Prompt text compression method and related equipment

Abstract

The invention discloses a prompt text compression method and related equipment. The method comprises the following steps: decomposing an original prompt text into a token sequence, calculating a self-information amount of each token based on a context conditional probability, determining an iteration limit threshold according to a target compression ratio, and iteratively removing a token with a low self-information amount until a compression target is reached. According to the scheme, the importance of the token is evaluated through the self-information amount, redundant and low-value information is accurately removed, and the problem that low-information-amount content cannot be effectively removed in the prior art is solved; based on a calculation mechanism of a language model, the defects of capability limitation and serious illusion of a large model are avoided; compared with a knowledge graph construction method, complex graph construction and traversal are not needed, the cost is greatly reduced, and the information processing time is remarkably shortened. Therefore, efficient and accurate compression of the prompt text is achieved, the information processing efficiency is improved, and the high-quality compressed prompt text better meeting the actual application requirement is obtained.

Description

Technical Field

[0001] The present application relates to the technical field of prompt text processing, and more specifically, to a prompt text compression method and related equipment. Background Art

[0002] In today's natural language processing field, effective processing strategies are particularly necessary, given that long prompt texts can lead to problems such as inefficient processing and inaccurate information extraction. Currently, there are two main approaches to processing long prompt texts. The first is to segment the long prompt text and then use a large model combined with strategies such as Stuff / MapReduce to filter out valid content. The second approach is to split the long prompt text into a knowledge graph. Using vectorization, the information node in the knowledge graph closest to the question is found. Using this information node, combined with relationships such as triples, the entire knowledge graph is traversed until the target length is reached.

[0003] However, these existing technologies have obvious flaws. For the first method, when extracting content through a large model, it will be limited by the compression capacity of the large model itself, and will also be affected by the compression prompt text, resulting in a serious illusion problem, that is, the extracted information may contain a lot of inaccuracies. In addition, the Stuff / MapReduce strategy cannot effectively remove redundant and low-information content in the original content, which will cause the processed information to still contain a large amount of useless information, affecting subsequent analysis and processing. For the second method, the cost of building a knowledge graph is high, its association relationships are complex and intertwined, and the dimensions are high. It takes a lot of time to extract information, which reduces processing efficiency.

[0004] Therefore, there is an urgent need for a new prompt text compression method that can achieve efficient and accurate compression of the prompt text while avoiding the various problems existing in the existing technology. Summary of the Invention

[0005] The present application provides a prompt text compression method and related equipment, which solves the problems of the existing technology such as severe hallucinations, inability to eliminate redundant information, high construction cost, and long processing time by decomposing the original text into a token sequence and calculating the self-information, and iteratively removes low self-information tokens based on a threshold value determined according to a target compression ratio, thereby achieving efficient and accurate text compression.

[0006] A prompt text compression method, comprising:

[0007] Break the original prompt text into a sequence of tokens;

[0008] Calculating the self-information of each token in the token sequence using a language model, wherein the self-information is determined based on a conditional probability of the token in the context;

[0009] Calculating the number of compressed tokens after the iteration is completed based on a preset target compression ratio and the number of original tokens in the token sequence, and determining a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence;

[0010] Based on the iteration limit threshold, iteratively remove tokens in the token sequence whose self-information amount is lower than the iteration limit threshold, and recalculate the self-information amount of the remaining tokens in the token sequence and the iteration limit threshold of the next iteration after each iteration;

[0011] When the number of remaining tokens reaches the compressed token number or the number of tokens in consecutive iterations no longer changes, a compression prompt text is output.

[0012] Optionally, for any round of iteration, the self-information of each token in the token sequence is calculated as follows:

[0013]

[0014] Where t is the number of iterations, is the i-th token in the token sequence at round t iteration.

[0015] Optionally, for any round of iteration, the iteration limit threshold is determined as follows:

[0016] Based on the threshold satisfaction condition, gradually approximating and determining an iterative limit threshold that meets the threshold satisfaction condition through a bisection method or a gradient descent method;

[0017] The threshold meets the conditions:

[0018]

[0019] Where t is the number of iterations, is the number of tokens in the token sequence at round t iteration, is the i-th token in the token sequence at round t iteration, is the iteration limit threshold for round t iteration, B is the number of compressed tokens, is the indicator function.

[0020] Optionally, after each iteration of removal, also include:

[0021] The semantic coherence test verifies whether the prompt text before and after iterative removal maintains the original semantics. If the semantics are broken, the fallback mechanism is triggered to undo the most recent iterative removal operation, adjust the iterative limit threshold, and perform iterative removal again until the semantic coherence requirement is met.

[0022] Optionally, also include:

[0023] Performing multimodal feature extraction on each token in the token sequence decomposed from the original prompt text, including part-of-speech tagging, syntactic dependency, and entity recognition results;

[0024] fusing the multimodal features of each token with weighted self-information to generate a composite importance score;

[0025] The iterative removal priority of each token is defined based on the composite score, with high-scoring tokens being retained first.

[0026] Optionally, also include:

[0027] monitoring in real time a resource load parameter of a target large language model used for subsequent processing of the compressed prompt text, and dynamically adjusting the target compression ratio according to available computing resources of the target large language model;

[0028] During the iteration process, the iteration limit threshold is modified based on the resource load parameter, and the modification calculation formula is:

[0029]

[0030] Where t is the number of iterations, is the iteration limit threshold after t rounds of iteration correction, is the iteration limit threshold before t rounds of iteration correction, is the current resource load, is the maximum resource load, is the load sensitivity factor.

[0031] A prompt text compression device, comprising:

[0032] A token decomposition unit, used to decompose the original prompt text into a sequence of tokens;

[0033] A self-information unit, configured to calculate the self-information of each token in the token sequence using a language model, wherein the self-information is determined based on a conditional probability of the token in a context;

[0034] A compression limiting unit, configured to calculate the number of compressed tokens after the iteration is completed according to a preset target compression ratio and the number of original tokens in the token sequence, and determine a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence;

[0035] an iterative removal unit, configured to iteratively remove tokens in the token sequence whose self-information is lower than the iteration limit threshold based on the iteration limit threshold, and recalculate the self-information of the remaining tokens in the token sequence and the iteration limit threshold for the next iteration after each iteration;

[0036] The output result unit is used to output the compression prompt text when the number of remaining tokens reaches the compressed token number or the number of tokens in continuous iterations no longer changes.

[0037] A prompt text compression device includes a memory and a processor;

[0038] The memory is used to store programs;

[0039] The processor is used to execute the program to implement each step of the prompt text compression method as described in any one of the above items.

[0040] A readable storage medium stores a computer program thereon, wherein when the computer program is executed by a processor, each step of the prompt text compression method as described in any one of the above items is implemented.

[0041] A computer program product includes a computer program, characterized in that when the computer program is run by a processor, it executes each step of the prompt text compression method as described in any one of the above items.

[0042] It can be seen from the above technical solution that the embodiment of the present application provides a prompt text compression method and related equipment, including decomposing the original prompt text into a token sequence; using a language model to calculate the self-information of each token in the token sequence, and the self-information is determined based on the conditional probability of the token in the context; according to the preset target compression ratio and the number of original tokens in the token sequence, the number of compressed tokens after the iteration is completed is calculated, and the corresponding iteration limit threshold is determined in combination with the self-information of each token; based on the iteration limit threshold, the tokens in the token sequence whose self-information is lower than the iteration limit threshold are iteratively removed, and the self-information of the remaining tokens and the iteration limit threshold of the next round of iteration are recalculated after each iteration; when the number of remaining tokens reaches the number of compressed tokens or the number of tokens no longer changes in consecutive iterations, the compressed prompt text is output.

[0043] This solution can accurately measure the importance of each token in the context by calculating the self-information of the token. The higher the self-information, the more critical the information contained in the token. Based on this, by setting an iteration limit threshold to remove tokens with low self-information, the redundant and low-information content in the original prompt text is effectively removed, solving the problem that redundant information cannot be effectively removed in the existing technology. At the same time, this method is based on the calculation method of the language model, which avoids the problem of being limited by the ability of the compressed large model itself and serious hallucinations when reading content extraction like the existing large model. Moreover, compared with the method of constructing a knowledge graph, this method does not need to construct a complex knowledge graph, which greatly reduces the cost, and since there is no need for complex knowledge graph traversal, the time for information processing will also be significantly reduced, improving processing efficiency. In this way, the method achieves efficient and accurate compression of the prompt text, obtains more valuable compressed prompt text, and meets the needs of practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0045] Figure 1 This is a flowchart of a prompt text compression method disclosed in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a prompt text compression device disclosed in an embodiment of the present application;

[0047] Figure 3 This is a hardware structure block diagram of a prompt text compression device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] The present application can be used in a variety of general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, and distributed computing environments including any of the above devices or devices.

[0050] Next, we will introduce the application scheme. This application proposes the following technical scheme, please see below for details.

[0051] Figure 1 This is a flowchart of a prompt text compression method disclosed in an embodiment of the present application.

[0052] like Figure 1 As shown, the method may include:

[0053] Step S1: Decompose the original prompt text into a token sequence.

[0054] Specifically, the input prompt text is broken down into a series of basic units, or token sequences, according to specific rules. This decomposition can be based on lexical analysis or byte-pair encoding. For English text, toolkits like NLTK (Natural Language Toolkit) and spaCy can segment text into distinct units based on distinct delimiters such as spaces and punctuation. For example, the text "Welcome to the world of AI." would be broken down into a token sequence like ["Welcome", "space", "to", "space", "the", "space", "world", "space", "of", "space", "AI", "."] Each word, space, and punctuation mark becomes a separate token.

[0055] For Chinese text, because there are no natural word separators like in English, dictionary-based word segmentation is often used. For example, when the Jieba word segmentation tool is fed the text "We love natural language processing technology," it uses its built-in dictionary to segment the text into a token sequence such as ["we," "love," "natural language processing," "technology"]. By identifying and segmenting the words in the text, the conversion from raw text to a token sequence is achieved.

[0056] Byte pair encoding is a method that gradually builds tokens from the character level. Initially, the text is split into characters. For example, the text "supercalifragilisticexpialidocious" would initially be represented as ["s", "u", "p", "e", "r", "c", "a", "l", "i", "f", "r", "a", "g", "i", "l", "i", "s", "t", "i", "c", "e", "x", "p", "i", "a", "l", "i", "d", "o", "c", "i", "o", "u", "s"]. The frequency of adjacent character combinations in the text is then counted, and the most frequent adjacent character combinations are merged. After multiple iterations of merging, more meaningful token units are gradually formed. Assuming that in a certain iteration, the combination "ca" appears frequently, it is merged into "ca". Subsequently, adjacent units are merged according to frequency until the set rules or conditions are met, ultimately obtaining a suitable token sequence. This approach can effectively handle out-of-vocabulary words and adapt to the characteristics of different languages and texts. It is understood that the decomposition method of the original prompt text into a token sequence should include, but is not limited to, the above method.

[0057] Step S2: Calculate the self-information of each token in the token sequence using a language model, where the self-information is determined based on the conditional probability of the token in the context.

[0058] Specifically, using a pre-trained language model, the amount of information contained in each token in a token sequence within its context is calculated to measure the token's importance. The token sequence is input into the language model, and the model outputs information about each token within its context. The model then calculates the probability of each token appearing in this context, known as the conditional probability.

[0059] Taking the BERT model (Bidirectional Encoder Representations from Transformers) as an example, when the token sequence obtained in step S1 is input into the BERT model, the model's internal multi-layer Transformer structure extracts features and encodes semantics from the token sequence, ultimately generating a hidden state representation corresponding to each token at the output layer. These hidden states contain semantic information about the token within the context of the entire text. Next, through specific computational operations, such as the Softmax function, the hidden states are processed and converted into a probability distribution, resulting in the conditional probability of each token based on its context. The conditional probability reflects the likelihood of the current token occurring given the previous token sequence.

[0060] Based on the obtained conditional probability, the self-information of each token is calculated according to the principle of calculating self-information, that is, the self-information is related to the negative logarithm of the conditional probability. A greater self-information indicates that the information carried by the token in the current context is more unique and important; a smaller self-information indicates that the token may have a higher degree of information redundancy in the current context. For example, in the text "I really enjoy reading books," the conditional probability of the token "books" is calculated using a language model, and then its self-information is calculated. If the probability of "books" appearing in similar contexts is relatively low, then its self-information will be high, indicating that it is relatively more critical in this text segment; conversely, if the probability is high, the self-information will be low, which may indicate that it is relatively common in this context and its information content is less prominent.

[0061] For any round of iteration, the self-information of each token in the token sequence is calculated as follows:

[0062]

[0063] Where t is the number of iterations, is the i-th token in the token sequence at round t iteration.

[0064] In the context of prompt text compression, the self-information of each token is calculated using the above formula to assess the information value of each token in the text. A token with low self-information is more likely to appear in the current context and carries relatively less unique information, making it redundant or low-value information. During the iterative token removal process, these low-self-information tokens are preferentially removed based on their self-information and the set iteration limit threshold, thereby gradually compressing the original prompt text and retaining the more critical, high-information portions, ultimately achieving text optimization.

[0065] Step S3: Calculate the number of compressed tokens after the iteration is completed according to the preset target compression ratio and the number of original tokens in the token sequence, and determine the corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence.

[0066] Specifically, this step aims to determine the number of compressed tokens that should be retained after the iterative operation based on the preset target compression ratio and the number of original tokens, and determine the corresponding iteration limit threshold based on the token self-information, providing a key basis for subsequent iterations to remove low-information tokens.

[0067] In this method, the preset target compression ratio α is a key parameter, and its value range is (0, 1). This parameter intuitively reflects the degree to which the original prompt text is expected to be compressed. For example, if α = 0.3, it means that the original prompt text is expected to be compressed to 30% of its original length. At the same time, the number N of original tokens in the original token sequence obtained through step S1. The compressed token number B after the iteration is calculated by the formula where the symbol represents the floor operation.

[0068] For example, if the number of original tokens N = 300 and the target compression ratio α = 0.4, then That is, after the compression iteration, it is expected that 120 compressed tokens will be retained. This calculation result clarifies the scale of tokens that need to be retained in the subsequent iteration operation and sets a quantitative goal for text compression.

[0069] After obtaining the expected compressed token number B, an iteration limit threshold τ needs to be determined to judge which tokens should be retained and which should be removed during the iteration. The self-information amount of each token in the token sequence has been calculated in step S2. The self-information amount reflects the richness and uniqueness of the information carried by each token in a specific context. By setting the threshold τ, the number of tokens whose self-information amount is greater than or equal to this threshold is approximately equal to the compressed token number B. When the condition ≥τ holds, the function value is 1, otherwise it is 0.

[0070] When actually determining this threshold, numerical calculation methods such as the bisection method and the gradient descent method can be used. Taking the bisection method as an example, first determine a reasonable initial threshold range, that is, the lower limit value l and the upper limit value u. Then take the middle value τ=(l + u) / 2 as the current trial threshold, and count the number count of tokens in the token sequence whose self-information amount is greater than or equal to this trial threshold τ. If count > B, it means that the current trial threshold τ is set too low, resulting in too many tokens satisfying the retention condition. At this time, update the lower limit value l to the current τ; if count < B, it means that the current trial threshold τ is set too high, and the number of tokens satisfying the retention condition is insufficient. The upper limit value u needs to be updated to the current τ. By continuously repeating the above adjustment process, gradually narrow the threshold range until a suitable τ value is found, so that the number of tokens satisfying the condition is close enough to the compressed token number B, thereby determining the iteration limit threshold that meets the requirements. This threshold will be used as an important criterion for judging whether a token is retained in the subsequent iteration operation, ensuring that the text compression process can proceed accurately and efficiently.

[0071] For any round of iteration, the determination method of the iteration limit threshold is:

[0072] Based on the threshold satisfaction condition, gradually approximating and determining an iterative limit threshold that meets the threshold satisfaction condition through a bisection method or a gradient descent method;

[0073] The threshold meets the conditions:

[0074]

[0075] Where t is the number of iterations, is the number of tokens in the token sequence at round t iteration, is the i-th token in the token sequence at round t iteration, is the iteration limit threshold at t iterations, B is the number of compressed tokens, and is the number of compressed tokens calculated based on the preset target compression ratio and the number of original tokens. is the indicator function.

[0076] Furthermore, in order to make the prompt text compression process better adapt to the computing resources of the target large language model in subsequent processing, this method further introduces a dynamic adjustment mechanism.

[0077] First, the resource load parameters of the target large language model used for subsequent processing of the compressed prompt text, such as CPU usage and memory utilization, are monitored in real time. The preset target compression ratio is dynamically adjusted based on the computing resources currently available to the target large language model. If available computing resources are sufficient, the compression level can be appropriately reduced, thereby increasing the target compression ratio. If resources are limited, the compression level can be increased, thereby decreasing the target compression ratio. Furthermore, during the iteration process, the iteration limit threshold is adjusted based on the resource load parameters.

[0078] That is, this application also includes:

[0079] monitoring in real time a resource load parameter of a target large language model used for subsequent processing of the compressed prompt text, and dynamically adjusting the target compression ratio according to available computing resources of the target large language model;

[0080] During the iteration process, the iteration limit threshold is modified based on the resource load parameter, and the modification calculation formula is:

[0081]

[0082] Where t is the number of iterations, is the iteration limit threshold after t rounds of iteration correction, is the iteration limit threshold before t rounds of iteration correction, is the current resource load, is the maximum resource load, is the load sensitivity factor.

[0083] When the current resource load is relatively large and close to the maximum resource load, the corrected iteration limit threshold increases, thereby removing more tokens with relatively low self-information content in the iteration to achieve more stringent compression to adapt to the tight computing resource situation; conversely, when the resource load is small, the corrected threshold is adjusted accordingly to enable the compression process to utilize resources more reasonably.

[0084] Step S4: Based on the iteration limit threshold, iteratively remove the tokens in the token sequence whose self-information content is lower than the iteration limit threshold, and recalculate the self-information content of the remaining tokens in the token sequence and the iteration limit threshold for the next iteration after each iteration.

[0085] Specifically, in this step, according to the iteration limit threshold determined in step S3, iterative processing is performed on the token sequence to gradually remove tokens with low self-information content and update relevant parameters.

[0086] In each iteration, the self-information content of each token in the token sequence calculated in step S2 is compared one by one with the iteration limit threshold of the current iteration. For those tokens whose self-information content is lower than the iteration limit threshold, it is considered that the unique information they carry in the current text context is relatively less, belonging to the category of redundant or low-value information, so they are removed from the token sequence. For example, in a token sequence of a text describing fruits, some particle tokens such as "的" and "了" often have relatively low self-information content and will be removed in this round of iteration if they are lower than the current iteration limit threshold.

[0087] After removing the corresponding tokens, since the composition of the token sequence has changed, the context environment of each remaining token has also changed. And the self-information content of a token is determined based on its conditional probability in the context, so it is necessary to recalculate the self-information content of the remaining tokens. By applying the language model and calculation method in step S2 again, the self-information content of the remaining tokens in the new context environment can be accurately obtained, so as to more accurately evaluate their information value in the current text.

[0088] Meanwhile, based on the updated token sequence and the recalculated self-information content of the remaining tokens, referring to the method and process for determining the iteration limit threshold in step S3, the iteration limit threshold required for the next iteration is calculated again. This step ensures that in each iteration process, the iteration limit threshold can be dynamically adjusted according to the actual situation of the current token sequence, enabling subsequent iterations to continuously and effectively advance towards the direction of the preset target compression ratio, further optimizing the text compression effect.

[0089] Step S5: When the number of remaining tokens reaches the compressed token number or the number of tokens no longer changes in consecutive iterations, output the compressed prompt text.

[0090] Specifically, during the iteration process, the number of remaining tokens is continuously monitored in real time. When the number of remaining tokens reaches the number of compressed tokens calculated in step S3 based on the preset target compression ratio and the number of original tokens, this clearly indicates that the current iteration has successfully achieved the preset text compression target. At this point, the iteration process can be terminated in a timely manner because the expected compression effect has been achieved and further iterations are meaningless.

[0091] Furthermore, if the number of tokens remains constant over multiple iterations, meaning no tokens are removed due to their self-information falling below the iteration limit, this means that the self-information distribution of the tokens in the current token sequence has reached a relatively stable state. In this case, further iterations will not effectively compress the text, so the iteration process should be terminated.

[0092] Once any of the above iteration termination conditions are met, the remaining token sequence is reassembled according to the natural order of the text to form the final text form, which is then output as the compressed prompt text. This compressed prompt text, obtained after multiple rounds of iterative processing, retains the key information of the original prompt text while removing redundant information to the greatest extent possible. This provides a more streamlined and efficient input for subsequent natural language processing tasks, such as input into large language models for inference and content generation, helping to improve processing efficiency and result accuracy.

[0093] It can be seen from the above technical solution that the embodiment of the present application provides a prompt text compression method and related equipment, including decomposing the original prompt text into a token sequence; using a language model to calculate the self-information of each token in the token sequence, and the self-information is determined based on the conditional probability of the token in the context; according to the preset target compression ratio and the number of original tokens in the token sequence, the number of compressed tokens after the iteration is completed is calculated, and the corresponding iteration limit threshold is determined in combination with the self-information of each token; based on the iteration limit threshold, the tokens in the token sequence whose self-information is lower than the iteration limit threshold are iteratively removed, and the self-information of the remaining tokens and the iteration limit threshold of the next round of iteration are recalculated after each iteration; when the number of remaining tokens reaches the number of compressed tokens or the number of tokens no longer changes in consecutive iterations, the compressed prompt text is output.

[0094] This solution can accurately measure the importance of each token in the context by calculating the self-information of the token. The higher the self-information, the more critical the information contained in the token. Based on this, by setting an iteration limit threshold to remove tokens with low self-information, the redundant and low-information content in the original prompt text is effectively removed, solving the problem that redundant information cannot be effectively removed in the existing technology. At the same time, this method is based on the calculation method of the language model, which avoids the problem of being limited by the ability of the compressed large model itself and serious hallucinations when reading content extraction like the existing large model. Moreover, compared with the method of constructing a knowledge graph, this method does not need to construct a complex knowledge graph, which greatly reduces the cost, and since there is no need for complex knowledge graph traversal, the time for information processing will also be significantly reduced, improving processing efficiency. In this way, the method achieves efficient and accurate compression of the prompt text, obtains more valuable compressed prompt text, and meets the needs of practical applications.

[0095] In some embodiments of the present application, to further improve the quality of prompt text compression, a semantic coherence check process is added. This process is performed after token removal in each iteration to ensure that the prompt text maintains its original semantics during the compression process.

[0096] Specifically, after each iterative removal, it also includes:

[0097] The semantic coherence test verifies whether the prompt text before and after iterative removal maintains the original semantics. If the semantics are broken, the fallback mechanism is triggered to undo the most recent iterative removal operation, adjust the iterative limit threshold, and perform iterative removal again until the semantic coherence requirement is met.

[0098] Specifically, after each iterative token removal, the prompt text before and after the iterative removal is semantically verified using a specialized semantic consistency check. This semantic consistency check leverages semantic analysis techniques from natural language processing, such as semantic role labeling and textual entailment relationship analysis, to assess the semantic consistency of the text. If the check detects a semantic discontinuity, meaning that the iterative removal operation has caused the text to become semantically incoherent, a rollback mechanism is triggered. This mechanism undoes the most recent iterative removal operation, restoring the token sequence to the state of the previous round. Simultaneously, the iteration limit threshold is adjusted, for example, by appropriately lowering the threshold to reduce the number of tokens removed in this round, or by adopting other appropriate adjustment strategies based on the specific situation. The iterative removal operation is then repeated, followed by another semantic consistency check. This cycle repeats until the prompt text after the iterative removal operation meets the semantic consistency requirements. This mechanism effectively avoids semantic loss caused by over-compression, ensuring the semantic integrity and accuracy of the compressed prompt text.

[0099] In addition, during each token removal process, this application also introduces a removal strategy based on composite importance scoring to more accurately retain key information.

[0100] Specifically, each removal process also includes:

[0101] Performing multimodal feature extraction on each token in the token sequence decomposed from the original prompt text, including part-of-speech tagging, syntactic dependency, and entity recognition results;

[0102] fusing the multimodal features of each token with weighted self-information to generate a composite importance score;

[0103] The iterative removal priority of each token is defined based on the composite score, with high-scoring tokens being retained first.

[0104] Specifically, multimodal feature extraction is first performed on each token in the token sequence derived from the decomposition of the original prompt text. This includes: part-of-speech tagging, which clarifies the part of speech of each token in the grammatical structure, such as noun, verb, adjective, etc., which helps understand the function and role of the token in the sentence; syntactic dependency analysis, which determines the grammatical relationship between the token and other tokens, such as the subject-verb relationship and the verb-object relationship, so as to grasp the overall structure of the sentence; and entity recognition, which identifies entities with specific meanings in the text, such as names of people, places, and institutions. These entities often carry important semantic information.

[0105] Next, the multimodal features extracted from each token are weighted and fused with the self-information calculated in step S2. Each feature is assigned a weight based on its importance in the semantic expression, and a weighted calculation is then used to generate a composite importance score for each token. For example, for text describing a specific event, entity recognition results may be given a higher weight; for text emphasizing grammatical structure, syntactic dependencies may be given a higher weight.

[0106] Finally, the generated composite importance score is used to define the iterative removal priority for each token. Tokens with higher scores represent greater overall importance within the text and are therefore prioritized for retention during the iterative removal process. This approach allows for more targeted selection and retention of key tokens when compressing prompt text, achieving compression while preserving the text's core semantics and important information to the greatest extent possible, improving the quality and practicality of the compressed prompt text.

[0107] The present application is described below with an example.

[0108] Assume the original prompt text is:

[0109] This prompt consists of 7 tokens. The goal is to compress the prompt to 50% of its original length, that is, to retain approximately tokens.

[0110] The steps are as follows:

[0111] 1. Calculate the self-information

[0112] For each token, use the language model to calculate its conditional probability to obtain the self-information. Suppose after calculation, the following self-information is obtained:

[0113] "Deep learning":

[0114] "Yes":

[0115] "Artificial intelligence":

[0116] "Of":

[0117] "Core":

[0118] "Technology":

[0119] ".":

[0120] 2. Determine the iteration limit threshold

[0121] To achieve the goal of retaining 3 tokens, an iteration limit threshold τ needs to be selected such that:

[0122]

[0123] Based on the above self-information, τ can be set around 6. In this way, only tokens with self-information greater than or equal to 6 are retained.

[0124] 3. Screen and remove tokens

[0125] Check whether each token is removed according to the iteration limit threshold of ⑥:

[0126] "Deep learning" (8≥6) → Retain

[0127] "Yes" (1<6) → Remove

[0128] "Artificial intelligence" (7≥6) → Retain

[0129] "Of" (1<6) → Remove

[0130] "Core" (6≥6) → Retain

[0131] "Technology" (5<6) → Remove

[0132] "." (1<6) → Remove

[0133] After this round of iteration, the compressed prompt sequence is:

[0134]

[0135] The original prompt, "Deep learning is the core technology of artificial intelligence," contains key information primarily in the three parts "deep learning," "artificial intelligence," and "core." After algorithmic processing, the compressed prompt becomes "Deep Learning Artificial Intelligence Core." This not only reduces the number of tokens from 7 to 3 (achieving a 50% compression rate), but also preserves key information, allowing the large language model to still understand the main meaning of the original prompt during subsequent inference. The above example illustrates only one iteration of removal. In practice, the self-information of the remaining tokens in the token sequence and the iteration limit threshold for the next iteration are recalculated, and iterative removal is continued until the iteration stop condition is met, at which point the compressed prompt text is output.

[0136] The following describes a prompt text compression device provided in an embodiment of the present application. The prompt text compression device described below and the prompt text compression method described above can be referenced to each other.

[0137] See also Figure 2 , Figure 2 This is a schematic diagram of a prompt text compression device disclosed in an embodiment of the present application.

[0138] like Figure 2 As shown, the prompt text compression device may include:

[0139] a token decomposition unit 110 for decomposing the original prompt text into a sequence of tokens;

[0140] A self-information unit 120 is configured to calculate the self-information of each token in the token sequence using a language model, wherein the self-information is determined based on a conditional probability of the token in a context;

[0141] A compression limiting unit 130 is configured to calculate the number of compressed tokens after the iteration is completed based on a preset target compression ratio and the number of original tokens in the token sequence, and determine a corresponding iteration limit threshold based on the self-information of each token in the token sequence;

[0142] an iterative removal unit 140, configured to iteratively remove tokens in the token sequence whose self-information is lower than the iteration limit threshold based on the iteration limit threshold, and recalculate the self-information of the remaining tokens in the token sequence and the iteration limit threshold for the next iteration after each iteration;

[0143] The result output unit 150 is configured to output a compression prompt text when the number of remaining tokens reaches the number of compressed tokens or the number of tokens does not change in consecutive iterations.

[0144] It can be seen from the above technical solution that the embodiment of the present application provides a prompt text compression method and related equipment, including decomposing the original prompt text into a token sequence; using a language model to calculate the self-information of each token in the token sequence, and the self-information is determined based on the conditional probability of the token in the context; according to the preset target compression ratio and the number of original tokens in the token sequence, the number of compressed tokens after the iteration is completed is calculated, and the corresponding iteration limit threshold is determined in combination with the self-information of each token; based on the iteration limit threshold, the tokens in the token sequence whose self-information is lower than the iteration limit threshold are iteratively removed, and the self-information of the remaining tokens and the iteration limit threshold of the next round of iteration are recalculated after each iteration; when the number of remaining tokens reaches the number of compressed tokens or the number of tokens no longer changes in consecutive iterations, the compressed prompt text is output.

[0145] This solution can accurately measure the importance of each token in the context by calculating the self-information of the token. The higher the self-information, the more critical the information contained in the token. Based on this, by setting an iteration limit threshold to remove tokens with low self-information, the redundant and low-information content in the original prompt text is effectively removed, solving the problem that redundant information cannot be effectively removed in the existing technology. At the same time, this method is based on the calculation method of the language model, which avoids the problem of being limited by the ability of the compressed large model itself and serious hallucinations when reading content extraction like the existing large model. Moreover, compared with the method of constructing a knowledge graph, this method does not need to construct a complex knowledge graph, which greatly reduces the cost, and since there is no need for complex knowledge graph traversal, the time for information processing will also be significantly reduced, improving processing efficiency. In this way, the method achieves efficient and accurate compression of the prompt text, obtains more valuable compressed prompt text, and meets the needs of practical applications.

[0146] Optionally, for any round of iteration, the self-information of each token in the token sequence is calculated as follows:

[0147]

[0148] Where t is the number of iterations, is the i-th token in the token sequence at round t iteration.

[0149] Optionally, for any round of iteration, the iteration limit threshold is determined as follows:

[0150] Based on the threshold satisfaction condition, gradually approximating and determining an iterative limit threshold that meets the threshold satisfaction condition through a bisection method or a gradient descent method;

[0151] The threshold meets the conditions:

[0152]

[0153] Where t is the number of iterations, is the number of tokens in the token sequence at round t iteration, is the i-th token in the token sequence at round t iteration, is the iteration limit threshold for round t iteration, B is the number of compressed tokens, is the indicator function.

[0154] Optionally, after each iteration of removal, also include:

[0155] The semantic coherence test verifies whether the prompt text before and after iterative removal maintains the original semantics. If the semantics are broken, the fallback mechanism is triggered to undo the most recent iterative removal operation, adjust the iterative limit threshold, and perform iterative removal again until the semantic coherence requirement is met.

[0156] Optionally, also include:

[0157] Performing multimodal feature extraction on each token in the token sequence decomposed from the original prompt text, including part-of-speech tagging, syntactic dependency, and entity recognition results;

[0158] fusing the multimodal features of each token with weighted self-information to generate a composite importance score;

[0159] The iterative removal priority of each token is defined based on the composite score, with high-scoring tokens being retained first.

[0160] Optionally, also include:

[0161] monitoring in real time a resource load parameter of a target large language model used for subsequent processing of the compressed prompt text, and dynamically adjusting the target compression ratio according to available computing resources of the target large language model;

[0162] During the iteration process, the iteration limit threshold is modified based on the resource load parameter, and the modification calculation formula is:

[0163]

[0164] Where t is the number of iterations, is the iteration limit threshold after t rounds of iteration correction, is the iteration limit threshold before t rounds of iteration correction, is the current resource load, is the maximum resource load, is the load sensitivity factor.

[0165] The prompt text compression device provided in the embodiment of the present application can be applied to prompt text compression equipment. Figure 3 The hardware structure diagram of the prompt text compression device is shown. Figure 3 ,The hardware structure of the prompt text compression device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0166] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0167] The processor 1 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0168] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory;

[0169] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0170] Break the original prompt text into a sequence of tokens;

[0171] Calculating the self-information of each token in the token sequence using a language model, where the self-information is determined based on a conditional probability of the token in context;

[0172] Calculating the number of compressed tokens after the iteration is completed based on a preset target compression ratio and the number of original tokens in the token sequence, and determining a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence;

[0173] Based on the iteration limit threshold, iteratively remove tokens in the token sequence whose self-information amount is lower than the iteration limit threshold, and recalculate the self-information amount of the remaining tokens in the token sequence and the iteration limit threshold of the next iteration after each iteration;

[0174] When the number of remaining tokens reaches the compressed token number or the number of tokens in consecutive iterations does not change, the compressed prompt text is output. Optionally, the detailed functions and extended functions of the program can refer to the above description.

[0175] The present application also provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0176] Break the original prompt text into a sequence of tokens;

[0177] Calculating the self-information of each token in the token sequence using a language model, where the self-information is determined based on a conditional probability of the token in context;

[0178] Calculating the number of compressed tokens after the iteration is completed based on a preset target compression ratio and the number of original tokens in the token sequence, and determining a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence;

[0179] Based on the iteration limit threshold, iteratively remove tokens in the token sequence whose self-information amount is lower than the iteration limit threshold, and recalculate the self-information amount of the remaining tokens in the token sequence and the iteration limit threshold of the next iteration after each iteration;

[0180] When the number of remaining tokens reaches the compressed token number or the number of tokens in consecutive iterations does not change any more, a compression prompt text is output.

[0181] Optionally, the refined functions and extended functions of the program may refer to the above description.

[0182] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the execution method is as follows:

[0183] Break the original prompt text into a sequence of tokens;

[0184] Calculating the self-information of each token in the token sequence using a language model, where the self-information is determined based on a conditional probability of the token in context;

[0185] Calculating the number of compressed tokens after the iteration is completed based on a preset target compression ratio and the number of original tokens in the token sequence, and determining a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence;

[0186] Based on the iteration limit threshold, iteratively remove tokens in the token sequence whose self-information amount is lower than the iteration limit threshold, and recalculate the self-information amount of the remaining tokens in the token sequence and the iteration limit threshold of the next iteration after each iteration;

[0187] When the number of remaining tokens reaches the compressed token number or the number of tokens in consecutive iterations does not change, the compressed prompt text is output. Optionally, the detailed functions and extended functions of the program can refer to the above description.

[0188] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0189] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0190] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A prompt text compression method, characterized in that: include: Break the original prompt text into a sequence of tokens; Calculating the self-information of each token in the token sequence using a language model, where the self-information is determined based on a conditional probability of the token in context; Calculating the number of compressed tokens after the iteration is completed based on a preset target compression ratio and the number of original tokens in the token sequence, and determining a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence; Based on the iteration limit threshold, iteratively remove tokens in the token sequence whose self-information amount is lower than the iteration limit threshold, and recalculate the self-information amount of the remaining tokens in the token sequence and the iteration limit threshold of the next iteration after each iteration; When the number of remaining tokens reaches the compressed token number or the number of tokens in consecutive iterations no longer changes, a compression prompt text is output.

2. The method according to claim 1, characterized in that For any round of iteration, the self-information of each token in the token sequence is calculated as follows: Where t is the number of iterations, is the i-th token in the token sequence at round t iteration.

3. The method according to claim 1, characterized in that For any round of iteration, the iteration limit threshold is determined as follows: Based on the threshold satisfaction condition, gradually approximating and determining an iterative limit threshold that meets the threshold satisfaction condition through a bisection method or a gradient descent method; The threshold meets the conditions: Where t is the number of iterations, is the number of tokens in the token sequence at round t iteration, is the i-th token in the token sequence at round t iteration, is the iteration limit threshold for round t iteration, B is the number of compressed tokens, is the indicator function.

4. The method according to claim 1, wherein After each iteration of removal, also include: The semantic coherence test verifies whether the prompt text before and after iterative removal maintains the original semantics. If the semantics are broken, the fallback mechanism is triggered to undo the most recent iterative removal operation, adjust the iterative limit threshold, and perform iterative removal again until the semantic coherence requirement is met.

5. The method according to claim 1, wherein Also includes: Performing multimodal feature extraction on each token in the token sequence decomposed from the original prompt text, including part-of-speech tagging, syntactic dependency, and entity recognition results; fusing the multimodal features of each token with weighted self-information to generate a composite importance score; The iterative removal priority of each token is defined based on the composite score, with high-scoring tokens being retained first.

6. The method according to claim 1, characterized in that Also includes: monitoring in real time a resource load parameter of a target large language model used for subsequent processing of the compressed prompt text, and dynamically adjusting the target compression ratio according to available computing resources of the target large language model; During the iteration process, the iteration limit threshold is modified based on the resource load parameter, and the modification calculation formula is: Where t is the number of iterations, is the iteration limit threshold after t rounds of iteration correction, is the iteration limit threshold before t rounds of iteration correction, is the current resource load, is the maximum resource load, is the load sensitivity factor.

7. A prompt text compression device, characterized in that: include: A token decomposition unit, used to decompose the original prompt text into a sequence of tokens; A self-information unit, configured to calculate the self-information of each token in the token sequence using a language model, wherein the self-information is determined based on a conditional probability of the token in a context; A compression limiting unit, configured to calculate the number of compressed tokens after the iteration is completed according to a preset target compression ratio and the number of original tokens in the token sequence, and determine a corresponding iteration limit threshold in combination with the self-information amount of each token in the token sequence; an iterative removal unit, configured to iteratively remove tokens in the token sequence whose self-information is lower than the iteration limit threshold based on the iteration limit threshold, and recalculate the self-information of the remaining tokens in the token sequence and the iteration limit threshold for the next iteration after each iteration; The output result unit is used to output the compression prompt text when the number of remaining tokens reaches the compressed token number or the number of tokens in continuous iterations no longer changes.

8. A prompt text compression device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the prompt text compression method according to any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the prompt text compression method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program executes the steps of the prompt text compression method according to any one of claims 1 to 6.

Images