Method, device and storage medium for reducing token consumption of large model
By monitoring and analyzing rule changes, dynamically adjusting and compressing prompt words, and generating optimal prompt words, the token consumption problem of large models when rules change is solved, achieving efficient token saving and ensuring analysis quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU YSCREDIT CO LTD
- Filing Date
- 2026-07-06
- Publication Date
- 2026-07-31
AI Technical Summary
When processing text in large models, changes in analysis rules lead to increased token consumption, and existing technologies struggle to effectively reduce token consumption while maintaining analysis quality.
By monitoring and analyzing rule changes, we can dynamically adjust and compress prompt words to generate optimal prompt words and reduce token consumption.
While ensuring analysis quality, it significantly reduces token consumption when large models process historical text, reduces computational resource consumption and manual modification costs, and improves response efficiency.
Smart Images
Figure CN122489730A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text processing technology, and in particular to a method, apparatus, and storage medium for reducing the token consumption of large models. Background Technology
[0002] In data tracing and analysis scenarios, it is often necessary to classify and label large amounts of historical text to support subsequent data queries and tracing. Traditional methods rely on manual reading and labeling, which is inefficient and difficult to scale. With the development of large language model technology, using large models to process historical text has become a feasible solution. By designing appropriate prompt words, the large model is guided to extract key information from the text, and then classification is performed in combination with preset rules. However, when the preset rules are updated, the original prompt words will no longer be suitable for the new rules, resulting in the inability to extract the required information. At this time, it is necessary to regenerate prompt words that are suitable for the new rules, and use the new prompt words to reprocess all historical text to update the category labels.
[0003] However, large models charge based on the number of input and output tokens when processing text. Token consumption is directly related to the length of the input content. Reprocessing all historical text with new prompts would result in significant token consumption, while arbitrarily reducing the number of characters in prompts could easily lead to a decrease in analysis accuracy. Therefore, how to effectively reduce token consumption while maintaining analysis quality when analysis rules change has become a pressing problem in this field. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a method, device, and storage medium for reducing token consumption in large models. By monitoring and analyzing changes in rules, the invention dynamically adjusts and compresses prompt words, thereby reducing token consumption when large models process historical text while ensuring analysis quality.
[0005] According to a first aspect of the present invention, a method for reducing token consumption in large models is provided, comprising the following steps: S1, based on the historical text set of the target object, determines the corresponding analysis task type according to the category label associated with each text unit in the historical text set, and determines the original prompt words associated with the analysis task type and the analysis accuracy of the corresponding original prompt words from the preset prompt word library; wherein, the category label is labeled by the preset analysis rules; the original prompt words are used to guide the large model to extract sentence fragments related to the preset analysis rules from the text units.
[0006] S2 monitors changes in preset analysis rules in real time. When a rule change is detected, it determines the category label corresponding to the preset analysis rule and the original prompt words corresponding to the category label. Based on the rule change, it adjusts the analysis dimension keywords in the original prompt words and generates new prompt words.
[0007] S3. The new prompt words are compressed using several preset compression strategies to generate multiple candidate prompt words. The quality loss is calculated based on the change in the analysis accuracy of each candidate prompt word relative to the original prompt word. The optimal prompt word is selected based on the quality loss of each candidate prompt word and the pre-calculated number of tokens.
[0008] S4, based on the optimal prompt words, guides the large model to reprocess the text units corresponding to the category labels associated with rule changes, thereby reducing token consumption during the large model's processing.
[0009] According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the above-described method for reducing large model token consumption.
[0010] According to a third aspect of the present invention, an electronic device is provided, including a processor and the aforementioned non-transitory computer-readable storage medium.
[0011] The present invention has at least the following beneficial effects: This invention provides a method to reduce token consumption in large models. First, it determines the analysis task type based on the category tags associated with each text unit in the target object's historical text set, and then identifies the original prompt words associated with each analysis task type and their corresponding analysis accuracy. This provides a basis for quickly locating affected prompt words when analysis rules change. When a rule change is detected, it determines the category tags corresponding to the preset analysis rules and the original prompt words corresponding to those tags. Based on the rule change, it adjusts the analysis dimension keywords in the original prompt words to generate new prompt words. By detecting rule changes in real time and accurately locating affected prompt words, it avoids ineffective optimization of all prompt words, reducing computational resource consumption. Furthermore, by specifically adjusting the analysis dimension keywords in the original prompt words according to the rule change, the new prompt words can quickly adapt to the new rules, reducing manual modification costs. Then, multiple compression strategies are employed to compress the new prompt words, generating multiple candidate prompt words. This reduces the token length of the prompt words themselves, thereby reducing the single-call consumption during large-scale model processing. Quality loss is then calculated based on changes in analysis accuracy. The optimal prompt word is selected based on the quality loss and token quantity of each candidate prompt word, achieving an optimal balance between token saving and analysis quality. Finally, based on the optimal prompt word, the large-scale model is guided to reprocess text units affected by rule changes. This invention reduces the token consumption of prompt words by dynamically adjusting and compressing them through monitoring and analyzing rule changes. Simultaneously, it avoids the high token consumption caused by reprocessing all text units using prompt words, significantly reducing the token consumption cost of large-scale models when processing historical text while ensuring analysis quality. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 A flowchart illustrating a method for reducing token consumption in large models, provided in an embodiment of the present invention. Detailed Implementation
[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0015] This invention provides a method for reducing token consumption in large models, such as... Figure 1 As shown, the method includes the following steps: S1, based on the historical text set of the target object, determines the corresponding analysis task type according to the category label associated with each text unit in the historical text set, and determines the original prompt words associated with the analysis task type and the analysis accuracy of the corresponding original prompt words from the preset prompt word library; it can be understood as: several analysis task types are predefined for the target object, each analysis task type corresponds to a set of category labels, and the category labels are marked by preset analysis rules, that is, each preset analysis rule corresponds to at least one category label.
[0016] Specifically, a template with at least one original prompt word is configured for each analysis task type. When a template with one original prompt word is configured, the analysis dimension keywords in the original prompt word are the keywords corresponding to the analysis task type. When a template with multiple original prompt words is configured, the analysis dimension keywords in the original prompt words include the keywords corresponding to each category tag, and optionally, keywords corresponding to the analysis task type. For example, when the target object is a cybersecurity incident, the analysis task types include data breach incidents, access violation incidents, and abnormal access incidents; among them, the category tags corresponding to abnormal access incidents include high-frequency access tags, remote login tags, and abnormal user tags. The corresponding original prompt word can be: Analyze the following text, extract descriptions related to abnormal access incidents and give an anomaly score:\n{text content}; or it can be: Analyze the following text, extract descriptions related to abnormal user tags and give an anomaly score:\n{text content}.
[0017] In other implementation scenarios, when the target object is industrial equipment, the analysis task types include equipment failure, safety violation records, and abnormal operation logs. Among them, the category tags corresponding to equipment failure include bearing failure, motor failure, and transmission system failure. When the target object is an enterprise, the analysis task types include financial status identification and public opinion risk analysis. Among them, the category tags corresponding to financial status identification include debt repayment ability risk, profitability risk, and cash flow risk.
[0018] In one embodiment, obtaining a historical text set specifically includes: preprocessing the obtained historical text data, wherein the preprocessing includes: removing irrelevant characters, segmenting sentences according to punctuation marks, segmenting words using a word segmentation tool, removing stop words, and retaining the core semantics of the text; dividing the preprocessed text into several text units in chronological order, and labeling each text unit with a category label determined by a preset analysis rule.
[0019] Specifically, category labels are generated using preset analysis rules, and the original prompt words guide the large model to extract sentence fragments related to the preset analysis rules from text units. The large model can be a large-scale language model with natural language understanding and generation capabilities, such as the GPT-4 model. For example, when the analysis task is data breach event identification, the corresponding original prompt words are "Please analyze the description of the data breach event in the following text unit, extract the core event, and give a preliminary score for the degree of breach:\n{text content}". The preset analysis rules for the corresponding data breach event description include the rule that when the initial score for the degree of breach exceeds a preset score threshold, the text unit is labeled as a serious data breach event.
[0020] Specifically, the historical text set is stored in a structured database. Each storage record contains a text identifier, generation timestamp, text content, and associated category tag corresponding to the text unit. A time index is also established for the text unit to enable rapid retrieval and location of the text unit in data tracing scenarios.
[0021] As described above, by establishing the association between category labels and prompt words, the standardization of text units was achieved, the analysis accuracy of the original prompt words was recorded in advance, providing a data foundation for subsequent optimization, and this step linked the rules, labels, and prompt words, providing a basis for quickly locating the affected prompt words when the rules change.
[0022] In a specific embodiment, the analysis accuracy of the corresponding original prompt word is obtained through the following steps: S101, Obtain the verification sample set with real category labels already labeled; this can be understood as: the verification samples are historical texts with real category labels already labeled.
[0023] S102, the original prompt words are applied to the verification sample set to guide the large model to extract several sentence fragments related to the preset analysis rules from each verification sample in the verification sample set, and input the extracted sentence fragments into the preset analysis rules to output the category label of each verification sample.
[0024] S103 compares the category label of each output verification sample with the corresponding true category label, and calculates the proportion of correct outputs as the analysis accuracy of the original prompt words.
[0025] S2 monitors changes in preset analysis rules in real time. When a rule change is detected, it determines the category label corresponding to the preset analysis rule and the original prompt words corresponding to the category label. Based on the rule change, it adjusts the analysis dimension keywords in the original prompt words and generates new prompt words. This can be understood as: first, determine the category label corresponding to the changed preset analysis rule, then determine the analysis task type corresponding to the category label, and map the corresponding original prompt words based on the analysis task type.
[0026] Specifically, the changes in the preset analysis rules include one or more of the following: adding or removing category labels and adding or removing analysis dimensions in the preset analysis rules.
[0027] Furthermore, the adjustment of the corresponding analysis dimension keywords in the original prompts based on rule-based content changes includes: When the rule change involves adding or removing category labels, the corresponding analysis dimension keywords should be added or removed from the original prompts.
[0028] When the rule change involves adding or removing analysis dimensions from the preset analysis rules, the corresponding analysis dimension keywords should be added or deleted from the original prompts. For example, if an analysis dimension is added to the preset analysis rules as: "Monthly revenue in the most recent quarter is less than 10% and gradually declines," then the following should be added to the original prompts: "Retrieve keywords about monthly revenue in the most recent quarter from the text," while the rest of the prompts remain unchanged.
[0029] The above-mentioned real-time detection of rule changes and precise location of affected prompt words avoids ineffective optimization of all prompt words, reduces the consumption of computing resources, and specifically adjusts the analysis dimension keywords in the original prompt words according to the content of rule changes, so that the new prompt words can quickly adapt to the new rules, reducing the cost of manual modification. This step realizes the automated mapping from rule changes to prompt word adjustments, and improves response efficiency.
[0030] S3. The new prompt words are compressed using several preset compression strategies to generate multiple candidate prompt words. The quality loss is calculated based on the change in the analysis accuracy of each candidate prompt word relative to the original prompt word. The optimal prompt word is selected based on the quality loss of each candidate prompt word and the pre-calculated number of tokens.
[0031] Specifically, the preset compression strategies include one or more combinations of redundant word deletion strategies, original prompt word merging strategies, and encoding replacement strategies.
[0032] Specifically, when implementing the redundant word removal strategy, semantic analysis tools from natural language processing are used to identify modifiers without information content, repetitive instructions, and redundant expressions in the prompts. These are then removed in batches without affecting the large model's understanding of the analysis requirements. In other words, the principle of redundant word removal is to retain the core instructions and analysis dimensions of the original prompts while removing meaningless modifiers, so that the large model can accurately understand the analysis task content.
[0033] Specifically, when implementing the encoding substitution strategy, a standardized instruction encoding table is established to map the complete original prompt word to the corresponding encoding. At the front end of the large model, the received short encoding is decoded based on the configured standardized instruction encoding table and restored to the complete prompt word instruction. In the process of token transmission and billing, the consumption is calculated based on the length of the encoded token.
[0034] In one specific embodiment, the original prompt word merging strategy includes: S01, calculate the semantic similarity between any two new prompt words. For example, calculate semantic similarity using cosine similarity.
[0035] S02, using each new prompt word as a node and the semantic similarity between new prompt words exceeding a preset threshold as an edge, construct a connected graph, and divide all new prompt words in the same connected component into the same merge group.
[0036] S03, for each merge group, extract the common instruction portion of all new prompt words within the merge group as a fixed prompt word template, and replace the differences in each new prompt word with variable placeholders to obtain the candidate prompt words corresponding to the merge group. For example, if the original prompt word A is: "Analyze high-frequency access events in the text, extract core events and give risk scores"; and the original prompt word B is: "Analyze abnormal user events in the text, extract core events and give risk scores"; then the candidate prompt word obtained after merging is: "Analyze {event type} events in the text, extract core events and give risk scores"; where {event type} is a variable placeholder. When processing text units using the large model later, the variable placeholder is dynamically filled with high-frequency access or abnormal user content according to the actual analysis task requirements. The filled prompt word is then input into the large model to guide the large model in processing the text.
[0037] As mentioned above, using multiple compression methods to compress prompt words can reduce the token length of the prompt words themselves, thereby reducing the single call consumption in the processing of large models. When accumulating and processing a large number of text units, it can significantly save the overall token overhead. Furthermore, generating multiple candidate prompt words based on different strategies and strategy combinations makes the compression scheme diverse, avoiding the problems of over-compression or under-compression that may be caused by a single strategy, and thus selecting the optimal compressed prompt words.
[0038] In one specific embodiment, the optimal suggestion words are selected through the following steps: S301, For any candidate suggestion word, calculate the quality loss of the candidate suggestion word; the quality loss of the candidate suggestion word conforms to the following formula: L = max(0, (R0 - R1) / R0), where L is the quality loss of the candidate suggestion, R0 is the analysis accuracy of the original suggestion, and R1 is the analysis accuracy of the candidate suggestion; R1 is obtained in the same way as R0, that is, by referring to steps S101-S103. In this formula, when R1 > R0, the quality loss is 0.
[0039] S302, candidate suggestion words with quality loss greater than a preset threshold are removed. The quality loss and token quantity of the remaining candidate suggestion words are normalized to the same preset numerical range. The cost parameter value of each candidate suggestion word is calculated based on the normalized quality loss value and the normalized token quantity value. In specific implementations, min-max normalization or Z-score normalization can be used for normalization processing. Those skilled in the art are familiar with the specific implementation steps of these two normalization methods, and will not be elaborated here.
[0040] Specifically, the cost parameter value of any candidate prompt word conforms to the following formula: C = T0 + λ × L0, where C is the cost parameter of the candidate prompt, T0 is the normalized value of the token quantity of the candidate prompt, L0 is the normalized value of the quality loss of the candidate prompt, and λ is the preset adjustment coefficient.
[0041] S303, select the candidate prompt word with the smallest corresponding cost parameter value as the optimal prompt word.
[0042] As described above, based on generating several candidate suggestion words using multiple compression strategies, the system effectively eliminated inferior candidate suggestion words that caused a significant drop in accuracy due to over-compression through quality loss. Through comprehensive evaluation of cost parameter values, the system achieved an optimal balance between token saving and analysis quality, ensuring that the compression process does not sacrifice analysis accuracy. Furthermore, the system can quickly generate low-token-consumption suggestion words that adapt to new rules when rules change, thereby improving the system's adaptability and overall processing efficiency.
[0043] Furthermore, the method also determines the optimal prompt word through the following steps: S10: Calculate the token saving rate based on the number of tokens for the optimal prompt and the number of tokens for the original prompt corresponding to the optimal prompt.
[0044] Specifically, the token saving rate meets the following conditions: η = (D - D0) / D × 100%, where η is the token saving rate, D is the number of tokens of the original prompt corresponding to the optimal prompt, and D0 is the number of tokens of the optimal prompt.
[0045] S20, if the token saving rate is lower than the preset target, return to step S3, and enhance the compression level of the preset compression strategy, regenerate candidate prompts and filter new optimal prompts until the token saving rate is not lower than the preset target or the iteration number threshold is reached; wherein, if the token saving rate is still lower than the preset target when the iteration number threshold is reached, the candidate prompt with the smallest corresponding cost parameter value during the iteration process is taken as the optimal prompt. Those skilled in the art can set the preset target and iteration number threshold according to actual needs.
[0046] Specifically, enhancing the compression level of the preset compression strategy includes: increasing the strictness of redundant word determination in the redundant word deletion strategy to expand the range of redundant words to be deleted, for example, expanding the part-of-speech filtering range from adjectives to adverbs and degree adverbs; lowering the preset threshold corresponding to semantic similarity in the original prompt word merging process so that more original prompt words meet the merging conditions; and adopting a higher compression rate instruction abbreviation encoding method in the encoding substitution strategy.
[0047] With the accuracy of analysis in the screening process already guaranteed in the aforementioned embodiments, the token saving rate is continuously optimized through the iterative feedback mechanism in this implementation step, achieving a gradual improvement in compression efficiency. When the iteration cannot improve further, the global optimal solution is automatically backtracked to ensure that the final output prompts achieve the best compression effect within the quality constraints.
[0048] S4, based on the optimal prompt words, guides the large model to reprocess the text units corresponding to the category labels associated with rule changes, so as to reduce the token consumption in the large model processing process; it can be understood as: the large model extracts the keywords or sentences corresponding to the changed rules from the text units affected by the rule changes based on the optimal prompt words, so as to update the category labels carried by the text units, so that the updated text units can be used for subsequent data tracing or query statistics.
[0049] As described above, by using the optimal prompt words to reduce token consumption, the optimal prompt words are used to reprocess text units affected by rule changes. This further avoids the high token consumption caused by reprocessing all text units using prompt words. The optimized prompt words enable batch updates of text unit category labels, which significantly reduces the processing cost of large models in rule change scenarios and greatly saves computing resources.
[0050] In another specific embodiment, the method further includes the following steps: The frequency f of regular changes within a historical time period is statistically analyzed, and the dynamic compression coefficient γ is calculated based on the frequency of regular changes; where γ = max(γ min ,1-f / f max ), γ minTo preset the minimum dynamic compression factor, f max This refers to the preset maximum frequency corresponding to a historical time period. In specific implementations, the historical time period can be the most recent year. Those skilled in the art can set the preset minimum dynamic compression factor and preset maximum frequency according to actual needs.
[0051] The product of the dynamic compression coefficient γ and the number of tokens for each original prompt word is used as the threshold for the number of tokens after compression of each original prompt word, so that the number of tokens for the candidate prompt words generated during the next compression is not greater than the corresponding token number threshold.
[0052] Considering that high-frequency change scenarios require frequent reprocessing of historical text and are sensitive to token consumption, while low-frequency change scenarios consume less tokens, it is necessary to prioritize ensuring analysis quality. The above-mentioned method of dynamically adjusting the compression level based on the frequency of rule changes is adopted. In high-frequency change scenarios, a stronger compression strategy is used to save a lot of repetitive processing overhead, while in low-frequency change scenarios, a gentle compression is used to prioritize ensuring analysis quality. Furthermore, a preset minimum compression coefficient is introduced to avoid excessive compression that would cause prompt words to become invalid, thus achieving adaptive matching between compression strength and rule changes.
[0053] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method for reducing large model token consumption provided in the above embodiments.
[0054] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.
[0055] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims
1. A method for reducing token consumption in large models, characterized in that, The method includes the following steps: S1, based on the historical text set of the target object, determines the corresponding analysis task type according to the category label associated with each text unit in the historical text set, and determines the original prompt words associated with the analysis task type and the analysis accuracy of the corresponding original prompt words from the preset prompt word library; wherein, the category label is labeled by the preset analysis rules; the original prompt words are used to guide the large model to extract sentence fragments related to the preset analysis rules from the text units; S2, monitors changes in preset analysis rules in real time. When a rule change is detected, it determines the category label corresponding to the preset analysis rule and the original prompt words corresponding to the category label. Based on the rule change, it adjusts the analysis dimension keywords in the original prompt words and generates new prompt words. S3. The new prompt words are compressed using several preset compression strategies to generate multiple candidate prompt words. The quality loss is calculated based on the change in the analysis accuracy of each candidate prompt word relative to the original prompt word. The optimal prompt word is selected based on the quality loss of each candidate prompt word and the pre-calculated number of tokens. S4, based on the optimal prompt words, guides the large model to reprocess the text units corresponding to the category labels associated with rule changes, thereby reducing token consumption during the large model's processing.
2. The method for reducing token consumption in large models according to claim 1, characterized in that, In step S1, the analysis accuracy of the corresponding original prompt word is obtained through the following steps: Obtain a validation sample set that has been labeled with real category tags; The original prompt words are applied to the verification sample set to guide the large model to extract several sentence fragments related to the preset analysis rules from each verification sample in the verification sample set. The extracted sentence fragments are then input into the preset analysis rules to output the category label of each verification sample. The category label of each output validation sample is compared with the corresponding true category label, and the proportion of correct outputs is calculated as the analysis accuracy of the original prompt words.
3. The method for reducing token consumption in large models according to claim 1, characterized in that, In step S2, the changes to the preset analysis rules include one or more of the following: adding or removing category labels and adding or removing analysis dimensions in the preset analysis rules. The adjustment of the corresponding analysis dimension keywords in the original prompts based on rule changes includes: When the rule change involves adding or removing category labels, add or remove the corresponding analysis dimension keywords from the original prompts. When the rule change involves adding or removing analysis dimensions in the preset analysis rules, the corresponding analysis dimension keywords will be added or deleted from the original prompts.
4. The method for reducing token consumption in large models according to claim 1, characterized in that, In step S3, the several preset compression strategies include one or more combinations of redundant word deletion strategy, original prompt word merging strategy, and encoding replacement strategy; The original prompt word merging strategy includes: Calculate the semantic similarity between every two new prompt words; Using each new prompt word as a node and the semantic similarity between new prompt words exceeding a preset threshold as an edge, a connected graph is constructed, and all new prompt words in the same connected component are assigned to the same merging group. For each merged group, extract the common instruction portion of all new prompts within the merged group as a fixed prompt template, and replace the differences in each new prompt with variable placeholders to obtain the candidate prompts corresponding to the merged group.
5. The method for reducing token consumption in large models according to claim 1, characterized in that, In step S3, the optimal suggestion words are selected through the following steps: For any candidate suggestion word, calculate the quality loss of the candidate suggestion word; the quality loss of the candidate suggestion word conforms to the following formula: L = max(0, (R0-R1) / R0), where L is the quality loss of the candidate suggestion word, R0 is the analysis accuracy of the original suggestion word, and R1 is the analysis accuracy of the candidate suggestion word; R1 and R0 are obtained in the same way. Candidate prompts with a quality loss greater than a preset threshold are removed. The quality loss and token count of the remaining candidate prompts are normalized to the same preset numerical range. The cost parameter value of each candidate prompt is calculated based on the normalized quality loss and token count values. The cost parameter value of any candidate prompt conforms to the following formula: C = T0 + λ × L0, where C is the cost parameter of the candidate prompt, T0 is the normalized value of the token quantity of the candidate prompt, L0 is the normalized value of the quality loss of the candidate prompt, and λ is the preset adjustment coefficient; The candidate prompt word with the smallest corresponding cost parameter value is selected as the optimal prompt word.
6. The method for reducing token consumption in large models according to claim 5, characterized in that, The method also determines the optimal prompt word through the following steps: Calculate the token saving rate based on the number of tokens for the optimal prompt word and the number of tokens for the original prompt word corresponding to the optimal prompt word; If the token saving rate is lower than the preset target, return to step S3, enhance the compression degree of the preset compression strategy, regenerate candidate prompt words and filter new optimal prompt words until the token saving rate is not lower than the preset target or reaches the iteration number threshold; wherein, when the token saving rate is still lower than the preset target when the iteration number threshold is reached, the candidate prompt word with the smallest corresponding cost parameter value during the iteration process is taken as the optimal prompt word.
7. The method for reducing token consumption in large models according to claim 1, characterized in that, The method further includes the following steps: a frequency f of the rule change in a historical time period, and calculating a dynamic compression coefficient γ based on the frequency of the rule change; wherein, γ = max(γ min , 1-f / f max ), γ min is a preset minimum dynamic compression coefficient, f max is a preset maximum frequency corresponding to the historical time period; The product of the dynamic compression coefficient γ and the number of tokens for each original prompt word is used as the threshold for the number of tokens after compression of each original prompt word, so that the number of tokens for the candidate prompt words generated during the next compression is not greater than the corresponding token number threshold.
8. The method for reducing token consumption in large models according to claim 1, characterized in that, The historical text set is stored in a structured database. Each storage record contains a text identifier, generation timestamp, text content, and associated category label corresponding to the text unit, and a time index is established for the text unit.
9. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the method for reducing large model token consumption as described in any one of claims 1-8.
10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.