Fault-tolerant processing method and system for long text in large model service, and storage medium

Abstract

The application relates to the technical field of large language model application, in particular to a fault-tolerant processing method and system for super-long text in a large model service and a storage medium. The method comprises the following steps: obtaining a to-be-processed super-long text and a maximum Token threshold of a large model context window, completing Tokenization and judging whether the Token is out of limit; starting a multi-level alternative solution containing fast compression, dynamic compression and chat history compression; starting an error recovery mechanism containing hierarchical exception processing, intelligent degradation and parameter verification; outputting a target text with a compliant Token number; the system comprises a text acquisition module, a multi-level alternative solution execution module, an error recovery module and an output module; the storage medium stores a computer program, and the computer program realizes the above method when executed, can be adapted to multiple models, meets the industrial super-long text processing demand, and solves the problems of the absence of a fault-tolerant mechanism, Token out-of-limit errors caused by insufficient boundary processing and unstable reasoning service in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of large language model application technology, specifically involving a fault-tolerant processing method and system for ultra-long texts in large model services, as well as a storage medium. Background Technology

[0002] With the rapid development of large language model technology, its application in fields such as intelligent assistants, professional customer service platforms, and collaborative office dialogue systems is becoming increasingly widespread. At this time, the efficient processing of long dialogue context has become the core challenge to ensure the performance of dialogue systems and user experience. Long dialogue context contains key logic, user needs and preferences, and scenario information in the dialogue history. Whether it can be accurately managed directly determines the depth of understanding of dialogue intent and the rationality of response generation by the large language model.

[0003] However, existing long context processing technologies in large language model dialogue systems still have significant shortcomings and are difficult to meet the needs of practical applications: First, there is a lack of fault tolerance mechanisms. Although some current technical solutions have introduced dynamic and static compression mechanisms that combine dynamic compression and static compression, when the length of the corresponding token after text processing by such compression mechanisms still exceeds the limit of the large language model context window, there is a lack of effective alternative processing solutions that can be activated immediately. This leads to the long context processing process being prone to interruption and unable to achieve continuous service output. Second, there is insufficient boundary handling. Existing technologies cannot accurately deal with edge cases that are close to the length of the large language model context window (i.e., the maximum number of tokens in the context window corresponding to the large model, denoted as session_len). For example, when the original length of the dialogue context is close to the upper limit of session_len, or the length after compression is near the critical value of session_len, it is difficult to achieve a balance between preserving the integrity of the content and the compliance of the length. This can easily lead to problems such as excessive truncation of context information or the length still exceeding the limit.

[0004] The aforementioned technical defects directly lead to a significant decrease in the stability of the large language model inference service. In particular, when processing ultra-long text-based dialogue scenarios such as engineering technical documents, project tenders, and commercial contracts, the token quantity limit error occurs frequently. This not only interrupts the current inference service process but may also cause the loss of key information in the dialogue history. This seriously affects the reliability of the large language model in professional fields (such as engineering collaboration and business negotiations) and fails to meet the requirements of industrial-grade dialogue systems for service stability, fault tolerance, and ultra-long text processing capabilities.

[0005] In view of this, the present invention is hereby proposed. Summary of the Invention

[0006] To address the aforementioned technical problems in existing technologies, this invention provides a fault-tolerant processing method and system for ultra-long text in large model services, as well as a storage medium, which solves the problems of lack of fault tolerance mechanisms, insufficient boundary processing leading to token over-limit errors, and instability of inference services in existing technologies.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] Firstly, the fault-tolerance methods for ultra-long texts in large model services include:

[0009] S1. Obtain the maximum number of tokens threshold for the context window corresponding to the long text to be processed and the large model;

[0010] S2. Activate the multi-level alternative solution execution strategy to compress the long text to be processed. The multi-level alternative solution execution strategy includes at least one of the following: fast compression strategy, dynamic compression strategy, and chat history compression strategy.

[0011] S3. During the execution of the multi-level alternative solution execution strategy, an error recovery mechanism is activated to handle abnormal situations. The error recovery mechanism includes at least one of the following: a layered exception handling architecture, an intelligent degradation strategy, and parameter verification and boundary protection.

[0012] S4. Output the compressed target text with a token count not exceeding the maximum token count threshold.

[0013] Furthermore, the fast compression strategy employs a six-level degradation strategy, specifically including:

[0014] Level 1 direct check: Determine whether the number of tokens corresponding to the text to be processed is less than or equal to the maximum number of tokens threshold. If so, return the text to be processed directly.

[0015] Second-level intelligent interception: If the number of tokens exceeds the limit, calculate the number of starting tokens and ending tokens to be retained, and intercept and combine the beginning and end parts of the token sequence.

[0016] Third-level omission token insertion: Insert an omission token sequence into the combined result. If the total length still exceeds the limit, retain the first maximum token number threshold tokens.

[0017] Level 4 character-level truncation: If the number of tokens still exceeds the limit, character-level truncation is performed based on an average relationship of 3 characters / token;

[0018] Level 5 sentence boundary truncation: Identifies sentence boundaries in the text after Level 4 truncation, preserving the complete content before the last punctuation mark;

[0019] Level 6 Forced Truncation: If no valid punctuation is found, return the first min(100, len(text)) characters of the original text as a minimum result.

[0020] Furthermore, the chat history compression strategy employs a seven-level degradation strategy, specifically including:

[0021] Level 1 message quantity control: If the total number of original chat history messages is greater than the preset maximum number of historical messages, all system messages are retained, and the most recent maximum number of non-system messages among user messages and assistant messages is retained, where the maximum number of non-system messages = max(1, preset maximum number of historical messages - number of system messages);

[0022] Second-level historical text compression: Convert the filtered chat history into continuous text and compress it using the fast compression strategy described above;

[0023] The third level retains the most recent message: If the text after the second level compression is empty or contains only whitespace characters, the most recent message in the chat history is retained;

[0024] Level 4 Single Message Compression: If the number of single message tokens retained in Level 3 is greater than the number of available tokens, the fast compression strategy described above is invoked to compress the message content;

[0025] Fifth-level aggressive compression: If the total number of tokens after the fourth-level processing is still greater than the number of available tokens, set the maximum number of tokens = the number of available tokens / / 3, and call the fast compression strategy again for compression;

[0026] Level 6 returns an empty list: If the limit is still exceeded after Level 5 processing, an empty message list is returned;

[0027] Level 7 Exception Recovery: If an exception occurs during the execution of the degradation strategy, the chat history messages are traversed in reverse to return the last user message; even in the event of an extreme situation, an empty value should be returned to avoid an error, ensuring a return value in 100% of cases.

[0028] Furthermore, the dynamic compression strategy employs a dual-path strategy, specifically including:

[0029] Main path fast compression: If the fast compression flag use_fast_compression is enabled, the fast compression strategy is invoked to compress the text to be processed, and the compressed text and a success flag are returned.

[0030] Alternate path segmented compression: If the main path fails or is not enabled, the text to be processed is divided into multiple text segments; for each text segment, its importance score is calculated, and the compression ratio is adjusted based on the score; the fast compression strategy is called to compress each segment, and the segments are merged to obtain the final compressed text.

[0031] Furthermore, the importance score is calculated using a hybrid scoring algorithm, specifically as follows:

[0032]

[0033] in, Score the keyword matching of a text segment; The TF-IDF score for the text segment.

[0034] Furthermore, the layered exception handling architecture includes three layers of protection, specifically including:

[0035] Top-level protection: Each main method in the multi-level alternative solution execution strategy is wrapped with a try-catch statement; if the method execution fails, the exception information is output and the preset result and failure flag are returned;

[0036] Mid-layer protection: critical calculation steps; if a step execution fails, output the exception information and return the preset default value;

[0037] Underlying protection: Boundary checks and parameter validation are performed on basic parameters: If the text is empty or contains only blank characters, the original text and failure flag are returned; if the maximum token quantity threshold is ≤0, empty text and failure flag are returned, but there is always a fallback mechanism. In extreme cases, the compressed text will be empty to ensure that the service will not be interrupted.

[0038] Furthermore, the intelligent degradation strategy includes:

[0039] Importance calculation downgrade: If keyword matching fails, return a base score of 0.5; if TF-IDF calculation fails, return a default importance score of 0.5; if the text fragment length is <50 characters, directly return an importance score of 0.5.

[0040] Compression operation degradation: If token encoding fails, a character-level truncation strategy is used; if intelligent truncation fails, a sentence boundary truncation strategy is used; if sentence boundary truncation fails, a forced truncation strategy is used.

[0041] Message processing degradation: If batch message compression fails, switch to single message compression; if single message compression fails, return an empty message list; if all compression fails, retain the last user message.

[0042] Secondly, the fault-tolerant processing system for ultra-long texts in large model services includes:

[0043] Text acquisition module: used to acquire the maximum number of tokens in the context window corresponding to the long text to be processed and the large model;

[0044] Multi-level alternative solution execution module: used to initiate a multi-level alternative solution execution strategy to compress the long text to be processed. The multi-level alternative solution execution strategy includes at least one of the following: fast compression strategy, dynamic compression strategy, and chat history compression strategy.

[0045] Error recovery module: used to activate the error recovery mechanism to handle abnormal situations during the execution of the multi-level alternative scheme execution strategy. The error recovery mechanism includes at least one of the following: layered exception handling architecture, intelligent degradation strategy, and parameter verification and boundary protection.

[0046] Output module: Used to output compressed target text with a token count not exceeding the maximum token count threshold.

[0047] Thirdly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described fault-tolerant processing method for ultra-long texts in large model services.

[0048] Compared with existing technologies, the present invention provides a fault-tolerant processing method and system for ultra-long text in the above-mentioned large model service, as well as a storage medium. The method includes: obtaining the ultra-long text to be processed and the maximum token threshold of the large model context window, completing tokenization and determining whether it exceeds the limit; initiating a multi-level alternative scheme including fast compression, dynamic compression, and chat history compression; initiating an error recovery mechanism including hierarchical anomaly handling, intelligent degradation, and parameter verification; outputting target text with a compliant token number; the system includes modules for text acquisition, execution of multi-level alternative schemes, error recovery, and output; the storage medium stores a computer program that implements the above method during execution, which can adapt to multiple models and meet the industrial-grade ultra-long text processing needs; and solves the problems of lack of fault tolerance mechanism, insufficient boundary handling leading to token exceeding the limit error, and unstable inference service in existing technologies. Attached Figure Description

[0049] Figure 1 A flowchart illustrating the fault-tolerant processing method for ultra-long texts in the large model service provided in this embodiment of the invention. Detailed Implementation

[0050] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0051] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.

[0052] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.

[0053] Example 1

[0054] See Figure 1 , Figure 1 This is a flowchart illustrating the fault-tolerant processing method for ultra-long texts in the large model service proposed in this invention. Specific steps may include:

[0055] S1. Obtain the text to be processed and the Token threshold: Obtain the maximum number of Tokens in the context window corresponding to the ultra-long text to be processed and the large model;

[0056] S11. Text Acquisition: Read the engineering technical document to be processed through the API interface. The document contains "Project Overview (800 Tokens), Technical Parameters (2200 Tokens), Implementation Plan (1500 Tokens), Risk Statement (700 Tokens)".

[0057] S12. Token Threshold Determination: Call the large model metadata interface to obtain the maximum number of tokens in the context window of the currently used model (e.g., GPT-3.5). Maximum number of tokens = 4096. Reserve 100 tokens to reserve space for subsequent response generation. Calculate the actual threshold that can be used for text compression (e.g., "the actual threshold that can be used for text compression is 4096-100=3996").

[0058] S13. Tokenization Processing: Supported models and their corresponding tokenizers convert the engineering technical documents to be processed into a token sequence; the original token count is calculated using the following formula: The comparison confirmed that 5200 > 3996, meaning that the number of original text tokens exceeded the actual usable threshold, and a multi-level alternative strategy needed to be implemented for compression processing.

[0059] S2. Initiate multi-level alternative execution strategy

[0060] The multi-level alternative solution execution strategy is activated to compress the long text to be processed. The multi-level alternative solution execution strategy includes at least one of the following: fast compression strategy, dynamic compression strategy, and chat history compression strategy.

[0061] S21. The fast compression strategy employs a six-level degradation strategy, specifically including:

[0062] Level 1 direct check: Determine whether the number of tokens corresponding to the text to be processed is less than or equal to the maximum number of tokens threshold. If so, return the text to be processed directly.

[0063] Second-level intelligent interception: If the number of tokens exceeds the limit, calculate the number of starting tokens and ending tokens to be retained, and intercept and combine the beginning and end parts of the token sequence.

[0064] Third-level omission token insertion: Insert an omission token sequence into the combined result. If the total length still exceeds the limit, retain the first maximum token number threshold tokens.

[0065] Level 4 character-level truncation: If the number of tokens still exceeds the limit, character-level truncation is performed based on an average relationship of 3 characters / token;

[0066] Level 5 sentence boundary truncation: Identifies sentence boundaries in the text after Level 4 truncation, preserving the complete content before the last punctuation mark;

[0067] Level 6 Forced Truncation: If no valid punctuation is found, return the first min(100, len(text)) Chinese characters of the original text as a minimum result.

[0068] S22. The chat history compression strategy adopts a seven-level degradation strategy, specifically including:

[0069] Level 1 message quantity control: If the total number of original chat history messages is greater than the preset maximum number of historical messages, all system messages are retained, and the most recent maximum number of non-system messages among user messages and assistant messages is retained, where the maximum number of non-system messages = max(1, preset maximum number of historical messages - number of system messages);

[0070] Second-level historical text compression: Convert the filtered chat history into continuous text and compress it using the fast compression strategy described above;

[0071] The third level retains the most recent message: If the text after the second level compression is empty or contains only whitespace characters, the most recent message in the chat history is retained;

[0072] Level 4 Single Message Compression: If the number of single message tokens retained in Level 3 is greater than the number of available tokens, the fast compression strategy described above is invoked to compress the message content;

[0073] Fifth-level aggressive compression: If the total number of tokens after the fourth-level processing is still greater than the number of available tokens, set the maximum number of tokens = the number of available tokens / / 3, and call the fast compression strategy again for compression;

[0074] The sixth level returns an empty list: If it is still over the limit after the fifth level processing, return an empty message list;

[0075] The seventh level exception recovery: If an exception occurs during the execution of the degradation strategy, traverse the chat history messages in reverse and return the last user message; If an extreme situation occurs finally, it should also be able to return a null value without reporting an error, ensuring that there is a return value in 100% of the cases.

[0076] S23. The dynamic compression strategy adopts a dual-path strategy, specifically including:

[0077] The main path fast compression: If the fast compression flag use_fast_compression is enabled, call the fast compression strategy to compress the text to be processed, and return the compressed text and the success flag;

[0078] The backup path segment compression: If the main path fails to execute or is not enabled, split the text to be processed into multiple text segments; For each text segment, calculate its importance score, and adjust the compression ratio based on the score; Call the fast compression strategy to compress each segment, and merge the segments to obtain the final compressed text.

[0079] Among them, the importance score is calculated using a hybrid scoring algorithm, and the specific formula is:

[0080]

[0081] Among them, is the keyword matching score of the text segment. If segment 2 contains keywords such as "core parameters, material standards", the score is 0.9, and if segment 1 contains the keyword "project name", the score is 0.5; is the TF-IDF score of the text segment. Using an improved algorithm, meaningless words with a length < 2 (such as "of, is") are ignored, and combining "the highest score of segment 2 is 0.9 (70% weight) + the average score of 0.7 (30% weight)", the calculated score is 0.9 * 0.7 + 0.7 * 0.3 = 0.84.

[0082] Let the basic compression ratio (compression_ratio) = 0.8, and adjust the compression intensity according to the segment compression ratio (segment_ratio): segment_ratio = 0.8 * (1.0 - 0.864 * 0.5) = 0.8 * (1 - 0.432) = 0.4544. The compression ratio of segment 2 is 45.44% (less compression, retaining more key information), and the compression ratio of segment 1 is 0.8 * (1 - 0.5 * 0.5) = 0.6 (more compression); Call the fast compression strategy to compress according to the ratio of each segment, and obtain the final text after merging.

[0083] S3. During the execution of the multi-level alternative solution execution strategy, an error recovery mechanism is activated to handle abnormal situations. The error recovery mechanism includes at least one of the following: a layered exception handling architecture, an intelligent degradation strategy, and parameter verification and boundary protection.

[0084] S31. The layered exception handling architecture includes three layers of protection, specifically:

[0085] Top-level protection: Each main method in the multi-level alternative solution execution strategy is wrapped with a try-catch statement; if the method execution fails, the exception information is output and the preset result and failure flag are returned;

[0086] Mid-layer protection: critical calculation steps; if a step execution fails, output the exception information and return the preset default value;

[0087] Underlying protection: Boundary checks and parameter validation are performed on basic parameters: If the text is empty or contains only blank characters, the original text and failure flag are returned; if the maximum token quantity threshold is ≤0, empty text and failure flag are returned, but there is always a fallback mechanism. In extreme cases, the compressed text will be empty to ensure that the service will not be interrupted.

[0088] Extreme cases include, but are not limited to: an empty chat history message list; text to be processed containing only garbled characters (such as ASCII non-printable characters); and an empty result after token encoding.

[0089] S32, Intelligent Degradation Strategies include:

[0090] Importance calculation downgrade: If keyword matching fails, return a base score of 0.5; if TF-IDF calculation fails, return a default importance score of 0.5; if the text fragment length is <50 characters (including Chinese and English, counted according to Unicode encoding), directly return an importance score of 0.5;

[0091] Compression operation degradation: If token encoding fails, a character-level truncation strategy is used; if intelligent truncation fails, a sentence boundary truncation strategy is used; if sentence boundary truncation fails, a forced truncation strategy is used.

[0092] Message processing degradation: If batch message compression fails, switch to single message compression; if single message compression fails, return an empty message list; if all compression fails, retain the last user message.

[0093] S4. Output the compressed target text with a token count not exceeding the maximum token count threshold.

[0094] Secondly, the fault-tolerant processing system for ultra-long texts in the large model service proposed in this invention may specifically include:

[0095] Text acquisition module: used to acquire the maximum number of tokens in the context window corresponding to the long text to be processed and the large model;

[0096] Multi-level alternative solution execution module: used to initiate a multi-level alternative solution execution strategy to compress the long text to be processed. The multi-level alternative solution execution strategy includes at least one of the following: fast compression strategy, dynamic compression strategy, and chat history compression strategy.

[0097] Error recovery module: used to activate the error recovery mechanism to handle abnormal situations during the execution of the multi-level alternative scheme execution strategy. The error recovery mechanism includes at least one of the following: layered exception handling architecture, intelligent degradation strategy, and parameter verification and boundary protection.

[0098] Parameter validation includes, but is not limited to: whether the number of available tokens is negative (if negative, set to 0); whether the number of text fragments exceeds the preset limit (e.g., no more than 20 fragments per split to avoid memory overflow); and whether the keyword matching library is empty (if empty, automatically load the default industry keyword library).

[0099] Output module: Used to output compressed target text with a token count not exceeding the maximum token count threshold.

[0100] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned fault-tolerant processing method for ultra-long texts in large model services.

[0101] In summary, the present invention has the following advantages:

[0102] 1. Significantly improved compression efficiency: By designing multi-level alternative solutions, the success rate of text compression is significantly improved; at the same time, compression performance is optimized, the average compression time is shortened, and the overall compression efficiency is effectively improved.

[0103] 2. Excellent text quality is maintained: Key text information is effectively preserved during compression; the semantic integrity of the text is improved by leveraging the TF-IDF scoring mechanism; and the user experience is improved in dialogue scenarios.

[0104] 3. Significantly enhanced system stability: It has comprehensive exception handling capabilities and can return usable results in various abnormal situations; effectively improving system availability and shortening error recovery time.

[0105] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A fault-tolerant processing method for super-long text in a large model service, characterized in that, include: S1. Obtain the maximum number of tokens threshold for the context window corresponding to the long text to be processed and the large model; S2. Activate the multi-level alternative solution execution strategy to compress the long text to be processed. The multi-level alternative solution execution strategy includes at least one of the following: fast compression strategy, dynamic compression strategy, and chat history compression strategy. The chat history compression strategy adopts a seven-level degradation strategy, specifically including: Level 1 Message Count Control: If the total number of original chat history messages exceeds the preset maximum number of historical messages, all system messages are retained, and the most recent non-system message count among user messages and assistant messages is also retained, where the maximum number of non-system messages = max(1, preset maximum number of historical messages). (Number of system messages); Second-level historical text compression: Convert the filtered chat history into continuous text and compress it using the fast compression strategy described above; The third level retains the most recent message: If the text after the second level compression is empty or contains only whitespace characters, the most recent message in the chat history is retained; Level 4 Single Message Compression: If the number of single message tokens retained in Level 3 is greater than the number of available tokens, the fast compression strategy described above is invoked to compress the message content; Fifth-level aggressive compression: If the total number of tokens after the fourth-level processing is still greater than the number of available tokens, set the maximum number of tokens = the number of available tokens / / 3, and call the fast compression strategy again for compression; Level 6 returns an empty list: If the limit is still exceeded after Level 5 processing, an empty message list is returned; Level 7 Exception Recovery: If an exception occurs during the execution of the degradation strategy, the chat history messages are traversed in reverse to return the last user message; if an extreme case occurs at the end, it should also be able to return a null value. S3. During the execution of the multi-level alternative solution execution strategy, an error recovery mechanism is activated to handle abnormal situations. The error recovery mechanism includes at least one of the following: a layered exception handling architecture, an intelligent degradation strategy, and parameter verification and boundary protection. S4. Output the compressed target text with a token count not exceeding the maximum token count threshold.

2. The fault-tolerant processing method for super-long text in a large model service according to claim 1, characterized in that, The fast compression strategy employs a six-level degradation strategy, specifically including: Level 1 direct check: Determine whether the number of tokens corresponding to the text to be processed is less than or equal to the maximum number of tokens threshold. If so, return the text to be processed directly. Second-level intelligent interception: If the number of tokens exceeds the limit, calculate the number of starting tokens and ending tokens to be retained, and intercept and combine the beginning and end parts of the token sequence. Third-level omission token insertion: Insert an omission token sequence into the combined result. If the total length still exceeds the limit, retain the first maximum token number threshold tokens. Level 4 character-level truncation: If the number of tokens still exceeds the limit, character-level truncation is performed based on an average relationship of 3 characters / token; Level 5 sentence boundary truncation: Identifies sentence boundaries in the text after Level 4 truncation, preserving the complete content before the last punctuation mark; Level 6 Forced Truncation: If no valid punctuation is found, return the first min(100, len(text)) characters of the original text as a minimum result. 3.The method of claim 1, wherein, The dynamic compression strategy employs a dual-path approach, specifically including: Main path fast compression: If the fast compression flag use_fast_compression is enabled, the fast compression strategy is invoked to compress the text to be processed, and the compressed text and a success flag are returned. Alternate path segmented compression: If the main path fails or is not enabled, the text to be processed is divided into multiple text segments; for each text segment, its importance score is calculated, and the compression ratio is adjusted based on the score; the fast compression strategy is called to compress each segment, and the segments are merged to obtain the final compressed text.

4. The fault-tolerant processing method for super-long text in a large model service according to claim 3, characterized in that, The importance score is calculated using a hybrid scoring algorithm, and the specific formula is as follows: in, Score the keyword matching of a text segment; The TF-IDF score for the text segment. 5.The method of claim 1, wherein, The layered exception handling architecture includes three layers of protection, specifically: Top-level protection: Each main method in the multi-level alternative solution execution strategy is wrapped with a try-catch statement; if the method execution fails, the exception information is output and the preset result and failure flag are returned; Mid-layer protection: critical calculation steps; if a step execution fails, output the exception information and return the preset default value; Underlying protection: Boundary checks and parameter validation are performed on basic parameters: If the text is empty or contains only blank characters, the original text and failure flag are returned; if the maximum token quantity threshold is ≤0, empty text and failure flag are returned, but there is always a fallback mechanism. In extreme cases, the compressed text will be empty to ensure that the service will not be interrupted. 6.The method of claim 1, wherein, The intelligent degradation strategy includes: Importance calculation downgrade: If keyword matching fails, return a base score of 0.5; if TF-IDF calculation fails, return a default importance score of 0.5; if the text fragment length is <50 characters, directly return an importance score of 0.

5. Compression operation degradation: If token encoding fails, a character-level truncation strategy is used; if intelligent truncation fails, a sentence boundary truncation strategy is used; if sentence boundary truncation fails, a forced truncation strategy is used. Message processing degradation: If batch message compression fails, switch to single message compression; if single message compression fails, return an empty message list; if all compression fails, retain the last user message.

7. A fault-tolerant processing system for super-long text in a large model service, characterized by, include: Text acquisition module: used to acquire the maximum number of tokens in the context window corresponding to the long text to be processed and the large model; Multi-level alternative solution execution module: used to initiate a multi-level alternative solution execution strategy to compress the long text to be processed. The multi-level alternative solution execution strategy includes at least one of the following: fast compression strategy, dynamic compression strategy, and chat history compression strategy. The chat history compression strategy adopts a seven-level degradation strategy, specifically including: Level 1 Message Count Control: If the total number of original chat history messages exceeds the preset maximum number of historical messages, all system messages are retained, and the most recent non-system message count among user messages and assistant messages is also retained, where the maximum number of non-system messages = max(1, preset maximum number of historical messages). (Number of system messages); Second-level historical text compression: Convert the filtered chat history into continuous text and compress it using the fast compression strategy described above; The third level retains the most recent message: If the text after the second level compression is empty or contains only whitespace characters, the most recent message in the chat history is retained; Level 4 Single Message Compression: If the number of single message tokens retained in Level 3 is greater than the number of available tokens, the fast compression strategy described above is invoked to compress the message content; Fifth-level aggressive compression: If the total number of tokens after the fourth-level processing is still greater than the number of available tokens, set the maximum number of tokens = the number of available tokens / / 3, and call the fast compression strategy again for compression; Level 6 returns an empty list: If the limit is still exceeded after Level 5 processing, an empty message list is returned; Level 7 Anomaly Recovery: If an anomaly occurs during the execution of the degradation strategy, the chat history messages are traversed in reverse to return the last user message; if an extreme case occurs at the end, it should also be able to return a null value; Error Recovery Module: Used to activate the error recovery mechanism to handle abnormal situations during the execution of the multi-level alternative scheme execution strategy. The error recovery mechanism includes at least one of the following: layered anomaly handling architecture, intelligent degradation strategy, and parameter verification and boundary protection; Output module: Used to output compressed target text with a token count not exceeding the maximum token count threshold.

8. A computer readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the fault-tolerant processing method for ultra-long texts in the large model service of any one of claims 1-6.

Images