Weak network environment-oriented method and device for optimizing interactive service of large language model

Abstract

The invention relates to a method and a device for optimizing a large language model interactive service for a weak network environment, which are particularly suitable for regions with limited network infrastructures. The method comprises the following steps: establishing an interactive self-adaptive hierarchical service architecture of a large language model, and dynamically adjusting LLM context management and generation control according to a network condition; establishing a semantic local cache mechanism oriented to large language model interaction, and performing local storage and offline response of LLM specific key data at a user equipment side; establishing an asynchronous fault-tolerant processing mechanism of large language model interaction, and providing an interaction continuity guarantee when the network is abnormal; and establishing a transmission optimization strategy of the large language model, and optimizing the transmission characteristics of the data. According to the method, the special problems of response delay, interaction interruption, bandwidth limitation and the like of interactive application of the large language model in a weak network environment are solved, and the service stability and the user interaction fluency of the large language model are effectively improved.

Description

Technical Field

[0001] This invention relates to a method and apparatus for optimizing the performance of large-scale language model interactive services in weak network environments, particularly suitable for areas with limited network infrastructure (such as rural or remote areas). Specifically, it involves optimization methods such as layered service architecture, local caching and offline functionality, adaptive latency handling, and network bandwidth optimization, thereby improving the service stability of large-scale language model interactive services in weak network environments, and belongs to the field of artificial intelligence. Background Technology

[0002] In recent years, with the widespread application of large-scale language models in various scenarios such as intelligent question answering, medical consultation, and educational tutoring, higher demands have been placed on the network environment. Especially in interactive services, the frequent and continuous information exchange between users and models is highly dependent on network stability, bandwidth, and response time. In urban areas and regions with well-developed network infrastructure, the user experience is generally smooth. However, in rural areas, remote mountainous regions, and developing areas, weak network infrastructure, frequent network fluctuations, high latency, and high packet loss rates severely impact the efficiency and availability of large language model services.

[0003] Current solutions attempt to reduce network dependence through model compression and local inference, but these methods typically sacrifice model performance and struggle to meet high-quality interaction requirements. For example, edge computing deployment (Cai, Fenglong, et al. "Edge-llm: A collaborative framework for large language model serving in edge computing." 2024 IEEE International Conference on Web Services (ICWS). IEEE, 2024.) proposed in existing literature improves response speed by deploying part of the model locally, but this method fails to effectively address the dynamic network changes in long-distance communication and places high demands on the computing power of terminal devices, making it difficult to deploy widely in resource-constrained environments. Similarly, while methods such as model compression (Zhu, Xunyu, et al. "Asurvey on model compression for large language models." Transactions of the Association for Computational Linguistics 12 (2024): 1556-15.) can reduce the inference burden of large-scale language models on the server side and improve response speed, they still fail to solve the communication latency and packet loss caused by the network environment, and cannot bring intuitive user experience improvement to end users in restricted network environments.

[0004] In existing technologies, large language model interaction services mainly suffer from the following drawbacks: 1) High dependence on network: The current model service is mainly based on cloud computing. All requests need to be uploaded to the server for processing in real time. Once the network quality deteriorates, the model response will slow down significantly or even be interrupted.

[0005] 2) Lack of flexible design: Most existing solutions do not consider dynamically adjusting model interaction parameters (such as context length and number of generated words) to cope with real-time network fluctuations.

[0006] 3) No fault tolerance mechanism for disconnection: Once communication is interrupted, users are often unable to continue their tasks, resulting in a service experience interruption.

[0007] 4) Low bandwidth utilization: A large amount of redundant data is still transmitted in weak network environments, further increasing the network burden.

[0008] 5) Limited evaluation methods: Existing tests are mostly conducted in ideal network environments, lacking performance verification for extreme weak network scenarios. Summary of the Invention

[0009] This invention aims to solve the above-mentioned technical problems and provides a method and apparatus for optimizing the performance of large-scale language model interactive services in weak network environments. Through layered service architecture, local caching and offline functions, adaptive latency processing and bandwidth optimization strategies, it effectively improves service stability and user interaction smoothness, and is especially suitable for areas with limited network infrastructure.

[0010] The technical solution adopted in this invention is as follows: A method for optimizing interactive services of large language models in weak network environments includes the following steps: Establish a layered service architecture for interactive adaptation of large language models, and dynamically adjust the context management and generation control of large language models according to network conditions; Establish a semantic local caching mechanism for interaction with large language models, and perform local storage and offline response of key data unique to large language models on user devices; Establish an asynchronous fault-tolerant processing mechanism for large language model interactions to ensure interaction continuity in the event of network anomalies; Establish a transmission optimization strategy for large language models to optimize data transmission characteristics.

[0011] Furthermore, the layered service architecture includes: When network conditions are good, enable the full context window to maintain the integrity of the dialogue history and the richness of the generated content; When network conditions deteriorate, the context window is automatically reduced and the generation length is limited to reduce data transmission and ensure the continuity of basic interactions. Real-time monitoring of network status enables flexible adaptation of large language model services to network conditions, allowing users to perceive only changes in response speed without experiencing functional interruptions.

[0012] Furthermore, the semantic local caching mechanism includes: Semantic compression is performed on historical dialogues to extract key information points and store them as structured knowledge fragments; Pre-generate standardized response content for common question-and-answer patterns in large language models; Context-differential storage is used to store historical information identifiers instead of the full text, and only incrementally changed content is transmitted; When the network is down, the local cache is used to perform basic semantic responses for the large language model, and the difference data is synchronized after the network is restored.

[0013] Furthermore, the asynchronous fault-tolerant processing mechanism includes: After the user enters the input, a placeholder response is returned immediately, and the complete response is processed asynchronously in the background. The response waiting time of the large language model is dynamically adjusted based on network latency to avoid timeout interruptions. The interactive interface continuously displays the "thinking" status to maintain user confidence and continuity in the interaction.

[0014] Furthermore, the transmission optimization strategy includes: Prioritize the transmission content of the large language model, prioritizing the transmission of core text content and delaying unnecessary multimedia elements; By reusing context identifiers, we can avoid repeatedly transmitting historical conversation content. Perform semantic compression transmission and dynamically adjust the compression strategy based on bandwidth.

[0015] Furthermore, the transmission content of the large language model is subject to priority management, wherein the priority includes: High priority: User input, context ID, large language model response results; Medium priority: Dialogue support information, formatting tags; Low priority: multimedia content, style elements.

[0016] An apparatus for optimizing interactive services of large language models in weak network environments, comprising: The layered service module is used to establish an interactive and adaptive layered service architecture for large language models, and dynamically adjust the context management and generation control of large language models according to network conditions. The semantic local caching module is used to establish a semantic local caching mechanism for interaction with large language models, and to store and respond to key data specific to large language models locally on the user device. The asynchronous fault-tolerant processing module is used to establish an asynchronous fault-tolerant processing mechanism for large language model interactions, providing a guarantee of interaction continuity in the event of network anomalies. The transmission optimization module is used to establish transmission optimization strategies for large language models and optimize the transmission characteristics of data.

[0017] This invention addresses the specific needs of large language model interactive services in weak network environments by proposing a specially optimized technical solution. Its core innovations and beneficial effects are as follows: 1) LLM-aware layered service architecture: For the first time, a dynamic adjustment mechanism for the context window of a large language model based on network state is proposed. The LLM interaction parameters are optimized in real time according to network quality, realizing intelligent adaptation of service quality and effectively solving the problem of LLM interaction continuity in weak network environment.

[0018] 2) Semantic-driven local caching mechanism: Design a semantic caching system for large language model interaction. Through dialogue summaries, response templates and differential storage, it can still provide basic LLM interaction capabilities in the event of network outage, which significantly improves service robustness.

[0019] 3) Asynchronous fault-tolerant interactive processing: Introducing LLM interaction decoupling and dynamic timeout mechanism to ensure that users have a continuous and stable interactive experience when the network fluctuates, avoiding the interruption problem of the traditional request-response mode.

[0020] 4) LLM content transmission optimization: In response to the characteristics of large language model text data transmission, priority management, differential transmission and intelligent compression strategies are adopted to maximize bandwidth utilization efficiency. Attached Figure Description

[0021] Figure 1 This diagram illustrates four key technologies for optimizing interactive services of large language models in weak network environments.

[0022] Figure 2 This is a framework diagram of a device for optimizing interactive services of large language models in weak network environments. Detailed Implementation

[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0024] This invention proposes an optimization method for large language model interactive services in weak network environments, specifically addressing the unique problems of response latency, interaction interruption, and bandwidth limitation in large language model interactive applications under weak network conditions. Figure 1 As shown, it specifically includes the following four key technologies: 1) LLM-aware layered service architecture This invention designs a layered service architecture for large language model interaction and adaptation, dynamically adjusting LLM context management and generation control based on network conditions, specifically including: a) When network conditions are good, enable the full context window (e.g., 4096 tokens) to maintain the integrity of the dialogue history and the richness of the generated content; b) When network conditions deteriorate (high latency or packet loss), automatically reduce the LLM context window and limit the generation length to reduce data transmission volume and ensure the continuity of basic interactions; c) The LLM interaction strategy is adjusted in real time by the client network status monitoring module to achieve flexible adaptation of service quality.

[0025] 2) LLM Semantic Local Caching and Offline Response This invention designs a semantic local caching mechanism for large language model interactions, implementing local storage of LLM-specific key data on the user device side, specifically including: a) Semantic summarization of dialogues: Semantic compression of historical dialogues, extraction of key information points, and storage as structured knowledge fragments; b) LLM Response Templates: Standardized response content pre-generated for common question-and-answer patterns in large language models; c) Context Differential Storage: Stores historical information identifiers instead of the full text, and only transmits incrementally changed content upon submission (i.e. each time the user initiates a dialogue with the server); d) When the network is down, the local cache is invoked to perform LLM basic-level semantic response. After the network is restored, the difference data is synchronized. That is, by comparing the newly added local data with the online data in both directions, the data to be synchronized is found. The server returns the data ID that needs to be synchronized to the client, and the client uploads the relevant data.

[0026] 3) LLM interactive asynchronous fault-tolerant processing This invention designs an asynchronous fault-tolerant processing mechanism (or asynchronous task scheduling mechanism) unique to large language model interactions, providing a guarantee of interaction continuity in the event of network anomalies, specifically including: a) Introduce an LLM interaction decoupling mechanism, where a placeholder response is returned immediately after user input, and the complete response is processed asynchronously in the background; b) Dynamic timeout adaptation: Dynamically adjust the LLM response waiting time based on network latency to avoid unnecessary timeout interruptions; c) Maintaining the interactive state: The interface continuously displays the "thinking" state to maintain user confidence and continuity in interaction.

[0027] 4) LLM data transfer optimization strategy This invention designs a communication content optimization scheme for large language models, specifically optimizing for the transmission characteristics of text-based data, including: a) LLM content priority management: Prioritize the transmission of core text content and delay unnecessary multimedia elements; b) Context Differential Transmission: Avoids repeated transmission of historical dialogue content by reusing context identifiers; c) Semantic compression transmission: Employ compression algorithms suitable for text features and dynamically adjust the compression strategy based on bandwidth.

[0028] The following details the specific implementation of the large language model interaction service optimization method for weak network environments, covering core aspects such as LLM interaction feature adaptation, network state awareness, hierarchical service strategy, semantic cache management, asynchronous fault tolerance processing, and data transmission optimization.

[0029] 1. System Architecture and Operating Environment This system is specifically designed for interactive scenarios involving large language models, including: 1.1) Client (User Terminal Equipment): a) LLM interactive interface: Web-based interactive applications; b) Network status monitoring: A monitoring module specifically optimized for LLM interactions; c) Semantic cache management: storage of dialogue summaries and response templates for large language models; d) Asynchronous task scheduling (asynchronous fault-tolerant processing mechanism): A task management mechanism unique to LLM interaction.

[0030] 1.2) Server-side (Cloud-based LLM service): a) Large language model inference engines: LLM-specific inference frameworks such as vLLM and SGLang; b) Context Management: LLM Dialogue Context Storage and Reuse System; c) Response generation: LLM response generation module optimized for weak networks.

[0031] The client and server communicate via HTTP / HTTPS protocols, supporting the long connection and disconnection recovery mechanisms unique to LLM interaction.

[0032] 2. Implementation of LLM Layered Service Architecture The layered service architecture dynamically adjusts LLM service parameters based on real-time network conditions.

[0033] 2.1) Network status monitoring: a) Key LLM interaction metrics such as RTT (Round Trip Time), packet loss rate, and available bandwidth; b) The sliding window algorithm is used for smooth processing to avoid instantaneous fluctuations affecting the stability of the LLM service.

[0034] 2.2) Network Status Classification: a) Good: RTT < 200ms, packet loss rate < 2%, bandwidth > 2Mbps; b) General: 200ms≤RTT<800ms, packet loss rate 2%-10%, bandwidth 0.5-2Mbps; c) Poor: RTT≥800ms, packet loss rate>10%, bandwidth<0.5Mbps.

[0035] 2.3) Dynamic adjustment of LLM services: Depending on the different network status levels, the client and server dynamically adjust their data processing and transmission as follows: a) Good state: Enable the full context window of the model (e.g., 4096 or more tokens), allow the generation of longer text (e.g., 512~1024 tokens), and maintain interactive coherence and information richness; b) Normal state: Reduce the context window to 2048 tokens, limit the length of a single generation to 256~512 tokens, and add a length penalty parameter to reduce the amount of data requested per request; c) Poor state: The context window is further reduced to 1024 tokens, the generation length is limited to 128~256 tokens, the length penalty parameter is further increased, and unnecessary functions (such as complex typesetting) are disabled.

[0036] After the client monitoring module determines the network status change, it notifies the server via network message. The server adjusts the model inference configuration based on the received context window and generation length parameter. The adjustment process is transparent to the user: the user can know that the current network problem has caused the response speed to change, but the conversation is still going on normally.

[0037] 3. Implementation of Semantic Local Caching 3.1) Cache content: a) Semantic summary of dialogue: Semantically compress past dialogues, extract key information points, and store them as structured text fragments (stored in knowledge graph or JSON format), rather than the original full text of the dialogue; b) LLM response template: pre-generated standardized response content, such as "I am temporarily unable to connect to the internet, please try again later" or "Your question has been recorded and will be answered in detail once you are connected to the internet." c) Differential identifier storage: Stores historical information identifiers instead of the full text content.

[0038] 3.2) Cache Management: After each successful interaction with the server, the client stores key information from the current conversation (such as summaries of the latest rounds of dialogue and the generated final answer) in its local cache. a) Update the latest conversation summary after a successful interaction; b) Adopt a time-based cache update strategy (e.g., 1-day validity period); c) Incremental synchronization of discrepancies after network recovery: The client actively uploads locally unsynchronized questions and temporary answers to the server through an incremental synchronization mechanism, and the server performs consistency verification and supplementary storage.

[0039] 4. Implementation of Asynchronous Fault Tolerance 4.1) Interactive decoupling: a) The "Thinking" status is displayed immediately after the user inputs the information; b) The background asynchronously processes the complete LLM response.

[0040] 4.2) Dynamic timeout: a) Base timeout of 5 seconds, dynamically extended based on network latency; b) Maximum timeout of 30 seconds to ensure the integrity of LLM response; c) If real-time monitoring detects that the network latency exceeds the threshold (e.g., 300ms), the timeout period will be extended proportionally. For example, for every 100ms increase in latency, the timeout period will increase by 1 second, up to a maximum of 30 seconds. During the response timeout period, the client's user interface will maintain a "responding" animation or progress bar to prevent user misoperation or duplicate submission.

[0041] 4.3) State maintenance: a) Maintain interactive state using local caching when the network is disconnected; b) Automatically resume unfinished interactions once the network is restored.

[0042] 5. Implementation of Data Transmission Optimization 5.1) Content Priority: a) High priority: User input, context ID, LLM response result; b) Medium priority: Dialogue support information, formatting tags; c) Low priority: multimedia content, style elements.

[0043] 5.2) Differential transmission: a) Avoid repeatedly transmitting the entire dialogue history by reusing context identifiers; b) Only transmit incremental data of the changed parts.

[0044] 5.3) Compression optimization: a) Text data is compressed using Gzip / Brotli; b) Dynamically adjust the compression level based on bandwidth; c) Image attachments use an efficient encoding format.

[0045] To verify the effectiveness of this invention in extreme weak network environments, a simulation test was designed to verify the synergistic effect of the core optimization strategies (layered architecture, local caching, adaptive latency, and bandwidth optimization). The experiment simulated 1000 users engaging in a one-hour Q&A session through an AI health consultation platform (a typical scenario of large-scale model interaction). Network parameters were set as follows: 10% IP switching probability (every 10 seconds), 20% connection interruption probability (every 10 seconds), 60% packet loss rate, 1000ms RTT, and 100Kbps bandwidth (stricter than typical values ​​in rural western China: 300Kbps bandwidth, 10% packet loss, etc.). The final health report submission success rate (i.e., the QoS of this application) was used as the core indicator, and ablation experiments were conducted to compare different optimization configurations. The experimental results are shown in Table 1. Table 1: QoS Indicators of Large-Scale Interactive Applications under Different Optimization Conditions in Weak Network Environments Experimental setup QoS (Success Rate) Integrate all optimization strategies (complete solution) 98.43% Unlayered service architecture 90.25% No local caching or offline functionality 65.50% No adaptive delay processing 70.17% No network bandwidth optimization 76.63% No optimizations (baseline) 43.60% Experimental data shows that the complete solution has significant advantages: with all optimizations integrated, QoS reaches 98.43% under extremely weak network conditions (60% packet loss, 100Kbps), with almost no interruption in user interaction. The lack of local caching and offline functionality caused the success rate to plummet to 65.50% (the largest drop), demonstrating their crucial role in reducing real-time dependence by reusing local data; the lack of adaptive latency processing reduced the success rate to 70.17%, highlighting the importance of dynamic timeouts and retries in maintaining continuous interaction; layered architecture and bandwidth optimization contributed to stability and transmission efficiency, respectively, with the success rate decreasing by 8.18%~21.8% when they were missing. The traditional solution without optimization has a success rate of only 43.60%. This invention addresses the combined challenges of high packet loss, long latency, and network outages under weak network conditions through multi-strategy collaboration.

[0046] In summary, this invention demonstrates excellent robustness in extremely weak network environments, and its technical strategy can be extended to rural and remote scenarios with limited network coverage, providing a reliable guarantee for inclusive AI services.

[0047] The following two examples illustrate the use of the algorithm and solution invented in this patent in specific applications.

[0048] Example 1: Rural users using interactive question-and-answer applications A user located in a mountainous area with weak network signal used a mobile app to ask the large model, "I haven't been feeling well lately, what tests should I get?" 1) The client detects an RTT of 900ms, a packet loss rate of 12%, and a bandwidth of 0.3Mbps, classifying it as a "poor" state; at this time: 2) The system automatically sets the context window to 1024 tokens and limits the generated token length to 128 tokens; 3) After the user enters a question, the client user interface immediately displays "Thinking...", the task scheduling module initiates an asynchronous request, and the timeout period is extended to 25 seconds; 4) The client only uploads the question text and context ID; the server reuses the historical context and returns a concise answer. 5) If the network is interrupted at this time, the system will return "The network is unstable. Your question has been saved and will be answered for you after the network is connected" and add the question to the synchronization queue. When the network is restored, the client will automatically retry and synchronize the unfinished interaction.

[0049] Example 2: Users using interactive question-and-answer applications in intermittent network environments such as subway elevators. Users interact with the model using tablets in subways or elevators, but the network connection is intermittent; at this time: 1) The client monitors the network status in real time and automatically reconnects and continues unfinished conversations when the network recovers; 2) By using a differential transmission mechanism, only the new question and context ID are sent, significantly reducing data traffic; 3) The compression module automatically selects Gzip Level 6 based on the current bandwidth to balance compression efficiency and processing speed; 4) Users are unaware of underlying network fluctuations throughout the process, resulting in a smooth and natural interactive experience.

[0050] In summary, this specific implementation method fully realizes the optimization method for large language model interactive services in weak network environments from five aspects: system architecture, network awareness, local capabilities, interactive fault tolerance, and bandwidth control. It ensures that stable, reliable, and responsive intelligent interactive services can still be provided under various network constraints.

[0051] Another embodiment of the present invention provides an apparatus for optimizing interactive services of large language models in weak network environments, such as... Figure 2 As shown, it includes: The layered service module is used to establish an interactive and adaptive layered service architecture for large language models, and dynamically adjust the context management and generation control of large language models according to network conditions. The semantic local caching module is used to establish a semantic local caching mechanism for interaction with large language models, and to store and respond to key data specific to large language models locally on the user device. The asynchronous fault-tolerant processing module is used to establish an asynchronous fault-tolerant processing mechanism for large language model interactions, providing a guarantee of interaction continuity in the event of network anomalies. The transmission optimization module is used to establish transmission optimization strategies for large language models and optimize the transmission characteristics of data.

[0052] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be referred to the corresponding process of the four key technologies in the aforementioned method embodiments, and will not be repeated here.

[0053] Another embodiment of the present invention provides a computer device (computer, server, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0054] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0055] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0056] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A method for optimizing interactive services of large language models in weak network environments, characterized in that, Includes the following steps: Establish a layered service architecture for interactive adaptation of large language models, and dynamically adjust the context management and generation control of large language models according to network conditions; Establish a semantic local caching mechanism for interaction with large language models, and perform local storage and offline response of key data unique to large language models on user devices; Establish an asynchronous fault-tolerant processing mechanism for large language model interactions to ensure interaction continuity in the event of network anomalies; Establish a transmission optimization strategy for large language models to optimize data transmission characteristics.

2. The method according to claim 1, characterized in that, The layered service architecture includes: When network conditions are good, enable the full context window to maintain the integrity of the dialogue history and the richness of the generated content; When network conditions deteriorate, the context window is automatically reduced and the generation length is limited to reduce data transmission and ensure the continuity of basic interactions. Real-time monitoring of network status enables flexible adaptation of large language model services to network conditions, allowing users to perceive only changes in response speed without experiencing functional interruptions.

3. The method according to claim 2, characterized in that, The layered service architecture includes: Network status classification includes: Ideal condition: RTT < 200ms, packet loss rate < 2%, bandwidth > 2Mbps; Under normal conditions: 200ms ≤ RTT < 800ms, packet loss rate 2%-10%, bandwidth 0.5-2Mbps; Poor condition: RTT ≥ 800ms, packet loss rate > 10%, bandwidth < 0.5Mbps; Based on different network status levels, the client and server make the following dynamic adjustments when processing and transmitting data: Good state: Enables full context window, allowing the generation of longer text, maintaining interactive coherence and information richness; Normal state: Reduce the context window to 2048 tokens, limit the length of a single generation to 256~512 tokens, and increase the length penalty parameter to reduce the amount of data requested per request; Poor state: Further reduce the context window to 1024 tokens, limit the generated length to 128~256 tokens, further increase the length penalty parameter, and disable unnecessary functions.

4. The method according to claim 1, characterized in that, The semantic local caching mechanism includes: Semantic compression is performed on historical dialogues to extract key information points and store them as structured knowledge fragments; Pre-generate standardized response content for common question-and-answer patterns in large language models; Context-differential storage is used to store historical information identifiers instead of the full text, and only incrementally changed content is transmitted; When the network is down, the local cache is used to perform basic semantic responses for the large language model, and the difference data is synchronized after the network is restored.

5. The method according to claim 1, characterized in that, The asynchronous fault-tolerant processing mechanism includes: After the user enters the input, a placeholder response is returned immediately, and the complete response is processed asynchronously in the background. The response waiting time of the large language model is dynamically adjusted based on network latency to avoid timeout interruptions. The interactive interface continuously displays the "thinking" status to maintain user confidence and continuity in the interaction.

6. The method according to claim 1, characterized in that, The transmission optimization strategy includes: Prioritize the transmission content of the large language model, prioritizing the transmission of core text content and delaying unnecessary multimedia elements; By reusing context identifiers, we can avoid repeatedly transmitting historical conversation content. Perform semantic compression transmission and dynamically adjust the compression strategy based on bandwidth.

7. The method according to claim 6, characterized in that, The priority management of the transmitted content of the large language model includes: High priority: User input, context ID, large language model response results; Medium priority: Dialogue support information, formatting tags; Low priority: multimedia content, style elements.

8. An apparatus for optimizing interactive services of large language models in weak network environments, characterized in that, include: The layered service module is used to establish an interactive and adaptive layered service architecture for large language models, and dynamically adjust the context management and generation control of large language models according to network conditions. The semantic local caching module is used to establish a semantic local caching mechanism for interaction with large language models, and to store and respond to key data specific to large language models locally on the user device. The asynchronous fault-tolerant processing module is used to establish an asynchronous fault-tolerant processing mechanism for large language model interactions, providing a guarantee of interaction continuity in the event of network anomalies. The transmission optimization module is used to establish transmission optimization strategies for large language models and optimize the transmission characteristics of data.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images