Cloud-edge collaborative big language model reasoning framework optimization method based on historical information matching

Abstract

A cloud-edge collaborative big language model reasoning framework optimization method based on historical information matching comprises the following steps: firstly, transmitting a big language model reasoning request to an access point deployed with a vector database, and accessing an edge end cloud collaborative reasoning system; the access point obtains request related information and system state information; determining an optimal reasoning decision variable by combining historical dialogue data in the vector database according to the semantic matching degree and utilizing a neighbor strategy optimization algorithm fused with trust domain pruning; performing dynamic scheduling and processing on the reasoning task through an edge end cloud collaborative reasoning framework; the access point, the cloud server and the edge server start cooperative reasoning task execution, each node continuously interacts, the cloud server is responsible for high-complexity semantic abstract calculation, the edge server undertakes parallel detail supplementary reasoning, the access point is responsible for information aggregation and result feedback, and meanwhile, current dialogue data is stored in a vector database; in the reasoning process, low delay and high accuracy are guaranteed according to a utility function optimization target.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of artificial intelligence, edge computing, and natural language processing, specifically to a cloud-edge collaborative large language model (LLM) inference optimization method based on historical information matching, which is suitable for efficient inference tasks in real-time interactive scenarios such as multi-turn continuous dialogue. Background Technology

[0002] In edge computing environments, cloud-edge collaborative reasoning provides an effective paradigm for deploying large-scale language models. This paradigm aims to mitigate the high latency, high energy consumption, and privacy risks associated with centralized cloud-based reasoning by rationally distributing computational tasks across cloud and edge devices, thereby supporting intelligent applications with low latency and high privacy requirements. However, existing collaborative reasoning frameworks still face significant challenges in complex interactive scenarios such as multi-turn continuous dialogues.

[0003] In these scenarios, there is a strong semantic connection and referential dependency between user queries and historical context. Taking intelligent customer service or personalized assistants as examples, the accurate understanding and answering of the current question often depends on a coherent grasp of the intent, entities, and topical context of previous rounds of dialogue. This complex contextual dependency significantly increases the difficulty of task scheduling and maintaining semantic continuity. Existing cloud-edge collaborative reasoning frameworks have significant shortcomings in this regard: on the one hand, they fail to fully consider the continuous modeling of historical context, which can easily lead to semantic gaps and error accumulation in dialogue, impairing reasoning accuracy; on the other hand, the historical information retrieval mechanisms used are often rather crude, and due to semantic ambiguity, they are unable to accurately filter out high-value information from lengthy dialogue histories, instead introducing noise. In addition, the inherent randomness of the large language model generation process also makes it difficult to stably train collaborative strategies, resulting in slow convergence. While pursuing efficiency, existing solutions also lack effective privacy protection design for dialogue history cached on the edge side, posing a risk of sensitive information leakage.

[0004] Therefore, existing frameworks fall short in ensuring context continuity, efficiently utilizing historical information, training stability, and balancing privacy and efficiency when dealing with the complex historical dependencies between tasks in multi-turn dialogue scenarios. Deeply integrating the contextual characteristics of multi-turn dialogues to design a collaborative reasoning mechanism that can accurately match historical information, provide stable training, and protect privacy has become a key breakthrough for improving the overall performance and practicality of cloud-edge collaborative LLM inference systems. Summary of the Invention

[0005] To overcome the shortcomings of existing cloud-edge collaborative large language model inference frameworks in multi-turn dialogue scenarios, such as insufficient context continuity, inefficient use of historical information, poor training stability, and difficulty in balancing privacy protection and efficiency, this invention provides a cloud-edge collaborative large language model inference optimization method based on historical information matching. This invention aims to reduce inference latency, improve response accuracy, accelerate policy training convergence, and reduce the risk of privacy leakage by constructing an efficient historical information matching and collaborative inference mechanism.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: An optimization method for an edge-cloud collaborative large language model inference framework based on historical context matching includes: When a user initiates a large language model inference request, the request is first transmitted to an access point that has deployed a vector database. The access point will automatically connect to the edge cloud collaborative inference system. The vector database is used to cache the multi-turn dialogue history information between the user and the model. After receiving a user's inference request, the access point obtains request-related information and system status information. The request-related information includes user identifier, query content, query length, maximum latency constraint and minimum accuracy requirement, and the system status information includes access point computing resources, edge server computing capabilities and communication network parameters. Based on the request-related information and system status information, combined with historical dialogue data in the vector database, and according to the semantic matching degree, the optimal inference decision variables are determined by using the nearest neighbor strategy optimization algorithm with fusion trust domain pruning. The decision variables include the size of the historical information window, the selection of the inference mode, and the edge server allocation scheme. Based on the aforementioned decision variables, the reasoning task is dynamically scheduled and processed through an edge-cloud collaborative reasoning framework.

[0007] Access points, cloud servers, and edge servers initiate collaborative inference tasks. During the inference process, the nodes continuously interact. The cloud server is responsible for high-complexity semantic abstraction calculations, the edge server undertakes parallel detailed supplementary inference, and the access point is responsible for information aggregation and result feedback. At the same time, the current dialogue data is stored in the vector database. The entire inference process optimizes the objective based on the utility function to ensure low latency and high accuracy.

[0008] Furthermore, based on request-related information, system state information, and historical dialogue data in the vector database, the optimal inference decision variables are determined using a nearest neighbor policy optimization algorithm that incorporates trust domain pruning, including: Based on the current query, the historical dialogue vectors are sorted by relevance. The effective value of each candidate historical dialogue is quantified by using a lightweight attention fine-tuning module in the Transformer encoder. A serial decision-making mechanism is adopted to sequentially determine the size of the historical information window, the inference mode, and the allocation of edge servers.

[0009] Trust domain constraints and caching mechanisms are introduced to suppress the randomness of reasoning in large language models; Optimize decision-making strategies through an actor-critic network architecture to balance task completion latency, accuracy, and system energy consumption.

[0010] Furthermore, the utility function is constructed as follows: With user satisfaction and system energy consumption as optimization objectives, user satisfaction models and system energy consumption models are established respectively. User satisfaction is calculated based on task completion delay and inference accuracy. The delay must meet the user's maximum delay constraint, and the accuracy must meet the user's minimum accuracy requirement. System energy consumption covers the energy consumption of historical information matching and fine-tuning at the access point, the transmission energy consumption and computing energy consumption between the cloud and the edge, and is calculated based on the power parameters, transmission power and computing resource consumption of each node. Based on the user satisfaction model and the system energy consumption model, a weighting factor is introduced to balance their influence, and a final utility function is constructed. This function is the difference between the user satisfaction reward value and the system energy consumption cost, which is used to quantify the comprehensive optimization effect of the reasoning task.

[0011] Furthermore, the multi-objective optimization model is expressed as:

[0012] In the formula, System utility function.

[0013] Furthermore, the utility function model is expressed as:

[0014] in Indicates user satisfaction. Indicates system energy consumption. Energy consumption weighting coefficient;

[0015] In the formula, This serves as the baseline return constant, used to ensure that the independent variable of the logarithmic function is positive; This indicates the maximum latency limit for the task. The total latency cost for completing task inference; Indicates the accuracy of the reasoning response in a reasoning task. This is the minimum accuracy requirement for this task; and This is the weighting factor.

[0016]

[0017] In the formula, This indicates the latency of the preceding vector retrieval. For the time delay of the fine-tuning part of the attention mechanism, It refers to the transmission latency of the inference task from the access point to the cloud. It's the inference latency of large cloud-based models. It is a Sketch collection The total latency from transmission to the completion of inference. This refers to the transmission latency of the inference task from the access point to the edge server in the standalone inference mode of the edge model. It is a Sketch collection Total inference latency on edge servers.

[0018]

[0019] In the formula, Energy consumption for fine-tuning the attention mechanism. It refers to the energy consumption of inference tasks from the access point to the cloud. It is the inference energy consumption of large cloud models. It is a Sketch collection Energy consumption for transmission from the cloud to the edge server It is a Sketch collection Inference energy consumption on edge servers This refers to the energy consumption of the inference task from the access point to the edge server in the standalone inference mode of the edge model. It represents the inference energy consumption of inference tasks on edge servers.

[0020] Furthermore, the optimal inference decision variables are determined using a nearest neighbor policy optimization algorithm that incorporates trust domain pruning, including: The nearest neighbor policy optimization algorithm based on fusion trust domain pruning solves the reasoning decision optimization problem. It generates multiple candidate solutions in the solution space through a serial decision mechanism; each candidate solution represents a configuration of decision variables.

[0021] By evaluating the task completion delay, inference accuracy, and system energy consumption corresponding to each candidate solution through a utility function, the multi-dimensional performance indicators are mapped to a single quantitative value to reflect the comprehensive optimization effect, and the candidate solution with the best comprehensive performance is selected as the final decision variable.

[0022] Furthermore, the edge-cloud collaborative reasoning framework includes an access point, a cloud server, and an edge server. Based on the decision variables, the framework dynamically schedules and processes reasoning tasks, including: The task processing path is determined based on the semantic matching degree and the decision result of the reasoning pattern.

[0023] In the cloud-edge collaborative reasoning path, the enhanced query is uploaded to the cloud server, where the large language model performs highly complex semantic abstraction calculations to generate a coarse-grained summary. Based on the resource status and task allocation scheme of the edge servers, the coarse-grained summary is distributed to the corresponding edge servers. Each edge server's smaller language model executes inference in parallel, supplementing detailed information to refine the answer.

[0024] In the local inference path at the edge, the access point directly sends the enhanced query to the selected edge server, and the edge small language model independently completes the complete inference based on the historical context enhancement information to generate the final answer.

[0025] After inference is completed, the edge server feeds back the results to the access point. The access point aggregates and integrates the inference results from multiple edge nodes to form a unified and coherent final response, which is then returned to the user.

[0026] The core of this invention lies in constructing a three-tiered collaborative architecture consisting of access nodes, a cloud-based large language model, and a lightweight edge model. This method enhances the semantic continuity of the query context through a historical information matching mechanism, integrating vector database retrieval and attention fine-tuning. It employs a precision-aware dynamic scheduling strategy to adaptively select the optimal inference path and introduces trust-domain-based policy pruning and near-end policy optimization algorithms to suppress randomness during inference and improve training stability and convergence speed. In specific implementation, historical dialogue context is first retrieved and fine-tuned using a vector database. Then, the access node, based on the scheduling strategy, decides to offload the computational task to the edge model for independent inference or initiate a cloud-edge collaborative inference process. The cloud is responsible for generating the response summary, while the edge model performs parallel refinement generation. This invention effectively reduces inference latency and improves task accuracy, making it suitable for scenarios with high real-time requirements, such as intelligent dialogue and interactive applications.

[0027] The beneficial effects of this invention are: This invention employs a historical context matching mechanism that integrates vector database retrieval and attention fine-tuning to accurately filter high-value information and enhance the semantic coherence of multi-turn dialogues. Combined with a near-end policy optimization algorithm using trust domain policy pruning, it suppresses the randomness of inference in large language models, improving training stability and convergence speed. Through precision-aware dynamic scheduling, it adaptively selects either edge-independent inference or cloud-edge collaborative paths, generating a core summary in the cloud and supplementing details in parallel at the edge, significantly reducing inference latency and improving task accuracy. A user-level access control mechanism at the access point matches only the current user's historical context, ensuring data privacy and security. This invention is suitable for scenarios with high requirements for real-time performance and interactive coherence, such as multi-turn intelligent dialogue and interactive question answering. Attached Figure Description

[0028] Figure 1 This is a system diagram of a cloud-edge collaborative large language model inference framework with historical context information matching.

[0029] Figure 2 This is a flowchart of the PPO-HCM-CELLM algorithm.

[0030] Figure 3 Flowchart for serial decision-making in the PPO-HCM-CELLM algorithm

[0031] Figure 4 Flowchart of the PPO-HCM-CELLM algorithm caching mechanism Detailed Implementation

[0032] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0033] An optimization method for an edge-cloud collaborative large language model inference framework based on historical context matching includes: Step 1: The system consists of an access point, a cloud-based LLM, and multiple edge lightweight models (SLMs). The access point deploys a vector database (such as Chroma) to store the embedding vectors of users' historical dialogues and is equipped with a lightweight Transformer encoder. Perform real-time text-to-vector conversion. The cloud server runs a large number of parameterized LLMs, with a computational capacity of... Edge deployment One SLM node, one edge server node Its computing power is The communication bandwidth between the access point and the cloud, and between each edge node, are respectively... and .

[0034] Step 2: Design a historical information matching mechanism. (In time slots) ,user Reasoning task Defined as a quintuple , representing the user ID, query conditions, query length, latency constraints, and accuracy requirements, respectively. The inference task then arrives at the access point.

[0035] Step 3: Pre-vector retrieval. First, the encoder is used to retrieve the current query vector. Convert to query vector Retrieve the user's historical dialogue vector set from the vector database. ,in For each historical dialogue round, a comprehensive score combining semantic similarity and temporal proximity is used to calculate the relevance of each historical record. in, This is the current total number of dialogue rounds. The weighting factor is used to balance semantic similarity and temporal proximity. The attenuation coefficient is used to control the attenuation rate of the influence from previous rounds. The rounds with the highest overall scores are selected. The items are used as candidate sets, and Dynamic constraints based on the currently available computing resources at the access point: in Computing resources for access points, The basic computational cost for the current query. This represents the average processing cost for a single historical record.

[0036] Step 4: Attention fine-tuning. Concatenate candidate historical texts with the current query to form a sequence. The input is processed by a lightweight Transformer encoder for value weight re-evaluation. Each candidate historical information is computed using a scaled dot product attention function. weight The scaling dot product attention formula is: , Hidden dimensions for the self-attention layer of a lightweight Transformer encoder, and filtering out those below a threshold. From low-value information, obtain an effective set of historical context. Finally, the historical information matching degree of this query is calculated: By integrating effective historical context with the current query, an enhanced query can be formed. Its length is .

[0037] Step 5: Access point based on matching degree Task Requirements And the real-time status of the system, through binary decision variables Dynamically select the reasoning path.

[0038] Step 6: Perform cloud-edge collaborative inference. Cloud-based inference: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Upload to the cloud-based LLM library to generate a semantic summary Sketch collection. The transmission delay is ,in , For uplink transmission power, For channel gain, The power is Gaussian white noise. The cloud inference latency model is... , ,in This indicates the number of layers in a large language model. The dimension of the hidden layer of the large language model in the cloud. For the attention head dimension of the model, It represents the number of attention heads in the model. It is the intermediate dimension of the model's feedforward network. , and for Constants related to the structural parameters of large language models. The corresponding energy consumption is expressed as... , This represents the average power consumption parameter of the large cloud model per unit time.

[0039] Edge parallel inference: The cloud will be the first One Sketch, length is Unload to edge server This decision is determined by variables. It indicates, and regarding Sketch satisfy Each edge node performs refined inference in parallel. (Sketch) Transmitted to the corresponding server The transmission delay is On the edge server The inference delay on can be expressed as ,in Indicates deployment on edge servers The number of layers in the small language model. Represents edge server computing power , and for Constants related to the structural parameters of each language model. (Sketches collection) The total latency from transmission to inference is The total energy consumption from transmission to inference completion is

[0040] Step 7: Perform edge-specific inference: Enhance the query Treat it as a single Sketch and unload it to the edge server The transmission delay is The transmission rate Determined by bandwidth and channel conditions, it is expressed as The inference latency for edge SLM is: The total energy consumption from transmission to inference completion is

[0041] Step 8: Establish an optimization problem model and solve it using a reinforcement learning algorithm. The historical information matching, scheduling decision-making, and resource allocation processes described above are modeled as an optimization problem aiming to maximize the overall system benefit. This problem is formulated as a Markov decision process, where the state space includes user query features, historical matching degree, and available computing resources, and the action space covers the number of historical information retrievals, inference path selection, and task unloading decisions. A proximal policy optimization algorithm based on trust domain pruning is used to train this decision process. By constraining the policy update step size and reusing similar decision experiences, training fluctuations caused by the randomness of large language model generation are suppressed, accelerating the stable convergence of the policy network. Specifically, the historical information window size... Reasoning-based decision making Edge server selection decision To optimize variables and maximize the utility function of all inference tasks in the system through reasonable scheduling, the mathematical model is established as follows:

[0042] Step 9: Define constraints. Constraint (1a) Each inference task can only choose one inference method; (1b) The unloading decision for each Sketch is a binary variable; Constraint (1c) Ensures that each Sketch must be assigned to an edge server; Constraint (1d) The total computational load of Sketches carried by the edge server must not exceed its computational resource capacity; (1e) The selected historical information window size cannot exceed It cannot be a non-positive number.

[0043] Step 10: Solve the scheduling strategy using the PPO-HCM-CELLM optimization algorithm. The long-term optimization problem is modeled as a Markov Decision Process (MDP) and solved. MDP Modeling: State ,in This represents the computing resources of each edge server at the current moment. (Time slot) action The reward function is designed to maximize overall utility. in , This serves as the baseline return constant, used to ensure that the independent variable of the logarithmic function is positive; Indicates the accuracy of the reasoning response in a reasoning task; and As a weighting factor; The total time delay for completing the reasoning task is represented by the following formula: The total energy consumption during the execution of the inference task is modeled as follows: , It is a weighting factor, mainly used to limit the order of magnitude of energy consumption.

[0044] Step 11: Select the PPO-HCM-CELLM algorithm, which is optimized based on the PPO algorithm, for solving the MDP problem. Introduce a caching mechanism to store effective training samples to reduce training interference caused by the randomness of LLM inference. This is used to make optimal decisions on the output historical information window size, inference mode, and Sketch edge allocation. The algorithm network is built using an actor-critic architecture, and the network parameters are updated through a policy pruning mechanism.

[0045] Step 12: Receive parameters as the access point of the actor. Then, determine the action. Vector databases are based on and Sort the user's historical information. The vector database receives... Before selection The historical information is entered into the attention fine-tuning module, and finally the matching degree is obtained. The actor is based on the received data. Determine the action Then perform the action. Reasoning task Uninstall it to the corresponding server. If... Enhance information Considered a sketch, its length regarded as The actor then bases its work on the received data. Status information Determine the action Finally, execute Each Sketch instance is unloaded to its corresponding edge server for inference.

[0046] Step 13: The actor receives... And determine the action Afterwards, they will be paired with the same reasoning task from the previous training session. and To make a comparison, if the time between the two rounds of training The difference is less than 10%, and If the data is the same, it can be considered a cache hit, and the data from the previous training round of this inference task can be directly read. Similarly, when the actor determines the action... Then, compare the corresponding parameters of the previous training round of the same inference task. If the cache is hit, directly obtain the actual inference response, inference latency, energy consumption and other data of each Sketch in the previous training round from the cache.

[0047] Step 14: Calculate the reward based on information such as latency and energy consumption of the task reasoning. and feedback and the next state At the same time, the sequence Experience samples are stored in the experience pool. During the network optimization phase, the objective function is used to update the Actor network and the Critic network.

[0048] Step 15: The trained PPO-HCM-CELLM algorithm provides the optimal scheduling decision.

[0049] This embodiment addresses cloud-edge collaborative inference scenarios involving multi-user, multi-turn continuous dialogue. It not only considers structural optimization of Large Language Model (LLM) inference tasks in terms of historical context matching and cloud-edge resource collaboration, but also proposes a reinforcement learning scheduling algorithm based on policy pruning to rationally optimize the scheduling of inference tasks with latency constraints and accuracy requirements. This invention assists access points in formulating efficient inference path selection strategies to minimize task response latency, reduce system energy consumption, and simultaneously improve inference accuracy and task completion rate.

[0050] For the scenario shown in Figure 1, where multiple user devices collaborate with cloud-based LLM servers and edge lightweight language model (SLMs) servers through an access point to process inference tasks, the specific implementation plan is as follows: First, the system comprises multiple user devices, an access point equipped with a Chroma vector database, a cloud server deploying a high-performance LLM (such as DeepSeek V3.1), and several edge servers deploying SLMs (such as the Qwen3 series). The access point integrates a lightweight Transformer encoder and an attention fine-tuning module, responsible for historical context storage and matching; the cloud-based LLM handles complex semantic reasoning and sketch generation, while the edge SLMs supplement the reasoning details. Within a time slot, user inference tasks randomly arrive at the access point, and each task includes key information such as user identifier, query content, length, maximum latency constraints, and minimum accuracy requirements.

[0051] Secondly, after the user's inference task arrives, the historical information matching process is initiated: the access point maps the current query to a semantic vector through the encoder, and uses cosine similarity combined with a time decay mechanism to retrieve the Top-K historical dialogues of the same user from the vector database; the attention fine-tuning module calculates the effective value weight of historical information, filters low-value redundant content, generates enhanced queries, and quantifies the matching degree. The decision-making inference path is based on the matching degree: if the matching degree meets the standard, the enhanced query is directly unloaded to the edge server, and the SLM completes the inference independently; if the matching degree does not meet the standard, it is first uploaded to the cloud LLM to generate a semantic sketch set, and then each sketch is unloaded to the edge server in parallel for refinement. Finally, the access point summarizes the results and returns them to the user.

[0052] Furthermore, an optimization problem is established with constraints such as access point computing resources and edge server computing capacity, and with historical information window size, inference mode decision, and edge server selection decision as optimization variables, aiming to maximize the overall system utility. User satisfaction is quantified by task completion latency and inference accuracy, and system energy consumption covers the energy consumption of the entire process of matching calculation, inference, and transmission.

[0053] Then, this optimization problem, being equivalent to the multiprocessor task scheduling problem, was proven to be NP-hard. Considering the memorylessness of LLM request arrivals and system state updates, it was modeled as a Markov decision process: the state space contains user task information and edge server resource states; the action space is the set of optimization variables mentioned above; and the reward function is the immediate system utility, taking into account both service quality and energy consumption costs.

[0054] Finally, regarding such Figure 2 3 and 4 propose a PPO-HCM-CELLM solution strategy based on the proximal policy optimization algorithm: a serial decision-making mechanism is designed to handle the temporal dependencies between variables, and the rationality is ensured through the "decision-execution-observation-re-decision" process; a caching mechanism based on trust regions is introduced to compare the matching degree difference between the current and historical training and the inference pattern, and historical data is reused when the cache is hit, thus suppressing the randomness of LLM generation; an actor-critic architecture is adopted, in which the actor network generates scheduling decisions, the critic network evaluates the state value through generalized advantage estimation, and the parameters are updated by combining the policy pruning mechanism to improve training stability and sample efficiency.

[0055] This invention addresses cloud-edge collaboration scenarios involving multi-user, multi-turn dialogues. By using historical context matching and cloud-edge resource collaboration optimization, it assists access points in developing efficient scheduling strategies for inference tasks with real-time and accuracy requirements. This reduces latency and energy consumption while improving inference accuracy and task completion rate.

[0056] The embodiments described in this specification are merely illustrative examples of implementations of the inventive concept. The scope of protection of this invention should not be limited to the specific forms presented in these embodiments, but also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. An optimization method for an edge-cloud collaborative large language model inference framework based on historical information matching, characterized in that: The method includes: When a user initiates a large language model inference request, the request is first transmitted to an access point that has deployed a vector database. The access point will automatically connect to the edge cloud collaborative inference system. The vector database is used to cache the multi-turn dialogue history information between the user and the model. After receiving a user's inference request, the access point obtains request-related information and system status information. The request-related information includes user identifier, query content, query length, maximum latency constraint and minimum accuracy requirement, and the system status information includes access point computing resources, edge server computing capabilities and communication network parameters. Based on the request-related information and system status information, combined with historical dialogue data in the vector database, and according to the semantic matching degree, the optimal inference decision variables are determined by using the nearest neighbor strategy optimization algorithm with fusion trust domain pruning. The decision variables include the size of the historical information window, the selection of the inference mode, and the edge server allocation scheme. Based on the aforementioned decision variables, the reasoning task is dynamically scheduled and processed through an edge-cloud collaborative reasoning framework; Access points, cloud servers, and edge servers initiate collaborative inference tasks. During the inference process, the nodes continuously interact. The cloud server is responsible for high-complexity semantic abstraction calculations, the edge server undertakes parallel detailed supplementary inference, and the access point is responsible for information aggregation and result feedback. At the same time, the current dialogue data is stored in the vector database. The entire inference process optimizes the objective based on the utility function to ensure low latency and high accuracy.

2. The method as described in claim 1, characterized in that, Based on request-related information, system status information, and historical dialogue data in the vector database, the optimal inference decision variables are determined using a nearest neighbor policy optimization algorithm that incorporates trust domain pruning, including: Based on the current query, the historical dialogue vectors are sorted by relevance. The effective value of each candidate historical dialogue is quantified by using a lightweight attention fine-tuning module in the Transformer encoder. A serial decision-making mechanism is adopted to sequentially determine the size of the historical information window, the inference mode, and the allocation of edge servers; Trust domain constraints and caching mechanisms are introduced to suppress the randomness of reasoning in large language models; Optimize decision-making strategies through an actor-critic network architecture to balance task completion latency, accuracy, and system energy consumption.

3. The method as described in claim 1, characterized in that, The utility function is constructed as follows: With user satisfaction and system energy consumption as optimization objectives, user satisfaction models and system energy consumption models are established respectively. User satisfaction is calculated based on task completion delay and inference accuracy. The delay must meet the user's maximum delay constraint, and the accuracy must meet the user's minimum accuracy requirement. System energy consumption covers the energy consumption of historical information matching and fine-tuning at the access point, the transmission energy consumption and computing energy consumption between the cloud and the edge, and is calculated based on the power parameters, transmission power and computing resource consumption of each node. Based on the user satisfaction model and the system energy consumption model, a weighting factor is introduced to balance their influence, and a final utility function is constructed. This function is the difference between the user satisfaction reward value and the system energy consumption cost, which is used to quantify the comprehensive optimization effect of the reasoning task.

4. The method as described in claim 3, characterized in that, The multi-objective optimization model is expressed as: In the formula, System utility function.

5. The method as described in claim 4, characterized in that, The utility function model is expressed as: in Indicates user satisfaction. Indicates system energy consumption. Energy consumption weighting coefficient; In the formula, This serves as the baseline return constant, used to ensure that the independent variable of the logarithmic function is positive; This indicates the maximum latency limit for the task. The total latency cost for completing task inference; Indicates the accuracy of the reasoning response in a reasoning task. This is the minimum accuracy requirement for this task; and As a weighting factor; In the formula, This indicates the latency of the preceding vector retrieval. For the time delay of the attention mechanism fine-tuning part, It refers to the transmission latency of the inference task from the access point to the cloud. It's the inference latency of large cloud-based models. It is a Sketch collection The total latency from transmission to the completion of inference. This refers to the transmission latency of the inference task from the access point to the edge server in the standalone inference mode of the edge model. It is a Sketch collection Total inference latency on edge servers; In the formula, Energy consumption for fine-tuning the attention mechanism. It refers to the energy consumption of inference tasks from the access point to the cloud. It is the inference energy consumption of large cloud models. It is a Sketch collection Energy consumption for transmission from the cloud to the edge server It is a Sketch collection Inference energy consumption on edge servers This refers to the energy consumption of the inference task from the access point to the edge server in the standalone inference mode of the edge model. It represents the inference energy consumption of inference tasks on edge servers.

6. The method as described in claim 5, characterized in that, The optimal inference decision variables are determined using a nearest neighbor policy optimization algorithm that incorporates trust domain pruning, including: The nearest neighbor policy optimization algorithm based on fusion trust domain pruning solves the reasoning decision optimization problem. It generates multiple candidate solutions in the solution space through a serial decision mechanism; each candidate solution represents a configuration of decision variables. By evaluating the task completion delay, inference accuracy, and system energy consumption corresponding to each candidate solution through a utility function, the multi-dimensional performance indicators are mapped to a single quantitative value to reflect the comprehensive optimization effect, and the candidate solution with the best comprehensive performance is selected as the final decision variable.

7. The method as described in claim 1, characterized in that, The edge-cloud collaborative inference framework includes an access point, a cloud server, and an edge server. Based on the decision variables, the framework dynamically schedules and processes inference tasks, including: The task processing path is determined based on the semantic matching degree and the decision-making results of the reasoning pattern; In the cloud-edge collaborative reasoning path, the enhanced query is uploaded to the cloud server, where the large language model performs highly complex semantic abstraction calculations to generate a coarse-grained summary. Based on the resource status and task allocation scheme of the edge servers, the coarse-grained summary is distributed to the corresponding edge servers, where smaller language models on each edge server execute reasoning in parallel, supplementing detailed information to refine the answer. In the local inference path at the edge, the access point directly sends the enhanced query to the selected edge server, and the edge small language model independently completes the complete inference based on the historical context enhancement information to generate the final answer. After inference is completed, the edge server feeds back the results to the access point. The access point aggregates and integrates the inference results from multiple edge nodes to form a unified and coherent final response, which is then returned to the user.

Images