Edge-end collaborative reasoning method and system based on large model characteristics and dynamic network

Abstract

The invention discloses an edge-end collaborative reasoning method and system based on large model characteristics and a dynamic network, and belongs to the technical field of large model deployment reasoning, and the collaborative reasoning method comprises the steps: constructing a relation model between edge-end network resources and model interaction data and an edge-end collaborative triggering mechanism based on the network resources, judging whether to trigger edge-end collaboration or not through the model confidence coefficient and the network delay; designing a collaborative reasoning enhancement strategy based on speculation sampling in different network environments to optimize a big and small model collaborative reasoning process; by introducing a cooperative trigger function based on confidence and network delay, the system can perceive model output quality and communication environment change in real time, on-demand cooperation is realized, dual requirements of real-time performance and performance of tasks are met on the premise of ensuring precision, and the real-time performance of the system is improved. Computing resource occupation, reasoning delay and network transmission bottleneck of a large model in an edge-end heterogeneous environment are fully considered, and reasoning efficiency is improved through an efficient collaborative reasoning strategy.

Description

Technical Field

[0001] This invention belongs to the field of large model deployment inference technology, and specifically relates to an edge-to-edge collaborative inference method and system based on the characteristics of large models and dynamic networks. Background Technology

[0002] Large-scale model deployments in edge scenarios are characterized by resource constraints, hardware heterogeneity, and diverse requirements. In the edge paradigm, large-scale model inference deployments typically face complex heterogeneous environments, such as significant differences in device computing resources, diverse task requirements, complex and variable network environments, and inconsistent user-specific needs. These diverse requirements necessitate that large-scale model inference not only be sufficiently flexible to cope with different environments but also ensure deployment efficiency.

[0003] However, current solutions are mostly manually designed, with customized inference deployment schemes for each scenario, which is not only costly but also severely impacts deployment efficiency. Furthermore, the lightweight models used on the edge are often proprietary models, whose performance is often lower than that of pre-trained large models. Therefore, how to fully leverage personalized knowledge to enhance model performance and create personalized "assistants" that meet specific needs has become a pressing practical problem. Against this backdrop, there is an urgent need to design an efficient edge inference deployment method that not only meets real-time and diverse requirements but also achieves efficient inference and performance improvement of lightweight large models in complex heterogeneous scenarios through intelligent adaptation and personalized optimization.

[0004] Large-scale collaborative reasoning is a key research area for edge-based inference deployments of large models. It involves distributing inference tasks across multiple devices or nodes to fully utilize heterogeneous resources. Traditional collaborative reasoning methods are primarily designed for small- to medium-scale models and lack optimization support for the high computational complexity of large-scale inference stages, making them difficult to apply effectively in large-scale collaborative reasoning. To address the challenges of optimizing large-scale inference, researchers have proposed a speculative decoding method based on the characteristics of autoregressive inference in large models. This method reduces the computational overhead of large models by shrinking the search space. Speculative sampling techniques, developed on this basis, further improve the decoding speed and resource utilization efficiency of Large Language Models (LLMs). Although speculative sampling techniques perform well in single-device LLM acceleration and device-side inference, their application in edge-based collaborative reasoning is still in its early stages of exploration. Currently, edge-based collaborative reasoning still faces many challenges, requiring solutions to complex resource allocation, network fluctuations, and real-time performance issues. In the prior art, patent CN118839766A provides a method for collaborative inference of Transformer models for multiple edge devices. This patent divides the original dataset into multiple subsets with an equal number of label categories. Based on the computing performance and resources of the edge devices, it segments the original model into multiple sub-models that retain only the important structures. It uses a sub-task distillation method to help the sub-models learn the features of the original model, and uses a model ensemble method to aggregate all sub-models. The sub-models in the trained ensemble model are then deployed to the corresponding edge devices. When an edge device receives an inference task request, all edge devices execute the inference of the sub-models in parallel. The central node uses a feature fusion module to fuse all intermediate results to obtain the final inference result, thus achieving collaborative inference of models across multiple edge devices. However, it uses static model segmentation, pre-splitting the Transformer model into multiple sub-models and assigning them to different devices, and then fusing the results during inference. This can lead to inference delays or failure to complete inference when the network is poor or the device sub-models are unavailable. Patent CN119358687A discloses a method and apparatus for collaborative reasoning using multiple heterogeneous models. The method includes: receiving a reasoning request and a prompt word submitted by a client; selecting multiple matching heterogeneous models to simultaneously predict a token based on the context information and preset conditions of the reasoning request; selecting the best prediction result from the prediction results of all heterogeneous models; concatenating the best prediction result with the prompt word; if the concatenated result does not complete the reasoning, submitting the concatenated result as a new prompt word to multiple matching heterogeneous models, and continuing iterative reasoning until the reasoning is completed. However, this patent selects the best token based on the complete prediction results of multiple models, requiring all models to complete their complete predictions before comparison, which introduces significant computational overhead and communication burden in collaborative reasoning. Summary of the Invention

[0005] To address the issues of low deployment efficiency of lightweight large models in edge computing, real-time inference problems caused by network latency and bandwidth limitations, and the complexity of resource scheduling and task allocation in edge-end collaborative computing, this invention fully considers the autoregressive sampling method of large models during inference. The aim of this invention is to provide an edge-end collaborative inference method based on the characteristics of large models and dynamic networks. This method aims to achieve dynamic adaptation of edge resources and inference tasks. By evaluating edge network conditions and the inference performance of large models, and combining speculative sampling techniques, the optimal collaborative inference strategy is automatically selected to improve inference efficiency and real-time performance.

[0006] To achieve the above objectives, in a first aspect, the present invention provides an edge-to-edge collaborative reasoning method based on the characteristics of large models and dynamic networks, comprising the following steps: S1, construct the relationship model between edge network resources and model interaction data, and the edge-end collaborative triggering mechanism based on network resources. The lightweight end-side model at time step... When generating a token, its confidence level is calculated. and data interaction delay Determine whether collaborative reasoning has been triggered; S2, Design collaborative inference enhancement strategies based on speculative sampling under different network environments, specifically including: Step 1) Use the lightweight end-side model Perform autoregressive sampling and continuous inference to generate Tokens, until the inference confidence level Below This triggers collaborative reasoning; Step 2) Based on network conditions Triggering edge-end model collaboration or edge-side heterogeneous model collaboration will generate Each token is concatenated with a prefix and input into the corresponding collaborative model. Perform one forward computation; Step 3) Compare the lightweight end-side models and collaborative model If the logits value is found to be inconsistent with the expected token generation, a collaborative model is adopted. Correct the result and repeat step 1); Step 4) If the generated results meet the requirements, then use the collaborative model. Continue generating the next token, repeating step 1).

[0007] Furthermore, the relational model described in S1 uses a lightweight end-side model at time steps. Generate tokens The confidence level is The confidence level The calculation method is as follows:

[0008] in, It generates tokens. The corresponding logits value, For vocabulary size; when confidence level Below the threshold At that time, request collaborative reasoning from the edge models.

[0009] Furthermore, in S1, let... For the generated first The amount of data per token To generate tokens The subsequent network bandwidth, if interaction with the pre-generated... If there are one token, then the delay The calculation method is as follows:

[0010] When the delay Less than the real-time threshold At that time, collaborative reasoning is requested.

[0011] Furthermore, in S1, the edge-to-edge collaborative triggering mechanism is as follows: According to time step network environment and model confidence Construct a binary association model And formalized into a collaborative reasoning trigger function. ,as follows: in Represents end-to-end reasoning, Represents edge-to-edge collaborative reasoning, symbol The symbol represents the logical "OR" relationship. It indicates a logical AND relationship.

[0012] Furthermore, the edge-to-edge collaborative triggering mechanism in S2, which adopts a collaborative inference strategy based on speculative sampling, is as follows: According to time step network environment and model confidence Establish a collaborative reasoning trigger function as follows:

[0013] in, Represents end-to-end reasoning, Represents edge-to-edge collaborative reasoning. This represents collaborative reasoning within a heterogeneous model cluster on the endpoint.

[0014] Furthermore, if network resources are limited and heterogeneous models on the edge are used for collaboration, then each collaboration will select the result with the highest inference confidence in the heterogeneous model cluster.

[0015] Secondly, the present invention provides an edge-end collaborative reasoning system based on large model characteristics and dynamic networks, for implementing the above-mentioned edge-end collaborative reasoning method based on large model characteristics and dynamic networks. The system includes: an edge-side reasoning engine module, a collaborative decision control module, and an edge-end resource awareness and scheduling module. The edge inference engine module is deployed on the terminal device and runs a lightweight edge model. and heterogeneous model clusters It performs autoregressive inference and calculates the confidence level of each generated token in real time. Meanwhile, the generated token sequences are cached for collaborative use; The collaborative decision control module receives confidence signals from the edge-side inference engine module and network latency information from the resource-aware scheduling module, and then uses the extended collaborative inference trigger function. Dynamically determine whether to use independent inference on the edge, collaborative inference on the edge, or collaborative inference of heterogeneous model clusters on the edge, and execute the speculative sampling verification process after triggering collaboration; The edge resource awareness and scheduling module continuously monitors the network status between the terminal and the edge server, and obtains network bandwidth in real time. And combined with the current number of tokens to be transmitted and the encoding size of each token Dynamically calculate round-trip communication delay This provides real-time resource constraint basis for the collaborative decision-making and control module.

[0016] Thirdly, the present invention provides a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads part or all of the computer executable program from the memory and executes it, and the processor can realize the above-mentioned edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks when executing part or all of the computer executable program.

[0017] Simultaneously, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform the aforementioned edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks.

[0018] Alternatively, a computer-readable storage medium may be provided, in which a computer program is stored, which, when executed by a processor, enables the aforementioned edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects: First, the reasoning method described in this invention achieves dynamic intelligent decision-making of reasoning strategies by introducing a confidence-based approach. With network latency With its collaborative triggering function, the system can perceive changes in model output quality and communication environment in real time, enabling on-demand collaboration while balancing inference accuracy and real-time performance. Secondly, this invention effectively improves the efficiency and robustness of edge-side large model inference. It applies speculative sampling technology to collaborative inference under various network conditions and innovatively introduces edge-side heterogeneous small model clusters as backup collaborative units when edge-side large models are unavailable. This not only inherits the advantages of speculative sampling in reducing the number of autoregressive decodings and improving inference throughput, but also effectively addresses real-world challenges such as network fluctuations and device offline, enhancing the availability and fault tolerance of the system.

[0020] Finally, taking into account inference accuracy, real-time performance, and resource efficiency, this invention effectively alleviates the computation, storage, and bandwidth bottlenecks of edge devices by using mechanisms such as on-demand triggering of collaboration, minimizing communication data volume, and sampling and verification, while ensuring generation quality. This provides a feasible technical path for the efficient deployment of large models in resource-constrained scenarios such as embedded systems, mobile terminals, and Internet of Things (IoT) devices. Attached Figure Description

[0021] Figure 1 This is a flowchart of the edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks according to the present invention.

[0022] Figure 2 This is the general framework of the edge-to-edge collaborative reasoning method based on the characteristics of large models and dynamic networks in this invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1, such as Figure 1 As shown in the figure, an edge-to-edge collaborative inference method based on large model characteristics and dynamic networks provided by this invention includes the following steps: S1, construct the relationship model between edge network resources and model interaction data, and the edge-end collaborative triggering mechanism based on network resources.

[0025] In this embodiment, a joint relationship model between end-side inference confidence and network transmission latency is first constructed to quantify the coupling relationship between model prediction reliability and communication feasibility. This relationship model uses a lightweight end-side model at time steps. Token confidence level and transmission delay under current network conditions The input is the decision criterion for whether to trigger collaborative reasoning.

[0026] Specifically, during the inference process of the lightweight model on the edge, for the time step Generated tokens , Given the logits value corresponding to the token, its confidence level is... The calculation formula is as follows:

[0027] in, It is the size of the vocabulary. Below the set threshold This indicates that the current client-side model uses tokens. The predictions are unreliable and require collaborative reasoning.

[0028] At the same time, to determine whether the edge-to-edge coordination conditions are met, it is necessary to calculate the data interaction latency under the current network conditions. .set up For the generated first The data size of each token (in bits). To generate tokens The subsequent network bandwidth (bit / s), if interaction with the generated... One token, then delay . It needs to be less than the real-time threshold. Only then can collaborative reasoning be triggered.

[0029] Based on the above two dimensions, this invention constructs a binary association model. The binary joint relation model is used to describe the coupling relationship between model prediction reliability and communication timeliness, and is formalized as a collaborative inference trigger function. :

[0030] in, Represents end-to-end reasoning, Represents edge-to-edge collaborative reasoning, symbol The symbol represents the logical "OR" relationship. To express a logical "AND" relationship, only when the confidence level is... Below the set threshold And network latency Less than the real-time threshold Edge-side collaborative reasoning will only be triggered when the condition is met; otherwise, reasoning will be performed on the edge side. Figure 2 As shown.

[0031] S2. Design collaborative reasoning enhancement strategies based on speculative sampling under different network environments.

[0032] To optimize edge inference efficiency, this implementation further designs a collaborative inference enhancement strategy based on speculative sampling. This strategy is applicable to different scenarios with good or limited network resources, such as... Figure 2 As shown.

[0033] The speculative sampling collaborative inference process is as follows: Step 1), use the edge-side lightweight model Perform autoregressive sampling and continuous inference to generate A token, until a certain time step Inference confidence Below This triggers collaborative reasoning.

[0034] Step 2), based on network conditions Trigger edge-side model collaboration or edge-side heterogeneous model collaboration. If Then use the isomorphic large model deployed on the side. ;like Then call the heterogeneous model cluster on the end side. Perform collaborative reasoning.

[0035] Simultaneously, the end-side lightweight model will be implemented. generated Each token is concatenated with a prefix and input into the corresponding collaborative model. Perform a forward computation to obtain the reference logits value.

[0036] Step 3), compare the lightweight models on the end side. With collaborative model The logits output is then subjected to softmax to obtain the output probability distributions of the two models, respectively. and The maximum probability difference is defined as ,in and Lightweight models for the edge side and collaborative model The probability of each being most likely to generate a token. When Greater than the set threshold If the end-side model prediction is significantly biased, a larger model should be used. The result is that the output segment is corrected, and step 1 is repeated.

[0037] Step 4): If the generated results meet the requirements, then use the collaborative model. Continue generating the next token, repeating step 1).

[0038] If using a heterogeneous model cluster on the edge. When performing collaborative inference, the result with the highest inference confidence from the heterogeneous model cluster on the endpoint is selected in each collaboration to compensate for the shortcomings of the lightweight model on the endpoint. To address the shortcomings and ensure the maximization of reasoning quality.

[0039] Based on this, the collaborative reasoning trigger function is further expanded. as follows:

[0040] in, Represents collaborative reasoning among heterogeneous model clusters on the edge. Represents edge-to-edge collaborative reasoning. This represents collaborative reasoning within a heterogeneous model cluster on the endpoint.

[0041] Example 2: This invention constructs a lightweight collaborative reasoning framework for large models with limited resources. In this embodiment, to achieve efficient reasoning of large models under resource-constrained environments, a lightweight collaborative reasoning framework for large models is constructed. Based on the above concept, this invention provides an edge-side collaborative reasoning system based on the characteristics of large models and dynamic networks, including an edge-side reasoning engine module, a collaborative decision control module, and an edge-side resource awareness and scheduling module. (1) End-side inference engine module The edge inference engine module is deployed on the terminal device and runs a lightweight edge model. and heterogeneous model clusters It is responsible for performing autoregressive inference and calculating the confidence level of each generated token in real time. Simultaneously, the generated token sequences are cached for collaborative use. The edge-side inference engine module supports completing full inference locally and also supports passing context to the collaborative model when needed. Perform verification or relay generation.

[0042] (2) Collaborative Decision Control Module As the scheduling hub of the entire framework, the collaborative decision control module receives confidence signals from the edge inference engine module and network latency information from the resource-aware scheduling module, and then determines the appropriate response based on the extended collaborative inference trigger function. The system dynamically determines whether to use independent inference on the edge, collaborative inference on the edge, or collaborative inference on the edge heterogeneous model cluster, and executes the speculative sampling verification process after triggering collaboration.

[0043] (3) Edge resource awareness and scheduling module The edge resource awareness and scheduling module continuously monitors the network status between the terminal and the edge server, and obtains network bandwidth in real time. And combined with the current number of tokens to be transmitted and the encoding size of each token Dynamically calculate round-trip communication delay This provides real-time resource constraint basis for the collaborative decision-making and control module.

[0044] This system combines a dynamic adaptation mechanism based on network resources and inference task performance, and improves inference efficiency through efficient collaborative inference strategies to ensure that real-time inference requirements are met.

[0045] The system constructed in this application combines a dynamic adaptation mechanism based on network resources and inference task performance. It fully considers the computational resource consumption, inference latency, and network transmission bottlenecks of large models in heterogeneous edge environments. It improves inference efficiency through an efficient collaborative inference strategy, ensuring that real-time inference requirements are met. Under good network conditions, a lightweight edge model is used for autoregressive sampling, while the large edge model corrects the generated results. When edge network resources are limited or the large edge model is unavailable, speculative sampling collaboration between heterogeneous small edge models is allowed to compensate for the performance limitations of a single small model. In resource-constrained edge environments, this framework provides flexible adaptability, ensuring that inference tasks can meet both real-time and performance requirements while maintaining accuracy.

[0046] For example, the technical solution of the present invention can be further described as follows: The user enters the question on a smartphone (device-side): "Please briefly explain the collaboration between large and small models." The device-side inference engine module first uses a lightweight main model. Autoregressive generation is performed, and the confidence level of each generated token is calculated. The confidence level output by the model is calculated when generating tokens in a "cooperative" manner. Below the preset threshold This indicates that the token has low reliability.

[0047] Meanwhile, the edge resource awareness and scheduling module detected that the network bandwidth under the current resource environment is 100Mbps, the length of the context to be transmitted is 20 tokens, the average encoding size of each token is 4 bytes, and estimated the round-trip delay. Below the latency threshold .

[0048] The collaborative decision control module is based on the trigger function. The decision is made to initiate edge-to-edge collaborative inference. Subsequently, the edge sends the complete context, including user input and the generated token, to the edge server. The edge-to-edge large model performs speculative sampling based on the same context to generate the logits distribution of the next token.

[0049] The distribution of logits returned from the edge receiver at the device side, compared with the local lightweight model. Compare the logits distributions and calculate the maximum probability difference. The difference is greater than the set threshold. Therefore, the output of this segment is corrected using the generation result of the edge model.

[0050] Ultimately, the system outputs highly accurate answers in a short period of time.

[0051] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks as described in the present invention.

[0052] The present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and the processor can implement the edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks described in the present invention when executing the computer executable program.

[0053] The computer device may be a laptop, a desktop computer, or a workstation.

[0054] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf programmable gate array (FPGA).

[0055] The memory described in this invention can be an internal storage unit of a laptop, desktop computer, or workstation, such as memory or hard disk; or it can be an external storage unit, such as a portable hard disk or flash memory card.

[0056] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0057] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for edge-end collaborative reasoning based on large model characteristics and dynamic networks, characterized in that, Includes the following steps: S1, construct a relationship model between edge network resources and model interaction data and an edge collaborative triggering mechanism based on network resources, and the edge lightweight model is triggered at a time step When generating a token, calculate its confidence And data interaction delay Determine whether to trigger collaborative reasoning; S2, Design collaborative inference enhancement strategies based on speculative sampling under different network environments, specifically including: Step 1) using end-side lightweight model Performing autoregressive sampling, continuously infer generation Tokens until inference confidence Is lower than Triggering collaborative inference; Step 2) according to network conditions Triggering edge model collaboration or end-side heterogeneous model collaboration, the generated Token is spliced with the prefix, and the corresponding collaborative model Perform a forward calculation; Step 3) Compare end-side lightweight model and the logits value of the collaborative model If it is found that a certain token generation is not as expected, the result of the collaborative model is used to correct it, and step 1) is repeated. Step 4) If the generated result meets the requirements, use the collaborative model Continue generating the next token, repeat step 1).

2. The edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks according to claim 1, characterized in that, The relationship model in S1 is an end-side lightweight model at a time step Generating tokens The confidence of The confidence is calculated as follows: wherein, is a generated token corresponding logits value, is a vocabulary size; when the confidence is lower than a threshold , request edge model collaborative inference.

3. The edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks according to claim 1, characterized in that, In S1, set the data amount of the generated first token, the network bandwidth after generating the token , if interaction is needed for the generated token, the delay is calculated as follows: When the latency is less than a real-time threshold , request collaborative reasoning.

4. The edge-end collaborative inference method based on large model characteristics and dynamic network of claim 1, wherein, In S1, the edge-to-edge collaborative triggering mechanism is as follows: According to the time step of the network environment and the model confidence , a binary joint relationship model is constructed and formalized as a collaborative reasoning trigger function as follows: wherein represents end-side reasoning, represents edge-end collaborative reasoning, symbol represents a logical "or" relationship, symbol represents a logical "and" relationship.

5. The edge-end collaborative inference method based on large model characteristics and dynamic network of claim 1, wherein, The edge-to-edge collaborative triggering mechanism in S2, which adopts a collaborative inference strategy based on speculative sampling, is as follows: According to time step network environment and model confidence Establish a collaborative reasoning trigger function as follows: in, Represents end-to-end reasoning, Represents edge-to-edge collaborative reasoning. This represents collaborative reasoning within a heterogeneous model cluster on the endpoint.

6. The edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks according to claim 5, characterized in that, If, under conditions of limited network resources, heterogeneous model collaboration is used on the end side, then each collaboration selects the result with the highest inference confidence among the heterogeneous model cluster.

7. An edge-to-edge collaborative reasoning system based on large model characteristics and dynamic networks, characterized in that, To implement the edge-end collaborative reasoning method based on large model characteristics and dynamic networks as described in any one of claims 1-6, the system includes: an edge-side reasoning engine module, a collaborative decision control module, and an edge-end resource awareness and scheduling module. The edge inference engine module is deployed on the terminal device and runs a lightweight edge model. and heterogeneous model clusters It performs autoregressive inference and calculates the confidence level of each generated token in real time. Meanwhile, the generated token sequences are cached for collaborative use; The collaborative decision control module receives confidence signals from the edge-side inference engine module and network latency information from the resource-aware scheduling module, and then uses the extended collaborative inference trigger function. Dynamically determine whether to use independent inference on the edge, collaborative inference on the edge, or collaborative inference of heterogeneous model clusters on the edge, and execute the speculative sampling verification process after triggering collaboration; The edge resource awareness and scheduling module continuously monitors the network status between the terminal and the edge server, and obtains network bandwidth in real time. And combined with the current number of tokens to be transmitted and the encoding size of each token Dynamically calculate round-trip communication delay This provides real-time resource constraint basis for the collaborative decision-making and control module.

8. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer-executable program, the processor reading part or all of the computer-executable program from the memory and executing it, and when the processor executes part or all of the computer-executable program, it can implement the edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks as described in any one of claims 1-6.

9. A computer program product containing instructions, characterized in that, When the instructions are executed on a computer, the computer causes the computer to perform claim 1. Any one of the six edge-to-edge collaborative reasoning methods based on the characteristics of large models and dynamic networks.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the edge-to-edge collaborative reasoning method based on large model characteristics and dynamic networks as described in any one of claims 1-6.

Images