A collaborative reasoning method and system for deterministic delay in edge computing scenarios

By dividing candidate models in edge computing scenarios and monitoring network bandwidth in real time, the model configuration is dynamically adjusted to solve the problems of delay and accuracy uncertainty caused by network fluctuations in edge-to-end collaborative reasoning, and achieve deterministic delay and high-precision collaborative reasoning.

CN118869510BActive Publication Date: 2025-09-19SOUTHEAST UNIV +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410878822.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-02
Publication Date
2025-09-19
Estimated Expiration
2044-07-02

AI Technical Summary

Technical Problem

Existing technologies cannot effectively cope with network bandwidth fluctuations in edge-to-edge collaborative inference, resulting in uncertainty in inference accuracy and latency. In particular, when network conditions are poor, the inference accuracy is significantly lost and deterministic latency cannot be achieved.

Method used

By obtaining edge-device collaborative inference scenario information, dividing candidate models and adding collaborative inference schedulers and network bandwidth monitoring points on edge devices, we can schedule and monitor network bandwidth in real time, select the optimal robustness model for deployment and transmission, dynamically adjust model configuration based on user needs, and optimize the neural network model structure to enhance the ability to resist data loss.

Benefits of technology

It achieves the determinism of inference latency in the case of network bandwidth fluctuations, while reducing accuracy loss, improving the robustness and adaptability of the model, and is suitable for applications such as edge intelligence and real-time audio and video.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118869510B_ABST
    Figure CN118869510B_ABST
Patent Text Reader

Abstract

The present invention discloses a collaborative reasoning method and system for deterministic delay in edge computing scenarios. The method obtains end-edge collaborative reasoning scenario information, determines the user's demand for reasoning delay and tolerance for loss of reasoning accuracy; prepares candidate models, divides the target model into shallow models and deep models; adds a collaborative reasoning scheduler and a network bandwidth monitoring point to the edge device; executes end-edge collaborative reasoning, deploys the initial model, the end device executes the shallow model, the edge device executes the deep model, and the execution result of the end device is transmitted to the edge device through the network based on a progressive transmission mechanism; the scheduler determines the delay for network transmission based on the user's demand for reasoning delay and the computing delay of the end and edge devices, and adjusts the reasoning configuration based on the user's tolerance for loss of reasoning accuracy. This method can optimize the training of neural network models and select the optimal robustness model for reasoning based on real-time network bandwidth fluctuations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of edge computing collaborative reasoning and neural network optimization, and mainly relates to a collaborative reasoning method and system with deterministic delay in edge computing scenarios. Background Art

[0002] With the continuous evolution of the Internet of Everything (IoE), the computing power and energy efficiency of IoT devices have significantly improved. Edge-device collaborative reasoning has become a promising solution for IoT devices to support mobile AI applications. Edge-device collaborative reasoning is an architecture in which end devices and edge servers collaborate to complete reasoning tasks in an edge computing environment. By fully leveraging edge server resources, edge-device collaborative reasoning effectively compensates for the insufficient computing power and storage capacity of end devices during the reasoning process. Specifically, a large deep neural network is divided into multiple partitions. The shallow partitions are executed on the end device, and the intermediate results are transmitted over the network to the edge server to complete the remaining model reasoning.

[0003] To achieve the goal of deterministic transmission latency and better cope with real-time bandwidth fluctuations, researchers are currently using a progressive transmission mechanism to complete the transmission of intermediate data for end-to-edge collaborative reasoning. This fixes the network transmission latency and reflects fluctuations in network bandwidth in real time in the amount of data transmitted. According to the progressive transmission scheme, the end device will send as much data as possible within the available time under network dynamics, and the edge server will activate the corresponding candidate model based on the amount of intermediate data received. Once the data transmission time becomes independent of network bandwidth fluctuations, the determinism of the entire inference time can be ensured. A solution that adapts to end-to-cloud inference in a progressive manner (CLIO) combines the collaborative reasoning of deep learning models with the progressive transmission of intermediate results.

[0004] However, the large amount of intermediate data that cannot be transmitted can lead to a significant decrease in model inference accuracy. Incremental transmission is detrimental to maintaining inference accuracy stability, requiring models with higher accuracy and robustness. The candidate model in the CLI solution has limited ability to cope with network fluctuations. When the amount of intermediate data transmitted changes by 40%, inference accuracy varies by as much as 10%. When the model's capabilities no longer meet the current requirements for inference latency and accuracy, inference latency will fluctuate with network bandwidth to maintain accuracy. During incremental transmission, the accuracy of a single model decreases significantly, leading to more frequent model replacement in real-world scenarios. Furthermore, even after training multiple split-point codes based on bandwidth coverage, the loss in inference accuracy remains significant under poor network conditions. Clearly, the modifications to the neural network model and the solution's unique training method do not provide a single model with strong resilience to data loss. Therefore, the existing solution cannot effectively respond to wireless network fluctuations, and the ability of a single model to maintain inference accuracy in the face of data loss is poor, making it impossible to achieve deterministic latency-based edge-to-edge collaborative inference. Summary of the Invention

[0005] The present invention is aimed at the problems existing in the prior art and provides a collaborative reasoning method and system for deterministic delay in edge computing scenarios. First, the end-edge collaborative reasoning scenario information is obtained to determine the user's demand for reasoning delay and the tolerance range for loss of reasoning accuracy; the target model is obtained, which is the model that the task needs to perform reasoning. After the target model is divided into a shallow model and a deep model, it becomes a candidate model; a candidate model group is prepared, which is a group of candidate models with multiple different split points; a collaborative reasoning scheduler and a network bandwidth monitoring point are added to the edge device; end-edge collaborative reasoning is executed, and the candidate model configured at the split point with the maximum network transmission delay is selected as the initial model for deployment, the end device executes the shallow model, and the edge device executes the deep model. The execution result of the end device is transmitted to the edge device through the network based on the progressive transmission mechanism; the scheduler determines the delay used for network transmission based on the user's demand for reasoning delay and the computing delay of the end and edge devices, and adjusts the reasoning configuration based on the user's tolerance range for loss of reasoning accuracy. This method can optimize the training of neural network models and select the optimal robustness model to perform inference based on real-time network bandwidth fluctuations. It can serve applications such as edge intelligence, real-time audio and video, and autonomous driving.

[0006] To achieve the above objectives, the present invention adopts a technical solution: a collaborative reasoning method and system for deterministic delay in edge computing scenarios, comprising the following steps:

[0007] S1: Obtain information about the device-edge collaborative inference scenario and determine the user's requirements for inference latency and tolerance for loss of inference accuracy. Prepare candidate models with multiple split points. Divide the target model into a shallow model and a deep model.

[0008] S2: Add a collaborative inference scheduler and network bandwidth monitoring point to the edge device to schedule collaborative inference solutions and monitor network bandwidth in real time;

[0009] S3: Performs device-edge collaborative inference. The candidate model configured at the split point with the maximum network transmission delay is selected as the initial model for deployment. The end device executes the shallow model, and the edge device executes the deep model. The execution results of the end device are transmitted to the edge device through the network based on the progressive transmission mechanism.

[0010] S4: The scheduler determines the latency used for network transmission based on the user's requirements for inference latency and the computing latency of end and edge devices, and adjusts the inference configuration based on the user's tolerance for loss of inference accuracy.

[0011] As an improvement of the present invention, the end-edge collaborative reasoning scenario information in step S1 includes at least: end device information, edge device information, network bandwidth information and target model information; the user's demand for inference delay is a positive real delay value, and the user's tolerance range for inference accuracy loss is a positive real precision value.

[0012] As another improvement of the present invention, the candidate model in step S1 is divided at different layers of the model, the model structure is optimized through the self-attention module and the reordering module, and the model parameters are adjusted using the data loss simulation method.

[0013] As another improvement of the present invention, in step S2, the network bandwidth monitoring point detects the network bandwidth of the transmission channel at a frequency of 1 Hz to obtain real-time network bandwidth information.

[0014] As another improvement of the present invention, step S3 specifically includes the following steps:

[0015] S31: deploying the initial model on the device according to the instructions of the scheduler, wherein the shallow model is deployed on the end device and the deep model is deployed on the edge device. The input data of the neural network model inference is located on the end device. The data is input into the shallow model on the end device and the inference process is started.

[0016] S32: The end device executes the reasoning of the shallow model. The intermediate data generated by the reasoning is transmitted to the maximum extent under the influence of the fluctuating network bandwidth according to the transmission delay decided by the scheduler. The execution result of the end device is transmitted to the edge device through the network based on the progressive transmission mechanism. The edge device receives the intermediate data transmitted by the end device through the network as the input of the deep model.

[0017] As another improvement of the present invention, in step S32, when the intermediate data is not completed, the end device abandons the uncompleted transmitted data, and the edge device fills the discarded intermediate data bits with 0 as the input of the deep model to complete the remaining reasoning task.

[0018] As a further improvement of the present invention, in step S4, the scheduler reduces the model's sensitivity to intermediate data loss by increasing the number of self-attention modules before the model split point.

[0019] As a further improvement of the present invention, when the network bandwidth fluctuates greatly, the scheduler adjusts the depth of the inference model executed on the end device, modifies the device calculation delay to affect the network transmission delay, and achieves a trade-off between inference delay and inference accuracy.

[0020] In order to achieve the above-mentioned purpose, the present invention also adopts the following technical solution: a collaborative reasoning system with deterministic delay in an edge computing scenario, comprising a computer program, which implements the steps of any of the above-mentioned methods when executed by a processor.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] (1) The present invention analyzes the specific sources of network transmission delay jitter in a fine-grained manner through understanding and analyzing the end-edge collaborative reasoning process and the neural network model optimization method, monitors real-time network bandwidth fluctuations during the transmission process, and selects the model reasoning with optimal robustness in real time during the reasoning process through a real-time scheduling method.

[0023] (2) The present invention is based on an improved scheduling method that balances model inference accuracy and inference delay. By dynamically adjusting the model inference scheme, while ensuring deterministic inference delay, it effectively reduces the loss of model inference accuracy caused by maintaining delay determinism, and maintains the inference delay and inference accuracy fluctuations within the range of user requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of model structure optimization for realizing the method of the present invention;

[0025] Figure 2 A schematic diagram of a reordering module for implementing the method of the present invention;

[0026] Figure 3 is a flow chart of the steps of the method of the present invention;

[0027] Figure 4 A flowchart of the scheduler decision-making in the method of the present invention;

[0028] Figure 5 The delay data diagram of the method of the present invention and the comparative method in different network bandwidth environments;

[0029] Figure 6 This is a graph showing the delay data of the method of the present invention and the comparative method under network fluctuation conditions. DETAILED DESCRIPTION

[0030] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0031] Example 1

[0032] A collaborative reasoning method for deterministic delay in edge computing scenarios, such as Figure 3 As shown, the following steps are included:

[0033] S1: Obtain information about the device-edge collaborative inference scenario, understand the user's requirements for inference latency and tolerance for loss of inference accuracy, and based on this information, divide the target model into shallow and deep parts. Prepare multiple candidate models at different split points based on this information.

[0034] The end-to-end collaborative reasoning scenario information includes at least: end device information, edge device information, network bandwidth information, and target model information; the user's demand for reasoning delay is a positive real delay value, and the user's tolerance range for reasoning accuracy loss is a positive real precision value.

[0035] Different candidate models are split at different layers of the model, and self-attention modules and reordering modules are introduced to optimize the model structure. The candidate models are fine-tuned using the data loss simulation method. Among them, the introduction of the self-attention module is to introduce a 3-layer sub-attention module at the model split point; Figure 2 As shown in the figure, the reordering module reorders the intermediate data to be transmitted before the split point according to the attention score, so that the data that is more important for maintaining the inference accuracy has a higher transmission priority; the data loss simulation method randomly simulates the number of data loss bits in each round of neural network model training.

[0036] S2: Add a collaborative inference scheduler and network bandwidth monitoring point to the edge device to schedule collaborative inference solutions and monitor network bandwidth in real time.

[0037] The scheduler and network bandwidth real-time monitoring point are deployed on edge devices. The network bandwidth real-time monitoring point is used to detect the transmission channel network bandwidth during the inference process.

[0038] S3: During device-edge collaborative inference, the scheduler selects an initial model and deploys it on the device. The end device executes the shallow model, and the edge device executes the deep model. The execution results of the end device are transmitted to the edge device over the network based on a progressive transmission mechanism.

[0039] S31: Following the instructions of the scheduler, the selected candidate models are deployed on the devices. The shallow model is deployed on the end device, and the deep model is deployed on the edge device. The neural network model inference input data is located on the end device. This data is input into the shallow model on the end device, initiating the inference process.

[0040] S32: The end device first performs inference on the shallow model. The intermediate data generated by the inference is transmitted as much as possible according to the transmission delay determined by the scheduler under the influence of fluctuating network bandwidth. Any unfinished data is directly discarded.

[0041] S33: The edge device receives the intermediate data transmitted by the end device through the network as the input of the deep model. When the intermediate data is incomplete, the edge device fills the discarded intermediate data bits with 0 as the input of the deep model to complete the remaining reasoning tasks.

[0042] The scheduler runs the optimal robust scheduling method, sends decision information to end devices and edge devices based on the detection of network bandwidth, and adjusts the deployed model configuration.

[0043] S4: During end-edge collaborative inference, the scheduler determines the latency for network transmission based on the user's requirements for inference latency and the computational latency of the end and edge devices. It further adjusts the inference configuration based on the user's tolerance for loss of inference accuracy.

[0044] When network bandwidth fluctuates greatly, the scheduler adjusts the depth of the inference model executed on the end device, modifies the device calculation latency and thus affects the network transmission latency, achieving a trade-off between inference latency and inference accuracy.

[0045] The collaborative reasoning method with deterministic delay in edge computing scenarios ultimately implemented by the present invention can be quickly applied to the target model to optimize and fine-tune the model, and monitor real-time network bandwidth fluctuations during the reasoning process. The scheduler executes the optimal robustness scheduling method to make real-time decisions and select the most robust reasoning model configuration for reasoning.

[0046] Example 2

[0047] This example uses a VGG-19 model for edge-to-edge collaborative reasoning in an edge computing scenario. The end device uses a Jetson Nano, and the edge device uses a server equipped with an NVIDIA GeForce RTX 2080 Ti GPU. Implementing a collaborative reasoning method with deterministic latency in an edge computing scenario includes the following steps:

[0048] S1: Obtaining edge-end collaborative inference scenario information, the end device performs floating-point operations at a rate F d , the floating point operation rate F of the edge device e , the user's demand for inference latency ρ and the tolerance range for inference accuracy loss α, VGG-19 has 45 layers and 410 million floating-point operations. The convolution layer, activation layer, and pooling layer are used as a split granularity, with a total of 5 potential split points; first, the VGG-19 model structure is modified as follows Figure 1 As shown in Figure 2, three layers of self-attention blocks and a reordering block are added before the split point of the model, and one layer of self-attention block is added after the split point. Furthermore, the modified model is fine-tuned using the data loss simulation method to train all candidate model parameters.

[0049] S2: Add a collaborative inference scheduler and a network bandwidth monitoring point to the edge device to schedule collaborative inference solutions and monitor network bandwidth in real time. The network bandwidth monitoring point detects the network bandwidth of the transmission channel at a frequency of 1 Hz based on the existing network bandwidth tool to obtain real-time network bandwidth information.

[0050] S3: During device-edge collaborative inference, the scheduler selects an initial model and deploys it on the device. The end device executes the shallow model, and the edge device executes the deep model. The execution results of the end device are transmitted to the edge device over the network based on a progressive transmission mechanism.

[0051] S31: Deploy the selected candidate model on the device according to the instructions of the scheduler. The scheduler is based on the floating-point operation number C of the shallow and deep models. d 、C e , the user's demand for inference latency ρ and the floating-point operation rate F of the end and edge devices d 、F e Infer the network transmission delay T t , select the maximum T t The candidate model configured with the split point is deployed as the initial model. The shallow part of the model is deployed on the end device, and the deep part of the model is deployed on the edge device.

[0052] S32: The end device first performs the reasoning of the shallow model. The execution result of the end device is transmitted to the edge device through the network based on the progressive transmission mechanism. t Transmit as much intermediate data as possible within the time.

[0053] In a fluctuating network bandwidth environment, there are two situations in which intermediate data is transmitted:

[0054] In the first one, all the intermediate data are transmitted, and the edge device continues to perform subsequent inference after receiving all the intermediate data.

[0055] The second type is that the edge device receives the intermediate data transmitted by the end device through the network as the input of the deep model. When some of the intermediate data exists in T t If the transmission is not completed within the specified time, the end device will abandon the unfinished data. The edge device will transmit the data within the specified time. t After the time has passed, the unfinished data is padded with 0 and used as the input of the deep model to continue executing subsequent inputs.

[0056] S4: During end-edge collaborative inference, the scheduler determines the latency for network transmission based on the user's requirements for inference latency and the computational latency of the end and edge devices. It further adjusts the inference configuration based on the user's tolerance for loss of inference accuracy.

[0057] The scheduler executes the optimal robust scheduling method to make runtime decisions. The scheduler decision process is as follows: Figure 4 Within a certain range, the depth of the shallow network is related to the model's robustness to incomplete intermediate data transmission. The deeper the shallow network, the more robust the model. The scheduler can reduce the model's sensitivity to intermediate data loss by increasing the number of self-attention blocks before the split point.

[0058] Scheduling decisions are based on the following three principles: First, to accommodate latency-sensitive tasks, one of our goals is to ensure deterministic inference latency. The progressive transmission method can complete the data transmission process within a limited delay, and based on the reordering module, the probability of successful transmission of more important data is higher. The amount of data transmitted m is related to the inference accuracy of the model Δac c The correlation between them can be obtained based on the candidate model test to obtain the bijective function f, Δacc = f(m); secondly, the traditional intermediate data transmission time is no longer directly related to the terminal device computing time, but is coupled together through the stability of the model. The network transmission delay can be expressed as Third, the scheduler’s decision goal is to execute as many blocks as possible for inference on the terminal device, thereby enhancing the robustness of the model during transmission. This strategy aims to enhance the performance and reliability of the model.

[0059] Figure 4 The operation mechanism of the runtime scheduler is explained, which determines the model split points and T t, which aims to enhance the model's adaptability to network fluctuations under deterministic delay conditions, while minimizing the impact of intermediate data loss on inference accuracy. Specifically, if the time required to fully transmit intermediate data to achieve the user's expected inference accuracy is less than the user-specified inference delay under the current bandwidth conditions, the scheduler will use this extra time to execute as many model layers as possible on the terminal device, thereby enhancing the model's robustness to network fluctuations. If the user-specified inference delay is strict under the current bandwidth conditions, the scheduler will explore the split point and the transmission delay T t Various combinations of

[15] are used to determine the optimal balance between the amount of intermediate data to be transferred and inference accuracy. This relationship is used to adjust the time allocated to device inference and data transfer and to select the most robust inference model.

[0060] The method of the present invention has an advantage in inference latency compared with the existing CLIO solution and the Joint Accuracy and Latency Awareness Solution for Edge Cloud Execution (JALAD). We used Jetson Nano as the terminal device and deployed an edge server with an NVIDIA GeForce RTX 2080 Ti GPU. We used the ImageNet dataset as the input of the model, created a wireless network to connect the edge server and the terminal device, and used the COMCAST tool12 to simulate various network conditions. Compared with the existing CLI0 solution and the Joint Accuracy and Latency Awareness Solution for Edge Cloud Execution (JALAD), the method of the present invention has a significant advantage in inference latency. Under various bandwidth settings, the method of the present invention outperforms other methods, such as Figure 5 As shown in Figure 3, our method achieves a 1.00 to 1.39x speedup on VGG-19 on the Ci fa10 dataset. When the VGG-19 model is fully executed on the end device without collaborative inference, our model degenerates to the original VGG-19 model.

[0061] In edge computing environments, network instability is a common problem. To address this issue, we tested the inference latency of different solutions under network bandwidth fluctuations, assuming the accuracy loss is less than 0.1%. The results are as follows: Figure 6It shows that due to the compression of intermediate data, the inference time of the JALAD solution will fluctuate dramatically with the fluctuations of network bandwidth and machine real-time inference performance. While CLIO and the method of the present invention can both perform inference at a specific time, CLIO has difficulty coping with bandwidth fluctuations and may adjust the inference delay, resulting in accuracy fluctuations of up to 21.92%. In contrast, the delay of the method of the present invention is only affected by changes in device performance, and the inference time fluctuation is less than 5ms. In the case of short-term bandwidth fluctuations, the correlation between inference accuracy and bandwidth in the method of the present invention and the CLIO model is shown. The experiment converted bandwidth fluctuations into intermediate data to complete the transmission rate. The results showed that the method of the present invention is always better than the CLIO solution, with an inference accuracy maintenance capability of 1.25 to 1.43 times. This further proves the superiority of the method of the present invention in dealing with network instability problems. In summary, the invention of this case discloses a collaborative reasoning method and system with deterministic delay in edge computing scenarios. First, the end-edge collaborative reasoning scenario information is obtained, the user's demand for reasoning delay and tolerance for loss of reasoning accuracy are received, and multiple candidate models are prepared based on the relevant information; a collaborative reasoning scheduler and a network bandwidth monitoring point are added to the edge device to schedule the collaborative reasoning plan in real time and monitor the network bandwidth; in the process of executing end-edge collaborative reasoning, the scheduler selects the initial model to be deployed on the device, the end device executes the shallow model, the edge device executes the deep model, and the execution result of the end device is transmitted to the edge device through the network based on the progressive transmission mechanism; in the process of executing end-edge collaborative reasoning, the scheduler combines the user's demand for reasoning delay and the computing delay of the end and edge devices to determine the delay used for network transmission, further adjusts the reasoning configuration based on the user's tolerance for loss of reasoning accuracy, and selects the candidate model with the best real-time robustness for deployment and reasoning. This method can optimize the training of neural network models and select the best robustness model to perform reasoning based on real-time network bandwidth fluctuations. The system of the present invention can provide deterministic latency for end-edge collaborative reasoning while maintaining high reasoning accuracy according to user needs, and can serve applications such as edge intelligence, real-time audio and video, and autonomous driving.

[0062] It should be noted that the above content merely illustrates the technical idea of ​​the present invention and cannot be used to limit the scope of protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications all fall within the scope of protection of the claims of the present invention.

Claims

1. A collaborative reasoning method for deterministic delay in edge computing scenarios, characterized by , including the following steps: S1: Obtain information about the device-edge collaborative inference scenario to determine the user's requirements for inference latency and tolerance for loss of inference accuracy. Obtain the target model, which is the inference model required for the task. The target model is divided into shallow and deep models, which become candidate models. Prepare a candidate model group, wherein the candidate model group is a group of candidate models with different cut-off points; S2: Add a collaborative inference scheduler and network bandwidth monitoring point to the edge device to schedule collaborative inference solutions in real time and monitor network bandwidth. The collaborative inference solution is to select the best candidate model to perform collaborative inference in different network environments. S3: Performs device-edge collaborative inference. The candidate model configured at the split point with the maximum network transmission delay is selected as the initial model for deployment. The end device executes the shallow model, and the edge device executes the deep model. The execution results of the end device are transmitted to the edge device through the network based on the progressive transmission mechanism. S31: Deploy the initial model on the device according to the scheduler's instructions. The shallow model is deployed on the end device, and the deep model is deployed on the edge device. The input data for neural network model inference is located on the end device. The data is input into the shallow model on the end device, and the inference process is started. S32: The end device executes reasoning on the shallow model. The intermediate data generated by the reasoning is transmitted to the maximum extent possible under the influence of fluctuating network bandwidth according to the network transmission delay determined by the scheduler. The execution results of the end device are transmitted to the edge device through the network based on the progressive transmission mechanism. The edge device receives the intermediate data transmitted by the end device through the network as the input of the deep model. S4: The scheduler determines the latency used for network transmission based on the user's requirements for inference latency and the computing latency of end devices and edge devices, and adjusts the inference configuration based on the user's tolerance for loss of inference accuracy.

2. The collaborative reasoning method for deterministic delay in edge computing scenarios according to claim 1, characterized in that: The end-edge collaborative reasoning scenario information in step S1 includes at least: end device information, edge device information, network bandwidth information and target model information; the user's demand for reasoning delay is a positive real delay value, and the user's tolerance value for reasoning accuracy loss is a positive real number.

3. The collaborative reasoning method for deterministic delay in edge computing scenarios according to claim 1, characterized in that: The candidate model in step S1 is segmented at different layers of the model, the model structure is optimized through the self-attention module and the reordering module, and the model parameters are adjusted using the data loss simulation method, wherein the data loss simulation method randomly simulates the number of data loss bits in each round of neural network model training.

4. The collaborative reasoning method for deterministic delay in edge computing scenarios according to claim 1, characterized in that: In step S2, the network bandwidth monitoring point detects the network bandwidth of the transmission channel at a frequency of 1 Hz to obtain real-time network bandwidth information.

5. The collaborative reasoning method for deterministic delay in edge computing scenarios according to claim 4, characterized in that: In step S32, when the intermediate data is not transmitted, the end device abandons the untransmitted data, and the edge device fills the discarded intermediate data bits with 0 as the input of the deep model to complete the remaining reasoning task.

6. The collaborative reasoning method for deterministic delay in edge computing scenarios according to claim 5, characterized in that: In step S4, the scheduler reduces the model's sensitivity to intermediate data loss by increasing the number of self-attention modules before the model split point.

7. The collaborative reasoning method for deterministic delay in edge computing scenarios according to claim 5, characterized in that: When network bandwidth fluctuates greatly, the scheduler modifies the device computing latency by adjusting the depth of the inference model executed on the end device.

8. A collaborative reasoning system for deterministic latency in edge computing scenarios, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Multi-compression-version cloud side DNN collaborative reasoning acceleration method

    CN116663644A

  • Cloud edge collaborative reasoning acceleration method and system based on deep neural network

    CN117195966A