Edge-Server DNN Inference via Intermediate Output Sharing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for real-time applications using deep neural networks (DNNs) face challenges when different tasks are executed on edge devices and cloud servers, as they require data transmission and recalculations, leading to inefficiencies in resource utilization and increased transmission amounts.

Innovation Solution

A processing method and system that utilize an edge device to execute inference on a lightweight model for a first task and transmit intermediate output values to a server device for a second task with higher operational demands, allowing the server to perform the necessary computations, thereby reducing transmission and calculation loads.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If different tasks are executed on edge device and server device separately, then task specialization is achieved, but transmission amount increases and resource utilization decreases

Engineering Contradiction:

Improvetask specializationVSAvoidtransmission amount

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent merges the execution flow of different tasks by transmitting intermediate output values from the edge device to the server device, allowing both tasks to share common computational results and avoid redundant data transmission

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The intermediate output values generated by the edge device serve multiple purposes: they are used for the first task locally and also serve as input for the second task on the server device, making the transmitted data multi-functional

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If different tasks are executed on edge device and server device separately, then task independence is achieved, but resource utilization decreases

Engineering Contradiction:

Improvetask independenceVSAvoidresource utilization

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent combines the computational resources of edge and server devices into a coordinated system where intermediate results are shared, allowing independent tasks to benefit from combined processing power without requiring complete data retransmission

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If intermediate output values are transmitted from edge device to server device, then resource utilization improves, but device complexity increases

Engineering Contradiction:

Improveresource utilizationVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the overall processing into distinct stages: edge device execution of the first task, transmission of intermediate results, and server device execution of the second task, allowing each component to remain relatively simple while achieving efficient resource utilization

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20240290083A1Processing method, processing system, and processing program

Publication Date: 2024.08.29 NT T INC

AI summary

A processing system (100) is a processing system that executes inference process in an edge device (30) and a server device (20). The edge device (30) includes an inference unit (31) that executes inference related to a first task on inference target data by using a DNN1; and a determination unit (32) that transmits an intermediate output value of the DNN1 used to execute the inference related to the first task to the server device (20) so that the server device (20) executes a second task which is different from the first task and has a higher operation amount than the first task.