Distributed DNN Inference Using Capability-Based Model Partitioning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing deep neural network (DNN) inference processing is computationally expensive and requires appropriate division to optimize performance across multiple devices.

Innovation Solution

A neural network distribution server measures AI processing capabilities of devices and determines optimal division points for DNNs, distributing them across multiple AI devices for efficient inference processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If DNN inference processing is performed using a single device, then the model can execute complicated and advanced DNN processing, but the calculation cost becomes excessively large

Engineering Contradiction:

ImproveDNN processing capabilityVSAvoidcalculation cost

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The DNN model is divided into multiple segments (first DNN, second DNN, etc.) that can be distributed across multiple devices. Each device executes a portion of the DNN processing, thereby reducing the calculation burden on any single device while maintaining the overall processing capability for complicated tasks

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If the DNN model size is increased to execute more complicated processing, then the processing capability is improved, but the calculation cost and device load increase

Engineering Contradiction:

ImproveDNN processing capabilityVSAvoidmodel size

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Large DNN models are segmented into smaller sub-models that can be distributed across multiple devices. This allows the system to handle complicated processing tasks with large models while each individual device only needs to manage smaller, more manageable model segments

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple devices are combined to form a distributed computing system that collectively executes the full DNN model. The combined processing power of multiple devices enables execution of large, complicated models without overloading any single device

Inventive Principle:

Principle #5Merging (Combining)

3Use of energy by moving object

If DNN is divided into multiple parts, then the calculation cost on individual devices is reduced, but appropriate division according to processing amount becomes necessary

Engineering Contradiction:

Improvecalculation costVSAvoiddivision complexity

Core Design Contradiction:

Use of energy by moving objectVSEase of operation

Solution Approach 1:

The system obtains processing capability information from each terminal device and uses this feedback to determine the optimal division strategy. By measuring actual device capabilities and adjusting the division accordingly, the system automatically finds the appropriate balance between load distribution and processing requirements without manual intervention

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20260093955A1Server device, terminal device, information processing method, and information processing system

Publication Date: 2026.04.02 SONY SEMICON SOLUTIONS CORP

AI summary

An information processing system configured to transmit a first command to a first electronic device requesting processing capability information of the first electronic device; receive first parameters from the first electronic device in response to the first command; divide a deep neural network (DNN) into at least a first DNN and a second DNN based on the first parameters received from the first electronic device; and transmit the first divided DNN to the first electronic device.