Neural Network Bottleneck Portion for Distributed Data Transmission

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In distributed neural networks, large amounts of data transmission can lead to increased load on transmission paths, reducing data processing speed due to speed limitations, especially when using existing neural network configurations.

Innovation Solution

The implementation of a bottleneck portion within the neural network where the amount of transmission data is minimized, allowing for efficient data processing and transmission, utilizing a two-device system where one device processes data using a first neural network and transmits the bottleneck portion to another device for further processing using a second neural network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If data is transmitted between distributed neural networks, then processing can be distributed across multiple devices, but the amount of transmission data increases causing load on transmission paths and reducing processing speed

Engineering Contradiction:

Improvedistributed processing capabilityVSAvoiddata processing speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The overall neural network is segmented into multiple parts distributed across different devices. The first neural network on the first device performs initial processing and outputs to a bottleneck portion, while the second neural network on the second device continues processing. This segmentation enables distributed processing while the bottleneck design ensures minimal data transmission between devices.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the parameter of data dimensionality by introducing a bottleneck portion that reduces the number of channels or features in the transmitted data. This parameter change (reducing data size) allows efficient transmission between distributed devices while maintaining the essential information needed for continued processing.

Inventive Principle:

Principle #35Parameter changes

2Loss of energy

If a bottleneck portion is introduced to minimize transmission data, then transmission load is reduced, but the neural network structure becomes more complex

Engineering Contradiction:

Improvetransmission data loadVSAvoidneural network structure

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The neural network is segmented into multiple parts with a bottleneck portion connecting them. This segmentation creates a structured architecture where the bottleneck serves as a controlled interface between segments, reducing transmission data while maintaining organized network structure rather than arbitrary complexity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The bottleneck portion acts as an intermediary between the first and second neural networks. It mediates the data flow by transforming and reducing the data representation, serving as a controlled interface that simplifies the connection between distributed devices while minimizing transmission requirements.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Quantity of substance

If the first neural network processes data locally, then transmission data amount is minimized, but ensuring the bottleneck portion contains sufficient information for the second neural network becomes challenging

Engineering Contradiction:

Improvetransmission data amountVSAvoidinformation sufficiency

Core Design Contradiction:

Quantity of substanceVSLoss of information

Solution Approach 1:

The bottleneck portion transforms the data parameters by reducing the number of channels or features while preserving essential information through learned representations. This parameter transformation allows minimal data transmission that still contains sufficient information for the second neural network to continue processing effectively.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The bottleneck portion serves as an information-preserving intermediary that extracts and transmits only the most essential features from the first neural network's output. This mediator function ensures that while data amount is minimized, the critical information needed for accurate continued processing by the second neural network is maintained.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250095356A1Information processing apparatus, information processing system, computer-readable recording medium, and information processing method

Publication Date: 2025.03.20 SONY SEMICON SOLUTIONS CORP

AI summary

An information processing apparatus (1) includes: a processing section (112, 122) that processes data using a first neural network (NN1) which constitutes a part of an overall neural network (NN) and includes, in an output portion, a bottleneck portion (BN12) where the amount of transmission data is minimum or extremely small in the overall neural network (NN); and a transmission section (124) that transmits data of the bottleneck portion (BN12) of the first neural network (NN1), obtained by the processing using the first neural network (NN1) performed by the processing section (112, 122), to the outside.