Compressed Neural Network Output Layer Tuning for Regression Accuracy

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional deep learning-based neural networks face performance degradation due to compression, leading to reduced output accuracy, particularly in regression problems.

Innovation Solution

Adjust the number of nodes in the output layer of a neural network model to mitigate compression effects, enhancing output resolution and inhibiting performance degradation by increasing nodes assigned to regression problems while adhering to computational constraints.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If scalar quantization is performed to compress the neural network model, then the amount of computation is reduced, but the output resolution and accuracy of the model deteriorate

Engineering Contradiction:

Improvecomputation speedVSAvoidoutput resolution

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent segments the neural network model into multiple computation units that can be processed in parallel. By dividing the model into smaller independent units, the system achieves both compression and maintains output resolution through distributed computation across multiple nodes in the output layer

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the parameter of the number of nodes in the output layer dynamically. By adjusting the node count based on computational constraints and accuracy requirements, the system optimizes the trade-off between computation speed and output resolution, allowing the model to adapt to different hardware configurations

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If the number of nodes in the output layer is increased to maintain output resolution, then the computation time increases, but the model accuracy improves

Engineering Contradiction:

Improveoutput accuracyVSAvoidcomputation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent introduces dynamic adjustment of the number of nodes in the output layer based on computational constraints. The system can adaptively change the node count during different operational phases or hardware configurations, optimizing the balance between computation time and output accuracy rather than using a fixed architecture

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent applies partial action by using a reduced number of nodes when full accuracy is not required, such as in inference phases or resource-constrained environments. This allows the system to achieve acceptable accuracy with reduced computation time, applying the full node count only when necessary

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentEP3518153B1Information processing method and information processing system

Publication Date: 2026.03.11 PANASONIC INTELLECTUAL PROPERTY CORP OF AMERICA

Figure 1
Figure 2
Figure 3

AI summary

An information processing method including the following executed using a computer: obtaining a neural network model that solves a regression problem; obtaining input data and label data corresponding to the input data; compressing a network of the neural network model to obtain a compressed model; and changing the label data and the number of nodes in the neural network model, based on information indicating performance of the compressed model, the number of nodes being assigned to the regression problem, the information being calculated using the label data and output data which is obtained by inputting the input data to the compressed model.