Hardware Neuron Layer Calculation with Simplified Activation Functions

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Physical models used in technical systems, such as internal combustion engines, face challenges in achieving real-time calculation accuracy due to complex correlations, requiring high processing capacity, which is difficult to meet with modern processing capabilities.

Innovation Solution

A method for calculating a neuron layer of a multi-layer perceptron model using a permanently hardwired processor core configured in hardware, employing simplified sigmoid and tan h functions based on zero-point mirroring of the exponential function, avoiding division and utilizing only multiplications and additions for efficient hardware-based calculation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a data-based functional model with a large number of nodes is used to achieve sufficient modeling accuracy, then the modeling precision is improved, but the processing capacity requirement increases

Engineering Contradiction:

Improvemodeling accuracyVSAvoidprocessing capacity

Core Design Contradiction:

Measurement precisionVSPower

Solution Approach 1:

The patent transforms the activation function calculation from a general mathematical operation into a hardware-optimized form by changing parameters: using piecewise linear approximation instead of exponential functions, avoiding division operations, and utilizing only multiplication and addition. This parameter transformation enables the hardware to achieve both high modeling accuracy with complex networks and real-time processing capabilities.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the software-based general-purpose processor calculation with a dedicated hardware circuit implementation. The activation function is hardwired into the processor core, substituting flexible software computation with fixed hardware logic that performs the same function more efficiently, thereby reducing processing capacity requirements while maintaining accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If a general-purpose processor is used to calculate data-based functional models, then the adaptability is improved, but the calculation speed deteriorates

Engineering Contradiction:

Improvemodel flexibilityVSAvoidcalculation speed

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The patent segments the activation function calculation into distinct operational ranges (positive and negative input domains) with dedicated calculation paths for each segment. This segmentation allows the hardware to quickly determine which calculation path to follow based on the input sign, dramatically improving calculation speed while maintaining the ability to handle different model configurations through programmable parameters.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary classification of the input value sign before entering the main calculation logic. By checking the sign bit first and pre-selecting the appropriate calculation path, the system avoids unnecessary computational steps, thereby improving calculation speed while maintaining adaptability to different activation function types through configurable parameters.

Inventive Principle:

Principle #10Preliminary action

3Manufacturing precision

If additional operational blocks are added to the hardware calculation unit to improve calculation accuracy, then the manufacturing precision is improved, but the device complexity increases

Engineering Contradiction:

Improvecalculation accuracyVSAvoidhardware structure

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent designs the hardware activation function unit to be universal and multi-functional, capable of implementing different activation function types (sigmoid, tanh, ReLU, and their variants) using the same hardware structure with configurable parameters. This universality allows the system to achieve high calculation accuracy for multiple model types without increasing device complexity through additional specialized blocks.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent merges the activation function calculation logic directly into the existing processor core architecture, combining it with the neural network computation units. This integration eliminates the need for separate operational blocks, reducing device complexity while maintaining high calculation accuracy through shared hardware resources and optimized data flow.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11216721B2Method for calculating a neuron layer of a multi-layer perceptron model with simplified activation function

Publication Date: 2022.01.04 ROBERT BOSCH GMBH

AI summary

A method for calculating a neuron layer of a multi-layer perceptron model that includes a permanently hardwired processor core configured in hardware for calculating a permanently predefined processing algorithm in coupled functional blocks, a neuron of a neuron layer of the perceptron model being calculated with the aid of an activation function, the activation function corresponding to a simplified sigmoid function and to a simplified tan h function, the activation function being formed by zero-point mirroring of the negative definition range of the exponential function.