Piecewise Linear Rectifier Unit for Neural Network Error Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural network systems face challenges with component errors, particularly in high-integration, low-power deep-learning structures, where channel interference leads to distorted values during read operations, necessitating a method to compensate for these errors.
Innovation Solution
The implementation of a modified activation function, specifically a piecewise linear rectification unit (PLRU), which classifies input values into sections with different slopes, reducing error propagation by limiting the output range and using gradient descent for parameter determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If high-integration, low-power deep-learning structures are used, then power consumption is reduced and integration is increased, but component errors and channel interference cause distorted values during read operations
Solution Approach 1:
The activation function is segmented into multiple linear sections with different slopes, where each section handles a specific range of input values. This segmentation allows the system to apply different transformation characteristics to different value ranges, effectively compensating for component errors by adjusting the slope in each segment to counteract distortion in that particular range.
Solution Approach 2:
The invention changes the parameters of the activation function (specifically the slopes of different linear sections) to compensate for component errors. By adjusting the slope parameters in each linear section based on the range of input values, the system can counteract the effects of channel interference and distorted read values, thereby improving reliability while maintaining low-power operation.
2Reliability
If piecewise linear rectification unit with multiple sections is used, then component errors are compensated, but device complexity increases
Solution Approach 1:
The activation function is divided into multiple linear sections, each with a simple linear transformation. While the overall function becomes more complex with multiple sections, each individual section remains computationally simple, allowing the system to achieve error compensation through the combination of multiple simple linear operations rather than a single complex non-linear operation.
Solution Approach 2:
The invention uses parameter changes (adjusting slopes and threshold values) in a systematic way to achieve error compensation. By optimizing the parameters of each linear section to handle specific input ranges, the system can compensate for component errors without requiring overly complex computational structures, as the complexity is managed through parameter optimization rather than structural elaboration.
Data Source
AI summary
An artificial neural network system of the present invention can include an input layer that receives input data; an output layer that outputs output data corresponding to the input data; and a hidden layer group including a multiple number of hidden layers that are positioned between the input layer and the output layer and are connected by connection weights, with the connection weights updated based on a difference between the output data and prediction data corresponding to the input data, where each of the hidden layers can include a hidden node that includes an activation function, which can output 0 if the input value is negative and can determine the output value based on linear functions having different slopes according to the range of the input value if the input value is positive, and the linear functions can have smaller slopes for greater distances from the origin point.