Generalized Linear Units for Deep Learning Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional object recognition approaches are inefficient and time-consuming, particularly when users attempt to identify specific items in images or audio, leading to frustration and incomplete transactions in electronic marketplaces.
Innovation Solution
The implementation of a deep learning network-based approach using generalized linear units (GLUs) as activation units improves the accuracy of object recognition and classification, enabling faster and more precise identification of items in images and audio signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional object recognition approaches are used, then the system can identify objects, but the recognition process is slow and time-consuming
Solution Approach 1:
The patent changes the activation function parameters from conventional ReLU to generalized linear units (GLUs) with learnable parameters. This parameter change enables the network to adaptively optimize the activation characteristics, improving both recognition speed and accuracy by allowing gradients to flow more effectively through deeper networks without saturation issues.
Solution Approach 2:
The patent introduces dynamic parameter learning where the activation function parameters are not fixed but learned during training. The GLU parameters are updated via backpropagation, making the network dynamically adapt to the specific characteristics of the data, thereby improving recognition performance while maintaining computational efficiency.
2Device complexity
If conventional activation functions are used in deep learning networks, then the network structure can be implemented, but the training difficulty increases with network depth
Solution Approach 1:
The patent transforms the static activation function into a dynamic one by introducing learnable parameters. This allows the network to automatically adjust the activation characteristics during training, making it easier to train deeper networks by preventing gradient saturation and improving convergence behavior.
Solution Approach 2:
The patent implements feedback mechanisms through backpropagation where the performance gradient is fed back to update the GLU parameters. This continuous feedback loop enables the network to self-adjust and optimize its activation functions during training, reducing the difficulty of training deep architectures.
3Productivity
If users manually search through products using various options, then they can locate specific items, but the process is time-consuming and frustrating
Solution Approach 1:
The patent replaces the manual mechanical searching process with an automated image recognition system. Users simply capture an image of the product, and the GLU-based deep learning network automatically identifies and retrieves product information, eliminating the need for manual filtering and searching through hundreds or thousands of products.
Data Source
AI summary
Tasks such as object classification from image data can take advantage of a deep learning process using convolutional neural networks. These networks can include a convolutional layer followed by an activation layer, or activation unit, among other potential layers. Improved accuracy can be obtained by using a generalized linear unit (GLU) as an activation unit in such a network, where a GLU is linear for both positive and negative inputs, and is defined by a positive slope, a negative slope, and a bias. These parameters can be learned for each channel or a block of channels, and stacking those types of activation units can further improve accuracy.