Deep Neural Network Activation Function Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural networks face challenges in efficiently training networks with multiple hidden layers, modeling complex non-linear relationships, and providing a unified user interface, leading to difficulties in selecting appropriate activation functions for predicting such relationships.
Innovation Solution
A method and system for predicting non-linear relationships in a deep neural network framework that involves receiving parameter values, selecting an activation function based on desired output, industry type, and application area, and modeling the network using stochastic gradient descent with back-propagation to train the network with multiple hidden layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional artificial neural networks are used, then the system can perform basic predictions, but it cannot efficiently train networks with multiple hidden layers or model complex non-linear relationships
Solution Approach 1:
The patent changes the parameter of activation function selection from manual/random to automated based on industry type and application area. This allows the system to handle complex non-linear relationships effectively without requiring users to understand activation function selection, thereby improving prediction accuracy for complex models while maintaining ease of use.
Solution Approach 2:
The system performs self-service by automatically selecting appropriate activation functions based on the problem domain and application type. This eliminates the need for user intervention in complex configuration decisions, enabling the network to efficiently train with multiple hidden layers and model complex relationships without increasing user burden.
2Adaptability or versatility
If manual activation function selection is used, then users can customize the model, but the process becomes complex and requires trial and error
Solution Approach 1:
The patent introduces an intermediary mechanism that automatically maps industry type and application area to appropriate activation functions. This intermediary layer preserves model adaptability by selecting domain-appropriate activation functions while shielding users from the complexity of activation function selection, thereby maintaining customization benefits without increasing operational complexity.
3Adaptability or versatility
If existing artificial neural networks are used, then the system can process data, but it lacks a unified user interface and cross-language support
Solution Approach 1:
The patent implements a unified system that handles multiple functions including data reception, activation function selection, model training, and prediction within a single framework. This universal interface supports cross-domain research and multiple programming languages while maintaining system coherence, thereby improving adaptability without proportionally increasing integration complexity.
Data Source
AI summary
A method and a system are provided for predicting a non-linear relationship between a plurality of parameters in a deep neural network framework. The method comprises receiving, by an application server, a plurality of parameter values associated with the plurality of parameters. The method further comprises selecting, by the application server, an activation function based on a desired output. In an embodiment, the desired output is based on an industry type and an application area of the plurality of parameters. The method further comprises predicting, by the application server, the non-linear relationship between the plurality of parameters by modelling the deep neural network framework based on the selected activation function.