Activation function generation method of neural network model

A technology of neural network model and activation function, applied in the direction of biological neural network model, neural architecture, etc., to achieve the effect of improving accuracy and improving the ability to learn nonlinear changes

Inactive Publication Date: 2017-09-01
SOUTH CHINA UNIV OF TECH
View PDF0 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a method for generating an activation function of a neural network model. The activation function generated by the method improves the abi

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Activation function generation method of neural network model
  • Activation function generation method of neural network model
  • Activation function generation method of neural network model

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0048] Example 1

[0049] This embodiment discloses a method for generating an activation function of a neural network model, such as figure 1 As shown, the steps are as follows:

[0050] S1. Select a number of different basic activation functions; in this step, generally choose 2 to 6 different basic activation functions; in this embodiment, the basic activation function can be Sigmoid function, Tanh function, ReLU function, PReLU function, PELU function or RReLU function. The multiple different basic activation functions in this embodiment are several of the above functions.

[0051] S2. Combine the multiple different basic activation functions selected in step S1 as the activation function of the neural network model. In this step, multiple different basic activation functions are combined to obtain the activation function f(x) of the neural network model in the following way:

[0052]

[0053] p n = 1-(p 1 +,p 2 +,…,+p n-1 );

[0054] Where p 1 ,p 2 ,...,P n Is the combination co...

Example Embodiment

[0064] Example 2

[0065] This embodiment discloses a method for generating an activation function of a neural network model, and the steps are as follows:

[0066] S1. Select a number of different basic activation functions; in this step, generally choose 2 to 6 different basic activation functions; in this embodiment, the basic activation function can be Sigmoid function, Tanh function, ReLU function, PReLU function, PELU function or RReLU function.

[0067] S2. Combine the multiple different basic activation functions selected in step S1 as the activation function of the neural network model. In this step, multiple different basic activation functions are combined to obtain the activation function f(x) of the neural network model in the following way:

[0068]

[0069] σ n (w n x) = 1-(σ 1 (w 1 x)+σ 2 (w 2 x)+,…+σ n-1 (w n-1 x));

[0070] Where σ i (w i x), i=1,2,...n represents the input as w i Sigmoid function of x, w 1 ,w 2 ,...,W n Is the combination coefficient of each basic a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention discloses an activation function generation method of a neural network model. The method includes the following steps that: S1, a plurality of different basic activation functions are selected; S2, the different basic activation functions selected in the S1 are combined so as to form the activation function of the neural network model; and S3, the activation function of the neural network model is updated with the iteration of the neural network model through a back propagation means. With the activation function generated by using the method of the invention can improve the nonlinear change learning capability of the neural network model, and the neural network model can respond differently to different input in a test stage, and a defect that a neural network uses a single activation function can be eliminated.

Description

technical field [0001] The invention relates to a field related to machine learning, in particular to a method for generating an activation function of a neural network model. Background technique [0002] In recent years, deep learning has achieved remarkable results in the field of computer vision, and one of the important factors is the development of activation functions. In the artificial neural network, the activation function of the neuron node defines the mapping to the output of the neuron. Simply put, the output of the neuron is processed by the activation function and then used as the output. [0003] The main function of the activation function in the neural network is to provide the nonlinear modeling ability of the network. Assuming that an example neural network only contains linear convolution and fully connected operations, then the network can only express linear mapping, even if the depth of the network is increased, it is still a linear mapping, and it i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04
CPCG06N3/048
Inventor 刘华钱生吴斯
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap