Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Neural network parameter initialization method and device

A neural network and initialization technology, applied in the field of neural network parameter initialization, can solve the problems of poor initialization performance and slow convergence speed of deep neural network parameters, and achieve the effect of improving convergence speed and performance

Pending Publication Date: 2020-03-24
HUAWEI TECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In the prior art, the randomization method and the method of transfer learning are used in the initialization of the deep neural network parameters, resulting in slow convergence speed and poor performance of deep neural network parameter initialization when training the deep neural network

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network parameter initialization method and device
  • Neural network parameter initialization method and device
  • Neural network parameter initialization method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0072] The technical solution in this application will be described below with reference to the accompanying drawings.

[0073] The technical solutions of the embodiments of this application can be applied to deep neural networks, for example: figure 1 A deep neural network 100 is shown.

[0074] figure 1 is a schematic diagram of a deep neural network 100 applicable to the embodiment of the present application. The diagram includes an input layer, a hidden layer, and an output layer.

[0075] A deep neural network literally means a deep neural network. A deep neural network consists of multiple layers such as figure 1 The leftmost layer shown is called the input layer, and the neurons in the input layer are called input neurons. figure 1 The rightmost layer shown is called the output layer, and the neurons located in the output layer are called output neurons, figure 1 The output layer shown in has only one output neuron. The layer between the input layer and the outpu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a neural network parameter initialization method and device in the field of artificial intelligence. The method comprises the following steps: determining a feature training setcorresponding to each local network of the neural network, and training a plurality of local networks based on the plurality of feature training sets, each local network in the plurality of local networks covering a part of the neural network; and taking all or part of the trained parameters of the plurality of local networks as initialization parameters of the neural network. According to the technical scheme provided by the invention, the neural network initialization parameters are obtained by training the plurality of local networks, so that the convergence rate and the parameter initialization performance of the neural network can be improved.

Description

technical field [0001] The present application relates to the field of artificial intelligence, and more specifically, to a method and device for initializing neural network parameters. Background technique [0002] Research on neural networks is divided into two directions. Among them, one research direction focuses on the process of biological information processing, called biological neural network; the other research direction focuses on engineering applications, called artificial neural network. Until the introduction of the concepts of deep network and deep learning in 2006, the neural network began to glow with a new round of life. [0003] Deep neural networks (DNN) refers to the deep neural network algorithm, which is a new popular topic in the field of machine learning in industry and academia in recent years. The DNN algorithm has successfully improved the recognition rate of the previous artificial neural network to a significant level. [0004] At present, de...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N3/04G06N3/08
CPCG06N3/084G06N3/045
Inventor 杨宁
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com