RNNLM system based on distributed neurons and design method thereof

A system design and design method technology, applied in neural learning methods, biological neural network models, etc., can solve problems such as concurrent neuron training, and achieve the effects of improving practicability, improving training efficiency, and reducing training time overhead.

Inactive Publication Date: 2016-11-30
JIANGSU UNIV +1
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a RNNLM system based on distributed neurons and its design method to solve the problem of concurrent training of neurons and improve the training efficiency of RNNLM, so as to reduce the training time overhead. The number of neurons and training samples to improve the practicality of RNNLM

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • RNNLM system based on distributed neurons and design method thereof
  • RNNLM system based on distributed neurons and design method thereof
  • RNNLM system based on distributed neurons and design method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0042] like figure 1 As shown, a distributed neuron-based RNNLM system includes a distributed neuron interaction node module server and multiple distributed neuron node module servers, in which acquisition parameters, calculation accuracy rate, calculation error rate, update Connect with the input layer, update with the last hidden layer connection, update with the output layer connection, training initial and distribution, update the output layer, result aggregation and update and other modules, the distribution and function description of the modules in the two types of servers are shown in Table 1 Show.

[0043] Table 1 Various types of servers and functional modules in the RNNLM system based on distributed neurons

[0044]

[0045] In a distributed neuron-based RNNLM system, the interaction process between the distributed neuron interaction node module server and the distributed neuron node module server is as follows: figure 2 shown.

Embodiment 2

[0047] According to the open source code of RNNLM, the distributed neuron-based RNNLM system was implemented in Spark using Scala, and the test environment was built with three servers, each server equipped with an Intel(R) Xeon(R) E5606 2.13GHz processor 2 1, 64G memory, the operating system is Centos6.7, the Spark version is RDMA-Spark-0.9.1, the network is 40GB Infiniband, the communication protocol is RDMA; the Driver node serves as the distributed neuron interaction node module server, and the Worker node serves as the distribution A neuron node module server, a Worker node runs multiple distributed neuron node modules to support a large number of distributed neurons. At the same time, the RNNLM open source code is used in one server to build a stand-alone RNNLM system, which works in one server, and the server configuration is the same as that of a server running a distributed neuron-based RNNLM system.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a RNNLM system based on distributed neurons and a design method thereof. In order to solve problems such as serial problems and inability of simulating biological neuron parallel execution characteristics, large training time cost, and difficulty in containing a plurality of neurons, the structure of the RNNLM system is changed by taking the distributed neurons capable of realizing parallel execution as a center, and then the RNNLM system based on the distributed neurons is designed. The design method comprises the structure based on the distributed type neurons, a distributed type neuron autonomic training method, and a distributed type neuron coordination method. The parallel execution of the biological neurons is simulated, and the training time cost of the RNNLM is effectively reduced, and therefore the neuron number and training samples in the RNNLM are increased under a precondition of reducing the training time cost, and the practicability of the RNNLM is improved.

Description

technical field [0001] The invention belongs to the field of statistics-based natural language processing, specifically relates to a language model based on a neural network, and designs a distributed neuron-based RNNLM system. Mainly change the RNNLM structure, reduce the training time overhead of the RNNLM system through the distributed training method, so that the number of neurons and training samples in the RNNLM can be increased while ensuring the reduction of the training time overhead, and the practicability of the RNNLM can be improved. Background technique [0002] Recurrent neural network (RNN) is a special kind of neural network. Different from the one-way input data and processing in traditional deep neural network, the hidden layer of recurrent connection is used instead of fixed multi-level hidden layer, which makes RNN save all historical information, and constructed a theoretically near-perfect neural network. At present, RNN has been widely used in non-lin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08
CPCG06N3/082
Inventor 牛德姣蔡涛彭长生薛瑞詹永照埃法·金斯利
Owner JIANGSU UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap