Neural network search distributed training system and training method based on evolutionary computation

A neural network and evolutionary computing technology, applied in the direction of biological neural network model, calculation, calculation model, etc., can solve the problems of time-consuming, achieve linear improvement of efficiency, avoid overhead, avoid the increase of cluster scale, and reduce the amount of data Effect

Pending Publication Date: 2020-06-23
SICHUAN UNIV
View PDF8 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] The purpose of the present invention is to provide a neural network search distributed training system and training method based on evolutionary calculation, provide a distributed training system and method, and hand over the most time-consuming neural network performance evaluation part to the distributed computing node to complete , to achieve a linear increase in efficiency and solve the time-consuming problem of the evolutionary neural network structure search algorithm

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network search distributed training system and training method based on evolutionary computation
  • Neural network search distributed training system and training method based on evolutionary computation
  • Neural network search distributed training system and training method based on evolutionary computation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0055] A neural network search distributed training system based on evolutionary computing, including server nodes and computing nodes, the computing nodes and server nodes communicate through Socket; each computing node includes at least one GPU;

[0056] The server node creates the main process, which is responsible for the processing of data packets, the evolution of the population, and the control of the entire cluster computing node. Since one server node corresponds to multiple computing nodes, the server node also creates a shared sending queue and receiving queue. The entire population (including the evolutionary individuals of each generation) is stored in the shared sending queue, and the individuals that need fitness evaluation (ie, the following "packet B") are extracted from the sending queue and sent to the computing nodes.

[0057] All computing nodes establish connections with server nodes through socket communication. Once the connection is established, the se...

experiment example

[0088] In order to verify whether the neural network search distributed system based on evolutionary computing proposed in this paper is effective, a small-scale cluster is constructed and tested on this platform. The experimental environment consists of 1 service node and 4 computing nodes. The configuration of the four nodes is the same. The specific standards are shown in Table 1:

[0089] operating system CentOS7 GPU GTX1080Ti Memory 16G External storage 1T R & D platform Pytorch Python 3.6

[0090] Laboratory design: Based on the network structure of DenseNet, use the genetic algorithm to encode the network structure, search the neural network structure, and find out the neural network with the highest accuracy. The hyperparameters of the entire algorithm are as follows:

[0091] Population size: p=20

[0092]Termination algebra: T=15

[0093] Crossover probability: p c =0.5

[0094] Mutation probability: p m =0.5

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a neural network search distributed training system and training method based on evolutionary computation, and solves the problem of time consumption of an evolutionary neuralnetwork search algorithm. A neural network search distributed training system based on evolutionary computation comprises a server node and at least one computing node, and the server node communicates with the computing nodes; the server node is used for generating each generation of population (one individual represents one neural network model) based on an evolutionary algorithm, asynchronouslyreceiving a request sent by the computing node, distributing to-be-evaluated individuals, receiving the individuals of which the computing node completes fitness calculation, and controlling the computing node at the same time; and the computing node is used for asynchronously sending a request instruction to the server node, receiving a data packet (including an individual to be evaluated), decoding the individual to generate a neural network, training and verifying on a specified data set, completing neural network performance evaluation, namely completing individual fitness evaluation, andfeeding back the evaluated individual to the server node.

Description

technical field [0001] The invention relates to an artificial intelligence technology, in particular to a neural network search distributed training system and training method based on evolutionary calculation. Background technique [0002] In recent years, deep neural networks have been widely used in various fields, such as image classification, speech recognition, etc., and have made remarkable achievements. In recent years, more complex networks have been proposed to solve more complex problems. For example, VGG-16 has more than 130 million parameters, occupies 500M memory space, and requires 15.3 billion floating-point operations to complete image recognition tasks. [0003] At present, most of the neural networks are designed manually. The design principles are derived from the accumulation of prior knowledge and network structure. Designing such a network with superior performance requires a lot of energy and computing power. A very difficult thing, many hyperparamet...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N20/00G06N3/12G06N3/063
CPCG06N20/00G06N3/126G06N3/063
Inventor 吕建成叶庆孙亚楠卢莉方智阳
Owner SICHUAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap