Unlock instant, AI-driven research and patent intelligence for your innovation.

Deep neural network with equilibrium solver

A neural network and balance point technology, applied in the field of training neural network systems, can solve problems such as time-consuming, computationally complex, lengthy training sessions and/or models

Pending Publication Date: 2021-02-09
ROBERT BOSCH GMBH +2
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Although it is known to share weights on some or all layers of a neural network (see e.g. [1]), thereby reducing the amount of data to be stored for the weights of a neural network, even if weights are shared by several layers, it is usually still necessary to provide Each layer stores ephemeral data separately for forward and backward passes
[0005] Besides the large amount of data to be stored in memory, another disadvantage is that propagating through all the layers of a deep neural network during training (but in some cases also during subsequent use) can be computationally time, resulting in lengthy training sessions and / or high latency of the model during use

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep neural network with equilibrium solver
  • Deep neural network with equilibrium solver
  • Deep neural network with equilibrium solver

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0110] Refer below figure 1 and figure 2 to describe the training of neural networks using an alternative to layer stacks of neural networks with mutually shared weights, then refer to Figure 3 to Figure 6 Neural networks and their training are described in more detail, and refer to Figure 7 to Figure 9 The use of trained neural networks to control or monitor physical systems, such as (semi) autonomous vehicles, is described.

[0111] figure 1 A system 100 for training a neural network is shown. System 100 may include an input interface for accessing training data 192 for the neural network. For example, if figure 1 As illustrated in , the input interface may include a data storage interface 180 that may access training data 192 from a data storage 190 . For example, the data storage interface 180 can not only be a memory interface or a persistent storage interface (persistent storage interface) (for example, a hard disk interface or an SSD interface), but also a p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A neural network may comprise an iterative function (z[i+1] = f(z[i], theta, c(x)). Such an iterative function is known in the field of machine learning to be representable by a stack of layers whichhave mutually shared weights. As described in this specification, this stack of layers may during training be replaced by the use of a numerical root-finding algorithm to find an equilibrium of the iterative function in which a further execution of the iterative function would not substantially further change the output of the iterative function. Effectively, the stack of layers may be replaced bya numerical equilibrium solver 480. The use of the numerical root-finding algorithm is demonstrated to greatly reduce the memory footprint during training while achieving similar accuracy as state-of-the-art prior art models.

Description

technical field [0001] The present invention relates to systems and computer-implemented methods for training neural networks. The invention also relates to trained neural networks. The present invention also relates to systems and computer-implemented methods for inferring using trained neural networks, such as controlling or monitoring a physical system based on its state inferred from sensor data. The present invention also relates to a computer-readable medium comprising transitory or non-transitory data representing instructions for a processor system to perform any computer-implemented method. Background technique [0002] Machine-learned ("trained") models are widely used in many real-life applications, such as autonomous driving, robotics, manufacturing, building control, and others. For example, a machine learnable model can be trained based on sensor data acquired by one or more sensors to infer the state of a physical system (such as an autonomous vehicle or rob...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G06N5/04
CPCG06N3/084G06N5/041G06N3/045G06N3/044
Inventor 白绍杰J·Z·科尔特M·肖伯
Owner ROBERT BOSCH GMBH