Bayesian deep learning memory optimization method
A technology of deep learning and optimization methods, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve problems such as increased memory overhead, and achieve the effect of reducing space
Pending Publication Date: 2020-04-10
QINGDAO RES INST OF BEIHANG UNIV
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] Since the size of the uncertainty matrix is M×N, it is necessary to open up the same size of memory space to store the feature matrix β, which increases the memory overhead by about 50%.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0044] Such as Figure 4 As shown, in this example, The dimension of the feature matrix β is changed from the original M×N to At this time, the characteristic matrix β is divided into β 1 , β 2 , β 3 , β 4 The four parts satisfy the following relational expression;
[0045]
[0046] At the same time, the sampled T uncertainty matrices H i (i=1,2,…,T), each uncertainty matrix can also be divided into Four parts, each part is inner product with the corresponding β,
[0047]
[0048] can get z 1 to z TThe T corresponding outputs. Different from existing methods, each round of calculation in this method produces T The sub-vectors of , after 4 iterations, T complete output vectors are obtained.
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a memory optimization method, in particular to a Bayesian deep learning memory optimization method based on decomposition and storage strategies. According to the method, thedimension of a feature matrix beta is changed from M*N to alpha M*N, wherein alpha is larger than 0 and is smaller than or equal to 1, at the moment, the feature matrix beta is divided into a featurematrix beta1, beta2 beta3,..., beta1 / a; for T sampled uncertainty matrixes Hi (i =1, 2, 3,...,T), each uncertainty matrix Hi is divided into 1 / a parts which includes H1, H2, H3,..., H1 / a;and the inner product of each part of the H1, H2, H3,...,H1 / a of the uncertainty matrix and the corresponding beta is obtained, and therefore, T complete output vectors from z1 to zT are obtained. With the method adopted, space for storing the feature matrix can be effectively reduced; the storage space is changed from original M*N to the alpha M*N; and additional memory overhead brought by decomposition and storage strategies can be reduced from 50% to alpha*50%.
Description
technical field [0001] The invention relates to a memory optimization method, in particular to a Bayesian deep learning memory optimization method based on decomposition and storage strategies. Background technique [0002] The combination of Bayesian methods and deep learning is called Bayesian deep learning or Bayesian deep neural network. Among them, deep neural network aims to build different types of learning models, while Bayesian inference aims to focus on training methods. Bayesian deep neural network is not a new type of network architecture, but provides a new neural network training algorithm. It not only has the powerful fitting ability of neural network, but also has the powerful ability of Bayesian method to represent uncertain information. It also has perfect mathematical theory support and good anti-overfitting ability. [0003] Deep learning and Bayesian deep learning use the same network structure, but the parameter expressions are different, such as fig...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06N3/063G06N3/04
CPCG06N3/08G06N3/063G06N3/045
Inventor 贾小涛杨建磊马宝健赵巍胜
Owner QINGDAO RES INST OF BEIHANG UNIV