A polynomial dendritic neural network and its prediction method

Abstract

The present invention is a polynomial dendritic neural network and a prediction method thereof, belonging to the field of neural networks. A polynomial dendritic neural network is composed of a plurality of neurons with dendritic functions layer by layer, and the polynomial dendritic neurons are composed of a plurality of synaptic signal receiving and processing modules, dendritic function modules, and nonlinear activation modules. A polynomial dendritic neural network prediction method comprises the following steps: S1: building a polynomial dendritic neural network according to the data structure and needs of the user; S2: pre-processing the user data and dividing it into a training set and a test set; S3: training the polynomial dendritic neural network using the training set, and testing it using the test set; S4: processing the user's new data using the trained polynomial dendritic neural network and outputting the result. The present invention can equivalently realize the prediction of the existing neural network. Compared with the existing dendritic neural network, it is not limited to the input and output of the same dimension and has a wider applicability.

Description

Technical Field

[0001] The invention relates to a polynomial dendritic neural network and a prediction method thereof, belongs to the field of neural networks, and particularly relates to a polynomial dendritic neural network and a prediction method thereof. Background Art

[0002] Deep neural networks (DNNs) are the foundation of many artificial intelligence applications. DNNs have achieved revolutionary success in areas such as computer vision, natural language processing, and speech recognition. However, a key bottleneck in practical applications is "interpretability", especially in risk-sensitive fields such as healthcare. These models are regarded as black boxes, and we do not know how they work and why they behave incorrectly. Therefore, this difficulty in interpreting limits the application and innovative research of deep neural networks. In recent years, the interpretability of deep learning has attracted increasing attention from academia and industry. The literature [1] reports many methods that aim to overcome this key weakness, sometimes at the expense of interpretability for accuracy. Each method is developed to provide a solution to a specific problem, and therefore, it explicitly or implicitly describes its definition of interpretability and explanation. Recent studies [2] have demonstrated a strong relationship between neural networks (NNs) and polynomial regression (PR) models. In addition, some special polynomial works, such as DD and Kileel et al.'s deep polynomial neural networks, show strong computability of some single neurons, which is consistent with recent findings on biological neurons that a single biological neuron can perform an XOR operation.

[0003] [1] Y. Dong, S. Hang, J. Zhu, and Z. Bo. Improving interpretability of deep neural networks with semantic information. IEEE, 2017.

[0004] [2] C. Xi, B. Khomtchouk, N. Matloff, and P. Mohanty. Polynomial regression as an alternative to neural nets. 2018.

[0005] [3] Z. Hui, J. Zhu, S. Rosset, and T. Hastie. Multi-class adaboost. 2009.

[0006] [4]SRSafavian and D.Landgrebe.A survey of decision tree classifiermethodology.IEEE Transactions on Systems, Man, and Cybernetics, 21(3):660-674, 1991.

[0007] [5] G¨unter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter. Selfnormalizing neural networks. CoRR, abs / 1706.02515, 2017. Summary of the invention

[0008] In view of this, the present invention provides a polynomial dendritic neural network and a prediction method thereof, which constructs a new neuron model, introduces a new dendritic function module and a nonlinear activation module, making it a combined polynomial regression model with biological significance; and a neural network architecture based thereon, making it a biologically interpretable neural network with powerful computing power.

[0009] To achieve the above objectives, combined Figure 1 , the present invention provides the following technical solutions:

[0010] A polynomial dendritic neural network, composed of multiple neurons with dendritic functions layer by layer, combined with Figure 2 , characterized in that the polynomial dendritic neuron is composed of multiple synaptic signal receiving and processing modules, dendritic function modules, and nonlinear activation modules; the receiving dimensions of the multiple synaptic signal receiving and processing modules are Input signal Then use the d-dimensional signal weight vector and signal bias Processing to obtain synaptic results; the dendrite function module is based on Receive information Using Dendrite Weights and dendritic bias Obtain dendrite function parameters; the nonlinear activation module is an activation function that uses the dendrite function parameters to act on the synaptic result;

[0011] The synaptic signal receiving and processing module inputs X=[x1, x2, ..., x d ] T With the synaptic weight vector W = [w1, w2, ..., w d] one by one, then sum the products, add the synaptic bias b, and get the

[0012] The dendritic function module will input With the dendrite weight vector A = [a1, a2, ..., a n ] one by one and then sum the product results, add the dendrite bias t, and get

[0013] The nonlinear activation module is implemented by using and The polynomial function of is used as the activation function to process the input and obtain the polynomial result

[0014] Preferably, combined with Figure 3 The polynomial dendritic neural network of layer l formed by the plurality of polynomial dendritic neurons layer by layer can be an APND structure, in which the polynomial dendritic neurons in the jth layer are related to the synaptic weight matrix W = [w1, w2, ..., w d ] one by one input X = [x1, x2, ..., x d ] T is the initial input of the polynomial dendritic neural network In the j-th layer of polynomial dendritic neurons, the dendritic weight matrix A = [a1, a2, ..., a n ] The one-to-one connected input is the n-dimensional output of the previous layer of the polynomial dendritic neural network The expression combined with the nonlinear activation module is Wherein, ο is the Hadamard multiplier.

[0015] Preferably, combined with Figure 4 The polynomial dendritic neural network of layer l formed by the multiple polynomial dendritic neurons layer by layer can be an EPDN structure, in which the synaptic weight matrix W = [w1, w2, ..., w n ] and the dendritic weight matrix A of the polynomial dendritic neuron = [a1, a2, ..., a n ] The one-to-one connected inputs are all the outputs of the previous layer of the polynomial dendritic neural network The expression combined with the nonlinear activation module is Wherein, ο is the Hadamard multiplier.

[0016] A polynomial dendritic neural network prediction method applied to a polynomial dendritic neural network is characterized by comprising the following steps:

[0017] S1: According to the user's data structure and needs, select APND structure or EPDN structure or other polynomial dendritic neural network to build;

[0018] S2: After preprocessing, the user data is divided into a training set and a test set;

[0019] S3: Use the training set to train the polynomial dendritic neural network, and use the test set to test it. After successful training, the synaptic weight vector W, synaptic bias b, dendritic weight vector A, dendritic bias t and number of layers l of each neuron are obtained;

[0020] S4: Use the trained polynomial dendritic neural network to process the user's new data and output the results.

[0021] Furthermore, the learning rule of the polynomial dendritic neural network during training is:

[0022] (1) Construct a loss function based on the user’s goal;

[0023] Preferably, the loss function is as follows:

[0024]

[0025] Among them, y is the groundtruth of the target, y L is the predicted output value of the polynomial dendritic neural network, m is the number of samples, and x is the set of all polynomial dendritic neurons in the polynomial dendritic neural network;

[0026] (2) Design the update formula for each polynomial dendritic neuron:

[0027]

[0028] Among them, i=1,…,d, j=1,…,n, and η is the learning rate parameter.

[0029] The beneficial effects of the present invention are as follows: the present invention provides a polynomial dendritic neural network and a prediction method thereof, constructs a new neuron model, introduces a new dendritic function module and a nonlinear activation module, making it a combined polynomial regression model with biological significance; and a neural network architecture based thereon, making it a biologically interpretable neural network with powerful computing power; the method of the present invention can equivalently realize the prediction of existing neural networks, and compared with existing dendritic neural networks, it is not limited to inputs and outputs of the same dimension and has a wider applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to make the purpose and technical solution of the present invention, the present invention provides the following drawings for explanation:

[0031] Figure 1 is a flow chart of the method of the present invention;

[0032] Figure 2 This is a diagram of the polynomial dendritic neuron architecture of the present invention;

[0033] Figure 3 This is the architecture diagram of the APND structure polynomial dendritic neural network in the present invention;

[0034] Figure 4 This is the architecture diagram of the EPND structure polynomial dendritic neural network in the present invention. DETAILED DESCRIPTION

[0035] Example: There are 70,000 cardiovascular disease data sets provided by a hospital. Each data set contains 11 dimensional features, namely X = [x1, x2, ..., x 11 ] T , corresponding to: age, height, special weight, gender, systolic blood pressure, diastolic blood pressure, cholesterol concentration, blood glucose concentration, smoking, alcohol intake and physical activity frequency; the data set is divided into training set, validation set and test set in a ratio of 6:2:2. In order to accurately predict the mortality rate of heart failure, we propose "a polynomial dendritic neural network and its prediction method".

[0036] The following will be combined with the attached Figure 1 , the preferred embodiments of the present invention are described in detail.

[0037] A polynomial dendritic neural network, composed of multiple neurons with dendritic functions layer by layer, combined with Figure 2 , characterized in that the polynomial dendritic neuron is composed of multiple synaptic signal receiving and processing modules, dendritic function modules, and nonlinear activation modules; the multiple synaptic signal receiving and processing modules receive an input signal X=[x1, x2,…, x] with a dimension of 11 11 ] T , and then use the d-dimensional signal weight vector W = [w1, w2, ..., w 11 ] and signal bias b to obtain synaptic results; the dendrite function module receives information according to n dimensions Using dendrite weights A = [a1, a2, ..., a n ] and dendrite bias t to obtain dendrite function parameters; the nonlinear activation module is an activation function that uses dendrite function parameters to act on synaptic results;

[0038] The synaptic signal receiving and processing module inputs With the synaptic weight vector W = [w1, w2, ..., w 11 ] one by one, then sum the products, add the synaptic bias b, and get the

[0039] The dendritic function module will input With the dendrite weight vector A = [a1, a2, ..., a n ] one by one and then sum the product results, add the dendrite bias t, and get

[0040] The nonlinear activation module is implemented by using and The polynomial function of is used as the activation function to process the input and obtain the polynomial result

[0041] Combination Figure 3 The polynomial dendritic neural network of layer l formed by the multiple polynomial dendritic neurons layer by layer can be an APND structure; in this structure, in the j-th layer of polynomial dendritic neurons, the synaptic weight matrix W = [w1, w2, ..., w d ] one by one input X = [x1, x2, ..., x d ] T is the initial input of the polynomial dendritic neural network X = [x1, x2, ..., x 11 ] T ; In the j-th layer of polynomial dendritic neurons, the dendritic weight matrix A = [a1, a2, ..., a n ] The one-to-one connected input is the n-dimensional output of the previous layer of the polynomial dendritic neural network The expression combined with the nonlinear activation module is Wherein, ο is the Hadamard multiplier.

[0042] At the same time, in order to better illustrate the method of the present invention, Figure 4 The polynomial dendritic neural network of layer l formed by the multiple polynomial dendritic neurons layer by layer can be an EPDN structure, in which the synaptic weight matrix W = [w1, w2, ..., w n ] and the dendritic weight matrix A of the polynomial dendritic neuron = [a1, a2, ..., a n ] The one-to-one connected inputs are all the outputs of the previous layer of the polynomial dendritic neural network The expression combined with the nonlinear activation module is Wherein, ο is the Hadamard multiplier.

[0043] A polynomial dendritic neural network prediction method applied to a polynomial dendritic neural network is characterized by comprising the following steps:

[0044] S1: According to the user's data structure and requirements, select the APND structured polynomial dendritic neural network for construction.

[0045] S2: Preprocess the user data.

[0046] S3: The polynomial dendritic neural network is trained using the training set and tested using the test set. After successful training, the synaptic weight vector W, synaptic bias b, dendritic weight vector A, dendritic bias t and number of layers l of each neuron are obtained.

[0047] Furthermore, the learning rule of the polynomial dendritic neural network during training is:

[0048] (2) According to the user's goal, the loss function is constructed as follows:

[0049]

[0050] Among them, y is the groundtruth of the target, y L is the predicted output value of the polynomial dendritic neural network, m is the number of samples, and x is the set of all polynomial dendritic neurons in the polynomial dendritic neural network;

[0051] (2) Design the update formula for each polynomial dendritic neuron:

[0052]

[0053] Among them, i=1,…,d, j=1,…,n, and η is the learning rate parameter.

[0054] After the training is completed, both neural networks of the present invention have 4 layers, and the number of neurons in each layer is 2-6-4-2 in sequence.

[0055] S4: Use the trained polynomial dendritic neural network to process the test set and validation set and output the results.

[0056] In order to better demonstrate the effect of the present invention, Adaboost[3], Decision Tree[4] and SeluNet[5] neural networks were selected to conduct comparative experiments on data. Each model and data set was trained for 30 epochs, batchSize=100, learning rate LR=0.001, and the results shown in Table 1 were obtained.

[0057] Table 1 Comparative experimental results of the embodiments

[0058]

[0059] It can be seen that the method of the present invention has sufficient accuracy and is superior to the prior art in most cases.

[0060] Furthermore, by analyzing the characteristics of the APND structured polynomial dendritic neural network, it can be intuitively concluded that age, systolic blood pressure and cholesterol concentration mainly affect the risk of death in patients with heart failure, which confirms the conclusion of medical research and achieves the purpose of interpretability.

[0061] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A cardiovascular mortality interpretability method based on a polynomial dendritic neural network, characterized in that: The invention is realized by a polynomial dendritic neural network, wherein the polynomial dendritic neural network is composed of a plurality of neurons with dendritic functions layer by layer, and the polynomial dendritic neurons are composed of a plurality of synaptic signal receiving and processing modules, a dendritic function module, and a nonlinear activation module; the plurality of synaptic signal receiving and processing modules receive an input signal X=[x1, x2, ..., x d ] T , and then use the d-dimensional signal weight vector W = [w1, w2, ..., w d ] and signal bias b to obtain synaptic results; the dendrite function module receives information according to n dimensions Using dendrite weights A = [a1, a2, ..., a n ] and dendrite bias t to obtain dendrite function parameters; the nonlinear activation module is an activation function that uses dendrite function parameters to act on synaptic results; The synaptic signal receiving and processing module receives the input X=[x1, x2, ..., x d ] T With the synaptic weight vector W = [w1, w2, ..., w d ] one by one, then sum the products, add the synaptic bias b, and get the The dendritic function module will input With the dendrite weight vector A = [a1, a2, ..., a n ] one by one and then sum the product results, add the dendrite bias t, and get The nonlinear activation module is implemented by using The polynomial function is used as the activation function to process the input and obtain the polynomial result Where X = [x1, x2, ..., x 11 ] T , corresponding to: age, height, weight, gender, systolic blood pressure, diastolic blood pressure, cholesterol concentration, blood sugar concentration, smoking, alcohol intake and physical activity frequency; The method comprises the following steps: S1: According to the user's data structure and needs, select APND structure or EPDN structure or other polynomial dendritic neural network to build; S2: After preprocessing, the user data is divided into a training set and a test set; S3: Use the training set to train the polynomial dendritic neural network, and use the test set to test it. After successful training, the synaptic weight vector W, synaptic bias b, dendritic weight vector A, dendritic bias t and number of layers l of each neuron are obtained; S4: Use the trained polynomial dendritic neural network to process the user's new data and output cardiovascular mortality results and corresponding interpretable analysis.

2. The cardiovascular mortality interpretability method based on polynomial dendritic neural network according to claim 1, characterized in that: The polynomial dendritic neural network comprises multiple polynomial dendritic neurons layer by layer, and the polynomial dendritic neural network of each layer may be an APND structure, in which the polynomial dendritic neurons of the jth layer are related to the synaptic weight matrix W=[w1, w2, ..., w d ] one by one input X = [x1, x2, ..., x d ] T is the initial input of the polynomial dendritic neural network In the j-th layer of polynomial dendritic neurons, the dendritic weight matrix A = [a1, a2, ..., a n ] The one-to-one connected input is the n-dimensional output of the previous layer of the polynomial dendritic neural network The expression combined with the nonlinear activation module is in, is the Hadamard multiplier.

3. The cardiovascular mortality interpretability method based on polynomial dendritic neural network according to claim 1, characterized in that: The polynomial dendritic neural network of the polynomial dendritic neural network may be an EPDN structure, wherein the synaptic weight matrix W=[w1, w2, ..., w n ] and the dendritic weight matrix A of the polynomial dendritic neuron = [a1, a2, ..., a n ] The connected inputs are all the outputs of the previous layer of the polynomial dendritic neural network. The expression combined with the nonlinear activation module is in, is the Hadamard multiplier.

4. The cardiovascular mortality interpretability method based on polynomial dendritic neural network according to claim 1, characterized in that: The learning rule of the polynomial dendritic neural network during training is: (1) Construct a loss function based on the user’s goal; (2) Design the update formula for each polynomial dendritic neuron: Among them, i=1,…,d, j=1,…,n, and η is the learning rate parameter.

Images