A deep learning multi-microgrid elastic combination decision-making method based on Boosting perception

Abstract

The present invention provides a deep learning multi-microgrid elastic combination decision method based on Boosting perception, which involves the fields of Boosting classifier, deep neural network and multi-microgrid combined operation scheduling considering operation elasticity. It includes three parts: microgrid combination optimization modeling, learning sample generation based on NATAF transformation, and deep neural network learning based on Boosting perceptron. The microgrid combination optimization modeling takes the power supply recovery capability of important loads as the goal and provides operators with backup power supply paths; the learning sample generation based on NATAF transformation expands the original samples to all feasible spaces considering variable correlation; the deep neural network learning based on Boosting perceptron adjusts the weights of the deep neural network through hierarchical iterative traversal of neurons, and adopts a cross-voting mechanism to upgrade the binary perception learning to multi-classification learning. Utilizing the algorithm provided by the present invention, the real-time performance is high, the overall elasticity is taken into consideration, and the applicability is not limited.

Description

Technical Field

[0001] The present invention relates to the fields of Boosting classifiers, deep neural networks and multi-microgrid combined operation scheduling considering operation elasticity, and in particular to a deep learning multi-microgrid elastic combination decision-making method based on Boosting perception. Background Art

[0002] Due to the increasing pressure of environmental protection, energy is gradually transitioning from a model that relies heavily on centralized power grid supply to a model that consumes a large amount of distributed renewable energy locally. However, the integration of energy systems with a high proportion of new energy has put forward higher requirements for interactive energy management, especially operation recovery and reconstruction in emergency situations.

[0003] Multiple microgrids (MGs) can benefit from the coordination of distributed resources and are considered a promising way to promote renewable energy consumption, reduce global costs and reduce randomness. However, the inherent uncertainty of renewable energy generation and demand in different distributed energy systems hinders the economical interactive operation, especially high-resolution distributed energy systems. Parallel operation or interaction with adjacent microgrids increases the risk of cascading failures and requires frequent adjustment of operation modes.

[0004] Therefore, there is an increasing focus on distributed and renewable energy resources such as multi-microgrids to take advantage of the integration benefits in the alliance paradigm shift. The resilience factor helps to avoid the loss of power to critical loads (CLs) in extreme unexpected situations. Many means, such as black start devices, strong power networks and backup supply plans, enhance the resilience of distribution systems. With the improvement of these hardware, the flexible scheduling of system redundancy can also greatly enhance the resilience of the system. This potentially drives the paradigm shift of developing resilient multi-microgrid combination (ROMF) operations to further enhance the system's adaptability to the environment.

[0005] ROMF dispatch provides an updated microgrid networking method and a differential power supply restoration method when the system operation mode changes or the topology is disturbed. One of its primary goals is to reduce system operating costs in the long term or transition period. On the other hand, in order to ensure continuous power supply to critical loads, recovery indicators need to be considered. Some scholars have proposed a resilience assessment method based on the analytic hierarchy process (AHP). In some applications, the elastic trapezoid helps capture key system degradation and recovery characteristics through time-dependent resilience metrics. Choquet integral provides an available method to quantify resilience indicators and integrate them into the target.

[0006] At the same time, in order to reduce the risk of emergencies, effective ROMF scheduling should support online decision-making and provide a near-optimal microgrid combination operation plan as soon as possible. The development of machine learning technology provides a promising solution for solving multi-attribute decision-making problems online.

[0007] So far, people have conducted extensive research on the optimal scheduling of multi-microgrid combinations. Some scholars have proposed a cooperative alliance game theory method to coordinate energy transactions among multiple microgrids with minimal operating costs. In terms of power supply for important loads, a heuristic method is proposed to approximate the optimal strategy of microgrid combinations after disturbances, which speeds up the optimization process compared to the optimal algorithm. However, this method has not yet been implemented online. Kwami et al. developed a model to handle the restoration process of radial and mesh distribution systems with little preprocessing of the input data. These methods are only applicable to the restoration of important loads in extreme power outages, but not to common insulation and local fault processing. According to the U.S. Department of Energy (DOE), 53.12% of power emergencies can be restored within 6 hours, with only partial load shedding and regional system separation occurring.

[0008] Therefore, those skilled in the art are committed to developing a method that can overcome all of the above problems at the same time. Summary of the invention

[0009] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is how to ensure real-time performance while considering overall elasticity. To achieve the above-mentioned purpose, a deep learning multi-microgrid elastic combination decision method based on Boosting perception includes three parts: microgrid combination optimization modeling, learning sample generation based on NATAF transformation, and deep neural network learning based on Boosting perceptron. The microgrid combination optimization modeling takes the power supply recovery capability of important loads as the goal and provides operators with backup power supply paths; the learning sample generation based on NATAF transformation expands the original samples to all feasible spaces considering variable correlations; the deep neural network learning based on Boosting perceptron adjusts the deep neural network weights through hierarchical iterative traversal of neurons and appropriate reduction of problems; the microgrid combination optimization modeling includes the elasticity indicator system of the microgrid, the cost plus elasticity target of the microgrid combination, and constraints.

[0010] Furthermore, the elasticity index system of the microgrid mainly includes four aspects, namely, integrated cost (IC), power supply ratio (PSR), path redundancy (PR), and source redundancy (SR); the calculation of the integrated cost is as follows:

[0011] Define the real-time cost of the i-th microgrid as It consists of three parts: adjustable power generation cost Interactive electricity settlement with neighbors and user payments;

[0012] Define Θ to be the microgrid set;

[0013] The comprehensive cost includes the operating costs of all the microgrids in a scheduling cycle, which is described as

[0014]

[0015] Where Ω i is the set of all groups in the microgrid i; d il ∈{0,1} represents the physical connection status between the microgrid i and the microgrid l; if d il =1, then the microgrid i and the microgrid l are connected;

[0016] p t It is the real-time electricity price; represents the load on node j; represents the real-time interactive power of the microgrid i;

[0017] Specifically, the adjustable power generation cost It can be written as

[0018]

[0019] in is the real-time dispatchable power generation of dispatchable unit n, {a n,i ,b n,i ,c n,i} is the equivalent cost coefficient set of the dispatchable unit n in the microgrid i;

[0020] λ t is the marginal cost at time t; it is described as follows:

[0021]

[0022] The power supply ratio is calculated as follows:

[0023] The power supply ratio measures load satisfaction from the perspective of the electricity user;

[0024]

[0025] Where N L is the total number of load nodes;

[0026] represents the active power demand of the jth node;

[0027] The path redundancy is defined as the ratio between the total number of paths between the dispatchable generators and important loads and the number of important loads:

[0028]

[0029] N CL Represents the total number of important loads; N path is the total number of paths between the dispatchable generating units and all the important loads;

[0030] A similar possible network (SPN) is defined as a feasible path combination without loops for supplying energy to the important loads; the SR is defined as the ratio of the reserve capacity to the rated capacity of all the important loads in the SPN:

[0031]

[0032] The superscript c indicates the important load; represents the real-time spare capacity of the cth important load; Represents the real-time power of the cth important load.

[0033] Furthermore, the cost plus elasticity target of the microgrid combination is calculated as follows:

[0034] The elasticity index set is defined as X = {X1, X2, L, X e};

[0035] because We have X i ∩X j =φ and

[0036] The importance of the elasticity indicator set X is regarded as the fuzzy measure of the set X, denoted as μ(X);

[0037] The λ-fuzzy measure satisfies the following conditions:

[0038]

[0039] Where μ(X i ) indicates the importance of elasticity indicators;

[0040] According to the boundary conditions, the equation can be rewritten as

[0041]

[0042] Definition: If h is a real-valued function and μ is the fuzzy measure on X, then the discrete Choquet integral of h with respect to the fuzzy measure μ is

[0043]

[0044] Among them, we assume that 0≤h(X1)≤h(X2)≤…≤h(Xε ),{x (i) ,x (i+1) ,…x (ε)} and C (ε+1) =0;

[0045] h(X i ) is the specific value of the indicator;

[0046] Therefore, the cost plus elasticity target of the microgrid portfolio is described as

[0047]

[0048] Where Ψ={d il ,s j ,c jm ,u ij}, i∈Θ, j∈Ξ is the set of decision variables.

[0049] Furthermore, the constraints include optimal scheduling constraints, connection constraints, node association constraints, node connection constraints, switch connection constraints, and load energy supply constraints;

[0050] (1) Optimal scheduling constraints

[0051] The power balance constraint is a global constraint that all microgrids must satisfy simultaneously:

[0052]

[0053] In addition to the global constraints, each microgrid has some individual constraints. For i∈Θ,

[0054]

[0055] in The real-time renewable energy generation of the microgrid is for the boundary; is the real-time charging state SOC of the energy storage in the microgrid i; η i is the discharge or charge efficiency; are the lower and upper bounds of discharge and charge, respectively; Represents lower and upper bounds on schedulable generation; are the minimum and maximum SOC respectively;

[0056] (2) Connection constraints

[0057] The static graph of the multi-microgrid system can be defined as G, A(G) = {a il} represents the correlation matrix of the graph, a il =1 indicates that there is a connection between the microgrid i and the microgrid l. The microgrid connectivity constraint can be as follows

[0058]

[0059] (3) Node association constraints

[0060] Define auxiliary binary variable u ij ∈{0,1};

[0061] Determine whether the load node j with power shortage risk belongs to the microgrid i∈Θ. If the microgrid i is responsible for the load node j, then u ij = 1, therefore, the constraint dependent on the load node can be described as

[0062]

[0063] in is the set of all the load nodes;

[0064] For power generation nodes or battery nodes installed in a microgrid, such nodes must belong to the microgrid and their affiliation cannot be changed; the equality constraints are as follows:

[0065]

[0066] Among them G i and B i It is the generator and battery bank of microgrid i;

[0067] (4) Node connection constraints

[0068] The topological structure of the distribution network is usually a radial tree, in which a node belongs to the microgrid i only if its parent node also belongs to the microgrid i; assuming θ i (j) is the parent node of node j of microgrid i,

[0069]

[0070] (5) Opening and closing connection constraints

[0071] If the line (j, m) belongs to the same microgrid, the switch on the line should be closed, c jm ∈{0,1} is the switch state of the path (j,m); a value of 1 indicates that the path between node j and node m is connected, while a value of 0 indicates that the switch between node j and node m is disconnected; therefore, the node-path constraint can be expressed as

[0072]

[0073] in is a set of nodes on the line (j,m);

[0074] (6) Load energy supply constraints

[0075] If the load at node j is powered by microgrid i That is u ij = 1, the switch associated with the load and node j should be closed, s j ∈{0,1} is a binary decision variable, with a value of 0 indicating that the load in node j is powered, and a value of 1 indicating that the load in node j is not powered; therefore, the constraint can be described as

[0076] u ij ≥s j

[0077] Furthermore, the calculation of learning sample generation based on NATAF transformation provides a probability model of photovoltaic power generation, wind power generation and load;

[0078] (1) Photovoltaic power generation probability model

[0079] Since photovoltaic power generation is mainly determined by light, the photovoltaic power generation shape has its own unique pattern and is easy to predict. In order to simplify the learning space, the photovoltaic power generation and light intensity are defined based on the maximum photovoltaic power generation distribution. The linear relationship between the two models significantly reduces the sampling dimension.

[0080]

[0081] in is the predicted daily photovoltaic power generation curve of node j, is the typical daily photovoltaic power generation curve of node j, G PV is the set of photovoltaic nodes;

[0082] (2) Wind power and load probability model

[0083] The number of wind turbines and the load nodes are defined as N WT and N L , the number of input random variables is equal to the number of wind turbines and load nodes N dim =N WT +N L The Nataf transform reconstructs the joint distribution when the probability distribution of the input variables is given. The default input is the probability distribution of wind power and load.

[0084] Assume that N dim The sample vector composed of variables is For a random variable S g , S g The cumulative distribution function can be described as

[0085]

[0086] Nataf theory and Cholesky decomposition take into account correlation and transform non-normal random variables into independent standard normal random variables;

[0087] In order to explore the correlation between regional wind power generation and load, Nataf theory is used to recalculate S as a standard normal distribution vector

[0088]

[0089] where Φ is the cumulative distribution function of the standard normal distribution, assuming and is the correlation coefficient matrix between Z and S, and Can be converted into each other;

[0090] Cholesky decomposition removes the correlation between random variable samples. Z is a symmetric positive definite matrix, which can be decomposed by Cholesky R Z =LL T to describe, L is a real non-singular lower triangular matrix, Z * is an independent standard normal random vector:

[0091]

[0092] Z * The correlation coefficient is actually the same as R Z The same, then, a random sample of arbitrary distribution is obtained by inverse Nataf transformation;

[0093] Assume N WL is the number of samples of wind power generation and load. After the PV power generation, wind power generation and load data are generated, wind power generation, load and PV power generation N ζ Each sample of is merged into a system sample, where z is uniformly distributed on [0,1]. This method is used to generate learning samples, and the sample dimension is N = N ζ ×N WL .

[0094] Furthermore, the deep neural network learning based on Boosting perceptron includes three parts, namely, a two-layer feedforward neural network, activation function linearization, and a Boosting-based classification learner; the two-layer feedforward neural network calculation process:

[0095] The classifier extends the single-layer perceptron to a DNN with linear neurons, and uses the layer-by-layer iterative traversal of the neurons to adjust the connection weights of the DNN;

[0096] The basic classifier can be modeled as a mapping x a →{-1,+1}, assuming is a feature set, the superscript represents sample a, and its sample classification is F a ∈{-1,+1};

[0097] For the convenience of analysis, it is assumed that the characteristics of sample a in layer b are L, The bias term weight of the threshold is equal to -1. The superscript and subscript represent the layer number and feature number respectively. In order to clearly represent the DNN, for the neuron k, is the weight of the qth input feature in the bth layer;

[0098] Therefore, the two-layer feedforward neural network consists of three layers, the 0th layer is Q input neurons [I1, I2, ..., I Q ], the first layer consists of K hidden neurons [H1,H2,…,H K ], the second layer is an output neuron O;

[0099] Here the hidden neurons are designed To learn the weights of the first layer, the hidden neurons H k The learning mechanism of can be described by the following equation:

[0100]

[0101] Where f1 is the activation function of layer 1, is the hidden neuron H k Output:

[0102] The calculation formula for the single output O of the ath sample is:

[0103]

[0104] Wherein f2 represents the activation function of the second layer;

[0105] Therefore, keeping the rest of the DNN unchanged, the hidden neuron H r The learning process can be described as:

[0106]

[0107] The hidden neuron H r The learning process is re-described as:

[0108]

[0109] Furthermore, the activation function linearization calculation process is:

[0110] Since the dimension of the learning task is high, the binary decision tree requires a large number of basic classifiers. Therefore, the learning problem of the output neuron is transformed into a perceptron, and the training pair (x a ,F a ) is transformed into a new training pair (Y a ,F a ),in

[0111] Assumptions Representing the linear output of a neuron before applying the activation function, the Taylor series of the Sigmoid function can be written as follows:

[0112]

[0113] The Taylor series linearization of the tanh activation function is:

[0114]

[0115] Piecewise functions can be estimated using continuous approximation functions. For example, the ReLU function can be represented by a linear piecewise function or estimated by the Softplus function:

[0116]

[0117] In general, the Taylor series representation of the activation function can be unified as:

[0118]

[0119] A nonlinear activation function is introduced to generalize the original input features to all series less than or equal to σ and applied to the subsequent DNN perceptron.

[0120] Furthermore, the calculation process of the Boosting-based classification learner is:

[0121] The single-node decision learner τ is regarded as a decision tree with only one factor. The decision tree can be defined by The linear combination W τ OK, where J τ is the threshold for distinguishing the two categories. The equivalent decision tree can be described as follows:

[0122] s τ (x a )=-[W τ ·(x a ) T -J τ ]

[0123] is obtained by combining the samples x a and threshold value;

[0124] Accordingly, the weight vector is expressed as s τ (x a ) represents the decision result of the learner τ, which can be expressed in homogeneous coordinates;

[0125]

[0126] The Boosting method selects several basic learners with adaptive weight distribution by learning training examples, and forms a complete learner by linearly combining the selected learners according to the weight distribution;

[0127] Using the decision tree Replacing the basic classifier and combining the algorithm operation, the final decision tree can be described as

[0128]

[0129] where a t is the total error rate of the existing training set;

[0130] DNN learning is the mapping of Boosting perceptron. In this mapping, the hidden neuron is equivalent to the basic classifier, and the output neuron O is similar to the final decision tree. Specifically, the weight of the hidden neuron r is Can be regarded as the weight of the basic classifier In addition, the weights of the hidden neurons r in the second layer are It corresponds exactly to the error rate a t Therefore, the two-stage learning in DNN learning is equivalent to learning the basic classifier The weights and the total error rate a t .

[0131] Furthermore, the binary classification DNN learning algorithm of the Boosting perceptron can be summarized as follows:

[0132]

[0133] Furthermore, the multi-classification DNN learning algorithm of the Boosting perceptron can be summarized as follows:

[0134]

[0135]

[0136] After multi-classification learning, samples can be classified according to the following steps:

[0137]

[0138] Compared with the prior art, the present invention has the following beneficial effects:

[0139] The proposed microgrid combination dispatch takes into account both operation economy and system resilience, adjusts the microgrid combination scheme online and provides a transition mode to improve resilience in emergency situations.

[0140] Based on Nataf theory and Cholesky decomposition, the proposed sample generation scheme fully exploits the intrinsic connection between RES generation and load and avoids the difficulty of solving the joint probability distribution.

[0141] It is proved that two-stage learning in deep neural network (DNN) learning is equivalent to learning the weights of basic classifiers. The proposed Boosting-based DNN adjusts the weights of DNN through hierarchical iterative traversal of neurons and learns a series of basic classifiers through the Boosting perception method.

[0142] Based on the trained DNN, flexible microgrid combination scheduling can be implemented online, and an approximately optimal combination solution can be quickly given without initialization, thereby greatly reducing emergency risks and operating costs.

[0143] The concept, specific structure and technical effects of the present invention are further described below to fully understand the purpose, characteristics and effects of the present invention. DETAILED DESCRIPTION

[0144] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0145] The present invention mainly consists of three parts:

[0146] (1) The dispatch center establishes a multi-microgrid combined operation optimization model oriented to elasticity goals;

[0147] (ii) Conduct correlation analysis on historical data based on Nataf theory, and generate large sample training samples from small sample historical data;

[0148] (III) Through the Boosting-based DNN multi-classifier, learning samples are used to determine the optimal multi-microgrid networking solution based on input information such as new energy forecasts and loads.

[0149] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A deep learning multi-microgrid elastic combination decision method based on Boosting perception, including microgrid combination optimization modeling, learning sample generation based on Nataf transform, and deep neural network learning based on Boosting perceptron. The microgrid combination optimization modeling takes the power supply restoration capability of important loads as the goal and provides operators with backup power supply paths; The learning samples are generated based on Nataf transformation, and the original samples are expanded to all feasible spaces considering the correlation of variables; the double-layer feedforward deep neural network is designed based on the Boosting perceptron principle, the weights of the deep neural network are adjusted through the hierarchical iterative traversal of neurons, and the cross-voting mechanism is adopted to upgrade the binary perception learning to multi-classification learning; The microgrid combination optimization modeling includes a microgrid elasticity index system, a cost plus elasticity target of the microgrid combination, and constraints; The microgrid elasticity index system has four indicators, namely, comprehensive cost IC, power supply ratio PSR, path redundancy PR, and source redundancy SR. The calculation of the comprehensive cost is as follows: Define the real-time cost of the i-th microgrid as It consists of three parts: adjustable power generation cost Interactive electricity settlement with neighbors and user payments; Define Θ to be the set of microgrids; The comprehensive cost includes the operating costs of all the microgrids in a scheduling cycle, which is described as Where Ω i is the set of all groups in microgrid i; d il ∈{0,1} represents the physical connection status between microgrid i and microgrid l; if d il =1, then the microgrid i and the microgrid l are connected; p t It is the real-time electricity price; represents the load on node j; represents the real-time interactive power of the microgrid i; Specifically, the adjustable power generation cost Written as in is the real-time dispatchable power generation of dispatchable unit n, {a n,i ,b n,i ,c n,i } is the equivalent cost coefficient set of the dispatchable unit n in the microgrid i, yes The square of λ t is the marginal cost at time t; it is described as follows: The power supply ratio is calculated as follows: The power supply ratio measures load satisfaction from the perspective of the electricity user; Where N L is the total number of nodes; represents the active power demand of the jth node; The path redundancy is defined as the ratio between the total number of paths between the dispatchable generators and important loads and the number of important loads: N CL Represents the total number of important loads; N path is the total number of paths between the dispatchable generating units and all the important loads; The similar possible network SPN is defined as a feasible path combination without loops, which is used to supply energy to the important loads; the SR is defined as the ratio of the spare capacity to the rated capacity of all the important loads in the SPN: The superscript c indicates the important load; represents the real-time spare capacity of the cth important load; represents the real-time power of the cth important load; The cost plus elasticity target of the microgrid combination is calculated as follows: The elasticity index set is defined as X = {X1,X2,···,X ε }; because Where i≠j, there is X i ∩X j =Φ and The importance of the elasticity indicator set X is regarded as the fuzzy measure of the set X, denoted as μ(X); The λ-fuzzy measure satisfies the following conditions: Where μ(X i ) indicates the importance of elasticity indicators; According to the boundary conditions, the equation can be rewritten as Definition: If h is a real-valued function and μ is the fuzzy measure on X, then the discrete Choquet integral of h with respect to the fuzzy measure μ is Among them, the default is 0≤h(X1)≤h(X2)≤…≤h(X ε ),{x (i) ,x (i+1) ,…x (ε) } and C (ε+1) =0; h(X i ) is the specific value of the indicator; Therefore, the cost plus elasticity target of the microgrid portfolio is described as where Ψ = {d il , s j , c jm , u ij}, i ∈ Θ, j ∈ Ξ; The calculation of learning sample generation based on Nataf transformation provides a probability model of photovoltaic power generation, wind power generation and load; (1) Photovoltaic power generation probability model Since photovoltaic power generation is determined by light and is easy to predict, in order to simplify the learning space, a linear relationship between photovoltaic power generation and light intensity l is defined based on the maximum photovoltaic power generation distribution. in is the predicted daily photovoltaic power generation curve of node j, is the typical daily photovoltaic power generation curve of node j, G PV is the set of photovoltaic nodes; (2) Wind power and load probability model The number of wind turbines and the nodes are defined as N WT and N L， The number of input random variables is equal to the number of wind turbines and load nodes N dim =N WT +N L The Nataf transform reconstructs the joint distribution when the probability distribution of the input variables is given. The default input is the probability distribution of wind power and load. Assume that N dim The sample vector composed of variables is For a random variable S g , S g The cumulative distribution function is described as Nataf theory and Cholesky decomposition take into account correlation and transform non-normal random variables into independent standard normal random variables; In order to explore the correlation between regional wind power generation and load, Nataf theory is used to recalculate S as a standard normal distribution vector where Φ is the cumulative distribution function of the standard normal distribution, assuming and is the correlation coefficient matrix between Z and S, and Mutual conversion; Cholesky decomposition removes the correlation between random variable samples. Z is a symmetric positive definite matrix, using Cholesky decomposition R Z =LL T to describe, L is a real non-singular lower triangular matrix, Z * is an independent standard normal random vector: Z * The correlation coefficient is actually the same as R Z The same, then, a random sample of arbitrary distribution is obtained by inverse Nataf transformation; Assume N WL is the number of samples of wind power generation and load. After the PV power generation, wind power generation and load data are generated, wind power generation, load and PV power generation N ξ Each sample of is merged into a system sample, where z is uniformly distributed on [0,1]. This method is used to generate learning samples, and the sample dimension is N = N ξ ×N WL ; The deep neural network learning based on Boosting perceptron includes three parts, namely, two-layer feedforward neural network, activation function linearization, and Boosting-based classifier; the two-layer feedforward neural network calculation process: The classifier extends a single-layer perceptron to a deep neural network DNN with linear neurons, and uses layer-by-layer iterative traversal of the neurons to adjust the connection weights of the DNN; The classifier is modeled as a mapping x a →{-1,+1}, assuming is a feature set, the superscript represents sample a, and its sample classification is F a ∈{-1,+1}; For the convenience of analysis, assume that the characteristics of sample a in layer b are The bias term weight of the threshold is equal to -1. The superscript and subscript represent the layer number and feature number respectively. In order to clearly represent the DNN, for the neuron k, is the weight of the qth input feature in the bth layer; Therefore, the two-layer feedforward neural network consists of three layers, the 0th layer is Q input neurons [I1, I2, ..., I Q ], the first layer consists of K hidden neurons [H1,H2,…,H K ], the second layer is an output neuron O; Here the hidden neurons are designed To learn the weights of the first layer, the hidden neurons H k The learning mechanism of is described by the following equation: Where f1 is the activation function of layer 1; The calculation formula for the single output O of the ath sample is: Wherein f2 represents the activation function of the second layer; Therefore, keeping the rest of the DNN unchanged, the hidden neuron H r The learning process is described as: The hidden neuron H r The learning process is re-described as: The activation function linearization calculation process: The learning problem of the output neuron is transformed into a perceptron, and the training pair (x a ,F a ) is transformed into a new training pair (Y a ,F a ),in Assumptions Representing the linear output of a neuron before applying the activation function, the Taylor series of the Sigmoid function is written as follows: The Taylor series linearization of the tanh activation function is: The piecewise function is estimated by a continuous approximation function, and the Relu function is represented by a linear piecewise function or estimated by the Softplus function: The Taylor series of the activation function is expressed as: Introduce a nonlinear activation function to generalize the original input features to all series less than or equal to σ, and apply it to the subsequent DNN perceptron; The calculation process of the Boosting-based classifier: Taking the single-node decision maker τ as a decision tree with only one factor, for The decision tree is defined in The linear combination W τ OK, where J τ is the threshold for distinguishing the two categories, and the equivalent decision tree is described as follows: s τ (x a )=-[W τ ·(x a ) T -J τ ] is obtained by combining the samples x a and threshold value; Accordingly, the weight vector is expressed as s τ (x a ) represents the decision result of the decision maker τ, which is expressed in homogeneous coordinates as; The Boosting method selects several classifiers with adaptive weight distribution by learning training examples, and forms a complete classifier by linearly combining the selected classifiers according to the weight distribution; Using the decision tree Instead of classifier, combined with algorithm operation, the final decision tree is described as where α τ is the total error rate of the existing training set; DNN learning is the mapping of Boosting perceptron. In this mapping, the hidden neuron is equivalent to the classifier, the output neuron O is similar to the final decision tree, and the weight of the hidden neuron r is is the weight of the classifier In addition, the weights of the hidden neurons r in the second layer are It corresponds to the total error rate α of the existing training set τ Therefore, the two-stage learning in DNN learning is equivalent to the classifier h t The weights and the total error rate α of the existing training set are calculated τ .

Images