Direction-of-Arrival Estimation Method Based on Tensorized Compressed Neural Network

Abstract

The present invention discloses a method for estimating the direction of arrival based on a tensorized compressed neural network, which mainly solves the problems existing in the existing methods, such as the huge scale of neural network weight parameters, large consumption of computing resources, and long training time. The implementation steps are as follows: tensor modeling of the signals received by a uniform planar array; derivation of a five-dimensional covariance tensor; generation of a state-space tensor based on inverse Tucker decomposition; construction of a tensorized compressed neural network and backpropagation of the loss function of the output layer; accelerated training of the tensorized compressed neural network and output of the direction-of-arrival estimation result. On the premise of retaining the structural characteristics of multi-dimensional received signals, the present invention maps the generation of the state space of the neural network layer to an inverse tensor decomposition process, compresses the large-scale weight parameters to be trained into the factor matrices of the inverse tensor decomposition, effectively accelerates the training process of the neural network, realizes high computational efficiency and low training loss for direction-of-arrival estimation, and can be used for source direction finding in complex environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of array signal processing, and particularly relates to statistical signal processing technology. Specifically, it is a method for estimating the direction of arrival based on a tensorized compressed neural network, which can be used for source direction finding in complex application environments. Background Technique

[0002] In various applications such as radar, sonar, wireless communication, medical imaging, and radio astronomy, direction-of-arrival (DOA) estimation is a core technology for target direction finding, positioning, navigation, and imaging. As the dimensions and scales of sensor arrays deployed in these applications increase day by day, the array received signals cover multi-dimensional complex spatio-temporal characteristics. To retain the structured information of multi-dimensional received signals, tensors, as a multi-dimensional extension form of vectors and matrices, have begun to be applied to the field of array signal processing. By using tensors to model multi-dimensional received signals and performing feature analysis and spatial information extraction on tensor signals, high-precision and high-resolution DOA estimation can be achieved. However, in actual application scenarios, the signal source propagation environment is often relatively complex, resulting in severe working conditions such as low signal-to-noise ratio and low sampling snapshot numbers. Traditional DOA estimation methods based on direct decomposition of tensor statistics perform severely degraded in such severe conditions and have high computational complexity.

[0003] To achieve DOA estimation with low computational complexity and resistance to severe conditions, neural networks have begun to be applied to array signal processing to achieve high-performance and high-efficiency extraction of signal angle information through a data-driven mode. However, for tensor signals and their statistics, existing neural networks need to vectorize them as inputs and use the weight parameter matrices of network layers to perform forward propagation calculations. In this way, the scale of the weight parameter matrices is necessarily proportional to the length of the vectorized signal statistics, and the huge scale of weight parameters causes problems such as high training costs and long training times for deep networks. Therefore, in view of the requirements of tensor signal processing for DOA estimation applications, there is an urgent need to develop a deep network architecture with low training loss and high efficiency. Summary of the Invention

[0004] The purpose of the present invention is to address the problems existing in the existing methods, such as the huge scale of neural network weight parameters, large consumption of computing resources, and long training time, and propose a method for estimating the direction of arrival based on a tensorized compressed neural network, providing a feasible idea and an effective solution for constructing a forward propagation mechanism of a tensorized compressed network layer by using tensor decomposition means to achieve high-efficiency and low-training-loss DOA estimation.

[0005] The purpose of the present invention is achieved through the following technical solutions: A method for estimating the direction of arrival based on a tensorized compressed neural network, the method comprising the following steps:

[0006] (1) The receiving end uses M×N physical antenna elements to construct a uniform planar array. Suppose there are K far-field narrowband uncorrelated signal sources from {(θ1,φ1),(θ2,φ2),…,θ K ,φ K )}, where θ k and φ k are the azimuth angle and elevation angle of the k-th incident signal source respectively, and k = 1, 2, …, K. The T snapshot sampling signals of the uniform planar array are superimposed in the third dimension to obtain a three-dimensional signal tensor which is modeled as:

[0007]

[0008] where, s k = [s k,1 , s k,2 , …, s k,T T is the multi-snapshot sampling signal waveform vector corresponding to the k-th incident signal source, ° represents the vector outer product, is the noise tensor independent of each signal source, a(μ k ) and a(v k ) are the steering vectors of the uniform planar array in the x-axis and y-axis directions respectively, and are expressed as:

[0009]

[0010]

[0011] where, μ k = sin(φ k )cos(θ k ), v k = sin(φ k )sin(θ k ), [·] T represents the transpose operation;

[0012] (2) Calculate the autocorrelation statistic of the three-dimensional signal tensor to obtain the second-order covariance tensor

[0013]

[0014] where, represents the power of the k-th signal source, represents the noise power, represents the four-dimensional identity tensor, <·,·> r ​Denote the tensor contraction operation of two tensors along the r-th dimension, and E[·] denote the operation of taking the mathematical expectation, (·) * denote the conjugate operation; extract the covariance tensor of the real and imaginary parts, and construct a five-dimensional real-valued covariance tensor:

[0015]

[0016] where Re(·) and Im(·) respectively denote the operations of taking the real and imaginary parts of a complex number, denote the tensor stacking operation along the r-th dimension;

[0017] (3) Perform an inverse Tucker decomposition on the five-dimensional real-valued covariance tensor to obtain a five-dimensional state space tensor of size H 1，1 ×H 1,2 ×H 1,3 ×H 1,4 ×H 1,5 corresponding to the first layer of the neural network

[0018]

[0019] where, and are the inverse Tucker factor matrices corresponding to the five dimensions, covering the weight parameters to be trained of the first layer of the neural network, f1(·) is the non-linear activation function corresponding to the first layer of the neural network, × r denote the tensor-matrix product along the r-th dimension;

[0020] (4) Generate the state space tensor of the next layer of the neural network based on the inverse Tucker decomposition of the state space tensor so as to obtain L - 1 five-dimensional state space tensors in sequence

[0021]

[0022] where, is the inverse Tucker factor matrix corresponding to the l-th layer of the neural network, f l (·) is the non-linear activation function corresponding to the l-th layer of the neural network, l = 2, 3, …, L, and thus a tensorized compressed neural network with a depth of L is constructed; use a six-dimensional weight tensor to weight the state space tensor of the L-th layer of the neural network to obtain the direction-of-arrival estimation value of the output layer:

[0023]

[0024] wherein, is the estimated value of the azimuth angle and elevation angle of the k-th signal source, represents the tensor inner product operation of two tensors along the r1, r2, …, r Y dimensions; during the network training process, a real-valued covariance tensor input is regarded as a training sample, and for the ζ-th training sample calculate the loss function between the estimated value of the direction of arrival of the output layer and the true direction of arrival :

[0025]

[0026] wherein, ‖·‖1 and ‖·‖2 respectively represent the 1-norm and 2-norm, and η represents a conversion threshold; derive the gradient of the output layer loss function Ψ with respect to the weight tensor and use it to update the weight tensor corresponding to the ζ-th training sample

[0027]

[0028] wherein, when ζ = 1, is the randomly initialized weight tensor, and α represents the learning rate; similarly, derive the gradient of the output layer loss function Ψ with respect to the state space tensor of the L-th layer neural network and update the inverse Tucker factor matrix corresponding to the ζ-th training sample

[0029]

[0030] wherein is the randomly initialized inverse Tucker factor matrix; thus, the gradient of the output layer loss function Ψ with respect to the state space tensor is backpropagated from the output layer to the input layer, and the corresponding inverse Tucker factor matrix is updated in turn;

[0031] (5) Use G training samples to iteratively update the inverse Tucker factor matrices {P 1,i , P l,i, where \(i = 1, 2, \ldots, 5\) and \(l = 2, 3, \ldots, L\); Input all training samples into the tensorized compressed neural network and complete the update of the inverse Tucker factor matrices as a process of updating the network weight parameters. Repeat the update of the network weight parameters for a set number of rounds to finally complete the accelerated training of the tensorized compressed neural network; Based on the trained tensorized compressed neural network, calculate the direction-of-arrival estimation result in the actual application scenario through low-complexity forward propagation.

[0032] Furthermore, in the covariance tensor derivation described in step (2), in practice, Approximately obtained by calculating the autocorrelation statistic of the three-dimensional signal tensor , that is, the sampling covariance tensor

[0033]

[0034] Furthermore, in step (3), the scale of the weight parameters corresponding to a single-layer neural network no longer needs to be in direct proportion to the signal scale \(2M\) 2 N 2 of the input covariance tensor, but only in direct proportion to the sum of the sizes of the five inverse Tucker factor matrices \(MH\) 1,1 + \(NH\) 1,2 + \(MH\) 1,3 + \(NH\) 1,4 + \(2H\) 1,5 Since \(H\) 1,1 , \(H\) 1,2 , \(H\) 1,3 , \(H\) 1,4 , \(H\) 1,5 < \(M, N\), the weight parameters to be trained are effectively compressed, greatly reducing the training cost.

[0035] The present invention has the following advantages compared with the prior art:

[0036] (1) Based on the tensor modeling and statistical analysis of multi-dimensional received signals, the present invention effectively retains the original structural attributes of multi-dimensional received signals, and by generating the state space tensor, ensures that the neural network can still effectively extract multi-dimensional spatio-temporal features during the forward propagation process, laying a foundation for achieving high-performance direction-of-arrival estimation;

[0037] (2) The present invention constructs the generation process of the state space as an inverse tensor decomposition process, thereby compressing the huge scale of weight parameters into the inverse tensor decomposition factor matrices, and then designs a multi-layer tensorized compressed neural network for direction-of-arrival estimation. Compared with traditional neural networks, it has the significant advantages of low training loss and high computational efficiency. Description of the Drawings

[0038] Figure 1It is the overall flowchart of the present invention.

[0039] Figure 2 It is a schematic diagram of the forward propagation process based on inverse Tucker decomposition designed by the present invention.

[0040] Figure 3 It is a comparison graph of the DOA estimation accuracy performance of the method proposed by the present invention under different signal-to-noise ratio conditions.

[0041] Figure 4 It is a comparison graph of the DOA estimation accuracy performance of the method proposed by the present invention under different numbers of sampling snapshots. Detailed implementation manners

[0042] The technical solution of the present invention will be further described in detail with reference to the accompanying drawings.

[0043] To solve the problems of the large scale of the depth network weight parameters, large consumption of computing resources, and long training time existing in the existing methods, the present invention proposes a method for direction-of-arrival estimation based on a tensorized compressed neural network. On the premise of retaining the structural characteristics of the multi-dimensional received signals, an efficient forward propagation calculation mechanism based on inverse tensor decomposition is established, and the tensorized compressed neural network constructed is used to achieve high-efficiency and low-training-loss direction-of-arrival estimation. Referring to Figure 1 , the implementation steps of the present invention are as follows:

[0044] Step 1: Tensor modeling of the signals received by the uniform planar array. At the receiving end, a uniform planar array is constructed using M×N physical antenna elements, and the element spacing d is taken as half of the wavelength λ of the incident narrowband signal, that is Assume that there are K far-field narrowband uncorrelated signal sources from {(θ1,φ1),(θ2,φ2),…,(θ K ,φ K )}, where θ k and φ k are the azimuth angle and elevation angle of the k-th incident signal source respectively, k = 1, 2,..., K; the T snapshot sampling signals of the uniform planar array are superimposed in the third dimension to obtain a three-dimensional signal tensor Modeled as:

[0045]

[0046] Among them, s k = [s k,1 , s k,2 ,..., s k,T T is the multi-snapshot sampling signal waveform vector corresponding to the k-th incident signal source, represents the vector outer product, ​is a noise tensor independent of each signal source, a(μ k ) and a(v k ) are the steering vectors of the uniform planar array in the x-axis and y-axis directions, respectively, expressed as:

[0047]

[0048]

[0049] where μ k = sin(φ k )cos(θ k ), ν k = sin(φ k )sin(θ k ), denotes the transpose operation;

[0050] Step 2: Five-dimensional covariance tensor derivation. By finding the autocorrelation statistics of the three-dimensional signal tensor , the second-order covariance tensor

[0051]

[0052] is obtained, where represents the power of the k-th signal source, represents the noise power, represents the four-dimensional identity tensor, <·,·> r denotes the tensor contraction operation of two tensors along the r-th dimension, E[·] denotes the mathematical expectation operation, (·) * denotes the conjugate operation; in practice, is approximately obtained by calculating the autocorrelation statistics of the three-dimensional signal tensor , that is, the sample covariance tensor Furthermore, the real and imaginary parts of the covariance tensor are extracted to construct a five-dimensional real-valued covariance tensor:

[0053]

[0054] where Re(·) and Im(·) denote the operations of taking the real and imaginary parts of a complex number, respectively, denotes the tensor stacking operation along the r-th dimension;

[0055] Step 3: State space tensor generation based on inverse Tucker decomposition. To retain the original structure of the signal features, define the size of the first layer of the neural network as H 1,1 ×H 1,2 ×H 1,3 ×H 1,4×H 1,5 Five - dimensional state - space tensor As Figure 2 shown, this state - space tensor is obtained from the inverse Tucker decomposition of a five - dimensional real - valued covariance tensor :

[0056]

[0057] where and are the inverse Tucker factor matrices corresponding to the five dimensions, covering the weight parameters of the first - layer neural network, f1(·) is the non - linear activation function corresponding to the first - layer neural network, × r denotes the tensor - matrix product along the r - th dimension; thus, the scale of the weight parameters corresponding to the first - layer neural network no longer needs to be proportional to the signal scale 2M 2 N 2 of the input covariance tensor, but is only proportional to the sum of the sizes of the five inverse Tucker factor matrices MH 1,1 +NH 1,2 +MH 1,3 +NH 1,4 +2H 1,5 Since H 1,1 , H 1,2 , H 1,3 , H 1,4 , H 1,5 < M, N, the weight parameters to be trained in the network are effectively compressed, greatly reducing the training cost;

[0058] Step 4: Construction of the tensorized compressed neural network and back - propagation of the loss function of the output layer. Similar to the generation process of the state - space tensor of the first - layer neural network, based on the inverse Tucker decomposition of the state - space tensor to generate the state - space tensor of the next - layer neural network, thus obtaining L - 1 five - dimensional state - space tensors in sequence

[0059]

[0060] where is the inverse Tucker factor matrix corresponding to the l - th layer neural network, f l (·) is the non - linear activation function corresponding to the l - th layer neural network, l = 2, 3,..., L, thus constructing a tensorized compressed neural network with depth L; using a six - dimensional weight tensor to weight the state - space tensor of the L - th layer neural network, the direction - of - arrival estimation value of the output layer is obtained:

[0061]

[0062] wherein, is the estimated value of the azimuth angle and elevation angle of the k-th signal source, represents the inner product operation of two tensors along the r1, r2,..., r Y dimensions; during the network training process, a real-valued covariance tensor input is regarded as a training sample, and for the ζ-th training sample ζ≥1, calculate the loss function between the estimated value of the direction of arrival at the output layer and the true direction of arrival :

[0063]

[0064] wherein, ‖·‖1 and ‖·‖2 respectively represent the 1-norm and 2-norm, η represents a conversion threshold for calculating a loss function, and generally η = 1 is set; derive the gradient of the loss function Ψ of the output layer with respect to the weight tensor and use it to update the weight tensor corresponding to the ζ-th training sample

[0065]

[0066] wherein, when ζ = 1, is the randomly initialized weight tensor, and α represents the learning rate; similarly, derive the gradient of the loss function Ψ of the output layer with respect to the state space tensor of the L-th layer neural network and update the inverse Tucker factor matrix corresponding to the ζ-th training sample

[0067]

[0068] wherein, is the randomly initialized inverse Tucker factor matrix; thus, backpropagate the gradient of the loss function Ψ of the output layer with respect to the state space tensor from the output layer to the input layer, and sequentially update the corresponding inverse Tucker factor matrix;

[0069] Step 5: Tensorized compressed neural network accelerated training and direction of arrival estimation result output. Use G training samples to iteratively update the inverse Tucker factor matrices {P 1,i , P l,i , i = 1, 2,..., 5}, l = 2, 3,..., L. Since the inverse Tucker factor matrices {P 1,i , Pl,i , where \(i = 1, 2, \ldots, 5\}, the scale of the weight parameters covered is greatly reduced, and the training process of the tensorized compressed neural network is effectively accelerated; all \(G\) training samples are input into the network and the inverse Tucker factor matrices are updated to complete one round of network weight parameter update process, and the network weight parameter update is repeated 10 rounds, and finally the accelerated training of the tensorized compressed neural network is completed; based on the trained tensorized compressed neural network, the direction-of-arrival estimation result in the actual application scenario is obtained through low-complexity forward propagation calculation.

[0070] The effects of the present invention will be further described below in conjunction with simulation examples.

[0071] Simulation example: A uniform planar array is used to receive incident signals, and its parameters are selected as \(M = 10\), \(N = 10\), that is, the uniform planar array of the architecture contains a total of 100 antenna elements. Compare the training efficiency of the tensorized compressed neural network proposed in the present invention with that of the traditional uncompressed neural network. Set the depth of the neural network to \(L = 3\), and the corresponding state space tensor sizes are \(7\times7\times7\times7\times2\), \(5\times5\times5\times5\times1\), \(3\times3\times3\times3\times1\), and the activation functions are set as \(\tanh(\cdot)\), \(\tanh(\cdot)\) and \(\text{ReLU}(\cdot)\) respectively. Assume that there are 2 incident signals. In the training stage, the network is trained by generating 19,200 samples. The signal azimuth and elevation angles corresponding to each sample are randomly generated within the range of \([0^{\circ}, 90^{\circ}]\), and the signal-to-noise ratio (SNR) is randomly selected from the set \(\{-15\text{dB}, -10\text{dB}, -5\text{dB}, 0\text{dB}\}\); the learning rates of the proposed tensorized compressed neural network and the traditional neural network are \(\alpha = 5.221\times10\) -3 and \(\alpha = 0.273\times10\) -3 . Experiments prove that the training time required for the traditional neural network is 541 minutes, while the proposed tensorized compressed neural network only requires 24 minutes, which greatly saves the training cost.

[0072] Furthermore, the proposed method is compared with the traditional direction-of-arrival estimation method based on rotational invariance, the direction-of-arrival estimation method based on canonical polyadic decomposition, and the direction-of-arrival estimation method based on the uncompressed deep network. The azimuth and elevation angles of 2 incident signal sources are randomly generated within the range of \([0^{\circ}, 90^{\circ}]\). Under the condition of the sampling snapshot number \(T = 200\), the performance comparison curve of the root-mean-square error (RMSE) varying with the SNR is plotted, as shown in Figure 3 ; under the condition of \(SNR = -5\text{dB}\), the performance comparison curve of the RMSE varying with the sampling snapshot number \(T\) is plotted, as shown in Figure 4 ; as shown in Figure 3and Figure 4 From the comparison results, it can be seen that, whether in different signal-to-noise ratio (SNR) scenarios or in different numbers of sampling snapshots (T) scenarios, the method proposed in the present invention can achieve better performance than the rotation-invariant method and has similar estimation accuracy to the canonical polyadic decomposition method. However, due to its low-complexity forward propagation calculation mechanism, the proposed method has higher operating efficiency. At the same time, compared with the traditional uncompressed neural network method, although the estimation accuracy of the proposed method decreases slightly, its training loss is smaller and it is more practical in actual application environments. In summary, the present invention realizes the estimation of the direction of arrival of multiple signal sources with low training loss and high computational efficiency by constructing a tensorized compressed neural network.

[0073] The above description is only the preferred embodiment of the present invention. Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible changes and modifications to the technical solution of the present invention, or modify it into an equivalent embodiment with equivalent changes, without departing from the scope of the technical solution of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the protection of the technical solution of the present invention.

Claims

1. A method for estimating the direction of arrival based on a tensorized compressed neural network, characterized in that, Including the following steps: (1) The receiving end uses M×N physical antenna elements to construct a uniform planar array; assume there are K far-field narrowband uncorrelated signal sources from the directions of {(θ1, φ1), (θ2, φ2), …, (θ K , φ K )}, where θ k and φ k are the azimuth angle and elevation angle of the k-th incident signal source respectively, and k = 1, 2, …, K; Superimpose the T snapshot sampling signals of the uniform planar array in the third dimension to obtain a three-dimensional signal tensor Modeled as: where s k = [s k，1 , s k，2 , …, s k，T T is the multi-snapshot sampling signal waveform vector corresponding to the k-th incident signal source, denotes the vector outer product, is the noise tensor independent of each signal source, a(μ k ) and a(v k ) are the steering vectors of the uniform planar array in the x-axis and y-axis directions, respectively, expressed as:​ where, μ k = sin(φ k )cos(θ k ), v k = sin(φ k )sin(θ k ), [·] T denotes the transpose operation; (2) Calculate the autocorrelation statistics of the three-dimensional signal tensor to obtain the second-order covariance tensor wherein, represents the power of the k-th signal source, represents the noise power, represents the four-dimensional unit tensor, <·,·> r represents the tensor contraction operation of two tensors along the r-th dimension, E[·] represents the mathematical expectation operation, (·) * represents the conjugate operation; extract the real part and imaginary part of the covariance tensor to construct a five-dimensional real-valued covariance tensor: where Re(·) and Im(·) respectively represent the operations of taking the real part and the imaginary part of a complex number, represents the tensor stacking operation along the r-th dimension; (3) Perform inverse Tucker decomposition on the five-dimensional real-valued covariance tensor to obtain a five-dimensional state space tensor 1,1 of size H 1,2 × H 1,3 × H 1,4 × H 1,5 corresponding to the first layer of the neural network Among them, and are the inverse Tucker factor matrices corresponding to five dimensions, covering the weight parameters to be trained in the first-layer neural network. f1(·) is the non-linear activation function corresponding to the first-layer neural network. × r represents the tensor-matrix product along the r-th dimension; (4) Based on the state space tensor Generate the state space tensor of the next layer of the neural network through inverse Tucker decomposition, thereby obtaining L-1 five-dimensional state space tensors in sequence Among them, is the inverse Tucker factor matrix corresponding to the l-th layer neural network, and f l (·) is the non-linear activation function corresponding to the l-th layer neural network, where l = 2, 3, …, L. Thus, a deep tensorized compressed neural network with depth L is constructed; a six-dimensional weight tensor is used to weight the state space tensor of the L-th layer neural network to obtain the estimated value of the direction of arrival of the output layer: Among them, is the estimated value of the azimuth angle and elevation angle of the k-th signal source, represents the inner product operation of two tensors along the r1, r2, …, r Y dimensions; during network training, an input real-valued covariance tensor is regarded as a training sample, and for the ζ-th training sample calculate the loss function between the estimated value of the direction of arrival at the output layer and the true direction of arrival as follows: where, ||·||1 and ||·||2 represent the 1-norm and 2-norm respectively, and η represents a conversion threshold; derive the gradient of the output layer loss function Ψ with respect to the weight tensor and use it to update the weight tensor corresponding to the ζ-th training sample Among them, when ζ = 1, is a randomly initialized weight tensor, and α represents the learning rate; similarly, the gradient of the output layer loss function Ψ with respect to the neural network state space tensor of the L-th layer is derived and the inverse Tucker factor matrix corresponding to the ζ-th training sample is updated. Among them is the randomly initialized inverse Tucker factor matrix; thus, the gradient of the output layer loss function Ψ with respect to the state space tensor is reversely propagated from the output layer to the input layer, and the corresponding inverse Tucker factor matrix is updated in turn; (5) Using G training samples Iteratively update the inverse Tucker factor matrices {P 1，i , P l，i , i = 1, 2, …, 5}, l = 2, 3, …, L; Input all training samples into the tensorized compressed neural network and complete the update of the inverse Tucker factor matrices as a process of updating the network weight parameters for one round. Repeat the network weight parameter update for the set number of rounds to finally complete the accelerated training of the tensorized compressed neural network; Based on the fully trained tensorized compressed neural network, calculate the direction-of-arrival estimation result in the actual application scenario through low-complexity forward propagation.

2. The method for estimating the direction of arrival based on the tensorized compressed neural network according to claim 1, wherein The covariance tensor derivation described in step (2) is, in practice, obtained approximately by calculating the autocorrelation statistic of the three-dimensional signal tensor , that is, the sampling covariance tensor 3. The direction-of-arrival estimation method based on a tensorized compressed neural network according to claim 1, wherein In step (3), the scale of the weight parameters corresponding to the one-layer neural network no longer needs to be proportional to the signal scale 2M of the input covariance tensor 2 N 2 but is only proportional to the sum of the sizes of the five inverse Tucker factor matrices MH 1，1 +NH 1，2 +MH 1，3 +NH 1，4 +2H 1，5 Since H 1，1 ,H 1，2 ,H 1，3 ,H 1，4 ,H 1，5 <M, N, the weight parameters to be trained are effectively compressed, greatly reducing the training cost.

Images