Normalization ReduNet white-box algorithm based on spherical coordinate system

Abstract

The invention discloses a normalized ReduNet white-box algorithm based on a spherical coordinate system, and belongs to the field of ReduNet white-box networks, and the method comprises the steps: obtaining input data with the sample size of m in an n-dimensional space, converting the input data from a Cartesian coordinate system to a spherical coordinate system, and obtaining the input data in the spherical coordinate system; enabling the radial coordinate to be 1, defining the input data under the spherical coordinate system on a unit spherical surface, and obtaining normalized data under the spherical coordinate system; and taking the normalized data in the spherical coordinate system as initial features to be input into the ReduNet white-box network. The method solves the problem that when an existing normalization method is applied to a ReduNet white-box network, updated features of each layer always need to be projected to a unit spherical surface to be normalized.

Description

Technical Field

[0001] The present invention belongs to the field of ReduNet white box networks, and in particular relates to a normalized ReduNet white box algorithm based on a spherical coordinate system. Background Art

[0002] In recent years, deep neural networks have achieved great success in many fields, such as image recognition, speech processing, natural language processing, Go game, etc. The practice of deep networks has captured people's imagination through its empirical success in learning useful representations from such large-scale real-world data. To a large extent, the rise of deep neural networks is attributed to the remarkable technological progress of scalable computing platforms and, equally important, the abundance of real or simulated data for network training and evaluation. However, the practice of deep networks has been shrouded in unexplainability from the beginning, largely because deep network architectures and their components are usually designed based on years of trial and error, trained from random initialization through backpropagation, and deployed as "black boxes". But many of these popular techniques and methods for designing and training deep networks are developed through heuristic and empirical methods, rather than rigorous mathematical principles, modeling, and analysis. In order to unveil the mystery of deep networks, a large number of recent studies on the mathematics of deep learning have greatly improved people's understanding of key parts of deep models, including overfitting and over-parameterization, optimization; however, these efforts are usually based on simplified models, e.g., networks with only a few layers; or rely on simplifying assumptions, e.g., ultra-wide deep networks; or only explain or justify one of the components or features of the network, such as random dropout methods.

[0003] On the other hand, the dominant approach in the above practice is still to construct a black box structure through repeated trials and back propagation. Therefore, the white box neural network ReduNet aims to explain the modern deep convolutional network from the principles of data compression and discriminative representation, and thus derives a basic iterative gradient ascent scheme for optimizing the rate reduction target, which is called the maximum code rate reduction principle MCR2.

[0004] Optimizing this goal will naturally produce a Figure 1 The multi-layer "white box" deep network shown, called ReduNet, has common features of modern deep networks. The deep architecture, linear and nonlinear operators, and even the parameters of the network are explicitly constructed layer by layer through forward propagation, all of which have precise optimization, statistical and geometric interpretations, and the effectiveness of the rate reduction target and white box network has been verified in existing technical solutions.

[0005] By observing the structure of the network, we can see that each layer of the network finally passes through This layer, that is, the updated features of each layer defined always need to be projected onto the unit sphere S n-1 Normalization can be understood as performing a unit norm normalization operation on Z of each layer after each update of new features, so that the dimensions of each layer are consistent. This step is the normalization method for feature scaling in training deep neural networks.

[0006] However, due to the large variance of neurons in neural networks, the model is sensitive to changes in the input distribution, resulting in poor generalization. In addition, as information moves forward along the layer, the variance may be amplified, especially in deep networks, where large variance can also exacerbate the shift of internal covariates. That is, as the parameters of the previous layer change, the distribution of each layer changes during training. The shift of internal covariates slows down training because each layer needs to constantly adapt to the new feature distribution. Traditionally, neural networks use dot products to calculate the pre-activation of neurons. The result of the dot product is unbounded and can be any value in the entire real space, increasing the risk of large variance. Using small weights can alleviate the above problems. Weight decay and maximum normalization are methods that try to make weights smaller. However, when weight decay or maximum normalization is used, the goal (or goal direction) of the original optimization problem changes, and they also bring additional hyperparameters.

[0007] Batch normalization normalizes the pre-activations using statistics computed over a mini-batch of training examples. The normalized values ​​are rescaled and re-shifted using additional parameters. Since batch normalization uses statistics on a mini-batch of examples, its effectiveness depends on the size of the batch. To overcome this, propagated normalization uses a data-independent mean parameter estimate, while layer normalization computes the mean and standard deviation of the same layer in a single training case. Weight normalization reparameterizes the input weight vector by dividing it. It decouples the length of the weight vector from its direction, thereby partially constraining the result of the dot product, but does not take into account the length of the input vector. All of the above methods introduce additional parameters to learn, making the model more complex.

[0008] The normalization operation in the ReduNet white-box network is actually the batch normalization and layer normalization methods mentioned above in the neural network. Since the updated features of each layer always need to be normalized by projecting onto the unit sphere, this raises a new perspective: is there a good normalization method that can speed up the convergence of the neural network while removing the unitization operation of each layer? Or is it possible to use a new network that can be trained by the gradient ascent method in the white-box network but does not rely on any statistics of large or small batches of examples in the above normalization methods?

[0009] Based on this idea, an existing technical solution proposes to use cosine similarity or central cosine similarity represented only by angles to replace dot products in neural networks, which is called cosine normalization. Another existing technical solution proposes a new hyperspherical convolution learning framework and gives an angle representation on the hypersphere. The sphere specification in the hyperspherical convolution network introduced in this way can be used as a new normalization method. Another existing technical solution proposes a spherification layer to represent all angular similarity information. Unlike other projection-based methods, this method ensures that representation learning always occurs on the hypersphere to completely eliminate the interference of normalization operations in training without losing any information. In addition, this method can also be applied to any network by replacing existing layers, verifying the wide applicability and scalability of the spherification layer.

[0010] Converting the data representation based on rectangular coordinates in traditional deep learning methods to data representation in spherical coordinates can expand the application scope of deep learning under different data representations. Since traditional normalization methods may face problems such as numerical instability and gradient vanishing when processing data in spherical coordinates, traditional normalization methods used in white-box neural networks still face challenges in the process of interpreting models. By introducing new normalization methods in spherical coordinates, can it be applied to white-box networks, enhance the interpretability of the network, improve the test accuracy of the network and accelerate the convergence speed? Therefore, it is very meaningful to study the normalization method based on spherical coordinates and its application in white-box neural networks.

[0011] Existing batch normalization must calculate the mean and variance in order to normalize the previous output in the batch. This statistical estimate is relatively accurate if the batch size is large, but the accuracy of the estimate continues to decrease as the batch size decreases. Since each layer needs to be normalized with a small batch of data first, and although the batch normalization method is not computationally intensive and the total number of iterations required for convergence is reduced, the time for each iteration will increase significantly, and it will increase further as the batch size increases.

[0012] Layer normalization requires the calculation of the mean and variance of the activations of each layer, which adds computational overhead. Although these calculations are negligible in most cases, in some environments with limited computing resources, these additional calculations may affect the overall performance.

[0013] Although layer normalization can improve training stability, it may not speed up training as significantly as batch normalization. Batch normalization uses statistics of small batches of data for normalization and usually converges faster. Layer normalization, on the other hand, may lead to relatively slow convergence due to the need to normalize each sample. Summary of the invention

[0014] In view of the above-mentioned deficiencies in the prior art, the present invention provides a normalized ReduNet white-box algorithm based on a spherical coordinate system, which solves the problem that when the existing normalization method is used in the ReduNet white-box network, the updated features of each layer always need to be normalized by projecting onto the unit sphere.

[0015] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a normalized ReduNet white box algorithm based on a spherical coordinate system, comprising the following steps:

[0016] S1. Obtain input data with a sample size of m in n-dimensional space, and convert the input data from a Cartesian coordinate system to a spherical coordinate system to obtain input data in a spherical coordinate system;

[0017] S2, let the radial coordinate be 1, define the input data in the spherical coordinate system on the unit sphere, and obtain the normalized data in the spherical coordinate system;

[0018] S3. Input the normalized data in the spherical coordinate system as the initial features into the ReduNet white box network.

[0019] Furthermore, the expression of the input data in the spherical coordinate system in step S1 is:

[0020]

[0021] Where X is the input data in the spherical coordinate system; x 1 is the 1-dimensional component of the input data in the Cartesian coordinate system; x 2 is the 2D component of the input data in the Cartesian coordinate system; x n-1 is the n-1 dimensional component of the input data in the Cartesian coordinate system; x n is the n-dimensional component of the input data in the Cartesian coordinate system; θ 1 is the angular coordinate corresponding to the 1D component of the input data; θ n-2 is the angular coordinate corresponding to the n-2 dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; r is the radial coordinate.

[0022] Furthermore, the expression of the normalized data in the spherical coordinate system in step S2 is:

[0023]

[0024] Among them, θ is the normalized data in the spherical coordinate system; θ 1 is the angular coordinate corresponding to the 1D component of the input data; θ 2 is the angular coordinate corresponding to the 2D component of the input data; θ 3 is the angular coordinate corresponding to the 3D component of the input data; θn-2 is the angular coordinate corresponding to the n-2 dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; x 1 is the 1-dimensional component of the input data in the Cartesian coordinate system; x 2 is the 2D component of the input data in the Cartesian coordinate system; x 3 is the 3D component of the input data in the Cartesian coordinate system; x n-1 is the n-1 dimensional component of the input data in the Cartesian coordinate system.

[0025] Furthermore, in step S3, the gradient of the maximum bit rate problem solved by the gradient ascent method in the ReduNet white box network is:

[0026]

[0027]

[0028] Where R is the coding rate; is the symbol of partial derivative; θ is the normalized data in spherical coordinate system; θ 1 is the normalized data of the first sample in the spherical coordinate system; θ 2 is the normalized data of the second sample in the spherical coordinate system; θ m is the normalized data of the mth sample in the spherical coordinate system; m is the total number of samples; Z 1 is the feature of the first sample; Z 2 is the characteristic of the second sample; Z m is the feature of the mth sample; G 1 is the transpose of the derivative matrix of the first sample; G 2 is the transpose of the derivative matrix of the second sample; G m is the transpose of the derivative matrix of the mth sample; T is the transpose; G i is the transpose of the derivative matrix of the i-th sample; J i is the derivative matrix of the i-th sample; Z i is the feature of the i-th sample; θ i is the normalized data of the i-th sample in the spherical coordinate system; θ 1 is the angular coordinate corresponding to the 1D component of the input data; θ 2 is the angular coordinate corresponding to the 2D component of the input data; θ 3 is the angular coordinate corresponding to the 3D component of the input data; θ p is the angular coordinate corresponding to the p-dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; n is the spatial dimension; i is the sample number.

[0029] Furthermore, the gradient ascent formula of the ReduNet white box network in step S3 is:

[0030]

[0031] Among them, θ l+1 is the angular coordinate of the l+1th layer network; θ l is the angular coordinate of the lth layer network; η is the learning rate; R is the coding rate; is the symbol of partial derivative; θ is the normalized data in spherical coordinate system; For the case of the l-th layer network; Z 1 is the feature of the first sample; Z 2 is the characteristic of the second sample; Z m is the feature of the mth sample; G 1 is the transpose of the derivative matrix of the first sample; G 2 is the transpose of the derivative matrix of the second sample; G m is the transpose of the derivative matrix of the mth sample.

[0032] The beneficial effects of the present invention are as follows: based on a new normalization method in a spherical coordinate system, the distribution characteristics of the spherical coordinate system data are fully utilized, and the data distribution is compressed on the unit sphere to improve the numerical stability and convergence speed of the model. The newly designed normalization method in a spherical coordinate system is embedded in a white-box neural network, which improves the interpretability, test accuracy and convergence speed of the network under different data representations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the ReduNet white box network in the background technology of the present invention.

[0034] Figure 2 The figure is a flow chart of the method of the present invention.

[0035] Figure 3 This is a graph showing the loss function of the spherical coordinate normalized network iterating with the number of layers in an embodiment of the present invention.

[0036] Figure 4 It is a curve graph of Theta-ReduNet and ReduNet network iteration with the number of layers in an embodiment of the present invention.

[0037] Figure 5 This is a graph showing the KNN classification accuracy results of the network in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0039] like Figure 2 As shown, in one embodiment of the present invention, a normalized ReduNet white box algorithm based on a spherical coordinate system includes the following steps:

[0040] S1. Obtain input data with a sample size of m in n-dimensional space, and convert the input data from a Cartesian coordinate system to a spherical coordinate system to obtain input data in a spherical coordinate system;

[0041] S2, let the radial coordinate be 1, define the input data in the spherical coordinate system on the unit sphere, and obtain the normalized data in the spherical coordinate system;

[0042] S3. Input the normalized data in the spherical coordinate system as the initial features into the ReduNet white box network.

[0043] The expression of the input data in the spherical coordinate system in step S1 is:

[0044]

[0045] Where X is the input data in the spherical coordinate system; x 1 is the 1-dimensional component of the input data in the Cartesian coordinate system; x 2 is the 2D component of the input data in the Cartesian coordinate system; x n-1 is the n-1 dimensional component of the input data in the Cartesian coordinate system; x n is the n-dimensional component of the input data in the Cartesian coordinate system; θ 1 is the angular coordinate corresponding to the 1D component of the input data; θ n-2 is the angular coordinate corresponding to the n-2 dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; r is the radial coordinate.

[0046] In this embodiment, in most neural networks, all input samples are represented as Cartesian coordinates in n-dimensional space. The pre-activation from the neuron is used to determine whether the neuron should be activated by using the following activation function. In this process, the calculation of the pre-activation uses weights and bias parameters, represented as Cartesian coordinates. These Cartesian coordinates can be converted from spherical coordinates. Given a vector x represented as Cartesian coordinates in n-dimensional space, x can be defined as spherical coordinates θ = [r, θ], which consists of a radial coordinate r and n-1 angular coordinates θ = [θ 1 ,θ 2 ,...θ n-1 ]. In this case, the kth sample of x can be calculated from the formula:

[0047]

[0048] In general, spherical coordinates are a 3D version of polar coordinates. However, the spherical coordinates in the above equation represent all n-dimensional versions of polar coordinates, where n is greater than or equal to 2.

[0049] The expression of the normalized data in the spherical coordinate system in step S2 is:

[0050]

[0051] Among them, θ is the normalized data in the spherical coordinate system; θ 1 is the angular coordinate corresponding to the 1D component of the input data; θ 2 is the angular coordinate corresponding to the 2D component of the input data; θ 3 is the angular coordinate corresponding to the 3D component of the input data; θ n-2 is the angular coordinate corresponding to the n-2 dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; x 1 is the 1-dimensional component of the input data in the Cartesian coordinate system; x 2 is the 2D component of the input data in the Cartesian coordinate system; x 3 is the 3D component of the input data in the Cartesian coordinate system; x n-1 is the n-1 dimensional component of the input data in the Cartesian coordinate system.

[0052] In this embodiment, a popular assumption for high-dimensional classification problems is that each class has a relatively low-dimensional intrinsic structure. ReduNet aims to learn low-dimensional intrinsic representations of data by extracting features from samples of different classes that belong to different linear subspaces, while samples from the same class are highly correlated and belong to the same linear subspace. The objective function is constructed using the rate-distortion function as a building block. Here, the application of spherical normalization in white-box networks is named Theta-ReduNet. From the spherical coordinate system, since each layer of the previous ReduNet network is always normalized by projecting onto the unit sphere, we define the radial coordinate r=1 in the transformation between the spherical coordinate system and the Cartesian coordinate system of data in n dimensions.

[0053] set up is the input data matrix with sample size m, where each is the i-th n-dimensional sample, belonging to one of the k classes, and the corresponding spherical coordinate data matrix is For a given precision ∈>0, Theta-ReduNet first converts the input variable from the rectangular coordinate system to the spherical coordinate system through the conversion between the spherical coordinate system and the Cartesian coordinate system of the data in N dimensions, and calculates it using the inverse formula of the calculation formula of the variance of the input data in layer normalization, and sets r=1, and the value of the initialization θ is obtained as:

[0054]

[0055] In this embodiment, through the above data preprocessing, the original first step preprocessing input data of ReduNet is converted into the initial value of the feature Z in the rectangular coordinates, and the current input data is converted into the initial value of the feature Z in the spherical coordinates. By defining the radius r = 1, the data is always defined on the unit sphere, which can omit the normalization operation after each step of feature iteration.

[0056] In step S3, the gradient of the maximum bit rate problem solved by the gradient ascent method in the ReduNet white box network is:

[0057]

[0058]

[0059] Where R is the coding rate; is the symbol of partial derivative; θ is the normalized data in spherical coordinate system; θ 1 is the normalized data of the first sample in the spherical coordinate system; θ 2 is the normalized data of the second sample in the spherical coordinate system; θ m is the normalized data of the mth sample in the spherical coordinate system; m is the total number of samples; Z 1is the feature of the first sample; Z 2 is the characteristic of the second sample; Z m is the feature of the mth sample; G 1 is the transpose of the derivative matrix of the first sample; G 2 is the transpose of the derivative matrix of the second sample; G m is the transpose of the derivative matrix of the mth sample; T is the transpose; G i is the transpose of the derivative matrix of the i-th sample; J i is the derivative matrix of the i-th sample; Z i is the feature of the i-th sample; θ i is the normalized data of the i-th sample in the spherical coordinate system; θ 1 is the angular coordinate corresponding to the 1D component of the input data; θ 2 is the angular coordinate corresponding to the 2D component of the input data; θ 3 is the angular coordinate corresponding to the 3D component of the input data; θ p is the angular coordinate corresponding to the p-dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; n is the spatial dimension; i is the sample number.

[0060] In this embodiment, the transformation under the new spherical coordinate system is applied to the white-box network, and the data composition distribution of the white-box network and the target iteration algorithm are re-inferred: after the original Cartesian coordinate system is transformed into the spherical coordinate system, the characteristics and iteration variables of the original network are Z = [Z 1 ,Z 2 ,...Z m ] becomes θ=[θ 1 ,θ 2 ,...θ m ], and at the same time, the gradient objective of the maximum coding rate problem (MCR2) solved by the gradient ascent method in the white-box network is given by Became At this time, the chain rule of the gradient can be verified to obtain the formula:

[0061]

[0062] At the same time, since the Z matrix is ​​composed of the θ matrix, its partial derivative with respect to the θ matrix can be easily obtained, and the transpose of the derivative matrix J is recorded as the G matrix, that is, the formula is:

[0063]

[0064] At this point, the gradient target of solving the maximum code rate problem (MCR2) using the gradient ascent method in the white box network becomes the following formula:

[0065]

[0066] The gradient ascent formula of the ReduNet white box network in step S3 is:

[0067]

[0068] Among them, θ l+1 is the angular coordinate of the l+1th layer network; θ l is the angular coordinate of the lth layer network; η is the learning rate; R is the coding rate; is the symbol of partial derivative; θ is the normalized data in spherical coordinate system; For the case of the l-th layer network; Z 1 is the feature of the first sample; Z 2 is the characteristic of the second sample; Z m is the feature of the mth sample; G 1 is the transpose of the derivative matrix of the first sample; G 2 is the transpose of the derivative matrix of the second sample; G m is the transpose of the derivative matrix of the mth sample.

[0069] In this embodiment, the feature iteration update step using the gradient ascent method in the white box network becomes the following formula. Therefore, the network represents different data by using only angles, and the iterative algorithm of the network target also achieves the optimal solution only by optimizing the angle. The optimization formula of the entire network is:

[0070]

[0071] This is the known formula of the original ReduNet network. Therefore, the original feature Z that needs to be updated and iterated is changed to the update angle θ, and the gradient ascent formula is also changed to:

[0072]

[0073] From the above formula, we can see that the normalization method in the spherical coordinate system can eliminate the normalization operation to the unit sphere after each layer iteration. At the same time, the neural network can reduce the complexity and speed of iteration by expressing all representations in different ways using only angles.

[0074] Simulation test results:

[0075] In the simulation experiment, 7000 samples of the MNIST training set are used to train the original network and the angle normalized network, and 3000 samples of the test set are used to evaluate the generalization error. All samples are preprocessed with dimension reduction n=87 and maximum normalization, and the parameters are selected as η=0.5, λ=500, ∈=0.2, δ=10-8. The experimental results are shown in Figure 3 shown.

[0076] like Figure 4 As shown in the figure, the simulation experiment results show the loss curve of the modified network. It can be seen that in the final MCR2 target, the loss increases with the increase of layers and decreases with the increase of layers. The target achieves the maximum result and reaches the convergence effect of the experiment at about 800 layers, which is in line with the target expectation.

[0077] The simulation experiment results show the comparison curves of Theta-ReduNet and ReduNet networks with the iteration of the number of layers. It can be seen that under the same parameter settings, Theta-ReduNet can improve the numerical stability and convergence speed of the model and reach convergence faster. Table 1 is a schematic comparison of the image classification accuracy of Theta-ReduNet and ReduNet networks.

[0078] Table 1

[0079]

[0080] like Figure 5 As shown in Figure 1, the simulation experiment results compare the classification accuracy of four different networks. The features extracted in Algorithm 1 can be used to determine which class the sample belongs to. This requires a suitable classifier, which can be a support vector machine, K-nearest neighbor or nearest subspace. Here, the test accuracy of four networks is compared using the KNN classifier, namely the original ReduNet network, the AR-ReduNet network with binary optimization, the Theta-ReduNet network modified from the original ReduNet network, and the Theta-ReduNet network modified from the AR-ReduNet network. By comparing the classification accuracy evaluation indicators of the four groups of experiments, it can be observed that the accuracy of the original ReduNet network modified after using the spherical coordinate system is significantly improved compared to that before modification. For the fixed distortion ∈=0.2, the Theta-ReduNet network is better than the original network for all layers from L=100 to 2000, and can achieve an accuracy similar to that of AR-ReduNet. In addition, using the spherical coordinate system normalization method on the more accurate AR-ReduNet can more effectively improve the effect. The experimental results reveal that the spherical coordinate system normalization simplifies the original network and is more suitable for the intrinsic structure of the data in some aspects, reducing some noise characteristics of the data, thereby improving the final classification performance of the classifier.

[0081] Analysis of simulation results: According to the experimental results, the representation of the spherical coordinate system simplifies the normalization of the original network and improves the classification accuracy. The following possible reasons are analyzed:

[0082] (1) Feature extraction: The distribution of data in the spherical coordinate system is easier to separate. Data that is originally nonlinearly separable in the rectangular coordinate system may show linear separability in the spherical coordinate system. It may be more consistent with the intrinsic structure of the data, making it easier for the model to learn the data pattern. At the same time, the spherical coordinate system may reduce the redundant information in the data. In the rectangular coordinate system, each component of the normalized vector Z is independent, but in high-dimensional space, there may be redundant information between these components. By converting to spherical coordinates, these points are actually represented in a more compact way, because there is a dependency between the angular components in the spherical coordinates. This representation reduces the redundant information that the model needs to learn, thereby improving the classification accuracy. Spherical coordinate transformation can convert linearly inseparable features in the original data into linearly separable features. In some application scenarios, the distribution of the original data in the rectangular coordinate system may be more complex, making it difficult to effectively classify it through a linear classifier. Through spherical coordinate transformation, these complex distributed data can be mapped to the spherical coordinate system, making the data distribution simpler and easier to classify.

[0083] (2) Data distribution: Data is more evenly distributed in spherical coordinates, which may make some classification algorithms more effective. Spherical coordinates may be more suitable for the intrinsic structure of the data, making the data easier to distinguish in the new feature space. In the spherical coordinate system, the data may be more concentrated near the sphere, which makes it easier for the classifier to capture the local features of the data and improve classification accuracy. In some cases, the intrinsic structure of the data may be closer to spherical coordinates than to rectangular coordinates. For example, if the data points are distributed on or near the unit sphere, then using spherical coordinates may be more natural to represent the data. In this case, the spherical coordinate transformation may make it easier for the model to learn the intrinsic pattern of the data.

Claims

1. A normalized ReduNet white-box algorithm based on a spherical coordinate system, characterized in that: The following steps are involved: S1. Obtain input data with a sample size of m in n-dimensional space, and convert the input data from a Cartesian coordinate system to a spherical coordinate system to obtain input data in a spherical coordinate system; S2, let the radial coordinate be 1, define the input data in the spherical coordinate system on the unit sphere, and obtain the normalized data in the spherical coordinate system; S3. Input the normalized data in the spherical coordinate system as the initial features into the ReduNet white box network.

2. The ReduNet white-box algorithm based on normalization in a spherical coordinate system according to claim 1, characterized in that: The expression of the input data in the spherical coordinate system in step S1 is: Where X is the input data in the spherical coordinate system; x1 is the 1D component of the input data in the Cartesian coordinate system; x2 is the 2D component of the input data in the Cartesian coordinate system; x n-1 is the n-1 dimensional component of the input data in the Cartesian coordinate system; x n is the n-dimensional component of the input data in the Cartesian coordinate system; θ1 is the angular coordinate corresponding to the 1-dimensional component of the input data; θ n-2 is the angular coordinate corresponding to the n-2 dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; r is the radial coordinate.

3. The ReduNet white-box algorithm based on normalization in a spherical coordinate system according to claim 1, characterized in that: The expression of the normalized data in the spherical coordinate system in step S2 is: Among them, θ is the normalized data in the spherical coordinate system; θ1 is the angular coordinate corresponding to the 1-dimensional component of the input data; θ2 is the angular coordinate corresponding to the 2-dimensional component of the input data; θ3 is the angular coordinate corresponding to the 3-dimensional component of the input data; θ n-2 is the angular coordinate corresponding to the n-2 dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1-dimensional component of the input data; x1 is the 1-dimensional component of the input data in the Cartesian coordinate system; x2 is the 2-dimensional component of the input data in the Cartesian coordinate system; x3 is the 3-dimensional component of the input data in the Cartesian coordinate system; x n-1 It is the n-1 dimensional component of the input data in the Cartesian coordinate system.

4. The ReduNet white-box algorithm based on normalization in a spherical coordinate system according to claim 1, characterized in that: In step S3, the gradient of the maximum bit rate problem solved by the gradient ascent method in the ReduNet white box network is: Where R is the coding rate; is the symbol of partial derivative; θ is the normalized data in spherical coordinate system; θ 1 is the normalized data of the first sample in the spherical coordinate system; θ 2 is the normalized data of the second sample in the spherical coordinate system; θ m is the normalized data of the mth sample in the spherical coordinate system; m is the total number of samples; Z 1 is the feature of the first sample; Z 2 is the characteristic of the second sample; Z m is the feature of the mth sample; G 1 is the transpose of the derivative matrix of the first sample; G 2 is the transpose of the derivative matrix of the second sample; G m is the transpose of the derivative matrix of the mth sample; T is the transpose; G i is the transpose of the derivative matrix of the i-th sample; J i is the derivative matrix of the i-th sample; Z i is the feature of the i-th sample; θ i is the normalized data of the spherical coordinate system of the i-th sample; θ1 is the angular coordinate corresponding to the 1-dimensional component of the input data; θ2 is the angular coordinate corresponding to the 2-dimensional component of the input data; θ3 is the angular coordinate corresponding to the 3-dimensional component of the input data; θ p is the angular coordinate corresponding to the p-dimensional component of the input data; θ n-1 is the angular coordinate corresponding to the n-1 dimensional component of the input data; n is the spatial dimension; i is the sample number.

5. The ReduNet white-box algorithm based on normalization in a spherical coordinate system according to claim 1, characterized in that: The gradient ascent formula of the ReduNet white box network in step S3 is: Among them, θ l+1 is the angular coordinate of the l+1th layer network; θ l is the angular coordinate of the lth layer network; η is the learning rate; R is the coding rate; is the symbol of partial derivative; θ is the normalized data in spherical coordinate system; For the case of the l-th layer network; Z 1 is the feature of the first sample; Z 2 is the characteristic of the second sample; Z m is the feature of the mth sample; G 1 is the transpose of the derivative matrix of the first sample; G 2 is the transpose of the derivative matrix of the second sample; G m is the transpose of the derivative matrix of the mth sample.

Images