A white-box neural network classifier, training method and computer-readable storage medium

Abstract

A white-box neural network classifier, a training method and a computer readable storage medium, wherein the white-box neural network classifier comprises a seed region perception layer, a topological relationship activation layer and a topological information integration layer connected with each other; the seed region perception layer is used for receiving an input sample and perceiving Mahalanobis distance between the input sample and a seed region, wherein the seed region comprises a plurality of N-dimensional ellipsoids; the topological relationship activation layer is used for mapping the Mahalanobis distance between the input sample and the seed region to a position relationship between the input sample and the seed region, and activating the position relationship between the input sample and the seed region; and the topological information integration layer is used for integrating the position relationship between the input sample and the seed region and mapping the position relationship to a corresponding classification set to generate a predicted classification result. The present application can perform data fitting on any group of input samples satisfying local consistency and corresponding categories, and can make a reliable prediction on the input sample after fitting.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and interpretable deep learning technology, and in particular to a white-box neural network classifier, training method and computer-readable storage medium. Background Technology

[0002] Deep learning technology has made groundbreaking progress in fields such as computer vision and natural language processing. However, traditional neural network models, represented by multilayer perceptrons and convolutional neural networks, suffer from the inherent defect of a black box decision-making process, i.e., the black box problem. Traditional models construct end-to-end mappings by stacking massive numbers of neurons and nonlinear activation functions, but the physical meaning of their internal weight matrices and feature channels is unclear, making it impossible for humans to understand "why" the model makes a certain decision. This poses a serious application obstacle in highly sensitive fields such as aerospace, medical diagnosis, and financial risk control. Furthermore, in the field of natural language processing, the black box characteristic makes neural networks difficult to compress; each compression operation incurs high compression costs, while uncompressed neural networks incur high operating costs.

[0003] To address the "black box" problem of high-performance neural network models, some methods attempt to partially explain the network using pre-trained model parameters combined with statistical knowledge. However, while complex and abstract mathematical methods can partially explain the model's characteristics, they still fail to solve the problem of the network model's opacity and inability to be concretely understood. To date, the black box problem of neural networks remains a pressing issue that needs to be addressed. Summary of the Invention

[0004] This invention provides a white-box neural network classifier, a training method, and a computer-readable storage medium to solve the technical problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows: This invention provides a white-box neural network classifier, comprising: The seed region sensing layer is used to receive input samples and sense the Mahalanobis distance between the input samples and the seed region, wherein the seed region consists of multiple N-dimensional ellipsoids; The topological relationship activation layer, connected to the seed region perception layer, is used to map the Mahalanobis distance between the input sample and the seed region to the positional relationship between the input sample and the seed region, and to activate the positional relationship between the input sample and the seed region. The topology information integration layer, connected to the topology relationship activation layer, is used to integrate the positional relationship between the input sample and the seed region and map the positional relationship to the corresponding classification set to generate the predicted classification result.

[0006] Furthermore, the parameters of the seed region perception layer, the topology relationship activation layer, and the topology information integration layer are optimized and adjusted according to the convergence function, which is the mean square error loss function.

[0007] Furthermore, the input samples have local consistency, where local consistency refers to the statistical features or geometric structures of the input samples having a similarity of up to a set value in their local neighborhood.

[0008] Furthermore, the Mahalanobis distance between the input sample and the seed region is calculated as follows: ; in, This represents the Mahalanobis distance between the input sample and the seed region; Indicates the input sample; M represents The parameter matrix of N includes the geometric center positions of M N-dimensional ellipsoids. ; R represents the curvature matrix of M N-dimensional ellipsoids; The semi-major axis matrix of N, the semi-major axis matrix R is composed of the semi-lengths of M N-dimensional ellipsoids on each axis; , They represent passing through orthogonal matrices respectively. The rotated input sample X and parameter matrix S; Represents matrix multiplication; Let represent an orthogonal matrix that satisfies . , Let be an N-dimensional identity matrix, representing the axis vectors of M N-dimensional ellipsoids; T denotes transpose; the subscript i in the summation symbol indicates the i-th vector in the corresponding parameter matrix, which has a dimension of N.

[0009] Furthermore, the orthogonal matrix The calculation formula is as follows: ; ; in, This means expanding an input matrix of dimension N to a matrix of dimension N. A diagonal matrix of N; The matrix A represents an equilateral triangular decomposition; A represents an N*N dimensional matrix; F represents an N dimensional matrix. Matrix A and matrix F are used to generate the semi-major axis matrix R and the orthogonal matrix. A random matrix whose values ​​are randomly generated; This represents an orthogonal matrix with undefined axis rotation directions; `sign(.)` extracts the sign of the input matrix parameters and constructs a matrix. If the input matrix parameters are positive, then the corresponding parameter for `sign(.)` is... If the input matrix parameter is negative, then the parameter corresponding to sign(.) is... .

[0010] Furthermore, the formula used to activate the positional relationship between the input sample and the seed region is as follows: ; in, This indicates the positional relationship between the activated input sample and the seed region; This indicates the positional relationship between the input sample and the seed region.

[0011] Furthermore, the expression for the predicted classification result is as follows: ; in, This represents the predicted classification result; Weight is an M The network's underlying matrix is ​​C, where C represents the number of specific types; Bias is a vector of dimension C; where... d represents the normalized exponential function. This indicates that the row vectors of the matrix are normalized.

[0012] A second aspect of the present invention also provides a method for training a white-box neural network classifier, comprising the following steps: S1. Obtain multiple input samples and construct a training dataset using the multiple input samples; the input samples in the training dataset have local consistency; S2. Initialize the network parameters of the white-box neural network classifier, where the semi-major axis matrix R uses the curvature matrix. Replace, and on the curvature matrix Perform initialization; S3. Input the training dataset into the initialized white-box neural network classifier and construct the convergence function. The convergence function is the error between the predicted classification result output by the white-box neural network classifier and the target vector. The target vector is the real classification result set by the user. S4. Backpropagate the white-box neural network classifier using the convergence function to update the network parameters of the white-box neural network classifier. S5. Repeat S3 to S4 until the convergence function converges, and obtain the trained white-box neural network classifier.

[0013] Furthermore, the formula for calculating the target vector in S3 is as follows: ; in, Represents the target vector; The characteristics of the target vector are: ; That is, for any class i, the sum of the target vectors is 0, and the sum of the entire matrix of the target vectors is 0; in, Indicates "restricted by"; Indicates the total number of data categories. Represents the target vector The j-th element in the i-th row.

[0014] A third aspect of the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described training method for a white-box neural network classifier.

[0015] The beneficial effects of this invention are: 1. This invention discloses a white-box neural network classifier, comprising a seed region perception layer, a topological relationship activation layer, and a topological information integration layer connected in sequence; the white-box neural network classifier can perform data fitting on any set of input samples that satisfy local consistency and their corresponding categories, and after fitting, it can make a reliable prediction on the input sample.

[0016] 2. In order to solve the black box problem of neural networks, this invention uses the topological relationship (i.e., positional relationship) between the N-dimensional ellipsoid and the input sample as a basis. By analyzing the ellipsoids and input samples contained in the corresponding sets of each category, it is possible to accurately determine which N-dimensional ellipsoid a certain input sample is in and which set this ellipsoid belongs to, etc.

[0017] 3. To solve the network compression problem, this invention discloses a training method for a white-box neural network classifier. This invention can effectively enable the network to converge quickly. The network only needs to activate a very small number of parameters to complete the calculation. Since most of them do not participate in the calculation, the network can be directly compressed to reduce the number of calculations. Attached Figure Description

[0018] Figure 1 This is a structural diagram of the white-box neural network classifier in this invention. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0020] Reference Figure 1This application provides a white-box neural network classifier, including: The seed region sensing layer receives input samples and senses the Mahalanobis distance between the input samples and the seed region, where the seed region comprises multiple N-dimensional ellipsoids. The seed region sensing layer contains the basic parameters required to describe the N-dimensional ellipsoids; the specific parameter values ​​of the seed region sensing layer depend on the input samples. The dimensions are defined, among which, Represents the set of real numbers; The topological relationship activation layer, connected to the seed region perception layer, is used to map the Mahalanobis distance between the input sample and the seed region to the positional relationship between the input sample and the seed region, and to activate the positional relationship between the input sample and the seed region. The topology information integration layer, connected to the topology positional relationship activation layer, integrates the positional relationships between input samples and seed regions and maps these relationships to corresponding classification sets. This yields the parameter visualization results of the topology information integration layer, which then generates predicted classification results based on these visualizations. Specifically, the integration method involves using sign information (positive or negative) to integrate the topology information regarding the positional relationships between the activated input samples and seed regions.

[0021] White-box neural network classifiers classify data by analyzing the positional relationship between input samples and their corresponding ellipsoids. This positional relationship (or topological relationship) includes three possibilities: inside the ellipsoid, on the ellipsoid, and outside the ellipsoid. The seed region perception layer measures the distance between the input sample and the seed region; the topological positional relationship activation layer extracts the topological relationship between the input sample and the seed region; and finally, the topological information integration layer completes the data classification.

[0022] The white-box neural network classifier disclosed in this invention has high accuracy in predicting classification results and improves its classification reliability. At the same time, network compression effectively reduces the operating cost of the model (white-box neural network classifier), making it particularly suitable for large-scale model application scenarios that require high reliability.

[0023] In some embodiments, the parameters of the seed region perception layer, the topological relationship activation layer, and the topological information integration layer are optimized and adjusted according to a convergence function. This convergence function ensures that the white-box neural network classifier can converge to the target output for any input sample with local consistency during training. The mean squared error loss function is selected as the convergence function.

[0024] In some embodiments, the input samples have local consistency, wherein local consistency refers to the statistical features or geometric structures of the input samples having a similarity of up to a set value in their local neighborhood.

[0025] In some embodiments, the Mahalanobis distance between the input sample and the seed region is calculated as follows: ; in, This represents the Mahalanobis distance between the input sample and the seed region; Indicates the input sample; M represents The parameter matrix of N represents the positions of the geometric centers of M N-dimensional ellipsoids; T denotes transpose; Represents matrix multiplication; matrix The following relationship must be satisfied: ; in, Let represent an orthogonal matrix that satisfies . , Let R be an N-dimensional identity matrix, representing the axis vectors of M N-dimensional ellipsoids; R represents M The semi-major axis matrix of N, the semi-major axis matrix R is composed of the semi-lengths of M N-dimensional ellipsoids on each axis; This means expanding an input matrix of dimension N to a matrix of dimension N. A diagonal matrix of N; By changing the formula for calculating the Mahalanobis distance between the input sample and the seed region, it can be modified to: ; Since tensor multiplication in the modified formula requires less space and is faster than dot product in parallel computation, the formula can be optimized as follows: ; To facilitate computer calculations, a curvature matrix is ​​used. Replacing the semi-major axis matrix R, the above calculation formula becomes: ; in, Represents the curvature matrix of M N-dimensional ellipsoids; , They represent passing through orthogonal matrices respectively. The input sample X and parameter matrix S are rotated; the subscript i in the summation symbol represents the i-th vector in the parameter matrix, which has dimension N.

[0026] In some embodiments, the orthogonal matrix The calculation formula is as follows: ; ; in, Represents equilateral triangular decomposition; A represents an N*N dimensional matrix; F represents an N dimensional matrix; matrices A and F serve as the generating semi-major axis matrix R and orthogonal matrix. A random matrix whose values ​​are randomly generated; This represents an orthogonal matrix with undefined axis rotation directions; `sign(.)` extracts the sign of the input matrix parameters and constructs a matrix. If the input matrix parameters are positive, then the corresponding parameter for `sign(.)` is... If the input matrix parameter is negative, then the parameter corresponding to sign(.) is... .

[0027] In some embodiments, the formula used to activate the positional relationship between the input sample and the seed region is as follows: ; in, This indicates the positional relationship between the activated input sample and the seed region; This indicates the positional relationship between the input sample and the seed region.

[0028] In some embodiments, the expression for the predicted classification result is as follows: ; in, This represents the predicted classification result; Weight is an M C is the network's underlying matrix, where C represents the number of specific types; Bias is a dimension C. A general matrix of 1; d This indicates that the row vectors of the matrix are normalized. This represents the normalized exponential function. The actual role matrix in information integration is... The characteristic of this matrix is ​​that, for a given category... The sum of the weights of the seeds contained in this species is 1, and each seed corresponds to an ellipsoidal region (N-dimensional ellipsoid) in the N-dimensional feature space. The matrix represents Within which seed regions does this type of data fall in the N-dimensional feature space?

[0029] A second aspect of the present invention also provides a method for training a white-box neural network classifier, comprising the following steps: S1. Obtain multiple input samples and construct a training dataset using the multiple input samples; the input samples in the training dataset have local consistency; S2. Initialize the network parameters of the white-box neural network classifier, where the semi-major axis matrix R uses the curvature matrix. Replace, and on the curvature matrix Perform initialization; S3. Input the training dataset into the initialized white-box neural network classifier and construct the convergence function. The convergence function is the error between the predicted classification result output by the white-box neural network classifier and the target vector. The target vector is the real classification result set by the user. S4. Backpropagate the white-box neural network classifier using the convergence function to update the network parameters of the white-box neural network classifier. S5. Repeat S3 to S4 until the convergence function converges, and obtain the trained white-box neural network classifier.

[0030] In some embodiments, S1 specifically includes the following steps: S11. Obtain multiple input samples, wherein the input samples have local consistency, that is, there are significant feature differences between input samples of the same type and input samples of different types. For example, suppose in the feature space... Within, for any set of category labels and arbitrary classification functions If any category label The geometric center position of the corresponding N-dimensional ellipsoid ,like Such that for any geometric center position and any input sample ,when There will always be a time for it. If the dataset satisfies local consistency, it can be classified using this classifier. The geometric center location is... This is used to illustrate that the input sample X clusters at this point. It is the feature space The distance metric formula used in the white-box neural network classifier training method is Mahalanobis distance. Indicates "existence"; This represents the distance from the input sample X to the seed region; S12. Construct a dataset using multiple input samples, and then divide the dataset into a training dataset and a test dataset according to a set ratio (e.g., 8:2).

[0031] In some embodiments, the number of network parameters of the white-box neural network classifier depends on the input samples. The dimensions are defined, and the network parameters of the white-box neural network classifier are as follows: ; in, This represents the number of network parameters in a white-box neural network classifier; k represents the number of ellipsoids assigned to each class.

[0032] In some embodiments, S3 specifically includes the following steps: S31. Select a suitable black-box neural network encoder to encode the input samples in the training dataset, mapping the input samples to an N-dimensional feature space. ; S32. Design training parameters. The trainer uses the Adam optimizer. The batch size should not be too large and should be gradually increased from 256. The learning rate adopts the cosine function variation curve and is initially set to 0.01, and is gradually adjusted during the training process. S33. Input the mapped input samples into the initialized white-box neural network classifier and construct a convergence function. The mean squared error loss function is selected as the convergence function.

[0033] In some embodiments, the formula for calculating the target vector in S3 is: ; in, Represents the target vector; The characteristics of the target vector are: ; That is, for any class i, the sum of the target vectors is 0, and the sum of the entire matrix of the target vectors is 0; in, Indicates "restricted by"; Indicates the total number of data categories. This represents the target vector of the i-th category. Represents the target vector The j-th element in the i-th row.

[0034] In some embodiments, the following steps are further included after step S5: S6. Evaluate the performance of the trained white-box neural network classifier. If the evaluation passes, compress the trained white-box neural network classifier. If it fails, return to S3 and continue training.

[0035] In some embodiments, S6 specifically includes the following steps: S61. Extracting the matrix from the white-box neural network classifier Sort the matrix by absolute values ​​and calculate the percentage (rate) of the first m values ​​in each matrix out of the total values. The percentage of rate is then denoted as the Energy Concentration Ratio (ECR). If the ECR > 90%, the model performance is complete and highly interpretable; otherwise, return to S3 and continue training. S62. Compress the parameters of the white-box neural network classifier based on the energy concentration ratio (ECR). For the position of the geometric center of the i-th N-dimensional ellipsoid... If the position In the matrix If the corresponding column does not have a high-energy value, then this position... Delete, iterate through all positions in the parameter matrix S. This completes the compression of the white-box neural network classifier; S63. Feed all training set data into the compressed white-box neural network classifier in sequence for classification prediction. For each category, extract the input sample closest to the center of the seed region in the feature space as the standard prototype of the seed.

[0036] In some embodiments, the following steps are further included after step S6: S7. Use the test dataset to test the white-box neural network classifier before and after compression. Test the accuracy of the white-box neural network classifier before and after compression respectively. If the accuracy does not decrease significantly, it means that the training and compression process of the white-box neural network classifier is successfully completed and the final white-box neural network classifier is obtained; otherwise, return to the corresponding step to retrain or compress.

[0037] In some embodiments, the following steps are further included after S7: S8. Deploy the final white-box neural network classifier to the device, and input samples that satisfy local consistency into the trained white-box neural network classifier on the device to obtain the classification result. The result range is within... .

[0038] If the classification result is positive in a certain dimension, it means that the input sample belongs to the corresponding category in that dimension. The higher the value, the higher the probability of belonging to that category, and an interpretation is made according to the standard prototype of the seed extracted in S6. If a certain category result is in the matrix There are multiple high-energy values. Extract the output of the input sample in the seed region perception layer, find the corresponding seed region that is activated, and output the sample that is closest to the geometric center of the seed region as the prototype of the classification standard.

[0039] If all classification results are negative, it means that the input sample does not belong to any known category. A prediction is given based on the maximum value of the classification result, but this prediction is unreliable and its authenticity is questionable.

[0040] A third aspect of the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described training method for a white-box neural network classifier.

[0041] A fourth aspect of the present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the above-described training method for a white-box neural network classifier.

[0042] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A white-box neural network classifier, characterized in that, include: The seed region sensing layer is used to receive input samples and sense the Mahalanobis distance between the input samples and the seed region, wherein the seed region consists of multiple N-dimensional ellipsoids; The topological relationship activation layer, connected to the seed region perception layer, is used to map the Mahalanobis distance between the input sample and the seed region to the positional relationship between the input sample and the seed region, and to activate the positional relationship between the input sample and the seed region. The topology information integration layer, connected to the topology relationship activation layer, is used to integrate the positional relationship between the input sample and the seed region and map the positional relationship to the corresponding classification set to generate the predicted classification result.

2. The white-box neural network classifier according to claim 1, characterized in that, The parameters of the seed region perception layer, topological relationship activation layer, and topological information integration layer are optimized and adjusted according to the convergence function, which is the mean square error loss function.

3. A white-box neural network classifier according to claim 1, characterized in that, The input samples have local consistency, where local consistency means that the input samples have statistical features or geometric structures with a similarity of up to a set value in their local neighborhood.

4. A white-box neural network classifier according to claim 1, characterized in that, The formula for calculating the Mahalanobis distance between the input sample and the seed region is: ； in, This represents the Mahalanobis distance between the input sample and the seed region; Indicates the input sample; M represents The parameter matrix of N includes the geometric center positions of M N-dimensional ellipsoids. ; R represents the curvature matrix of M N-dimensional ellipsoids; The semi-major axis matrix of N, the semi-major axis matrix R is composed of the semi-lengths of M N-dimensional ellipsoids on each axis; , They represent passing through orthogonal matrices respectively. The rotated input sample X and parameter matrix S; Represents matrix multiplication; Let represent an orthogonal matrix that satisfies . , Let be an N-dimensional identity matrix, representing the axis vectors of M N-dimensional ellipsoids; T denotes transpose; the subscript i in the summation symbol indicates the i-th vector in the corresponding parameter matrix, which has a dimension of N.

5. A white-box neural network classifier according to claim 4, characterized in that, The orthogonal matrix The calculation formula is as follows: ； ； in, This means expanding an input matrix of dimension N to a matrix of dimension N. A diagonal matrix of N; The matrix A represents an equilateral triangular decomposition; A represents an N*N dimensional matrix; F represents an N dimensional matrix. Matrix A and matrix F are used to generate the semi-major axis matrix R and the orthogonal matrix. A random matrix whose values ​​are randomly generated; This represents an orthogonal matrix with undefined axis rotation directions; `sign(.)` extracts the sign of the input matrix parameters and constructs a matrix. If the input matrix parameters are positive, then the corresponding parameter for `sign(.)` is... If the input matrix parameter is negative, then the parameter corresponding to sign(.) is... .

6. A white-box neural network classifier according to claim 5, characterized in that, The formula used to activate the positional relationship between the input sample and the seed region is: ； in, This indicates the positional relationship between the activated input sample and the seed region; This indicates the positional relationship between the input sample and the seed region.

7. A white-box neural network classifier according to claim 6, characterized in that, The expression for the predicted classification result is as follows: ； in, This represents the predicted classification result; Weight is an M The network's underlying matrix is ​​C, where C represents the number of specific types; Bias is a vector of dimension C; where... d represents the normalized exponential function. This indicates that the row vectors of the matrix are normalized.

8. A training method for a white-box neural network classifier according to any one of claims 1 to 7, characterized in that, Includes the following steps: S1. Obtain multiple input samples and construct a training dataset using the multiple input samples; the input samples in the training dataset have local consistency; S2. Initialize the network parameters of the white-box neural network classifier, where the semi-major axis matrix R uses the curvature matrix. Replace, and on the curvature matrix Perform initialization; S3. Input the training dataset into the initialized white-box neural network classifier and construct the convergence function. The convergence function is the error between the predicted classification result output by the white-box neural network classifier and the target vector. The target vector is the real classification result set by the user. S4. Backpropagate the white-box neural network classifier using the convergence function to update the network parameters of the white-box neural network classifier. S5. Repeat S3 to S4 until the convergence function converges, and obtain the trained white-box neural network classifier.

9. The training method for a white-box neural network classifier according to claim 8, characterized in that, The formula for calculating the target vector in S3 is as follows: ； in, Represents the target vector; The characteristics of the target vector are: ； That is, for any class i, the sum of the target vectors is 0, and the sum of the entire matrix of the target vectors is 0; in, Indicates "restricted by"; Indicates the total number of data categories. Represents the target vector The j-th element in the i-th row.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the training method for the white-box neural network classifier as described in claim 8 or 9.

Images