Data classification method for improving neural network based on partition pseudo-inverse method

Abstract

The invention discloses a data classification method for improving a neural network based on a partition pseudo-inverse method, which belongs to the field of data classification and comprises the following steps of: initializing a training data set and a data set needing to be classified; constructing a Bernstein polynomial neural network for data classification and orderly arranging hidden layer neurons; solving hidden layer parameters of the Bernstein polynomial neural network by using a training data set and a division pseudo-inverse method to obtain the Bernstein polynomial neural network with negatively correlated hidden layer neuron weight and response frequency; and inputting a data set needing to be classified into the improved Bernstein polynomial neural network to obtain a data classification result. The Bernstein neural network hidden layer neuron weight is negatively correlated with the response frequency based on the division pseudo-inverse method, the generalization of the neural network in the data classification technology can be improved, and the method is effectively applied to the data classification task.

Description

Technical Field

[0001] The present invention relates to the field of data classification, and in particular to a data classification method based on a partitioning pseudo-inverse method to improve a neural network. Background Art

[0002] Data classification is a complex task that involves processing large amounts of data and analyzing multiple influencing factors. When processing and analyzing data, special attention should be paid to its nonlinear characteristics and high dimensionality. Neural networks excel in the field of data classification. The performance of a neural network is not only related to its approximation ability, but also the generalization ability of its model. Highly correlated to both is the number of neurons in the neural network. When the number of neurons is too small, the approximation performance of the neural network will drop significantly, while when the number of neurons is too large, overfitting problems often occur.

[0003] Bernstein polynomial neural network (BPolyNN) is a neural network that can be applied to data classification. It adopts a multiple cross-validation method on a validation set and a method of gradually growing neurons to adaptively determine the optimal number of neurons. Although Bernstein polynomial neural network performs well in data classification tasks, it still has the limitation of insufficient generalization. In order to solve the problem of insufficient generalization, the introduction of the partition pseudo-inverse method makes the hidden layer neuron weights of the Bernstein neural network negatively correlated with the response frequency, thereby improving the generalization of the neural network model and effectively applying it to data classification tasks. The present invention proposes a classification technology based on the partition pseudo-inverse method to improve the neural network, which can improve the generalization of the neural network in the classification technology and effectively apply it to data classification tasks. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a data classification method for improving a neural network based on a partitioning pseudo-inverse method.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A data classification method based on partition pseudo-inverse method to improve neural network includes the following steps:

[0007] S1. Initialize the training data set and the data set to be classified;

[0008] S2. Construct a Bernstein polynomial neural network for data classification and arrange the hidden layer neurons in an orderly manner;

[0009] S3, using the training data set and the partitioned pseudo-inverse method to solve the hidden layer parameters of the Bernstein polynomial neural network, and obtaining a Bernstein polynomial neural network in which the hidden layer neuron weights are negatively correlated with the response frequency;

[0010] S4. Input the data set to be classified into the Bernstein polynomial neural network obtained in S3 to obtain the data classification result.

[0011] Furthermore, the Bernstein polynomial neural network in S2 is expressed as:

[0012]

[0013] In the formula, Y is the data classification result vector output by the network; B represents the input of the last layer of the network; represents the weight of the neural network; is the number of output classification attributes; is the number of classification data samples; is the total number of neurons in the hidden layer; X is the classification data input; s r (x) represents the output of the rth hidden layer neuron, w r is the weight of the rth hidden layer neuron; is the Chebyshev polynomial, represents the coefficients of Chebyshev polynomials, For the number of items, Indicates The highest order of the Vychebyshev polynomial.

[0014] Furthermore, the S3 specifically includes the following steps:

[0015] S31, training the Bernstein polynomial neural network using the training data set data;

[0016] S32. Use the partitioned pseudo-inverse method to calculate the weights of the Bernstein polynomial neural network, so that the weights of the hidden layer neurons with high frequency response are small, and the weights of the hidden layer neurons with low frequency response are large.

[0017] Furthermore, the specific method of training the Bernstein polynomial neural network in S31 is:

[0018]

[0019] Where Γ is the label of the input training data, is the neural network weight, and ||·||2 is the calculation of the second norm.

[0020] Furthermore, the weight of the Bernstein polynomial neural network in S32 is expressed as:

[0021]

[0022] In the formula, is the weight of the i-th partition label subset; Γ i is the label of the i-th input data; I represents the unit matrix; D i is the ridge regression diagonal matrix; is the matrix transpose symbol.

[0023] Furthermore, the data classification process in S4 is expressed as:

[0024]

[0025] In the formula, X' is the input data to be classified, and Y' is the output of data classification result. is the neural network weight.

[0026] The present invention has the following beneficial effects:

[0027] In the previous data classification technology based on neural network, the weights corresponding to the neurons with high frequency response in the hidden layer of the neural network may be relatively large, which is not conducive to the generalization performance of the neural network. In addition, in the existing data classification technology based on neural network, too many neurons in the neural network usually lead to overfitting, which in turn leads to the problem of insufficient generalization. The present invention makes the weights of the neurons in the hidden layer of the Bernstein neural network negatively correlated with the response frequency based on the partition pseudo-inverse method, which can improve the generalization of the neural network in the data classification technology and be effectively applied to data classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of implementing data classification in the present invention. DETAILED DESCRIPTION

[0029] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0030] A data classification method based on partition pseudo-inverse method to improve neural network, such as Figure 1 As shown, the following steps are included:

[0031] S1. Initialize the training data set and the data set to be classified;

[0032] Read data from the file and normalize it to the [0,1] interval. Fill the missing attribute values ​​of incomplete attribute samples with the mean of the corresponding attribute. Divide the dataset into a training set and a test set (60% for training and 40% for testing). The training set is used to train the model, and the test set is used to evaluate the final performance of the model.

[0033] S2. Construct a Bernstein polynomial neural network for data classification and arrange the hidden layer neurons in an orderly manner;

[0034] The Bernstein polynomial neural network can be defined as

[0035]

[0036] In the formula, Y is the data classification result vector output by the network; B represents the input of the last layer of the network; represents the weight of the neural network; is the number of output classification attributes; is the number of classification data samples; is the total number of neurons in the hidden layer; X is the classification data input; s r (x) represents the output of the rth hidden layer neuron, w r is the weight of the rth hidden layer neuron, is the Chebyshev polynomial; represents the coefficients of Chebyshev polynomials, For the number of items, Indicates The highest order of the Vyssey Chebyshev polynomial; the Chebyshev polynomial is defined as follows:

[0037]

[0038] Further sorting the pairs by hierarchical dictionary The elements are arranged so that the hidden layer neurons are arranged in order according to the polynomial order. Consider the outputs of two different hidden layer neurons: and Further construct two vectors according to the subscripts: and Then there are and If one of the following two conditions is met, then g <h:

[0039] 1)||β||1=||γ||1 and β i -γ i <0, where β i -γ i yes The first non-zero element of ;

[0040] 2)||β||1>||γ||1.

[0041] Thus, the hidden layer neurons can be arranged in an orderly manner

[0042] S3, using the training data set and the partitioned pseudo-inverse method to solve the hidden layer parameters of the Bernstein polynomial neural network, and obtaining a Bernstein polynomial neural network in which the hidden layer neuron weights are negatively correlated with the response frequency;

[0043] In this embodiment, the following steps are specifically included:

[0044] S31. Use the training data set to train the Bernstein polynomial neural network. The training method is:

[0045]

[0046] Where Γ is the label of the input training data, is the neural network weight, ||·||2 is the two-norm calculation

[0047] S32. Use the partitioned pseudo-inverse method to calculate the weights of the Bernstein polynomial neural network, so that the weights of the hidden layer neurons with high frequency response are small, and the weights of the hidden layer neurons with low frequency response are large.

[0048] The partitioned pseudo-inverse method solves the weights of the neural network by the following formula Consider the weight calculation for dividing the training dataset label subsets:

[0049]

[0050] Where z represents the number of divided label subsets;

[0051]

[0052]

[0053] is a diagonal matrix that satisfies for any i>j>0 T represents the transpose operation; I represents the identity matrix.

[0054] S4. Input the data set to be classified into the Bernstein polynomial neural network obtained in S3 to obtain the data classification result.

[0055] The data classification process is as follows:

[0056]

[0057] Where X ′ Indicates the data input that needs to be classified, Y ′ Represents the data classification result output.

[0058] In this embodiment, the samples come from a medical dispute data set provided by a hospital. The data set consists of medical dispute data collected by different departments of the hospital. The data set provides 5,800 samples with disputes and 1,449 samples without disputes. After randomly shuffling the order, 60% of the data set was used as a training set and 40% was used as a test set. As shown in Table 1, on the provided medical dispute data set, the present invention achieved a test accuracy and a test accuracy rate second only to ensemble learning, and the neural network structure achieved a certain generalization improvement. These results show that the present invention can effectively perform data classification, and provide strong support for future applications in similar data classification tasks.

[0059] Table 1

[0060]

[0061]

[0062] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0063] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0064] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0065] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0066] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A data classification method based on partition pseudo-inverse method to improve neural network, characterized in that: The steps include: S1. Initialize the training data set and the data set to be classified; S2. Construct a Bernstein polynomial neural network for data classification and arrange the hidden layer neurons in an orderly manner; S3, using the training data set and the partitioned pseudo-inverse method to solve the hidden layer parameters of the Bernstein polynomial neural network, and obtaining a Bernstein polynomial neural network in which the hidden layer neuron weights are negatively correlated with the response frequency; S4. Input the data set to be classified into the Bernstein polynomial neural network obtained in S3 to obtain the data classification result.

2. The data classification method based on the partition pseudo-inverse method to improve the neural network according to claim 1 is characterized in that: The Bernstein polynomial neural network in S2 is expressed as: In the formula, Y is the data classification result vector output by the network; B represents the input of the last layer of the network; represents the weight of the neural network; is the number of output classification attributes; is the number of classification data samples; is the total number of neurons in the hidden layer; X is the classification data input; s r (x) represents the output of the rth hidden layer neuron, w r is the weight of the rth hidden layer neuron; is the Chebyshev polynomial, represents the coefficients of Chebyshev polynomials, For the number of items, Indicates The highest order of the Vychebyshev polynomial.

3. The data classification method based on the partition pseudo-inverse method to improve the neural network according to claim 1 is characterized in that: The S3 specifically includes the following steps: S31, training the Bernstein polynomial neural network using the training data set data; S32. Use the partitioned pseudo-inverse method to calculate the weights of the Bernstein polynomial neural network, so that the weights of the hidden layer neurons with high frequency response are small, and the weights of the hidden layer neurons with low frequency response are large.

4. The data classification method based on the partition pseudo-inverse method to improve the neural network according to claim 1 is characterized in that: The specific method of training the Bernstein polynomial neural network in S31 is: Where Γ is the label of the input training data, is the neural network weight, and ||·||2 is the calculation of the second norm.

5. The data classification method based on partition pseudo-inverse method to improve neural network according to claim 1 is characterized in that: The weight of the Bernstein polynomial neural network in S32 is expressed as: In the formula, is the weight of the i-th partition label subset; Γ i is the label of the i-th input data; I represents the identity matrix; D i is the ridge regression diagonal matrix; T is the symbol for matrix transpose.

6. The data classification method based on the partition pseudo-inverse method to improve the neural network according to claim 1 is characterized in that: The data classification process in S4 is expressed as: In the formula, X' is the input data to be classified, and Y' is the output of data classification result. is the neural network weight.

Images