A width sparse classification method and device

Abstract

The present application provides a width sparse classification method and device, including: constructing a target width learning model; substituting the data set to be trained into the target width learning model to obtain an output matrix; determining the classification boundary of the data set to be trained based on the output matrix; judging the size of the classification boundary and a preset threshold; if the classification boundary is greater than the preset threshold, classifying the data set to be trained using the target width learning model to obtain a classification result; if the classification boundary is less than or equal to the preset threshold, classifying the data set to be trained using a sparse representation classification method to obtain a classification result. Combining the width learning system with the sparse representation classification method can make adaptive selections based on the actual data set to be trained, solving the problem of inaccurate classification caused by a single classification method, such as poor stability, weak anti-interference ability, and cumbersome calculations.

Description

Technical Field

[0001] The present application relates to the technical field of data classification, and in particular to a method and device for width-sparse classification. Background Art

[0002] Classification problems are a hot topic in the field of machine learning. The width learning system (BLS) is a wide and flat neural network. Compared with other networks, BLS has a simple structure and consists only of feature nodes, enhancement nodes, and target coefficients. At the same time, the modeling process of BLS is more efficient. BLS uses a ridge regression algorithm that only requires one iteration to solve the target coefficient, while the deep network uses a gradient descent method that requires multiple iterations to solve the parameters. Sparse representation classification (SRC) is a representation-based classification method. The basic idea of ​​SRC to solve the classification problem is that an unknown test sample can be approximated by a sparse linear combination of known samples in the dictionary.

[0003] However, in actual application, due to the differences between different data sets, it is difficult to reduce the error and complexity of BLS in different environments by only using empirical regularization parameters, making it difficult for the model to obtain the optimal classification hyperplane, which limits the generalization performance of BLS. For classification tasks with noisy samples, especially image classification tasks, traditional BLS has difficulty handling inaccurate features, and has problems such as poor stability and weak anti-interference ability, resulting in unsatisfactory classification results. SRC will be troubled by norm optimization problems and dictionary redundancy problems during the classification process, and the test samples will fall into an extremely time-consuming query process, resulting in unsatisfactory classification results. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a width sparse classification method, including:

[0005] Constructing a target width learning model, wherein a regularization parameter in the target width learning model is determined by a leave-one-out cross-validation method;

[0006] Substituting the training data set into the target width learning model to obtain an output matrix;

[0007] Determine the classification boundary of the to-be-trained data set based on the output matrix;

[0008] Determine the size of the classification boundary and a preset threshold;

[0009] If the classification boundary is greater than the preset threshold, the target width learning model is used to classify the training data set to obtain a classification result;

[0010] If the classification boundary is less than or equal to the preset threshold, a sparse representation classification method is used to classify the training data set to obtain a classification result.

[0011] Optionally, the step of constructing a target width learning model includes:

[0012] Constructing feature nodes of the data set to be trained and enhanced nodes of the data set to be trained using randomly generated weights and biases to obtain a node matrix of the data set to be trained;

[0013] Calculating a mean square error set of the node matrix, and determining a minimum root mean square error in the mean square error set;

[0014] determining a target regularization parameter based on the minimum mean square error;

[0015] A target width learning model is constructed according to the target regularization parameter.

[0016] Optionally, the step of calculating the root mean square error set of the node matrix includes:

[0017] Decomposing the node matrix by using the SVD method to obtain a decomposed node matrix;

[0018] Based on the decomposed node matrix:

[0019]

[0020] The root mean square error set of the node matrix is ​​calculated, where N is the total number of the training data sets, y i and is the actual output and expected output of the i-th data, (HAT BLS ) ii Indicates HAT BLS The i-th value on the matrix diagonal, HAT BLS The matrix is ​​the HAT matrix of BLS.

[0021] Optionally, the step of substituting the training data set into the target width learning model to obtain an output matrix includes:

[0022] Substituting the training data set into the target width learning model to obtain the target coefficient;

[0023] The output matrix is ​​obtained based on the target coefficients and the node matrix.

[0024] Optionally, the step of determining the classification boundary of the to-be-trained data set based on the output matrix includes:

[0025] Determine a maximum value and a second maximum value of an output vector in the output matrix, wherein the maximum value and the second maximum value of the output vector represent categories of data;

[0026] A difference between a maximum value of the output vector and a second maximum value of the output vector is calculated, and the difference is used as a classification boundary of the data set to be trained.

[0027] Optionally, the step of using the target width learning model to classify the to-be-trained data set to obtain a classification result includes:

[0028] The argmax function is used to classify the training data set to obtain a classification result.

[0029] Optionally, the step of classifying the training data set using a sparse representation classification method to obtain a classification result further includes:

[0030] Determining at least two target index values ​​in the output matrix of the to-be-trained data set according to a preset rule;

[0031] Based on the target index value, base atoms having the same label as the target index value are selected in the training data set;

[0032] Building a sparse representation dictionary based on the base atoms;

[0033] The to-be-trained data set is classified based on the sparse representation dictionary to obtain a classification result.

[0034] Optionally, the step of classifying the to-be-trained data set based on the sparse representation dictionary to obtain a classification result includes:

[0035] Calculating sparse coefficients based on the sparse representation dictionary, and taking the minimum sparse coefficient as a target sparse coefficient;

[0036] A residual is calculated according to the target sparse coefficient, and the to-be-trained data set is classified according to the residual to obtain a classification result.

[0037] Optionally, the step of classifying the to-be-trained data set according to the residual to obtain a classification result includes:

[0038] The training data set is classified by using a method of finding the minimum residual to obtain the classification result.

[0039] In a second aspect, the embodiment of the present application further provides a width sparse classification device, including:

[0040] Constructing a model for constructing a target width learning model, wherein a regularization parameter in the target width learning model is determined by a leave-one-out cross-validation method;

[0041] Substituting into a model, for substituting the data set to be trained into the target width learning model to obtain an output matrix;

[0042] A boundary determination model, used to determine the classification boundary of the to-be-trained data set based on the output matrix;

[0043] A judgment model is used to judge the size of the classification boundary and a preset threshold;

[0044] A classification model is used to classify the data set to be trained using the target width learning model to obtain a classification result if the classification boundary is greater than the preset threshold; and is also used to classify the data set to be trained using a sparse representation classification method to obtain a classification result if the classification boundary is less than or equal to the preset threshold.

[0045] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the width-sparse classification method described above are executed.

[0046] The width sparse classification method and device provided in the embodiment of the present application are suitable for machine learning scenarios. Compared with the classification method in the prior art, the accuracy of the classification hyperplane obtained by BLS is improved by optimizing the selection method of the regularization parameter, avoiding the problem that only the empirical regularization parameter used in the prior art is difficult to reduce the error and complexity of BLS in different environments, making it difficult for the model to obtain the optimal classification hyperplane, and then the classification is inaccurate. At the same time, the classification method is selected in the sparse representation classification SRC and the width learning system BLS according to the classification boundary, so that the classification result is more accurate, and the classification method described in the present application has a wider scope of application.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A flow chart of a width sparse classification method provided by an embodiment of the present application is shown;

[0050] Figure 2 A schematic diagram of the structure of a width sparse classification device provided in an embodiment of the present application is shown;

[0051] Figure 3A schematic diagram of the structure of a storage medium device provided in an embodiment of the present application is shown;

[0052] Figure 4 A regularization parameter selection diagram provided in an embodiment of the present application is shown;

[0053] Figure 5 A schematic diagram showing the accuracy of the classification of the ImageNet dataset according to an embodiment of the present application is shown;

[0054] Figure 6 A schematic diagram showing the time taken for classifying the ImageNet dataset in an embodiment of the present application is shown;

[0055] Figure 7 A schematic diagram showing the accuracy of the classification of the USPS data set by the embodiment of the present application is shown;

[0056] Figure 8 A schematic diagram showing the time taken when classifying a USPS data set according to an embodiment of the present application is shown;

[0057] Fig. 9 A schematic diagram showing the accuracy of the classification of the Pavia dataset according to an embodiment of the present application is shown;

[0058] Fig.10 A schematic diagram showing the time taken when classifying the Pavia dataset according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0060] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied to computer learning scenarios.

[0061] Research has found that due to the differences between different data sets, it is difficult to reduce the error and complexity of the BLS in different environments by using only empirical regularization parameters, making it difficult for the model to obtain the optimal classification hyperplane, which limits the generalization performance of BLS. For classification tasks with noisy samples, especially image classification tasks, traditional BLS has difficulty handling inaccurate features, and has problems such as poor stability and weak anti-interference ability, resulting in unsatisfactory classification results. SRC will be trapped in the problem of norm optimization and dictionary redundancy during the classification process, and the test samples will fall into an extremely time-consuming query process, resulting in unsatisfactory classification results.

[0062] Based on this, the present application embodiment provides a width sparse classification method, such as Figure 1 As shown, including:

[0063] S101, constructing a target width learning model, wherein a regularization parameter in the target width learning model is determined by a leave-one-out cross-validation method;

[0064] In a possible implementation, the step of building a target width learning model includes:

[0065] Constructing feature nodes of the data set to be trained and enhanced nodes of the data set to be trained using randomly generated weights and biases to obtain a node matrix of the data set to be trained;

[0066] Calculating a mean square error set of the node matrix, and determining a minimum root mean square error in the mean square error set;

[0067] determining a target regularization parameter based on the minimum mean square error;

[0068] A target width learning model is constructed according to the target regularization parameter.

[0069] For example, the linear function takes the input data X = [x1, x2, ..., x N ] is converted into mapping features and stored in the feature node.

[0070] Z i =φ i (XW ei +β ei )

[0071] Where X∈R (N×S) , N is the total number of input data, S is the dimension of each input data, W ei and β ei is a random weight coefficient with appropriate dimension, i = 1, 2, …, n, n is the number of feature node groups, the number of feature nodes in each group is k, and φ(·) is a linear function;

[0072] All the mapping features are integrated and recorded as Z n :

[0073] Z n ≡[Z1,…,Z n ]

[0074] Among them, Z n ∈R (N×nk) , nk is the total number of feature nodes;

[0075] The nonlinear function transforms the feature node Z n Further transformed into enhanced nodes;

[0076]

[0077] in, and is a random weight coefficient with appropriate dimension, m is the total number of enhanced nodes, j = 1,…,m, ξ(·) is a linear transformation function;

[0078] All the mapping features are integrated and recorded as H m :

[0079] H m ≡[H1,…,H m ]

[0080] Among them, H m ∈R (l+u)×m , m is the total number of enhanced nodes;

[0081] Integrate all feature nodes and enhancement nodes.

[0082] A=[Z n |H m ]

[0083] Among them, A∈R N×(nk+m) ;

[0084] Solve for the pseudo-inverse of the node matrix A:

[0085] A + =(A T A+θI) -1 A T

[0086] Use the pseudo-inverse of A to obtain the HAT matrix of BLS, HAT BLS :

[0087] HAT BLS =A(A T A+θI) -1 A T

[0088] Among them, θ is the regularization parameter;

[0089] In a possible implementation, the step of calculating the root mean square error set of the node matrix includes:

[0090] Decomposing the node matrix by using the SVD method to obtain a decomposed node matrix;

[0091] For example, the node matrix A is decomposed by SVD method, which is recorded as A=UDV T , then convert the above formula into:

[0092] HAT BLS =A(VDU T UDV T +θI) -1 A T

[0093] =AV(D 2 +θI) -1 V T A T

[0094] Where D is a diagonal matrix;

[0095] Based on the decomposed node matrix:

[0096]

[0097] The root mean square error set of the node matrix is ​​calculated, where N is the total number of the training data sets, y i and is the actual output and expected output of the i-th data, (HAT BLS ) ii Indicates HAT BLS The i-th value on the matrix diagonal, HAT BLS The matrix is ​​the HAT matrix of BLS.

[0098] The root mean square error set of the node matrix is ​​calculated, that is, the regularization parameter candidate set [θ min ,θ max The MSE of each θ in ] PRESS Value, through the formula:

[0099]

[0100] Select MSE PRESS The smallest value of θ is taken as the optimal regularization parameter, denoted as θ opt , using the optimal regularization parameter θ opt Construct the objective function:

[0101]

[0102] Where Y = [y1, y2, …, y N ] is the label corresponding to the input data X, and the optimal target coefficient β opt This can be solved by ridge regression:

[0103] β opt =(A T A+θ opt I) -1 A T Y

[0104] Where I represents the identity matrix;

[0105] Will

[0106] β opt =(A T A+θ opt I) -1 A T Y

[0107] Convert to

[0108] β opt =V(D 2 +θ opt I) -1 V T A T Y,

[0109] Obtaining the optimal target coefficient facilitates calculation and improves calculation efficiency.

[0110] S102, substituting the training data set into the target width learning model to obtain an output matrix;

[0111] In a possible implementation, the step of substituting the training data set into the target width learning model to obtain an output matrix includes:

[0112] Substituting the data set to be trained into the target width learning model to obtain the target coefficient;

[0113] The output matrix is ​​obtained based on the target coefficients and the node matrix.

[0114] For example, construct test data X = [x1, x2, ..., x L ]Node matrix A in the target width learning model test :

[0115]

[0116] Among them, A test∈, L is the number of test data, n is the number of feature node groups, k is the number of feature nodes in each group, and m is the number of enhanced nodes.

[0117] Using the target coefficient β opt and the node matrix A test The classification hyperplane O can be obtained:

[0118] O=A test β opt

[0119] Among them, O∈R L×C , C is the total number of labeled data categories, and the classification hyperplane O is the output matrix.

[0120] S103, determining the classification boundary of the to-be-trained data set based on the output matrix;

[0121] In a possible implementation, the step of determining the classification boundary of the to-be-trained data set based on the output matrix includes:

[0122] Determine a maximum value and a second maximum value of an output vector in the output matrix, wherein the maximum value and the second maximum value of the output vector represent categories of data;

[0123] A difference between a maximum value of the output vector and a second maximum value of the output vector is calculated, and the difference is used as a classification boundary of the data set to be trained.

[0124] For example, each test data O i , i=1,..,L is determined by the position of the maximum value in each column of the output matrix. That is, the output vector O i , i=1,..,L, each column value represents the probability of each category, and the category of the data is consistent with the index of the maximum value position, then, through:

[0125]

[0126] Reflects the classification boundary of each sample: where i=1,..,L, and is the output vector O i The first and second largest values ​​of Td i is the classification boundary, Td i The larger the value, the better the classification boundary and the higher the accuracy of data classification. If the value is smaller, the less obvious the classification boundary and the lower the accuracy of data classification.

[0127] S104, determining the size of the classification boundary and a preset threshold;

[0128] For example, according to actual needs, the threshold γ is set to further improve the effect of data classification.

[0129] S105. If the classification boundary is greater than the preset threshold, the target width learning model is used to classify the data set to be trained to obtain a classification result; if the classification boundary is less than or equal to the preset threshold, a sparse representation classification method is used to classify the data set to be trained to obtain a classification result.

[0130] By constructing a target width learning model, wherein the regularization parameter in the target width learning model is determined by the leave-one-out cross-validation method; optimizing the selection method of the regularization parameter, the accuracy of the classification hyperplane obtained by BLS is improved, and the problem that only using empirical regularization parameters in the prior art is difficult to reduce the error and complexity of BLS in different environments, making it difficult for the model to obtain the optimal classification hyperplane, and then the classification is inaccurate, the data set to be trained is substituted into the target width learning model to obtain an output matrix; based on the output matrix, the classification boundary of the data set to be trained is determined; the size of the classification boundary and the preset threshold is judged; if the classification boundary is greater than the preset threshold, the target width learning model is used to classify the data set to be trained to obtain a classification result; if the classification boundary is less than or equal to the preset threshold, the sparse representation classification method is used to classify the data set to be trained to obtain a classification result. According to the classification boundary, the classification method is selected in the sparse representation classification SRC and the width learning system BLS, so that the classification result is more accurate, and the application scope of the classification method described in this application is wider.

[0131] In a possible implementation, the step of using the target width learning model to classify the training data set to obtain a classification result includes:

[0132] The argmax function is used to classify the training data set to obtain a classification result.

[0133] Exemplarily, the category of the test data can be calculated according to the classification hyperplane O:

[0134]

[0135] in, L is the number of test data, C is the total number of data categories, Y test Represents the category of the test data.

[0136] In a possible implementation, the step of classifying the training data set using a sparse representation classification method to obtain a classification result further includes:

[0137] Determining at least two target index values ​​in the output matrix of the to-be-trained data set according to a preset rule;

[0138] Based on the target index value, base atoms having the same label as the target index value are selected in the training data set;

[0139] Building a sparse representation dictionary based on the base atoms;

[0140] The to-be-trained data set is classified based on the sparse representation dictionary to obtain a classification result.

[0141] Exemplarily, the output matrix O is obtained j μ maximum value indices, i.e., target index values, based on the target index values ​​for the test data x j Filter the base atoms with the same label as the target index value, and build an adaptive sparse sub-dictionary based on the base atoms Adaptive sparse sub-dictionary That is, the sparse representation dictionary classifies the to-be-trained data set based on the sparse representation dictionary to obtain a classification result.

[0142] In a possible implementation, the step of classifying the to-be-trained data set based on the sparse representation dictionary to obtain a classification result includes:

[0143] Calculating sparse coefficients based on the sparse representation dictionary, and taking the minimum sparse coefficient as a target sparse coefficient;

[0144] A residual is calculated according to the target sparse coefficient, and the to-be-trained data set is classified according to the residual to obtain a classification result.

[0145] For example, assuming is the data set, C is the total number of data categories, is the data dimension, k c represents the total number of data belonging to class c, A=[A1,A2…,A C ] is a dictionary;

[0146] pass

[0147]

[0148] Query the optimal sparse coefficient of data y, where ||·||0 is norm, is the optimal sparse coefficient.

[0149] because The norm is difficult to be solved, so the above formula is transformed into:

[0150]

[0151] Among them, ||·||1 is the l1 norm, ||·||2 is the l2 norm, and ε is the error value;

[0152] Use the optimal sparse coefficient to calculate the residual:

[0153]

[0154] Among them, δ C (·) is the mapping function, c is the category, It means that the characteristic function will is mapped to a new vector whose non-zero elements are The items associated with class c in .

[0155] In a possible implementation, the step of classifying the to-be-trained data set according to the residual to obtain a classification result includes:

[0156] The training data set is classified by using a method of finding the minimum residual to obtain the classification result.

[0157] For example, by Find the minimum residual to achieve classification, where C is the total number of categories.

[0158] In a possible implementation, Figure 2 As shown, the embodiment of the present application also provides a width sparse classification device, including:

[0159] Constructing a model 201 for constructing a target width learning model, wherein a regularization parameter in the target width learning model is determined by a leave-one-out cross-validation method;

[0160] Substituting into model 202, for substituting the training data set into the target width learning model to obtain an output matrix;

[0161] A boundary determination model 203, used to determine the classification boundary of the to-be-trained data set based on the output matrix;

[0162] A judgment model 204 is used to judge the size of the classification boundary and a preset threshold;

[0163] The classification model 205 is used to classify the data set to be trained by using the target width learning model to obtain a classification result if the classification boundary is greater than the preset threshold; and is also used to classify the data set to be trained by using a sparse representation classification method to obtain a classification result if the classification boundary is less than or equal to the preset threshold.

[0164] An embodiment of the present application also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the width-sparse classification method as described above are performed.

[0165] The present application also provides a computer-readable storage medium. Figure 3 As shown, the computer-readable storage medium 300 stores a computer program 310, and when the computer program is executed by a processor, the steps of the width-sparse classification method described above are executed.

[0166] Embodiment 1:

[0167] like Figure 4 As shown in the regularization parameter selection diagram provided in the embodiment of the present application, the regularization parameter of the traditional BLS is e -20.7 , performance index MSE PRESS The optimal regularization parameter e provided in the embodiment of the present application is -5.9 , minimum MSE PRESS The results show that compared with traditional BLS, the MSE of BLS-LOO is PRESS Therefore, the experiment sets the regularization parameter obtained by leave-one-out cross-validation method as e -5.9 The results show that under the ImageNet dataset, the embodiment of the present application can find a better regularization parameter and has better generalization performance.

[0168] Embodiment 2:

[0169] like Figure 5-Figure 6 As shown, ImageNet is a large-scale image classification dataset. This example uses a subset of it, which contains 5 categories and a total of 7341 samples. In addition, each data is feature extracted using the DeCaf method, and the 6th layer features are taken for experimental use, referred to as DeCaf6 features. Each sample has a total of 4096 dimensions. The experiment takes 250 samples (50 for each category) from the dataset as training samples, and the remaining data is used as test samples. The relevant parameters of SVM are the default values ​​in the Lib-SVM toolbox. The number of nodes of ELM is 2000. The regularization parameter of SRC is 0.1, and the sparse coefficient is calculated. The regularization parameter of BLS is an empirical value, e -20.7 (2 -30 ), the node combination is 300-1500. The regularization parameter of this application is [e -28 ,e -27.9 ,e -27.8 ,…,e 12] is selected. The node selection process of this application is shown in Table 1:

[0170] Table 1: Classification performance of different numbers of nodes in the target width learning model

[0171]

[0172]

[0173] As can be seen from Table 1, when the feature nodes are 300 and the enhancement nodes are 1750, the test accuracy of the target width learning model is 86.48%, which is better than other node combinations. When the feature nodes are 300 and the enhancement nodes are 500, the target width learning model has the shortest time. When the combination of feature nodes and enhancement nodes is 350-1500, the STD value of the target width learning model is the smallest and the stability is optimal. Comprehensive experimental results, in this case, the node combination of the target width learning model is 300-1500.

[0174] In the ImageNet dataset, in order to analyze the influence of the combination of the plane error value γ and the adaptive dictionary dimension μ on the DBSRC test accuracy and classification time, the plane error value range is set to [0.025, 0.05, 0.075, …, 0.40], and the adaptive dictionary dimension value range is set to [1, 2, …, 5]. Figure 6 and Figure 7 As shown, when the plane error γ remains unchanged, as the adaptive sub-dictionary dimension μ increases, the classification accuracy will first increase and then decrease. When the dictionary dimension μ remains unchanged, as γ increases, the classification accuracy will first increase and then decrease. Whether increasing the plane error γ or increasing the dictionary dimension μ, the classification time of DBSRC will increase. Therefore, the method described in this application can flexibly select a combination of γ and μ to obtain the optimal classification accuracy or better classification efficiency. Considering the impact of the combination of plane error γ and dictionary dimension μ on the performance of DBSRC, the experiment sets the combination of plane error γ and dictionary dimension μ (γ, μ) to (0.1, 4).

[0175] The performance of SVM, ELM, SRC, BLS, target width learning BLS-LOO and the classification method DBSRC described in this application were compared under the same conditions. Each method was run independently 10 times. The experiment selected three indicators: average test accuracy, average total classification time and average test accuracy variance for comparison. The experimental results are shown in Table 2:

[0176] Table 2: Classification performance of each method on the ImageNet dataset

[0177]

[0178] As can be seen from Table 2, the classification time of BLS is 1.06s, which is faster than several other classification models. The stability of SVM is better than several other classification models. The classification accuracy of the method described in this application is 86.70%, which is better than SVM, ELM, SRC, BLS and target width learning. The classification time of the method described in this application is 526.54s less than that of SRC. The STD value of DBSRC is 0.22, which is smaller than ELM, BLS and target width learning. The experimental results show that the method described in this application has good classification effect and efficiency for the ImageNet dataset.

[0179] Embodiment three:

[0180] This example selects the USPS dataset to verify the effectiveness of the classification method described in this application. The dataset consists of handwritten digit images from 0 to 9, with a total of 9298 images. Each digit is a grayscale image with 16×16 pixels. The experiment selects 2000 image data (200 per category) as the training set, and the remaining 7298 samples as the test set. Each column in the original USPS image will be directly used as a feature. The relevant parameters of SVM are the default values ​​in the Lib-SVM toolbox. The number of nodes of ELM is 3000. The regularization parameter of SRC is 0.1, and the sparse coefficient is calculated. The regularization parameter of BLS is an empirical value, e -20.7 (2 -30 ), the node combination is 100-2000. The regularization parameter of the method described in the present application is the same as that of the target width learning model. The regularization parameter of the target width learning model is [e -21 ,e -20.9 ,e -20.8 ,…,e -1 ] is selected. As the value of θ increases, the performance indicator MSE of the target width learning model PRESS There will be a phenomenon of first decreasing and then increasing. -11 When MSE PRESS The value is 0.6212, which is smaller than other θ. Therefore, in the USPS dataset, the experiment selects the optimal θ value of the target width learning model as e -11 The experimental results show that the target width learning model can optimize parameters with the help of the LOO algorithm and improve the generalization performance of the model. The node combination of the experimental method described in this application is the same as that of the target width learning model. The node selection process of the target width learning model is shown in Table 3:

[0181] Table 3: Classification performance of different numbers of nodes in the target width learning model

[0182]

[0183]

[0184] As can be seen from Table 3, both feature nodes and enhanced nodes will affect the performance of BLS-LOO. If the test accuracy of BLS-LOO is required to be high, a combination of 100 feature nodes and 3000 enhanced nodes is selected. If the classification time of BLS-LOO is required to be shorter, a combination of feature nodes and enhanced nodes of 100-1500 is selected. If the model is required to be more stable, a node combination of 350-2000 is preferred. Considering the impact of each node combination on the performance of BLS-LOO, the experiment selects 100 feature nodes and 2000 enhanced nodes, that is, 100-2000.

[0185] In order to test and analyze the influence of the combination of plane error value γ and adaptive sub-dictionary dimension μ on DBSRC test accuracy and classification time in USPS data, the plane error value range is set to [0.01, 0.05, 0.10, 0.15, …, 0.45, 0.50], and the adaptive dictionary dimension value range is set to [1, 2, …, 10]. The results of test accuracy and classification time are shown in the figure below. Figure 7 and Figure 8 shown.

[0186] When γ is fixed and the μ value is 1-6, the test accuracy of the classification method DBSRC described in this application shows an upward phenomenon; when γ is fixed and the μ value is greater than 6, the test accuracy of DBSRC will show a stable trend. This also verifies the importance of adaptive sub-dictionary dimension selection. When the μ value is fixed and the γ value is 0.35, the test accuracy of DBSRC is optimal. When the γ and μ values ​​increase at the same time, the classification time of DBSRC will be significantly improved. Considering the impact of the combination of γ and μ on the accuracy and efficiency of DBSRC, the experiment sets γ and μ to 0.35 and 2 respectively.

[0187] The performance of SVM, ELM, SRC, BLS, BLS-LOO and DBSRC were compared under the same conditions. Each method was run independently 10 times. The experiment selected three indicators: average test accuracy, average total classification time and average test accuracy variance for comparison. The experimental results are shown in Table 4:

[0188] Table 4: Classification performance of each method on the USPS dataset

[0189]

[0190] From Table 4, we can see that the STD value of BLS-LOO is 0.27, which is higher than SVM, SRC and DBSRC. SRC suffers from the sample query process and has the longest classification time of 994.33s, while the classification time of DBSRC is only 9.79s. It is worth mentioning that the classification accuracy of DBSRC is slightly better than that of SRC. BLS has the shortest classification time of only 0.72s, but the test accuracy of BLS is 3.26% and 3.97% lower than that of BLS-LOO and DBSRC, respectively. The results show that DBSRC has good classification performance under USPS data set.

[0191] Embodiment 4:

[0192] In order to further verify the effectiveness of DBSRC, the Pavia dataset is selected for this experiment. Pavia is the scene data of the University of Pavia collected by the Reflection Optical System Imaging Spectrometer (ROSIS). The original data has 103 spectral bands, the spectral coverage range is 0.43-0.86μm, and the spatial resolution is 1.3m. The dataset contains 9 classes and a total of about 42776 samples. This experiment uses a combination of principal component analysis and local binary method to reduce the dimension and extract features of the data. After processing, each sample has a total of 1771 dimensions. The division of the data training set and test set is shown in Table 5.

[0193] The relevant parameters of SVM are the default values ​​in the Lib-SVM toolbox. The number of nodes of ELM is 3000. The regularization parameter of SRC is 0.1, and the sparsity coefficient is calculated. The regularization parameter of BLS is an empirical value, e -20.7 (2 -30 ), the node combination is 400-2000. The regularization parameter of DBSRC is the same as that of BLS-LOO. The regularization parameter of BLS-LOO is [e -22 ,e -21.9 ,e -21.8 ,…,e -2 ] to select.

[0194] Table 5: Division of each type of training data and test data set in the Pavia dataset

[0195]

[0196] When the value of θ is e -20.7 When PRESS is 1.121, which is equal to BLS (Without LOO). -13.9 When PRESSThe result shows that BLS-LOO can obtain the MSE by optimizing the parameters. PRESS The smallest regularization parameter. Therefore, the experiment selects the optimal regularization parameter θ of BLS-LOO as e -13.9 .

[0197] The node combination of DBSRC is the same as that of BLS-LOO. The node selection process of BLS-LOO is shown in Table 6:

[0198] Table 6: Classification performance of different numbers of nodes in BLS-LOO

[0199]

[0200]

[0201] As can be seen from Table 6, with the increase of feature nodes or enhancement nodes, the classification time of BLS-LOO will increase. When the feature nodes are fixed and the number of enhancement nodes is greater than 2250, the increase in the test accuracy of BLS-LOO is limited. When the enhancement nodes are fixed, with the increase of feature nodes, the test accuracy of BLS-LOO will increase first and then level off. Considering the impact of different node combinations on the performance of BLS-LOO, the experiment selected 400 feature nodes and 2000 enhancement nodes, i.e. 400-2000.

[0202] In order to test and analyze the influence of the combination of the plane error value γ and the adaptive sub-dictionary dimension μ on the DBSRC test accuracy and classification time in Pavia data, the plane error value range is set to [0.02, 0.04, …, 0.20], and the adaptive dictionary dimension value range is set to [1, 2, …, 9]. The results of test accuracy and classification time are shown in the figure below. Fig. 9 and Fig.10 shown.

[0203] The performance of DBSRC will be affected by the combination of γ and μ. As γ and μ increase, the classification time of DBSRC will increase. When γ is fixed, and μ is greater than 4, the increase in DBSRC classification time is greater. When γ is fixed and μ is 5, the test accuracy of DBSRC is better than other values. When γ and μ are 0.12 and 5 respectively, the classification accuracy of DBSRC is better, which is 96.88%. Based on the experimental results, the experiment selects γ and μ of DBSRC as 0.14 and 2 respectively.

[0204] The performance of SVM, ELM, SRC, BLS, BLS-LOO and DBSRC were compared under the same conditions. Each method was run independently 10 times. The experiment selected three indicators: average test accuracy, average total classification time and average test accuracy variance for comparison. The experimental results are shown in Table 7:

[0205] Table 7: Classification results of various methods on the Pavia dataset

[0206]

[0207]

[0208] As can be seen from Table 7, the STD value of ELM is 0.36, which is higher than that of other classification models. The classification time of BLS is 3.87s, which is shorter than that of other classification models. DBSRC has the highest test accuracy, reaching 96.80%. The test accuracy of BLS-LOO is 96.20%, which is 1.43%, 4.89%, and 2.95% higher than SVM, ELM, and BLS, respectively. The test accuracy of SRC is 96.73%, which is only slightly lower than DBSRC. However, the classification time of SRC is 13985.39s, which is much longer than DBSRC. The results show that DBSRC can complete the classification task under the Pavia dataset with the best test accuracy, good efficiency and stability.

[0209] In summary, in three different datasets, ImageNet, USPS and Pavia, BLS-LOO can use LOO cross-validation to optimize the regularization parameters of the model. More importantly, LOO cross-validation can find regularization parameters that are more suitable for the data according to the characteristics of the data and obtain a better classification hyperplane. Compared with the empirical regularization parameters of traditional BLS, the regularization parameters selected by BLS-LOO will obtain a smaller MSE. PRESS, so that the model has better performance. Both feature nodes and enhancement nodes affect the performance of BLS-LOO. Specifically, whether adding feature nodes or enhancement nodes, the classification time of BLS-LOO will increase. As the number of feature nodes or enhancement nodes increases, the classification accuracy of BLS-LOO will increase first and then level off. Therefore, in practical applications, the number of nodes can be flexibly selected according to specific needs, and the BLS-LOO structure can be reasonably constructed to obtain ideal classification effects. In the ImageNet dataset, DBSRC selects γ and μ values ​​as 0.1 and 4, respectively. In the USPS dataset, DBSRC selects γ and μ values ​​as 0.35 and 2, respectively. In the Pavia dataset, DBSRC selects γ and μ values ​​as 0.14 and 2, respectively. The results show that in the classification tasks of different datasets, DBSRC can select appropriate γ and μ according to the characteristics of the data to obtain ideal classification accuracy. When γ is fixed, as the μ value increases, the redundancy of the dictionary increases, making it difficult to continuously improve the accuracy. The classification time of DBSRC is affected by the combination of the plane error γ and the dictionary dimension μ. When γ is fixed, as μ increases, the classification time of DBSRC will also increase. The reason is that as the μ value increases, the size of the sub-dictionary increases, and the sample query process will also take longer. When μ is fixed, as γ increases, the classification time of DBSRC will continue to increase. The main reason is that the increase in γ will lead to an increase in the number of samples that need to be further classified by SRC. The results show that DBSRC can use the adaptive sub-dictionary selection strategy to reduce dictionary redundancy and obtain ideal classification results. In addition, DBSRC can further classify noise samples using the decision threshold selection strategy, thereby better completing the classification task. Compared with ELM, BLS has higher test accuracy and stronger stability, because BLS can extract data features by constructing feature nodes and enhancement nodes to reduce data redundancy. Compared with SVM, BLS can quickly build a model through the ridge regression algorithm, thereby obtaining better classification efficiency. Compared with BLS, BLS-LOO can complete the classification task with higher test accuracy. This is because BLS-LOO can optimize the regularization parameters of the model through the LOO cross-validation method, thereby obtaining a better classification hyperplane and improving the classification performance of the model. Compared with SRC, DBSRC can reduce dictionary redundancy through dictionary selection strategy, achieve fast and accurate query of test samples, and improve classification effect and efficiency. Compared with BLS-LOO, DBSRC can process noise samples through threshold selection strategy, thus having better classification performance.

[0210] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0211] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0212] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0214] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0215] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A width sparse classification method, characterized in that: include: Constructing a target width learning model, wherein a regularization parameter in the target width learning model is determined by a leave-one-out cross-validation method; Substituting the training data set into the target width learning model to obtain an output matrix; Determine the classification boundary of the to-be-trained data set based on the output matrix; Determine the size of the classification boundary and a preset threshold; If the classification boundary is greater than the preset threshold, the target width learning model is used to classify the training data set to obtain a classification result; If the classification boundary is less than or equal to the preset threshold, a sparse representation classification method is used to classify the training data set to obtain a classification result; The step of constructing the target width learning model includes: Constructing feature nodes of the data set to be trained and enhanced nodes of the data set to be trained using randomly generated weights and biases to obtain a node matrix of the data set to be trained; Calculating a mean square error set of the node matrix, and determining a minimum root mean square error in the mean square error set; determining a target regularization parameter based on the minimum mean square error; constructing a target width learning model according to the target regularization parameter; The step of calculating the root mean square error set of the node matrix comprises: Decomposing the node matrix by using the SVD method to obtain a decomposed node matrix; Based on the decomposed node matrix: The root mean square error set of the node matrix is ​​calculated, where is the total number of the data sets to be trained, and is the actual output and expected output of the i-th data, express The i-th value on the matrix diagonal, The matrix is ​​the width of the learning model matrix; The step of determining the classification boundary of the to-be-trained data set based on the output matrix comprises: Determine a maximum value and a second maximum value of an output vector in the output matrix, wherein the maximum value and the second maximum value of the output vector represent categories of data; Calculating a difference between a maximum value of the output vector and a second maximum value of the output vector, wherein the difference is used as a classification boundary of the data set to be trained; The step of classifying the training data set using the sparse representation classification method to obtain a classification result also includes: Determining at least two target index values ​​in the output matrix of the to-be-trained data set according to a preset rule; Based on the target index value, base atoms having the same label as the target index value are selected in the training data set; Building a sparse representation dictionary based on the base atoms; The to-be-trained data set is classified based on the sparse representation dictionary to obtain a classification result.

2. The width sparse classification method according to claim 1, characterized in that: The step of substituting the training data set into the target width learning model to obtain an output matrix includes: Substituting the data set to be trained into the target width learning model to obtain the target coefficient; The output matrix is ​​obtained based on the target coefficients and the node matrix.

3. The width sparse classification method according to claim 1, characterized in that: The step of using the target width learning model to classify the training data set to obtain a classification result includes: The argmax function is used to classify the training data set to obtain a classification result.

4. The width sparse classification method according to claim 1, characterized in that: The step of classifying the training data set based on the sparse representation dictionary to obtain a classification result includes: Calculating sparse coefficients based on the sparse representation dictionary, and taking the smallest sparse coefficient as the target sparse coefficient; A residual is calculated according to the target sparse coefficient, and the to-be-trained data set is classified according to the residual to obtain a classification result.

5. The width sparse classification method according to claim 4, characterized in that: The step of classifying the training data set according to the residual to obtain a classification result includes: The training data set is classified by using a method of finding the minimum residual to obtain the classification result.

6. A width-sparse classification device, executing the classification method according to any one of claims 1 to 5, characterized in that: include: Constructing a model for constructing a target width learning model, wherein a regularization parameter in the target width learning model is determined by a leave-one-out cross-validation method; Substituting into a model, for substituting the data set to be trained into the target width learning model to obtain an output matrix; A boundary determination model, used to determine the classification boundary of the to-be-trained data set based on the output matrix; A judgment model is used to judge the size of the classification boundary and a preset threshold; A classification model is used to classify the data set to be trained using the target width learning model to obtain a classification result if the classification boundary is greater than the preset threshold; and is also used to classify the data set to be trained using a sparse representation classification method to obtain a classification result if the classification boundary is less than or equal to the preset threshold.

Images