Handwritten numeral classification method and device, medium and equipment
By using principal component analysis and spectrum correction techniques in Hilbert space, the problem of information loss in the dimensionality reduction process of handwritten digit classification methods in the prior art is solved, and the recognition accuracy of handwritten digits is improved.
Patent Information
- Application Number
- CN202410946025.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
The existing handwritten numeric classification methods will lead to information loss when reducing the dimensionality of high-dimensional and linear inseparable data, resulting in low recognition accuracy.
The handwritten digital image data is collected in the linearly separable Hilbert space, and the overall eigenvalue and projected eigenvalue are obtained through principal component analysis, spectrum correction is carried out, spectrum correction discriminant analysis function is constructed, and the categories of handwritten numbers to be classified are used. property.
By retaining more data set information, the information loss during the data dimensionality reduction process is reduced and the recognition accuracy of handwritten numbers is improved.
Smart Images

Figure CN118918595A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method, device, medium and equipment for classifying handwritten digits. Background Art
[0002] Handwritten digit classification technology is an important branch in the field of image processing and machine learning. It uses computer algorithms to identify and classify handwritten digits in images. Digital signatures are an important application field of handwritten digit recognition technology. In financial, legal and commercial documents, the legitimacy verification of digits is crucial to ensure the integrity and authenticity of documents. By classifying and verifying handwritten digits, the legitimate rights and interests of individuals and enterprises can be effectively protected.
[0003] In the prior art, CN103093235B discloses a handwritten digit recognition method based on improved distance kernel principal component analysis. The method first performs binarization preprocessing on the sample, maps the processed sample data to the kernel feature space using the distance kernel, divides the training sample set into N subsets using K-means clustering in the kernel feature space, then calculates the eigenvector of the kernel matrix in the kernel space to obtain the transformation matrix, extracts features using the transformation matrix, puts the extracted feature data set into a support vector machine to establish a training model, and finally uses the model to predict the recognition rate of the test sample.
[0004] However, the above method will cause the loss of image information in the process of reducing the dimensionality of high-dimensional, linearly inseparable data such as handwritten digit images, resulting in low recognition accuracy of handwritten digits. Summary of the invention
[0005] Based on this, it is necessary to provide a handwritten digit classification method, device, medium and equipment to address the above technical problems.
[0006] This manual adopts the following technical solutions:
[0007] This specification provides a handwritten digit classification method, including:
[0008] Collecting handwritten digital image data with category labels, and mapping the handwritten digital image data into a linearly separable Hilbert space;
[0009] In the Hilbert space, the overall eigenvalue λ of the handwritten digital image data is obtained j , the overall eigenvalue λ j has a corresponding overall eigenvector, the overall eigenvalue λ j Used to characterize the degree of discreteness in the direction of the overall feature vector corresponding to the handwritten digital image data;
[0010] In the Hilbert space, the handwritten digital data is projected to the high variance direction through principal component analysis to obtain high variance data; the covariance matrix of the high variance data is calculated, and the covariance matrix is decomposed to obtain the one-to-one corresponding projection eigenvalues l j and the projected eigenvector, the projected eigenvalue l j Used to characterize the degree of discreteness in the direction of the projected eigenvector corresponding to the projected high variance data;
[0011] Use the population eigenvalue λ j Replace the projected eigenvalue l j , perform spectrum correction on the projection covariance matrix, construct a spectrum correction discriminant analysis function based on the spectrum corrected projection covariance matrix, and use the spectrum correction discriminant analysis function to calculate the category attributes of the handwritten digits to be classified.
[0012] Furthermore, the overall covariance matrix Σ of the handwritten digital image data with category labels is:
[0013]
[0014] Among them, σ 2 is an undetermined value used to measure the size of the non-spiked eigenvalue, I P is the p*p dimensional identity matrix, λ j The eigenvalue deviating from the center of the distribution is called the Spiked eigenvalue, v j is its corresponding eigenvector, v j T Yes j The transpose of ; k = {1,2}, r1 represents the number of large Spiked eigenvalues, and r2 represents the number of small Spiked eigenvalues.
[0015] Furthermore, the projecting of the handwritten digital data to the high variance direction by principal component analysis specifically includes:
[0016] Calculate the sample mean of data in Hilbert space H and the sample covariance matrix S Φ :
[0017]
[0018] Among them, n i represents the number of training samples of the i-th category, is the jth sample of the i-th class sample;
[0019] n represents the total sample size, It is a sample Samples after projection to Hilbert space; Q = [Q0, Q1], Q0 is the sample vector after centering of class 0 samples; represents the sample mean of the i-th type of training samples in the Hilbert space; S Φ is the sample covariance matrix; T represents transpose;
[0020] For the sample covariance matrix S Φ Perform kernel transformation to obtain the kernel transformation sample matrix and kernel transformed sample covariance matrix
[0021] Kernel Transformation Sample Covariance Matrix Constructing the projection matrix Using the Projection Matrix Project the original handwritten digit data.
[0022] Furthermore, the sample covariance matrix S Φ Perform kernel transformation to obtain the kernel transformation sample matrix and kernel transformed sample covariance matrix Specifically include:
[0023] For the sample covariance matrix S Φ Spectral decomposition, obtain n-dimensional coordinates {w1,w2,...,w n};
[0024] Project the data in the Hilbert space H to the n-dimensional coordinates {w1,w2,...,w n}, we get the transformed eigenvector y=[y1,y2,...,y m ] T ;
[0025] Based on the transformed feature vector y=[y1,y2,...,y m ] T Calculate the kernel transform sample matrix and kernel transformed sample covariance matrix
[0026]
[0027] in, is the jth sample of the i-th class sample after transformation, Representation sample Sample after projection into Hilbert space; are the covariance matrices of class 0 and class 1 obtained based on the transformed samples, respectively.
[0028] Furthermore, the sample covariance matrix S ΦSpectral decomposition, obtain n-dimensional coordinates {w1,w2,...,w n}, specifically including:
[0029] The sample covariance matrix S Φ Perform spectral decomposition and obtain:
[0030] QQ T =WDW T
[0031] W=[w1,w2,…,w n ]
[0032] D = diag[λ1, λ2, ..., λ n ]
[0033] Among them, w1~w n is the n eigenvector basis, λ1~λ n is the characteristic value;
[0034] According to QQ T =WDW T Get QQ T w i =λ i w i ;
[0035] On QQ T w i =λ i w i Multiply both sides by Q T , get (Q T Q)Q T w i =λ i Q T w i ;
[0036] Let R = QQ T , then Q T w i is the eigenvector of R, which is related to QQ T have the same eigenvalues;
[0037] Perform spectral decomposition of matrix R:
[0038] R=ΓEΓ T
[0039] Γ=[γ1,γ2,…,γ n ]
[0040] Among them, E is the unit matrix, γ1~γ n is the decomposed vector;
[0041] Then the eigenvector of the sample covariance matrix is expressed as:
[0042]
[0043] At this time, {w1,w2,...,w n} is the n-dimensional basis coordinate in the Hilbert space, projecting the mapped sample Φ(x) onto the eigenvector basis {w1,w2,...,w n}, we get the transformed eigenvector y=[y1,y2,...,y m ] T .
[0044] Furthermore, the kernel-based transform sample covariance matrix Constructing the projection matrix Using the Projection Matrix Project the original handwritten digital data, including:
[0045] Kernel transform sample covariance matrix Perform eigendecomposition to obtain the eigenvectors corresponding to the first k eigenvalues;
[0046] Construct a projection matrix based on the eigenvector corresponding to the largest eigenvalue
[0047] Construct a projection matrix based on the eigenvectors corresponding to the first k eigenvalues
[0048] Multiply the original handwritten digital data by the projection matrix to obtain the reduced dimension data matrix:
[0049]
[0050] in, is the eigenvector matrix, is a diagonal matrix of eigenvectors, yes The inverse of
[0051] is the original handwritten digital data in Hilbert space, Is The projection matrix is composed of the first k columns of eigenvectors. The value of k is determined by the sample dimension p and the PCA dimensionality reduction ratio dim. k = floor(p*dim), where floor is rounded down. is the data matrix after dimensionality reduction.
[0052] Furthermore, the spectrum correction discriminant analysis function is:
[0053]
[0054] Among them, y is the handwritten digit image to be classified, are the sample means of different categories of digital images in the Hilbert space; is the projection covariance matrix after spectrum correction, and π0 and π1 are the sample prior probabilities of different categories of digital images.
[0055] A handwritten digit classification device, comprising:
[0056] A data acquisition module, used for collecting handwritten digital image data with category labels and mapping the handwritten digital image data into a linearly separable Hilbert space;
[0057] Correction module, used to obtain the overall eigenvalue λ of handwritten digital image data in Hilbert space j , the overall eigenvalue λ j has a corresponding overall eigenvector, the overall eigenvalue λ j Used to characterize the discrete degree of handwritten digital image data in the direction of its corresponding overall feature vector;
[0058] It is used to project the handwritten digital data to the high variance direction through principal component analysis in the Hilbert space to obtain high variance data; calculate the covariance matrix of the high variance data, and decompose the covariance matrix to obtain the one-to-one corresponding projection eigenvalues l j and the projected eigenvector, the projected eigenvalue l j It is used to characterize the degree of dispersion of the projected high variance data in the direction of the corresponding projected eigenvector;
[0059] For the overall eigenvalue λ j Replace the projected eigenvalue l j Performing spectrum correction on the projection covariance matrix, and constructing a spectrum correction discriminant analysis function based on the spectrum corrected projection covariance matrix;
[0060] The classification module is used to calculate the category attributes of the handwritten digits to be classified using the spectrum correction discriminant analysis function.
[0061] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned handwritten digit classification method is implemented.
[0062] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned handwritten digit classification method when executing the program.
[0063] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:
[0064] The present invention first obtains the overall eigenvalue in the linearly separable Hilbert space, and then obtains the projection eigenvalue of the projected data. Since the overall eigenvalue is obtained based on the original data and contains more data set information, and the projection eigenvalue is obtained based on the data after projection dimensionality reduction, the overall data information can be retained more completely by replacing the projection eigenvalue with the overall eigenvalue, thereby reducing the information loss in the data dimensionality reduction process, and enabling the discriminant function to have a stronger ability to distinguish the category attributes of handwritten digits, thereby achieving a higher handwritten digit recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0066] Figure 1 A flow chart of a handwritten digit classification method provided in this manual;
[0067] Figure 2 A schematic diagram of a handwritten digit classification device provided in this manual;
[0068] Figure 3 A schematic diagram of a computer device for implementing a handwritten digit classification method provided in this specification. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0070] With the rapid development of computers, data acquisition has become more convenient, and the data we study has gradually become high-dimensional or even infinite-dimensional, and classical classification algorithms have shown their limitations. In high-dimensional settings, the estimate of the mean associated with the covariance matrix for each class is highly inaccurate, resulting in severe degradation of classification performance. Linear discriminant analysis was first proposed by RA Fisher in 1936. Under the assumption that the data is Gaussian, has completely known statistics and a common covariance matrix across classes, this model has been proven to be the optimal classifier with the lowest misclassification rate. However, this model is not suitable for high-dimensional data forms in modern contexts. In 2008, Fan Jianqing et al. estimated the impact of noise and proposed a solution based on feature selection in their paper. In 2015, Zollanvari et al. used the latest results in random matrix theory to obtain the optimal estimate of the regularization parameter and used regularized LDA to solve the problem of the irreversibility of the sample covariance matrix. In 2020, Houssem Sifaou et al. proposed a model with spiked covariance matrix features—I-LDA. While reducing the number of parameter estimates and the complexity of calculations, it retains the original structure of the overall covariance matrix. Its classification performance is also better than other existing LDA models. However, when considering the spiked model structure, this method ignores the effective information brought by the small spiked eigenvalues.
[0071] The above improved models are all linear classification models. In other words, nonlinear classification data cannot be effectively classified by a linear classifier (straight line, plane), and the above improved models cannot be effectively applied to nonlinear classification data. In discriminant analysis, especially linear discriminant analysis (LDA), the "spike model" is an assumption that some eigenvalues are significantly larger than other eigenvalues. This term is often used to describe the covariance structure of data, especially in multivariate analysis. In the context of LDA, the spike model assumes that the data has significant variability in a low-dimensional subspace and little variability in other directions. This structure can be reflected by the size of the eigenvalue: the eigenvectors corresponding to large eigenvalues define the main direction of change of the data, which are very important in discriminant analysis because they can distinguish different categories. The eigenvectors corresponding to small eigenvalues indicate that the data has little variability in these directions, and these directions may not be very helpful for classification.
[0072] Spectral correction of the spike model is usually performed to deal with the small sample problem, that is, when the sample size is smaller than the feature dimension, the intra-class covariance matrix may be singular, which makes it impossible to directly use methods such as linear discriminant analysis (LDA). The purpose of spectral correction is to adjust the eigenvalues of the intra-class covariance matrix to make it reversible, so that subsequent discriminant analysis can be performed. A common method of spectral correction is to correct the eigenvalues of the intra-class covariance matrix so that all eigenvalues are greater than a small positive number. In this way, the corrected intra-class covariance matrix is no longer singular and its reversibility can be guaranteed. The following are the basic steps for spectral correction of the spike model: 1. Calculate the intra-class covariance matrix: For each class, calculate its sample covariance matrix, and then sum it up to obtain the total intra-class covariance matrix Sw. 2. Eigendecomposition: Perform eigendecomposition on the intra-class covariance matrix Sw to obtain eigenvalues and eigenvectors. 3. Spectral correction: Correct the eigenvalues to ensure that all eigenvalues are greater than 0, so that the corrected intra-class covariance matrix becomes positive definite. 4. Reconstruct the corrected intra-class covariance matrix: Use the corrected eigenvalues and the original eigenvectors to reconstruct the corrected intra-class covariance matrix: in is the corrected eigenvalue, u i is the corresponding eigenvector, and k is the number of eigenvalues. 5. Perform discriminant analysis: Use the modified within-class covariance matrix Swc and between-class covariance matrix Sb to perform linear discriminant analysis or other multivariate analysis methods.
[0073] Based on this, the present application discloses a spectrally corrected kernel linear classifier, SC-KLDA. The model cites the theoretical results of large-dimensional random matrices, and proposes that when the dimension of the observed sample is of the same order of magnitude as the sample size (n→∞, p→∞, p / n→c), the distribution of the eigenvalues of the population covariance matrix obeys the Marcenko-PasturLow curve. The present application records the eigenvalues that deviate from the center of the distribution as large and small Spiked eigenroots, and the model with this feature is recorded as a model of a spiked covariance matrix. The classification algorithm performed under the premise of this model structure includes the following steps: obtaining the original data with class labels; given a nonlinear mapping Φ, mapping the input data space to the Hilbert space H, so that the subsequent PCA dimensionality reduction and the construction of the classification model are performed in the Hilbert space: dividing the labeled data into a training set and a validation set according to a fixed ratio; calculating the mean and sample covariance matrix of each category of data in the training set to obtain the common covariance matrix of the set samples; performing spectral decomposition on the common covariance matrix to obtain the corresponding eigenvalues and orthogonal eigenvectors; selecting the eigenvectors corresponding to the "k" features with the largest eigenvalues (i.e., the largest variance) to form a data matrix after feature simplification, and using this matrix to map the original data vector to a low-dimensional space to obtain the eigenvalues and eigenvectors in the low-dimensional space; replacing the eigenvalues of the covariance matrix after feature simplification with the estimated values of the overall eigenvalues, and performing spectral correction on the sample covariance matrix structure to form a spectral correction discriminant analysis function, by which the category attributes of any new sample can be determined. In the handwritten digit classification method provided in the embodiment of the present application, it is considered to introduce a kernel method to map nonlinearly separable sample data in a low-dimensional space to a high-dimensional space, and use a linear classifier in the high-dimensional space to solve the classification problem. Experiments have shown that the introduction of the kernel method can effectively improve the classification performance. In addition, the method adds a small Spiked eigenroot to the spike model structure, and performs spectral correction on the overall covariance matrix estimator, so that it more completely retains the structural information of the overall covariance matrix, and controls the estimation error within a smaller range.
[0074] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0075] Figure 1-2 This is a flow chart of a handwritten digit classification method in this specification. The digit recognition method proposed in this application is applicable to the scenario where the overall covariance matrix Σ adopts the following specific form:
[0076]
[0077] Among them, I P is the p*p dimensional identity matrix, λ j The eigenvalues that deviate from the center of the distribution are called Spiked eigenvalues, vj is the corresponding eigenvector; r1 represents the number of large Spiked eigenroots, and r2 represents the number of small Spiked eigenroots.
[0078] The main process of a handwritten digit classification method is as follows: mapping the original handwritten digit data to the Hilbert space, projecting the handwritten digit data in the Hilbert space through principal component analysis, obtaining the projection covariance matrix and corresponding projection eigenvalues of the projected data; performing spectral correction on the projection covariance matrix by replacing the projection eigenvalues with the overall eigenvalues; and using the spectral correction discriminant analysis function to calculate the category attributes of the handwritten digits to be classified.
[0079] More specific steps are: obtain the original data with class labels; given a nonlinear mapping Φ, map the input original data space to the Hilbert space H; divide the data in the Hilbert space into a training set and a validation set according to a fixed ratio; calculate the mean and sample covariance matrix of each category of data in the training set, and obtain the common covariance matrix of the set samples as an estimator of the covariance matrix of the unknown population; perform spectral decomposition on the common covariance matrix to obtain the corresponding eigenvalues and orthogonal eigenvectors; directly replace the eigenvalues of the sample covariance matrix decomposition with the eigenvalues corresponding to the population covariance matrix. Eigenvalue, spectral correction is performed on the sample covariance matrix structure to obtain the estimator of the population covariance matrix; the eigenvectors corresponding to the "k" features with the largest eigenvalues (i.e., the largest variance) are selected to form a data matrix after feature simplification, and the original data vector is mapped to a low-dimensional space using this matrix to obtain the eigenvalues and eigenvectors in the low-dimensional space; the eigenvalues of the covariance matrix after feature simplification are replaced with the estimated values of the population eigenvalues, and the sample covariance matrix structure is spectrally corrected to form a spectral correction discriminant analysis function, through which the category attributes of any new sample can be determined. The following is the expansion of each step:
[0080] Step 1: Get the original data with class labels. The standard data set for handwritten digit recognition contains a total of 10 subsets. The present invention selects two subsets from the data set as the overall data of the first category of samples and the overall data of the second category of samples, denoted as I0 and I1 respectively; the sample dimension and feature quantity of category I0 are: 6131*784; the sample dimension and feature quantity of category I1 are: 5851*784.
[0081] Step 2: Given a nonlinear mapping Φ, map the input original data space to the Hilbert space H to obtain the handwritten digit mapping data Φ(X).
[0082] Step 3: Calculate the sample mean and covariance matrix of the Hilbert space data using the following formula:
[0083]
[0084] Among them, n represents the total sample size, n i represents the number of training samples of the i-th category, is the jth sample of the i-th class sample, It is a sample Samples after projection to Hilbert space; Q = [Q0, Q1], Q0 is the sample vector after centering of class 0 samples; represents the sample mean of the i-th type of training samples in the Hilbert space; S Φ is the sample covariance matrix.
[0085] Step 4: Using the kernel technique, we can express the linear expression of the training samples in the space of Zhang Cheng. First, we perform spectral decomposition on SΦ to obtain QQ T =WDW T (W=[w1,w2,...,w n ]), we can get the following expression:
[0086] QQ T w i =λ i w i
[0087] Multiply both sides of the equation by Q T
[0088] (Q T Q)Q T w i =λQ T w i
[0089] Let R = Q T Q, obviously, Q T w i is the eigenvector of R, which is related to QQ T have the same eigenvalues. The spectral decomposition of the matrix R is: R = ΓEΓ T (Γ=[γ1,γ2,...,γ n ]), at this time, the eigenvector of the sample covariance matrix can be expressed as:
[0090]
[0091] Now, {w1,w2,...,w n} is the m-dimensional basis coordinate in the Hilbert space, projecting the mapped sample Φ(x) onto the eigenvector basis {w1,w2,...,w n}, we can get the transformed eigenvector y=[y1,y2,...,y m ] T ,in
[0092]
[0093] Step 5: Calculate the sample mean and combined covariance matrix after kernel transformation according to the following formula:
[0094]
[0095] in,
[0096] Step 6: Use PCA to simplify the data in Hilbert space. Calculate the covariance matrix of the data matrix Get its eigenvalues and eigenvectors, select the matrix composed of the eigenvectors corresponding to the "k" features with the largest eigenvalues (i.e., the largest variance), which is the data feature matrix after feature simplification The value of "k" is determined by the sample dimension (p) and the PCA dimensionality reduction ratio (dim), that is, k = floor (p*dim) (floor means rounding down).
[0097] Step 7: Select the eigenvectors corresponding to the "k" features with the largest eigenvalues (i.e., the largest variance) to form a data matrix after feature simplification Use the matrix to map the original data vector to a low-dimensional space to obtain the data in the low-dimensional space, as well as the eigenvalues and eigenvectors of the data matrix;
[0098]
[0099] in Is The projection matrix composed of the first k columns of eigenvectors, is the eigenvector matrix, is a diagonal matrix of eigenvectors;
[0100] Step 8: Directly replace the eigenvalues of the sample covariance matrix decomposition with the eigenvalues corresponding to the population covariance matrix, perform spectral correction on the sample covariance matrix structure, and obtain the estimator of the population covariance matrix. This function can be used to determine the category attributes of any new sample.
[0101]
[0102] This application is a new idea for classification and identification when the dimension of the observed sample is of the same order of magnitude as the sample size. On the basis of retaining the original covariance matrix structure, the overall covariance matrix estimator is spectrally corrected. At the same time, the PCA algorithm is used to map high-dimensional data to low-dimensional space, simplify the features of the original data, reduce the complexity of calculations, and improve classification performance.
[0103] Moreover, based on the existing classification model, this application further optimizes the algorithm structure: based on the original structure of the population covariance matrix estimator, the study of small Spiked eigenvalues is added, and the eigenvalues of the sample covariance matrix decomposition are decomposed into j Directly replaced by the eigenvalue λ corresponding to the overall covariance matrix j , the sample covariance matrix structure is spectrally corrected. The structural characteristics of the population covariance matrix are retained more completely, so that although the estimator is not a consistent estimate of the population covariance matrix, its estimation error can be controlled within a smaller ideal range.
[0104] Secondly, this application uses the kernel method to solve the problem that linearly inseparable data cannot be applied to linear classification models: given a mapping function, the nonlinearly separable sample data in the low-dimensional space is mapped to the high-dimensional space, and the kernel technology is introduced in the high-dimensional space to avoid complex and dense point set calculations, and finally a linear classifier is used in the high-dimensional space to solve the classification problem.
[0105] Finally, the present invention combines the use of PCA to screen data features, which reduces the data dimension while retaining the dimensional data containing the main information to improve the classification efficiency of the model.
[0106] The above is a handwritten digit classification method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding handwritten digit classification device. A handwritten digit classification device provided in this specification includes:
[0107] A data acquisition module, used for collecting handwritten digital image data with category labels and mapping the handwritten digital image data into a linearly separable Hilbert space;
[0108] Correction module, used to obtain the overall eigenvalue λ of handwritten digital image data in Hilbert space j , the overall eigenvalue λ j has a corresponding overall eigenvector, the overall eigenvalue λ j Used to characterize the discrete degree of handwritten digital image data in the direction of its corresponding overall feature vector;
[0109] It is used to project the handwritten digital data to the high variance direction through principal component analysis in the Hilbert space to obtain high variance data; calculate the covariance matrix of the high variance data, and decompose the covariance matrix to obtain the one-to-one corresponding projection eigenvalues l j and the projected eigenvector, the projected eigenvalue l j It is used to characterize the degree of dispersion of the projected high variance data in the direction of the corresponding projected eigenvector;
[0110] For the overall eigenvalue λ j Replace the projected eigenvalue l j Perform spectrum correction on the projection covariance matrix, and construct the spectrum correction discriminant analysis function based on the projection covariance matrix after spectrum correction:
[0111]
[0112] Among them, y is the handwritten digital image to be classified, u0 and u1 are the sample means of digital images of different categories respectively; is the projection covariance matrix after spectrum correction, π0 and π1 are the sample prior probabilities of different categories of digital images;
[0113] The classification module is used to calculate the category attributes of the handwritten digits to be classified using the spectrum correction discriminant analysis function.
[0114] The specific definition of the handwritten digit classification device can be found in the definition of the handwritten digit classification method above, which will not be repeated here. Each module in the above-mentioned handwritten digit classification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0115] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provided handwritten digit classification method.
[0116] This manual also provides Figure 3 The structural diagram of the computer device shown in FIG. Figure 3 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provided handwritten digit classification method.
[0117] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0118] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A handwritten digit classification method, characterized in that: include: Collecting handwritten digital image data with category labels, and mapping the handwritten digital image data into a linearly separable Hilbert space; In the Hilbert space, the overall eigenvalue λ of the handwritten digital image data is obtained j , the overall eigenvalue λ j has a corresponding overall eigenvector, the overall eigenvalue λ j Used to characterize the degree of discreteness in the direction of the overall feature vector corresponding to the handwritten digital image data; In the Hilbert space, the handwritten digital data is projected to the high variance direction through principal component analysis to obtain high variance data; the covariance matrix of the high variance data is calculated, and the covariance matrix is decomposed to obtain the one-to-one corresponding projection eigenvalues l j and the projected eigenvector, the projected eigenvalue l j Used to characterize the degree of discreteness in the direction of the projected eigenvector corresponding to the projected high variance data; Use the population eigenvalue λ j Replace the projected eigenvalue l j , perform spectrum correction on the projection covariance matrix, construct a spectrum correction discriminant analysis function based on the spectrum corrected projection covariance matrix, and use the spectrum correction discriminant analysis function to calculate the category attributes of the handwritten digits to be classified.
2. The handwritten digit classification method according to claim 1, characterized in that: The overall covariance matrix Σ of the handwritten digital image data with category labels is: Among them, σ 2 is an undetermined value used to measure the size of the non-spiked eigenvalue, I P is the p*p dimensional identity matrix, λ j The eigenvalue deviating from the center of the distribution is called the Spiked eigenvalue, v j is its corresponding eigenvector, v j T Yes j The transpose of ; k = {1,2}, r1 represents the number of large Spiked eigenvalues, and r2 represents the number of small Spiked eigenvalues.
3. The handwritten digit classification method according to claim 2, characterized in that: The projecting of the handwritten digital data to the high variance direction by principal component analysis specifically includes: Calculate the sample mean of data in Hilbert space H and the sample covariance matrix S Φ : Among them, n i represents the number of training samples of the i-th category, is the jth sample of the i-th class sample; n represents the total sample size, It is a sample Samples after projection to Hilbert space; Q = [Q0, Q1], Q0 is the sample vector after centering of class 0 samples; represents the sample mean of the i-th type of training samples in the Hilbert space; S Φ is the sample covariance matrix; T represents transpose; For the sample covariance matrix S Φ Perform kernel transformation to obtain the kernel transformation sample matrix and kernel transformed sample covariance matrix Kernel Transformation Sample Covariance Matrix Constructing the projection matrix Using the Projection Matrix Project the original handwritten digit data.
4. The handwritten digit classification method according to claim 3, characterized in that: The sample covariance matrix S Φ Perform kernel transformation to obtain the kernel transformation sample matrix and kernel transformed sample covariance matrix Specifically include: Decompose the sample covariance matrix SΦ spectrally to obtain the n-dimensional coordinates {w1,w2,...,w n }; Project the data in the Hilbert space H to the n-dimensional coordinates {w1,w2,...,w n }, we get the transformed eigenvector y=[y1,y2,...,y m ] T ; Based on the transformed feature vector y=[y1,y2,...,y m ] T Calculate the kernel transform sample matrix and kernel transformed sample covariance matrix in, is the jth sample of the i-th class sample after transformation, Representation sample Sample after projection into Hilbert space; are the covariance matrices of class 0 and class 1 obtained based on the transformed samples, respectively.
5. The handwritten digit classification method according to claim 4, characterized in that: The sample covariance matrix SΦ is decomposed to obtain n-dimensional coordinates {w1,w2,...,w n }, specifically including: The sample covariance matrix S Φ Perform spectral decomposition and obtain: QQ T =WDW T W=[w1,w2,…,w n ] D=diag[λ1,λ2,…,λ n ] Among them, w1~w n is the n eigenvector basis, λ1~λ n is the characteristic value; According to QQ T =WDW T Get QQ T w i =λ i w i ; On QQ T w i =λ i w i Multiply both sides by Q T , get (Q T Q)Q T w i =λ i Q T w i ; Let R = QQ T , then Q T w i is the eigenvector of R, which is related to QQ T have the same eigenvalues; Perform spectral decomposition of matrix R: R=GEG T Γ=[γ1,γ2,…,γ n ] Among them, E is the unit matrix, γ1~γ n is the decomposed vector; Then the eigenvector of the sample covariance matrix is expressed as: At this time, {w1,w2,...,w n } is the n-dimensional coordinate in the Hilbert space, projecting the mapped sample Φ(x) onto the eigenvector basis {w1,w2,...,w n }, we get the transformed eigenvector y=[y1,y2,...,y m ] T .
6. The handwritten digit classification method according to claim 3, characterized in that: The kernel-based transform sample covariance matrix Constructing the projection matrix Using the Projection Matrix Project the original handwritten digital data, including: Kernel transform sample covariance matrix Perform eigendecomposition to obtain the eigenvectors corresponding to the first k eigenvalues; Construct a projection matrix based on the eigenvectors corresponding to the first k eigenvalues Multiply the original handwritten digital data by the projection matrix to obtain the reduced dimension data matrix: in, is the eigenvector matrix, is a diagonal matrix of eigenvectors, yes The inverse of is the original handwritten digital data in Hilbert space, Is The projection matrix is composed of the first k columns of eigenvectors. The value of k is determined by the sample dimension p and the PCA dimensionality reduction ratio dim. k = floor(p*dim), where floor is rounded down. is the data matrix after dimensionality reduction.
7. The handwritten digit classification method according to claim 1, characterized in that: The spectrum correction discriminant analysis function is: Among them, y is the handwritten digit image to be classified, are the sample means of different categories of digital images in the Hilbert space; is the projection covariance matrix after spectrum correction, and π0 and π1 are the sample prior probabilities of different categories of digital images.
8. A handwritten digit classification device, characterized in that: include: Correction module, used to obtain the overall eigenvalue λ of handwritten digital image data in Hilbert space j , the overall eigenvalue λ j has a corresponding overall eigenvector, the overall eigenvalue λ j Used to characterize the discrete degree of handwritten digital image data in the direction of its corresponding overall feature vector; It is used to project the handwritten digital data to the high variance direction through principal component analysis in the Hilbert space to obtain high variance data; calculate the covariance matrix of the high variance data, and decompose the covariance matrix to obtain the one-to-one corresponding projection eigenvalues l j and the projected eigenvector, the projected eigenvalue l j It is used to characterize the degree of dispersion of the projected high variance data in the direction of the corresponding projected eigenvector; For the overall eigenvalue λ j Replace the projected eigenvalue l j Performing spectrum correction on the projection covariance matrix, and constructing a spectrum correction discriminant analysis function based on the spectrum corrected projection covariance matrix; The classification module is used to calculate the category attributes of the handwritten digits to be classified using the spectrum correction discriminant analysis function.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
A Handwritten Digit Recognition Method Based on Improved Distance Kernel Principal Component Analysis
CN103093235B
