Picture feature extraction method and system, and device and medium

Abstract

The present invention relates to the technical field of image classification. Disclosed are a picture feature extraction method and apparatus, and a device and a medium. The method comprises the following steps: acquiring picture data having a label; calculating a within-class scatter matrix and a between-class scatter matrix of the picture data; performing spectrum correction and regularization on the within-class scatter matrix of the picture data; and constructing a conversion matrix, and using the conversion matrix to perform dimension reduction processing on a picture data sample, so as to extract picture features. In the method, a spiked model is introduced into the structure of a population covariance matrix of picture data; moreover, feature values of a within-class scatter matrix of the picture data are replaced with feature values of the population covariance matrix, and spectrum correction and regularization are performed on the structure of the within-class scatter matrix of the picture data, such that structure information of the population covariance matrix is more completely preserved, and an estimation error is controlled within a smaller range, thereby achieving a higher recognition effect in the high-dimensional condition of a 2D scene.

Description

Image feature extraction method, system, device and medium Technical Field

[0001] The present application relates to the field of image classification technology, and in particular to a method, system, device and medium for extracting image features. Background Art

[0002] With the development and advancement of science and technology, high-dimensional data has become common in many fields, and dimensionality reduction is key to solving these problems. Linear Discriminant Analysis (LDA) is a well-known supervised dimensionality reduction method. When working with matrix data, classic LDA typically flattens the matrix into a single vector. This results in a high-dimensional, low-data scenario, known as the small sample size problem. This leads to singularity issues in classic LDA, which necessitated the development of 2DLDA. While 2DLDA addresses the singularity issues of classic LDA, it remains highly sensitive to outliers and noise. To enhance the stability of 2DLDA, researchers both domestically and internationally have proposed various improvements, but these are all designed to address the situation where the matrix dimension is relatively low relative to the number of samples. However, with the passage of time, data dimensionality is increasing, most notably with the increasing number of pixels in images. This exposes 2DLDA to the same small sample size problem as classic LDA.

[0003] With the advancement of technology, more and more matrix data has emerged, the most typical of which is image data. When solving such problems, LDA typically flattens the matrix into a vector. However, this approach makes the data dimension very large relative to the number of samples, not only causing the small sample size problem but also increasing computational difficulty and cost. Therefore, in 2005, M Li proposed 2DLDA to address this problem. However, because 2DLDA uses the F-norm as a distance metric, it is still sensitive to outliers and noise. To address this issue, improvements have been made to 2DLDA. In 2020, Sifaou H et al. (2020) proposed the first improvement to the classic LDA in a high-dimensional context based on the Spiked Model (I-LDA). This method effectively avoids the difficulties of selecting regularization parameters in R-LDA and the highly inaccurate estimation of the covariance matrix and mean associated with each class in high-dimensional LDA, which leads to a significant decline in classification performance. HuaLi et al. (2022) proposed a new improved LDA based on the Spiked Model, which is superior to the classic R-LDA and can be comparable to the KNN and SVM classifiers with lower computational complexity. It is called spectral correction and regularized LDA (SRLDA).

[0004] It can be seen that the above improved models do not take into account the occurrence of high-dimensional situations in 2D scenes, so that in the extraction of image features, improper processing of 2D high-dimensional data leads to poor results in the subsequent model building process and large errors in image feature extraction.

[0005] Summary of the Invention

[0006] Based on this, it is necessary to provide an image feature extraction method, system, device and medium to address the above technical problems.

[0007] This manual adopts the following technical solutions:

[0008] This specification provides a method for extracting image features, including:

[0009] Obtaining labeled 2D image data samples, wherein the population covariance matrix ∑ of the image data samples has a Spiked structure;

[0010] Calculate the intra-class scatter matrix S of each sample of the image data w And the inter-class scatter matrix S of each sample b , and the image data class scatter matrix S w The eigenvalue λ and eigenvector u of each sample;

[0011] Replace the image data intra-class scatter matrix S with the sample eigenvalue l of the overall covariance matrix ∑ of the image data w The sample eigenvalue λ and the intra-class scatter matrix S of the image data sample w The structure is spectrally modified and regularized to have a Spiked structure, and the regularized intra-class scatter matrix S of the original image data is replaced by the intra-class scatter matrix w , let the new intra-class scatter matrix be

[0012] Construct a transformation matrix W, use the transformation matrix W to reduce the dimension of the image data samples, and extract the features of the image samples.

[0013] Preferably, the population covariance matrix ∑ of the image data sample has a Spiked structure, specifically including: when the dimension of the image data sample is of the same order of magnitude as the sample size, the distribution of the eigenvalue l of the population covariance matrix ∑ obeys the Marcenko-PasturLow curve, and the eigenvalues ​​deviating from the distribution center are respectively recorded as large and small Spiked eigenroots.

[0014] Preferably, the intra-class scatter matrix S of the image data w The structure is spectrally corrected and regularized, including:

[0015] Replace the intra-class scatter matrix S of the image data with the eigenvalue l of the overall covariance matrix ∑ of the image data w The eigenvalue λ of the image data is reconstructed and the intra-class scatter matrix S is constructed. w The structure of the new image data makes the intra-class scatter matrix S w The overall covariance matrix ∑ of the image data has the same structure as the intra-class scatter matrix S w Perform spectrum correction;

[0016] The intra-class scatter matrix S of the image data after spectrum correction w Perform regularization.

[0017] Preferably, the step of constructing a transformation matrix W and performing dimensionality reduction processing on the image data samples using the transformation matrix W to extract features of the image samples specifically includes:

[0018] Calculate the intra-class scatter matrix of image data The inverse matrix of is multiplied by the inter-class scatter matrix S b The eigenvalue ξ and eigenvector w of

[0019] Arrange the obtained eigenvalues ​​ξ in descending order, and take the eigenvectors w corresponding to the first K eigenvalues ​​ξ to form the transformation matrix W;

[0020] Combine the transformation matrix W with the image data sample matrix X ij Multiply the image data sample matrix X ij Project to a lower-dimensional space and obtain the projection matrix Y ij , that is, to perform dimensionality reduction processing on the image data samples.

[0021] Preferably, the intra-class scatter matrix S of the calculated image data is w and the inter-class scatter matrix S b , including the following steps:

[0022] Clean the image data and remove the same rows or columns in each image data;

[0023] Calculate the class sample mean and overall sample mean of the cleaned image data;

[0024] Use the class sample mean and the overall sample mean of the image data to calculate the intra-class scatter matrix S w and the inter-class scatter matrix S b .

[0025] Preferably, the image data intra-class scatter matrix S w The eigenvalue λ and eigenvector u are obtained by analyzing the intra-class scatter matrix S of the image data samples. w Obtained by spectral decomposition.

[0026] This specification provides an image feature extraction system, including:

[0027] An image data sample acquisition module is used to acquire labeled image data samples, and the overall covariance matrix ∑ of the image data samples has a Spiked structure;

[0028] Calculation module, used to calculate the intra-class scatter matrix S of image data w and the inter-class scatter matrix S b , and the image data class scatter matrix S w The eigenvalue λ and eigenvector u of

[0029] Image data intra-class scatter matrix S w The spectrum correction and regularization module is used to replace the image data class scatter matrix S with the eigenvalue l of the image data sample population covariance matrix ∑ w The eigenvalue λ of the image data is calculated, and the intra-class scatter matrix S is calculated. w Perform spectrum correction and regularization, and replace the original image data class scatter matrix S w , let the new intra-class scatter matrix be

[0030] The image feature extraction module is used to construct a transformation matrix W, use the transformation matrix W to reduce the dimension of the image data samples, and extract the features of the image samples.

[0031] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned image feature extraction method.

[0032] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned image feature extraction method is implemented.

[0033] This specification provides an image feature extraction method. Compared with the existing technology, its beneficial effects are as follows:

[0034] In the method for image feature extraction proposed in this specification, a Spiked model is introduced into the structure of the original image data sample population covariance matrix, and the eigenvalues ​​of the spectral decomposition of the intra-class scatter matrix of the image data sample are replaced with the eigenvalues ​​of the population covariance matrix of the image data sample. The intra-class scatter matrix structure of the image data sample is spectrally corrected and regularized, so that the structural information of the population covariance matrix is ​​more completely retained, and the estimation error is controlled within a smaller range, thereby achieving a higher recognition effect in high-dimensional situations in 2D scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] FIG1 is a flow chart of a method for extracting image features provided in this specification; DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] Example

[0039] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0040] As shown in Figure 1, this specification provides a method for image feature extraction - SR-2DLDA based on spectrum correction and regularization. The model quotes the theoretical results of large-dimensional random matrices, and proposes that when the dimension of the observed sample is of the same order of magnitude as the sample size (n→∞, p→∞, p / n→c), the distribution of the eigenvalues ​​of the population covariance matrix obeys the Marcenko-PasturLow curve. The present invention records the eigenvalues ​​that deviate from the center of the distribution as large and small Spiked eigenroots, respectively, and the model with this feature is recorded as a model of the spiked covariance matrix. The classification algorithm performed under the premise of this model structure includes the following steps:

[0041] Obtain image data and labels; clean the image data and remove the same rows or columns in each image data; calculate the class sample mean and the overall sample mean of the cleaned image data; use the class sample mean and the overall sample mean to calculate the intra-class scatter matrix S respectively w and the inter-class scatter matrix S b ; Calculate the eigenvalue λ and eigenvector u of the intra-class scatter matrix, and directly replace the eigenvalue λ of the intra-class scatter matrix decomposition with the eigenvalue l corresponding to the overall covariance matrix ∑; for the intra-class scatter matrix S w The structure performs spectrum correction and regularization, and replaces the original intra-class scatter matrix S w , let the new intra-class scatter matrix be and calculate The eigenvalues ​​ξ and eigenvectors w of the image are arranged in descending order, and the eigenvalues ​​ξ are selected to form the eigenvectors w corresponding to the first k eigenvalues ​​ξ to form the transformation matrix W; the transformation matrix is ​​used to reduce the dimension of the image data, that is, to extract the image features; finally, the images are classified according to the distance between each sample and the center of each class.

[0042] This embodiment specifically includes the following steps:

[0043] S1: Obtain labeled image data. The ORL face dataset is used in this case. The ORL face dataset contains 400 images of 40 different people. It was created by the Olivetti Research Laboratory in Cambridge, UK between April 1992 and April 1994. All images are stored in PGM format, which is a grayscale image with an image size of 92 width and 112 height. All images are taken against a dark uniform background and are of the frontal face (some with a slight side tilt). Each person's image is collected at different times, under different lighting conditions, with different facial expressions (eyes open / closed, smiling / not smiling), and facial details (wearing glasses / not wearing glasses).

[0044] The overall covariance matrix ∑ of the obtained labeled image data has a Spiked structure, which is

[0045] Among them, σ 2 is the covariance corresponding to the non-Spiked eigenvalue, I = I1∪I2, I1 = {1,…,r1}, I2 = {-r2,…,-1}, r = r1+r2, r1 represents the number of large Spiked eigenvalues, r2 represents the number of small Spiked eigenvalues, are column vectors and are mutually orthogonal.

[0046] S2: Calculate the intra-class scatter matrix S of the sample w and the inter-class scatter matrix S b and the intra-class scatter matrix S w The eigenvalues ​​λ and eigenvectors u of include:

[0047] Among them, S w represents the intra-class scatter matrix, S b represents the between-class scatter matrix, represents the jth sample of the i-th class, c represents the number of samples divided into c classes, that is, the number of sample classes, n represents the total sample size, n i represents the number of samples in the i-th category.

[0048] For the intra-class scatter matrix S w Perform spectral decomposition and get

[0049] Among them, λ i is the eigenvalue, u i is the feature vector.

[0050] S3: Replace the image data class scatter matrix S with the eigenvalue l of the image data sample population covariance matrix ∑ w The eigenvalue λ of the image data is calculated, and the intra-class scatter matrix S is calculated. w Perform spectrum correction and regularization, and replace the original image data class scatter matrix S w , let the new intra-class scatter matrix be

[0051] Replace the intra-class scatter matrix S with the eigenvalue l of the population covariance matrix ∑ w The eigenvalue λ of , and reconstruct the intra-class scatter matrix S w The structure of the class scatter matrix S w has the same structure as the overall covariance matrix, i.e., spectrum correction, so we get

[0052] Among them, σ 2 is the covariance corresponding to the non-Spiked eigenvalue, I = I1∪I2, I1 = {1,…,r1}, I2 = {-r2,…,-1}, r = r1+r2, r1 represents the number of large Spiked eigenvalues, r2 represents the number of small Spiked eigenvalues, are column vectors and are mutually orthogonal.

[0053] The intra-class scatter matrix after spectrum correction Regularize it, and we get

[0054] Among them, γ1,γ2 are regularization parameters, δ>0, In practical application, only The regularization parameters γ1 and γ2 can be confirmed.

[0055] S4: Construct a transformation matrix W, use the transformation matrix W to reduce the dimension of the image data sample, and extract the image sample features, including:

[0056] S41: Calculate the intra-class scatter matrix of image data The inverse matrix of is multiplied by the inter-class scatter matrix S b The eigenvalue ξ and eigenvector w of

[0057] in, is the intra-class scatter matrix of image data The inverse matrix of .

[0058] S42: Arrange the obtained eigenvalues ​​ξ in descending order, ξ1≥ξ2≥…≥ξ p , take the eigenvectors w corresponding to the first K eigenvalues ​​ξ to form the transformation matrix W=[w1,...,w k ].

[0059] S43: Combine the transformation matrix W with the image data sample matrix X ij Multiply the image data sample matrix X ij Project to a lower-dimensional space and obtain the projection matrix Y ij , Y ij =W T X ij

[0060] S5: Classify the images based on the distance between each sample and the center of each type of sample after dimensionality reduction, including:

[0061] S51: Use the transformation matrix W to transform the centers of various samples in the original image data Perform the conversion and calculate the matrix after the conversion of various sample centers

[0062] Where i=1,...,n.

[0063] S52: Calculate the projection matrix Y ij Each sample Y and each sample center distance;

[0064] S53: Classify sample Y into the class to which the center of the sample class closest to it belongs.

[0065] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0066] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for extracting image features, characterized in that: include: Obtaining labeled 2D image data samples, wherein the population covariance matrix ∑ of the image data samples has a Spiked structure; Calculate the intra-class scatter matrix S of each sample of the image data w And the inter-class scatter matrix S of each sample b , and the image data class scatter matrix S w The eigenvalue λ and eigenvector u of each sample; Replace the image data intra-class scatter matrix S with the sample eigenvalue l of the overall covariance matrix ∑ of the image data w The sample eigenvalue λ of the image data sample is calculated, and the intra-class scatter matrix S w The structure is spectrally modified and regularized to make it have a Spiked structure, and the regularized intra-class scatter matrix is ​​used to replace the intra-class scatter matrix S of the original image data w , let the new intra-class scatter matrix be Construct a transformation matrix W, use the transformation matrix W to reduce the dimension of the image data samples, and extract the features of the image samples.

2. A method for extracting image features as claimed in claim 1, characterized in that: The population covariance matrix ∑ of the image data sample has a Spiked structure, specifically including: when the dimension of the image data sample is of the same order of magnitude as the sample size, the distribution of the eigenvalue l of the population covariance matrix ∑ obeys the Marcenko-PasturLow curve, and the eigenvalues ​​deviating from the distribution center are respectively recorded as large and small Spiked eigenroots.

3. A method for extracting image features as claimed in claim 1, characterized in that: The intra-class scatter matrix S of the image data w The structure is spectrally corrected and regularized, including: Replace the intra-class scatter matrix S of the image data with the eigenvalue l of the overall covariance matrix ∑ of the image data w The eigenvalue λ of the image data is reconstructed and the intra-class scatter matrix S is constructed. w The structure of the new image data makes the intra-class scatter matrix S w The overall covariance matrix ∑ of the image data has the same Spiked structure to w Perform spectrum correction; The intra-class scatter matrix S of the image data after spectrum correction w Perform regularization.

4. A method for extracting image features as claimed in claim 1, characterized in that: The construction of the conversion matrix W, using the conversion matrix W to perform dimensionality reduction processing on the image data sample, and extracting the image sample features specifically includes: Calculate the intra-class scatter matrix of image data The inverse matrix of is multiplied by the inter-class scatter matrix S b The eigenvalue ξ and eigenvector w of Arrange the obtained eigenvalues ​​ξ in descending order, and take the eigenvectors w corresponding to the first K eigenvalues ​​ξ to form the transformation matrix W; The transformation matrix W and the image data sample matrix X ij Multiply the image data sample matrix X ij Project to a lower dimensional space and obtain the projection matrix Y ij , that is, to perform dimensionality reduction processing on the image data samples.

5. A method for extracting image features as claimed in claim 1, characterized in that: The intra-class scatter matrix S of the calculated image data w and the inter-class scatter matrix S b , including the following steps: Clean the image data and remove the same rows or columns in each image data; Calculate the class sample mean and overall sample mean of the cleaned image data; Use the class sample mean and the overall sample mean of the image data to calculate the intra-class scatter matrix S w and the inter-class scatter matrix S b .

6. A method for extracting image features as claimed in claim 1, characterized in that: The image data intra-class scatter matrix S w The eigenvalue λ and eigenvector u of the image data sample are obtained by calculating the intra-class scatter matrix S w Obtained by spectral decomposition.

7. A picture feature extraction system, characterized in that: include: An image data sample acquisition module is used to acquire labeled image data samples; The calculation module is used to calculate the intra-class scatter matrix S of the image data. w and the inter-class scatter matrix S b , and the image data class scatter matrix S w The eigenvalue λ and eigenvector u of The intra-class scatter matrix S of the image data w The spectrum correction and regularization module is used to scatter the matrix S within the image data class w The eigenvalue λ of is replaced by the eigenvalue l of the overall covariance matrix ∑ of the image data sample, and The intra-class scatter matrix S of the image data w Perform spectrum correction and regularization, and replace the original image data intra-class scatter matrix S w , let the new intra-class scatter matrix be The image feature extraction module is used to construct a transformation matrix W, use the transformation matrix W to reduce the dimension of the image data samples, and extract the image sample features.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 6 is implemented.

Images