Image classification method, device, equipment and storage medium

Abstract

The present application discloses an image classification method, device, equipment and storage medium, the method comprising: pre-establishing an image feature extraction model based on a multi-view uncorrelated discrimination analysis algorithm and a local discriminant typical correlation algorithm; when extracting features, the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features; obtains a set of images to be classified and inputs them into the image feature extraction model for feature extraction, inputs the first feature extraction result into the image classification model for image classification, and obtains the corresponding classification result. In this way, a local discriminant typical correlation algorithm is added to the existing multi-view uncorrelated discrimination analysis algorithm, so that when extracting features, the local correlation between images of the same type is enhanced, the local correlation between images of different types is weakened, and redundant information of image features is removed, so that the extracted image features are more significant, and the subsequent image classification effect is improved.

Description

Technical Field

[0001] The present application relates to pattern recognition technology, and in particular to an image classification method, device, equipment and storage medium. Background Art

[0002] Multi-view learning improves the accuracy of data prediction by learning from multiple perspectives. Therefore, multi-view learning has been widely used in the field of computer vision and has become a hot research direction in machine learning. Among them, Multiview Uncorrelated Linear Discriminant Analysis (MULDA) can be regarded as a combination of Canonical Correlation Analysis (CCA) and Uncorrelated Linear Discriminant Analysis (ULDA). The feature vectors extracted by MULDA in each view are uncorrelated in the common subspace, and the redundant information between views is effectively removed, while maximizing the discriminability of each view and the correlation between views.

[0003] However, since CCA is an unsupervised learning method and cannot utilize the class label information of sample views, sufficient discriminant information cannot be retained when MULDA is used to extract view features using CCA, resulting in reduced view classification performance.

[0004] Application Contents

[0005] In order to solve the above technical problems, the present application provides an image recognition method, device, equipment and storage medium.

[0006] The technical solution of this application is implemented as follows:

[0007] In a first aspect, an image classification method is provided, the method comprising:

[0008] Preliminarily establishing an image feature extraction model based on a multi-view uncorrelated discrimination analysis algorithm and a local discrimination canonical correlation algorithm; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction;

[0009] Acquire a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified;

[0010] Inputting the image set to be classified into the image feature extraction model to perform feature extraction, and obtaining a first feature extraction result;

[0011] The first feature extraction result is input into an image classification model to perform image classification, so as to obtain a classification result of the image set to be classified.

[0012] In the above scheme, the image feature extraction model is established in advance based on the multi-view unrelated discrimination analysis algorithm and the local discriminant canonical correlation algorithm, including: establishing the image feature extraction model based on the multi-view unrelated discrimination analysis algorithm and the local discriminant canonical correlation algorithm; obtaining an image sample set to be identified; wherein the image sample set contains image samples of the same type and image samples of different types; inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction, and obtaining at least two first feature extraction results; wherein each type of image sample includes multiple images; inputting the at least two first feature extraction results into the image classification model for image classification, and obtaining the classification results of the at least two types of image samples; based on the classification results and category labels of the at least two types of image samples, correcting the image feature extraction model to obtain a trained image feature extraction model.

[0013] In the above scheme, the image sample set includes a first sample set and a second sample set; wherein the second sample set is obtained by processing the first sample set based on a preset image processing strategy; the inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction to obtain at least two first feature extraction results includes: preprocessing the image samples in the first sample set to obtain at least two types of first sample data; preprocessing the image samples in the second sample set to obtain at least two types of second sample data; constructing a corresponding first projection matrix for each type of first sample data, and constructing a second projection matrix for each type of second sample data; wherein the first projection matrix and the second projection matrix of the same type of first type sample data and the second type sample data form a pair of projection matrices; using the same type of first projection matrix and the second projection matrix to respectively project the corresponding first sample data and the second sample data into a low-dimensional space, so as to realize feature extraction of the first sample data and the second sample data.

[0014] In the above scheme, the construction of the corresponding first projection matrix for each class of first sample data and the construction of the second projection matrix for each class of second sample data include: constructing the second local intra-class correlation matrix and the second local inter-class correlation matrix corresponding to each class of first sample data and the second sample data based on each class of first sample data and the second sample data, a predefined first local intra-class correlation matrix and a predefined first local inter-class correlation matrix; constructing the second inter-class dispersion matrix, the second intra-class dispersion matrix and the second total dispersion matrix for each class of first sample data and the second sample data based on each class of first sample data and the second sample data, a predefined first inter-class dispersion matrix, a first intra-class dispersion matrix and a first total dispersion matrix; using the second local intra-class correlation matrix, the second local inter-class correlation matrix, the second inter-class dispersion matrix, the second intra-class dispersion matrix and the second total dispersion matrix as preset functions Input parameters; wherein the preset function is constructed by the multi-view uncorrelated identification analysis algorithm and the local discriminant typical correlation algorithm; the preset function is solved to obtain multiple first eigenvalues ​​corresponding to each type of first sample data and multiple second eigenvalues ​​corresponding to the second sample data; for the multiple first eigenvalues, the first d first eigenvalues ​​are selected from the multiple first eigenvalues ​​in descending order, and d first eigenvectors corresponding to the first d first eigenvalues ​​are obtained; the first d second eigenvalues ​​are selected from the multiple second eigenvalues ​​in descending order, and d second eigenvectors corresponding to the first d second eigenvalues ​​are obtained; based on the d first eigenvectors of each type of first sample data, a first projection matrix corresponding to each type of first sample data is constructed; based on the d second eigenvectors of each type of second sample data, a second projection matrix corresponding to each type of second sample data is constructed.

[0015] In the above scheme, the use of the same type of first projection matrix and second projection matrix to project the corresponding first sample data and second sample data to a low-dimensional space respectively includes: projecting the corresponding first sample data to a low-dimensional space through the first projection matrix to obtain new first sample data; projecting the corresponding second sample data to a low-dimensional space through the second projection matrix to obtain new second sample data; based on a parallel fusion strategy, adding the new first sample data and the new second sample data to obtain a corresponding second feature extraction result; or, based on a serial fusion strategy, splicing the new first sample data and the new second sample data to obtain a corresponding second feature extraction result.

[0016] In the above scheme, the at least two second feature extraction results are input into the image classification model for image classification to obtain the classification results of the at least two categories of image samples, including: calculating the distance between the second feature extraction result of the target class sample image and M standard features in the standard feature library to obtain M distance values; wherein the target class sample image is any one of the at least two categories of image samples; sorting the M distance values ​​from small to large to obtain the first k distance values; wherein M is greater than k, and M and k are positive integers; taking the category information of the k standard features corresponding to the first k distance values; and determining the classification result of the target class sample image based on the category information of the k standard features.

[0017] In the above scheme, the step of inputting the image set to be classified into the image feature extraction model for feature extraction includes: preprocessing the image set to be classified to obtain a data set corresponding to the image set to be classified; and inputting the data set into the image feature extraction model for feature extraction.

[0018] In a second aspect, an image classification device is provided, the device comprising: an image feature extraction module and an image classification module;

[0019] Wherein, the image feature extraction module includes a creation unit, an acquisition unit and an extraction unit;

[0020] The creation unit is used to establish the image feature extraction model based on the multi-view uncorrelated discrimination analysis algorithm and the local discrimination typical correlation algorithm in advance; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction;

[0021] The acquisition unit is used to acquire a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified;

[0022] The extraction unit is used to input the image set to be classified into the image feature extraction model to perform feature extraction to obtain a first feature extraction result;

[0023] The image classification module is used to input the first feature extraction result into the image classification model to perform image classification, so as to obtain the classification result of the image set to be classified.

[0024] According to a third aspect, an image classification device is provided, comprising: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the aforementioned method when running the computer program.

[0025] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0026] By adopting the above technical scheme, an image feature extraction model is established in advance based on a multi-view uncorrelated discrimination analysis algorithm and a local discriminant typical correlation algorithm; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction; obtains a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified; the set of images to be classified is input into the image feature extraction model for feature extraction to obtain a first feature extraction result; the first feature extraction result is input into the image classification model for image classification to obtain a classification result of the set of images to be classified. In this way, a local discriminant typical correlation algorithm is added to the existing multi-view uncorrelated discrimination analysis algorithm, so that the local correlation between images of the same type is enhanced during feature extraction, the local correlation between images of different types is weakened, and redundant information of image features is removed, so that the extracted image features are more significant, and the subsequent image classification effect is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the process of the image classification method in the embodiment of the present application;

[0028] Figure 2 This is a schematic diagram of the image classification process in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the training process of the image feature extraction model in an embodiment of the present application;

[0030] Figure 4 This is a schematic diagram of the relationship between the recognition rate and the nearest neighbor parameter k in the ORL face recognition database in the embodiment of the present application;

[0031] Figure 5 This is a schematic diagram of the relationship between the recognition rate and the nearest neighbor parameter k in the AR face recognition database in the embodiment of the present application;

[0032] Figure 6 Schematic diagram of the recognition rate of each algorithm on the ORL face recognition database in the embodiment of the present application;

[0033] Figure 7 Schematic diagram of the recognition rate of each algorithm on the AR face recognition database in the embodiment of the present application;

[0034] Figure 8 This is a schematic diagram of the structure of the image classification device in the embodiment of the present application;

[0035] Fig. 9 Schematic diagram of the structure of the image classification device in the embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0037] The present application embodiment provides an image classification method. Figure 1 Schematic diagram of the process of the image classification method in the embodiment of the present application. Figure 1 As shown, the image classification method may specifically include:

[0038] Step 101: Preliminarily establishing an image feature extraction model based on a multi-view uncorrelated discrimination analysis algorithm and a local discrimination canonical correlation algorithm; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction;

[0039] It should be noted that in order to avoid the problem that the existing Multiview Uncorrelated Linear Discriminant Analysis (MULDA) algorithm cannot retain sufficient discriminant information when extracting image features, resulting in reduced image classification performance, this application proposes a new MULDA algorithm. The new MULDA algorithm is used to extract features from images, so that the local correlation between images of the same type is enhanced during feature extraction, the local correlation between images of different types is weakened, and the redundant information of image features is removed, thereby improving the image classification performance.

[0040] Specifically, a new MULDA algorithm is obtained by combining the existing MULDA algorithm with the Locality Discriminative Canonical Correlation Analysis (LDCCA) algorithm, wherein the LDCCA algorithm is used to enhance the local correlation between images of the same type and to weaken the local correlation between images of different types.

[0041] Here, the image feature extraction model is a trained image feature extraction model, which is used to extract features from images. The specific training process will be described in detail later.

[0042] Figure 2 Schematic diagram of the image classification process in the embodiment of the present application. Figure 2As shown, a complete image classification process is generally divided into four steps, namely data acquisition 201 (corresponding to step 102), data preprocessing 202 (corresponding to step 103), feature extraction 203 (corresponding to step 104) and recognition and classification 204 (corresponding to step 105).

[0043] Step 102: obtaining a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified;

[0044] It should be noted that the objects to be classified may be people, animals or objects, etc.

[0045] Here, the image set to be classified is obtained by shooting, or the image set to be classified can be obtained from a face recognition database, wherein the face recognition database includes ORL, AR, and the like.

[0046] Here, the image set to be classified can be obtained by shooting multiple different images of the same object to be classified, or by shooting a part of the image and then performing a preprocessing operation on the part of the image to obtain a processed image, and the unprocessed and processed images form the image set to be classified.

[0047] Alternatively, multiple different images of the same object to be classified are obtained from a face recognition database, or a portion of an image is obtained, and then a preprocessing operation is performed on the portion of the image to obtain a processed image, and the unprocessed and processed images are combined into an image set to be classified.

[0048] Step 103: inputting the image set to be classified into the image feature extraction model to perform feature extraction, and obtaining a first feature extraction result;

[0049] It should be noted that before inputting the image set to be classified into the image feature extraction model, the image set to be classified needs to be converted into image data that can be recognized by a computer.

[0050] In some embodiments, this step specifically includes: preprocessing the image set to be classified to obtain a data set corresponding to the image set to be classified; and inputting the data set into the image feature extraction model for feature extraction.

[0051] It should be noted that the preprocessing process is to convert the image into image data that can be recognized by the computer. In addition, it can also remove factors that affect the feature extraction and recognition and classification steps to provide unified and standardized data and improve recognition and classification performance.

[0052] Here, preprocessing generally includes smoothing, filtering, centering, grayscale processing and wavelet decomposition.

[0053] Since image shooting is affected by different factors, including lighting, shooting angle, etc., which have more or less influence on the recognition and classification process, the wavelet decomposition mentioned above can simultaneously extract the time domain information and frequency domain information of the image, which can eliminate the influence of these interference factors as much as possible, which is beneficial to the subsequent image classification. Among them, the low-frequency features of the original image are extracted through different forms of wavelet decomposition, and then the subsequent feature extraction and classification operations are performed.

[0054] Step 104: Input the first feature extraction result into an image classification model to perform image classification, and obtain a classification result of the image set to be classified.

[0055] Here, the image classification model is used to classify the image, wherein the image classification model can be established by a KNN algorithm, a SVM algorithm, or a CNN algorithm.

[0056] Here, the execution subject of step 101 to step 104 may be a processor of the image classification device.

[0057] In this way, a local discriminant typical correlation algorithm is added on the basis of the existing multi-view unrelated discrimination analysis algorithm, so that the local correlation between images of the same type is enhanced during feature extraction, the local correlation between images of different types is weakened, and the redundant information of image features is removed, so that the extracted image features are more significant and the subsequent image classification effect is improved.

[0058] The following is a detailed description of the training process of the image feature extraction model. Figure 3 Schematic diagram of the training process of the image feature extraction model in the embodiment of the present application, such as Figure 3 As shown, specifically,

[0059] Step 301: establishing an image feature extraction model based on a multi-view uncorrelated discrimination analysis algorithm and a local discriminant canonical correlation algorithm;

[0060] It should be noted that the image feature extraction model is a model established before training. The following steps specifically train the image feature extraction model.

[0061] Step 302: Obtain an image sample set to be identified; wherein the image sample set includes image samples of the same type and image samples of different types;

[0062] It should be noted that the same type of image samples or different types of image samples include multiple different images.

[0063] Step 303: inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction to obtain at least two second feature extraction results; wherein each type of image sample includes multiple images;

[0064] It should be noted that finding the projection matrix of the image sample set is the key to feature extraction. The process of projecting the sample data set corresponding to the image sample set through the projection matrix is ​​to extract features from the sample data set. Here, the projection matrix is ​​constructed through the image feature extraction model.

[0065] In some embodiments, the image sample set includes a first sample set and a second sample set; wherein the second sample set is obtained by processing the first sample set based on a preset image processing strategy; this step specifically includes: preprocessing the image samples in the first sample set to obtain at least two categories of first sample data; preprocessing the image samples in the second sample set to obtain at least two categories of second sample data; constructing a corresponding first projection matrix for each category of first sample data, and constructing a second projection matrix for each category of second sample data; wherein the first projection matrix and the second projection matrix of the same type of first category sample data and second category sample data form a pair of projection matrices; using the same type of first projection matrix and second projection matrix, the corresponding first sample data and second sample data are respectively projected into a low-dimensional space to achieve feature extraction of the first sample data and the second sample data.

[0066] Here, the image sample set includes a first sample set and a second sample set, the first sample set includes at least two categories of first sample sets, and the second sample set includes at least two categories of second sample sets. Each category of preprocessed first sample data and second sample data is used as input of the image feature extraction model to obtain multiple projection vectors corresponding to the first sample data and multiple projection vectors corresponding to the second sample data, respectively. The multiple projection vectors corresponding to the first sample data constitute a first projection matrix, and the multiple projection vectors corresponding to the second sample data constitute a second projection matrix.

[0067] In some embodiments, the constructing of a corresponding first projection matrix for each class of first sample data and the constructing of a second projection matrix for each class of second sample data include: constructing a second local intra-class correlation matrix and a second local inter-class correlation matrix corresponding to each class of first sample data and second sample data based on each class of first sample data and second sample data, a predefined first local intra-class correlation matrix and a first local inter-class correlation matrix; constructing a second inter-class dispersion matrix, a second intra-class dispersion matrix and a second total dispersion matrix for each class of first sample data and second sample data based on each class of first sample data and second sample data, a predefined first inter-class dispersion matrix, a first intra-class dispersion matrix and a first total dispersion matrix; using the second local intra-class correlation matrix, the second local inter-class correlation matrix, the second inter-class dispersion matrix, the second intra-class dispersion matrix and the second total dispersion matrix as preset functions. The method comprises the steps of: selecting the first d first eigenvalues ​​from the plurality of first eigenvalues ​​in descending order to obtain d first eigenvectors corresponding to the first d first eigenvalues; selecting the first d second eigenvalues ​​from the plurality of second eigenvalues ​​in descending order to obtain d second eigenvectors corresponding to the first d second eigenvalues; constructing a first projection matrix corresponding to each class of first sample data based on the d first eigenvectors of each class of first sample data; and constructing a second projection matrix corresponding to each class of second sample data based on the d second eigenvectors of each class of second sample data.

[0068] The specific steps of this embodiment are described as follows in conjunction with the corresponding algorithm:

[0069] Here, let X (i.e., the first sample data) and Y (i.e., the second sample data) be two centralized sample sets, X = [x1, x2, ..., x n ]=[X1,X2,…,X c ]∈R p×n , where x j ∈R p (1≤j≤n) represents a single sample, n represents the number of samples, p represents the dimension of the sample, and c represents the number of categories of the sample. Indicates that the i-th category has n i samples. Similarly, there is a sample set Y = [y1, y2, …, y n ]=[Y1,Y2,…,Y c ]∈R q×n , where q represents the dimension of the sample.

[0070] Regarding the LDCCA algorithm, specifically, the goal of LDCCA is to find a pair of projection vectors w x and w y , so that the correlation between the extracted features of the same type of k-nearest neighbor samples is maximized, and the correlation between the features of the different types of k-nearest neighbor samples is minimized, which is conducive to classification. Its identification criteria can be expressed as follows:

[0071]

[0072] in and Represents the covariance matrix of X and Y respectively. When calculating the canonical correlation coefficient, C xx and C yy The coefficients in Can be omitted. ζ is used for balance and The factor, and They are the local intra-class correlation matrix (i.e., the first local intra-class correlation matrix) and the local inter-class correlation matrix (i.e., the first local inter-class correlation matrix), which are defined as follows:

[0073]

[0074] In the above formula, N I (x i ) represents x i k nearest neighbors in the same class, that is, the nearest neighbors in the same class as x i The set of most similar points; N E (x i ) is x i The inter-class k nearest neighbors represent the neighbors in other classes that are similar to x i The set of most similar points. Similarly, N I (y i )N E (y i ) has a similar meaning. The algorithm uses the standard Euclidean distance to measure the similarity between data points.

[0075] Based on the first local intra-class correlation matrix and the first local inter-class correlation matrix, the first sample data and the second sample data of each class are input to obtain the constructed second local intra-class correlation matrix and the second local inter-class correlation matrix.

[0076] From the MLDA algorithm obtained by combining the CCA and LDA algorithms, the first inter-class dispersion matrix can be obtained and The first kind of internal dispersion matrix and The first total dispersion matrix and The specific details are not given here. Input the first sample data and the second sample data of each class to obtain the constructed second class dispersion matrix and The second kind of internal dispersion matrix and The second total dispersion matrix and in,

[0077] Next, the second local intra-class correlation matrix, the second local inter-class correlation matrix, the second inter-class discreteness matrix, the second intra-class discreteness matrix and the second total discreteness matrix are used as input parameters of a preset function (i.e., formula (1)), and the preset function is solved to obtain a pair of projection matrices.

[0078] The specific steps for solving the preset function are as follows:

[0079] Assume (w xj ,w yj ), (j=1,2,…,r-1) represents the vector pair corresponding to the first j largest eigenvalues ​​obtained by the MLDA algorithm. The multi-view irrelevant discriminant analysis and local discriminant canonical correlation combined algorithm of this application is to find the rth vector pair (w xr ,w yr ), and satisfy the irrelevant constraints at the same time, the optimization model of this application is as follows:

[0080]

[0081] η>0 is a fine-tuning parameter used to balance the relative importance between the LDCCA part and the MULDA part. As defined in equation (2). In formula (1), and The two constraints are transformed into one constraint The Lagrangian function corresponding to the optimization problem of formula (1) is as follows:

[0082]

[0083] Among them, λ, α j and β j is the Lagrange multiplier, and (5) is respectively xr and w yr Taking the partial derivative and making the result equal to 0, we get:

[0084]

[0085] (6) and (7) are multiplied on the left by and After adding, we can get:

[0086]

[0087] From equation (8), we can see that the value of λ is equal to the objective function value of equation (1). We can get r-1 formulas:

[0088]

[0089] Formula (9) is transformed into another form as follows:

[0090]

[0091] Let e ​​= [α1α2…α r-1 ] Τ , Then equation (10) can be expressed as follows:

[0092]

[0093] From formula (11), we can get:

[0094]

[0095] Similarly, multiply (7) on the left After a series of effective transformations,

[0096] Let b = [β1β2…β r-1 ] Τ , We can get:

[0097]

[0098] Set variables e and B x Substituting into (6) we can get:

[0099]

[0100] Substituting (12) into (14), we can obtain:

[0101]

[0102] Similarly, variables b and B y After substituting into (7) and undergoing a series of transformations, we can obtain:

[0103]

[0104] Separate order Then the original problem can be transformed into the following generalized eigenvalue solution problem:

[0105]

[0106] Solve the generalized characteristic equation and take the eigenvector pairs corresponding to the first d largest eigenvalues ​​to construct the projection matrix W x =[w x1 ,w x2 ,…,w xd ] (i.e. the first projection matrix) and W y =[w y1 ,w y2 ,…,w yd ] (i.e., the second projection matrix), where d satisfies the condition 1≤d≤min(p,q,c).

[0107] In some embodiments, the use of the same type of first projection matrix and second projection matrix to project the corresponding first sample data and second sample data to a low-dimensional space respectively includes: projecting the corresponding first sample data to the low-dimensional space through the first projection matrix to obtain new first sample data; projecting the corresponding second sample data to the low-dimensional space through the second projection matrix to obtain new second sample data; based on a parallel fusion strategy, adding the new first sample data and the new second sample data to obtain a corresponding second feature extraction result; or, based on a serial fusion strategy, splicing the new first sample data and the new second sample data to obtain a corresponding second feature extraction result.

[0108] Here, for the projection matrix W x and W y The feature extraction is performed through the parallel fusion strategy (i.e., FS1), which corresponds to formula (18), and the feature extraction is performed through the serial fusion strategy (i.e., FS2), which corresponds to formula (19).

[0109]

[0110] According to two different feature fusion strategies, feature extraction is performed on at least two types of first sample data and second sample data respectively to obtain at least two types of second feature extraction results.

[0111] Step 304: inputting the at least two second feature extraction results into an image classification model to perform image classification, and obtaining classification results of the at least two categories of image samples;

[0112] Here, the image classification model is used to classify the image, wherein the image classification model can be established by a KNN algorithm, a SVM algorithm, or a CNN algorithm.

[0113] In some embodiments, this step specifically includes: calculating the distance between the first feature extraction result of the target class sample image and M standard features in the standard feature library to obtain M distance values; wherein the target class sample image is any one of the at least two categories of image samples; sorting the M distance values ​​from small to large to obtain the first k distance values; wherein M is greater than k, and M and k are positive integers; obtaining the category information of the k standard features corresponding to the first k distance values; and determining the classification result of the target class sample image based on the category information of the k standard features.

[0114] Here, in order to study the influence of the neighbor parameter k of the local intra-class correlation matrix and the local inter-class correlation matrix involved in this application on the experimental recognition performance, experiments were conducted on the ORL and AR face recognition databases, respectively. The neighbor parameter k of the samples ranged from 1 to 5, and a separate LDCCA algorithm was used for comparison. At the same time, the serial feature fusion strategy of formula (19) was used to extract features. The experiment yielded Figure 4 Schematic diagram of the relationship between the recognition rate and the nearest neighbor parameter k on the ORL face recognition database, and Figure 5 Schematic diagram of the relationship between the recognition rate and the nearest neighbor parameter k on the AR face recognition database.

[0115] from Figure 4 and Figure 5 It can be seen that the recognition rate of the algorithm in this application is higher than that of the LDCCA algorithm, and the effect is more stable. Figure 4 and 5 This indicates that the recognition rate of the algorithm of this application is not sensitive to the value of k. In order to better compare the performance of the algorithms, the number of inter-class and intra-class neighbors k of this application is uniformly set to the number of training samples minus one.

[0116] Step 305: Based on the classification results and category labels of the at least two categories of image samples, the image feature extraction model is modified to obtain a trained image feature extraction model.

[0117] Based on the above steps, in order to verify the effectiveness of the algorithm of the present application, face recognition experiments were carried out on the ORL and AR face recognition databases, and the results were compared with MLDA, LDCCA, Fisherfaces and CCA, and compared with the MULDA algorithm and its improved algorithm (MULDA-m). The experiment was also carried out with the traditional CCA algorithm using the feature fusion strategy of formula (19) to extract features. The fine-tuning parameters η and ξ of the algorithm of the present application were determined between [0.001, 0.01, 0.1, 1, 10] through cross-validation, the parameter λ was 1, the parameter d satisfied the condition 1≤d≤min(p,q,c), and d was 40 and 45 on the ORL and AR face databases, respectively. All the algorithms mentioned above were tested 20 times in turn, and finally the nearest neighbor classifier was used for classification. The number of neighbors of the nearest neighbor classifier was 5, and the average recognition rate was calculated for comparison. The comparison results are shown in Figure 6 and Figure 7 It is known that the recognition rate of the algorithm in this application is the highest. Among them, Figure 6 Schematic diagram of the recognition rate of each algorithm on the ORL face recognition database in the embodiment of the present application. Figure 7 Schematic diagram of the recognition rate of each algorithm on the AR face recognition database in the embodiment of the present application.

[0118] In this way, a local discriminant typical correlation algorithm is added on the basis of the existing multi-view unrelated discrimination analysis algorithm, so that the local correlation between images of the same type is enhanced during feature extraction, the local correlation between images of different types is weakened, and the redundant information of image features is removed, so that the extracted image features are more significant and the subsequent image classification effect is improved.

[0119] An image classification device is provided in an embodiment of the present application. Figure 8 Schematic diagram of the structure of the image classification device in the embodiment of the present application. Figure 8 As shown, the device comprises: the device comprises: an image feature extraction module 80 and an image classification module 81;

[0120] The image feature extraction module 80 includes a creation unit 801, an acquisition unit 802, and an extraction unit 803;

[0121] The creation unit 801 is used to establish the image feature extraction model based on the multi-view uncorrelated discrimination analysis algorithm and the local discriminant canonical correlation algorithm in advance; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction;

[0122] The acquisition unit 802 is used to acquire a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified;

[0123] The extraction unit 803 is used to input the image set to be classified into the image feature extraction model to perform feature extraction to obtain a first feature extraction result;

[0124] The image classification module 81 is used to input the first feature extraction result into the image classification model to perform image classification, so as to obtain the classification result of the image set to be classified.

[0125] In some embodiments, the pre-establishing of an image feature extraction model based on a multi-view unrelated discrimination analysis algorithm and a local discriminant canonical correlation algorithm includes: establishing an image feature extraction model based on a multi-view unrelated discrimination analysis algorithm and a local discriminant canonical correlation algorithm; obtaining an image sample set to be identified; wherein the image sample set includes image samples of the same type and image samples of different types; inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction to obtain at least two first feature extraction results; wherein each type of image sample includes multiple images; inputting the at least two second feature extraction results into an image classification model for image classification to obtain classification results of the at least two types of image samples; based on the classification results and category labels of the at least two types of image samples, correcting the image feature extraction model to obtain a trained image feature extraction model.

[0126] In some embodiments, the image sample set includes a first sample set and a second sample set; wherein the second sample set is obtained by processing the first sample set based on a preset image processing strategy; the inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction to obtain at least two first feature extraction results includes: preprocessing the image samples in the first sample set to obtain at least two types of first sample data; preprocessing the image samples in the second sample set to obtain at least two types of second sample data; constructing a corresponding first projection matrix for each type of first sample data, and constructing a second projection matrix for each type of second sample data; wherein the first projection matrix and the second projection matrix of the same type of first type sample data and the second type sample data form a pair of projection matrices; using the same type of first projection matrix and the second projection matrix to respectively project the corresponding first sample data and the second sample data into a low-dimensional space to achieve feature extraction of the first sample data and the second sample data.

[0127] In some embodiments, the constructing of a corresponding first projection matrix for each class of first sample data and the constructing of a second projection matrix for each class of second sample data include: constructing a second local intra-class correlation matrix and a second local inter-class correlation matrix corresponding to each class of first sample data and second sample data based on each class of first sample data and second sample data, a predefined first local intra-class correlation matrix and a first local inter-class correlation matrix; constructing a second inter-class dispersion matrix, a second intra-class dispersion matrix and a second total dispersion matrix for each class of first sample data and second sample data based on each class of first sample data and second sample data, a predefined first inter-class dispersion matrix, a first intra-class dispersion matrix and a first total dispersion matrix; and converting the second local intra-class correlation matrix, the second local inter-class correlation matrix, the second inter-class dispersion matrix, the second intra-class dispersion matrix and the second total dispersion matrix into a matrix. The method comprises the steps of: using a matrix as an input parameter of a preset function; wherein the preset function is constructed by the multi-view uncorrelated discriminant analysis algorithm and the local discriminant typical correlation algorithm; solving the preset function to obtain a plurality of first eigenvalues ​​corresponding to each type of first sample data and a plurality of second eigenvalues ​​corresponding to the second sample data; selecting the first d first eigenvalues ​​from the plurality of first eigenvalues ​​in descending order, and obtaining d first eigenvectors corresponding to the first d first eigenvalues; selecting the first d second eigenvalues ​​from the plurality of second eigenvalues ​​in descending order, and obtaining d second eigenvectors corresponding to the first d second eigenvalues; constructing a first projection matrix corresponding to each type of first sample data based on the d first eigenvectors of each type of first sample data; and constructing a second projection matrix corresponding to each type of second sample data based on the d second eigenvectors of each type of second sample data.

[0128] In some embodiments, the use of the same type of first projection matrix and second projection matrix to project the corresponding first sample data and second sample data to a low-dimensional space respectively includes: projecting the corresponding first sample data to the low-dimensional space through the first projection matrix to obtain new first sample data; projecting the corresponding second sample data to the low-dimensional space through the second projection matrix to obtain new second sample data; based on a parallel fusion strategy, adding the new first sample data and the new second sample data to obtain a corresponding second feature extraction result; or, based on a serial fusion strategy, splicing the new first sample data and the new second sample data to obtain a corresponding second feature extraction result.

[0129] In some embodiments, the inputting the at least two second feature extraction results into the image classification model for image classification to obtain the classification results of the at least two categories of image samples includes: calculating the distance between the second feature extraction result of the target class sample image and M standard features in the standard feature library to obtain M distance values; wherein the target class sample image is any one of the at least two categories of image samples; sorting the M distance values ​​from small to large to obtain the first k distance values; wherein M is greater than k, and M and k are positive integers; obtaining the category information of the k standard features corresponding to the first k distance values; and determining the classification result of the target class sample image based on the category information of the k standard features.

[0130] In some embodiments, inputting the image set to be classified into the image feature extraction model for feature extraction includes: preprocessing the image set to be classified to obtain a data set corresponding to the image set to be classified; and inputting the data set into the image feature extraction model for feature extraction.

[0131] In this way, a local discriminant typical correlation algorithm is added on the basis of the existing multi-view unrelated discrimination analysis algorithm, so that the local correlation between images of the same type is enhanced during feature extraction, the local correlation between images of different types is weakened, and the redundant information of image features is removed, so that the extracted image features are more significant and the subsequent image classification effect is improved.

[0132] The present application also provides an image classification device, Fig. 9 Schematic diagram of the structure of the image classification device in the embodiment of the present application. Fig. 9 As shown, the image classification device includes: a processor 901 and a memory 902 configured to store a computer program that can be run on the processor;

[0133] The processor 901 is configured to execute the steps of the shooting control method in the aforementioned embodiment when running the computer program.

[0134] Of course, in practical applications, Fig. 9 As shown, the components in the image classification device are coupled together via a bus system 903. It is understood that the bus system 903 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 903 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig. 9 Various buses are labeled as bus system 903.

[0135] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0136] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0137] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0138] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0140] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0141] In addition, all functional units in the embodiments of the present application can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes: mobile storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0142] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0143] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0144] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0145] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image classification method, characterized in that: The method comprises: An image feature extraction model is established in advance based on a multi-view uncorrelated discrimination analysis algorithm and a local discrimination canonical correlation algorithm; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction; the image feature extraction model is: In formula (1), T is the matrix transposition operation, w xr and w yr is the rth vector pair corresponding to the first sample data and the second sample data of each category, and are the second inter-class discreteness matrices corresponding to the first sample data and the second sample data of each class, The second local intra-class correlation matrix and the second local inter-class correlation matrix corresponding to each class of the first sample data and the second sample data, and are the second total discreteness matrix corresponding to the first sample data and the second sample data of each category, w xj and w yj are the first r-1 vector pairs corresponding to the first sample data and the second sample data of each type; the first sample data is data obtained by processing the acquired image, the second sample data is data obtained by processing the processed image, and the processed image is an image obtained by processing the acquired image; Acquire a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified; Inputting the image set to be classified into the image feature extraction model to perform feature extraction, and obtaining a first feature extraction result; The first feature extraction result is input into an image classification model to perform image classification, so as to obtain a classification result of the image set to be classified.

2. The method according to claim 1, characterized in that: The image feature extraction model is established in advance based on the multi-view irrelevant discrimination analysis algorithm and the local discrimination canonical correlation algorithm, including: Initialize the image feature extraction model based on the multi-view uncorrelated discriminant analysis algorithm and the local discriminant canonical correlation algorithm; Acquire an image sample set to be identified; wherein the image sample set includes image samples of the same type and image samples of different types; Inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction to obtain at least two second feature extraction results; wherein each type of image sample includes multiple images; Inputting the at least two second feature extraction results into an image classification model to perform image classification, thereby obtaining classification results of the at least two categories of image samples; Based on the classification results and category labels of the at least two categories of image samples, the image feature extraction model is modified to obtain a trained image feature extraction model.

3. The method according to claim 2, characterized in that The image sample set includes a first sample set and a second sample set; wherein the second sample set is obtained by processing the first sample set based on a preset image processing strategy; The step of inputting at least two types of image samples in the image sample set into the image feature extraction model for feature extraction to obtain at least two second feature extraction results includes: Preprocessing the image samples in the first sample set to obtain at least two types of first sample data; Preprocessing the image samples in the second sample set to obtain at least two types of second sample data; Constructing a corresponding first projection matrix for each type of first sample data, and constructing a second projection matrix for each type of second sample data; wherein the first projection matrix and the second projection matrix of the first type of sample data and the second type of sample data of the same type form a pair of projection matrices; The first and second sample data are projected to a low-dimensional space using a first projection matrix and a second projection matrix of the same type, respectively, to achieve feature extraction of the first and second sample data.

4. The method according to claim 3, characterized in that The step of constructing a corresponding first projection matrix for each type of first sample data and constructing a second projection matrix for each type of second sample data includes: Based on each class of first sample data and second sample data, a predefined first local intra-class correlation matrix and a predefined first local inter-class correlation matrix, construct a second local intra-class correlation matrix and a second local inter-class correlation matrix corresponding to each class of first sample data and second sample data; Based on each class of first sample data and second sample data, a predefined first inter-class dispersion matrix, a first intra-class dispersion matrix and a first total dispersion matrix, construct a second inter-class dispersion matrix, a second intra-class dispersion matrix and a second total dispersion matrix of each class of first sample data and second sample data; The second local intra-class correlation matrix, the second local inter-class correlation matrix, the second inter-class dispersion matrix, the second intra-class dispersion matrix and the second total dispersion matrix are used as input parameters of a preset function; wherein the preset function is constructed by the multi-view uncorrelated discriminant analysis algorithm and the local discriminant canonical correlation algorithm; the construction method of the preset function is as shown in formula (1); Solving the preset function to obtain a plurality of first eigenvalues ​​corresponding to each type of first sample data and a plurality of second eigenvalues ​​corresponding to the second sample data; Select first d first eigenvalues ​​from the plurality of first eigenvalues ​​in descending order, and obtain d first eigenvectors corresponding to the first d first eigenvalues; Selecting first d second eigenvalues ​​from the plurality of second eigenvalues ​​in descending order, and obtaining d second eigenvectors corresponding to the first d second eigenvalues; Based on the d first eigenvectors of each type of first sample data, a first projection matrix corresponding to each type of first sample data is constructed; based on the d second eigenvectors of each type of second sample data, a second projection matrix corresponding to each type of second sample data is constructed.

5. The method according to claim 3, characterized in that: The method of using the first projection matrix and the second projection matrix of the same type to project the corresponding first sample data and the second sample data to the low-dimensional space respectively includes: Projecting the corresponding first sample data to a low-dimensional space by using the first projection matrix to obtain new first sample data; Projecting the corresponding second sample data to a low-dimensional space by using the second projection matrix to obtain new second sample data; Based on the parallel fusion strategy, the new first sample data and the new second sample data are added to obtain a corresponding second feature extraction result; Alternatively, based on a serial fusion strategy, the new first sample data and the new second sample data are concatenated to obtain a corresponding second feature extraction result.

6. The method according to claim 2, characterized in that The step of inputting the at least two second feature extraction results into an image classification model for image classification to obtain classification results of the at least two types of image samples includes: Calculate the distance between the second feature extraction result of the target class sample image and the M standard features in the standard feature library to obtain M distance values; wherein the target class sample image is any one of the at least two types of image samples; Sort the M distance values ​​from small to large to obtain the first k distance values; wherein M is greater than k, and M and k are positive integers; Obtaining category information of k standard features corresponding to the first k distance values; Based on the category information of the k standard features, a classification result of the target class sample image is determined.

7. The method according to claim 1, characterized in that The step of inputting the image set to be classified into the image feature extraction model for feature extraction comprises: Preprocessing the image set to be classified to obtain a data set corresponding to the image set to be classified; The data set is input into the image feature extraction model for feature extraction.

8. An image classification device, characterized in that: The device comprises: an image feature extraction module and an image classification module; Wherein, the image feature extraction module includes a creation unit, an acquisition unit and an extraction unit; The creation unit is used to establish an image feature extraction model based on a multi-view uncorrelated discrimination analysis algorithm and a local discrimination typical correlation algorithm in advance; the image feature extraction model enhances the local correlation between images of the same type, weakens the local correlation between images of different types, and removes redundant information of image features during feature extraction; the image feature extraction model is: In formula (1), T is the matrix transposition operation, w xr and w yr is the rth vector pair corresponding to the first sample data and the second sample data of each category, and are the second inter-class discreteness matrices corresponding to the first sample data and the second sample data of each class, The second local intra-class correlation matrix and the second local inter-class correlation matrix corresponding to each class of the first sample data and the second sample data, and are the second total discreteness matrix corresponding to the first sample data and the second sample data of each category, w xj and w yj are the first r-1 vector pairs corresponding to the first sample data and the second sample data of each type; the first sample data is data obtained by processing the acquired image, the second sample data is data obtained by processing the processed image, and the processed image is an image obtained by processing the acquired image; The acquisition unit is used to acquire a set of images to be classified; wherein the set of images to be classified includes different images of the same object to be classified; The extraction unit is used to input the image set to be classified into the image feature extraction model to perform feature extraction to obtain a first feature extraction result; The image classification module is used to input the first feature extraction result into the image classification model to perform image classification, so as to obtain the classification result of the image set to be classified.

9. An image classification device, characterized in that: The image classification device comprises: a processor and a memory configured to store a computer program that can be run on the processor. Wherein, the processor is configured to execute the steps of the method according to any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images