A multi-angle face feature extraction method

Abstract

The present invention discloses a multi-angle face feature extraction method, which enriches the face angle span of multi-angle face recognition. Based on the original GRMF, algorithm improvements are made and parameter selection is carried out according to experiments. A computer program can be used to classify multi-angle face images, improve the accuracy of image classification, and be more applicable to the real environment. Using a head pose database, with two images for each pose, the purpose is to be able to train and test the algorithm on known and unknown faces. The people in the database may or may not wear glasses and have different skin colors. The background is natural, neutral and tidy to focus on facial operations. The face position on each image is marked in a separate text file. Under the same experimental data, the invention increases the amount of information of GRMF features in multi-angle face images, enriches the coverage of face angles, and improves the accuracy rate. It can effectively classify multi-angle face images and improve the classification efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a multi-angle face feature extraction method. Background Art

[0002] In recent years, with the expansion of terminals, the maturity of Internet technology and the popularization of electronic devices, video surveillance has not only brought convenience to people, but also provided strong protection for the safety of people's lives and property. Due to the angle problem of human faces in surveillance, a person can have multiple angles, and how to quickly identify based on local angles is a problem that needs to be solved.

[0003] In recent years, with the introduction of deep learning theory, many new face recognition methods have emerged. Gaussian Markov random field model (GMRF feature) has a good effect in extracting image texture features, but further research on the GMRF model is lacking, and there is no selective treatment of the characteristics of face recognition, such as ignoring the texture information of the face. The order of the Gaussian Markov random field needs to be selected in real time for different problems; at the same time, in the Gaussian Markov random field, when selecting spatial pixels, the traditional method is to perform linear calculations on the pixels; it is difficult to extract comprehensive complex texture analysis.

[0004] The existing GMRF feature extraction method has shortcomings in the feature description of multi-angle face images, such as too few local area divisions, missing texture information, and missing spatial information of facial texture structure, which leads to poor multi-angle face image classification effect. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-angle face feature extraction method to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a multi-angle face feature extraction method, comprising the following steps:

[0007] Step 1: Collect and import modules, take and store color face images;

[0008] Step 2: Grayscale processing module, grayscales the color face image and saves it, performs face segmentation and face Fourier transform on the grayscale face image, and forms frequency domain information of new processed face images in different regions;

[0009] Step 3: A generation module is used to pre-process the frequency domain information of the face images in different regions in step 1 into blocks to generate image blocks with different features;

[0010] Step 4: Calculation module. Different feature image blocks enter the GMRF model to extract feature points to form a feature vector to be estimated, and then the information of variance and mean is calculated to form the GMRF features after block processing.

[0011] Step 5: Merging module. The GMRF features extracted in Step 3 are stitched and combined, the face categories are labeled, and they are packaged together with the feature vectors in Step 2.

[0012] Step 6: Output module. The packaged feature vectors and labels in Step 4 are input into the SVM classifier for training and recognition, and the classification results are obtained by comparing with other multi-angle algorithms.

[0013] Further, in Step 2, the steps of extracting frequency domain information after graying the color face image are as follows: gray the face image and convert it into a two-dimensional array of double precision, and then perform Fourier transform on the converted two-dimensional array.

[0014] Further, in Step 3, the steps of dividing the feature image into blocks are as follows: read the features of the frequency domain image after Fourier transform, and find out the length and width of the feature image respectively;

[0015] Calculate the block point positions of each method according to the size of the feature image block. After calculation, take values and save the processed feature image according to different blocks.

[0016] Further, in Step 4, the steps of forming the GMRF features after block processing are as follows: first read the different feature image blocks saved in Step 3;

[0017] Then extract the feature points of the RMF features for different feature image blocks to become the feature point vectors to be calculated.

[0018] Further, in Step 4, after obtaining the feature vectors of each feature image block, splice them together to form the feature vector of an entire feature map and package and save it together with the label of the feature map for subsequent training.

[0019] Further, in Step 6, the steps of classifying multi-angle face images using feature vectors and SVM classification algorithms and comparing the classification accuracy by using the influence of the GRMF model of different orders on the classification to judge whether the classification result is correct are as follows:

[0020] Use the SVM multi-classification algorithm to classify multi-angle face images with feature vectors;

[0021] Then evaluate using the average accuracy result.

[0022] An apparatus for implementing a multi-angle face feature extraction method, the apparatus includes:

[0023] A collection and import module for capturing or importing color face images;

[0024] A grayscale processing module for grayscaling the color face image, saving it, performing face segmentation and face Fourier transform on the grayscaled face image, and forming the frequency domain information of the processed face images in different regions;

[0025] A generation module for performing block preprocessing on the frequency domain information of the face images in different regions in Step 1 to generate different feature image blocks;

[0026] A calculation module for different feature image blocks to enter the GMRF model to extract feature points to form an estimated feature vector, and then calculate the information of variance and mean to form the GMRF features after block processing;

[0027] A merging module for splicing and combining the GMRF features extracted in Step 3, labeling the face categories, and packing them together with the feature vectors in Step 2;

[0028] An output module for inputting the packed feature vectors and labels in Step 4 into an SVM classifier for training and recognition, and obtaining the classification result by comparing with other multi-angle algorithms.

[0029] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the multi-angle face feature extraction method are implemented.

[0030] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the multi-angle face feature extraction method are implemented.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] The present invention enriches the face angle span of multi-angle face recognition. Based on the original GRMF, algorithm improvements and parameter selection according to experiments are carried out. The computer program can be used to classify multi-angle face images, improve the accuracy of image classification, reduce the workload, and be more applicable to the real environment.

[0033] The face database selected for this invention is the head pose database, which is a benchmark of 2,790 monocular face images of 15 individuals, with translation and tilt angles ranging from -90 degrees to +90 degrees. For each individual, two series of 93 images can be used. The purpose of having two images for each pose is to be able to train and test the algorithm on known and unknown faces. The people in the database wear glasses or not and have different skin colors. The background is natural, neutral, and tidy to focus on facial operations. The position of the face on each image is marked in a separate text file.

[0034] This invention focuses on feature extraction of face pictures at different angles, has feature robustness compared to other algorithms, and also has good classification effects. The Vander Lugt correlator + zero-mean normalized cross-correlation realizes face recognition at different angles, with an overall recognition rate of 75.38%.

[0035] Face regions are extracted from the original multi-angle face images, and after extraction, the size is adjusted to 128 * 128 pixels. After gray-scaling and dividing the extracted face images into blocks, they are respectively input into the GRMF model, and after feature extraction, they are stitched together for classification and recognition. The classification effect at this time is 70.29%, which not only does not improve compared to the above method, but instead decreases by 5.09%. The reason for thinking may be that when adjusting the image size, the pixels in the image space are not symmetrical; it has a negative impact on feature point selection of GRMF.

[0036] After only gray-scaling and dividing the face images and then entering the second-order GRMF for feature extraction and then classification, the classification effect at this time is 77.39%, which is 2.01% higher than the above method. Since the low-order GRMF model pays too much attention to detail description and thus ignores the relationship between the whole. Therefore, this invention increases the order of GRMF on the basis of the second order. Experiments are respectively carried out on Gaussian Markov models of order 2, 4, 5, 8, and 14. The highest recognition rate reaches 94.78%, which is 17.39% higher than that of the second order; the experiment proves that the high-order GRMF model has a robust description ability for face angle changes.

[0037] Considering the influence of Fourier transform on face feature point selection again, the real part, imaginary part, modulus, and included angle after Fourier transform are respectively extracted and calculated; then GRMF feature extraction is carried out on them, and finally recognition and classification are carried out. The best among them is 99.57%.

[0038] To sum up: Under the condition of the same experimental data, this invention increases the information volume of GRMF features on multi-angle face images, enriches the coverage of face angles, and improves the accuracy. It can effectively classify multi-angle face images, improve the classification efficiency, and reduce the workload of manual classification. Description of the Drawings

[0039] Figure 1 is the flowchart of the method of the present invention;

[0040] Figure 2 is the schematic diagram of the GMRF feature point extraction method involved in the present invention;

[0041] Figure 3 is the schematic diagram of the extraction of the face area involved in the present invention;

[0042] Figure 4 is the schematic diagram of the features obtained by Fourier transform of the multi-angle face images involved in the present invention;

[0043] Figure 5 is the schematic diagram of the multi-block method of the multi-angle face images involved in the present invention;

[0044] Figure 6a is the comparison of the classification effects of the multi-angle face images at different GRMF orders in the method of extracting and reconstructing the face images in the present invention;

[0045] Figure 6b is the comparison of the classification effects of the multi-angle face images at different GRMF orders in directly extracting the face information in the present invention;

[0046] Figure 6c is the comparison of the classification effects of the multi-angle face images at different GRMF orders after Fourier transform in the present invention;

[0047] Figure 7 is the partial display of the multi-angle face database used in the present invention;

[0048] Figure 8 is the schematic diagram of the composition of the multi-angle face feature extraction device provided by the embodiment of the present invention;

[0049] Figure 9 is the schematic diagram of the structure of the computer device provided by the embodiment of the present invention. Detailed implementation manners

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] Please refer to Figures 1-9 , the present invention provides a technical solution: a multi-angle face feature extraction method, including the following steps:

[0052] Step 1: Collect the import module and capture and store color face images

[0053] Step 2: Grayscale the color face images and save them. Perform face segmentation and face Fourier transform on the grayscaled face images to form the frequency domain information of the processed face images in different regions;

[0054] Step 3: Perform block preprocessing on the frequency domain information of the face images in different regions in Step 1 to generate feature image blocks;

[0055] Step 4: The different feature image blocks enter the GMRF model to extract feature points to form the feature vectors to be estimated, and then calculate the information of variance and mean to form the GMRF features after block processing;

[0056] Step 5: Stitch and combine the GMRF features extracted in Step 3, label the face categories, and package them together with the feature vectors in Step 2;

[0057] Step 6: Input the packaged feature vectors and labels in Step 4 into the SVM classifier for training and recognition, and obtain the classification results by comparing with other multi-angle algorithms.

[0058] The face database selected in the present invention is the Head Pose Image Database (HPID), which is a benchmark of 2,790 monocular face images of 15 people. Its translation and tilt angles range from -90 degrees to +90 degrees. For each person, two series of 93 images (93 different poses) can be used. The purpose of having two images for each pose is to be able to train and test the algorithm on known and unknown faces. The people in the database may or may not wear glasses and have different skin colors. The background is natural, neutral, and tidy to focus on facial operations. The face positions on each image are marked in a separate text file.

[0059] The Vander Lugt correlator (VLC) + zero-mean normalized cross-correlation (ZNCC) achieved face recognition at different angles, with an overall recognition rate of 75.38%.

[0060] In the said Step 2, the steps of extracting the frequency domain information after grayscaling the color face images are as follows: Grayscale the face images and convert them into two-dimensional arrays of double precision, and then perform Fourier transform on the converted two-dimensional arrays. Two-dimensional continuous function Fourier transform is defined as: ;

[0061] If is a real function, and its Fourier transform is symmetric, i.e.: ;

[0062] The frequency spectrum of the Fourier transform is symmetric ;

[0063] Given , the inverse Fourier transform can be used to obtain : ;

[0064] Two-dimensional polar coordinate representation of the Fourier transform: ;

[0065] The amplitude or frequency spectrum is: ;

[0066] and are the real part and the imaginary part of respectively. The phase angle or phase spectrum is: ;

[0067] The power spectrum is: ;

[0068] Origin transformation of ;

[0069] Multiply by , and transform the origin of to in the rate coordinate system, which is the center of the region .

[0070] The correspondence between frequency components and the appearance of the image can be utilized. Some enhancement tasks that are difficult to express in the spatial domain become very common in the frequency domain; at the same time, filtering is more intuitive in the frequency domain, and it can explain certain properties of spatial domain filtering. For multi-angle face images, their angular transformation has completely different properties in spatial domain and frequency domain processing.

[0071] In the third step described above, the steps for partitioning the feature image are as follows: Read the features of the frequency domain image after Fourier transform, and find out the length and width of the feature image respectively; Calculate the partitioning points of each method according to the size of the partition, and after calculation, take values from the processed feature image according to different partitions and save them.

[0072] In step 4, the steps to form the Gaussian Markov random field (GMRF) features after block processing are as follows: First, read the block feature image patches saved in step 3; then extract the feature points of the GMRF features of different orders from them to obtain the feature point vectors to be calculated. The specific steps of the feature point extraction method are as follows:

[0073] Let be the set of points on the network, , assuming that the given texture is a zero-mean Gaussian random process, then the GMRF model can be represented by a linear equation, and this equation contains multiple unknown parameters. The specific formula is as follows: ;

[0074] Among them, represents the GMRF neighborhood of point , represents the coefficient, is the Gaussian noise sequence and its mean is zero. Because the pixel point neighborhood is symmetric, , the above formula can be written as: ;

[0075] Among them, is the point in the closed annular region . Applying equation (2-27) to each point in the region can obtain equations about and :

[0076]

[0077]

[0078] …

[0079]

[0080] …

[0081]

[0082] …

[0083] ;

[0084] Representing all the equations composed of in matrix form can be written as: ;

[0085] The above formula is the linear model of the Gaussian Markov random field, is about all matrix is the eigenvector to be estimated for the model. Taking the second-order GRMF model as an example:

[0086] ;

[0087] Neighborhood: Mean and variance: ;

[0088] For each pixel, we use the covariance matrix defined in a window W and the , parameters , and through the least squares estimation (LSE) : ;

[0089] ;

[0090] ;

[0091] ;

[0092] represents the number of pixels in the window W. Due to the symmetry of the correlation function, we can estimate 4 parameters. We can obtain a feature space .

[0093] The following estimation and solution formula can be obtained.

[0094] ;

[0095] ;

[0096] In the formula, is the asymptotically consistent estimation of the Gaussian Markov random field model parameters, and gives the mean square error of the GMRF model parameter estimation.

[0097] G matrix algorithm model for each order GMRF model (for a sliding window, each non-edge pixel can calculate a column vector, which is a column of the G matrix. Therefore, the number of rows of the G matrix is the number of non-edge pixels, and the number of columns is the number of estimated parameters (determined by the order of the GMRF model):

[0098] (1) Second-order model

[0099] ;

[0100] , is a 4-dimensional vector ;

[0101] (2) Fourth-order model

[0102] ;

[0103] , is a 10-dimensional vector , is a 10-dimensional vector;

[0104] (3) Fifth-order model

[0105] ;

[0106] , is a 10-dimensional vector , is a 12-dimensional vector;

[0107] (4) Eighth-order model

[0108] ; , is a 10-dimensional vector , is a 22-dimensional vector;

[0109] (5) Fourteenth-order model

[0110] ;

[0111] , is a 10-dimensional vector; , is a 40-dimensional vector.

[0112] In the fourth step described above, after obtaining the feature vectors of each feature image block, they are spliced together to form the feature vector of a whole feature map and saved together with the label of the feature map to prepare for subsequent training.

[0113] In the sixth step described above, the steps of classifying multi-angle face images using the feature vector and the SVM classification algorithm and comparing the classification accuracy by using the influence of the GRMF model of different orders on the classification to judge whether the classification result is correct are as follows:

[0114] Use the SVM multi-classification algorithm to classify multi-angle face images with the feature vector;

[0115] Then evaluate using the mean average precision (MAP) result, where:

[0116] 。

[0117] It should be noted that the present invention is based on the system framework of general face recognition, and modifies the extraction process of previous face features in four parts according to the characteristics of multi-angle human face images. The four parts are: feature preprocessing of multi-angle faces, multi-angle face image block division algorithm, GRMF for extracting feature points of face images, and GRMF for calculating feature vectors from feature points. The method process of the present invention is to classify multi-angle face images using a set computer execution program, aiming to expand the angle of face image classification and improve the working efficiency of face recognition and save human resources.

[0118] In terms of only changing the model representation of different orders of GRMF for feature point extraction, as the order of the Gaussian Markov random field increases, the classification effect is also proportional. When its order is 14, the best expected effect is achieved, reaching 94.78%; it is 3.33% and 0.29% higher than the 5-order and 8-order GRMF models respectively.

[0119] Considering the excellent effect of the properties of Fourier transform in images, the present invention also preprocesses multi-angle face images with Fourier transform. Classification experiments are respectively carried out on the real part, imaginary part, spectrum, phase angle and their combinations after Fourier transform. Under the same GRMF model, the recognition effect of the single real part is the highest, reaching 99.57%.

[0120] In addition, the classification effects under other classifiers are also compared, and the results show that this feature performs the best under the SVM multi-classifier among these three.

[0121] An apparatus for a method of extracting real multi-angle face features, the apparatus includes:

[0122] A collection and import module, used for photographing or importing color face images;

[0123] A grayscale processing module, used for grayscale processing and saving of color face images, performing face segmentation and face Fourier transform on the grayscale face images, and forming frequency domain information of face images in different processed regions;

[0124] A generation module, used for performing block preprocessing on the frequency domain information of face images in different regions in step one to generate different feature image blocks;

[0125] A calculation module, used for different feature image blocks to enter the GMRF model to extract feature points to form an estimated feature vector, and then calculating the information of variance and mean to form the GMRF feature after block processing;

[0126] A merging module, which is used to splice and combine the GMRF features extracted in the third step, label the face categories, and package them together with the feature vectors in the second step;

[0127] An output module, which is used to input the packaged feature vectors and labels in the fourth step into an SVM classifier for training and recognition, and obtain the classification result by comparing with other multi-angle algorithms.

[0128] A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the multi-angle face feature extraction method are implemented.

[0129] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-angle face feature extraction method are implemented.

[0130] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, physical database sharding, or other media used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0131] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-angle face feature extraction method, characterized in that: It includes the following steps: Step 1: Collect the import module and capture and store the color face image; Step 2: The grayscale processing module grayscales the color face image and saves it, performs face segmentation and face Fourier transform on the grayscaled face image to form the frequency domain information of the processed face images in different regions; Step 3: The generation module performs block preprocessing on the frequency domain information of the face images in different regions in Step 1 to generate different feature image blocks; Step 4: The calculation module, the different feature image blocks enter the GMRF model to extract feature points to form the feature vectors to be estimated, and then calculate the information of variance and mean to form the GMRF features after block processing; Step 5: The merging module splices and combines the GMRF features extracted in Step 3, labels the face categories, and packages them together with the feature vectors in Step 2; Step 6: The output module inputs the packaged feature vectors and labels in Step 4 into the SVM classifier for training and recognition, and obtains the classification result by comparing with other multi-angle algorithms.

2. The multi-angle face feature extraction method according to claim 1, wherein: In the said Step 2, the steps of extracting the frequency domain information after grayscaling the color face image are: grayscaling the face image and converting it into a double-precision two-dimensional array, and then performing Fourier transform on the converted two-dimensional array.

3. A multi-angle face feature extraction method according to claim 1, characterized in that: In the said Step 3, the steps regarding the block division of the feature image are: reading the features of the frequency domain image after Fourier transform, and respectively obtaining the length and width values of the feature image; Calculating the block division points of each method according to the sizes of different feature image blocks, and after calculation, taking values and saving the processed feature image according to different block divisions.

4. A multi-angle face feature extraction method according to claim 1, characterized in that: In the said Step 4, the steps of forming the GMRF features after block processing are: first reading the different feature image blocks saved in Step 3; Then extracting the feature points of the RMF features from the different feature image blocks to become the feature point vectors to be calculated.

5. The multi-angle face feature extraction method according to claim 4, characterized in that: In the said Step 4, after obtaining the feature vectors of each feature image block, splicing them together to form the feature vector of a whole feature map and packaging it together with the label of the feature map for subsequent training.

6. The multi-angle face feature extraction method according to claim 1, characterized in that: In the said Step 6, the steps of classifying the multi-angle face images using the feature vectors and the SVM classification algorithm and comparing the classification accuracy by using the influence of the GRMF model of different orders on the classification to judge whether the classification result is correct are: Using the SVM multi-classification algorithm to classify the feature vectors for multi-angle face images; Then evaluating using the average accuracy result.

7. An apparatus for implementing the multi-angle face feature extraction method described in claim 1, the apparatus includes: The collection and import module is used to capture or import the color face image; The grayscale processing module is used to grayscale the color face image and save it, perform face segmentation and face Fourier transform on the grayscaled face image to form the frequency domain information of the processed face images in different regions; The generation module is used to perform block preprocessing on the frequency domain information of the face images in different regions in Step 1 to generate different feature image blocks; A calculation module, which is used for different feature image blocks to enter the GMRF model to extract feature points to form a feature vector to be estimated, and then calculate the information of variance and mean to form the GMRF features after block processing; A merging module, which is used for splicing and combining the GMRF features extracted in step three, labeling the face categories, and packing them together with the feature vectors in step two; An output module, which is used to input the feature vectors and labels packed in step four into the SVM classifier for training and recognition, and obtain the classification result by comparing with other multi-angle algorithms.

Images