An automatic face feature recognition system and method based on image processing

Abstract

The application provides a kind of face feature automatic identification system and method based on image processing, it is related to face recognition technical field, the present application is generated by using face structure information and is guided multi-scale feature fusion and illumination decomposition by heat map, effectively enhance the feature expression of key area of five organs, while inhibiting background interference, improved Retinex algorithm combines heat map and realizes adaptive illumination component estimation, overcome the halo effect of traditional method in edge area, ensure the illumination invariance of reflection component feature, by generating accurate occlusion mask from reflection component map, and extracting local discriminative feature to unoccluded area, it can effectively deal with the challenge brought by occlusion, finally, the weighted fusion of global reflection component feature and local discriminative feature, combined with principal component dimension reduction and nearest neighbor classification, constitute clear, collaborative work identification process, realize reliable identity recognition under the double difficulty of illumination change and local occlusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and particularly relates to a face feature automatic recognition system and method based on image processing. BACKGROUND

[0002] In the prior art, the face feature automatic recognition method usually faces many challenges in complex environments. The traditional method often directly extracts and matches the features of the input image, which is difficult to effectively deal with interference factors such as light changes. The conventional recognition process usually includes face detection, feature extraction and classification recognition three basic steps, and the feature extraction link depends on global features or single local feature descriptors. For the light problem, some technical solutions try to use histogram equalization or homomorphic filtering for preprocessing, but these methods are easy to lose image details or introduce noise. Although the multi-scale feature fusion strategy can combine information at different levels, how to effectively integrate the structural priori knowledge is still a difficulty. The feature decomposition method in the prior art often processes the light and reflection components independently, and lacks collaborative optimization with the overall recognition process. Therefore, a face recognition scheme that can organically combine multi-source information, has good interpretability and can adapt to complex scenes is needed.

[0003] Therefore, it is necessary to provide a face feature automatic recognition system and method based on image processing to solve the above technical problems. SUMMARY

[0004] To solve the above technical problems, the present application provides a face feature automatic recognition system and method based on image processing, which can combine multi-source information and adapt to complex scenes.

[0005] The present application provides a face feature automatic recognition method based on image processing, comprising:

[0006] S1: Extracting multi-scale feature maps and face structure information of the face image to be recognized, generating a comprehensive Gaussian heat map based on the face structure information, and fusing the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heat map;

[0007] S2: Decomposing the joint feature map into a light component map and a reflection component map through a Retinex algorithm combined with the comprehensive Gaussian heat map, and extracting a reflection component feature from the reflection component map;

[0008] S3: Based on the reflection component map, generating an occlusion mask through adaptive threshold segmentation and morphological operation, and performing spatial weighting on the joint feature map through the occlusion mask to obtain a regional occlusion feature map;

[0009] S4: Extracting local discriminative features of the unoccluded region in the regional occlusion feature map;

[0010] S5: After principal component analysis dimension reduction is performed on the reflection component feature and the local discriminative feature, the reflection component feature and the local discriminative feature are fused to generate a comprehensive feature vector; based on the comprehensive feature vector and a pre-constructed face feature library, a recognition result is generated through a nearest neighbor classifier.

[0011] Preferably, in step S1, the generating step of the comprehensive Gaussian heat map comprises:

[0012] A pixel grid with the same resolution as the face image to be recognized is constructed, and the key point coordinates in the face structure information are mapped into the pixel grid, each minimum non-divisible element in the pixel grid representing a pixel point;

[0013] For a single key point, the squared distance of each pixel point in the pixel grid to the key point is calculated to obtain a distance matrix corresponding to the key point;

[0014] The distance matrix corresponding to each key point is processed by a Gaussian function to obtain a Gaussian heat map corresponding to each key point, and all the Gaussian heat maps are superimposed and fused to be normalized into a comprehensive Gaussian heat map.

[0015] Preferably, in step S1, the fusion step of the joint feature map comprises:

[0016] The comprehensive Gaussian heat map with the original resolution is down-sampled to the resolution corresponding to the feature map of each scale through bilinear interpolation;

[0017] The down-sampled comprehensive Gaussian heat map is pixel-by-pixel multiplied with the feature map of the corresponding scale to obtain a spatial weighted feature map of each scale;

[0018] All the spatial weighted feature maps are spliced in the channel dimension to obtain a preliminary joint feature map;

[0019] The preliminary joint feature map is subjected to principal component analysis dimension reduction processing to generate a joint feature map.

[0020] Preferably, in step S2, the decomposition step of the illumination component map and the reflection component map comprises:

[0021] For each pixel position in the joint feature map, the heat value of the pixel at the corresponding position in the comprehensive Gaussian heat map is read, and the scale parameter of each pixel position in the joint feature map is calculated through a pre-set mapping function;

[0022] Based on the scale parameter of each pixel position in the joint feature map, a Gaussian kernel is generated in combination with a pre-set Gaussian kernel generation formula;

[0023] The joint feature map is subjected to Gaussian filtering processing through the Gaussian kernel to obtain an illumination component map;

[0024] The reflection component image is separated by performing a logarithmic domain subtraction operation on the combined feature map and the illumination component map.

[0025] Preferably, in step S2, the reflection component feature is obtained by calculating a gray gradient direction histogram of the reflection component image and counting the direction distribution thereof.

[0026] Preferably, in step S3, the generating of the occlusion mask comprises:

[0027] The reflection component image is converted into a gray image, a weighted average value of pixels within a local neighborhood window of each pixel in the gray image is calculated based on a preset local neighborhood window, and a preset constant is subtracted to obtain a local threshold value of each pixel, if the pixel value is greater than the local threshold value, the pixel value is assigned as 255, otherwise, the pixel value is assigned as 0 to generate an initial binary mask;

[0028] The initial binary mask is dilated using a rectangular structure of a preset size, and then the dilated initial binary mask is eroded using the rectangular structure to obtain a secondary binary mask;

[0029] All connected regions are identified by performing contour finding on the secondary binary mask, and the pixel area of each connected region is calculated to screen out a region with the largest area as a target occlusion region;

[0030] A blank mask image with the same size as the image to be recognized is created, the target occlusion region is filled as 255 and the remaining regions are filled as 0 on the blank mask image to generate an occlusion mask.

[0031] Preferably, in step S5, the step of generating the comprehensive feature vector comprises:

[0032] The reflection component feature after dimension reduction and the local discriminative feature after dimension reduction are weighted according to a preset weight coefficient, and then concatenated in the vector dimension to generate a comprehensive feature vector.

[0033] Preferably, in step S5, before the recognition result is generated by the nearest neighbor classifier, the step further comprises:

[0034] The comprehensive feature vector is subjected to L2 norm normalization processing to make its module length 1.

[0035] Preferably, in step S5, based on the comprehensive feature vector and the pre-constructed face feature library, the step of generating the recognition result by the nearest neighbor classifier comprises:

[0036] The Euclidean distance between the comprehensive feature vector and each template feature vector in the pre-constructed face feature library is calculated, and the identity of the template corresponding to the minimum Euclidean distance is taken as the preliminary recognition result.

[0037] The application provides a face feature automatic recognition system based on image processing, comprising:

[0038] A feature extraction and fusion module is configured to extract a multi-scale feature map and face structure information of a face image to be recognized, generate a comprehensive Gaussian heat map based on the face structure information, and fuse the multi-scale feature map into a joint feature map based on the comprehensive Gaussian heat map;

[0039] An image decomposition module is configured to decompose the joint feature map into an illumination component map and a reflection component map by a Retinex algorithm combined with the comprehensive Gaussian heat map, and extract a reflection component feature from the reflection component map;

[0040] An occlusion processing module is configured to generate an occlusion mask based on the reflection component map by adaptive threshold segmentation and morphological operation, and perform spatial weighting on the joint feature map by the occlusion mask to obtain a regional occlusion feature map;

[0041] A local feature extraction module is configured to extract a local discriminative feature of an unoccluded region in the regional occlusion feature map;

[0042] A recognition decision module is configured to fuse the reflection component feature and the local discriminative feature after principal component analysis and dimension reduction to generate a comprehensive feature vector, and perform recognition by a nearest neighbor classifier based on the comprehensive feature vector and a pre-constructed face feature library to generate a recognition result.

[0043] Compared with related technologies, the face feature automatic recognition system and method based on image processing provided by the application has the following beneficial effects:

[0044] The heat map generated based on the face structure information is used to guide the multi-scale feature fusion and illumination decomposition, which effectively enhances the feature expression of the key regions of the five organs, and suppresses the background interference. The improved Retinex algorithm combined with the heat map realizes adaptive illumination component estimation, overcomes the halo effect in the edge region of the traditional method, and ensures the illumination invariance of the reflection component feature. The accurate occlusion mask is generated from the reflection component map, and the local discriminative feature is extracted from the unoccluded region, so that the system can effectively cope with the challenge brought by the mask and other occlusions. Finally, the weighted fusion of the global reflection component feature and the local discriminative feature, combined with the principal component dimension reduction and the nearest neighbor classification, constitutes a clear and collaborative recognition process. Reliable identity recognition is realized under the dual difficulties of illumination change and local occlusion. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The application provides a flowchart of a face feature automatic recognition method based on image processing;

[0046] Figure 2A module structure diagram of a face feature automatic recognition system based on image processing. DETAILED DESCRIPTION

[0047] The application will be described in further detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the application and not to limit the application. In addition, it should be noted that, for the sake of description, only the parts related to the application are shown in the drawings, not all the structures. Furthermore, the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0048] In addition, it should be noted that, for the sake of description, only the parts related to the application are shown in the drawings, not all the structures. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The processes can correspond to methods, functions, procedures, subroutines, etc.

[0049] Embodiment one

[0050] A face feature automatic recognition method based on image processing, in the specific implementation process, as shown in the figure, Figure 1 a flowchart of a face feature automatic recognition method based on image processing is shown, which includes:

[0051] Step S1: Extracting the multi-scale feature map and the face structure information of the face image to be recognized, generating a comprehensive Gaussian heat map based on the face structure information, and fusing the multi-scale feature map into a joint feature map based on the comprehensive Gaussian heat map.

[0052] Specifically, in step S1, the generation step of the comprehensive Gaussian heat map includes:

[0053] A pixel grid with the same resolution as the face image to be recognized is constructed, and the key point coordinates in the face structure information are mapped into the pixel grid, and each minimum indivisible element in the pixel grid represents a pixel point;

[0054] For a single key point, the square distance of each pixel point in the pixel grid to the key point is calculated to obtain the distance matrix corresponding to the key point;

[0055] The distance matrix corresponding to each key point is processed by a Gaussian function to obtain the Gaussian heat map corresponding to each key point, and all the Gaussian heat maps are superimposed and fused to be normalized into a comprehensive Gaussian heat map.

[0056] Specifically, in step S1, the fusion step of the joint feature map includes:

[0057] down-sampling the comprehensive Gaussian heat map of the original resolution to the resolution corresponding to the feature map of each scale by bilinear interpolation;

[0058] pixel-by-pixel multiplying the down-sampled comprehensive Gaussian heat map with the feature map of the corresponding scale to obtain a spatially weighted feature map of each scale;

[0059] splicing all the spatially weighted feature maps in the channel dimension to obtain a preliminary joint feature map;

[0060] performing principal component analysis dimension reduction processing on the preliminary joint feature map to generate a joint feature map.

[0061] In the specific implementation process, first, multi-scale feature maps of the face image to be recognized are extracted by a convolutional neural network. For example, the multi-scale feature maps contain feature information of three different scales, and the scales are one-half, one-quarter and one-eighth of the original image, respectively. At the same time, face structure information is obtained by a 68-point face key point detection algorithm. The face structure information is composed of the horizontal and vertical coordinates of the 68 key points. Then, a comprehensive Gaussian heat map is generated based on the face structure information. First, a pixel grid with the same resolution as the face image to be recognized is constructed. Each minimum unit of the pixel grid corresponds to a pixel point. The coordinates of the 68 key points are mapped into the pixel grid. The squared distance from all pixel points in the pixel grid to each key point is calculated to generate a distance matrix corresponding to the key point. Then, a Gaussian function with a standard deviation of 3 is used to process each distance matrix to obtain a Gaussian heat map corresponding to each key point. Finally, all 68 Gaussian heat maps are superimposed, and each element in the superimposed matrix is normalized to make the numerical range between 0 and 1, thereby generating the final comprehensive Gaussian heat map. Finally, the multi-scale feature maps are fused into a joint feature map based on the comprehensive Gaussian heat map. For example, the comprehensive Gaussian heat map of the original resolution is down-sampled to the same resolution as the one-half, one-quarter and one-eighth scale feature maps by bilinear interpolation, respectively. Then, the down-sampled heat maps of each scale are pixel-by-pixel multiplied with the feature maps of the corresponding scale to obtain spatially weighted feature maps of three scales. The three spatially weighted feature maps are spliced in the channel dimension to form a high-dimensional preliminary joint feature map. Finally, principal component analysis dimension reduction processing is performed on the preliminary joint feature map. For example, the cumulative variance contribution rate threshold is set to 95%, the main feature components are retained, and the final joint feature map is generated.

[0062] Step S2: decompose the joint feature map into an illumination component map and a reflection component map by a Retinex algorithm combined with a comprehensive Gaussian heat map, and extract the reflection component feature from the reflection component map.

[0063] Specifically, in step S2, the decomposition step of the illumination component map and the reflection component map includes:

[0064] For each pixel position in the joint feature map, read the heat value of the pixel at the corresponding position in the comprehensive Gaussian heat map, and calculate the scale parameter of each pixel position in the joint feature map by a preset mapping function;

[0065] Based on the scale parameter of each pixel position in the joint feature map, generate a Gaussian kernel combined with a preset Gaussian kernel generation formula;

[0066] Gaussian filtering is performed on the joint feature map by the Gaussian kernel to obtain the illumination component map;

[0067] The reflection component map is separated by the logarithmic domain subtraction operation of the joint feature map and the illumination component map.

[0068] Specifically, in step S2, the reflection component feature is obtained by calculating the gray gradient direction histogram of the reflection component map and counting the direction distribution.

[0069] In the implementation process, first, for each pixel position in the joint feature map, the heat value of the pixel at the corresponding position in the comprehensive Gaussian heat map is read, the heat value is a floating point number between 0 and 1, and the scale parameter of each pixel position is calculated through a preset linear mapping function, the maximum scale parameter is set to 15 and the minimum scale parameter is set to 3, and the specific calculation method is to subtract the product of the difference between the heat value and the maximum and minimum scale parameters from the maximum scale parameter; based on the scale parameter of each pixel position, a corresponding Gaussian kernel is generated in combination with a preset Gaussian kernel generation formula, the size of the Gaussian kernel is dynamically determined according to the scale parameter, and the calculation formula is that the kernel size is equal to 2 times the scale parameter multiplied by 3, and the result is rounded up and then added by 1; the generated Gaussian kernel is used to perform pixel-by-pixel adaptive Gaussian filtering on the joint feature map to obtain an illumination component map conforming to the Retinex model, and in the filtering process, the Gaussian kernel generated by using the corresponding scale parameter of each pixel is used for neighborhood weighted average; then, the reflection component map is separated through the logarithmic domain subtraction operation of the joint feature map and the illumination component map in strict accordance with the mathematical expression of the Retinex algorithm, specifically, the joint feature map and the illumination component map are converted to the logarithmic domain with the natural constant as the base, then subtracted, and then the result is converted back to the linear domain through the exponential operation to obtain the reflection component map; finally, the reflection component feature is extracted from the reflection component map, for example, the image is divided into 8x8 pixel cell units by calculating the gray gradient direction histogram of the reflection component map, the gradient direction and amplitude of the pixels in each cell unit are calculated, the gradient direction of 0 to 180 degrees is divided into 9 intervals, the histogram of the gradient direction distribution in each cell unit is counted, then the adjacent cell units are grouped into blocks for intra-block normalization processing, and finally the histogram features of all the blocks are concatenated to form a 128-dimensional reflection component feature vector, that is, the reflection component feature.

[0070] Step S3: based on the reflection component map, an occlusion mask is generated through adaptive threshold segmentation and morphological operation, and the joint feature map is spatially weighted through the occlusion mask to obtain a regional occlusion feature map.

[0071] Specifically, in step S3, the generation step of the occlusion mask includes:

[0072] The reflection component map is converted into a gray image, the weighted average value of the pixels in the local neighborhood window of each pixel in the gray image is calculated based on a preset local neighborhood window, and a preset constant is subtracted to obtain the local threshold value of each pixel, if the pixel value is greater than the local threshold value, the value is 255, otherwise the value is 0, and an initial binary mask is generated;

[0073] The initial binary mask is dilated using a rectangular structure of a preset size, and then the dilated initial binary mask is eroded using the rectangular structure to obtain a secondary binary mask;

[0074] The secondary binary mask is subjected to contour searching to identify all connected regions, the pixel area of each connected region is calculated, and the region with the largest area is selected as the target occlusion region;

[0075] A blank mask graph with the same size as the image to be recognized is created, the target occlusion region is filled with 255 and the remaining regions are filled with 0 in the blank mask graph to generate an occlusion mask.

[0076] Specifically, first, the reflection component graph is converted into a gray image, the Gaussian weighted average value of the pixels in the local neighborhood window of each pixel in the gray image is calculated based on a preset local neighborhood window, and a preset constant 5 is subtracted to obtain the local threshold value of each pixel. For example, the size of the local neighborhood window is set to 35x35 pixels. If the pixel value is greater than its local threshold value, it is assigned a value of 255, otherwise it is assigned a value of 0 to generate an initial binary mask. A 3x3 pixel rectangular structuring element is used to perform dilation operation on the initial binary mask, and the dilation operation expands the white region to fill small holes. Then the same size rectangular structuring element is used to perform erosion operation on the initial binary mask after the dilation operation, and the erosion operation shrinks the white region to eliminate isolated noise points to obtain a secondary binary mask. The secondary binary mask is subjected to contour searching to identify all connected regions, the pixel area of each connected region is calculated, and the region with the largest area is selected as the target occlusion region. A blank mask graph with the same size as the image to be recognized is created, all pixels of the blank mask graph have an initial value of zero, the target occlusion region is filled with 255 and the remaining regions are kept as 0 in the blank mask graph to generate a final occlusion mask. Finally, the spatial weighting of the joint feature graph is performed through the occlusion mask, the pixel value of the joint feature graph corresponding to the occlusion region with a value of 255 in the occlusion mask is multiplied by 0, and the pixel value of the joint feature graph corresponding to the unoccluded region with a value of 0 in the occlusion mask remains unchanged to obtain a region occlusion feature graph.

[0077] Step S4: Extracting local discriminative features of the unoccluded region in the region occlusion feature graph.

[0078] In the implementation process, the domain occlusion feature map has been spatially weighted by the occlusion mask to the joint feature map, so that the feature values of the non-occluded area are retained and the feature values of the occluded area are suppressed; when extracting the local discriminative features of the non-occluded area, for example, based on the 68 key point coordinates in the face structure information, taking each key point coordinate as the center, an image block in a circular area with a radius of 15 pixels is extracted as a local analysis area, and if the circular area intersects with the non-occluded area determined by the occlusion mask, the intersection area is subjected to feature extraction; for the local image block of each intersection area, the local binary pattern feature is calculated, a circular neighborhood structure with a radius of 2 pixels and eight neighborhood points is used to calculate the gray value comparison result of each pixel point and its neighborhood pixels to generate an 8-bit binary pattern, and the uniform pattern histogram of the local image block in the intersection area is counted to obtain a 59-dimensional feature vector; at the same time, the gradient direction histogram feature of the local image block of each intersection area is calculated, the local image block of the intersection area is divided into four cell units, each cell unit has a size of 7*7 pixels, the gradient amplitude and direction of each pixel point are calculated, the direction range from zero to 180 degrees is evenly divided into nine intervals, the gradient amplitude accumulation of nine directions in each cell unit is counted to form the gradient direction histogram of the cell unit, and the histograms of the four cell units are concatenated into a 36-dimensional gradient histogram feature vector; the local binary pattern feature vector and the gradient direction histogram feature vector corresponding to the local image block area of each key point are spliced to form a 95-dimensional local feature descriptor; finally, the local feature descriptors of all non-occluded key points are concatenated in the spatial order of the key points to generate a final local discriminative feature vector, which is the local discriminative feature.

[0079] Step S5: After principal component analysis and dimensionality reduction on the reflection component feature and the local discriminative feature, the comprehensive feature vector is generated, and based on the comprehensive feature vector and the pre-constructed face feature library, the recognition is performed through the nearest neighbor classifier to generate the recognition result.

[0080] Specifically, in step S5, the step of generating the comprehensive feature vector by fusion includes:

[0081] After the dimensionality-reduced reflection component feature and the dimensionality-reduced local discriminative feature are weighted according to the preset weight coefficient, the vector dimensions are concatenated and spliced to generate the comprehensive feature vector.

[0082] Specifically, before the recognition is performed through the nearest neighbor classifier in step S5 to generate the recognition result, it further includes:

[0083] The L2 norm normalization processing is performed on the comprehensive feature vector to make its module length 1.

[0084] Specifically, in step S5, based on the comprehensive feature vector and the pre-constructed face feature library, the recognition is performed by a nearest neighbor classifier to generate a recognition result, and the step of generating the recognition result comprises:

[0085] The Euclidean distance between the comprehensive feature vector and each template feature vector in the pre-constructed face feature library is calculated, and the identity of the template corresponding to the minimum Euclidean distance is taken as the preliminary recognition result.

[0086] In the specific implementation process, first, the reflection component feature and the local discriminative feature are respectively subjected to principal component analysis dimension reduction processing, for example, the cumulative variance contribution rate threshold is set to 95%, the main feature components are retained, the reflection component feature is reduced from 128 dimensions to 64 dimensions, and the local discriminative feature is reduced from its original dimension to 32 dimensions; then, the dimension-reduced reflection component feature and the dimension-reduced local discriminative feature are weighted and fused according to a pre-set weight coefficient, for example, the weight coefficient of the reflection component feature is set to 0.6, the weight coefficient of the local discriminative feature is set to 0.4, the two feature vectors are respectively subjected to weighted calculation, and then are concatenated in series in the vector dimension to generate a 96-dimensional comprehensive feature vector; then, the comprehensive feature vector is subjected to L2 norm normalization processing, the L2 norm of the vector is calculated, and each element in the vector is divided by the norm value, so that the length of the processed comprehensive feature vector is 1; finally, the normalized comprehensive feature vector is recognized based on a pre-constructed face feature library, the feature library stores comprehensive feature vectors of multiple templates and their corresponding identity labels, the Euclidean distance between the to-be-recognized comprehensive feature vector and each template comprehensive feature vector in the feature library is calculated, the minimum Euclidean distance value is found after traversing all the templates, and the identity label of the template corresponding to the minimum Euclidean distance is taken as the preliminary recognition result, for example, the similarity determination threshold is set to 0.8, if the minimum Euclidean distance is less than the similarity determination threshold, the preliminary recognition result is confirmed as the final recognition result, and if the minimum Euclidean distance is greater than or equal to the similarity determination threshold, it is determined as an unknown face.

[0087] The working principle of the face feature automatic recognition method based on image processing provided by the application is as follows:

[0088] This invention first generates a comprehensive Gaussian heatmap using the coordinates of facial key points, quantifying the structural importance of different facial regions, with the facial features exhibiting higher heatmap values. This comprehensive Gaussian heatmap then serves a dual purpose: first, in the feature fusion stage, it guides the fusion of multi-scale feature maps through spatial weighting, highlighting the feature contributions of important regions; second, in the image decomposition stage, its heatmap values ​​are used to dynamically adjust the scale parameters of the Gaussian filter in the Retinex algorithm. Small-scale kernels are used in the high-heatmap-valued facial feature edge regions to preserve details, while large-scale kernels are used in the low-heatmap-valued background or smooth regions to achieve smoothing, thereby adaptively estimating illumination distribution. The process involves measuring and decomposing the reflectance component map to obtain a purer one. Then, leveraging the reflectance component map's insensitivity to illumination, an occlusion mask is generated. This mask is then used to precisely locate the occluded region through adaptive thresholding and morphological operations. The joint feature map is then spatially weighted again using this mask to suppress invalid regions and obtain a regional occlusion feature map. Subsequently, local discriminative features are extracted from the unoccluded region and fused with the aforementioned reflectance component features after dimensionality reduction, forming a comprehensive feature vector that combines global illumination invariance with local detail discriminability. Finally, a nearest neighbor classifier is used to match the features in the feature library to complete identity recognition, ultimately achieving stable and reliable face recognition in complex environments.

[0089] Example 2

[0090] An automatic facial feature recognition system based on image processing, in its specific implementation, such as... Figure 2 The diagram illustrates a modular structure of an image processing-based automatic facial feature recognition system according to the present invention, comprising:

[0091] The feature extraction and fusion module 100 is used to extract multi-scale feature maps and facial structure information of the face image to be identified, generate a comprehensive Gaussian heat map based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heat map.

[0092] The image decomposition module 200 is used to decompose the joint feature map into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heat map, and to extract the reflection component features from the reflection component map.

[0093] The occlusion processing module 300 is used to generate an occlusion mask based on the reflection component map through adaptive threshold segmentation and morphological operations, and to spatially weight the joint feature map through the occlusion mask to obtain a region occlusion feature map.

[0094] The local feature extraction module 400 is used to extract local discriminative features of the unoccluded region in the region occlusion feature map;

[0095] The recognition decision module 500 is used for fusing the reflection component features and the local discriminative features after principal component analysis dimension reduction, generating a comprehensive feature vector, and performing recognition based on the comprehensive feature vector and a pre-constructed face feature library through a nearest neighbor classifier to generate a recognition result.

[0096] The working principle of the face feature automatic recognition system based on image processing provided by the application is as follows:

[0097] The feature extraction and fusion module 100 is used for simultaneously extracting a multi-scale feature map and face structure information of an input face image, and generating a quantized comprehensive Gaussian heat map based on the face structure information, and then fusing the multi-scale feature map into a unified joint feature map by using the heat map as a spatial weight guide; then the image decomposition module 200 is used for realizing adaptive image decomposition by running an improved Retinex algorithm, i.e., dynamically adjusting the Gaussian filter scale of illumination estimation by using the heat value of each pixel point in the heat map, so as to output an illumination component map and further separate a reflection component map, and finally extracting reflection component features from the reflection component map; then the occlusion processing module 300 is used for accurately generating a binary mask for marking an occlusion area by adaptive threshold segmentation and morphological operation, and using the mask to perform spatial weighting on the joint feature map to suppress invalid areas, and finally outputting a region occlusion feature map highlighting unoccluded information; then the local feature extraction module 400 is used for locating the unoccluded area and extracting local discriminative features from the unoccluded area; and the recognition decision module 500 is used for firstly performing principal component analysis dimension reduction on the reflection component features and the local discriminative features to eliminate redundancy, then fusing the features according to a preset weight to generate a comprehensive feature vector, and finally performing Euclidean distance comparison between the vector and a template in a pre-built feature library through a nearest neighbor classifier, so as to output a final identity recognition result.

[0098] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0099] Those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructing the relevant hardware by means of a program, and the program can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to carry or store data in a computer readable manner.

[0100] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. An automatic facial feature recognition method based on image processing, characterized in that, The automatic identification method includes: S1: Extract multi-scale feature maps and facial structure information from the face image to be identified, generate a comprehensive Gaussian heatmap based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heatmap; S2: The joint feature map is decomposed into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heatmap, and the reflection component features are extracted from the reflection component map. S3: Based on the reflection component map, an occlusion mask is generated through adaptive threshold segmentation and morphological operations, and the joint feature map is spatially weighted by the occlusion mask to obtain the region occlusion feature map. S4: Extract local discriminative features of the unoccluded region in the region occlusion feature map; S5: Principal component analysis is performed on the reflection component features and local discriminative features to reduce dimensionality and then they are fused to generate a comprehensive feature vector. Based on the comprehensive feature vector and a pre-built face feature library, a nearest neighbor classifier is used for recognition to generate the recognition result. In step S1, the steps for generating the composite Gaussian heatmap include: Construct a pixel grid with the same resolution as the face image to be identified, and map the coordinates of key points in the face structure information to the pixel grid. Each smallest indivisible element in the pixel grid represents a pixel. For a single keypoint, calculate the squared distance from each pixel in the pixel grid to the keypoint to obtain the distance matrix corresponding to that keypoint; The distance matrix corresponding to each key point is processed by Gaussian function to obtain the Gaussian heatmap corresponding to each key point. All Gaussian heatmaps are superimposed, merged and normalized to form a comprehensive Gaussian heatmap. In step S1, the fusion step of the joint feature maps includes: The original resolution Gaussian heatmap is downsampled to the resolution corresponding to the feature map at each scale using bilinear interpolation; The downsampled composite Gaussian heatmap is multiplied pixel by pixel with the feature map at the corresponding scale to obtain the spatially weighted feature map at each scale. All spatially weighted feature maps are concatenated along the channel dimension to obtain a preliminary joint feature map; Principal component analysis is used to reduce the dimensionality of the preliminary joint feature map to generate a new joint feature map. In step S2, the decomposition steps of the illumination component map and the reflection component map include: For each pixel location in the joint feature map, the heat value of the corresponding pixel in the comprehensive Gaussian heat map is read, and the scale parameter of each pixel location in the joint feature map is calculated through a preset mapping function; A Gaussian kernel is generated based on the scale parameter of each pixel position in the joint feature map and a preset Gaussian kernel generation formula. The illumination component map is obtained by performing Gaussian filtering on the joint feature map using a Gaussian kernel. The reflection component map is separated by logarithmic subtraction of the joint feature map and the illumination component map.

2. The automatic facial feature recognition method based on image processing according to claim 1, characterized in that, In step S2, the reflection component features are obtained by calculating the gray-level gradient direction histogram of the reflection component map and statistically analyzing its direction distribution.

3. The automatic facial feature recognition method based on image processing according to claim 2, characterized in that, In step S3, the step of generating the occlusion mask includes: The reflection component map is converted into a grayscale image. The weighted average value of the pixels in the local neighborhood window of each pixel in the grayscale image is calculated based on the preset local neighborhood window, and a preset constant is subtracted to obtain the local threshold of each pixel. If the pixel value is greater than its local threshold, it is assigned a value of 255; otherwise, it is assigned a value of 0, thus generating an initial binary mask. The initial binary mask is expanded using a rectangular structure of a preset size, and then the expanded initial binary mask is eroded using the rectangular structure to obtain the secondary binary mask. Contour search is performed on the secondary binary mask to identify all connected regions. The pixel area of ​​each connected region is calculated, and the region with the largest area is selected as the target occlusion region. Create a blank mask image with the same size as the face image to be identified. Fill the occluded area of ​​the target with 255 and fill the rest with 0 on the blank mask image to generate the occlusion mask.

4. The automatic facial feature recognition method based on image processing according to claim 3, characterized in that, In step S5, the step of fusing and generating a comprehensive feature vector includes: The dimension-reduced reflection component features and the dimension-reduced local discriminative features are weighted according to preset weight coefficients and then concatenated in the vector dimension to generate a comprehensive feature vector.

5. The automatic facial feature recognition method based on image processing according to claim 4, characterized in that, In step S5, identification is performed using a nearest neighbor classifier. Before generating the identification result, the following steps are also included: The comprehensive feature vector is normalized using the L2 norm so that its magnitude is 1.

6. The automatic facial feature recognition method based on image processing according to claim 5, characterized in that, In step S5, the steps of generating recognition results by using a nearest neighbor classifier to identify faces based on the comprehensive feature vector and a pre-built face feature library include: Calculate the Euclidean distance between the comprehensive feature vector and the feature vector of each template in the pre-built face feature library, and take the identity identifier of the template corresponding to the smallest Euclidean distance as the preliminary recognition result.

7. An automatic facial feature recognition system based on image processing, characterized in that, An automatic facial feature recognition method based on image processing, as described in any one of claims 1 to 6, wherein the automatic recognition system comprises: The feature extraction and fusion module is used to extract multi-scale feature maps and facial structure information from the face image to be identified, generate a comprehensive Gaussian heatmap based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heatmap. The image decomposition module is used to decompose the joint feature map into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heatmap, and to extract the reflection component features from the reflection component map. The occlusion processing module is used to generate an occlusion mask based on the reflection component map through adaptive threshold segmentation and morphological operations, and to spatially weight the joint feature map using the occlusion mask to obtain a region occlusion feature map. The local feature extraction module is used to extract local discriminative features of the unoccluded areas in the region occlusion feature map; The recognition decision module is used to perform principal component analysis to reduce the dimensionality of the reflection component features and local discriminative features and then fuse them to generate a comprehensive feature vector. Based on the comprehensive feature vector and a pre-built face feature library, the nearest neighbor classifier is used to perform recognition and generate recognition results. In the feature extraction and fusion module, the steps for generating the comprehensive Gaussian heatmap include: Construct a pixel grid with the same resolution as the face image to be identified, and map the coordinates of key points in the face structure information to the pixel grid. Each smallest indivisible element in the pixel grid represents a pixel. For a single keypoint, calculate the squared distance from each pixel in the pixel grid to the keypoint to obtain the distance matrix corresponding to that keypoint; The distance matrix corresponding to each key point is processed by Gaussian function to obtain the Gaussian heatmap corresponding to each key point. All Gaussian heatmaps are superimposed, merged and normalized to form a comprehensive Gaussian heatmap. In the feature extraction and fusion module, the fusion steps of the joint feature maps include: The original resolution Gaussian heatmap is downsampled to the resolution corresponding to the feature map at each scale using bilinear interpolation; The downsampled composite Gaussian heatmap is multiplied pixel by pixel with the feature map at the corresponding scale to obtain the spatially weighted feature map at each scale. All spatially weighted feature maps are concatenated along the channel dimension to obtain a preliminary joint feature map; Principal component analysis is used to reduce the dimensionality of the preliminary joint feature map to generate a new joint feature map. In the image decomposition module, the decomposition steps of the illumination component map and the reflection component map include: For each pixel location in the joint feature map, the heat value of the corresponding pixel in the comprehensive Gaussian heat map is read, and the scale parameter of each pixel location in the joint feature map is calculated through a preset mapping function; A Gaussian kernel is generated based on the scale parameter of each pixel position in the joint feature map and a preset Gaussian kernel generation formula. The illumination component map is obtained by performing Gaussian filtering on the joint feature map using a Gaussian kernel. The reflection component map is separated by logarithmic subtraction of the joint feature map and the illumination component map.

Images