Face feature automatic recognition system and method based on image processing

Abstract

The invention provides an automatic face feature recognition system and method based on image processing, relates to the technical field of face recognition, and aims to guide multi-scale feature fusion and illumination decomposition by using a thermodynamic diagram generated by face structure information, effectively enhance feature expression of key areas of five sense organs, suppress background interference and improve recognition efficiency. An improved Retinex algorithm is combined with a thermodynamic diagram to realize adaptive illumination component estimation, the halo effect of a traditional method in an edge region is overcome, illumination invariance of reflection component features is ensured, and the illumination invariance of the reflection component features is ensured by generating an accurate shielding mask from a reflection component diagram and extracting local discriminative features from an unshielded region. The method can effectively cope with challenges brought by shelters, finally, weighted fusion of global reflection component features and local discriminative features is combined with principal component dimensionality reduction and nearest neighbor classification, a recognition process with clear hierarchy and cooperative work is formed, and reliable identity recognition under double difficulties of illumination variation and local sheltering is realized.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to an automatic facial feature recognition system and method based on image processing. Background Technology

[0002] In existing technologies, automatic facial recognition methods typically face numerous challenges in complex environments. Traditional methods often directly extract and match features from the input image, making it difficult to effectively handle interference factors such as changes in illumination. A typical recognition process usually includes face detection, feature extraction, and classification, with feature extraction often relying on global features or single local feature descriptors. To address illumination issues, some solutions attempt to use histogram equalization or homomorphic filtering for preprocessing, but these methods are prone to losing image details or introducing noise. While multi-scale feature fusion strategies can combine information from different levels, effectively integrating prior structural knowledge remains a challenge. Existing feature decomposition methods often handle illumination and reflection components independently, lacking synergistic optimization with the overall recognition process. Therefore, a facial recognition solution that can organically combine multi-source information, possesses good interpretability, and adapt to complex scenarios is needed.

[0003] Therefore, it is necessary to provide an automatic facial feature recognition system and method based on image processing to solve the above-mentioned technical problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an automatic facial feature recognition system and method based on image processing, achieving the beneficial effect of facial recognition that can combine multi-source information and adapt to complex scenarios.

[0005] This invention provides an automatic facial feature recognition method based on image processing, comprising:

[0006] S1: Extract multi-scale feature maps and facial structure information from the face image to be identified, generate a comprehensive Gaussian heatmap based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heatmap;

[0007] S2: The joint feature map is decomposed into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heatmap, and the reflection component features are extracted from the reflection component map.

[0008] S3: Based on the reflection component map, an occlusion mask is generated through adaptive threshold segmentation and morphological operations, and the joint feature map is spatially weighted by the occlusion mask to obtain the region occlusion feature map.

[0009] S4: Extract local discriminative features of the unoccluded region in the region occlusion feature map;

[0010] S5: Principal component analysis is used to reduce the dimensionality of the reflection component features and local discriminative features, and then the features are fused to generate a comprehensive feature vector. Based on the comprehensive feature vector and a pre-built face feature library, the nearest neighbor classifier is used for recognition to generate the recognition result.

[0011] Preferably, in step S1, the step of generating the comprehensive Gaussian heatmap includes:

[0012] Construct a pixel grid with the same resolution as the face image to be identified, and map the coordinates of key points in the face structure information to the pixel grid. Each smallest indivisible element in the pixel grid represents a pixel.

[0013] For a single keypoint, calculate the squared distance from each pixel in the pixel grid to the keypoint to obtain the distance matrix corresponding to that keypoint;

[0014] The distance matrix corresponding to each key point is processed by Gaussian function to obtain the Gaussian heatmap corresponding to each key point. All Gaussian heatmaps are superimposed, merged and normalized to form a comprehensive Gaussian heatmap.

[0015] Preferably, in step S1, the fusion step of the joint feature maps includes:

[0016] The original resolution Gaussian heatmap is downsampled to the resolution corresponding to the feature map at each scale using bilinear interpolation;

[0017] The downsampled composite Gaussian heatmap is multiplied pixel by pixel with the feature map at the corresponding scale to obtain the spatially weighted feature map at each scale.

[0018] All spatially weighted feature maps are concatenated along the channel dimension to obtain a preliminary joint feature map;

[0019] Principal component analysis is used to reduce the dimensionality of the preliminary joint feature map, generating a new joint feature map.

[0020] Preferably, in step S2, the decomposition steps of the illumination component map and the reflection component map include:

[0021] For each pixel location in the joint feature map, the heat value of the corresponding pixel in the comprehensive Gaussian heat map is read, and the scale parameter of each pixel location in the joint feature map is calculated through a preset mapping function;

[0022] A Gaussian kernel is generated based on the scale parameter of each pixel position in the joint feature map and a preset Gaussian kernel generation formula.

[0023] The illumination component map is obtained by performing Gaussian filtering on the joint feature map using a Gaussian kernel.

[0024] The reflection component map is separated by logarithmic subtraction of the joint feature map and the illumination component map.

[0025] Preferably, in step S2, the reflection component features are obtained by calculating the gray-level gradient direction histogram of the reflection component map and statistically analyzing its direction distribution.

[0026] Preferably, in step S3, the step of generating the occlusion mask includes:

[0027] The reflection component map is converted into a grayscale image. The weighted average value of the pixels in the local neighborhood window of each pixel in the grayscale image is calculated based on the preset local neighborhood window, and a preset constant is subtracted to obtain the local threshold of each pixel. If the pixel value is greater than its local threshold, it is assigned a value of 255; otherwise, it is assigned a value of 0, thus generating an initial binary mask.

[0028] The initial binary mask is expanded using a rectangular structure of a preset size, and then the expanded initial binary mask is eroded using the rectangular structure to obtain the secondary binary mask.

[0029] Contour search is performed on the secondary binary mask to identify all connected regions. The pixel area of ​​each connected region is calculated, and the region with the largest area is selected as the target occlusion region.

[0030] Create a blank mask image with the same size as the face image to be identified. Fill the occluded area of ​​the target with 255 and fill the rest with 0 on the blank mask image to generate the occlusion mask.

[0031] Preferably, in step S5, the step of fusing and generating a comprehensive feature vector includes:

[0032] The dimension-reduced reflection component features and the dimension-reduced local discriminative features are weighted according to preset weight coefficients and then concatenated in the vector dimension to generate a comprehensive feature vector.

[0033] Preferably, in step S5, before generating the recognition result by performing identification using a nearest neighbor classifier, the following steps are also included:

[0034] The comprehensive feature vector is normalized using the L2 norm so that its magnitude is 1.

[0035] Preferably, in step S5, the step of generating the recognition result by recognizing the face based on the comprehensive feature vector and the pre-built face feature library using a nearest neighbor classifier includes:

[0036] Calculate the Euclidean distance between the comprehensive feature vector and the feature vector of each template in the pre-built face feature library, and take the identity identifier of the template corresponding to the smallest Euclidean distance as the preliminary recognition result.

[0037] This invention provides an automatic facial feature recognition system based on image processing, comprising:

[0038] The feature extraction and fusion module is used to extract multi-scale feature maps and facial structure information from the face image to be identified, generate a comprehensive Gaussian heatmap based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heatmap.

[0039] The image decomposition module is used to decompose the joint feature map into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heatmap, and to extract the reflection component features from the reflection component map.

[0040] The occlusion processing module is used to generate an occlusion mask based on the reflection component map through adaptive threshold segmentation and morphological operations, and to spatially weight the joint feature map using the occlusion mask to obtain a region occlusion feature map.

[0041] The local feature extraction module is used to extract local discriminative features of the unoccluded areas in the region occlusion feature map;

[0042] The recognition decision module is used to perform principal component analysis to reduce the dimensionality of the reflection component features and local discriminative features and then fuse them to generate a comprehensive feature vector. Based on the comprehensive feature vector and a pre-built face feature library, the recognition is performed by a nearest neighbor classifier to generate the recognition result.

[0043] Compared with related technologies, the automatic facial feature recognition system and method based on image processing provided by the present invention has the following beneficial effects:

[0044] This invention utilizes heatmaps generated from facial structural information to guide multi-scale feature fusion and illumination decomposition, effectively enhancing the feature representation of key facial regions while suppressing background interference. An improved Retinex algorithm, combined with heatmaps, achieves adaptive illumination component estimation, overcoming the halo effect in edge regions inherent in traditional methods and ensuring the illumination invariance of reflectance component features. By generating accurate occlusion masks from the reflectance component map and extracting local discriminative features from unoccluded areas, the system effectively addresses challenges posed by occlusions such as masks. Finally, a weighted fusion of global reflectance component features and local discriminative features, combined with principal component dimensionality reduction and nearest neighbor classification, constitutes a hierarchical and collaborative recognition process. This achieves reliable identity recognition under the dual challenges of varying illumination and local occlusion. Attached Figure Description

[0045] Figure 1 This is a flowchart of an automatic facial feature recognition method based on image processing according to the present invention;

[0046] Figure 2This is a block diagram of an automatic facial feature recognition system based on image processing according to the present invention. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0048] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as being processed sequentially, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0049] Example 1

[0050] An automatic facial feature recognition method based on image processing, in specific implementation, such as... Figure 1 The diagram illustrates a flowchart of an automatic facial feature recognition method based on image processing according to the present invention, comprising:

[0051] Step S1: Extract multi-scale feature maps and facial structure information from the face image to be identified, generate a comprehensive Gaussian heatmap based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heatmap.

[0052] Specifically, in step S1, the steps for generating the comprehensive Gaussian heatmap include:

[0053] Construct a pixel grid with the same resolution as the face image to be identified, and map the coordinates of key points in the face structure information to the pixel grid. Each smallest indivisible element in the pixel grid represents a pixel.

[0054] For a single keypoint, calculate the squared distance from each pixel in the pixel grid to the keypoint to obtain the distance matrix corresponding to that keypoint;

[0055] The distance matrix corresponding to each key point is processed by Gaussian function to obtain the Gaussian heatmap corresponding to each key point. All Gaussian heatmaps are superimposed, merged and normalized to form a comprehensive Gaussian heatmap.

[0056] Specifically, in step S1, the fusion step of the joint feature maps includes:

[0057] The original resolution Gaussian heatmap is downsampled to the resolution corresponding to the feature map at each scale using bilinear interpolation;

[0058] The downsampled composite Gaussian heatmap is multiplied pixel by pixel with the feature map at the corresponding scale to obtain the spatially weighted feature map at each scale.

[0059] All spatially weighted feature maps are concatenated along the channel dimension to obtain a preliminary joint feature map;

[0060] Principal component analysis is used to reduce the dimensionality of the preliminary joint feature map, generating a new joint feature map.

[0061] In the specific implementation process, firstly, a multi-scale feature map of the face image to be identified is extracted using a convolutional neural network. For example, the multi-scale feature map contains feature information at three different scales, namely one-half, one-quarter, and one-eighth of the original image. Simultaneously, a 68-point facial landmark detection algorithm is used to obtain facial structure information, which consists of the horizontal and vertical coordinates of 68 landmarks. Next, a comprehensive Gaussian heatmap is generated based on the facial structure information. First, a pixel grid with the same resolution as the face image to be identified is constructed, where each smallest unit of the pixel grid corresponds to a pixel. The coordinates of the 68 landmarks are mapped to this pixel grid. For each landmark, the squared distance from all pixels in the pixel grid to that landmark is calculated to generate a distance matrix corresponding to that landmark. Then, a Gaussian function with a standard deviation of 3 is used to process each distance matrix to obtain the Gaussian heatmap corresponding to each landmark. Finally, all 68 Gaussian heatmaps are superimposed, and each element in the superimposed matrix is ​​normalized to a value between 0 and 1, generating the final composite Gaussian heatmap. Based on this composite Gaussian heatmap, multi-scale feature maps are fused into a joint feature map. For example, the original resolution composite Gaussian heatmap is downsampled to the same resolution as the 1 / 2, 1 / 4, and 1 / 8 scale feature maps using bilinear interpolation. Then, the downsampled heatmaps at each scale are multiplied pixel-by-pixel with the corresponding scale feature maps to obtain spatially weighted feature maps at three scales. These three spatially weighted feature maps are then concatenated along the channel dimension to form a high-dimensional preliminary joint feature map. Finally, principal component analysis is performed on this preliminary joint feature map for dimensionality reduction. For example, the cumulative variance contribution rate threshold is set to 95%, retaining the main feature components to generate the final joint feature map.

[0062] Step S2: The joint feature map is decomposed into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heatmap, and the reflection component features are extracted from the reflection component map.

[0063] Specifically, in step S2, the decomposition steps of the illumination component map and the reflection component map include:

[0064] For each pixel location in the joint feature map, the heat value of the corresponding pixel in the comprehensive Gaussian heat map is read, and the scale parameter of each pixel location in the joint feature map is calculated through a preset mapping function;

[0065] A Gaussian kernel is generated based on the scale parameter of each pixel position in the joint feature map and a preset Gaussian kernel generation formula.

[0066] The illumination component map is obtained by performing Gaussian filtering on the joint feature map using a Gaussian kernel.

[0067] The reflection component map is separated by logarithmic subtraction of the joint feature map and the illumination component map.

[0068] Specifically, in step S2, the reflection component features are obtained by calculating the gray-level gradient direction histogram of the reflection component map and statistically analyzing its direction distribution.

[0069] In the specific implementation process, firstly, for each pixel position in the joint feature map, the heat value of the corresponding pixel in the comprehensive Gaussian heatmap is read. The heat value is a floating-point number between 0 and 1. The scale parameter of each pixel position is calculated using a preset linear mapping function. This mapping function sets the maximum scale parameter to 15 and the minimum scale parameter to 3. Specifically, the calculation method is to subtract the product of the heat value and the difference between the maximum and minimum scale parameters from the maximum scale parameter. Based on the scale parameter of each pixel position, a corresponding Gaussian kernel is generated using a preset Gaussian kernel generation formula. The size of the Gaussian kernel is dynamically determined according to the scale parameter. The calculation formula is that the kernel size equals 2 multiplied by 3 times the scale parameter, rounded up, and then added by 1. The generated Gaussian kernel is used to perform pixel-by-pixel adaptive Gaussian filtering on the joint feature map to obtain an illumination component map that conforms to the Retinex model. During the filtering process, each pixel uses the Gaussian kernel generated with its corresponding scale parameter. The process begins with a neighborhood-weighted average. Following the mathematical expression of the Retinex algorithm, the reflection component map is separated by logarithmic subtraction between the joint feature map and the illumination component map. Specifically, the joint feature map and the illumination component map are transformed to the logarithmic domain with the natural constant as the base, then subtracted. The result is then transformed back to the linear domain using exponential operations to obtain the reflection component map. Finally, reflection component features are extracted from the reflection component map. For example, by calculating the gray-level gradient direction histogram of the reflection component map, the image is divided into 8×8 pixel cells. Within each cell, the gradient direction and magnitude of the pixels are calculated. The gradient direction from 0 to 180 degrees is divided into 9 intervals. The histogram of the gradient direction distribution within each cell is statistically analyzed. Then, adjacent cell cells are grouped into blocks for intra-block normalization. Finally, the histogram features of all blocks are concatenated to form a 128-dimensional reflection component feature vector, i.e., the reflection component features.

[0070] Step S3: Based on the reflection component map, generate an occlusion mask through adaptive threshold segmentation and morphological operations, and spatially weight the joint feature map using the occlusion mask to obtain the region occlusion feature map.

[0071] Specifically, in step S3, the step of generating the occlusion mask includes:

[0072] The reflection component map is converted into a grayscale image. The weighted average value of the pixels in the local neighborhood window of each pixel in the grayscale image is calculated based on the preset local neighborhood window, and a preset constant is subtracted to obtain the local threshold of each pixel. If the pixel value is greater than its local threshold, it is assigned a value of 255; otherwise, it is assigned a value of 0, thus generating an initial binary mask.

[0073] The initial binary mask is expanded using a rectangular structure of a preset size, and then the expanded initial binary mask is eroded using the rectangular structure to obtain the secondary binary mask.

[0074] Contour search is performed on the secondary binary mask to identify all connected regions. The pixel area of ​​each connected region is calculated, and the region with the largest area is selected as the target occlusion region.

[0075] Create a blank mask image with the same size as the face image to be identified. Fill the occluded area of ​​the target with 255 and fill the rest with 0 on the blank mask image to generate the occlusion mask.

[0076] Specifically, the reflection component image is first converted into a grayscale image. Based on a preset local neighborhood window, the Gaussian weighted average of the pixels within the local neighborhood window of each pixel in the grayscale image is calculated, and a preset constant of 5 is subtracted to obtain the local threshold for each pixel. For example, the local neighborhood window size is set to 35×35 pixels. If the pixel value is greater than its local threshold, it is assigned a value of 255; otherwise, it is assigned a value of 0, generating an initial binary mask. The initial binary mask is then dilated using a rectangular structuring element with a preset size of 3×3 pixels. This dilation operation expands the white area to fill small holes. Then, a rectangular structuring element of the same size is used to erode the dilated initial binary mask. This erosion operation shrinks the white area to eliminate isolation. Noise points are identified to obtain a secondary binary mask. Contour lookup is performed on the secondary binary mask to identify all connected regions. The pixel area of ​​each connected region is calculated, and the region with the largest area is selected as the target occlusion region. A blank mask image with the same size as the face image to be identified is created, with all pixels initially set to zero. The target occlusion region is filled with 255 pixels on the blank mask, while the remaining regions are kept at 0, generating the final occlusion mask. Finally, the joint feature map is spatially weighted using the occlusion mask. The pixel values ​​of the joint feature map corresponding to the occluded regions with a value of 255 in the occlusion mask are multiplied by 0, while the pixel values ​​of the joint feature map corresponding to the unoccluded regions with a value of 0 in the occlusion mask remain unchanged, resulting in a region occlusion feature map.

[0077] Step S4: Extract local discriminative features of the unoccluded areas in the region occlusion feature map.

[0078] In the specific implementation process, the domain occlusion feature map has undergone spatial weighting processing on the joint feature map through occlusion masking, so that the feature values ​​of the unoccluded region are preserved while the feature values ​​of the occluded region are suppressed. When extracting local discriminative features of the unoccluded region, for example, based on the coordinates of 68 key points in the face structure information, an image patch with a radius of 15 pixels is extracted as the local analysis region with each key point coordinate as the center. If the circular region intersects with the unoccluded region determined by the occlusion mask, feature extraction is performed on the intersection region. For the local image patch of each intersection region, its local binary pattern feature is calculated. A circular neighborhood structure with a radius of 2 pixels and eight neighboring points is used to calculate the gray value comparison result of each pixel with its neighboring pixels, generating an 8-bit binary pattern. The uniform pattern histogram within the local image patch of the intersection region is statistically analyzed to obtain a 59-dimensional pattern. The feature vector is calculated; simultaneously, the gradient orientation histogram feature of the local image patch in each intersection region is calculated, and the local image patch in the intersection region is divided into 4 cell units, each cell unit is 7×7 pixels in size. The gradient magnitude and direction of each pixel are calculated, and the directional range from 0 degrees to 180 degrees is evenly divided into 9 intervals. The gradient magnitude of the 9 directions in each cell unit is accumulated to form the gradient orientation histogram of that cell unit. The histograms of the four cell units are concatenated to form a 36-dimensional gradient histogram feature vector; the local binary pattern feature vector and the gradient orientation histogram feature vector of the local image patch region corresponding to each key point are concatenated to form a 95-dimensional local feature descriptor; finally, the local feature descriptors of all unoccluded key points are concatenated according to the spatial order of the key points to generate the final local discriminative feature vector, which is the local discriminative feature.

[0079] Step S5: Perform principal component analysis to reduce the dimensionality of the reflection component features and local discriminative features, and then fuse them to generate a comprehensive feature vector. Based on the comprehensive feature vector and the pre-built face feature library, the nearest neighbor classifier is used for recognition to generate the recognition result.

[0080] Specifically, in step S5, the step of fusing and generating a comprehensive feature vector includes:

[0081] The dimension-reduced reflection component features and the dimension-reduced local discriminative features are weighted according to preset weight coefficients and then concatenated in the vector dimension to generate a comprehensive feature vector.

[0082] Specifically, in step S5, identification is performed using a nearest neighbor classifier, and the process before generating the identification result includes:

[0083] The comprehensive feature vector is normalized using the L2 norm so that its magnitude is 1.

[0084] Specifically, in step S5, the steps of generating the recognition result by recognizing the face based on the comprehensive feature vector and the pre-built face feature library through a nearest neighbor classifier include:

[0085] Calculate the Euclidean distance between the comprehensive feature vector and the feature vector of each template in the pre-built face feature library, and take the identity identifier of the template corresponding to the smallest Euclidean distance as the preliminary recognition result.

[0086] In the specific implementation process, firstly, principal component analysis is performed on the reflection component features and local discriminant features to reduce their dimensionality. For example, the cumulative variance contribution rate threshold is set to 95%, retaining the main feature components, reducing the reflection component features from 128 dimensions to 64 dimensions, and the local discriminant features from their original dimensions to 32 dimensions. Next, the dimensionality-reduced reflection component features and the dimensionality-reduced local discriminant features are weighted and fused according to preset weight coefficients. For example, the weight coefficient of the reflection component features is set to 0.6, and the weight coefficient of the local discriminant features is set to 0.4. After weighting the two feature vectors separately, they are concatenated and spliced ​​in the vector dimension to generate a 96-dimensional comprehensive feature vector. Then, the comprehensive feature vector is normalized using the L2 norm, and the vector is calculated. The L2 norm is used to divide each element of the vector by this norm value, so that the magnitude of the processed comprehensive feature vector is 1. Finally, recognition is performed based on the normalized comprehensive feature vector and a pre-built face feature library. This feature library stores the comprehensive feature vectors of multiple templates and their corresponding identity identifiers. The Euclidean distance between the comprehensive feature vector to be recognized and the comprehensive feature vector of each template in the feature library is calculated. After traversing all templates, the minimum Euclidean distance value is found. The identity identifier of the template corresponding to the minimum Euclidean distance is taken as the preliminary recognition result. For example, the similarity judgment threshold is set to 0.8. If the minimum Euclidean distance is less than the similarity judgment threshold, the preliminary recognition result is confirmed as the final recognition result. If the minimum Euclidean distance is greater than or equal to the similarity judgment threshold, it is determined to be an unknown face.

[0087] The working principle of the automatic facial feature recognition method based on image processing provided by this invention is as follows:

[0088] This invention first generates a comprehensive Gaussian heatmap using the coordinates of facial key points, quantifying the structural importance of different facial regions, with the facial features exhibiting higher heatmap values. This comprehensive Gaussian heatmap then serves a dual purpose: first, in the feature fusion stage, it guides the fusion of multi-scale feature maps through spatial weighting, highlighting the feature contributions of important regions; second, in the image decomposition stage, its heatmap values ​​are used to dynamically adjust the scale parameters of the Gaussian filter in the Retinex algorithm. Small-scale kernels are used in the high-heatmap-valued facial feature edge regions to preserve details, while large-scale kernels are used in the low-heatmap-valued background or smooth regions to achieve smoothing, thereby adaptively estimating illumination distribution. The process involves measuring and decomposing the reflectance component map to obtain a purer one. Then, leveraging the reflectance component map's insensitivity to illumination, an occlusion mask is generated. This mask is then used to precisely locate the occluded region through adaptive thresholding and morphological operations. The joint feature map is then spatially weighted again using this mask to suppress invalid regions and obtain a regional occlusion feature map. Subsequently, local discriminative features are extracted from the unoccluded region and fused with the aforementioned reflectance component features after dimensionality reduction, forming a comprehensive feature vector that combines global illumination invariance with local detail discriminability. Finally, a nearest neighbor classifier is used to match the features in the feature library to complete identity recognition, ultimately achieving stable and reliable face recognition in complex environments.

[0089] Example 2

[0090] An automatic facial feature recognition system based on image processing, in its specific implementation, such as... Figure 2 The diagram illustrates the modular structure of an image processing-based automatic facial feature recognition system according to the present invention, comprising:

[0091] The feature extraction and fusion module 100 is used to extract multi-scale feature maps and facial structure information of the face image to be identified, generate a comprehensive Gaussian heat map based on the facial structure information, and fuse the multi-scale feature maps into a joint feature map based on the comprehensive Gaussian heat map.

[0092] The image decomposition module 200 is used to decompose the joint feature map into an illumination component map and a reflection component map by combining the Retinex algorithm with the comprehensive Gaussian heatmap, and to extract the reflection component features from the reflection component map.

[0093] The occlusion processing module 300 is used to generate an occlusion mask based on the reflection component map through adaptive threshold segmentation and morphological operations, and to spatially weight the joint feature map through the occlusion mask to obtain a region occlusion feature map.

[0094] The local feature extraction module 400 is used to extract local discriminative features of the unoccluded region in the region occlusion feature map;

[0095] The recognition decision module 500 is used to perform principal component analysis to reduce the dimensionality of the reflection component features and local discriminative features and then fuse them to generate a comprehensive feature vector. Based on the comprehensive feature vector and a pre-built face feature library, the recognition is performed by a nearest neighbor classifier to generate the recognition result.

[0096] The working principle of the automatic facial feature recognition system based on image processing provided by this invention is as follows:

[0097] This invention employs a feature extraction and fusion module 100 to simultaneously extract multi-scale feature maps and facial structure information from an input face image. Based on the facial structure information, a quantized comprehensive Gaussian heatmap is generated. This heatmap is then used as a spatial weight guide to fuse the multi-scale feature maps into a unified joint feature map. Next, an image decomposition module 200 runs an improved Retinex algorithm, dynamically adjusting the Gaussian filtering scale for illumination estimation using the heatmap value of each pixel in the heatmap, achieving adaptive image decomposition. This outputs an illumination component map and further separates the reflection component map, finally extracting reflection component features from the reflection component map. Subsequently, an occlusion processing module 300 uses an adaptive thresholding... Value segmentation and morphological operations accurately generate a binary mask for identifying occluded regions. This mask is then used to spatially weight the joint feature map to suppress invalid regions, ultimately outputting an occluded feature map that highlights unoccluded information. Next, the local feature extraction module 400 locates the unoccluded regions and extracts highly discriminative local features, i.e., local discriminative features. The recognition decision module 500 first performs principal component analysis to reduce the dimensionality of the reflectance component features and local discriminative features to eliminate redundancy, then fuses them according to preset weights to generate a comprehensive feature vector. Finally, a nearest neighbor classifier compares this vector with templates in a pre-built feature library using Euclidean distance, thus outputting the final identity recognition result.

[0098] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0099] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0100] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. An automatic face feature recognition method based on image processing, characterized in that, The automatic recognition method comprises: S1: extracting a multi-scale feature map and face structure information of a to-be-recognized face image, generating a comprehensive Gaussian heat map based on the face structure information, and fusing the multi-scale feature map into a joint feature map based on the comprehensive Gaussian heat map; S2: decomposing the joint feature map into an illumination component map and a reflection component map through a Retinex algorithm combined with the comprehensive Gaussian heat map, and extracting a reflection component feature from the reflection component map; S3: generating an occlusion mask based on the reflection component map through adaptive threshold segmentation and morphological operation, and performing spatial weighting on the joint feature map through the occlusion mask to obtain a regional occlusion feature map; S4: extracting a local discriminative feature of an unoccluded region in the regional occlusion feature map; S5: fusing the reflection component feature and the local discriminative feature after principal component analysis dimension reduction to generate a comprehensive feature vector, and performing recognition through a nearest neighbor classifier based on the comprehensive feature vector and a pre-constructed face feature library to generate a recognition result.

2. The automatic face feature recognition method based on image processing according to claim 1, characterized in that, In step S1, the generation step of the comprehensive Gaussian heat map comprises: A pixel grid with the same resolution as the to-be-recognized face image is constructed, and the key point coordinates in the face structure information are mapped into the pixel grid, each minimum indivisible element in the pixel grid representing a pixel point; For a single key point, the squared distance of each pixel point in the pixel grid to the key point is calculated to obtain a distance matrix corresponding to the key point; The distance matrix corresponding to each key point is processed by a Gaussian function to obtain a Gaussian heat map corresponding to each key point, and all Gaussian heat maps are superimposed and fused to be normalized into a comprehensive Gaussian heat map.

3. The method according to claim 2, wherein, In step S1, the fusion step of the joint feature map comprises: The comprehensive Gaussian heat map of the original resolution is down-sampled to the resolution corresponding to the feature map of each scale through bilinear interpolation; The down-sampled comprehensive Gaussian heat map is multiplied with the feature map of the corresponding scale pixel by pixel to obtain a spatially weighted feature map of each scale; All spatially weighted feature maps are spliced in the channel dimension to obtain a preliminary joint feature map; The preliminary joint feature map is processed by principal component analysis dimension reduction to generate a joint feature map.

4. The automatic face feature recognition method based on image processing according to claim 3, characterized in that, In step S2, the decomposition step of the illumination component map and the reflection component map comprises: For each pixel position in the joint feature map, the heat value of the pixel at the corresponding position in the comprehensive Gaussian heat map is read, and the scale parameter of each pixel position in the joint feature map is calculated through a preset mapping function; Based on the scale parameter of each pixel position in the joint feature map, a Gaussian kernel is generated based on a preset Gaussian kernel generation formula; The joint feature map is processed by Gaussian filtering through the Gaussian kernel to obtain an illumination component map; The reflection component map is separated by performing logarithmic domain subtraction operation between the joint feature map and the illumination component map.

5. The method according to claim 4, wherein, In step S2, the reflection component feature is obtained by calculating the gray gradient direction histogram of the reflection component map and counting its direction distribution.

6. The method according to claim 5, wherein, In step S3, the generation step of the occlusion mask comprises: The reflection component image is converted into a gray image, a weighted average value of pixels within a local neighborhood window of each pixel in the gray image is calculated based on a preset local neighborhood window, and a preset constant is subtracted to obtain a local threshold value of each pixel, if a pixel value is greater than the local threshold value, the pixel value is assigned 255, otherwise, the pixel value is assigned 0, and an initial binary mask is generated; The initial binary mask is dilated using a rectangular structure of a preset size, and the dilated initial binary mask is eroded using the rectangular structure to obtain a secondary binary mask; Contour finding is performed on the secondary binary mask to identify all connected regions, pixel areas of each connected region are calculated, and the largest area is selected as a target occlusion region; A blank mask image with the same size as the to-be-identified face image is created, the target occlusion region is filled with 255 and the remaining regions are filled with 0 in the blank mask image, and an occlusion mask is generated.

7. The method according to claim 6, wherein, In step S5, the step of fusing to generate a comprehensive feature vector includes: The reflection component feature after dimension reduction and the local discriminative feature after dimension reduction are weighted according to a preset weight coefficient, and then concatenated in the vector dimension to generate a comprehensive feature vector.

8. The method according to claim 7, wherein, In step S5, before the identification result is generated by the nearest neighbor classifier, it further includes: The comprehensive feature vector is subjected to L2 norm normalization processing to make its module length 1.

9. The method according to claim 8, wherein, In step S5, based on the comprehensive feature vector and the pre-constructed face feature library, the identification result is generated by the nearest neighbor classifier, and the step includes: The Euclidean distance between the comprehensive feature vector and each template feature vector in the pre-constructed face feature library is calculated, and the identity of the template corresponding to the minimum Euclidean distance is taken as the preliminary identification result.

10. An automatic face feature recognition system based on image processing, characterized in that, The automatic recognition system is applied to the face feature automatic recognition method based on image processing in any one of claims 1 to 9, and the automatic recognition system comprises: A feature extraction and fusion module is configured to extract a multi-scale feature map and face structure information of a to-be-identified face image, generate a comprehensive Gaussian heat map based on the face structure information, and fuse the multi-scale feature map into a joint feature map based on the comprehensive Gaussian heat map; An image decomposition module is configured to decompose the joint feature map into an illumination component image and a reflection component image by using a Retinex algorithm in combination with the comprehensive Gaussian heat map, and extract a reflection component feature from the reflection component image; An occlusion processing module is configured to generate an occlusion mask based on the reflection component image by using adaptive threshold segmentation and morphological operation, and obtain a region occlusion feature map by spatially weighting the joint feature map based on the occlusion mask; A local feature extraction module is configured to extract a local discriminative feature of an unoccluded region in the region occlusion feature map; An identification and decision module is configured to fuse the reflection component feature and the local discriminative feature after principal component analysis and dimension reduction to generate a comprehensive feature vector, and identify the comprehensive feature vector and a pre-constructed face feature library by using a nearest neighbor classifier to generate an identification result.

Images