Multi-feature combined face recognition method and device
Through the multi-feature joint face recognition method, convolutional neural network and feature fusion technology are used to solve the robustness and accuracy of traditional face recognition technology in complex environments, achieving higher recognition accuracy and robustness sex.
Patent Information
- Application Number
- CN202410975030.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional face recognition technology relies on a single feature extraction method, resulting in limited robustness and accuracy in complex environments.
Using multi-feature joint face recognition method, nonlinear features are obtained through image acquisition, processing, feature extraction, training model and feature matching steps, and nonlinear features are obtained by using convolutional neural networks, and multi-group features are spliced, weighted summing, feature mapping and Model integration to improve identification accuracy.
In complex environments, it improves recognition accuracy, enhances the robustness of the system, reduces the misidentification rate, and is easy to expand, and is suitable for a variety of occasions.
Smart Images

Figure CN118968587A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of face recognition technology, and in particular to a multi-feature combined face recognition method and device. Background Art
[0002] With the decline in computer hardware costs and the rapid development of various technologies, the scope of affairs that computers can help humans solve is increasing day by day. Computers have been able to upgrade from simple calculations to extracting the data and information people need from massive images, which has made information interaction more and more frequent in today's society, thus giving rise to a large number of online transactions. Most of the current security transaction verification methods rely on human memory or external devices, such as passwords, verification codes, identity certificates and other external information. This information is easily obtained by criminals through technical means, which poses a great threat to personal privacy or property. Therefore, using biometric recognition instead of traditional external information has become the first choice of various manufacturers or terminals. In biometric recognition, face recognition has always been a widely concerned research topic due to its complexity and wide application fields.
[0003] With the development of information technology, face recognition has become an important means of security authentication and personal identity verification. Traditional face recognition technology often relies on a single feature extraction method, such as based on geometric features, texture features or skin color features, which are limited in robustness and accuracy in complex environments; therefore, we propose a multi-feature joint face recognition method and device to solve this problem. Summary of the invention
[0004] The purpose of the present invention is to solve the shortcomings of traditional face recognition technology that often relies on a single feature extraction method such as based on geometric features, texture features or skin color features, and these methods have limited robustness and accuracy in complex environments, and to propose a multi-feature joint face recognition method and device.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A multi-feature joint face recognition method comprises the following steps:
[0007] S1. Image acquisition: For each face to be detected, collect its facial image;
[0008] S2, image processing: grayscale, normalize and remove noise from the collected facial images;
[0009] S3, feature extraction: extracting various types of facial features from the processed facial image;
[0010] S4, training model: using the algorithm to train face library samples and obtain a training model;
[0011] S5, feature matching: Matching multiple types of facial features through training models;
[0012] S6. Result export: output feedback information based on the facial feature matching results.
[0013] Preferably, in S1, when performing image acquisition, the following method is adopted:
[0014] S101, camera: directly photographing the face to be detected by the camera;
[0015] S102, monitoring: capturing facial images in the video captured by the monitoring camera;
[0016] S103, network collection: browsing web page information and collecting face images.
[0017] Preferably, in S3, when performing feature extraction, the following steps are included:
[0018] S301, dividing the image into 64×64 regions, encoding the pixels in each region, and then collecting the color data and texture data of each pixel;
[0019] S302, obtaining nonlinear features of the face image through a convolutional neural network, reducing the image resolution and retaining the features, and constructing a multi-level identification method for the face image through multiple sets of features.
[0020] Preferably, in S5, when performing feature matching, the following steps are included:
[0021] S501, concatenation: connect multiple feature maps in the depth dimension to form a richer feature representation;
[0022] S502, weighted summation: different feature maps are added element by element, and a weight coefficient is multiplied in front of each feature map;
[0023] S503, feature mapping: use convolution operation to adjust the size of the feature map so that it can be fused in the spatial dimension;
[0024] S504, Model Ensemble: Improve overall performance by training multiple models and combining their prediction results;
[0025] S505, multi-scale feature fusion: by capturing and matching feature information at different scales and levels.
[0026] The present invention also provides a multi-feature joint face recognition device, comprising: an image acquisition unit, an image processing unit, a feature extraction unit, a training model construction unit, a feature matching unit and a result output unit, wherein the image acquisition unit is used to acquire facial images of human faces, the image processing unit is used to preprocess the acquired images, the feature extraction unit is used to perform multi-feature extraction on the processed images, the training model construction unit is used to train face library samples using an algorithm and obtain a training model, wherein the feature matching unit is used to match multiple groups of types of facial features through the training model, and the result output unit is used to output facial feature matching results, the image acquisition unit is connected to the image processing unit, the feature processing unit is connected to the feature extraction unit, the training model construction unit and the feature matching unit are connected to the feature matching unit, and the feature matching unit is connected to the result output unit.
[0027] Preferably, the image acquisition unit includes an acquisition hardware module, a network acquisition module and a storage module. The acquisition hardware module includes a camera for capturing facial images and a surveillance camera for acquiring videos with faces. The storage module is used to temporarily store the acquired image information. Both the acquisition hardware module and the network acquisition module are connected to the storage module.
[0028] Preferably, the feature matching unit includes a splicing module, a weighted summation module, a feature mapping module, a model integration module and a feature fusion module. The splicing module is used to connect multiple feature maps in the depth dimension to form a richer feature representation. The weighted summation unit adds different feature maps element by element and multiplies a weight coefficient in front of each feature map. The feature mapping module uses a convolution operation to adjust the size of the feature map so that it can be fused in the spatial dimension; the model integration module improves the overall performance by training multiple models and combining their prediction results. The multi-scale feature fusion module captures feature information at different scales and levels and matches them.
[0029] Preferably, the result output unit includes a result temporary storage module, a result verification module and a communication unit. The result temporary storage unit is used to temporarily store the matching results. The result verification unit is used to compare and verify multiple matching results in the same group, and when deviations occur in the verification, re-extract features and match. When no deviations occur in the verification, the matching result information is sent out through the communication unit. The result temporary storage module is connected to the result verification module, and the result verification module is connected to the communication unit.
[0030] Compared with the prior art, the present invention provides a multi-feature combined face recognition method and device, which has the following beneficial effects:
[0031] (1) directly photographing the face to be detected by a camera, intercepting the face image in the video captured by the surveillance camera, browsing web information and collecting face images, the storage module temporarily stores the collected image information, and the image processing unit retrieves the face image information in the storage module, and performs grayscale, normalization and noise removal processing;
[0032] (2) The image is divided into 64×64 regions through the feature extraction module, the pixels in each region are encoded, and then the color data and texture data of each pixel are collected. The nonlinear features of the face image are obtained through the convolutional neural network, the image resolution is reduced while retaining the features, and a multi-level identification method of the face image is constructed through multiple sets of features;
[0033] (3) The training model construction unit uses an algorithm to train face library samples and obtain a training model. The splicing module connects multiple feature maps in the depth dimension to form a richer feature representation. The weighted summation unit adds different feature maps element by element and multiplies a weight coefficient in front of each feature map. The feature mapping module uses convolution operations to adjust the size of the feature map so that it can be fused in the spatial dimension. The model integration module improves the overall performance by training multiple models and combining their prediction results. The multi-scale feature fusion module captures feature information at different scales and levels and matches them.
[0034] (3) The result temporary storage unit is used to temporarily store the matching results, and the result verification unit is used to compare and verify multiple matching results in the same group, and when a deviation occurs in the verification, feature extraction and matching are performed again, and when no deviation occurs in the verification, matching result information is sent out through the communication unit;
[0035] (4) The combined use of multiple features improves the recognition accuracy in complex environments. The fusion of different types of features enhances the robustness of the system and reduces the misrecognition rate. The system can be applied in a variety of situations and is easy to expand. New feature extraction modules can be added as needed to cope with different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A flowchart of a multi-feature joint face recognition method proposed by the present invention;
[0037] Figure 2 This is a system block diagram of a multi-feature combined face recognition device proposed by the present invention.
[0038] In the figure: 1. Image acquisition unit; 2. Image processing unit; 3. Feature extraction unit; 4. Training model construction unit; 5. Feature matching unit; 6. Result output unit. DETAILED DESCRIPTION
[0039] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0040] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0041] Reference Figure 1 , a multi-feature joint face recognition method, comprising the following steps:
[0042] S1. Image acquisition: For each face to be detected, collect its facial image;
[0043] S2, image processing: grayscale, normalize and remove noise from the collected facial images;
[0044] S3, feature extraction: extracting various types of facial features from the processed facial image;
[0045] S4, training model: using the algorithm to train face library samples and obtain a training model;
[0046] S5, feature matching: Matching multiple types of facial features through training models;
[0047] S6. Result export: output feedback information based on the facial feature matching results.
[0048] In this embodiment, in S1, when performing image acquisition, the following method is adopted:
[0049] S101, camera: directly photographing the face to be detected by the camera;
[0050] S102, monitoring: capturing facial images in the video captured by the monitoring camera;
[0051] S103, network collection: browsing web page information and collecting face images.
[0052] In this embodiment, in S3, when performing feature extraction, the following steps are included:
[0053] S301, dividing the image into 64×64 regions, encoding the pixels in each region, and then collecting the color data and texture data of each pixel;
[0054] S302, obtaining nonlinear features of the face image through a convolutional neural network, reducing the image resolution and retaining the features, and constructing a multi-level identification method for the face image through multiple sets of features.
[0055] In this embodiment, in S5, when performing feature matching, the following steps are included:
[0056] S501, concatenation: connect multiple feature maps in the depth dimension to form a richer feature representation;
[0057] S502, weighted summation: different feature maps are added element by element, and a weight coefficient is multiplied in front of each feature map;
[0058] S503, feature mapping: use convolution operation to adjust the size of the feature map so that it can be fused in the spatial dimension;
[0059] S504, Model Ensemble: Improve overall performance by training multiple models and combining their prediction results;
[0060] S505, multi-scale feature fusion: by capturing feature information at different scales and levels and matching them.
[0061] Reference Figure 2 The present invention also provides a multi-feature joint face recognition device, including: an image acquisition unit, an image processing unit, a feature extraction unit, a training model construction unit, a feature matching unit and a result output unit, wherein the image acquisition unit is used to acquire facial images of human faces, the image processing unit is used to preprocess the acquired images, the feature extraction unit is used to extract multiple features from the processed images, the training model construction unit is used to train face library samples using an algorithm and obtain a training model, wherein the feature matching unit is used to match multiple groups of types of facial features through the training model, the result output unit is used to output facial feature matching results, the image acquisition unit is connected to the image processing unit, the feature processing unit is connected to the feature extraction unit, the training model construction unit and the feature matching unit are connected to the feature matching unit, and the feature matching unit is connected to the result output unit.
[0062] In this embodiment, the image acquisition unit includes an acquisition hardware module, a network acquisition module and a storage module. The acquisition hardware module includes a camera for capturing facial images and a surveillance camera for acquiring videos with faces. The storage module is used to temporarily store the acquired image information. The acquisition hardware module and the network acquisition module are both connected to the storage module.
[0063] In this embodiment, the feature matching unit includes a splicing module, a weighted summation module, a feature mapping module, a model integration module and a feature fusion module. The splicing module is used to connect multiple feature maps in the depth dimension to form a richer feature representation. The weighted summation unit adds different feature maps element by element and multiplies a weight coefficient in front of each feature map. The feature mapping module uses a convolution operation to adjust the size of the feature map so that it can be fused in the spatial dimension; the model integration module improves the overall performance by training multiple models and combining their prediction results. The multi-scale feature fusion module captures feature information at different scales and levels and matches them.
[0064] In this embodiment, the result output unit includes a result temporary storage module, a result verification module and a communication unit. The result temporary storage unit is used to temporarily store the matching results. The result verification unit is used to compare and verify multiple matching results in the same group, and when deviations occur in the verification, re-extract features and match. When no deviations occur in the verification, the matching result information is sent through the communication unit. The result temporary storage module is connected to the result verification module, and the result verification module is connected to the communication unit.
[0065] In this embodiment, when in use, the face to be detected is directly photographed by a camera, the face image in the video captured by the surveillance camera is intercepted, and the face image is collected by browsing web information, and the storage module temporarily stores the collected image information;
[0066] The facial image information in the storage module is retrieved through the image processing unit, and grayscale, normalized and noise removed;
[0067] The feature extraction module divides the image into 64×64 areas, encodes the pixels in each area, and then collects the color data and texture data of each pixel. The convolutional neural network is used to obtain the nonlinear features of the face image, reduce the image resolution and retain the features, and construct a multi-level identification method for the face image through multiple sets of features;
[0068] The training model building unit uses an algorithm to train face library samples and obtain a training model. The splicing module connects multiple feature maps in the depth dimension to form a richer feature representation. The weighted summation unit adds different feature maps element by element and multiplies a weight coefficient in front of each feature map. The feature mapping module uses convolution operations to adjust the size of the feature map so that it can be fused in the spatial dimension. The model integration module improves the overall performance by training multiple models and combining their prediction results. The multi-scale feature fusion module captures feature information at different scales and levels and matches them.
[0069] The result temporary storage unit is used to temporarily store the matching results, and the result verification unit is used to compare and verify multiple matching results in the same group. When deviation occurs in the verification, feature extraction and matching are performed again. When no deviation occurs in the verification, the matching result information is sent out through the communication unit.
[0070] The standard parts used in the present invention can all be purchased from the market, and special-shaped parts can be customized according to the instructions and the drawings. The specific connection methods of each part adopt conventional means such as mature bolts, rivets, welding, etc. in the prior art. Machinery, parts and equipment all adopt conventional models in the prior art, and the circuit connection adopts the conventional connection method in the prior art, which will not be described in detail here.
Claims
1. A multi-feature joint face recognition method, characterized in that: The steps include: S1. Image acquisition: For each face to be detected, collect its facial image; S2, image processing: grayscale, normalize and remove noise from the collected facial images; S3, feature extraction: extracting various types of facial features from the processed facial image; S4, training model: using the algorithm to train face library samples and obtain a training model; S5, feature matching: Matching multiple types of facial features through training models; S6. Result export: output feedback information based on the facial feature matching results.
2. A multi-feature joint face recognition method according to claim 1, characterized in that: In S1, when performing image acquisition, the following method is adopted: S101, camera: directly photographing the face to be detected by the camera; S102, monitoring: capturing facial images in the video captured by the monitoring camera; S103, network collection: browsing web page information and collecting face images.
3. The multi-feature combined face recognition method according to claim 1, characterized in that: In the above S3, when performing feature extraction, the following steps are included: S301, dividing the image into 64×64 regions, encoding the pixels in each region, and then collecting the color data and texture data of each pixel; S302, obtaining nonlinear features of the face image through a convolutional neural network, reducing the image resolution and retaining the features, and constructing a multi-level identification method for the face image through multiple sets of features.
4. The multi-feature combined face recognition method according to claim 1, characterized in that: In the above S5, when performing feature matching, the following steps are included: S501, concatenation: connect multiple feature maps in the depth dimension to form a richer feature representation; S502, weighted summation: different feature maps are added element by element, and a weight coefficient is multiplied in front of each feature map; S503, feature mapping: use convolution operation to adjust the size of the feature map so that it can be fused in the spatial dimension; S504, Model Ensemble: Improve overall performance by training multiple models and combining their prediction results; S505, multi-scale feature fusion: by capturing and matching feature information at different scales and levels.
5. A multi-feature joint face recognition device, comprising: An image acquisition unit, an image processing unit, a feature extraction unit, a training model construction unit, a feature matching unit and a result output unit. The image acquisition unit is used to acquire facial images of human faces. The image processing unit is used to pre-process the acquired images. The feature extraction unit is used to extract multiple features from the processed images. The training model construction unit is used to train human face library samples using an algorithm and obtain a training model. The feature matching unit is used to match multiple groups of facial features through a training model, the result output unit is used to output the facial feature matching results, the image acquisition unit is connected to the image processing unit, the feature processing unit is connected to the feature extraction unit, the training model construction unit and the feature matching unit are connected to the feature matching unit, and the feature matching unit is connected to the result output unit.
6. The multi-feature combined face recognition device according to claim 4, characterized in that: The image acquisition unit includes an acquisition hardware module, a network acquisition module and a storage module. The acquisition hardware module includes a camera for taking facial images and a surveillance camera for acquiring videos with faces. The storage module is used to temporarily store the acquired image information. The acquisition hardware module and the network acquisition module are both connected to the storage module.
7. The multi-feature combined face recognition device according to claim 4, characterized in that: The feature matching unit includes a splicing module, a weighted summation module, a feature mapping module, a model integration module and a feature fusion module. The splicing module is used to connect multiple feature maps in the depth dimension to form a richer feature representation. The weighted summation unit adds different feature maps element by element and multiplies a weight coefficient in front of each feature map. The feature mapping module uses a convolution operation to adjust the size of the feature map so that it can be fused in the spatial dimension; the model integration module improves the overall performance by training multiple models and combining their prediction results. The multi-scale feature fusion module captures feature information at different scales and levels and matches them.
8. The multi-feature combined face recognition device according to claim 4, characterized in that: The result output unit includes a result temporary storage module, a result verification module and a communication unit. The result temporary storage unit is used to temporarily store the matching results. The result verification unit is used to compare and verify multiple matching results in the same group, and when deviations occur in the verification, re-extract features and match. When no deviations occur in the verification, the matching result information is sent out through the communication unit. The result temporary storage module is connected to the result verification module, and the result verification module is connected to the communication unit.
