Face recognition method and device, computer device and storage medium

Abstract

The application relates to a face recognition method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a face image of a to-be-recognized object, and extracting a planar feature and a depth feature from the face image; performing first amplification processing on the planar feature to obtain an amplified planar feature used for enhancing face contour details of the to-be-recognized object; performing fusion processing on the amplified planar feature and the depth feature to obtain a fusion feature map of the face image; performing second amplification processing on the fusion feature map to obtain an amplified fusion feature map used for enhancing face dynamic change details of the to-be-recognized object; and performing face recognition based on the amplified fusion feature map to obtain a face recognition result of the to-be-recognized object. The method can extract detailed features in the face image, and can perform face recognition according to the detailed features, so that the accuracy and efficiency of face recognition can be improved. Moreover, the method can save training costs in the case of scarce face image data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a face recognition method and device, computer equipment and storage medium. BACKGROUND

[0002] With the development of financial technology, financial institutions can use face recognition technology for users to perform payment, permission authentication and other operations. In the process of face recognition, the face image of the to-be-recognized object is usually collected, and the face features in the face image are compared with the face features of the user who has recorded.

[0003] However, the existing face recognition technology is relatively rough in extracting and recognizing face features, usually only focusing on the surface features and static features of the face, ignoring the detailed features and dynamic features of the face, so that illegal users may use other users' face photos to pass the face recognition.

[0004] Therefore, the recognition accuracy of the existing face recognition technology is not high, and it is difficult to cope with high-precision face recognition scenarios such as real face dynamic state recognition and local face detail recognition. SUMMARY

[0005] Therefore, it is necessary to provide a face recognition method, device, computer equipment, computer readable storage medium and computer program product capable of extracting detailed features in a face image and performing face recognition according to multiple detailed features, thereby improving the accuracy and efficiency of face recognition, and saving training costs in the case of scarce face image data.

[0006] In a first aspect, the present application provides a face recognition method, comprising:

[0007] obtaining a face image of a to-be-recognized object, and extracting planar features and depth features from the face image;

[0008] performing first amplification processing on the planar features to obtain amplified planar features for enhancing the face contour details of the to-be-recognized object;

[0009] performing fusion processing on the amplified planar features and the depth features to obtain a fusion feature map of the face image;

[0010] performing second amplification processing on the fusion feature map to obtain an amplified fusion feature map for enhancing the face dynamic change details of the to-be-recognized object;

[0011] performing face recognition based on the amplified fusion feature map to obtain a face recognition result of the to-be-recognized object.

[0012] In one embodiment, the first magnification process of the planar feature to obtain the magnified planar feature includes:

[0013] The planar features are convolved to obtain convolutional planar features;

[0014] The channel self-interaction model performs channel information self-interaction processing on the convolutional plane features to obtain target convolutional plane features and channel weight matrix of the target convolutional plane features. Each channel of the target convolutional plane features contains different feature information, and the elements of the channel weight matrix represent the degree of influence between any two channels in the convolutional plane features.

[0015] The magnified plane features are obtained based on the convolutional plane features and the channel weight matrix.

[0016] In one embodiment, the channel self-interaction model performs channel information self-interaction processing on the convolutional plane features to obtain target convolutional plane features and a channel weight matrix of the target convolutional plane features, including:

[0017] The convolutional plane features are vectorized to obtain the first convolutional plane features and the second convolutional plane features;

[0018] The first convolutional plane feature and the second convolutional plane feature are subjected to channel information highlighting processing to obtain the target convolutional plane feature used to highlight the channel information of the convolutional plane feature;

[0019] The channel weight matrix is ​​obtained by normalizing each column of the target convolutional plane features.

[0020] In one embodiment, obtaining the magnified plane features based on the convolutional plane features and the channel weight matrix includes:

[0021] Based on the channel weight matrix, the transpose features of the convolutional plane features are subjected to a first reshaping process to calculate the degree of influence between the feature information of different channels in the convolutional plane features, and the first reshaping features are obtained.

[0022] After multiplying the first reshaped feature with the scaling parameter, a second reshaped process is performed with the convolutional planar feature to fuse the reshaped planar feature in the face image with the original planar feature, thereby obtaining the magnified planar feature.

[0023] In one embodiment, the fusion processing of the magnified planar features and the depth features to obtain the fused feature map of the face image includes:

[0024] The magnified planar features are subjected to pooling processing to obtain a magnified planar feature map;

[0025] The magnified planar feature map is fused with the depth feature map to obtain the fused feature map of the face image.

[0026] In one embodiment, the depth features are multiple, and the magnified planar feature map is a plurality of feature images with different resolutions. The step of fusing the magnified planar feature map with the depth features to obtain the fused feature map of the face image includes:

[0027] Obtain the fusion weights for each of the magnified planar feature maps;

[0028] For multiple magnified planar feature maps with different resolutions and multiple depth features, each magnified planar feature and each depth feature are weighted and fused one-to-one according to the fusion weight of each magnified planar feature map to obtain multiple fused feature maps with different resolutions.

[0029] In one embodiment, the second magnification processing of the fused feature map to obtain a magnified fused feature map for enhancing the details of facial dynamic changes of the object to be identified includes:

[0030] Differential sampling processing is performed on the multiple fused feature maps of different resolutions to obtain multiple differentially sampled fused feature maps of different resolutions. The differential sampling processing is used to capture the feature differences between the high-resolution fused feature map and the low-resolution fused feature map.

[0031] The multiple differentially sampled fused feature maps at different resolutions are downsampled to obtain multiple downsampled fused feature maps at different resolutions.

[0032] The multiple downsampled fused feature maps of different resolutions are convolved to obtain the magnified fused feature map.

[0033] In one embodiment, the step of performing differential sampling processing on the multiple fused feature maps of different resolutions to obtain multiple differentially sampled fused feature maps of different resolutions includes:

[0034] Upsampling is performed on the multiple fused feature maps of different resolutions to obtain multiple upsampled fused feature maps of different resolutions;

[0035] For the multiple fused feature maps and the multiple upsampled fused feature maps, the difference between each pair of fused feature maps with the same resolution and the upsampled fused feature map is calculated to capture the feature differences between the two, resulting in multiple difference fused feature maps with different resolutions.

[0036] For the multiple difference fusion feature maps of different resolutions, each difference fusion feature map is subjected to convolution processing to obtain multiple difference sampling fusion feature maps of different resolutions.

[0037] In one embodiment, the step of performing face recognition based on the magnified and fused feature map to obtain a face recognition result for the object to be recognized includes:

[0038] Perform planar information recognition and depth information recognition on the image being identified;

[0039] If both the planar information recognition and the depth information recognition results are successful, then the face recognition result is successful.

[0040] If the recognition result for at least one of the planar information and the depth information is a failure, then the face recognition result is a failure.

[0041] In one embodiment, acquiring a facial image of the object to be identified and extracting planar features and depth features from the facial image includes:

[0042] The planar image acquisition device and the depth image acquisition device of the binocular image acquisition device are used to acquire the planar image and the depth image of the face of the object to be identified, respectively.

[0043] The planar features are extracted from the planar image of the face, and the depth features are extracted from the depth image of the face.

[0044] Secondly, this application also provides a face recognition device, the device comprising:

[0045] The feature extraction module is used to acquire the face image of the object to be identified, and extract planar features and depth features from the face image;

[0046] The first magnification module is used to perform a first magnification process on the planar features to obtain magnified planar features for enhancing the facial contour details of the object to be identified;

[0047] The feature fusion module is used to fuse the magnified planar features and the depth features to obtain the fused feature map of the face image;

[0048] The second magnification module is used to perform a second magnification process on the fused feature map to obtain an magnified fused feature map for enhancing the details of the dynamic changes of the face of the object to be identified;

[0049] The face recognition module is used to perform face recognition based on the magnified and fused feature map to obtain the face recognition result of the object to be recognized.

[0050] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0051] Acquire a facial image of the object to be identified, and extract planar features and depth features from the facial image;

[0052] The planar features are subjected to a first magnification process to obtain magnified planar features used to enhance the facial contour details of the object to be identified;

[0053] The magnified planar features and the depth features are fused to obtain the fused feature map of the face image;

[0054] The fused feature map is subjected to a second magnification process to obtain an magnified fused feature map used to enhance the details of the dynamic changes of the face of the object to be identified;

[0055] Face recognition is performed based on the magnified and fused feature map to obtain the face recognition result of the object to be identified.

[0056] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0057] Acquire a facial image of the object to be identified, and extract planar features and depth features from the facial image;

[0058] The planar features are subjected to a first magnification process to obtain magnified planar features used to enhance the facial contour details of the object to be identified;

[0059] The magnified planar features and the depth features are fused to obtain the fused feature map of the face image;

[0060] The fused feature map is subjected to a second magnification process to obtain an magnified fused feature map used to enhance the details of the dynamic changes of the face of the object to be identified;

[0061] Face recognition is performed based on the magnified and fused feature map to obtain the face recognition result of the object to be identified.

[0062] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0063] Acquire a facial image of the object to be identified, and extract planar features and depth features from the facial image;

[0064] The planar features are subjected to a first magnification process to obtain magnified planar features used to enhance the facial contour details of the object to be identified;

[0065] The magnified planar features and the depth features are fused to obtain the fused feature map of the face image;

[0066] The fused feature map is subjected to a second magnification process to obtain an magnified fused feature map used to enhance the details of the dynamic changes of the face of the object to be identified;

[0067] Face recognition is performed based on the magnified and fused feature map to obtain the face recognition result of the object to be identified.

[0068] The aforementioned face recognition methods, devices, computer equipment, storage media, and computer program products acquire a face image of the object to be recognized and extract planar and depth features from the face image. During subsequent face recognition, both types of face features can be referenced simultaneously, ensuring the comprehensiveness of the recognition information. Then, by performing a first magnification process on the planar features, magnified planar features are obtained to enhance the details of the face contour of the object to be recognized, thereby highlighting the details of the face contour. The magnified planar features and depth features are then fused to obtain a fused feature map of the face image, thus fusing two different types of face detail features to obtain richer face features. A second magnification process is performed on the fused feature map to obtain a magnified fused feature map to enhance the details of the dynamic changes in the face of the object to be recognized, enabling face recognition of the object in both dynamic and static directions. Finally, face recognition is performed based on the magnified fused feature map to obtain the face recognition result of the object to be recognized. Therefore, the above face recognition process can improve the accuracy and efficiency of face recognition.

[0069] Furthermore, by employing the method of this application to perform multi-feature extraction and feature amplification on face images, it is possible to obtain information-rich face features based on small sample features. These face features can be used to train face recognition models or networks, thereby saving the cost of obtaining training samples and the computational resources required for training. This reduces the investment in data labels and training costs while ensuring the accuracy of model training. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1This is a diagram illustrating the application environment of a face recognition method in one embodiment.

[0072] Figure 2 This is a flowchart illustrating a face recognition method in one embodiment;

[0073] Figure 3 This is a schematic diagram illustrating the acquisition of magnified planar features in one embodiment;

[0074] Figure 4 This is a schematic diagram of a channel self-interaction model processing planar features in one embodiment;

[0075] Figure 5 This is a general schematic diagram of face image processing in one embodiment;

[0076] Figure 6 This is a schematic diagram illustrating differential sampling processing of multiple fused feature maps in one embodiment;

[0077] Figure 7 This is a structural block diagram of a face recognition device in one embodiment;

[0078] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0080] Financial institutions are increasingly adopting facial recognition technology to support user transactions and authentication, a process known as online verification. Traditionally, facial recognition involves acquiring an image of the person to be identified and comparing its features with those stored in the user's database for verification. However, current facial recognition technologies have limitations. They primarily focus on extracting and recognizing facial features, typically capturing only surface and static features while neglecting detailed facial features and dynamic changes. This makes online verification technology heavily reliant on image comparison, weakening liveness detection during the verification process. This can allow malicious actors to bypass facial recognition systems using other users' photos. Consequently, existing facial recognition technologies suffer from low accuracy and struggle to handle scenarios requiring high precision, such as recognizing the dynamic and static states of real faces and identifying local facial details.

[0081] The face recognition method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers. Using the facial recognition method of this application, terminal 102 can perform facial recognition on the object to be recognized and send the facial recognition result to server 104 for subsequent authorization authentication.

[0082] In the application scenario of this application, terminal 102 acquires a face image of the object to be identified and extracts planar features and depth features from the face image; performs a first magnification process on the planar features to obtain magnified planar features used to enhance the details of the face contour of the object to be identified; performs a fusion process on the magnified planar features and depth features to obtain a fused feature map of the face image; performs a second magnification process on the fused feature map to obtain a magnified fused feature map used to enhance the details of the dynamic changes of the face of the object to be identified; and performs face recognition based on the magnified fused feature map to obtain the face recognition result of the object to be identified.

[0083] In one exemplary embodiment, such as Figure 2 As shown, a face recognition method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps 202 to 206, wherein:

[0084] Step S202: Obtain the face image of the object to be identified, and extract planar features and depth features from the face image.

[0085] Planar features refer to the surface information of a face, such as the position, shape, color, and texture of facial key points like the facial contours, eyes, nose, and mouth. Depth features refer to the distance information between pixels mapped from a 3D portrait to a 2D image and the imaging matrix. They can represent the stereoscopic depth information of a face, i.e., the distance and depth differences between different parts of the face.

[0086] In one embodiment, multiple key points can be identified in the face image first, and then image processing and analysis can be performed on the coverage area of ​​each key point to extract the planar features of the face image. In another embodiment, a depth camera, depth sensor, or other device capable of providing depth information can be used to extract depth features. These devices can calculate the distance between different pixels or key points in the face image to obtain depth features.

[0087] Step S204: Perform a first magnification process on the planar features to obtain magnified planar features used to enhance the facial contour details of the object to be identified.

[0088] In one embodiment, the first magnification of the planar features may refer to employing a series of image processing techniques to enhance the detailed information in the face image of the object to be identified, such as the details of the facial contours. Through this first magnification process, the planar features can be transformed into clearer and more discriminative magnified planar features, thereby improving the accuracy of subsequent face recognition.

[0089] In practical implementation, the channel self-interaction model of this application, or the few-sample channel interaction module within the channel self-interaction model, can be used to analyze and adjust the planar features in a face image. The channel self-interaction model can help identify subtle differences and fine structures in a face image, thereby obtaining more detailed and accurate feature information. This allows for the differentiation of different face images through more detailed features, thus improving the recognition performance for different faces.

[0090] Step S206: The magnified planar features and depth features are fused to obtain the fused feature map of the face image.

[0091] In one embodiment, fusing planar and depth features means combining the information from these two types of features to create a comprehensive facial image feature from different data sources. This fusion process can fully utilize the feature information from both planar and depth features. For example, it can fuse information such as shape, color, and texture contained in planar features with the depth information of depth features to improve the ability to describe and discriminate the details of the facial image of the object to be identified.

[0092] Step S208: Perform a second magnification process on the fused feature map to obtain an magnified fused feature map used to enhance the details of the dynamic changes of the face of the object to be identified.

[0093] In one embodiment, a second amplification process, including operations such as convolution and feature mapping, can be performed on the fused feature map using a channel self-interaction model. It is understood that since the fused feature map has already incorporated information from amplified planar and depth features, the second amplification process can amplify the relationship between changes in facial contours, facial features, texture, and depth, further enhancing the dynamic details in the feature map. This improves the ability to perceive the dynamic changes in the face of the object to be identified, as well as the accuracy and stability of face recognition in different states.

[0094] Step S210: Perform face recognition based on the magnified and fused feature map to obtain the face recognition result of the object to be recognized.

[0095] In one embodiment, the face recognition result may include face detection results and depth detection results. The face detection result can be used to detect planar information of the face to verify whether the object to be identified is an authorized target user. Correspondingly, the depth detection result can be used to detect whether the depth information of the face is accurate, which can be understood as a kind of liveness detection to prevent unauthorized users from using static images of the object to be identified to perform fraudulent face recognition.

[0096] In one embodiment, a logical AND operation can be performed on the facial recognition results from the two parts mentioned above. It can be understood that through this logical AND operation, only when both facial recognition results are correct can it be determined that the facial recognition was performed on behalf of the person being identified, without any other user impersonating them or using static images for fraudulent identification. Only then can the facial recognition result for the person being identified be determined as "yes" or "recognition passed." Conversely, if the logical AND operation result is incorrect, meaning one or both facial recognition results are incorrect, the system will determine the facial recognition result as "no" or "recognition failed," thus rejecting further operations for the person being identified or requiring further identification and verification.

[0097] This method enhances the security of facial recognition by combining planar and depth information of the face to reduce the risk of fraudulent identification, ensuring that only genuine user faces with correct depth information can pass the recognition.

[0098] The aforementioned face recognition methods, devices, computer equipment, storage media, and computer program products acquire a face image of the object to be recognized and extract planar and depth features from the face image. During subsequent face recognition, both types of face features can be referenced simultaneously, ensuring the comprehensiveness of the recognition information. Then, by performing a first magnification process on the planar features, magnified planar features are obtained to enhance the details of the face contour of the object to be recognized, thereby highlighting the details of the face contour. The magnified planar features and depth features are then fused to obtain a fused feature map of the face image, thus fusing two different types of face detail features to obtain richer face features. A second magnification process is performed on the fused feature map to obtain a magnified fused feature map to enhance the details of the dynamic changes in the face of the object to be recognized, enabling face recognition of the object in both dynamic and static directions. Finally, face recognition is performed based on the magnified fused feature map to obtain the face recognition result of the object to be recognized. Therefore, the above face recognition process can improve the accuracy and efficiency of face recognition.

[0099] Furthermore, by employing the method of this application to perform multi-feature extraction and feature amplification on face images, it is possible to obtain information-rich face features based on small sample features. These face features can be used to train face recognition models or networks, thereby saving the cost of obtaining training samples and the computational resources required for training. This reduces the investment in data labels and training costs while ensuring the accuracy of model training.

[0100] In one exemplary embodiment, such as Figure 3 The diagram shown illustrates the process of generating a page data packet in one embodiment. Step S206 includes the following steps:

[0101] Step S302: Perform convolution processing on the planar features to obtain convolutional planar features.

[0102] In one embodiment, planar features can be convolved using a convolution kernel. For example, a 3x3 convolution kernel (Conv3x3) can be slid across the image of the planar features and convolve with local regions of the image to detect different information in the planar features, thus obtaining convolved planar features f∈R. C×H×W Where C represents the number of channels, H represents the height, and W represents the width.

[0103] Step S304: The channel self-interaction model performs channel information self-interaction processing on the convolutional plane features to obtain the target convolutional plane features and the channel weight matrix of the target convolutional plane features.

[0104] Please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating how a channel self-interaction model processes planar features in one embodiment. For example...Figure 4 As shown, convolutional plane features can be input into a channel self-interaction model. In one embodiment, channel information self-interaction processing refers to the interaction, integration, or adjustment of feature information between different channels in the convolutional plane features to improve feature representation and better capture the importance and correlation between feature information of different channels.

[0105] Each channel of the target convolutional plane feature can contain different feature information. For example, each channel can contain various feature information such as edges, textures, colors, and shapes of different parts of the face, so that the feature information of the target convolutional plane feature obtained through the self-interactive model has strong diversity and richness.

[0106] In this matrix, the elements of the channel weight matrix represent the degree of influence between any two channels in the convolutional plane features. For example, the i-th value in the j-th column of the channel weight matrix X can be Xj. ij , representing the degree of influence of the information in the i-th channel on the information in the j-th channel. In one embodiment, X ij It can be calculated using the following formula:

[0107]

[0108] Where exp is the natural exponential function, used to transform the input value (f) i .f j Mapping to nonnegative real numbers is often used to emphasize differences between numerical values. i and f j These are the features of the i-th and j-th channels in the convolutional plane, respectively. f represents the sum of all channels i and f j Summation, where c is the number of channels.

[0109] It is understandable that if f i and f j If the correlation between them is high, meaning they contain similar information, then X ij A larger value for X indicates that the i-th channel has a greater influence on the j-th channel. Conversely, if the correlation between them is low, X... ij A smaller value indicates that the i-th channel has a smaller influence on the j-th channel. Each element of the channel weight matrix accurately gives the degree of mutual influence between the feature information of different channels in the convolutional plane feature, so as to more accurately extract and utilize the features of face images in subsequent face recognition tasks.

[0110] In one exemplary embodiment, step S304 may include:

[0111] The convolutional plane features are vectorized to obtain the first convolutional plane features and the second convolutional plane features;

[0112] The first convolutional plane feature and the second convolutional plane feature are processed to highlight the channel information, resulting in the target convolutional plane feature used to highlight the channel information of the convolutional plane feature.

[0113] Normalize each column of the target convolutional plane features to obtain the channel weight matrix.

[0114] In one embodiment, the convolutional planar features f∈R can be... C×H×W Vectorization is performed by stacking and flattening the pixels of each channel in the 3D convolutional plane feature into a one-dimensional vector, resulting in the first convolutional plane feature f1∈R. C×N Second convolutional plane features f2∈R N×C , where N = H × W.

[0115] In one embodiment, the target convolutional planar feature f3 can be calculated using the following formula:

[0116] f3 = f1 × f2, f3 ∈ R C×C ;

[0117] The target convolutional planar feature f3 is a C×C matrix. Each element in the target convolutional planar feature f3 represents the product between a channel of the first planar feature and a channel of the second planar feature, in order to obtain the correlation and importance information between the feature information of the channels of the two planar features.

[0118] Step S306: Obtain the magnified plane features based on the convolutional plane features and the channel weight matrix.

[0119] It is understandable that, since the channel weight matrix can represent the degree of influence of the information contained between any two channels in the convolutional plane feature, the convolutional plane feature can be processed by the channel weight matrix to highlight the important information in a specific channel and obtain magnified planar features to enhance the facial contour details of the object to be identified.

[0120] In one exemplary embodiment, step S306 may include:

[0121] Based on the channel weight matrix, the transpose features of the convolutional plane features are reshaped in the first remodeling process to calculate the degree of influence between the feature information of different channels in the convolutional plane features, and thus obtain the first remodeled features.

[0122] After multiplying the first reshaped feature with the scaling parameter, a second reshaped process is performed with the convolutional planar feature to fuse the reshaped planar feature in the face image with the original planar feature, resulting in an enlarged planar feature.

[0123] In one embodiment, the magnified planar feature E∈R can be calculated using the following formula. C×H×W :

[0124]

[0125] Among them, E j This represents the j-th element in the magnified planar feature E, where β is the scaling parameter, C represents the number of channels, i represents the channel index, and f i X represents the i-th channel in the convolutional planar feature. ij f represents the element in the i-th row and j-th column of the channel weight matrix X. j This represents the j-th channel in the convolutional planar feature.

[0126] like Figure 4 As shown, in one embodiment, The first reshaping feature is equivalent to weighting each channel in the convolutional plane feature f using the channel weight matrix X, so that the convolutional plane feature can be accurately enhanced based on the importance of each channel in the subsequent process.

[0127] like Figure 4 As shown, in one embodiment, the second reshaping process refers to performing an element-wise summation operation on the first reshaping feature and the original convolutional planar feature, thereby enabling the enhanced feature information in the first reshaping feature to be fused into the complete original feature information in a weighted summation manner to obtain the complete weighted summation feature information, thereby ultimately obtaining the magnified planar feature used to enhance the facial contour details of the object to be identified.

[0128] In one exemplary embodiment, step S202 may include:

[0129] The planar image acquisition device and the depth image acquisition device of the binocular image acquisition device are used to acquire the planar image and the depth image of the face of the object to be identified, respectively.

[0130] Extract planar features from a planar image of a face, and extract depth features from a depth image of a face.

[0131] As can be understood, binocular image acquisition devices can simulate the way human eyes observe the world. Through their included planar image acquisition devices and depth image acquisition devices, they can simultaneously acquire information from different dimensions, thereby providing rich facial features for subsequent processing and analysis. For example, these features can be used in applications such as face recognition and facial liveness detection, thereby improving the accuracy and security of face recognition.

[0132] In one exemplary embodiment, step S206 may include:

[0133] Pooling is applied to the magnified planar features to obtain the magnified planar feature map;

[0134] The magnified planar feature map is fused with the depth feature map to obtain the fused feature map of the face image.

[0135] Please refer to Figure 5 , Figure 5 This is a general schematic diagram illustrating the processing of a face image in one embodiment. Figure 5 As shown, the magnified planar features output from the small sample channel interaction module can retain important feature information through max pooling, and then be fused with deep features. In specific implementations, methods such as channel fusion and feature concatenation can be used to fuse the above two types of features, ultimately obtaining a fused feature map corresponding to the face image. The fused feature map can combine planar and depth information to provide a more comprehensive and richer feature representation.

[0136] By using the above methods, feature information from different dimensions can be fused together, so that subsequent face recognition tasks can improve the efficiency and accuracy of face recognition by more accurately analyzing and processing the fused features.

[0137] The depth features are multiple, and the magnified planar feature map consists of multiple feature images at different resolutions. To amplify weak signals in liveness detection, a weak signal feature fusion module can be used to further process the fused features. In one embodiment, multiple feature images at different resolutions can be output from multiple different layers of the backbone network in the face recognition network. For example... Figure 5 As shown, taking a four-layer backbone network as an example, H can be output respectively. 32×32 H 64×64 H 128×128 and H 256×256 These four magnified planar feature maps are used as inputs to the weak signal feature fusion module. The subscripts represent the resolution of each magnified planar feature map.

[0138] In an exemplary embodiment, the step of fusing the magnified planar feature map with the depth feature map to obtain a fused feature map of the face image includes:

[0139] Obtain the fusion weights for each magnified planar feature map;

[0140] For multiple magnified planar feature maps and multiple depth features at different resolutions, each magnified planar feature and each depth feature are weighted and fused one-to-one according to the fusion weight of each magnified planar feature map to obtain multiple fused feature maps at different resolutions.

[0141] In one embodiment, multiple fused feature maps F at different resolutions can be obtained using the following calculation formula:

[0142] F 256×256 =H 256×256 ×V×a;

[0143] F 128×128 =H 128×128 ×V×b;

[0144] F 64×64 =H 64×64 ×V×c;

[0145] F 32×32 =H 32×32 ×V×d;

[0146] Where V represents the depth feature, and a, b, c, and d are the fusion weights corresponding to different magnified planar feature maps. It can be understood that fusing a magnified planar feature map of a certain resolution with the depth feature yields a fused feature map of the same resolution, and the subscripts of F and H represent their respective resolutions.

[0147] In an exemplary embodiment, step S208 includes:

[0148] Differential sampling is performed on multiple fused feature maps of different resolutions to obtain multiple differentially sampled fused feature maps of different resolutions. The differential sampling is used to capture the feature differences between the high-resolution fused feature map and the low-resolution fused feature map.

[0149] Multiple differentially sampled fused feature maps at different resolutions are downsampled to obtain multiple downsampled fused feature maps at different resolutions.

[0150] Multiple downsampled fused feature maps of different resolutions are convolved to obtain magnified fused feature maps.

[0151] The purpose of differential sampling is to capture the differences between high-resolution and low-resolution fusion feature maps, helping the model better understand the information differences between different resolution levels. Subsequently, downsampling is performed on the resulting multiple differentially sampled fusion feature maps to match and align the differences between the maps at different resolutions. Further processing via convolution is then used to obtain richer information. This allows the weak signal feature fusion module to optimize the fusion and differential features at different resolution levels, improving the model performance of the face recognition network.

[0152] In an exemplary embodiment, the step of performing differential sampling processing on multiple fused feature maps of different resolutions to obtain multiple differentially sampled fused feature maps of different resolutions includes:

[0153] Upsampling is performed on multiple fused feature maps at different resolutions to obtain multiple upsampled fused feature maps at different resolutions.

[0154] For multiple fused feature maps and multiple upsampled fused feature maps, the difference between each pair of fused feature maps with the same resolution and the upsampled fused feature map is calculated to capture the feature differences between the two, resulting in multiple difference fused feature maps with different resolutions.

[0155] For multiple difference fusion feature maps at different resolutions, each difference fusion feature map is convolved to obtain multiple difference sampling fusion feature maps at different resolutions.

[0156] In one embodiment, multiple differential sampling fusion feature maps S can be obtained using the following formula:

[0157] S 256×256 =C(F 256×256 -U(F 128×128 ));

[0158] S 128×128 =C(F 128×128 -U(F 64×64 ));

[0159] S 64×64 =C(F 64×64 -U(F 32×32 ));

[0160] Where C (Conv) represents performing a convolution operation, U (Upsample) represents upsampling the feature at this resolution, and D (Downsample) represents downsampling the feature at this resolution. For example... Figure 6 As shown, the fused feature map F can be processed separately. 32×32 F 64×64 and F 128×128Upsampling is performed to obtain the upsampled fused feature map F6. ′ 4×64 F1 ′ 28×128 and F2 ′ 56×256 It can be understood that the purpose of calculating multiple differentially sampled fusion feature maps is to further enhance the representational ability of features by capturing the feature differences between the fusion feature map and the upsampled fusion feature map at different resolution levels. This allows the face recognition model or network used to implement this method to better understand information at different resolution levels, thereby improving the efficiency and accuracy of face recognition.

[0161] In one embodiment, the magnified fused feature map O can be calculated using the following formula:

[0162] O 32×32 =C(D(S) 256×256 ),D(S 128×128 ),D(S 64×64 ),F 32×32 );

[0163] Among them, multiple differential sampling and fusion feature maps S can be performed separately. 256×256 S 128×128 and S 64×64 Downsampling is performed to downsample the resolution of the above differentially sampled fused feature maps to 32×32. After merging and fusing, a magnified fused feature map O with the same 32×32 resolution is generated. 32×32 By adjusting the difference sampling fusion feature maps of different resolutions to the same spatial size and then stitching and convolving them, the face recognition model or network of this application can directly learn useful information from features of different depths, thereby improving the performance of the model.

[0164] In an exemplary embodiment, step S210 includes:

[0165] Perform planar information recognition and depth information recognition on the image being identified;

[0166] If both the planar information recognition and the depth information recognition results are successful, then the face recognition result is successful.

[0167] If the recognition result for at least one of the planar information and the depth information is a failure, then the face recognition result is a failure.

[0168] As mentioned in step S210 above, a logical AND operation can be performed on the face recognition results of the two types of information. The face recognition operations for these two types of information can be performed independently to determine whether the planar features and depth features match the known information. Specifically, if both planar information recognition and depth information recognition are successful, meaning both match the known information, the face recognition result is considered successful. This is equivalent to the object being recognized undergoing dual recognition and passing the recognition. Conversely, if at least one of the planar information recognition and depth information recognition results in failure, meaning one does not match the known information, the face recognition result is considered unsuccessful, indicating that the object being recognized failed this face recognition attempt.

[0169] The above methods enable multi-level verification to determine the identity of the object to be identified. Only when both planar and depth information are verified successfully will the face recognition result be confirmed, thereby improving the accuracy and security of face recognition.

[0170] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0171] Based on the same inventive concept, this application also provides a face recognition device for implementing the face recognition method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more face recognition device embodiments provided below can be found in the limitations of the face recognition method described above, and will not be repeated here.

[0172] In one exemplary embodiment, such as Figure 7 As shown, a face recognition device is provided, including: a feature extraction module 410, a first magnification module 420, a feature fusion module 430, a second magnification module 440, and a face recognition module 450, wherein:

[0173] The feature extraction module 410 is used to acquire the face image of the object to be identified and extract planar features and depth features from the face image;

[0174] The first magnification module 420 is used to perform a first magnification process on the planar features to obtain magnified planar features for enhancing the facial contour details of the object to be identified;

[0175] The feature fusion module 430 is used to fuse magnified planar features and depth features to obtain a fused feature map of the face image.

[0176] The second amplification module 440 is used to perform a second amplification processing on the fused feature map to obtain an amplified fused feature map for enhancing the details of the dynamic changes of the face of the object to be identified;

[0177] The face recognition module 450 is used to perform face recognition based on magnified and fused feature maps to obtain the face recognition result of the object to be recognized.

[0178] The aforementioned face recognition methods, devices, computer equipment, storage media, and computer program products acquire a face image of the object to be recognized and extract planar and depth features from the face image. During subsequent face recognition, both types of face features can be referenced simultaneously, ensuring the comprehensiveness of the recognition information. Then, by performing a first magnification process on the planar features, magnified planar features are obtained to enhance the details of the face contour of the object to be recognized, thereby highlighting the details of the face contour. The magnified planar features and depth features are then fused to obtain a fused feature map of the face image, thus fusing two different types of face detail features to obtain richer face features. A second magnification process is performed on the fused feature map to obtain a magnified fused feature map to enhance the details of the dynamic changes in the face of the object to be recognized, enabling face recognition of the object in both dynamic and static directions. Finally, face recognition is performed based on the magnified fused feature map to obtain the face recognition result of the object to be recognized. Therefore, the above face recognition process can improve the accuracy and efficiency of face recognition.

[0179] Furthermore, by employing the method of this application to perform multi-feature extraction and feature amplification on face images, it is possible to obtain information-rich face features based on small sample features. These face features can be used to train face recognition models or networks, thereby saving the cost of obtaining training samples and the computational resources required for training. This reduces the investment in data labels and training costs while ensuring the accuracy of model training.

[0180] The various modules in the aforementioned face recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0181] Figure 8In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a facial recognition method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0182] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0183] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0184] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0185] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0187] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A face recognition method, characterized in that, The method includes: Acquire a facial image of the object to be identified, and extract planar features and depth features from the facial image; The planar features are subjected to a first magnification process to obtain magnified planar features used to enhance the facial contour details of the object to be identified; The magnified planar features and the depth features are fused to obtain the fused feature map of the face image; The fused feature map is subjected to a second magnification process to obtain an magnified fused feature map used to enhance the details of the dynamic changes of the face of the object to be identified; Face recognition is performed based on the magnified and fused feature map to obtain the face recognition result of the object to be identified; The process of fusing the magnified planar features and the depth features to obtain a fused feature map of the face image includes: performing pooling processing on the magnified planar features to obtain a magnified planar feature map; and fusing the magnified planar feature map with the depth features to obtain a fused feature map of the face image. The depth features are multiple, and the magnified planar feature maps are multiple feature images with different resolutions. The magnified planar feature maps are fused with the depth features to obtain a fused feature map of the face image. This includes: obtaining the fusion weight of each magnified planar feature map; and, for the multiple magnified planar feature maps with different resolutions and the multiple depth features, performing a one-to-one weighted fusion process on each magnified planar feature map and each depth feature according to the fusion weight of each magnified planar feature map to obtain multiple fused feature maps with different resolutions.

2. The method according to claim 1, characterized in that, The first magnification process of the planar feature to obtain the magnified planar feature includes: The planar features are convolved to obtain convolutional planar features; The channel self-interaction model performs channel information self-interaction processing on the convolutional plane features to obtain target convolutional plane features and channel weight matrix of the target convolutional plane features. Each channel of the target convolutional plane features contains different feature information, and the elements of the channel weight matrix represent the degree of influence between any two channels in the convolutional plane features. The magnified plane features are obtained based on the convolutional plane features and the channel weight matrix.

3. The method according to claim 2, characterized in that, The channel self-interaction model performs channel information self-interaction processing on the convolutional plane features to obtain target convolutional plane features and the channel weight matrix of the target convolutional plane features, including: The convolutional plane features are vectorized to obtain the first convolutional plane features and the second convolutional plane features; The first convolutional plane feature and the second convolutional plane feature are subjected to channel information highlighting processing to obtain the target convolutional plane feature used to highlight the channel information of the convolutional plane feature; The channel weight matrix is ​​obtained by normalizing each column of the target convolutional plane features.

4. The method according to claim 3, characterized in that, The step of obtaining the magnified plane features based on the convolutional plane features and the channel weight matrix includes: Based on the channel weight matrix, the transpose features of the convolutional plane features are subjected to a first reshaping process to calculate the degree of influence between the feature information of different channels in the convolutional plane features, and the first reshaping features are obtained. After multiplying the first reshaped feature with the scaling parameter, a second reshaped process is performed with the convolutional planar feature to fuse the reshaped planar feature in the face image with the original planar feature, thereby obtaining the magnified planar feature.

5. The method according to claim 1, characterized in that, The second amplification process of the fused feature map to obtain an amplified fused feature map for enhancing the details of dynamic facial changes of the object to be identified includes: Differential sampling processing is performed on the multiple fused feature maps of different resolutions to obtain multiple differentially sampled fused feature maps of different resolutions. The differential sampling processing is used to capture the feature differences between the high-resolution fused feature map and the low-resolution fused feature map. The multiple differentially sampled fused feature maps at different resolutions are downsampled to obtain multiple downsampled fused feature maps at different resolutions. The multiple downsampled fused feature maps of different resolutions are convolved to obtain the magnified fused feature map.

6. The method according to claim 5, characterized in that, The step of performing differential sampling processing on the multiple fused feature maps of different resolutions to obtain multiple differentially sampled fused feature maps of different resolutions includes: Upsampling is performed on the multiple fused feature maps of different resolutions to obtain multiple upsampled fused feature maps of different resolutions; For the multiple fused feature maps and the multiple upsampled fused feature maps, the difference between each pair of fused feature maps with the same resolution and the upsampled fused feature map is calculated to capture the feature differences between the two, resulting in multiple difference fused feature maps with different resolutions. For the multiple difference fusion feature maps of different resolutions, each difference fusion feature map is subjected to convolution processing to obtain multiple difference sampling fusion feature maps of different resolutions.

7. The method according to any one of claims 1 to 6, characterized in that, The process of performing face recognition based on the magnified and fused feature map to obtain the face recognition result for the object to be recognized includes: Perform planar information recognition and depth information recognition on the facial image of the object to be identified; If both the planar information recognition and the depth information recognition results are successful, then the face recognition result is successful. If the recognition result for at least one of the planar information and the depth information is a failure, then the face recognition result is a failure.

8. The method according to claim 1, characterized in that, The process of acquiring a facial image of the object to be identified and extracting planar and depth features from the facial image includes: The planar image acquisition device and the depth image acquisition device of the binocular image acquisition device are used to acquire the planar image and the depth image of the face of the object to be identified, respectively. The planar features are extracted from the planar image of the face, and the depth features are extracted from the depth image of the face.

9. A face recognition device, used to execute the face recognition method according to any one of claims 1 to 8, characterized in that, The device includes: The feature extraction module is used to acquire the face image of the object to be identified, and extract planar features and depth features from the face image; The first magnification module is used to perform a first magnification process on the planar features to obtain magnified planar features for enhancing the facial contour details of the object to be identified; The feature fusion module is used to fuse the magnified planar features and the depth features to obtain the fused feature map of the face image; The second magnification module is used to perform a second magnification process on the fused feature map to obtain an magnified fused feature map for enhancing the details of the dynamic changes of the face of the object to be identified; The face recognition module is used to perform face recognition based on the magnified and fused feature map to obtain the face recognition result of the object to be recognized.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images