A face occlusion recognition method and device, electronic equipment and storage medium
By processing occluded face images using convolutional neural networks and multi-scale attention networks to generate an attention weight matrix, the problem of requiring extensive manual intervention for face recognition in occluded scenarios is solved, achieving highly efficient face recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING ZHIDA TIANJIE COMMERCIAL OPERATION MANAGEMENT CO LTD
- Filing Date
- 2023-01-09
- Publication Date
- 2026-07-24
AI Technical Summary
Current technologies for facial recognition in occluded scenarios require significant manual intervention, resulting in high labor costs and poor recognition performance.
A convolutional neural network is used for feature extraction, and a multi-scale attention network is used to process the facial feature map to generate an attention weight matrix, which is then input into the recognition network to output the recognition result.
It greatly reduces the need for manual intervention, significantly improves facial recognition performance in occluded scenarios, and reduces labor costs.
Smart Images
Figure CN116129499B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence, and more particularly to a method, apparatus, electronic device, and storage medium for occluded facial recognition. Background Technology
[0002] With the continuous development of deep learning, the facial recognition effect in general scenarios is getting better and better, but when faced with obstructed scenarios such as masks and sunglasses, the facial recognition effect is often very poor.
[0003] Existing methods for solving face recognition in occluded scenarios typically rely on manually generating large-scale occluded face data and then using this data to train and improve recognition performance. However, this approach not only requires significant manual intervention and incurs high labor costs, but also fails to fundamentally address the problem of face recognition in occluded scenarios, resulting in still relatively poor performance. Summary of the Invention
[0004] In view of this, the present disclosure provides a method, apparatus, electronic device and storage medium for occluded face recognition, in order to solve the problems in the prior art that face recognition in occluded scenarios requires a lot of manual intervention, resulting in high labor costs and poor recognition performance.
[0005] A first aspect of this disclosure provides a method for occluded face recognition, including:
[0006] Obtain a face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image;
[0007] The face image to be identified is input into a convolutional neural network for feature extraction to obtain a face feature map;
[0008] The face feature map is input into a multi-scale attention network for processing to obtain the attention weight matrix of the face feature map;
[0009] The face feature map and the attention weight matrix are input into the recognition network, and the face recognition result of the face image to be recognized is output.
[0010] A second aspect of this disclosure provides a face recognition device that occludes faces, including:
[0011] The acquisition module is configured to acquire a face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image;
[0012] The extraction module is configured to input the face image to be identified into a convolutional neural network for feature extraction to obtain a face feature map;
[0013] The processing module is configured to input the face feature map into a multi-scale attention network for processing to obtain the attention weight matrix of the face feature map;
[0014] The recognition module is configured to input the face feature map and the attention weight matrix into the recognition network and output the face recognition result of the face image to be recognized.
[0015] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.
[0016] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.
[0017] Compared with the prior art, the beneficial effects of this disclosed embodiment include at least the following: by acquiring a face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image; inputting the face image to be identified into a convolutional neural network for feature extraction to obtain a face feature map; inputting the face feature map into a multi-scale attention network for processing to obtain an attention weight matrix of the face feature map; inputting the face feature map and the attention weight matrix into a recognition network to output the face recognition result of the face image to be identified, which greatly reduces the degree of manual intervention, reduces labor costs, and can fundamentally solve the face recognition problem in occluded scenarios, significantly improving the face recognition effect. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a schematic flowchart of a face recognition method with occlusion provided in an embodiment of this disclosure;
[0020] Figure 2 This is a schematic diagram of the network structure of an occluded face recognition model in the occluded face recognition method provided in this embodiment of the present disclosure;
[0021] Figure 3 This is a schematic diagram of the network structure of a recognition network 203 in the occluded face recognition method provided in this embodiment of the present disclosure;
[0022] Figure 4 This is a schematic diagram of the structure of a face recognition device with occlusion provided in an embodiment of this disclosure;
[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.
[0025] A method and apparatus for occluded face recognition according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.
[0026] Figure 1 This is a schematic flowchart of a face recognition method with occlusion provided in an embodiment of this disclosure. Figure 1 As shown, the occluded face recognition method includes:
[0027] Step S101: Obtain the face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image.
[0028] The image to be identified can be an image / picture containing a face captured by a camera device (such as a monocular camera, a binocular camera, a surveillance camera, etc.). It can also be an image / picture containing a face extracted from a video stream captured by the camera device. The face image to be identified can be an unobstructed face image or an obstructed face image. An unobstructed face image typically refers to a face image showing the entire face without any obstructions. An obstructed face image typically refers to a face image showing a mask, glasses or sunglasses, a hat, or a face image showing only a side profile (not a full-face view).
[0029] Step S102: Input the face image to be identified into a convolutional neural network for feature extraction to obtain a face feature map.
[0030] Convolutional neural networks can be deep residual networks, such as the ResNet50 convolutional neural network. The ResNet50 convolutional neural network contains 49 convolutional layers and one fully connected layer.
[0031] Using the ResNet50 convolutional neural network to extract features from the face images to be recognized can not only reduce the number of parameters to a certain extent, but also reduce the amount of computation, thereby improving the efficiency of face recognition.
[0032] Step S103: Input the face feature map into a multi-scale attention network for processing to obtain the attention weight matrix of the face feature map.
[0033] A facial image often contains rich feature information, including low-level features such as color, brightness, texture, and orientation, as well as high-level features such as pose, expression, age, and ethnicity. This embodiment processes the facial feature map by inputting it into the multi-scale attention network designed in this disclosure, which can extract the rich feature information from the facial image better, faster, and more comprehensively, thereby improving the subsequent facial recognition performance.
[0034] The attention weight matrix is a 512-dimensional matrix composed of the attention values (0 to 1 values) of each channel of the face feature map obtained after passing through a multi-scale attention network.
[0035] Step S104: Input the face feature map and the attention weight matrix into the recognition network, and output the face recognition result of the face image to be recognized.
[0036] The technical solution provided in this disclosure involves acquiring a face image to be recognized, which may be an occluded or unoccluded face image; inputting the face image to be recognized into a convolutional neural network for feature extraction to obtain a face feature map; inputting the face feature map into a multi-scale attention network for processing to obtain an attention weight matrix of the face feature map; and inputting the face feature map and the attention weight matrix into a recognition network to output the face recognition result of the face image to be recognized. This significantly reduces the degree of manual intervention and lowers labor costs, while fundamentally solving the face recognition problem in occluded scenarios and significantly improving the face recognition effect.
[0037] Figure 2 This is a schematic diagram of the network structure of an occluded face recognition model in the occluded face recognition method provided in this embodiment.
[0038] like Figure 2As shown, the occluded face recognition model includes a convolutional neural network 201, a multi-scale attention network 202, and a recognition network 203. The convolutional neural network 201 includes a first network layer 2011, a second network layer 2012, a third network layer 2013, a fourth network layer 2014, and a fifth network layer 2015 connected in sequence. The first network layer 2011 does not contain residual blocks and mainly includes convolutional blocks and pooling blocks. The second network layer 2012, the third network layer 2013, the fourth network layer 2014, and the fifth network layer 2015 all contain residual blocks. The multi-scale attention network 202 includes a first input layer 2021, a second input layer 2022, and a third input layer 2023; a feature fusion layer 2024 connected to the first input layer 2021, the second input layer 2022, and the third input layer 2023; a global average pooling layer 2025 connected to the feature fusion layer 2024; a first fully connected layer 2026 connected to the global average pooling layer 2025; and an activation function layer 2027 connected to the first fully connected layer 2026. A second network layer 2012 is connected to the first input layer 2021, a third network layer 2013 is connected to the second input layer 2022, a fourth network layer 2014 is connected to the third input layer 2023, and a fifth network layer 2015 is connected to the recognition network 203.
[0039] In some embodiments, step S102 above specifically includes the following steps:
[0040] The image of the face to be identified is input into the first network layer, and the first feature map is output.
[0041] The first feature map is input into the second network layer, and the second feature map is output.
[0042] The second feature map is input into the third network layer, and the third feature map is output.
[0043] The third feature map is input into the fourth network layer, and the fourth feature map is output.
[0044] The fourth feature map is input into the fifth network layer, and a face feature map is output.
[0045] Combination Figure 2Suppose the face image to be identified is image A of person A wearing a mask. Image A is input into the first network layer 2011 of the convolutional neural network 201 in the occluded face recognition model. After convolution, regularization, activation function and max pooling calculations, the first feature map A1 is output. Then, the first feature map A1 is input into the second network layer 2012 of the convolutional neural network 201. After convolution operation in the second network layer 2012, the second feature map A2 is output. Then, the second feature map A2 is input into the convolutional neural network 2012. In the third network layer 2013 of the convolutional neural network 201, after convolution operation, the third feature map A3 is output. Then, the third feature map A3 is input into the fourth network layer 2014 of the convolutional neural network 201. After convolution operation, the fourth feature map A4 is output. Finally, the fourth feature map A4 is input into the fifth network layer 2015 of the convolutional neural network 201. After convolution operation, the face feature map A5 is output.
[0046] In some embodiments, step S103 above specifically includes the following steps:
[0047] The face feature map and the fourth feature map are input into the third input layer, and the first fused feature map is output.
[0048] The first fused feature map and the third feature map are input into the second input layer, and the second fused feature map is output.
[0049] The second fused feature map and the second feature map are input into the first input layer, and the third fused feature map is output.
[0050] The first fused feature map, the second fused feature map, and the third fused feature map are input into the feature fusion layer, and the total fused feature map is output.
[0051] The total fused feature map is sequentially input into the global average pooling layer, the first fully connected layer, and the activation function layer to obtain the attention weight matrix of the face feature map.
[0052] In one embodiment, the face feature map and the fourth feature map are input into the third input layer, and a first fused feature map is output, specifically including:
[0053] The facial feature map is upsampled to obtain an upsampled feature map;
[0054] The upsampled feature map and the fourth feature map are input into the third input layer to add the upsampled feature map and the fourth feature map to obtain the first fused feature map.
[0055] Based on the above example, the face feature map A5 output by the fifth network layer 2015 is upsampled by 2 times to obtain the upsampled feature map P1. Then, the upsampled feature map P1 and the fourth feature map A4 output by the fourth network layer 2014 are input into the third input layer 2023 to add the upsampled feature map P1 and the fourth feature map A4 to obtain the first fused feature map Q1.
[0056] Similarly, the first fusion feature map Q1 obtained above is upsampled by 2 times (equivalent to upsampling the face feature map A5 by 4 times) to obtain the upsampled feature map P2. Then, the upsampled feature map P2 and the third feature map A3 output by the third network layer 2013 are input into the second input layer 2022 to add the upsampled feature map P2 and the third feature map A3 to obtain the second fusion feature map Q2.
[0057] The second fusion feature map Q2 obtained above is upsampled by 2 times (equivalent to upsampling the face feature map A5 by 8 times) to obtain the upsampled feature map P3. Then, the upsampled feature map P3 and the second feature map A2 output by the second network layer 2012 are input into the first input layer 2021 to add the upsampled feature map P3 and the second feature map A2 to obtain the third fusion feature map Q3.
[0058] Subsequently, the first fusion feature map Q1 output by the third input layer 2023, the second fusion feature map Q2 output by the second input layer 2022, and the third fusion feature map Q3 output by the first input layer 2021 are all input into the feature fusion layer 2024 of the multi-scale attention network 202 to add and fuse the first fusion feature map Q1, the second fusion feature map Q2, and the third fusion feature map Q3 to obtain the total fusion feature map Q.
[0059] Next, the total fused feature map Q output by the feature fusion layer 2024 is sequentially input into the global average pooling layer 2025, the first fully connected layer 2026, and the activation function layer 2027 (sigmoid activation function layer) in the multi-scale attention network 202 for processing, outputting the attention weight matrix of the face feature map A5, that is, obtaining the attention value (a value from 0 to 1) for each channel of the face feature map A5. Assuming that the face feature map A5 has 512 channels, then the attention weight matrix is a 512-dimensional vector.
[0060] In general, in occluded scenarios, the occluded portion of the face causes a loss of facial feature information. Therefore, during face recognition, if one of the pre-stored face image and the face image to be recognized is occluded while the other is not (e.g., the pre-stored face image is unoccluded, but the face image to be recognized is occluded), significant discrepancies in information comparison will occur. Therefore, how to enable the machine to effectively learn the effective feature representation of occluded faces becomes crucial to the success of face recognition in occluded scenarios. In this embodiment, by sequentially upsampling the face feature map A5 output by the fifth network layer 2015 by 2, 4, and 8 times, a feature extraction pyramid of different scales is constructed. This allows the high-level semantic information extracted by the fifth network layer 2015 to be passed down layer by layer to the lower layers, ensuring that the lower layers simultaneously possess both high-level semantic information and detailed texture information. This results in richer and more comprehensive facial feature information, while simultaneously reducing the weight of facial features for occluded portions of the face. Then, the feature fusion layer 2024 fuses the local fusion feature information extracted from the first input layer 2021, the second input layer 2022, and the third input layer 2023 to obtain global fusion feature information, which can significantly improve the face feature recognition effect and efficiency in occluded scenes.
[0061] Figure 3 This is a schematic diagram of the network structure of a recognition network 203 in the occluded face recognition method provided in this embodiment.
[0062] like Figure 3 As shown, the recognition network 203 includes a feature map generation network layer 2031, a second fully connected layer 2032, and a classification layer 2033.
[0063] In some embodiments, the facial feature map and the attention weight matrix are input into the recognition network, and the facial recognition result of the face image to be recognized is output, specifically including the following steps:
[0064] The face feature map and the attention weight matrix are input into the feature map generation network layer to generate an attention feature map of the face feature map;
[0065] The attention feature map is sequentially input into the second fully connected layer and the classification layer, and the face recognition result of the face image to be recognized is output.
[0066] Based on the above example, the face feature map A5 output by the fifth network layer 2015 and the attention weight matrix output by the multi-scale attention network 202 are input into the feature map generation network layer 2031 of the recognition network 203 to multiply the face feature map A5 and the attention weight matrix to generate the attention feature map M of the face feature map A5.
[0067] Finally, the attention feature map M is sequentially input into the second fully connected layer 2032 and the classification layer 2033 (such as a softmax classification layer) in the recognition network 203 for processing, and the face recognition result of the face image to be recognized is output. Assuming there are 100 face categories, the output face recognition result of the face image to be recognized is the face recognition probability value corresponding to these 100 face categories.
[0068] In some embodiments, the above-described occluded face recognition method further includes the following steps:
[0069] Retrieve the category tags of face images from the face database;
[0070] Calculate the face recognition loss value based on the face recognition result and the classification label.
[0071] In practical applications, facial images are typically pre-collected and stored in a facial database. Each facial image is then labeled manually or by machine, with the labeling including the face ID and coordinates of key facial feature points. The classification label here primarily refers to the face ID; one facial image corresponds to one classification label.
[0072] After acquiring the image of the face to be identified, the image is input into... Figure 2 The occluded face recognition model shown is used for identification, and the output is the probability value that the face image to be identified may correspond to each face image pre-stored in the face database. Assuming there are 100 face images in the face database, the face recognition result output by the occluded face recognition model provided in this disclosure includes 100 probability values, that is, the similarity between the face image to be identified and the 100 face images in the face database.
[0073] Based on the face recognition results output by the occluded face recognition model and the classification labels of face images in the face database, a face recognition loss value is calculated. This loss value is then used to guide the network optimization training of the occluded face recognition model, which can further improve the face recognition effect and efficiency of the model in occluded scenarios.
[0074] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0075] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.
[0076] Figure 4 This is a schematic diagram of the structure of a face recognition device with occlusion provided in an embodiment of this disclosure. Figure 4 As shown, the occluded face recognition device includes:
[0077] The acquisition module 401 is configured to acquire a face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image;
[0078] Extraction module 402 is configured to input the face image to be identified into a convolutional neural network for feature extraction to obtain a face feature map;
[0079] The processing module 403 is configured to input the face feature map into a multi-scale attention network for processing to obtain the attention weight matrix of the face feature map;
[0080] The recognition module 404 is configured to input the face feature map and the attention weight matrix into the recognition network and output the face recognition result of the face image to be recognized.
[0081] In some embodiments, the convolutional neural network includes a first network layer, a second network layer, a third network layer, a fourth network layer, and a fifth network layer connected in sequence; the multi-scale attention network includes a first input layer, a second input layer, and a third input layer; a feature fusion layer connected to the first input layer, the second input layer, and the third input layer; a global average pooling layer connected to the feature fusion layer; a first fully connected layer connected to the global average pooling layer; and an activation function layer connected to the first fully connected layer; the second network layer is connected to the first input layer; the third network layer is connected to the second input layer; the fourth network layer is connected to the third input layer; and the fifth network layer is connected to the recognition network.
[0082] In some embodiments, the extraction module 402 may specifically include:
[0083] The first extraction unit is configured to input the face image to be identified into the first network layer and output a first feature map.
[0084] The second extraction unit is configured to input the first feature map into the second network layer and output the second feature map;
[0085] The third extraction unit is configured to input the second feature map into the third network layer and output the third feature map;
[0086] The fourth extraction unit is configured to input the third feature map into the fourth network layer and output the fourth feature map;
[0087] The fifth extraction unit is configured to input the fourth feature map into the fifth network layer and output a face feature map.
[0088] In some embodiments, the processing module 403 described above may specifically include:
[0089] The first fusion unit is configured to input the face feature map and the fourth feature map into the third input layer and output the first fused feature map;
[0090] The second fusion unit is configured to input the first fused feature map and the third feature map into the second input layer and output the second fused feature map;
[0091] The third fusion unit is configured to input the second fused feature map and the second feature map into the first input layer and output the third fused feature map;
[0092] The fourth fusion unit is configured to input the first fusion feature map, the second fusion feature map, and the third fusion feature map into the feature fusion layer and output the total fusion feature map;
[0093] The attention calculation unit is configured to sequentially input the total fused feature map into the global average pooling layer, the first fully connected layer, and the activation function layer to obtain the attention weight matrix of the face feature map.
[0094] In some embodiments, the first fusion unit described above may be specifically configured as follows:
[0095] The facial feature map is upsampled to obtain an upsampled feature map;
[0096] The upsampled feature map and the fourth feature map are input into the third input layer to add the upsampled feature map and the fourth feature map to obtain the first fused feature map.
[0097] In some embodiments, the recognition network described above includes a feature map generation network layer, a second fully connected layer, and a classification layer. The recognition module 404 described above may specifically include:
[0098] The generation unit is configured to input the face feature map and the attention weight matrix into the feature map generation network layer to generate an attention feature map of the face feature map;
[0099] The classification unit is configured to sequentially input the attention feature map into the second fully connected layer and the classification layer, and output the face recognition result of the face image to be recognized.
[0100] In some embodiments, the above-described occluded face recognition device may further include:
[0101] The tag acquisition module is configured to retrieve the category tags of face images from the face database;
[0102] The loss calculation module is configured to calculate the face recognition loss value based on the face recognition result and the classification label.
[0103] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.
[0104] Figure 5 This is a schematic diagram of the electronic device 5 provided in an embodiment of this disclosure. Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the various method embodiments described above. Alternatively, when the processor 501 executes the computer program 503, it implements the functions of each module / unit in the various device embodiments described above.
[0105] Electronic device 5 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 5 may include, but is not limited to, processor 501 and memory 502. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown, or different components.
[0106] The processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0107] The memory 502 can be an internal storage unit of the electronic device 5, such as a hard disk or RAM of the electronic device 5. The memory 502 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 5. The memory 502 can also include both internal and external storage units of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device.
[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0109] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0110] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.
Claims
1. A method for facial recognition under occlusion, characterized in that, include: Obtain a face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image; The face image to be identified is input into a convolutional neural network for feature extraction to obtain a face feature map; The face feature map is input into a multi-scale attention network for processing to obtain the attention weight matrix of the face feature map; The face feature map and the attention weight matrix are input into the recognition network, and the face recognition result of the face image to be recognized is output. The convolutional neural network includes a first network layer, a second network layer, a third network layer, a fourth network layer, and a fifth network layer connected in sequence; The multi-scale attention network includes a first input layer, a second input layer, and a third input layer; a feature fusion layer connected to the first input layer, the second input layer, and the third input layer; a global average pooling layer connected to the feature fusion layer; a first fully connected layer connected to the global average pooling layer; and an activation function layer connected to the first fully connected layer. The second network layer is connected to the first input layer, the third network layer is connected to the second input layer, the fourth network layer is connected to the third input layer, and the fifth network layer is connected to the recognition network; The face image to be identified is input into a convolutional neural network for feature extraction to obtain a face feature map, including: The image of the face to be identified is input into the first network layer, and the first feature map is output. The first feature map is input into the second network layer, and the second feature map is output. The second feature map is input into the third network layer, and the third feature map is output. The third feature map is input into the fourth network layer, and the fourth feature map is output. The fourth feature map is input into the fifth network layer, and a face feature map is output. The facial feature map is input into a multi-scale attention network for processing to obtain the attention weight matrix of the facial feature map, including: The face feature map and the fourth feature map are input into the third input layer, and the first fused feature map is output. The first fused feature map and the third feature map are input into the second input layer, and the second fused feature map is output. The second fused feature map and the second feature map are input into the first input layer, and the third fused feature map is output. The first fused feature map, the second fused feature map, and the third fused feature map are input into the feature fusion layer, and the total fused feature map is output. The total fused feature map is sequentially input into the global average pooling layer, the first fully connected layer, and the activation function layer to obtain the attention weight matrix of the face feature map.
2. The method according to claim 1, characterized in that, The face feature map and the fourth feature map are input into the third input layer, and the first fused feature map is output, including: The facial feature map is upsampled to obtain an upsampled feature map; The upsampled feature map and the fourth feature map are input into the third input layer to add the upsampled feature map and the fourth feature map to obtain the first fused feature map.
3. The method according to claim 1, characterized in that, The recognition network includes a feature map generation network layer, a second fully connected layer, and a classification layer; The facial feature map and the attention weight matrix are input into the recognition network, and the facial recognition result of the face image to be recognized is output, including: The face feature map and the attention weight matrix are input into the feature map generation network layer to generate an attention feature map of the face feature map; The attention feature map is sequentially input into the second fully connected layer and the classification layer, and the face recognition result of the face image to be recognized is output.
4. The method according to claim 1, characterized in that, The method further includes: Retrieve the category tags of face images from the face database; Calculate the face recognition loss value based on the face recognition result and the classification label.
5. A face recognition device with occlusion, characterized in that, The apparatus is used to implement the method as described in any one of claims 1 to 4, the apparatus comprising: The acquisition module is configured to acquire a face image to be identified, wherein the face image to be identified is an occluded face image or an unoccluded face image; The extraction module is configured to input the face image to be identified into a convolutional neural network for feature extraction to obtain a face feature map; The processing module is configured to input the face feature map into a multi-scale attention network for processing to obtain the attention weight matrix of the face feature map; The recognition module is configured to input the face feature map and the attention weight matrix into the recognition network and output the face recognition result of the face image to be recognized.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.
7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.