Face recognition method and apparatus

Abstract

The present disclosure provides a face recognition method and apparatus, which includes the steps of: obtaining a first feature map of a recognition target face image; performing depthwise convolution on the first feature map to obtain a second feature map; performing attention flow processing on the second feature map to obtain a third feature map; and sequentially performing channel addition convolution, attention flow processing, channel reduction convolution, and attention flow processing on the third feature map to obtain a target feature map corresponding to the first feature map.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and particularly to a face recognition method and apparatus.

Background Art

[0002] In the actual application process, face recognition technology is often deployed on the cloud and edge terminals. Due to the limitations of the computing power and storage resources of edge terminals such as embedded terminals, the face recognition model of edge terminals must meet the requirements of high precision, while also meeting the requirements of a small model size, low calculation complexity, and fast inference speed.

[0003] In related technologies, common lightweight networks that can implement face recognition tasks include SqueezeNet, MobileNet, ShuffleNet, etc. Due to the particularity of the face structure, the accuracy of these models in face recognition tasks is insufficient. MobileFaceNet, a mobile-terminal lightweight network designed specifically for face recognition tasks, is based on MobileNet, adopts a smaller expansion rate, and replaces the global average pooling layer with a global depthwise convolutional layer. However, the main construction module of MobileFaceNet still adopts a common residual bottleneck module, and the calculations of each module are the same, so there is still a problem of insufficient accuracy.

Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a face recognition method, apparatus, electronic device, and computer-readable storage medium to solve the problem of low accuracy of face recognition models in the prior art.

[0005] A first embodiment of the present disclosure provides a face recognition method that includes the steps of: obtaining a first feature map of a face image to be recognized; performing a depth convolution on the first feature map to obtain a second feature map; performing an attention flow process on the second feature map to obtain a third feature map; and sequentially performing a channel addition convolution, an attention flow process, a channel reduction convolution, and an attention flow process on the third feature map to obtain a corresponding target feature map of the first feature map.

[0006] A second embodiment of the present disclosure provides a face recognition device comprising: an acquisition module for acquiring a first feature map of a face image to be recognized; a convolution module for performing depth convolution on the first feature map and acquiring a second feature map; an attention flow module for performing attention flow processing on the second feature map and acquiring a third feature map; and a mixed processing module for sequentially performing channel addition convolution, attention flow processing, channel reduction convolution and attention flow processing on the third feature map and acquiring a corresponding target feature map of the first feature map.

[0007] A third embodiment of the present disclosure provides an electronic device comprising memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the steps of the above method are realized when the processor executes the computer program.

[0008] A fourth embodiment of the present disclosure provides a computer-readable storage medium in which a computer program is stored, and which enables the steps of the above method when the computer program is executed by a processor.

[0009] The embodiments of this disclosure offer beneficial effects compared to the prior art, including the ability to perform feature mapping for face recognition by combining convolution processing and attention flow processing, thereby promoting the flow of attention in multiple directional dimensions, and ultimately improving the recognition accuracy of the face recognition model by giving the resulting feature map high discriminative power in each directional dimension.

[0010] Specifically, the embodiments of this disclosure provide a lightweight attention flow module with a very low tensor dimension, resulting in very small computational complexity for convolution of low-dimensional tensors and enabling fast overall operation. If the entire network performs feature extraction in a low-dimensional space, there is a very high possibility of information incompleteness and insufficient feature robustness. In the embodiments of this disclosure, the number of channels is expanded by setting an expansion coefficient during the intermediate convolution process, thereby improving the feature extraction capability of the entire module and achieving an excellent balance between computational complexity and feature representation capability.

[0011] In the embodiments of this disclosure, the entire attention flow module combines operations such as different types of convolution, channel expansion and compression, and attention flow techniques to flow and transform the attention flow of interest for the face recognition task across space and channels, making feature fusion more efficient and ensuring that the feature map is effectively focused on the region of interest in face recognition. Furthermore, this attention flow module has the advantages of having fewer parameters, lower computational load, and higher speed. [Brief explanation of the drawing]

[0012] To more clearly explain the technical solutions in the embodiments of this disclosure, the following is a brief introduction of the drawings necessary for describing the embodiments or the prior art. Clearly, the drawings described below represent only a few embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these, assuming no creative work is required.

[0013] [Figure 1]This is a schematic diagram illustrating the application scenario of the embodiments of this disclosure. [Figure 2] This is a flowchart of the face recognition method provided in the embodiments of this disclosure. [Figure 3] This is a flowchart of the attention flow processing provided in the embodiments of this disclosure. [Figure 4] This is a flowchart of another face recognition method provided in the embodiments of this disclosure. [Figure 5] This is a schematic diagram of the facial recognition device provided in the embodiments of this disclosure. [Figure 6] This is a schematic diagram of the electronic device provided in the embodiments of this disclosure. [Modes for carrying out the invention]

[0014] In the following description, specific details such as particular system structures and technologies are provided for illustrative purposes, rather than for limitation, to ensure a thorough understanding of the embodiments of this disclosure. However, those skilled in the art should understand that the disclosure can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, apparatus, circuits, and methods are omitted to avoid unnecessary details interfering with the description of this disclosure.

[0015] The facial recognition method and apparatus according to the embodiments of this disclosure will be described in detail below with reference to the drawings.

[0016] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure. The application scenario may include terminal devices 101, 102 and 103, a server 104 and a network 105.

[0017] Terminal devices 101, 102, and 103 may be hardware or software. If terminal devices 101, 102, and 103 are hardware, they may be various electronic devices having a display and communicating with server 104, including but not limited to smartphones, robots, laptops, desktop computers, etc. (for example, 102 may be a robot). If terminal devices 101, 102, and 103 are software, they may be installed on the above-mentioned electronic devices. Terminal devices 101, 102, and 103 may be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of this disclosure are not limited thereto. Furthermore, various applications such as data processing applications, instant messaging tools, social platform software, search applications, and shopping applications may be installed on terminal devices 101, 102, and 103.

[0018] Server 104 may be a server that provides various services, such as a backend server that receives requests sent by terminal devices that establish a communication connection with it, and the backend server can perform processing such as receiving and analyzing the requests sent by terminal devices and generate processing results. Server 104 may be a single server, a server cluster consisting of several servers, or a cloud computing service center, and the embodiments of this disclosure are not limited thereto.

[0019] Note that the server 104 may be hardware or software. When the server 104 is hardware, it may be various electronic devices that provide various services to the terminal devices 101, 102, and 103. When the server 104 is software, it may be multiple software or software modules that provide various services to the terminal devices 101, 102, and 103, or it may be a single software or software module that provides various services to the terminal devices 101, 102, and 103. The embodiments of the present disclosure do not limit this.

[0020] The network 105 may be a wired network connected using coaxial cables, twisted pairs, and optical fibers, or it may be a wireless network that can realize interconnection of various communication devices without the need for wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc. The embodiments of the present disclosure do not limit this.

[0021] The target user can communicate and connect with the server 104 via the network 105 by the terminal devices 101, 102, and 103 so as to receive and transmit information. Note that the specific types, numbers, and combinations of the terminal devices 101, 102, and 103, the server 104, and the network 105 can be adjusted according to the actual needs of the application scenario. The embodiments of the present disclosure do not limit this.

[0022] In the related art, edge terminals such as embedded terminals have limited computing power and storage resources and can only support relatively small model sizes. On the other hand, in general lightweight large-scale models for face recognition, the face recognition accuracy is not high.

[0023] To solve this technical problem, embodiments of the present disclosure provide a face recognition solution, in which a general model for lightweight face feature extraction, which is simple and effective, is designed to specifically design a face recognition model with real-time response for edge terminals and embedded devices so as to improve the accuracy of face recognition.

[0024] Specifically, the technical solution of the embodiments of the present disclosure proposes a general attention flow technology, which can effectively capture attention in space and channels respectively, improve the feature discrimination ability through a non-linear mapping that can be learned for each channel, and the whole technology can extract an effective feature combination method and promote the flow of attention in multiple direction dimensions.

[0025] FIG. 2 is a flowchart of the face recognition method provided by the embodiments of the present disclosure. The method provided by the embodiments of the present disclosure may be executed by any electronic device with computer processing capabilities, such as a terminal or a server. As shown in FIG. 2, the face recognition method includes the following steps.

[0026] Step S201: Obtain a first feature map of the face image to be recognized.

[0027] Specifically, the first feature map is a 4D tensor, and the dimensions of the tensor are (N, C, H, W), where N represents the number of batch images, C represents the number of channels, H represents the height, and W represents the width. The first feature map is obtained by performing feature extraction on the face image to be recognized.

[0028] Step S202: Perform depthwise convolution processing on the first feature map to obtain a second feature map.

[0029] Specifically, depthwise convolution (DWConv) performs convolution operations within each independent channel. In general convolution, each convolution kernel is calculated once for each channel, whereas in depthwise convolution, each convolution kernel calculates only one channel.

[0030] Step S203 involves performing attention flow processing on the second feature map to obtain the third feature map.

[0031] Specifically, attention flow processing allows attention to flow between space and channels, enabling more effective feature fusion.

[0032] Step S204 involves sequentially performing channel addition convolution, attention flow processing, channel reduction convolution, and attention flow processing on the third feature map to obtain the corresponding target feature map for the first feature map.

[0033] Specifically, channel addition convolution and channel reduction convolution are two corresponding general convolution calculation processes: first, channel addition convolution is performed to increase the number of channels, and then channel reduction convolution is performed to return the number of channels to the original number.

[0034] According to the technical solutions of the embodiments of this disclosure, attention flow processing can extract effective feature combination schemes and facilitate the flow of attention in multiple directional dimensions. By designing and combining attention flow processing techniques with different types of convolutions, it is possible to simultaneously satisfy the requirements of face recognition tasks and the requirements for lightweight embedded devices, and to achieve higher recognition accuracy with fewer parameters compared to conventional techniques.

[0035] As shown in Figure 3, the attention flow processing in steps S203 and S204 includes the following steps.

[0036] Step S301 involves performing a flattening process on the first and second dimensions input to the feature map to obtain a first intermediate feature map.

[0037] Specifically, the first dimension may be height, and the second dimension may be width. Assuming the input feature map is f1, flattening the two dimensions of height and width of f1 transforms the dimension (N,C,H,W) into (N,C,R), where R = H * W.

[0038] Step S302 is to obtain a second intermediate feature map based on the first intermediate feature map and the first learnable parameter matrix.

[0039] In the technical solutions of the embodiments of this disclosure, a first product of a first intermediate feature map and the function value of its logistic regression function softmax can be obtained, and a second intermediate feature map can be obtained from the mean of the first product. Specifically, the first intermediate feature map is right-multiplied by a first learnable parameter matrix to obtain a tensor, and the Hadamard product of the tensor and the softmax function value of the tensor is calculated to obtain a matrix, and the matrix is ​​averaged in a certain dimension to obtain a second intermediate feature map. The first learnable parameter matrix can learn attention flow information in spatial dimensions.

[0040] Step S303 is performed to obtain a spatial attention feature map based on the product of the second intermediate feature map and the input feature map.

[0041] Specifically, a spatial attention feature map is a feature map that fuses spatial attention features.

[0042] Step S304 involves obtaining a channel attention feature map based on a second learnable parameter matrix, a third learnable parameter matrix, and a spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix.

[0043] Specifically, the spatial attention feature map may be right-multiplied by a second learnable parameter matrix to obtain the second product, the second product may be sparsified, and then right-multiplied by a third learnable parameter matrix to obtain the channel attention feature map. The second and third learnable parameter matrices can learn attention flow information in the channel dimension, capture feature relationships between different channels, learn the weight of each channel, and make the features more discriminative with respect to the information of each channel.

[0044] Step S305 involves obtaining an attention flow feature map based on the spatial attention feature map and the channel attention feature map.

[0045] Specifically, when obtaining an attention flow feature map based on a spatial attention feature map and a channel attention feature map, a nonlinear mapping process may be performed on the spatial attention feature map to obtain a third intermediate feature map, a fourth intermediate feature map may be obtained based on the product of the third intermediate feature map and the channel attention feature map, and a nonlinear mapping process may be performed on the fourth intermediate feature map to obtain an attention flow feature map. The attention flow feature map obtained from the spatial attention feature map and the channel attention feature map allows for learning of attention flow information in the spatial dimension and the channel dimension, thereby enhancing the accuracy of attention flow in the spatial dimension and the channel dimension.

[0046] Steps S301 to S305 will be explained in detail below.

[0047] In step S301, assuming the input feature map is f1 and its dimensions are (N, C, H, W), flattening the two dimensions H and W of f1 transforms the dimensions to (N, C, R), and a second intermediate feature map can be obtained, where R = H * W.

[0048] Features are learned in the dimension H*W, and attention is made to flow in the spatial dimension. In the embodiment of this disclosure, a first learnable parameter matrix Q1 is introduced, with dimensions (R,r)(r <R)である。

[0049] JPEG0007886582000001.jpg108170

[0050] In the embodiments of this disclosure, the introduction of a first learnable parameter matrix Q1 is for obtaining r types of spatial linear transformation results, and all combinations of representative features in space can be extracted. In the extracted face feature map, each spatial pixel has the same receptive field, but since these receptive fields are mapped to different regions of the original image, their contribution to the final recognition task is also different, and therefore different weights should be assigned to different pixels. The first learnable parameter matrix Q1 can be used to learn attention to features in H*W dimensions, allowing the attention to flow in the spatial dimension and producing a fusion effect of various feature combinations.

[0051] JPEG0007886582000002.jpg35170

[0052] JPEG0007886582000003.jpg11170

[0053] In step S304, a spatial attention feature map with dimensions (N,C,H,W) is introduced into the second learnable parameter matrix Q2 and the third learnable parameter matrix Q3 and processed, and the channel attention feature map f1 is created. c Obtain it.

[0054] Specifically, the dimension of the second learnable parameter matrix Q2 is (C, C / / p), and the dimension of the third learnable parameter matrix Q3 is (C / / p, C), where C is a natural number. As can be seen from this, the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix. f1 c Multiplying Q2 by (right) gives the dimension (N, C / / p), and after ReLU sparsification, multiplying Q3 by (right) gives the channel attention feature map f1 c We can obtain a value whose dimension is (N,C).

[0055] The specific calculation process is shown in equation (3) below. JPEG0007886582000004.jpg18170

[0056] In step S305, by introducing the second learnable parameter matrix Q2 and the third learnable parameter matrix Q3 into the channel attention feature map output in step S304, attention flow information in the channel dimension can be learned. The design of this part focuses more on the feature relationships between channels, capturing the feature relationships between different channels to learn the weight of each channel, and making the features more discriminative with respect to the information of each channel. p represents the scaling factor, and by designing the parameter p, the computation amount can be reduced and the size of the model can be controlled.

[0057] JPEG0007886582000005.jpg59170

[0058] JPEG0007886582000006.jpg53170 needs to be obtained through learning.

[0059] In the process of processing data using a nonlinear mapping scheme, for negative inputs, the positive and negative responses of the convolutional kernel should be acceptable rather than simply mapping inputs with a direct Relu value of 0 or less to 0 and outputting that as the output. In other words, for faces, it is recognized that negative inputs need to be learned. Applying such a nonlinear mapping scheme allows for the learning of more complex relationships within the data. Next, it is beneficial to learn the mapping values ​​in the depth direction, i.e., to perform independent weight learning for each channel. This can be considered an attention learning scheme between different channels, improving the accuracy of the flow of attention between channels. Furthermore, with this channel-by-channel mapping scheme, as the depth increases, the nonlinear mapping gradually becomes more "nonlinear." That is, the model tends to retain information in shallow networks and enhance discriminative power in deep networks. In other words, typically, low-level feature maps have high resolution and weak semantic information but rich spatial information, while high-level feature maps have low resolution but strong semantic information.

[0060] JPEG0007886582000007.jpg29170

[0061] JPEG0007886582000008.jpg59170

[0062] This represents a feature map where both the direction and channel direction of JPEG0007886582000009.jpg24170 have sufficient flow, and the attention flow of interest spans the entire feature space.

[0063] JPEG0007886582000010.jpg90170

[0064] In the embodiments of this disclosure, one attention flow module can be formed from the attention flow technology as a basic composition module of a neural network. This module, through a precise convolutional module design based on the specificity of facial structures, can extract facial features with strong discriminative power with minimal computation, and can effectively concentrate attention on regions of the feature map that are advantageous for recognition tasks.

[0065] When applying the attention flow module in steps S201 to S204, the implementation process of steps S201 to S204 will be described in detail below.

[0066] JPEG0007886582000011.jpg74170

[0067] Here, the stride length is a configurable hyperparameter that varies depending on the network design. In the embodiments of this disclosure, based on the design philosophy of small modules, depth convolution can be used instead of conventional convolution to reduce the number of parameters, and calculations show that the number of parameters for depth convolution is 1 / C of that of conventional convolution. Note that the 3x3 convolution here may be replaced with a larger convolution kernel such as 5x5 or 7x7, but 3x3 convolution offers the best cost performance.

[0068] JPEG0007886582000012.jpg41170

[0069] In step S204, the convolution process for adding channels includes the steps of performing a convolution on the input feature map with the number of channels increased by N times, and then performing batch normalization on the convolution result, where N is a natural number, and performing a convolution on the input feature map with the number of channels reduced by 1 / N, and then performing batch normalization on the convolution result. Specifically, in step S204, the following steps may be executed in order.

[0070] JPEG0007886582000013.jpg53170

[0071] JPEG0007886582000014.jpg29170

[0072] JPEG0007886582000015.jpg48170

[0073] JPEG0007886582000016.jpg29170

[0074] In embodiments of this disclosure, a lightweight attention flow module is provided, which is precisely designed for facial recognition technology, and the techniques used herein, such as convolutional design, linear and nonlinear mapping, all adhere to the following two principles: firstly, to reduce network parameters, save computational resources, and increase computational speed; and secondly, to perform more effective feature fusion in spatial and channel dimensions, enhance feature representation capabilities, and extract more discriminative facial features.

[0075] In the embodiments of this disclosure, the basic number of channels in the attention flow module may be set to 64, resulting in a very low tensor dimension, a very small amount of computation required for convolution of low-dimensional tensors, and a fast overall operating speed. If the entire network performs feature extraction in a low-dimensional space, there is a very high possibility of information incompleteness and insufficient feature robustness. In the embodiments of this disclosure, the number of channels is expanded by setting an expansion coefficient during the intermediate convolution process, thereby improving the feature extraction capability of the entire module and achieving an excellent balance between computational load and feature representation capability.

[0076] In the embodiments of this disclosure, the entire attention flow module combines operations such as different types of convolution, channel expansion and compression, and attention flow techniques to flow and transform the attention flow of interest for the face recognition task across space and channels, making feature fusion more efficient and ensuring that the feature map is effectively focused on the region of interest in face recognition. Furthermore, this attention flow module has the advantages of having fewer parameters, lower computational load, and higher speed.

[0077] As shown in Figure 4, the facial recognition method provided in the embodiments of this disclosure includes the following steps:

[0078] Step S401 involves inputting the face image to be recognized into a convolutional layer and a normalization layer, where the convolutional kernel is 3x3, the number of channels is 64, and the step length is 1. In one specific embodiment, the resolution of the face image to be recognized is (1,3,112,112). The resolution of the feature map output in step S401 is (1,64,112,112).

[0079] Step S402 involves inputting the feature map acquired in the previous step into an attention flow module with a base channel count of 64, an expansion coefficient of 1, and a placement stride length of 2. The resolution of the feature map output in step S402 is (1,64,56,56).

[0080] Step S403 involves inputting the feature map acquired in the previous step into an attention flow module with a base channel count of 64, an expansion coefficient of 1, and a placementable stride length of 1. The resolution of the feature map output in step S403 is (1,64,56,56).

[0081] Step S404 involves inputting the feature map obtained in the previous step into an attention flow module with a base channel count of 64, an expansion coefficient of 2, and a placement stride of 2. The resolution of the feature map output in step S404 is (1,64,28,28).

[0082] Step S405 involves inputting the feature map acquired in the previous step into four attention flow modules, each with a base channel count of 64, an expansion coefficient of 2, and a placement stride length of 1. The resolution of the feature map output in step S405 is (1,64,28,28).

[0083] Step S406 involves inputting the feature map acquired in the previous step into an attention flow module with a base channel count of 128, an expansion coefficient of 2, and a placement stride of 2. The resolution of the feature map output in step S406 is (1,128,14,14).

[0084] Step S407 involves inputting the feature map obtained in the previous step into six attention flow modules, each with a base channel count of 128, an expansion coefficient of 2, and a placement stride length of 1. The resolution of the feature map output in step S407 is (1,128,14,14).

[0085] Step S408 involves inputting the feature map obtained in the previous step into an attention flow module with a base channel count of 128, an expansion coefficient of 2, and a placement stride of 2. The resolution of the feature map output in step S408 is (1,128,7,7).

[0086] Step S409 involves inputting the feature map obtained in the previous step into two attention flow modules, each with a base channel count of 128, an expansion coefficient of 2, and a placementable stride length of 1. The resolution of the feature map output in step S409 is (1,128,7,7).

[0087] Step S410 is performed by inputting the feature map obtained in the previous step into a convolutional layer and a normalization layer, both of which have a 1x1 convolutional kernel and 512 channels. The resolution of the feature map output in step S410 is (1,512,7,7).

[0088] Step S411 involves inputting the feature map obtained in the previous step into a convolutional layer and a normalization layer, both of which have a 7x7 convolutional kernel and 512 channels. The resolution of the feature map output in step S411 is (1,512,1,1).

[0089] Step S412 is performed by flattening the feature map obtained in the previous step, then calculating a fully joined matrix of (512,512), and obtaining a 512-dimensional vector as the target feature map.

[0090] In the face recognition method shown in Figure 4, steps S402 and S403 may be considered as one step, steps S404 and S405 as one step, steps S406 and S407 as one step, and steps S408 and S409 as one step. The number of attention flow modules included in each step is (2, 5, 7, 3), respectively. However, this combination of attention flow modules is merely illustrative, and other combinations of attention flow modules can also achieve the technical effects of the technical solutions of the embodiments of this disclosure.

[0091] The technical solutions of the embodiments of this disclosure provide a general-purpose attention flow technique that can effectively capture attention in space and on channels, respectively, improve feature discrimination through learnable nonlinear mapping for each channel, and the technique as a whole can extract an effective feature combination scheme and facilitate the flow of attention in multiple directional dimensions.

[0092] According to the face recognition method of the embodiment of this disclosure, a combination of convolution processing and attention flow processing is used to perform feature map processing for face recognition, promote the flow of attention in multiple directional dimensions, and ultimately give the obtained feature map high discriminative power in each directional dimension, thereby improving the recognition accuracy of the face recognition model.

[0093] The following are apparatus embodiments of the present disclosure for carrying out the method embodiments of the present disclosure. The facial recognition apparatus described below can be cross-referenced with the facial recognition method described above. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0094] Figure 5 is a schematic diagram of a face recognition device provided in an embodiment of the present disclosure. As shown in Figure 5, the face recognition device comprises the following parts.

[0095] The acquisition module 501 may be used to acquire a first feature map of a face image to be recognized.

[0096] Specifically, the first feature map is a 4-dimensional tensor with dimensions (N, C, H, W), where N is the number of batch images, C is the number of channels, H is the height, and W is the width. The first feature map is obtained by performing feature extraction on the face image to be recognized.

[0097] A convolution module 502 may be used to perform depth convolution on a first feature map and obtain a second feature map.

[0098] Specifically, in depth convolution, the convolution operation is performed within each independent channel. In general convolution, each convolution kernel is calculated once for each channel, whereas in depth convolution, each convolution kernel calculates only one channel.

[0099] The attention flow module 503 may be used to perform attention flow processing on a second feature map and obtain a third feature map.

[0100] Specifically, attention flow processing allows attention to flow between space and channels, enabling more effective feature fusion.

[0101] A mixed processing module may be used to obtain a target feature map corresponding to the first feature map by sequentially performing channel addition convolution, attention flow processing, channel reduction convolution, and attention flow processing on a third feature map.

[0102] Specifically, channel addition convolution and channel reduction convolution are two corresponding general convolution calculation processes: first, channel addition convolution is performed to increase the number of channels, and then channel reduction convolution is performed to return the number of channels to the original number.

[0103] According to the technical solutions of the embodiments of this disclosure, attention flow processing can extract effective feature combination schemes and facilitate the flow of attention in multiple directional dimensions. By designing and combining attention flow processing techniques with different types of convolutions, it is possible to simultaneously satisfy the requirements of face recognition tasks and the requirements for lightweight embedded devices, and to achieve higher recognition accuracy with fewer parameters compared to conventional techniques.

[0104] In embodiments of this disclosure, the attention flow module 503 may be used for the following steps: obtaining a first intermediate feature map by performing a flattening process on the first and second dimensions input to the feature map; obtaining a second intermediate feature map based on the first intermediate feature map and a first learnable parameter matrix; obtaining a spatial attention feature map based on the product of the second intermediate feature map and the input feature map; obtaining a channel attention feature map based on a second learnable parameter matrix, a third learnable parameter matrix and a spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix; and obtaining an attention flow feature map based on the spatial attention feature map and the channel attention feature map.

[0105] In the technical solutions of the embodiments of this disclosure, the first product of the first intermediate feature map and the function value of the logistic regression function softmax can be obtained, and a second intermediate feature map can be obtained from the mean of the first product. Specifically, the first intermediate feature map is right multiplied by the first learnable parameter matrix to obtain a tensor, and the Hadamard product of the tensor and the softmax function value of the tensor is calculated to obtain a matrix, and the matrix is ​​averaged over a certain dimension to obtain a second intermediate feature map.

[0106] Specifically, the spatial attention feature map is a feature map that fuses spatial attention. The first learnable parameter matrix can learn attention flow information in the spatial dimension. The second and third learnable parameter matrices can learn attention flow information in the channel dimension, capturing feature relationships between different channels and learning the weight of each channel so that the features have greater discriminative power over the information of each channel. The attention flow feature map obtained from the spatial attention feature map and the channel attention feature map can learn attention flow information in the spatial dimension and the channel dimension, thereby enhancing the accuracy of attention flow in the spatial dimension and the channel dimension.

[0107] In the embodiments of this disclosure, the attention flow module 503 may be used to perform a nonlinear mapping process on a spatial attention feature map to obtain a third intermediate feature map, obtain a fourth intermediate feature map based on the product of the third intermediate feature map and the channel attention feature map, perform a nonlinear mapping process on the fourth intermediate feature map to obtain an attention flow feature map.

[0108] In the embodiments of this disclosure, applying such a nonlinear mapping method allows for the learning of more complex relationships within the data. Learning the mapping values ​​in the depth direction, i.e., independent weight learning for each channel, is beneficial, as this can be considered an attention learning method between different channels, improving the accuracy of the flow of attention between channels. Furthermore, with respect to this channel-by-channel mapping method, as the depth increases, the nonlinear mapping gradually becomes more "nonlinear," meaning the model tends to retain information in shallow networks and enhance discriminative power in deep networks. In other words, typically, low-level feature maps have high resolution and weak semantic information but are rich in spatial information, while high-level feature maps have low resolution but strong semantic information.

[0109] In the embodiments of this disclosure, the attention flow module 503 may be used to obtain the first product of the first intermediate feature map and the function value of the logistic regression function softmax, and to obtain the second intermediate feature map from the mean of the first product.

[0110] In the embodiments of this disclosure, the attention flow module 503 may be used to obtain a channel attention feature map by right multiplying the spatial attention feature map by a second learnable parameter matrix to obtain the second product, sparsifying the second product, right multiplying it by a third learnable parameter matrix.

[0111] In the embodiments of this disclosure, the introduction of the first learnable parameter matrix Q1 is for obtaining r types of spatial linear transformation results, and all combinations of representative features in space can be extracted. In the extracted face feature map, each spatial pixel has the same receptive field, but since these receptive fields are mapped to different regions of the original image, their contribution to the final recognition task is also different, and therefore different weights should be assigned to different pixels. The first learnable parameter matrix Q1 can be used to learn the attention of features in the H*W dimension, allowing the attention to flow in the spatial dimension and achieving a fusion effect of various feature combinations. The introduction of the second learnable parameter matrix Q2 and the third learnable parameter matrix Q3 allows for learning attention flow information in the channel dimension. The design of this part focuses more on the feature relationships between channels, capturing the feature relationships between different channels to learn the weight of each channel, and making the features more discriminative with respect to the information of each channel.

[0112] In embodiments of the present disclosure, the mixed processing module 504 may be used for the following: the channel addition convolution process includes the steps of performing a convolution on an input feature map with the number of channels increased by N times, and performing a batch normalization process on the convolution result, where N is a natural number; and the steps include performing a convolution on an input feature map with the number of channels reduced by 1 / N, and performing a batch normalization process on the convolution result.

[0113] In the embodiments of this disclosure, the convolution module 502 may be used to perform depth convolution on a first feature map, perform batch normalization on the depth convolution result, and obtain a second feature map.

[0114] In embodiments of this disclosure, a lightweight attention flow module is provided, which is precisely designed for facial recognition technology, and the techniques used herein, such as convolutional design, linear and nonlinear mapping, all adhere to the following two principles: firstly, to reduce network parameters, save computational resources, and increase computational speed; and secondly, to perform more effective feature fusion in spatial and channel dimensions, enhance feature representation capabilities, and extract more discriminative facial features.

[0115] In the embodiments of this disclosure, the basic number of channels in the attention flow module may be set to 64, resulting in a very low tensor dimension, a very small amount of computation required for convolution of low-dimensional tensors, and a fast overall operating speed. If the entire network performs feature extraction in a low-dimensional space, there is a very high possibility of information incompleteness and insufficient feature robustness. In the embodiments of this disclosure, the number of channels is expanded by setting an expansion coefficient during the intermediate convolution process, thereby improving the feature extraction capability of the entire module and achieving an excellent balance between computational load and feature representation capability.

[0116] In the embodiments of this disclosure, the entire attention flow module combines operations such as different types of convolution, channel expansion and compression, and attention flow techniques to flow and transform the attention flow of interest for the face recognition task across space and channels, making feature fusion more efficient and ensuring that the feature map is effectively focused on the region of interest in face recognition. Furthermore, this attention flow module has the advantages of having fewer parameters, lower computational load, and higher speed.

[0117] The technical solutions of the embodiments of this disclosure provide a general-purpose attention flow technique that can effectively capture attention in space and on channels, respectively, improve feature discrimination through learnable nonlinear mapping for each channel, and the technique as a whole can extract an effective feature combination scheme and facilitate the flow of attention in multiple directional dimensions.

[0118] Each functional module of the facial recognition device in the exemplary embodiments of the Disclosure corresponds to the steps of the exemplary embodiments of the facial recognition method described above, and for details not disclosed in the device embodiments of the Disclosure, refer to the embodiments of the facial recognition method described above.

[0119] According to the face recognition device of the embodiment of this disclosure, a combination of convolution processing and attention flow processing is used to perform feature map processing for face recognition, promote the flow of attention in multiple directional dimensions, and ultimately give the obtained feature map high discriminative power in each directional dimension, thereby improving the recognition accuracy of the face recognition model.

[0120] Figure 6 is a schematic diagram of an electronic device 6 provided in an embodiment of the present disclosure. As shown in Figure 6, the electronic device 6 of the embodiment comprises a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable by the processor 601. When the processor 601 executes the computer program 603, it realizes the steps in each of the above method embodiments. Alternatively, when the processor 601 executes the computer program 603, it realizes the functions of each module in each of the above device embodiments.

[0121] The electronic device 6 may be a desktop computer, a notebook computer, a palmtop computer, or a cloud server. The electronic device 6 may include, but is not limited to, a processor 601 and memory 602. A person skilled in the art will understand that Figure 6 is merely an example of the electronic device 6 and does not limit it, and that it may include more or fewer components than those shown, or different components.

[0122] The processor 601 may be a Central Processing Unit (CPU), or it may be another general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0123] Memory 602 may be the internal storage unit of the electronic device 6, for example, the hard disk or RAM of the electronic device 6. Memory 602 may also be an external storage device of the electronic device 6, for example, a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., installed in the electronic device 6. Memory 602 may include not only the internal storage unit of the electronic device 6 but also external storage devices. Memory 602 is for storing computer programs and other programs and data required by the electronic device.

[0124] Those skilled in the art will understand that, for the convenience and simplicity of explanation, only the division of each functional unit and module described above has been given as an example, but in actual application, the above functions can be completed by assigning them to different functional units and modules as needed, that is, by dividing the internal structure of the device into different functional units or modules, all or some of the functions described above can be completed. Each functional unit and module in the embodiment may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit, and the integrated unit may be implemented in hardware form or in the form of a software functional unit.

[0125] The integrated module may be implemented in the form of a software function unit and may be stored on a computer-readable storage medium when sold or used as an independent product. With this understanding, the disclosure can implement all or part of the processes in the methods of the above embodiments by directing the relevant hardware with a computer program, the computer program may be stored on a computer-readable storage medium, and the steps of each of the above embodiments can be implemented when the computer program is executed by a processor. The computer program may include computer program code, which may be in source code format, object code format, executable file, or some intermediate format. The computer-readable storage medium may include any entity or device capable of carrying the computer program code, recording media, U disks, removable hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media. Furthermore, the contents contained in a computer-readable storage medium may be increased or decreased as appropriate in accordance with the requirements of legislation and patent practice within the jurisdiction. For example, in a certain jurisdiction, based on legislation and patent practice, the computer-readable storage medium may not contain electrical carrier signals or telecommunications signals.

[0126] The embodiments described above are merely for illustrating, and not limiting, the technical solutions of the Disclosure. While the Disclosure has been described in detail with reference to the embodiments described above, those skilled in the art should understand that they may still modify the technical solutions described in the embodiments described above, or substitute some of their technical features with equivalent ones, but that such modifications or substitutions should not cause the substance of the corresponding technical solution to deviate from the idea and scope of the technical solutions of the embodiments of the Disclosure, and that they should all fall within the scope of protection of the Disclosure.

Claims

1. A facial recognition method, The steps include obtaining the first feature map of the face image to be recognized, The steps include performing depth-direction convolution on the first feature map to obtain a second feature map, The steps include performing attention flow processing on the aforementioned second feature map to obtain a third feature map, The process includes the steps of sequentially performing a channel addition convolution process, an attention flow process, a channel reduction convolution process, and an attention flow process on the third feature map to obtain the corresponding target feature map of the first feature map. A computer-based facial recognition method characterized by the following features.

2. The aforementioned attention flow processing is, The steps include: performing a flattening process on the first and second dimensions input to the feature map to obtain a first intermediate feature map; A step of obtaining a second intermediate feature map based on the first intermediate feature map and the first learnable parameter matrix, A step of obtaining a spatial attention feature map based on the product of the aforementioned second intermediate feature map and the input feature map, A step of obtaining a channel attention feature map based on a second learnable parameter matrix, a third learnable parameter matrix, and the spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix, The step of obtaining an attention flow feature map based on the spatial attention feature map and the channel attention feature map includes: The method according to feature 1.

3. The step of obtaining an attention flow feature map based on the spatial attention feature map and the channel attention feature map is: The steps include performing a nonlinear mapping process on the spatial attention feature map to obtain a third intermediate feature map, A step of obtaining a fourth intermediate feature map based on the product of the third intermediate feature map and the channel attention feature map, The step includes performing the nonlinear mapping process on the fourth intermediate feature map and obtaining the attention flow feature map. The method according to feature 2.

4. The step of obtaining a second intermediate feature map based on the first intermediate feature map and the first learnable parameter matrix is: The steps include obtaining the first product of the first intermediate feature map and its logistic regression function value, The step of obtaining the second intermediate feature map based on the mean value of the first product includes The method according to feature 2.

5. The step of obtaining a channel attention feature map based on a second learnable parameter matrix, a third learnable parameter matrix, and the spatial attention feature map is: The steps include: right multiplying the spatial attention feature map by the second learnable parameter matrix to obtain the second product; The process includes the steps of performing sparsification on the second product, right multiplying it by the third learnable parameter matrix, and obtaining the channel attention feature map. The method according to feature 2.

6. The aforementioned channel addition convolution process includes the steps of performing a convolution on the input feature map with the number of channels increased by N times, and then performing batch normalization on the convolution result, where N is a natural number. The channel reduction convolution process includes the steps of performing a convolution on the input feature map to reduce the number of channels to 1 / N, and then performing a batch normalization process on the convolution result. The method according to feature 1.

7. The step of performing depth convolution on the first feature map to obtain a second feature map is: The process includes the steps of performing depth convolution on the first feature map, performing batch normalization on the depth convolution result, and obtaining the second feature map. The method according to feature 6.

8. It is a facial recognition device, An acquisition module for obtaining the first feature map of the face image to be recognized, A convolution module for performing depth-direction convolution on the first feature map to obtain a second feature map, An attention flow module for performing attention flow processing on the aforementioned second feature map and obtaining a third feature map, The system includes a mixed processing module for obtaining a corresponding target feature map of the first feature map by sequentially performing channel addition convolution, attention flow processing, channel reduction convolution, and attention flow processing on the third feature map. A facial recognition device characterized by the following features.

9. An electronic device comprising memory, a processor, and a computer program stored in the memory and executable by the processor, The processor, when executing the computer program, realizes the steps of the method according to claim 1. An electronic device characterized by the following features.

10. A computer-readable storage medium on which computer programs are stored, The computer program, when executed by the processor, realizes the steps of the method according to claim 1. A computer-readable storage medium characterized by the following features.

Images