Face recognition method and device

Abstract

The present disclosure relates to the technical field of computers, and provides a face recognition method and device. The method comprises: acquiring a first feature map of a face image to be recognized; performing depth-by-depth convolution processing on the first feature map to obtain a second feature map; performing attention flow conversion processing on the second feature map to obtain a third feature map; and sequentially performing channel-increasing convolution processing, attention flow conversion processing, channel-reducing convolution processing and attention flow conversion processing on the third feature map to obtain a target feature map corresponding to the first feature map.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a face recognition method and apparatus. Background Technology

[0002] In practical applications, facial recognition technology often needs to be deployed in the cloud and at the edge. Due to the limited computing power and storage resources of edge devices such as embedded terminals, edge facial recognition models need to meet the requirements of high accuracy, small model size, low computational complexity, and fast inference speed.

[0003] Among related technologies, common lightweight networks for face recognition tasks include SqueezeNet, MobileNet, and ShuffleNet. However, due to the unique structure of the face, these models suffer from poor accuracy in face recognition. MobileFaceNet, a lightweight mobile network specifically designed for face recognition, uses a smaller dilation rate based on MobileNet and replaces global average pooling layers with global depthwise convolutional layers. However, MobileFaceNet's main building blocks still employ common residual bottleneck modules, and the computation of each module is identical, thus also exhibiting the problem of poor accuracy. Summary of the Invention

[0004] In view of this, the present disclosure provides a face recognition method, apparatus, electronic device, and computer-readable storage medium to solve the problem of poor accuracy of face recognition models in the prior art.

[0005] A first aspect of this disclosure provides a face recognition method, the method comprising: acquiring a first feature map of a face image to be recognized; performing depthwise convolution processing on the first feature map to obtain a second feature map; performing attention flow processing on the second feature map to obtain a third feature map; and sequentially performing convolution processing with increasing channels, attention flow processing, convolution processing with decreasing channels, and attention flow processing on the third feature map to obtain a target feature map corresponding to the first feature map.

[0006] A second aspect of this disclosure provides a face recognition device, comprising: an acquisition module for acquiring a first feature map of a face image to be recognized; a convolution module for performing depthwise convolution processing on the first feature map to obtain a second feature map; an attention transfer module for performing attention transfer processing on the second feature map to obtain a third feature map; and a hybrid processing module for sequentially performing convolution processing with increasing channels, attention transfer processing, convolution processing with decreasing channels, and attention transfer processing on the third feature map to obtain a target feature map corresponding to the first feature map.

[0007] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The beneficial effects of this disclosed embodiment compared with the prior art are: by combining convolution processing and attention flow processing to process the feature map of face recognition, attention is promoted to flow in multiple directional dimensions, so that the final feature map has high discriminative power in each directional dimension, thereby improving the recognition accuracy of the face recognition model.

[0010] Specifically, this disclosure proposes a lightweight attention transfer module with a very low tensor dimension. The convolution computation of low-dimensional tensors is very small, enabling faster overall operation. If the entire network extracts features in a low-dimensional space, it is highly likely to result in incomplete information and unrobust features. This disclosure addresses this by performing channel dilation with a set expansion coefficient during the intermediate convolution process, thereby improving the feature extraction capability of the entire module and achieving a delicate balance between computational cost and feature representation capability.

[0011] In this embodiment, the entire attention flow module combines different types of convolution, channel expansion and compression, attention flow techniques, etc., to make the attention flow of the face recognition task flow and transform between spaces and channels, making feature fusion more efficient. The feature map is finally effectively focused on the region of interest for face recognition. In addition, the attention flow module also has the advantages of fewer parameters, less computation, and faster speed. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;

[0014] Figure 2 This is a flowchart illustrating a face recognition method provided in an embodiment of this disclosure;

[0015] Figure 3 This is a schematic diagram of the attention flow processing provided in the embodiments of this disclosure;

[0016] Figure 4 This is a flowchart illustrating another face recognition method provided in this embodiment of the present disclosure;

[0017] Figure 5 This is a schematic diagram of the structure of a face recognition device provided in an embodiment of this disclosure;

[0018] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0020] The face recognition method and apparatus according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure. The application scenario may include terminal devices 101, 102, and 103, server 104, and network 105.

[0022] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays that support communication with server 104, including but not limited to smartphones, robots, laptops, and desktop computers (for example, 102 can be a robot). When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. Terminal devices 101, 102, and 103 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not limit this. Furthermore, various applications can be installed on terminal devices 101, 102, and 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0023] Server 104 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 104 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. This embodiment of the disclosure does not impose any limitations on these aspects.

[0024] It should be noted that server 104 can be either hardware or software. When server 104 is hardware, it can be various electronic devices that provide various services to terminal devices 101, 102, and 103. When server 104 is software, it can be multiple software programs or software modules that provide various services to terminal devices 101, 102, and 103, or it can be a single software program or software module that provides various services to terminal devices 101, 102, and 103. This disclosure does not limit the scope of the embodiments.

[0025] Network 105 can be a wired network using coaxial cable, twisted pair, and fiber optic connection, or it can be a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc. This disclosure does not limit the scope of the network.

[0026] Target users can establish a communication connection with server 104 via network 105 through terminal devices 101, 102, and 103 to receive or send information, etc. It should be noted that the specific types, quantities, and combinations of terminal devices 101, 102, and 103, server 104, and network 105 can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any limitations on this.

[0027] In related technologies, edge devices such as embedded terminals have limited computing power and storage resources, which can only support smaller model sizes, while general lightweight large face models do not have high face recognition accuracy.

[0028] To address this technical problem, this disclosure provides a face recognition solution. This solution improves the accuracy of face recognition by designing a simple, effective, lightweight, and universal model for extracting facial features, and by designing a real-time response face recognition model specifically for edge and embedded devices.

[0029] Specifically, the technical solution of this disclosure proposes a general attention flow technology that can effectively capture attention in space and channels, and improve feature discrimination through a channel-by-channel learnable nonlinear mapping method. The entire technology can extract effective feature combination methods and promote the flow of attention in multiple directional dimensions.

[0030] Figure 2 This is a schematic flowchart of a face recognition method provided in an embodiment of this disclosure. The method provided in this embodiment can be executed by any electronic device with computer processing capabilities, such as a terminal or server. Figure 2 As shown, the face recognition method includes:

[0031] Step S201: Obtain the first feature map of the face image to be identified.

[0032] Specifically, the first feature map is a 4-dimensional tensor with dimensions (N, C, H, W), where N represents the number of images in the batch, C represents the number of channels, H represents the height, and W represents the width. The first feature map is obtained by extracting features from the face image to be recognized.

[0033] Step S202: Perform depthwise convolution on the first feature map to obtain the second feature map.

[0034] Specifically, depthwise convolution (DWConv) performs convolution operations within each independent channel. In regular convolution, each convolution kernel performs calculations once for each channel, while in depthwise convolution, each convolution kernel performs calculations only for one channel.

[0035] Step S203: Perform attention flow processing on the second feature map to obtain the third feature map.

[0036] Specifically, attention flow processing enables attention to flow between space and channels, thereby achieving more effective feature fusion.

[0037] Step S204: Perform convolution processing with added channels, attention flow processing, convolution processing with reduced channels, and attention flow processing on the third feature map in sequence to obtain the target feature map corresponding to the first feature map.

[0038] Specifically, convolution processing that increases the number of channels and convolution processing that decreases the number of channels are two corresponding conventional convolution calculation processes. First, convolution processing that increases the number of channels is performed, and then convolution processing that decreases the number of channels is performed to restore the number of channels to the previous number.

[0039] According to the technical solution of this disclosure, attention flow processing can extract effective feature combination methods, promoting the flow of attention in multiple directional dimensions. By designing and combining attention flow processing technology with different types of convolution, the requirements of face recognition tasks and the lightweight requirements of embedded devices can be met simultaneously. Compared with the prior art, higher recognition accuracy can be achieved with fewer parameters.

[0040] like Figure 3 As shown, the attention flow processing in steps S203 and S204 includes the following steps:

[0041] Step S301: Flatten the first and second dimensions of the input feature map to obtain the first intermediate feature map.

[0042] Specifically, the first dimension can be height, and the second dimension can be width. Assuming the input feature map is f1, flattening the height and width dimensions of f1 transforms the dimensions (N, C, H, W) into (N, C, R), where R = H * W.

[0043] Step S302: Obtain the second intermediate feature map based on the first intermediate feature map and the first learnable parameter matrix.

[0044] In the technical solution of this disclosure embodiment, a first intermediate feature map can be obtained by multiplying the first intermediate feature map by the function value of its logistic regression function softmax, and then a second intermediate feature map can be obtained based on the mean of the first product. Specifically, the first intermediate feature map can be right-multiplied by a first learnable parameter matrix to obtain a tensor. The softmax function value of this tensor and the Hadamard product of the tensor are then calculated to obtain a matrix. The average of this matrix along a certain dimension is then taken to obtain the second intermediate feature map. The first learnable parameter matrix can learn attention flow information in the spatial dimension.

[0045] Step S303: Obtain the spatial attention feature map based on the product of the second intermediate feature map and the input feature map.

[0046] Specifically, the spatial attention feature map is a feature map that incorporates spatial attention.

[0047] Step S304: Obtain the channel attention feature map based on the second learnable parameter matrix, the third learnable parameter matrix, and the spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix.

[0048] Specifically, the spatial attention feature map can be right-multiplied by the second learnable parameter matrix to obtain the second product; the second product is then sparsified and right-multiplied by the third learnable parameter matrix to obtain the channel attention feature map. The second and third learnable parameter matrices can learn attention flow information along the channel dimension. By capturing the feature relationships between different channels, the weights of each channel can be learned, making the features more discriminative of information from each channel.

[0049] Step S305: Obtain the attention flow feature map based on the spatial attention feature map and the channel attention feature map.

[0050] Specifically, when obtaining the attention flow feature map based on the spatial attention feature map and the channel attention feature map, a non-linear mapping process can be performed on the spatial attention feature map to obtain a third intermediate feature map; the product of the third intermediate feature map and the channel attention feature map is used to obtain a fourth intermediate feature map; and a non-linear mapping process is performed on the fourth intermediate feature map to obtain the attention flow feature map. Based on the attention flow feature map obtained from the spatial and channel attention feature maps, attention flow information in both the spatial and channel dimensions can be learned, thereby enhancing the accuracy of attention flow in both dimensions.

[0051] The following is a detailed description of steps S301 to S305:

[0052] In step S301, assuming the input feature map is f1 with dimensions (N, C, H, W), the two dimensions H and W of f1 are flattened, and the dimensions are transformed to (N, C, R), to obtain the second intermediate feature map, where R = H*W.

[0053] In order to learn the attention along the feature H*W dimension, so that the attention flows in the spatial dimension, in this embodiment of the disclosure, a first learnable parameter matrix Q1 is introduced, with dimensions (R, r) (r < R).

[0054] In step S302, the first intermediate feature map obtained after dimensional transformation is right-multiplied by Q1 to obtain a tensor f′1 of dimension (N, C, r). Performing a softmax operation on the r dimension of f′1 yields a tensor A of dimension (N, C, r). s f'1 and A s Multiplying corresponding elements along the r dimension yields f'1 and A. s The Hadamard product yields a matrix M1 of size (N, C, r), where M1 represents a fusion of multiple feature combinations. The larger r is, the higher the complexity. Averaging M1 along the r-axis (average) compresses the dimension to 1, resulting in the second intermediate feature map. Its dimension is (N, C), and the specific calculation process is shown in the following formula (1):

[0055]

[0056] In this embodiment, the first learnable parameter matrix Q1 is introduced to calculate r spatial linear transformation results, which can extract representative feature combinations in the space. In the extracted face feature map, although each spatial pixel has the same receptive field, these receptive fields map to different regions of the original image, thus contributing differently to the final recognition task. Therefore, different weights should be assigned to different pixels. Using the first learnable parameter matrix Q1, attention along the H*W dimension of the features can be learned, allowing attention to flow across the spatial dimension and resulting in a fusion result of multiple feature combinations.

[0057] In step S303, the second intermediate feature map output in step S301 is... Multiplying by f1 yields the spatial attention feature map. Its dimensions are (N, C, H, W), and the specific calculation process is shown in the following formula (2):

[0058]

[0059] in, This is a feature map that incorporates spatial attention.

[0060] In step S304, the spatial attention feature map of dimension (N, C, H, W) is introduced into the second learnable parameter matrix Q2 and the third learnable parameter matrix Q3 for processing to obtain the channel attention feature map.

[0061] Specifically, the second learnable parameter matrix Q2 has dimensions (C, C / / p), and the third learnable parameter matrix Q3 has dimensions (C / / p, C), where C is a natural number. It is evident that the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and vice versa. Multiplying by Q2 on the right yields the dimension (N, C / / p). After ReLU sparsification, multiplying by Q3 on the right again yields the channel attention feature map. Its dimensions are (N, C).

[0062] The specific calculation process is shown in the following formula (3):

[0063]

[0064] In step S305, the second learnable parameter matrix Q2 and the third learnable parameter matrix Q3 are introduced into the channel output in step S304 to learn attention flow information in the channel dimension. This part of the design focuses more on the feature relationships between channels. By capturing the feature relationships between different channels, the weights of each channel are learned, making the features more discriminative for each channel's information. p represents the scaling factor. Designing parameter p can reduce the amount of computation and control the model size.

[0065] Dao attention feature map By performing a nonlinear mapping, the third intermediate feature fs can be obtained. The specific calculation process is shown in the following formulas (4) and (5):

[0066]

[0067]

[0068] in, i represents the i-th channel, which means performing a channel-by-channel nonlinear mapping on the feature map f'1, and the nonlinear mapping function for each channel can be different, with the mapping parameters for each channel ∈ i and k i It needs to be learned.

[0069] In data processing using nonlinear mapping, for negative inputs, compared to the ReLU method of directly mapping inputs with a value of 0 or less to 0 for output, it can be assumed that both positive and negative responses of the convolutional kernel should be accepted. In other words, it can be considered that faces need to learn negative inputs. Applying this nonlinear mapping method allows for the learning of more complex relationships within the data. Secondly, learning mapping values ​​per depth, i.e., learning channel-independent weights, is beneficial. This can be seen as a way to learn attention between different channels, enhancing the accuracy of attention flow between channels. Furthermore, with this channel-by-channel mapping method, the nonlinear mapping gradually becomes more "nonlinear" as the depth increases. That is, the model tends to retain information in shallower layers and enhance discriminative power in deeper layers. This is generally considered to mean that low-level feature maps have high resolution, weak semantic information, but rich spatial information, while high-level feature maps have low resolution but strong semantic information.

[0070] Furthermore, f s and Multiplication yields the fourth intermediate feature map f c The dimensions are (N, C, H, W), and the specific calculation process is shown in the following formula (6):

[0071]

[0072] To further enhance the expressive power of the features, the fourth intermediate feature map f is... cA nonlinear mapping is performed to obtain the attention flow feature map f. C The specific calculation process is shown in the following formulas (7) and (8):

[0073]

[0074]

[0075] in, f c The feature map represents the full flow of attention in both spatial and channel directions, until the flow of attention of interest spans the entire feature space.

[0076] From the above, we can see that f c The dimension of the input feature map f1 is (N, C, H, W), which is consistent with the dimension of the input feature map f1. Therefore, this attention transfer technique can be plugged into any module and any position in the neural network as a plug-and-play module, making it quite flexible in use. This attention transfer technique mainly achieves more effective feature fusion by transferring attention between space and channels, and enhances feature representation by learning a non-linear mapping through positive and negative responses channel by channel, thereby extracting more discriminative facial features. If we define this attention transfer technique as an SC function, with input f1 and output f... C Then we can obtain the following formula for attention flow (9):

[0077] f C =SC(f1) (9)

[0078] In this embodiment, an attention flow module can be formed based on the attention flow technology as a basic component of the neural network. This module can be designed with refined convolution modules tailored to the specific characteristics of facial structure, thereby achieving the function of extracting highly discriminative facial features with minimal computation, and effectively focusing the attention of the feature map on the region that is beneficial to the recognition task.

[0079] When the attention flow module is applied in steps S201 to S204, the implementation process of steps S201 to S204 can be described in detail as follows:

[0080] In step S202, the first feature map can be processed by depthwise convolution, and the depthwise convolution result can be batch normalized to obtain the second feature map. Specifically, depthwise convolution (DWConv) can be performed with a convolution kernel of n×n (n>1), input channels of C, output channels of C, padding of 1, and stride of s. Then, batch normalization (BatchNorm, abbreviated as BN) is performed to calculate the result f′1. Taking n=3 as an example, the specific calculation process is shown in the following formula (10):

[0081] f'1=BN(DWConv(f1,3×3)) (10)

[0082] The stride, which varies depending on the network design, is a configurable hyperparameter. In this embodiment, based on the idea of ​​designing small-sized modules, depthwise convolution is used instead of ordinary convolution to reduce the number of parameters. It can be calculated that the number of parameters for depthwise convolution is 1 / C of that for ordinary convolution. It should be noted that the 3×3 convolution can be replaced with larger convolution kernels such as 5×5 or 7×7, but the 3×3 convolution offers the best cost-effectiveness.

[0083] In step S203, the output f'1 from step S202 is subjected to the aforementioned attention flow calculation to obtain... The specific calculation process is shown in the following formula (11):

[0084]

[0085] In step S204, the convolution processing for increasing channels includes: performing convolution processing on the input feature map with N times the number of channels, and batch normalizing the convolution result, where N is a natural number; the convolution processing for decreasing channels includes: performing convolution processing on the input feature map with 1 / N channels, and batch normalizing the convolution result. Specifically, in step S204, the following steps can be performed sequentially:

[0086] The output of step S202 A convolution calculation (Conv) is performed with a 1×1 kernel, C input channels, C*expension output channels, and a stride of 1. Then, batch normalization is performed to obtain the result f2. The specific calculation process is shown in the following formula (12):

[0087]

[0088] Performing the attention flow calculation on f2 as described above, we obtain The specific calculation process is shown in the following formula (13):

[0089]

[0090] Will A convolutional calculation is performed with a 1×1 kernel, C*expension of input channels, C of output channels, and a stride of 1. Then, batch normalization is performed to obtain the result f3. The specific calculation process is shown in the following formula (14):

[0091]

[0092] Perform the attention flow calculation on f3 as described above, and obtain The specific calculation process is shown in the following formula (15):

[0093]

[0094] This disclosure proposes a lightweight attention flow module, which is finely designed for face recognition technology. The convolution design, linear and nonlinear mapping techniques follow two principles: first, to reduce network parameters, save computation, and improve processing speed; second, to perform more effective feature fusion in spatial and channel dimensions, enhance feature expression capabilities, and extract more discriminative face features.

[0095] In this embodiment, the attention flow module can be designed with a base number of 64 channels, resulting in a very low tensor dimension. The computational cost of convolution in low-dimensional tensors is also very small, enabling faster overall operation. If the entire network extracts features in a low-dimensional space, it is highly likely to lead to incomplete information and unrobust features. In this embodiment, channel dilation with a set expansion coefficient is performed during the intermediate convolution process, thereby improving the feature extraction capability of the entire module and achieving a delicate balance between computational cost and feature representation capability.

[0096] In this embodiment, the entire attention flow module combines different types of convolution, channel expansion and compression, attention flow techniques, etc., to make the attention flow of the face recognition task flow and transform between spaces and channels, making feature fusion more efficient. The feature map is finally effectively focused on the region of interest for face recognition. In addition, the attention flow module also has the advantages of fewer parameters, less computation, and faster speed.

[0097] like Figure 4 As shown in the embodiments of this disclosure, a face recognition method includes the following steps:

[0098] Step S401: The face image to be recognized is input into a convolutional layer and a normalization layer with a 3×3 kernel, 64 channels, and a stride of 1. In one specific embodiment, the resolution of the face image to be recognized is (1, 3, 112, 112). The feature map output by step S401 has a resolution of (1, 64, 112, 112).

[0099] In step S402, the feature map obtained in the previous step is input into an attention flow module with 64 basic channels, an expansion coefficient of 1, and a configurable stride of 2. The feature map output in step S402 has a resolution of (1, 64, 56, 56).

[0100] Step S403: Input the feature map obtained in the previous step into an attention flow module with 64 basic channels, an expansion coefficient of 1, and a configurable step size of 1. The feature map output by step S403 has a resolution of (1, 64, 56, 56).

[0101] Step S404: Input the feature map obtained in the previous step into an attention flow module with 64 basic channels, an expansion coefficient of 2, and a configurable stride of 2. The feature map output by step S404 has a resolution of (1, 64, 28, 28).

[0102] In step S405, the feature map obtained in the previous step is input into an attention flow module with 4 basic channels, 64 channels, an expansion coefficient of 2, and a configurable step size of 1. The feature map output in step S405 has a resolution of (1, 64, 28, 28).

[0103] Step S406: Input the feature map obtained in the previous step into an attention flow module with 128 basic channels, an expansion coefficient of 2, and a configurable stride of 2. The feature map output by step S406 has a resolution of (1, 128, 14, 14).

[0104] Step S407: Input the feature map obtained in the previous step into an attention flow module with 6 basic channels of 128, an expansion coefficient of 2, and a configurable step size of 1. The feature map output by step S407 has a resolution of (1, 128, 14, 14).

[0105] In step S408, the feature map obtained in the previous step is input into an attention flow module with 128 basic channels, an expansion coefficient of 2, and a configurable stride of 2. The feature map output by step S408 has a resolution of (1, 128, 7, 7).

[0106] Step S409: Input the feature map obtained in the previous step into two basic attention flow modules with 128 channels, an expansion coefficient of 2, and a configurable step size of 1. The feature map output by step S409 has a resolution of (1, 128, 7, 7).

[0107] Step S410: Input the feature map obtained in the previous step into a convolutional layer with a kernel of 1×1 and a normalization layer with 512 channels. The feature map output by step S410 has a resolution of (1, 512, 7, 7).

[0108] Step S411: Input the feature map obtained in the previous step into a convolutional layer with a kernel of 7×7 and a channel count of 512, and a normalization layer. The feature map output by step S411 has a resolution of (1, 512, 1, 1).

[0109] Step S412: After flattening the feature map obtained in the previous step, perform a (512, 512) fully connected matrix calculation to obtain a 512-dimensional vector as the target feature map.

[0110] In such Figure 4 In the face recognition method shown, steps S402 and S403 can be regarded as one stage, steps S404 and S405 can be regarded as one stage, steps S406 and S407 can be regarded as one stage, and steps S408 and S409 can be regarded as one stage. The number of attention flow modules included in each stage is (2, 5, 7, 3), but the combination of attention flow modules is only an exemplary description. Other combinations of attention flow modules can also achieve the technical effects of the technical solution of the present disclosure embodiment.

[0111] The technical solution of this disclosure proposes a general attention flow technology that can effectively capture attention in space and channels, and improve feature discrimination through a channel-by-channel learnable nonlinear mapping method. The entire technology can extract effective feature combination methods and promote the flow of attention in multiple directional dimensions.

[0112] According to the face recognition method of this disclosure, the feature map processing of face recognition is performed by combining convolution processing and attention flow processing, which promotes the flow of attention in multiple directional dimensions, so that the final feature map has high discriminative power in each directional dimension, thereby improving the recognition accuracy of the face recognition model.

[0113] The following are embodiments of the apparatus disclosed herein, which can be used to execute the embodiments of the method disclosed herein. The face recognition apparatus described below corresponds to the face recognition method described above. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the method embodiments of this disclosure.

[0114] Figure 5 This is a schematic diagram of a face recognition device provided in an embodiment of this disclosure. Figure 5 As shown, the facial recognition device includes:

[0115] The acquisition module 501 can be used to acquire the first feature map of the face image to be identified.

[0116] Specifically, the first feature map is a 4-dimensional tensor with dimensions (N, C, H, W), where N represents the number of images in the batch, C represents the number of channels, H represents the height, and W represents the width. The first feature map is obtained by extracting features from the face image to be recognized.

[0117] The convolution module 502 can be used to perform depthwise convolution on the first feature map to obtain the second feature map.

[0118] Specifically, depthwise convolution performs convolution operations within each independent channel. In regular convolution, each convolution kernel performs a calculation for each channel once, while in depthwise convolution, each convolution kernel performs a calculation for only one channel.

[0119] The attention transfer module 503 can be used to perform attention transfer processing on the second feature map to obtain the third feature map.

[0120] Specifically, attention flow processing enables attention to flow between space and channels, thereby achieving more effective feature fusion.

[0121] The hybrid processing module 504 can be used to sequentially perform convolution processing with increased channels, attention flow processing, convolution processing with decreased channels, and attention flow processing on the third feature map to obtain the target feature map corresponding to the first feature map.

[0122] Specifically, convolution processing that increases the number of channels and convolution processing that decreases the number of channels are two corresponding conventional convolution calculation processes. First, convolution processing that increases the number of channels is performed, and then convolution processing that decreases the number of channels is performed to restore the number of channels to the previous number.

[0123] According to the technical solution of this disclosure, attention flow processing can extract effective feature combination methods, promoting the flow of attention in multiple directional dimensions. By designing and combining attention flow processing technology with different types of convolution, the requirements of face recognition tasks and the lightweight requirements of embedded devices can be met simultaneously. Compared with the prior art, higher recognition accuracy can be achieved with fewer parameters.

[0124] In this embodiment of the disclosure, the attention flow module 503 can also be used to: flatten the first and second dimensions of the input feature map to obtain a first intermediate feature map; obtain a second intermediate feature map based on the first intermediate feature map and a first learnable parameter matrix; obtain a spatial attention feature map based on the product of the second intermediate feature map and the input feature map; obtain a channel attention feature map based on the second learnable parameter matrix, a third learnable parameter matrix, and the spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix; and obtain an attention flow feature map based on the spatial attention feature map and the channel attention feature map.

[0125] In the technical solution of this disclosure embodiment, a first intermediate feature map can be obtained by multiplying the first intermediate feature map with the function value of its logistic regression function softmax, and then a second intermediate feature map can be obtained based on the mean of the first product. Specifically, the first intermediate feature map can be right-multiplied by a first learnable parameter matrix to obtain a tensor. The softmax function value of the tensor and the Hadamard product of the tensor are then calculated to obtain a matrix, and the average of the matrix on a certain dimension is taken to obtain the second intermediate feature map.

[0126] Specifically, the spatial attention feature map is a feature map that incorporates spatial attention. The first learnable parameter matrix can learn attention flow information in the spatial dimension. The second and third learnable parameter matrices can learn attention flow information in the channel dimension. By capturing the feature relationships between different channels and learning the weights of each channel, the features can be made more discriminative of information from each channel. The attention flow feature map obtained from the spatial attention feature map and the channel attention feature map can learn attention flow information in both the spatial and channel dimensions, thereby enhancing the accuracy of attention flow in both dimensions.

[0127] In this embodiment of the disclosure, the attention flow module 503 can also be used to perform nonlinear mapping processing on the spatial attention feature map to obtain a third intermediate feature map; obtain a fourth intermediate feature map based on the product of the third intermediate feature map and the channel attention feature map; and perform nonlinear mapping processing on the fourth intermediate feature map to obtain an attention flow feature map.

[0128] In this embodiment of the disclosure, this nonlinear mapping method can learn more complex relationships in the data. Learning mapping values ​​per depth, i.e., learning channel-independent weights, is beneficial; it can be seen as a way of learning attention between different channels, enhancing the accuracy of attention flow between channels. Furthermore, for this channel-by-channel mapping method, the nonlinear mapping gradually becomes more "nonlinear" as the depth increases. That is, the model tends to retain information in shallower layers and strengthen discriminative power in deeper layers. This is generally considered to mean that lower-layer feature maps have high resolution, weak semantic information, but rich spatial information, while higher-layer feature maps have low resolution but strong semantic information.

[0129] In this embodiment of the disclosure, the attention flow module 503 can also be used to obtain a first product of the first intermediate feature map and its logistic regression function value; and to obtain a second intermediate feature map based on the mean of the first product.

[0130] In this embodiment of the disclosure, the attention flow module 503 can also be used to multiply the spatial attention feature map by the second learnable parameter matrix to obtain a second product; perform sparsification on the second product and multiply it by the third learnable parameter matrix to obtain a channel attention feature map.

[0131] In this embodiment, the first learnable parameter matrix Q1 is introduced to calculate r spatial linear transformation results, which can extract representative feature combinations in the space. In the extracted face feature map, although each spatial pixel has the same receptive field, these receptive fields map to different regions of the original image, contributing differently to the final recognition task. Therefore, different weights should be assigned to different pixels. Using the first learnable parameter matrix Q1, attention in the H*W dimension of the features can be learned, allowing attention to flow in the spatial dimension and obtaining a fusion result of multiple feature combinations. The second learnable parameter matrix Q2 and the third learnable parameter matrix Q3 can learn attention flow information in the channel dimension. This part of the design focuses more on the feature relationships between channels, learning the weights of each channel by capturing the feature relationships between different channels, making the features more discriminative of information from each channel.

[0132] In this embodiment of the disclosure, the hybrid processing module 504 can also be used to perform convolution processing that increases the number of channels by N times on the input feature map and batch normalize the convolution result, where N is a natural number; and to perform convolution processing that decreases the number of channels by 1 / N on the input feature map and batch normalize the convolution result.

[0133] In this embodiment of the disclosure, the convolution module 502 can also be used to perform depthwise convolution processing on the first feature map and batch normalize the depthwise convolution results to obtain the second feature map.

[0134] This disclosure proposes a lightweight attention flow module, which is finely designed for face recognition technology. The convolution design, linear and nonlinear mapping techniques follow two principles: first, to reduce network parameters, save computation, and improve processing speed; second, to perform more effective feature fusion in spatial and channel dimensions, enhance feature expression capabilities, and extract more discriminative face features.

[0135] In this embodiment, the attention flow module can be designed with a base number of 64 channels, resulting in a very low tensor dimension. The computational cost of convolution in low-dimensional tensors is also very small, enabling faster overall operation. If the entire network extracts features in a low-dimensional space, it is highly likely to lead to incomplete information and unrobust features. In this embodiment, channel dilation with a set expansion coefficient is performed during the intermediate convolution process, thereby improving the feature extraction capability of the entire module and achieving a delicate balance between computational cost and feature representation capability.

[0136] In this embodiment, the entire attention flow module combines different types of convolution, channel expansion and compression, attention flow techniques, etc., to make the attention flow of the face recognition task flow and transform between spaces and channels, making feature fusion more efficient. The feature map is finally effectively focused on the region of interest for face recognition. In addition, the attention flow module also has the advantages of fewer parameters, less computation, and faster speed.

[0137] The technical solution of this disclosure proposes a general attention flow technology that can effectively capture attention in space and channels, and improve feature discrimination through a channel-by-channel learnable nonlinear mapping method. The entire technology can extract effective feature combination methods and promote the flow of attention in multiple directional dimensions.

[0138] Since the functional modules of the face recognition device in the example embodiments of this disclosure correspond to the steps of the example embodiments of the face recognition method described above, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the face recognition method described above.

[0139] According to the face recognition device of this disclosure, the feature map processing for face recognition is performed by a combination of convolution processing and attention flow processing, which promotes the flow of attention in multiple directional dimensions, so that the final feature map has high discriminative power in each directional dimension, thereby improving the recognition accuracy of the face recognition model.

[0140] Figure 6 This is a schematic diagram of the electronic device 6 provided in an embodiment of this disclosure. Figure 6 As shown, the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the various method embodiments described above. Alternatively, when the processor 601 executes the computer program 603, it implements the functions of each module in the various device embodiments described above.

[0141] Electronic device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 6 may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or different components.

[0142] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0143] The memory 602 can be an internal storage unit of the electronic device 6, such as a hard disk or RAM of the electronic device 6. The memory 602 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 6. The memory 602 can also include both internal and external storage units of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0144] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If the integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0146] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A face recognition method, characterized in that, The method is applied to an edge device, and the method includes: Obtain the first feature map of the face image to be identified; The first feature map is subjected to depthwise convolution to obtain the second feature map; The second feature map is input into the attention flow module for attention flow processing to obtain the third feature map, wherein the basic number of channels of the attention flow module is 64; The third feature map is sequentially subjected to convolution processing with increased channels, attention flow processing, convolution processing with decreased channels, and attention flow processing to obtain the target feature map corresponding to the first feature map. The channel-increasing convolution processing includes: performing a convolution process on the input feature map to increase the number of channels by N times, where N is the dilation coefficient, and the channel-increasing convolution processing expands the number of channels to 64×N; The channel reduction convolution process includes: performing convolution processing on the input feature map to reduce the number of channels to 1 / N; The attention flow processing includes: Flatten the first and second dimensions of the input feature map to obtain the first intermediate feature map; A second intermediate feature map is obtained based on the first intermediate feature map and the first learnable parameter matrix; A spatial attention feature map is obtained by multiplying the second intermediate feature map and the input feature map; A channel attention feature map is obtained based on the second learnable parameter matrix, the third learnable parameter matrix, and the spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix. The spatial attention feature map is subjected to channel-by-channel nonlinear mapping processing to obtain a third intermediate feature map, wherein the nonlinear mapping functions of different channels are different. The fourth intermediate feature map is obtained by multiplying the third intermediate feature map and the channel attention feature map; The fourth intermediate feature map is subjected to the channel-by-channel nonlinear mapping process to obtain the attention flow feature map.

2. The method according to claim 1, characterized in that, Obtaining a second intermediate feature map based on the first intermediate feature map and the first learnable parameter matrix includes: Obtain the first product of the first intermediate feature map and its logistic regression function value; The second intermediate feature map is obtained based on the mean of the first product.

3. The method according to claim 1, characterized in that, The channel attention feature map is obtained based on the second learnable parameter matrix, the third learnable parameter matrix, and the spatial attention feature map, including: The spatial attention feature map is right-multiplied by the second learnable parameter matrix to obtain the second product; The second product is sparsified and then right-multiplied by the third learnable parameter matrix to obtain the channel attention feature map.

4. The method according to claim 1, characterized in that, After performing the aforementioned convolution process that adds channels, the method further includes: batch normalizing the convolution results; After performing the channel reduction convolution process, the method further includes batch normalization of the convolution results.

5. The method according to claim 4, characterized in that, The first feature map is subjected to depthwise convolution to obtain the second feature map, which includes: The first feature map is subjected to depthwise convolution, and the depthwise convolution result is batch normalized to obtain the second feature map.

6. A face recognition device, characterized in that, The device includes: The acquisition module is used to acquire the first feature map of the face image to be identified; The convolution module is used to perform depthwise convolution processing on the first feature map to obtain the second feature map; An attention flow module is used to perform attention flow processing on the second feature map to obtain a third feature map, wherein the basic number of channels of the attention flow module is 64; The hybrid processing module is used to sequentially perform convolution processing with increased channels, attention flow processing, convolution processing with decreased channels, and attention flow processing on the third feature map to obtain the target feature map corresponding to the first feature map; The channel-increasing convolution processing includes: performing a convolution process on the input feature map to increase the number of channels by N times, where N is the dilation coefficient, and the channel-increasing convolution processing expands the number of channels to 64×N; The channel reduction convolution process includes: performing convolution processing on the input feature map to reduce the number of channels to 1 / N; The attention flow processing includes: Flatten the first and second dimensions of the input feature map to obtain the first intermediate feature map; A second intermediate feature map is obtained based on the first intermediate feature map and the first learnable parameter matrix; A spatial attention feature map is obtained by multiplying the second intermediate feature map and the input feature map; A channel attention feature map is obtained based on the second learnable parameter matrix, the third learnable parameter matrix, and the spatial attention feature map, wherein the first dimension of the second learnable parameter matrix is ​​equal to the second dimension of the third learnable parameter matrix, and the first dimension of the third learnable parameter matrix is ​​equal to the second dimension of the second learnable parameter matrix. The spatial attention feature map is subjected to channel-by-channel nonlinear mapping processing to obtain a third intermediate feature map, wherein the nonlinear mapping functions of different channels are different. The fourth intermediate feature map is obtained by multiplying the third intermediate feature map and the channel attention feature map; The fourth intermediate feature map is subjected to the channel-by-channel nonlinear mapping process to obtain the attention flow feature map.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images