Lightweight face recognition method and system based on feature weighted fusion
By introducing channel attention and dynamic angle margin cutting technology in MobileFaceNet, the problems of environmental complexity and the impact of wrong data in natural scene face recognition are solved, the recognition accuracy and stability are improved, and more efficient model training is achieved and identify performance.
Patent Information
- Application Number
- CN202311172658.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-09-12
AI Technical Summary
Existing face recognition models are difficult to recognize in natural scenes, and are greatly affected by environmental complexity and wrong data, resulting in insufficient recognition accuracy and stability.
Using the improved MobileFaceNet, called SEMobileFaceNet, improves the stability of feature extraction and model training by adding channel attention mechanisms in the BottleBlock module and dynamically adjusting the angle margin in Arcface and cropping gradients.
It improves the accuracy and recall rate of face recognition in natural scenes, enhances the generalization ability of the model, reduces the instability and false detection rate of the training process, and improves the recognition performance.
Smart Images

Figure CN117409455B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of deep learning image processing, and in particular to a lightweight face recognition method, system, storage medium and computing device based on feature weighted fusion. Background Art
[0002] With the development of the Internet and the improvement of computing power in recent years, face recognition technology has been widely used in daily life and security fields, providing people with an efficient, accurate and convenient way of identity recognition and verification. Face recognition technology refers to a biometric technology based on face images or videos, which performs identity authentication or recognition by analyzing the characteristics and patterns of the face. However, due to the influence of various interference factors such as hardware, signal transmission, different postures and expressions, different lighting, occlusion, perspective distortion, etc., the difficulty of face recognition in natural scenes has increased dramatically. The widespread application of face recognition technology can have a profound impact on society. It can help improve public safety and prevent crime. It can also be used for social management, demographics, etc. Therefore, it is very necessary to apply face recognition technology to natural scenes.
[0003] With the rapid development of deep neural networks, some scholars have applied deep neural networks as feature extractors to the field of natural scene face recognition and achieved good results. However, the feature extraction of face data images by face recognition models is often performed on the entire image, and the background in the image will have a certain impact on the recognition accuracy of the face recognition model; at the same time, deep learning requires a large amount of labeled data for training, but the data quality of the collected large-scale face data sets is often difficult to guarantee. The impact of erroneous data will not only cause the face recognition model to oscillate during the training process, but also affect the recognition accuracy of the face recognition model; when the data contains a large amount of erroneous data, it may even cause face recognition training to fail. Summary of the invention
[0004] The first purpose of the present invention is to provide a lightweight face recognition method based on feature weighted fusion, aiming at the characteristics of the existing MobileFaceNet, taking into account the great influence of erroneous data on the training of face recognition models, and the difficulties in natural scene face recognition caused by complex environment. The method uses an improved MobileFaceNet, called SEMobileFaceNet, which has higher accuracy and recall rate in natural scene face recognition, better generalization ability, and can effectively cope with face recognition in complex environments. The network can be continuously optimized through training, and the training process is more efficient and stable, thereby improving the network recognition performance.
[0005] The second object of the present invention is to provide a lightweight face recognition system based on feature weighted fusion.
[0006] A third object of the present invention is to provide a storage medium.
[0007] A fourth object of the present invention is to provide a computing device.
[0008] The first object of the present invention is achieved by the following technical solutions: a lightweight face recognition method based on feature weighted fusion, the method is based on an improved MobileFaceNet to achieve real-time face recognition in natural scenes, the improved MobileFaceNet is called SEMobileFaceNet; SEMobileFaceNet is an improvement on the BottleBlock module and edge loss Arcface of the original MobileFaceNet, wherein the improvement on the BottleBlock module is: adding SE, using channel attention to weightedly fuse features of different channels, and improving the utilization of key features; the improvement on the edge loss Arcface is: converting the angle margin into a dynamic adjustment, ignoring or clipping part of the gradient according to the size of the gradient value;
[0009] The specific implementation of the lightweight face recognition method includes the following steps:
[0010] 1) Collect face data and pre-process the face data, including data cleaning and data enhancement; the face data includes face image data and face ID data as a label;
[0011] 2) Input the face image data into SEMobileFaceNet, train SEMobileFaceNet, and obtain the trained SEMobileFaceNet; wherein, during the training process, the face image data will first be converted into intermediate features through other pre-modules in SEMobileFaceNet before passing through SE, and the intermediate features will enter the adaptive average pooling and convolution modules in turn after entering SE to obtain channel weights, and then multiply the channel weights by the input intermediate features and add them to the input intermediate features to obtain weighted fused face features, and the obtained face features and the labels in the face data can be obtained by the improved Arcface to obtain the loss of SEMobileFaceNet, and then the gradient of SEMobileFaceNet is calculated based on the loss of SEMobileFaceNet, and after obtaining the gradient, the SEMobileFaceNet weights are updated using the gradient descent algorithm;
[0012] 3) Use the trained SEMobileFaceNet in actual natural scenes. First, input the facial image of the user to be identified into the trained SEMobileFaceNet to obtain the user's facial features, and store the user's facial features and user information in the face database. In subsequent use, input the facial image captured by the device into the trained SEMobileFaceNet to obtain real-time features, and compare the real-time features with the facial features in the face database by cosine similarity. Based on the comparison results, it can be determined whether the captured image is a user in the face database.
[0013] Furthermore, in step 1), the data enhancement is as follows: random horizontal flipping, image regularization, and random saturation and hue changes are performed on the face image data; through this data enhancement, the robustness and effectiveness of SEMobileFaceNet can be effectively improved; then, the enhanced image is converted to a size of 112×112 by cropping and scaling.
[0014] Further, in step 2), the SEMobileFaceNet comprises a strem module, a backbone module and a head module; the preprocessed face image data is input into the SEMobileFaceNet, the input image information is first read, and the early feature C is obtained by fast downsampling through the strem module. c2dw , where C c2dw It is the output feature of the second layer of SEMobileFaceNet;
[0015] Next, the early features C obtained by the strem module c2dw It is input to the backbone module, where the backbone module is composed of multiple stacked BottleBlock modules containing SE; the backbone module has a total of three stacked blocks, and the last layer of each stacked block is downsampled by 2 times to reduce the feature image size to half of the original; the early feature C c2dw After passing through three backbone modules, the original feature C is obtained c5 ;
[0016] In the BottleBlock module with SE, the feature C output from the previous layer is i-1 Input to the depth-separable convolution block to get the output feature C i , the depthwise separable convolutional block consists of three convolutional modules F c 、F dw 、F p The calculation process is shown in formula (1):
[0017] C i =Fp (F dw (F c (C i-1 ))) (1)
[0018] In the formula, i represents the current network layer, i-1 represents the previous network layer, C i-1 Represents the features of the previous layer output, F c represents grouped convolution, F dw represents channel-by-channel convolution, F p represents channel fusion convolution, C i Represents the output features after three layers of convolution;
[0019] In order to enhance the utilization of key channel feature information, the output feature C of the depthwise separable convolutional block is i Input to SE to get channel attention weight As shown in formula (2):
[0020]
[0021] In the formula, AvgPool represents the average pooling operation, MLP represents the number of channels compressed and then expanded using two convolutional layers, and σ represents the Sigmoid function;
[0022] After getting the channel attention channel weight After that, the channel attention weight The output feature C obtained by the depth-separable convolution block i Multiply them together to get the weighted features, and finally add the weighted features to the features C output from the previous layer. i-1 Add and fuse to get the output feature C of BottleBlock module i+1 , as shown in formula (3):
[0023]
[0024] Early Features C c2dw The original feature C is obtained through the backbone module c5 , then the original feature C c5 Input the head module to get the face embedding feature. Finally, the face embedding feature is passed through the BatchNormal function to get the final output face feature C o .
[0025] Further, in step 2), the improved Arcface is used to assist SEMobileFaceNet for training; the output facial features C oThe labels in the face data are input into the improved Arcface to get the loss of SEMobileFaceNet, and then the gradient descent algorithm is used to optimize SEMobileFaceNet. Finally, the distance between the facial features of the same face ID output by SEMobileFaceNet is continuously shortened;
[0026] The purpose of improving Arcface is to reduce the impact of erroneous samples and difficult samples in face data on model training and improve the accuracy of the model. The expression of Arcface is shown in formula (4):
[0027]
[0028] Where, L Arc represents the Arcface loss value, N represents the number of training samples, k represents the kth training sample, e represents the natural constant, s represents the scaling factor, x k represents the kth training sample feature vector, y k represents the kth training sample label, j represents the jth weight, W j represents the jth weight vector, θ j W j With x k The angle between the two, m represents the angle margin, Represents x k and The angle of Represents the weight vector of the last layer of SEMobileFaceNet;
[0029] The improvement of the angle margin m of Arcface is: the selection of the angle margin m is changed from the original fixed value to a sampling value sampled from a dynamic normal distribution; wherein the dynamic normal distribution is a normal distribution whose mean increases from an initial value to the final target value as the number of training times increases; the sampling method of the angle margin m is shown in formula (5):
[0030]
[0031] In the formula, μ represents the mean of the normal distribution, ε represents the standard deviation of the normal distribution, t represents the number of current trainings, epoch represents the total number of trainings required, and β t represents the change weight from the first round to the last round of training, α max Indicates the maximum angle margin, α min Indicates the minimum angle margin;
[0032] According to the loss calculated by Arcface, the gradient grad can be obtained; when the gradient grad∈[λ min ,λmax ], the face data corresponding to the gradient grad is considered to be the correct sample, and back propagation is performed; when the gradient grad∈[λ min ,-clipcalue) or grad∈(clipcalue,λ max ], the face data corresponding to the gradient grad is considered to be a difficult sample, and the gradient grad is limited to the interval [-clipcalue, clipcalue] to prevent the gradient from being too large and causing model oscillation; when the gradient When the gradient grad is within the interval, the face data corresponding to the gradient grad is considered to be an error sample, and the gradient grad is set to 0 and does not participate in back propagation. The calculation process is shown in formula (6):
[0033]
[0034] In the formula, clipcalue represents the clipping threshold, λ max represents the maximum gradient threshold, λ min represents the minimum gradient threshold;
[0035] After a preset number of initial trainings, in subsequent training, large gradient values generated during the training process will be pruned or discarded.
[0036] Further, in step 3), the facial image of the user to be identified is input into the trained SEMobileFaceNet to obtain the facial features of the user, and then the facial features and user information of the user are stored in the face database for face registration; wherein, when performing face registration, the facial features of each user and the ID information in the facial information are unique and correspond to each other, and the user must ensure that the facial image of the user has high image quality during the face registration process; when performing face comparison, a similarity threshold is set, and when the real-time features obtained from the captured face image and the facial features of all users in the face database have similarities all lower than the threshold, the captured face image is identified as a stranger's face image, and when the real-time features obtained from the captured face image and the facial features of users in the face database have one or more similarities greater than the threshold, the user information with the highest similarity is taken as the recognition result of the captured face.
[0037] The second object of the present invention is achieved by the following technical solution: a lightweight face recognition system based on feature weighted fusion, used to implement the above-mentioned lightweight face recognition method based on feature weighted fusion, which includes:
[0038] A data preprocessing module is used to collect and preprocess facial data;
[0039] The face recognition module realizes real-time face recognition in natural scenes based on SEMobileFaceNet. First, the face image of the user to be identified is input into the trained SEMobileFaceNet to obtain the user's face features, and the user's face features and user information are stored in the face database; in subsequent use, the face image captured by the device is input into the trained SEMobileFaceNet to obtain real-time features, and the real-time features are compared with the face features in the face database by cosine similarity. Based on the comparison results, it can be determined whether the captured image is a user in the face database.
[0040] The third objective of the present invention is achieved through the following technical solution: a storage medium storing a program, which, when executed by a processor, implements the above-mentioned lightweight face recognition method based on feature weighted fusion.
[0041] The fourth purpose of the present invention is achieved through the following technical solution: a computing device, comprising a processor and a memory for storing processor executable programs, when the processor executes the program stored in the memory, implements the above-mentioned lightweight face recognition method based on feature weighted fusion.
[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0043] 1. Aiming at the difficulties of natural scene face recognition, the present invention makes improvements based on MobileFaceNet. First, SE is added to the BottleBlock module; second, the angle margin of the edge loss Arcface is transformed into a dynamic adjustment, and some gradient values are clipped or discarded. The improved MobileFaceNet has higher accuracy and recall rate in natural scene face recognition, and better generalization ability.
[0044] 2. After being improved on MobileFaceNet, the present invention can effectively improve face recognition in complex environments. SE can more effectively identify and utilize key features in images and reduce missed detections and false detections.
[0045] 3. The present invention has broad application prospects. Its end-to-end training method can effectively reduce costs and improve the accuracy of face recognition in natural scenes. At the same time, the improvement of Arcface makes model training more efficient and stable, and has a certain market and prospect. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is the overall architecture diagram of SEMobileFaceNet. In the figure, the C block is the convolution operation, and C dw is the depth-wise separable convolution operation, C dThe squares are downsampling convolution operations.
[0047] Figure 2 This is a schematic diagram of the structure of the BottleBlock module with SE added. In the figure, Conv_g_block is a grouped convolution operation, Conv_dw_block is a channel-by-channel convolution operation, Conv_p_block is a channel fusion convolution operation, AvgPool is an average pooling operation, and MLP is a double-layer convolution operation that compresses and then expands the number of channels. is the pixel corresponding multiplication operation, It is the pixel corresponding addition operation.
[0048] Figure 3 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION
[0049] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.
[0050] Example 1
[0051] like Figure 1 and Figure 2 As shown, this embodiment discloses a lightweight face recognition method based on feature weighted fusion, which is based on an improved MobileFaceNet to achieve real-time face recognition in natural scenes, and the improved MobileFaceNet is called SEMobileFaceNet; SEMobileFaceNet is an improvement on the BottleBlock module and edge loss Arcface of the original MobileFaceNet, wherein the improvement on the BottleBlock module is: adding SE, using channel attention to weightedly fuse features of different channels, and improving the utilization of key features; the improvement on the edge loss Arcface is: converting the angle margin into a dynamic adjustment, and ignoring or clipping part of the gradient according to the size of the gradient value; the specific implementation of the lightweight face recognition method based on feature weighted fusion includes the following steps:
[0052] 1) Collect face data and preprocess the face data; the face data includes face image data and face ID data as a label; perform random horizontal flipping, image regularization and random saturation and hue changes on the face image data; then transform the enhanced image into a size of 112×112 by cropping and scaling; through this data enhancement method, the performance of SEMobileFaceNet can be effectively improved, and it is also more robust and effective for natural scene face recognition.
[0053] 2) Input the face image data into SEMobileFaceNet, train SEMobileFaceNet, and obtain the trained SEMobileFaceNet; during the training process, the face image data will first be converted into intermediate features through other pre-modules in SEMobileFaceNet before passing through SE. After entering SE, the intermediate features will successively enter the adaptive average pooling and convolution modules to obtain channel weights, and then multiply the channel weights by the input intermediate features and add them to the input intermediate features to obtain the weighted fused face features. The obtained face feature information and the labels in the face data are combined through the improved ArcFac e can get the loss of SEMobileFaceNet; then the gradient of SEMobileFaceNet can be calculated based on the loss of SEMobileFaceNet; after obtaining the gradient, the SEMobileFaceNet weight is updated using the gradient descent algorithm; among them, the improved Arcface angle margin is randomly sampled from the dynamic normal distribution; the mean of its dynamic normal distribution will increase from an initial value to the final target value as the number of training increases; after a certain number of initial trainings, in the subsequent training process, the larger gradient values generated during the training process will be cut or discarded; the specific situation is as follows:
[0054] The SEMobileFaceNet comprises a strem module, a backbone module and a head module. The preprocessed face image data is input into the SEMobileFaceNet, the input image information is first read, and the early feature C is obtained by fast downsampling through the strem module. c2dw , where C c2dw It is the output feature of the second layer of SEMobileFaceNet;
[0055] Next, the early features C obtained by the strem module c2dw It is input to the backbone module, where the backbone module is composed of multiple stacked BottleBlock modules containing SE; the backbone module has a total of three stacked blocks, and the last layer of each stacked block is downsampled by 2 times to reduce the feature image size to half of the original; the early feature C c2dw After passing through three backbone modules, the original feature C is obtained c5 ;
[0056] In the BottleBlock module with SE, the feature C output from the previous layer is i-1 Input to the depth-separable convolution block to get the output feature C i , the depthwise separable convolutional block consists of three convolutional modules Fc 、F dw 、F p The calculation process is shown in formula (1):
[0057] C i =F p (F dw (F c (C i-1 ))) (1)
[0058] In the formula, i represents the current network layer, i-1 represents the previous network layer, C i-1 Represents the features of the previous layer output, F c represents grouped convolution, F dw represents channel-by-channel convolution, F p represents channel fusion convolution, C i Represents the output features after three layers of convolution;
[0059] In order to enhance the utilization of key channel feature information, the output features of the depthwise separable convolutional block are input into SE to obtain the channel attention weights. As shown in formula (2):
[0060]
[0061] In the formula, AvgPool represents the average pooling operation, MLP represents the number of channels compressed and then expanded using two convolutional layers, and σ represents the Sigmoid function;
[0062] After getting the channel attention channel weight After that, the channel attention weight The output feature C obtained by the depth-separable convolution block i Multiply them together to get the weighted features, and finally add the weighted features to the features C output from the previous layer. i-1 Add and fuse to get the output feature C of BottleBlock module i+1 , as shown in formula (3):
[0063]
[0064] Early Features C c2dw The original feature C is obtained through the backbone module c5 , then the original feature C c5 Input the head module to get the face embedding feature, and finally pass the face embedding feature through the BatchNormal function to get the final output face feature C o ;
[0065] By adding SE, we can enhance the utilization of key features and improve the accuracy and robustness of SEMobileFaceNet;
[0066] Use the improved Arcface to assist SEMobileFaceNet for training; output facial features C o The labels in the face data are input into the improved Arcface to get the loss of SEMobileFaceNet, and then the gradient descent algorithm is used to optimize SEMobileFaceNet, and finally the distance between the facial features of the same face ID output by SEMobileFaceNet is continuously shortened;
[0067] The purpose of improving Arcface is mainly to reduce the impact of erroneous samples and difficult samples in face data on model training and improve the accuracy of the model. The expression of Arcface is shown in formula (4):
[0068]
[0069] Where, L Arc represents the Arcface loss value, N represents the number of training samples, k represents the kth training sample, e represents the natural constant, s represents the scaling factor, x k represents the kth training sample feature vector, y k represents the kth training sample label, j represents the jth weight, W j represents the jth weight vector, θ j W j With x k The angle between the two, m represents the angle margin, Represents x k and The angle of Represents the weight vector of the last layer of SEMobileFaceNet;
[0070] The improvement of the angle margin m of Arcface is: the selection of the angle margin m is changed from the original fixed value to a sampling value sampled from a dynamic normal distribution; wherein the dynamic normal distribution is a normal distribution whose mean increases from an initial value to the final target value as the number of training times increases; the sampling method of the angle margin m is shown in formula (5):
[0071]
[0072] In the formula, μ represents the mean of the normal distribution, ε represents the standard deviation of the normal distribution, t represents the number of current trainings, epoch represents the total number of trainings required, and β t represents the change weight from the first round to the last round of training, αmax Indicates the maximum angle margin, α min Indicates the minimum angle margin;
[0073] According to the loss calculated by Arcface, we can get the gradient grad; when the gradient grad∈[λ min ,λ max ], the face data corresponding to the gradient grad is considered to be the correct sample, and the back propagation is performed normally; when the gradient is grad∈[λ min ,-clipcalue) or grad∈(clipcalue,λ max ], the face data corresponding to the gradient grad is considered to be a difficult sample, and the gradient grad will be limited to the interval [-clipcalue, clipcalue] to prevent the gradient from being too large and causing model oscillation; when the gradient When the gradient grad is within the interval, the face data corresponding to the gradient grad is considered to be an error sample, and the gradient grad is set to 0 and does not participate in back propagation. The calculation process is shown in formula (6):
[0074]
[0075] In the formula, clipcalue represents the clipping threshold, λ max represents the maximum gradient threshold, λ min represents the minimum gradient threshold;
[0076] By improving arcface, the stability of the model training process can be improved. At the same time, by discarding abnormal data, the accuracy of the model can be improved to a certain extent.
[0077] 3) Use the trained SEMobileFaceNet for actual natural scenes. First, input the facial image of the user to be identified into the trained SEMobileFaceNet to obtain the user's facial features, and then store the user's facial features and user information into the face database for face registration. When performing face registration, the facial features of each user and the ID information in the facial information are unique and correspond to each other, and the user must ensure that the user's facial image has high image quality during the face registration process. When performing face comparison, a similarity threshold is set. When the real-time features obtained from the captured face image and the facial features of all users in the face database have similarities lower than the threshold, the captured face image is identified as a stranger's face image. When the real-time features obtained from the captured face image and the facial features of users in the face database have one or more similarities greater than the threshold, the user information with the highest similarity is taken as the recognition result of the captured face.
[0078] Example 2
[0079] This embodiment discloses a lightweight face recognition system based on weighted feature fusion, which is used to implement the lightweight face recognition method based on weighted feature fusion described in Embodiment 1. Figure 3 As shown, the system includes the following functional modules:
[0080] A data preprocessing module is used to collect and preprocess facial data;
[0081] The face recognition module realizes real-time face recognition in natural scenes based on SEMobileFaceNet. First, the face image of the user to be identified is input into the trained SEMobileFaceNet to obtain the user's face features, and the user's face features and user information are stored in the face database; in subsequent use, the face image captured by the device is input into the trained SEMobileFaceNet to obtain real-time features, and the real-time features are compared with the face features in the face database by cosine similarity. Based on the comparison results, it can be determined whether the captured image is a user in the face database.
[0082] Example 3
[0083] This embodiment discloses a storage medium storing a program. When the program is executed by a processor, the lightweight face recognition method based on feature weighted fusion described in Embodiment 1 is implemented.
[0084] The storage medium in this embodiment can be a disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), a USB flash drive, a mobile hard disk, or other media.
[0085] Example 4
[0086] This embodiment discloses a computing device, including a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the lightweight face recognition method based on feature weighted fusion described in Embodiment 1 is implemented.
[0087] The computing device described in this embodiment may be a desktop computer, a laptop computer, a smart phone, a PDA handheld terminal, a tablet computer, a programmable logic controller (PLC), or other terminal devices with a processor function.
[0088] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.
Claims
1. A lightweight face recognition method based on feature weighted fusion, characterized in that: This method is based on the improved MobileFaceNet to achieve real-time face recognition in natural scenes. The improved MobileFaceNet is called SEMobileFaceNet. SEMobileFaceNet is an improvement on the BottleBlock module and edge loss Arcface of the original MobileFaceNet. The improvement on the BottleBlock module is: adding SE and using channel attention to weightedly fuse the features of different channels to improve the utilization of key features; the improvement on the edge loss Arcface is: converting the angle margin into dynamic adjustment, ignoring or clipping part of the gradient according to the size of the gradient value; The specific implementation of the lightweight face recognition method includes the following steps: 1) Collect facial data and pre-process the facial data, including data cleaning and data enhancement; the facial data includes facial image data and facial ID data as a label; 2) Input the face image data into SEMobileFaceNet, train SEMobileFaceNet, and obtain the trained SEMobileFaceNet; wherein, during the training process, the face image data will first be converted into intermediate features through other pre-modules in SEMobileFaceNet before passing through SE, and the intermediate features will enter the adaptive average pooling and convolution modules in turn after entering SE to obtain channel weights, and then multiply the channel weights by the input intermediate features and add them to the input intermediate features to obtain weighted fused face features, and the obtained face features and the labels in the face data can be obtained by the improved Arcface to obtain the loss of SEMobileFaceNet, and then the gradient of SEMobileFaceNet is calculated based on the loss of SEMobileFaceNet, and after obtaining the gradient, the SEMobileFaceNet weights are updated using the gradient descent algorithm; Use the improved Arcface to assist SEMobileFaceNet for training; output facial features C o The labels in the face data are input into the improved Arcface to get the loss of SEMobileFaceNet, and then the gradient descent algorithm is used to optimize SEMobileFaceNet. Finally, the distance between the facial features of the same face ID output by SEMobileFaceNet is continuously shortened; The purpose of improving Arcface is to reduce the impact of erroneous samples and difficult samples in face data on model training and improve the accuracy of the model. The expression of Arcface is shown in formula (4): Where, L Arc represents the Arcface loss value, N represents the number of training samples, k represents the kth training sample, e represents the natural constant, s represents the scaling factor, x k represents the kth training sample feature vector, y k represents the kth training sample label, j represents the jth weight, W j represents the jth weight vector, θ j W j With x k The angle between the two, m represents the angle margin, Represents x k and The angle of Represents the weight vector of the last layer of SEMobileFaceNet; The improvement of the angle margin m of Arcface is: the selection of the angle margin m is changed from the original fixed value to a sampling value sampled from a dynamic normal distribution; wherein the dynamic normal distribution is a normal distribution whose mean increases from an initial value to the final target value as the number of training times increases; the sampling method of the angle margin m is shown in formula (5): In the formula, μ represents the mean of the normal distribution, ε represents the standard deviation of the normal distribution, t represents the number of current trainings, epoch represents the total number of trainings required, and β t represents the change weight from the first round to the last round of training, α max Indicates the maximum angle margin, α min Indicates the minimum angle margin; According to the loss calculated by the improved Arcface, the gradient grad can be obtained; when the gradient grad∈[λ min ,λ max ], the face data corresponding to the gradient grad is considered to be the correct sample, and back propagation is performed; when the gradient grad∈[λ min ,-clipcalue) or grad∈(clipcalue,λ max ], the face data corresponding to the gradient grad is considered to be a difficult sample, and the gradient grad is limited to the interval [-clipcalue, clipcalue] to prevent the gradient from being too large and causing model oscillation; when the gradient When the gradient grad is within the interval, the face data corresponding to the gradient grad is considered to be an error sample, and the gradient grad is set to 0 and does not participate in back propagation. The calculation process is shown in formula (6): In the formula, clipcalue represents the clipping threshold, λ max represents the maximum gradient threshold, λ min represents the minimum gradient threshold; After a preset number of initial trainings, in subsequent training, large gradient values generated during the training process will be clipped or discarded; 3) Use the trained SEMobileFaceNet in actual natural scenes. First, input the facial image of the user to be identified into the trained SEMobileFaceNet to obtain the user's facial features, and store the user's facial features and user information in the face database. In subsequent use, input the facial image captured by the device into the trained SEMobileFaceNet to obtain real-time features, and compare the real-time features with the facial features in the face database by cosine similarity. Based on the comparison results, it can be determined whether the captured image is a user in the face database.
2. The lightweight face recognition method based on feature weighted fusion according to claim 1 is characterized in that: In step 1), the data enhancement is as follows: random horizontal flipping, image regularization and random saturation and hue changes are performed on the face image data; through this data enhancement, the robustness and effectiveness of SEMobileFaceNet can be effectively improved; then, the enhanced image is converted to a size of 112×112 by cropping and scaling.
3. The lightweight face recognition method based on feature weighted fusion according to claim 2 is characterized in that: In step 2), the SEMobileFaceNet includes a strem module, a backbone module and a head module; the preprocessed face image data is input into SEMobileFaceNet, the input image information is first read, and the early feature C is obtained by fast downsampling through the strem module. c2dw , where C c2dw It is the output feature of the second layer of SEMobileFaceNet; Next, the early features C obtained by the strem module c2dw It is input to the backbone module, where the backbone module is composed of multiple stacked BottleBlock modules containing SE; the backbone module has a total of three stacked blocks, and the last layer of each stacked block is downsampled by 2 times to reduce the feature image size to half of the original; the early feature C c2dw After passing through three backbone modules, the original feature C is obtained c5 ; In the BottleBlock module with SE, the feature C output from the previous layer is i-1 Input to the depth-separable convolution block to get the output feature C i , the depthwise separable convolutional block consists of three convolutional modules F c 、F dw 、F p The calculation process is shown in formula (1): C i =F p (F dw (F c (C i-1 ))) (1) In the formula, i represents the current network layer, i-1 represents the previous network layer, C i-1 Represents the features of the previous layer output, F c represents grouped convolution, F dw represents channel-by-channel convolution, F p represents channel fusion convolution, C i Represents the output features after three layers of convolution; In order to enhance the utilization of key channel feature information, the output feature C of the depthwise separable convolutional block is i Input to SE to get channel attention weight As shown in formula (2): In the formula, AvgPool represents the average pooling operation, MLP represents the number of channels compressed and then expanded using two convolutional layers, and σ represents the Sigmoid function; After getting the channel attention channel weight After that, the channel attention weight The output feature C obtained by the depth-separable convolution block i Multiply them together to get the weighted features, and finally add the weighted features to the features C output from the previous layer. i-1 Add and fuse to get the output feature C of BottleBlock module i+1 , as shown in formula (3): Early Features C c2dw The original feature C is obtained through the backbone module c5 , then the original feature C c5 Input the head module to get the face embedding feature. Finally, the face embedding feature is passed through the BatchNormal function to get the final output face feature C o .
4. The lightweight face recognition method based on feature weighted fusion according to claim 3 is characterized in that: In step 3), the facial image of the user to be identified is input into the trained SEMobileFaceNet to obtain the facial features of the user, and then the facial features and user information of the user are stored in the face database for face registration; wherein, when performing face registration, the facial features of each user and the ID information in the facial information are unique and correspond to each other, and the user must ensure that the facial image of the user has high image quality during the face registration process; when performing face comparison, a similarity threshold is set, and when the real-time features obtained from the captured face image and the facial features of all users in the face database have similarities all lower than the threshold, the captured face image is identified as a stranger's face image, and when the real-time features obtained from the captured face image and the facial features of users in the face database have one or more similarities greater than the threshold, the user information with the highest similarity is taken as the recognition result of the captured face.
5. A lightweight face recognition system based on feature weighted fusion, characterized in that: A lightweight face recognition method based on feature weighted fusion for implementing any one of claims 1 to 4, comprising: A data preprocessing module is used to collect and preprocess facial data; The face recognition module realizes real-time face recognition in natural scenes based on SEMobileFaceNet. First, the face image of the user to be identified is input into the trained SEMobileFaceNet to obtain the user's face features, and the user's face features and user information are stored in the face database; in subsequent use, the face image captured by the device is input into the trained SEMobileFaceNet to obtain real-time features, and the real-time features are compared with the face features in the face database by cosine similarity. Based on the comparison results, it can be determined whether the captured image is a user in the face database.
6. A storage medium storing a program, characterized in that: When the program is executed by a processor, the lightweight face recognition method based on feature weighted fusion described in any one of claims 1 to 4 is implemented.
7. A computing device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the lightweight face recognition method based on feature weighted fusion described in any one of claims 1 to 4 is implemented.
