Privacy protection face recognition method and system based on feature compensation

Abstract

The invention discloses a privacy protection face recognition method and system based on feature compensation, and belongs to the technical field of image processing privacy protection. A face image is collected, and the face image is preprocessed to obtain a preprocessed face image; performing blurring processing on the preprocessed face image to obtain a first blurred image; carrying out LBP (Local Binary Pattern) filtering on the preprocessed face image, and carrying out fuzzy processing on the face image subjected to LBP filtering to obtain a second fuzzy image; performing compensation feature extraction by using the preprocessed face image and the first blurred image; performing compensation feature extraction by using the face image after LBP filtering and the second blurred image; performing feature extraction by using the first blurred image and the first compensation feature group; performing feature extraction by using the second blurred image and the second compensation feature group; performing dimension addition on the first feature vector and the second feature vector to obtain a feature vector; and matching the feature vectors with feature vectors in a database to obtain a face recognition result.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing privacy protection, and in particular relates to a privacy protection face recognition method and system based on feature compensation. Background Art

[0002] The widespread application of face recognition technology in security, finance, social and other fields has improved convenience and efficiency, but it has also brought serious privacy protection issues. At present, in order to deal with the risk of face data leakage and abuse, a variety of privacy protection technologies have been proposed. For example, data encryption technology prevents unauthorized access by encrypting the storage and transmission of face images or feature vectors. Such methods include symmetric encryption, asymmetric encryption and homomorphic encryption. Among them, homomorphic encryption can directly operate data in an encrypted state, but its computational overhead is large and it is difficult to meet real-time requirements. In addition, differential privacy technology adds noise to the data to ensure that even if an attacker obtains statistical information, the specific data of a single user cannot be inferred. This method provides theoretical guarantees for privacy protection, but the introduced noise will reduce the accuracy of the face recognition model, especially in scenarios that require high-precision recognition. Federated learning technology reduces the risk of privacy leakage by distributing the model training process on multiple devices, without the need to centrally upload face data. However, this method is susceptible to communication overhead and model synchronization problems, and there may be hidden dangers of adversarial attacks. In general, although these technologies provide certain solutions for face recognition privacy protection, they still have shortcomings in terms of computational efficiency, model performance, and practical applications. Algorithm design needs to be further optimized to achieve a balance between privacy protection and recognition performance.

[0003] In face recognition systems, the weak computing power of client edge devices is a major challenge, especially in large-scale application scenarios that require real-time processing and analysis. Edge devices such as smart door locks, surveillance cameras, and mobile devices are often limited by processor performance, storage space, and energy budget, making it difficult to run complex face recognition algorithms. Current mainstream face recognition algorithms rely on deep learning models, which usually contain a large number of parameters and computationally intensive operations such as convolution operations and matrix multiplications, resulting in the need for powerful hardware support for model inference. However, edge devices are difficult to match the computing power of server-level models, and may experience high latency or even fail to complete recognition tasks when running these models. In addition, in order to run models on edge devices, model compression and pruning are usually required, which can reduce the computing requirements and storage usage of the model, but may also lead to a decrease in recognition accuracy, especially in recognition tasks in complex environments. Another problem is that edge devices usually need to process data in real time, which places higher requirements on computing efficiency and power consumption. High power consumption not only limits the use scenarios of the device, but may also cause the device to overheat, further reducing performance. Therefore, how to make good use of the powerful computing power of the server to share computing under the limited computing power of edge devices and achieve efficient and accurate face recognition under the premise of privacy protection is still an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to overcome the problem that efficient and accurate face recognition cannot be achieved under the premise of privacy protection, and proposes a privacy protection face recognition method and system based on feature compensation.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme: In a first aspect, the present invention provides a privacy-preserving face recognition method based on feature compensation, comprising the following steps: Collecting a face image, preprocessing the face image to obtain a preprocessed face image; performing blur processing on the preprocessed face image to obtain a first blurred image; performing LBP filtering on the preprocessed face image to obtain an LBP filtered face image; performing blur processing on the LBP filtered face image to obtain a second blurred image; Use the preprocessed face image and the first blurred image to extract compensation features to obtain a first compensation feature group; use the LBP filtered face image and the second blurred image to extract compensation features to obtain a second compensation feature group; Using the first blurred image and the first compensating feature group to perform feature extraction, obtaining a first feature vector; using the second blurred image and the second compensating feature group to perform feature extraction, obtaining a second feature vector; Add the dimensions of the first eigenvector and the second eigenvector to obtain an eigenvector; The feature vector is matched with the feature vector in the face feature vector database to obtain the face recognition result.

[0006] Furthermore, the blur processing method adopts Gaussian blur, median blur, mean blur or a mixed blur of several blur methods.

[0007] Furthermore, the preprocessing is performed by MTCNN, and the size of the face after preprocessing is 112×112 pixels.

[0008] Furthermore, the preprocessed face image and the first blurred image are used to extract compensation features, and the first compensation feature group is obtained as follows: Subtracting each corresponding pixel of the preprocessed face image from that of the first blurred image to obtain a first compensated image, using the first compensated image as an input of a lightweight feature extraction neural network for feature extraction, and selecting three intermediate feature vectors in an intermediate layer of the lightweight feature extraction neural network as a first compensated feature group; The face image after LBP filtering and the second blurred image are used to extract compensation features, and the second compensation feature group is obtained as follows: Subtract each corresponding pixel of the LBP filtered face image from the second blurred image to obtain a second compensated image, and use the second compensated image as the input of a lightweight feature extraction neural network for feature extraction. Three intermediate feature vectors are selected in the intermediate layer of the lightweight feature extraction neural network as the second compensated feature group.

[0009] Furthermore, the lightweight feature extraction neural network adopts MobileNet or ShuffleNet.

[0010] Furthermore, the process of extracting features using the first blurred image and the first compensation feature group to obtain the first feature vector is: The first blurred image is used as the input of the first feature extraction neural network, a first intermediate vector having the same dimension as the first intermediate feature vector in the first compensation feature group is selected from the intermediate layer of the first feature extraction neural network, the first intermediate feature vector is added to the corresponding dimension component of the first intermediate vector as a first fused feature vector, and the first fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a second intermediate vector of the same dimension as the second intermediate feature vector in the first compensation feature group, adding the second intermediate feature vector and the corresponding dimension component of the second intermediate vector as a second fused feature vector, and the second fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a third intermediate vector of the same dimension as the third intermediate feature vector in the first compensation feature group, adding the third intermediate feature vector to the corresponding dimension component of the third intermediate vector as a third fused feature vector, and continuing to propagate the third fused feature vector backward in the feature extraction neural network to obtain the first feature vector; The first feature extraction neural network adopts ArcFace or FaceNet.

[0011] Furthermore, the process of extracting features using the second blurred image and the second compensation feature group to obtain the second feature vector is as follows: The second blurred image is used as the input of the second feature extraction neural network, a first intermediate vector having the same dimension as the first intermediate feature vector in the second compensation feature group is selected from the intermediate layer of the second feature extraction neural network, the first intermediate feature vector is added to the corresponding dimension component of the first intermediate vector as a first fused feature vector, and the first fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a second intermediate vector of the same dimension as the second intermediate feature vector in the second compensation feature group, adding the second intermediate feature vector and the corresponding dimension component of the second intermediate vector as a second fused feature vector, and the second fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a third intermediate vector of the same dimension as the third intermediate feature vector in the second compensation feature group, adding the third intermediate feature vector and the corresponding dimension component of the third intermediate vector as a third fused feature vector, and the third fused feature vector continues to propagate backward in the feature extraction neural network to obtain a second feature vector; The second feature extraction neural network adopts ArcFace or FaceNet.

[0012] In a second aspect, the present invention provides a privacy protection face recognition system based on feature compensation, comprising: An image acquisition module is used to acquire a face image and preprocess the face image to obtain a preprocessed face image; An LBP filtering module is used to perform LBP filtering on the preprocessed face image to obtain a face image after LBP filtering; The image blur processing module is used to blur the pre-processed face image to obtain a first blurred image; blur the LBP filtered face image to obtain a second blurred image; A feature compensation extraction module is used to extract compensation features using the preprocessed face image and the first blurred image to obtain a first compensation feature group; and to extract compensation features using the LBP filtered face image and the second blurred image to obtain a second compensation feature group; A feature extraction module is used to perform feature extraction using the first blurred image and the first compensation feature group to obtain a first feature vector; perform feature extraction using the second blurred image and the second compensation feature group to obtain a second feature vector; and perform dimension addition of the first feature vector and the second feature vector to obtain a feature vector; The feature matching module is used to match the feature vector with the feature vector in the database to obtain the face recognition result.

[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the privacy-preserving face recognition method based on feature compensation when executing the computer program.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a privacy-preserving face recognition method based on feature compensation is described.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention proposes a privacy-preserving face recognition method based on feature compensation. Starting from the concept of feature compensation, the present invention addresses the contradiction that the client device has weak computing power in real application scenarios, but the original face image is not convenient to be directly transmitted to the cloud server as private information. The present invention is a privacy-preserving face recognition system based on a convolutional neural network model. It can achieve efficient face recognition while protecting the original face image information from being leaked. Compared with the existing technology, the present invention has strong environmental adaptability, simple operation and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings: Figure 1 The present invention is a flowchart of a privacy protection face recognition method based on feature compensation.

[0017] Figure 2 This is a structural diagram of a privacy-preserving face recognition system based on feature compensation according to the present invention.

[0018] Figure 3 This is a diagram of an electronic device for a privacy-preserving face recognition method based on feature compensation according to the present invention.

[0019] Figure 4 Diagram of the process being handled for the client.

[0020] Figure 5 Diagram of the process of extracting server feature vectors. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] Embodiment 1 See also Figure 1 , a privacy-preserving face recognition method based on feature compensation, comprising the following steps: Collect the face image, pre-process the face image to obtain the pre-processed face image; the first step is to collect the face image, and then use image processing technology to pre-process the image to improve the image quality, laying a good foundation for the subsequent blur processing and feature extraction. The pre-processed face image is blurred to obtain the first blurred image; this step is to protect personal privacy, reduce the detailed information in the image, and avoid the leakage of sensitive information. The pre-processed face image is LBP filtered to obtain the LBP filtered face image; LBP filtering can capture the local texture features of the image, which plays an important role in face recognition. The LBP filtered face image is blurred to obtain the second blurred image; LBP (local binary pattern) filtering can extract the texture features of the image, and the subsequent blur processing is to further protect the privacy of these features.

[0023] The preprocessed face image and the first blurred image are used to extract compensation features to obtain the first compensation feature group; the purpose of this step is to recover some important features lost due to blurring from the blurred image. The LBP filtered face image and the second blurred image are used to extract compensation features to obtain the second compensation feature group; by combining the LBP filtered image with the blurred image, richer and more stable features can be extracted.

[0024] Using the first blurred image and the first compensating feature group to perform feature extraction, obtaining a first feature vector; using the second blurred image and the second compensating feature group to perform feature extraction, obtaining a second feature vector; The first eigenvector and the second eigenvector are dimensionally added to obtain the eigenvector; this step aims to integrate feature information from different sources to improve recognition accuracy.

[0025] The feature vector is matched with the feature vector in the face feature vector database to obtain the face recognition result.

[0026] This embodiment effectively reduces the detailed information in the image through blur processing, thereby protecting personal privacy to a certain extent. By using the pre-processed face image, the LBP filtered image and the blurred image to perform compensatory feature extraction, some important features lost due to blur can be restored, thereby improving the accuracy of recognition. The feature information from different sources is integrated, and the various feature information in the image is fully utilized, further improving the accuracy of recognition.

[0027] The method of this embodiment has broad application prospects in the field of face recognition, especially in situations where personal privacy needs to be protected, such as intelligent monitoring, mobile payment, identity authentication, etc. By combining feature compensation and fuzzy processing, the method can achieve efficient and accurate face recognition while protecting personal privacy, providing strong technical support for related applications.

[0028] Embodiment 2 See also Figure 2 , a privacy-preserving face recognition system based on feature compensation, comprising: An image acquisition module is used to acquire a face image and preprocess the face image to obtain a preprocessed face image; An LBP filtering module is used to perform LBP filtering on the preprocessed face image to obtain a face image after LBP filtering; The image blur processing module is used to blur the pre-processed face image to obtain a first blurred image; blur the LBP filtered face image to obtain a second blurred image; A feature compensation extraction module is used to extract compensation features using the preprocessed face image and the first blurred image to obtain a first compensation feature group; and to extract compensation features using the LBP filtered face image and the second blurred image to obtain a second compensation feature group; A feature extraction module is used to perform feature extraction using the first blurred image and the first compensation feature group to obtain a first feature vector; perform feature extraction using the second blurred image and the second compensation feature group to obtain a second feature vector; and perform dimension addition of the first feature vector and the second feature vector to obtain a feature vector; The feature matching module is used to match the feature vector with the feature vector in the database to obtain the face recognition result.

[0029] The image acquisition module of this embodiment is responsible for acquiring face images and preprocessing these images to obtain preprocessed face images with higher quality and more suitable for subsequent processing. The LBP filtering module applies local binary pattern (LBP) filtering to the preprocessed face images. LBP filtering can capture local texture features of images, which are crucial for face recognition. The image blur processing module blur processing is intended to protect personal privacy, reduce detailed information in the image, and avoid leakage of sensitive information. The first blurred image and the second blurred image obtained are prepared for subsequent feature compensation extraction. The feature compensation extraction module aims to restore some important features lost due to blurring from the blurred image, thereby improving the accuracy of recognition. Through this step, the first compensation feature group and the second compensation feature group can be obtained. The feature extraction module aims to integrate feature information from different sources to improve the robustness and accuracy of recognition. The feature matching module calculates the similarity between feature vectors through matching algorithms (such as cosine similarity, Euclidean distance, etc.). According to the similarity result, it is judged whether the input face image matches a face in the database, thereby obtaining the face recognition result.

[0030] The blur processing module effectively reduces the detailed information in the image and protects personal privacy. The feature compensation extraction module can restore some of the lost important features from the blurred image and improve the accuracy of recognition. The feature extraction module integrates feature information from different sources and makes full use of the various feature information in the image, further improving the robustness and accuracy of recognition.

[0031] Embodiment 3 See also Figure 3 , an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the privacy protection face recognition method based on feature compensation when executing the computer program.

[0032] Embodiment 4 A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a privacy-preserving face recognition method based on feature compensation is described.

[0033] Embodiment 5 Lightweight feature extraction neural networks are a type of deep learning model designed in recent years to address the problem of limited computing power of edge devices. This type of network uses structural optimization and model compression technology to reduce the amount of computation and storage requirements while maintaining model performance as much as possible. Typical lightweight networks include MobileNet, ShuffleNet, and EfficientNet. They significantly reduce computational complexity through technologies such as deep separable convolution, grouped convolution, and automated model search, and are particularly suitable for use in face recognition tasks. In the process of facial feature extraction, neural networks are usually designed as multiple consecutive feature extraction stages, and each stage gradually extracts more advanced and abstract facial features. The network structure of MobileNet is mainly composed of depthwise convolution and pointwise convolution; the core architecture of ShuffleNet includes key components such as deep separable convolution, channel shuffle mechanism, grouped convolution, and adaptive average pooling; Specifically, the input of the feature extraction network is usually a standardized face image. The first stage extracts high-resolution low-level features, which usually retain more spatial information, such as local features such as edges and textures. As the network deepens, the resolution gradually decreases, and the number of feature maps gradually increases. These features are more inclined to global semantic information, such as the location of key points and structural features of the face. The feature extraction results of each stage can be regarded as a set of feature maps. The resolution and semantic information of these feature maps change with the level of the network: the early feature maps have high resolution and shallow semantic information, while the later feature maps have low resolution but contain more discernible deep features. Such a hierarchical design not only makes it easier for the network to capture the multi-scale features of the face in a staged manner, but also enables the model to strike a balance between computational complexity and recognition performance.

[0034] Through this staged feature extraction method, the face recognition network can effectively compress the amount of calculation, while adjusting the network size and number of parameters in each stage according to the task requirements. For example, the computational overhead can be further reduced by reducing the number of channels or resolution of the early high-resolution feature map. In addition, the characteristics of multi-stage feature extraction also allow the model to flexibly apply feature fusion technology during training and inference, thereby improving the final recognition accuracy. This design concept is widely used in lightweight models and provides technical support for real-time and efficient face recognition on edge devices.

[0035] LBP (Local Binary Pattern) is an image processing method for texture feature extraction and is widely used in the field of computer vision. The core idea of ​​LBP is to generate a binary pattern by comparing the grayscale values ​​of a pixel and its neighboring pixels. Specifically, for each pixel, several pixels around it are used as neighborhoods. If the grayscale value of the neighboring pixels is higher than that of the central pixel, it is marked as 1, and if it is lower, it is marked as 0, thus obtaining a binary number. After converting this value to decimal, it can be used to describe the local texture features of the neighborhood of the pixel. Through LBP filtering, local features of the image such as edges, corners and texture patterns can be effectively captured and converted into a texture feature map. This method has the characteristics of simple and efficient calculation, and is also highly robust to changes in illumination. It is particularly suitable for tasks such as face recognition and can enhance the expression of local texture information in images.

[0036] Existing privacy-preserving face recognition methods that use secure multi-party computing methods require multiple information transfers, which lead to low system efficiency. In addition, when the system is actually deployed, the collector often does not have a multi-party secure computing engine, which limits the actual deployment and use of the solution. Existing privacy-preserving face recognition methods do not fully utilize the powerful computing power of the server, and require high computing power on the client device, which is not convenient for actual deployment and use.

[0037] See also Figure 2 The privacy-preserving face recognition system based on feature compensation designed in this scheme adopts the client-server mode, which includes two entities, the client and the server. The client includes four modules: image acquisition module, LBP filtering module, image blur processing module, and compensation feature extraction module; the server includes two modules: feature extraction module and feature matching module. When using this system for face recognition, refer to Figure 4 and Figure 5 , the specific steps are as follows: 1. Client: 1. Collect facial images Use the camera to collect the face image X0, and then use the pre-trained MTCNN (Multi-task Cascaded Convolutional Networks) to preprocess the image, detect and obtain the preprocessed face image X1 with a size of 112×112 pixels.

[0038] MTCNN is a multi-task face detection model. The model uses three CNN cascade algorithm structures to simultaneously complete face detection and facial feature point extraction.

[0039] 2. LBP filtering Perform LBP filtering on image X1 to obtain image X2.

[0040] 3. Blurring The image X1 is blurred to obtain a blurred image M1; the image X2 is blurred to obtain a blurred image M2. The specific blurring method may be Gaussian blurring, median blurring, mean blurring or a mixed blurring of several blurring methods.

[0041] 4. Compensation feature extraction (1) Obtaining compensation image Subtract each corresponding pixel of the image X1 from the corresponding blurred image M1 to obtain the compensated image B1; subtract each corresponding pixel of the image X2 from the corresponding blurred image M2 to obtain the compensated image B2.

[0042] (2) Obtaining compensation feature group The compensated image B1 is used as the input of a lightweight feature extraction neural network S for feature extraction. Specific neural networks that can be used include MobileNet, ShuffleNet, etc. Three intermediate vectors T1, T2, and T3 are selected from the intermediate layer of the feature extraction neural network S in turn as the compensation feature group T, T = {T1, T2, T3}.

[0043] The compensated image B2 is used as the input of the lightweight feature extraction neural network S for feature extraction. Specific neural networks that can be used include MobileNet, ShuffleNet, etc. Three intermediate vectors T are selected from the intermediate layer of the feature extraction neural network S in sequence. a 、T b 、T c As the compensation feature set T', T' = {T a , T b , T c}.

[0044] 5. Data transmission The blurred image M1, the blurred image M2 and the compensation feature groups T and T' are sent to the server.

[0045] 2. Server: 1. Feature extraction The blurred image M1 is used as the input of the feature extraction neural network P for feature extraction. Specific neural networks that can be used include ArcFace, FaceNet, etc. From the intermediate layer of the feature extraction neural network P, select the intermediate vector F1 of the same dimension as the feature vector T1 in the compensation feature group T, add the corresponding dimensional components of T1 and F1 as the fused feature vector R1, and continue to propagate backward in the feature extraction neural network P. Select the intermediate vector F2 of the same dimension as the feature vector T2 in the compensation feature group T, add the corresponding dimensional components of T2 and F2 as the fused feature vector R2, and continue to propagate backward in the feature extraction neural network P. Select the intermediate vector F3 of the same dimension as the feature vector T3 in the compensation feature group T, add the corresponding dimensional components of T3 and F3 as the fused feature vector R3, and continue to propagate backward in the feature extraction neural network P to obtain the feature vector X1.

[0046] The blurred image M2 is used as the input of another feature extraction neural network Q for feature extraction. Specific neural networks that can be used include ArcFace, FaceNet, etc. The feature vector T in the compensation feature group T' is selected from the middle layer of the feature extraction neural network Q. a Intermediate vector F of the same dimension a , T a and F a The corresponding dimension components are added as the fused feature vector R a , continue to propagate backward in the feature extraction neural network Q. Select and compensate the feature vector T in the feature group T' b Intermediate vector F of the same dimension b , T b and F b The corresponding dimension components are added as the fused feature vector R b , continue to propagate backward in the feature extraction neural network Q. Select and compensate the feature vector T in the feature group T c Intermediate vector F of the same dimension c , T c and F c The corresponding dimension components are added as the fused feature vector R c , continue to propagate backward in the feature extraction neural network Q to obtain the feature vector X2.

[0047] Add the corresponding dimension components of X1 and X2 as the feature vector X.

[0048] ArcFace is an advanced face recognition technology that focuses on facial image feature extraction. ArcFace maps facial images to a high-dimensional feature space through a deep neural network. In this space, the feature distances of similar faces (faces belonging to the same person) are close, and the feature distances of different faces (faces belonging to different people) are far, thus achieving efficient and accurate face recognition.

[0049] FaceNet is a deep learning model designed to solve problems such as face recognition, verification, and clustering. The core idea of ​​FaceNet is to map face images into a compact Euclidean space so that the distance in the Euclidean space directly reflects the similarity of face images. This mapping process is achieved through a deep convolutional network, and the output of the network is a 128-dimensional vector, namely the embedding. This vector can be regarded as a compact representation of the face image, which contains enough information for recognition and has high computational efficiency. FaceNet maps face images to a 128-dimensional Euclidean space through a convolutional neural network (CNN) to achieve functions such as face verification, recognition, and clustering. During the training process, FaceNet uses an optimization objective called the triplet loss function. Triplet refers to a data unit consisting of three samples, including an anchor sample, a positive sample (of the same type as the anchor), and a negative sample (of a different type from the anchor). FaceNet learns effective embeddings by minimizing the distance between the anchor and the positive sample, while maximizing the distance between the anchor and the negative sample.

[0050] 2. Feature matching The feature vector X is matched with the feature vector in the database to obtain the face recognition result and send it to the client. The database is a database of registered face feature vectors stored on the server. When performing face recognition, the server performs face recognition by matching the feature vector with the feature vector stored in the database.

[0051] This solution extracts features from the original image and the blurred image based on the original image and the blurred image based on the filtered image of the original image, and then performs feature fusion, so that the server can efficiently and accurately complete the face recognition task under the condition of only obtaining the blurred face image and the corresponding compensation feature vector. Extracting compensation features in different frequency domains from images in different frequency domains makes the information contained in the compensation features more diverse and comprehensive, thereby improving the accuracy of face recognition. Combining images in different frequency domains (original images and LBP filtered images) allows two independent sub-networks to learn high-dimensional features in different frequency domains, and finally fuses them to obtain feature vectors for matching.

[0052] This solution does not require additional hardware support, and can generate fuzzy face images and feature information required for face recognition. The steps are simple and can be widely used in practice. This solution adopts the client-server model, which transfers the complex calculation steps to the server side while ensuring image privacy, thereby improving system efficiency.

[0053] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, read-only optical disk, optical storage, etc.) containing computer-usable program code.

[0054] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0055] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0056] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention is described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the present invention.

Claims

1. A privacy-preserving face recognition method based on feature compensation, characterized in that: The following steps are involved: Collecting a face image, preprocessing the face image to obtain a preprocessed face image; performing blur processing on the preprocessed face image to obtain a first blurred image; performing LBP filtering on the preprocessed face image to obtain an LBP filtered face image; performing blur processing on the LBP filtered face image to obtain a second blurred image; Using the preprocessed face image and the first blurred image to extract compensation features, obtaining a first compensation feature group; Using the LBP filtered face image and the second blurred image to extract compensation features, to obtain a second compensation feature group; Using the first blurred image and the first compensating feature group to perform feature extraction, obtaining a first feature vector; using the second blurred image and the second compensating feature group to perform feature extraction, obtaining a second feature vector; Add the dimensions of the first eigenvector and the second eigenvector to obtain an eigenvector; The feature vector is matched with the feature vector in the face feature vector database to obtain the face recognition result.

2. The privacy-preserving face recognition method based on feature compensation according to claim 1, characterized in that: The blurring method may be Gaussian blurring, median blurring, mean blurring or a mixed blurring of several blurring methods.

3. The privacy-preserving face recognition method based on feature compensation according to claim 1, characterized in that: The preprocessing is performed by MTCNN, and the size of the face after preprocessing is 112×112 pixels.

4. The privacy-preserving face recognition method based on feature compensation according to claim 1, characterized in that: The preprocessed face image and the first blurred image are used to extract compensation features, and the first compensation feature group is obtained as follows: Subtracting each corresponding pixel of the preprocessed face image from that of the first blurred image to obtain a first compensated image, using the first compensated image as an input of a lightweight feature extraction neural network for feature extraction, and selecting three intermediate feature vectors in an intermediate layer of the lightweight feature extraction neural network as a first compensated feature group; The face image after LBP filtering and the second blurred image are used to extract compensation features, and the second compensation feature group is obtained as follows: Subtract each corresponding pixel of the LBP filtered face image from the second blurred image to obtain a second compensated image, and use the second compensated image as the input of a lightweight feature extraction neural network for feature extraction. Three intermediate feature vectors are selected in the intermediate layer of the lightweight feature extraction neural network as the second compensated feature group.

5. The privacy-preserving face recognition method based on feature compensation according to claim 4, characterized in that: The lightweight feature extraction neural network adopts MobileNet or ShuffleNet.

6. The privacy-preserving face recognition method based on feature compensation according to claim 4, characterized in that: The process of extracting features using the first blurred image and the first compensation feature group to obtain the first feature vector is as follows: The first blurred image is used as the input of the first feature extraction neural network, a first intermediate vector having the same dimension as the first intermediate feature vector in the first compensation feature group is selected from the intermediate layer of the first feature extraction neural network, the first intermediate feature vector is added to the corresponding dimension component of the first intermediate vector as a first fused feature vector, and the first fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a second intermediate vector of the same dimension as the second intermediate feature vector in the first compensation feature group, adding the second intermediate feature vector and the corresponding dimension component of the second intermediate vector as a second fused feature vector, and the second fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a third intermediate vector of the same dimension as the third intermediate feature vector in the first compensation feature group, adding the third intermediate feature vector to the corresponding dimension component of the third intermediate vector as a third fused feature vector, and the third fused feature vector continues to propagate backward in the feature extraction neural network to obtain the first feature vector; The first feature extraction neural network adopts ArcFace or FaceNet.

7. The privacy-preserving face recognition method based on feature compensation according to claim 4, characterized in that: The process of extracting features using the second blurred image and the second compensation feature group to obtain the second feature vector is as follows: The second blurred image is used as the input of the second feature extraction neural network, a first intermediate vector having the same dimension as the first intermediate feature vector in the second compensation feature group is selected from the intermediate layer of the second feature extraction neural network, the first intermediate feature vector is added to the corresponding dimension component of the first intermediate vector as a first fused feature vector, and the first fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a second intermediate vector of the same dimension as the second intermediate feature vector in the second compensation feature group, adding the second intermediate feature vector and the corresponding dimension component of the second intermediate vector as a second fused feature vector, and the second fused feature vector continues to propagate backward in the feature extraction neural network; Selecting a third intermediate vector of the same dimension as the third intermediate feature vector in the second compensation feature group, adding the third intermediate feature vector and the corresponding dimension component of the third intermediate vector as a third fused feature vector, and the third fused feature vector continues to propagate backward in the feature extraction neural network to obtain a second feature vector; The second feature extraction neural network adopts ArcFace or FaceNet.

8. A privacy-preserving face recognition system based on feature compensation, characterized in that: include: An image acquisition module is used to acquire a face image and preprocess the face image to obtain a preprocessed face image; An LBP filtering module is used to perform LBP filtering on the preprocessed face image to obtain a face image after LBP filtering; The image blur processing module is used to blur the pre-processed face image to obtain a first blurred image; blur the LBP filtered face image to obtain a second blurred image; A feature compensation extraction module, used to extract compensation features using the preprocessed face image and the first blurred image to obtain a first compensation feature group; Using the LBP filtered face image and the second blurred image to extract compensation features, to obtain a second compensation feature group; A feature extraction module is used to perform feature extraction using the first blurred image and the first compensation feature group to obtain a first feature vector; perform feature extraction using the second blurred image and the second compensation feature group to obtain a second feature vector; and perform dimension addition of the first feature vector and the second feature vector to obtain a feature vector; The feature matching module is used to match the feature vector with the feature vector in the database to obtain the face recognition result.

9. An electronic device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a privacy-preserving face recognition method based on feature compensation as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for privacy-preserving face recognition based on feature compensation described in any one of claims 1 to 7 is implemented.

Images