Facial data privacy protection method and apparatus, privacy protection robot, and medium

Abstract

The present disclosure relates to a facial data privacy protection method and apparatus, a privacy protection robot, and a medium. The method comprises: constructing an image data set; training a human eye detection model and a facial landmark localization model separately on the basis of the image data set to obtain a trained human eye detection model and a trained facial landmark localization model; acquiring image data waiting for privacy protection, and separately inputting said image data into the trained human eye detection model and the trained facial landmark localization model to obtain a human eye detection result and a facial landmark localization result; and on the basis of the human eye detection result and the facial landmark localization result, using a preset post-processing algorithm to perform privacy protection processing on said image data to generate image data that has undergone privacy protection. Compared with the prior art, embodiments of the present disclosure can improve the accuracy and robustness of privacy protection, improve the real-time performance and efficiency of privacy protection, and ensure that sensitive information is effectively protected.

Description

Face data privacy protection method and device, privacy protection robot and medium

[0001] The present disclosure claims priority to the Chinese patent application No. 202411356868.5, filed on September 27, 2024, and entitled "Face data privacy protection method and device, privacy protection robot and medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of privacy protection, and particularly relates to a face data privacy protection method and device, a privacy protection robot and a medium. BACKGROUND

[0003] In recent years, with the rapid development of artificial intelligence and image processing technology, computer vision has been widely applied in medical diagnosis, surgical assistance and patient monitoring, etc. These technologies can help medical staff analyze the health status of patients and improve the accuracy of diagnosis. However, medical data, especially face image data containing personal information, has very high privacy protection requirements. Therefore, in the field of face image technology, accurate collection and privacy protection of face image data have always been two important technical challenges. Although traditional privacy protection measures such as data desensitization, anonymization processing and data encryption can protect patient privacy to some extent, these methods often cannot balance the practicality and privacy of data at the same time in the use of face image data.

[0004] At present, some research institutions have developed some algorithms and systems, which mainly rely on a single model to recognize and mask the facial feature parts. Commonly used models include detection models, feature point positioning models and feature extraction models. The detection model is an algorithm used in image processing to locate and identify the facial feature region in the image; the feature point positioning model is used to further accurately locate the key feature points of the face, such as eyes, nose, mouth, etc., in the detected face region, which provides accurate reference for subsequent face analysis and processing; the feature extraction model is used to extract important feature information that can reflect the state of the face from the face feature points or regions, such as texture, color, shape, etc., which is crucial for further face analysis and recognition tasks.

[0005] The above three schemes can meet the needs of face feature recognition and positioning to some extent, but they also have some problems, which make it difficult to popularize the use of face image data collection in hospitals:

[0006] 1) The recognition accuracy of the detection model will decrease significantly in complex background, different lighting conditions, angle changes and face parts being blocked (such as wearing glasses, masks or hair blocking).

[0007] 2) Feature point positioning models usually rely on pre-defined shape models such as Active Shape Model (ASM) and Active Appearance Model (AAM), and this reliance can significantly reduce the positioning accuracy of the model when dealing with face shapes that are inconsistent with the training data set, for example, when the shape of facial features is significantly different from the pre-defined shape in the model, the model may not be able to accurately match and locate these feature points.

[0008] 3) Feature extraction models aim to extract key features by simplifying data, but this process can result in the loss of some important information in the original data, which can affect the performance of the model when dealing with complex scenarios or changing conditions, making it difficult for the model to fully capture all important features in the data, thereby affecting the accuracy and effectiveness of subsequent tasks. SUMMARY

[0009] (I) Technical problems to be solved

[0010] The technical problem to be solved by the present disclosure is to solve the problem that the existing face feature part recognition and mask method has poor recognition accuracy in complex scenes, resulting in poor accuracy and robustness of subsequent privacy protection, and failing to meet the real-time and efficiency requirements of privacy protection, and failing to effectively protect sensitive information.

[0011] (II) Technical solutions

[0012] To solve the above technical problems, the embodiments of the present disclosure provide a face data privacy protection method and device, a privacy protection robot and a medium, comprising:

[0013] In a first aspect, the present disclosure provides a face data privacy protection method, comprising:

[0014] Constructing an image data set, the image data set comprising a large amount of image data containing faces;

[0015] Training an eye detection model and a feature point positioning model based on the image data set, respectively, to obtain a trained eye detection model and a trained feature point positioning model;

[0016] Obtaining image data to be protected for privacy, and inputting the image data to be protected for privacy into the trained eye detection model and the trained feature point positioning model, respectively, to obtain eye detection results and feature point positioning results;

[0017] Based on the eye detection results and the feature point positioning results, a pre-set post-processing algorithm is used to perform privacy protection processing on the image data to be protected for privacy, to generate image data protected for privacy.

[0018] In a second aspect, the present disclosure provides a face data privacy protection device, the device comprising:

[0019] a construction unit configured to construct an image data set comprising a large amount of image data containing faces;

[0020] a training unit configured to train a human eye detection model and a feature point positioning model respectively based on the image data set, to obtain a trained human eye detection model and a trained feature point positioning model;

[0021] an obtaining unit configured to obtain image data to be protected, input the image data to be protected into the trained human eye detection model and the trained feature point positioning model respectively, and obtain a human eye detection result and a feature point positioning result;

[0022] a generating unit configured to perform privacy protection processing on the image data to be protected based on the human eye detection result and the feature point positioning result by using a preset post-processing algorithm, and generate image data after privacy protection.

[0023] In a third aspect, the present disclosure provides an electronic device, in particular a privacy protection robot, comprising:

[0024] a memory;

[0025] a processor; and

[0026] a computer program;

[0027] The computer program is stored in the memory and configured to be executed by the processor to implement the face data privacy protection method of the first aspect.

[0028] In a fourth aspect, the present disclosure provides a computer storage medium having a computer program stored thereon, the computer program being executed by a processor to implement some or all steps of the face data privacy protection method provided in the first aspect of the present disclosure.

[0029] In a fifth aspect, the present disclosure further provides a computer program product comprising a computer program or instructions, which, when executed by a processor, implement the face data privacy protection method according to the first aspect.

[0030] (III) Advantages

[0031] The above technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art:

[0032] The face data privacy protection method, device, privacy protection robot and medium provided by the embodiment of the present disclosure, by constructing an image data set, the image data set includes a large number of image data containing face images, training a human eye detection model and a feature point positioning model based on the image data set respectively, obtaining a trained human eye detection model and a trained feature point positioning model, obtaining image data to be protected, inputting the image data to be protected into the trained human eye detection model and the trained feature point positioning model respectively, obtaining a human eye detection result and a feature point positioning result, based on the human eye detection result and the feature point positioning result, using a preset post-processing algorithm to perform privacy protection processing on the image data to be protected, and generating image data after privacy protection. Compared with the prior art, the embodiment of the present disclosure can improve the accuracy and robustness of privacy protection, improve the real-time performance and efficiency of privacy protection, and can ensure that sensitive information is effectively protected. The embodiment of the present disclosure fuses the output results of the human eye detection model and the feature point positioning model, accurately locates the eye coordinates and range in the image, comprehensively considers the output results of multiple models, reduces the bit error rate, enhances the accuracy and robustness of the model, maintains high accuracy under various face image data acquisition conditions, can process various types of face image data, and improves the application universality and flexibility in various scenes.

[0033] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings incorporated in the specification and constituting a part hereof illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0036] FIG. 1 is a flowchart of a face data privacy protection method provided by an embodiment of the present disclosure;

[0037] FIG. 2 is a schematic diagram of the overall architecture of the face data privacy protection method provided by an embodiment of the present disclosure;

[0038] FIG. 3 is a flowchart of a face data privacy protection method provided by another embodiment of the present disclosure;

[0039] FIG. 4 is a flowchart of a face data privacy protection method provided by another embodiment of the present disclosure;

[0040] FIG. 5 is a diagram of an eye detection model architecture provided by an embodiment of the present disclosure;

[0041] FIG. 6 is a diagram of a feature point positioning model architecture provided by an embodiment of the present disclosure;

[0042] FIG. 7 is a diagram of a workflow of a privacy protection robot and a privacy protection module provided by an embodiment of the present disclosure;

[0043] FIG. 8 is a diagram of a workflow of a preset post-processing algorithm provided by an embodiment of the present disclosure;

[0044] FIG. 9 is a diagram of a workflow of an integrated privacy protection module provided by an embodiment of the present disclosure;

[0045] FIG. 10 is a diagram of a specific process of embedding of a privacy protection module provided by an embodiment of the present disclosure;

[0046] FIG. 11 is a diagram of a comparison between a privacy protection robot and a full output mode of a privacy protection robot provided by an embodiment of the present disclosure;

[0047] FIG. 12 is a diagram of a structure of a face data privacy protection device provided by an embodiment of the present disclosure;

[0048] FIG. 13 is a diagram of a structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0050] In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some of the embodiments of the present disclosure, not all the embodiments.

[0051] All actions of obtaining information or data in the present application are carried out in compliance with the corresponding data protection regulations and policies of the place, and with the authorization of the corresponding information or data owner.

[0052] To solve the problem, the present embodiment provides a face data privacy protection method, which will be introduced below in combination with specific embodiments.

[0053] FIG. 1 is a flowchart of a face data privacy protection method according to an embodiment of the present disclosure. The method is executed by an electronic device, which can be a privacy protection device, specifically a privacy protection robot. The method can be applied to a scenario of privacy protection of face data, and can improve the accuracy and robustness of privacy protection, and improve the real-time performance and efficiency of privacy protection. It can be understood that the face data privacy protection method provided by the embodiments of the present disclosure can also be applied to other scenarios.

[0054] The face data privacy protection method shown in FIG. 1 will be introduced as follows, and the specific steps of the method are as follows:

[0055] S101, constructing an image data set, the image data set including a large amount of image data containing face images.

[0056] As shown in FIG. 2, an image data set is constructed, and the image data set includes a large amount of image data containing face images. The construction of the image data set is the basis for model training. Through systematic and standardized image data collection, labeling and data set production, the model is trained under the support of diversified and high-quality data, thereby providing a solid foundation for subsequent eye detection and feature point positioning. Through the construction of the image data set, the performance and reliability of the model can be effectively improved, and the model can accurately recognize and process face and eye feature information in actual application.

[0057] S102, training an eye detection model and a feature point positioning model based on the image data set respectively, to obtain a trained eye detection model and a trained feature point positioning model.

[0058] In this step, as shown in FIG. 2, after the construction of the image data set, the model is trained using the image data set to obtain a trained eye detection model and a trained feature point positioning model. Optionally, the image data set includes a training set, a validation set and a test set. In the training process, the labeling information of the training set will guide the model to learn how to accurately complete the eye detection and feature point positioning tasks. The data of the validation set is used to adjust the hyperparameters of the model to ensure the performance of the model outside the training set and to avoid overfitting. After the model training is completed, the performance evaluation is performed based on the test set to confirm the actual detection ability and robustness of the model.

[0059] S103, obtaining image data to be protected, inputting the image data to be protected into the trained eye detection model and the trained feature point positioning model respectively, and obtaining eye detection results and feature point positioning results.

[0060] In this step, based on the image data collected by the image acquisition device, the trained eye detection model and the trained feature point positioning model are used to output the eye detection result and the feature point positioning result.

[0061] In S104, based on the eye detection result and the feature point positioning result, a preset post-processing algorithm is used to perform privacy protection processing on the image data to be protected, and image data after privacy protection is generated.

[0062] In this step, in order to ensure that the model can run stably in diversified scenes, a post-processing algorithm needs to be written to fully utilize the output results of the eye detection model and the feature point positioning model. The main task of the post-processing algorithm is to analyze and judge the detected eyes and their feature points, so as to determine the area that needs to be protected. Specifically, the algorithm processes the eye and other facial feature point data output by the model to generate a privacy mask or other protection measures to ensure that sensitive information is effectively protected. The embodiment of the present disclosure automatically identifies and protects sensitive parts while collecting facial images, ensuring real-time completion of privacy protection. Real-time processing avoids the time consumption of a large number of image processing in the later stage, significantly improves the efficiency, and the privacy can be protected in real time, reducing the risk of information leakage.

[0063] The embodiment of the present disclosure trains the eye detection model and the feature point positioning model based on the image data set, which includes a large amount of image data containing facial images, obtains the image data to be protected, inputs the image data to be protected into the trained eye detection model and the trained feature point positioning model, obtains the eye detection result and the feature point positioning result, and uses the preset post-processing algorithm to perform privacy protection processing on the image data to be protected based on the eye detection result and the feature point positioning result, and generates image data after privacy protection. Compared with the prior art, the embodiment of the present disclosure can improve the accuracy and robustness of privacy protection, improve the real-time performance and efficiency of privacy protection, and ensure that sensitive information is effectively protected. The embodiment of the present disclosure fuses the output results of the eye detection model and the feature point positioning model, accurately locates the eye coordinates and range in the image, comprehensively considers the output results of multiple models, reduces the error rate, enhances the accuracy and robustness of the model, maintains high accuracy under various facial image data collection conditions, can process various types of facial image data, and improves the application universality and flexibility in various scenes.

[0064] FIG. 3 is a flowchart of a facial data privacy protection method provided by another embodiment of the present disclosure, as shown in FIG. 3, the method includes the following steps:

[0065] S201. Collect a large amount of image data containing face images.

[0066] In this step, a large amount of image data containing face images is collected using an image collection device. For example, a standard robot is used to collect face images in a target environment. The standard robot is equipped with a high-definition camera and a fill light, and can shoot high-resolution images under various lighting conditions. Secondly, the standard processing module built-in the standard robot performs color calibration processing on the images to ensure the quality and consistency of the data. The processed image data will be saved to the robot's memory for subsequent labeling and model training.

[0067] Optionally, the main structure and functions of the standard robot are shown in Table 1.

[0068] Table 1 Main structure and function table of standard robot

[0069] The following points need to be noted during data collection:

[0070] a. Use standard color cards for image calibration: In order to improve the performance of the image calibration algorithm in the standard processing module, an accessory (standard color card) needs to be held during each image collection, and the color card needs to be completely included in the collected image range.

[0071] b. Diversified collection posture and distance: In order to improve the stability and generality of the model, each collector needs to collect data at different distances (0.5 meters, 0.75 meters and 1 meter). At each distance, the collector needs to change different facial postures and angles (such as front, side, top view, and overhead view), and take pictures under different lighting conditions (such as bright, dark, backlight, etc.).

[0072] c. Diversified collectors: In order to improve the feature extraction performance of the model, the collectors should be composed of different age groups, genders, and vision conditions (whether wearing glasses or not).

[0073] The specific collection strategy includes shooting environment arrangement, collection angle adjustment, lighting condition control, etc. See Table 2 for specific sample data collection strategy.

[0074] Table 2 Single sample data collection strategy

[0075] S202. Label each image data, label the eye bounding box of each image, obtain an eye detection data set, and divide the eye detection data set into a training set, a validation set and a test set of the eye detection model based on a preset first proportion.

[0076] After the image data is collected, the collected image data is labeled. The main tasks of the labeling process include determining the bounding box position and feature point position of the eyes in each image. In this step, the bounding box of the eyes needs to be labeled for each image, and these labeling information will provide necessary data support for the human eye detection model training. After labeling the collected image data, the labeled image data is organized into a human eye detection dataset. The human eye detection dataset is divided into a training set, a validation set, and a test set of the human eye detection model in a ratio of 8:1:1. This division method helps the model to fully learn and adjust during the training process, while optimizing parameters during the validation phase and performing comprehensive performance evaluation during the test phase, which can also avoid the problem of model overfitting.

[0077] S203, label each of the image data, label the feature point coordinates of each image to obtain a feature point positioning dataset, and divide the feature point positioning dataset into a training set, a validation set and a test set of the feature point positioning model based on a preset second ratio.

[0078] After the image data is collected, the collected image data is labeled. The main tasks of the labeling process include determining the bounding box position and feature point position of the eyes in each image. In this step, the bounding box of the eyes needs to be labeled for each image, and these labeling information will provide necessary data support for the human eye detection model training. After labeling the collected image data, the labeled image data is organized into a human eye detection dataset. The human eye detection dataset is divided into a training set, a validation set, and a test set of the human eye detection model in a ratio of 8:1:1. This division method helps the model to fully learn and adjust during the training process, while optimizing parameters during the validation phase and performing comprehensive performance evaluation during the test phase, which can also avoid the problem of model overfitting.

[0079] Data labeling plays a key role in the present disclosure. It significantly enhances the training effect of the model and the overall performance of the system by providing supervision information, clarifying learning goals, improving data quality, and enriching data diversity. Data labeling is performed on the collected images, covering human eye detection and feature point positioning. The present disclosure uses Labelme as a data labeling tool, which is an open-source image labeling platform widely used in various computer vision tasks. Labelme supports manual labeling of image data, including rectangular bounding boxes, polygons, and control points, etc. for object regions, which are used to generate training data. In the present disclosure, rectangular bounding boxes are used to label the eyes in the image to clearly indicate the position of the eyes in the image, thereby providing training data for the human eye detection model. At the same time, key feature points are labeled on the outlines of the eyes and eyebrows, thereby providing training data for the feature point positioning model.

[0080] It should be noted that the preset first ratio and the preset second ratio can also be other ratio values, and the embodiments of the present disclosure are only for ease of explanation and illustration, and are not limited specifically.

[0081] S204, training the human eye detection model based on the human eye detection dataset, updating the model parameters of the human eye detection model, and if the human eye detection result of each image output by the updated human eye detection model has converged to the labeled eye bounding box of each image, a trained human eye detection model is obtained.

[0082] In this step, the pre-trained detection model is trained based on the human eye detection dataset to teach the model to accurately recognize the eye features of the human face in the image. Through a large number of images labeled with eye regions and their bounding box information, the model learns the features and positions of the eyes. During the training process, the model is evaluated and adjusted in real time through the validation set to ensure its robustness and accuracy on different data distributions. The process of training the human eye detection model is mainly to establish the mapping relationship between the input image and the output result through a large number of labeled training data. As shown in FIG. 5, image features are extracted through downsampling operation to generate feature maps, then the scale of the feature maps is restored through upsampling, and the feature maps are fused into the fully connected layer to finally output the human eye detection result. The downsampling operation uses convolutional neural network (CNN) to extract image features, the fully connected layer further processes and combines these features, and the output layer generates the final human eye detection result. This training process enables the model to accurately detect the eye region and its features, thereby improving the detection capability of the system.

[0083] S205, training the feature point positioning model based on the feature point positioning dataset, updating the model parameters of the feature point positioning model, and if the feature point positioning result of each image output by the updated feature point positioning model has converged to the labeled feature point coordinates of each image, a trained feature point positioning model is obtained.

[0084] In this step, based on the feature point positioning dataset, the pre-trained feature point model is trained to teach the model to accurately identify and locate the key feature points (such as eyes and eyebrows) on the face. Through a large number of facial images labeled with accurate feature point positions, the model learns the distribution and positional relationship of feature points. During the training process, the validation set is used for real-time evaluation and adjustment to ensure the robustness and accuracy of the model under different facial expressions, poses and lighting conditions. The feature point positioning model is trained to accurately locate the key feature points (eyes and eyebrows) on the face. The model uses a convolutional neural network (CNN) as the main feature extraction network, as shown in Figure 6. The CNN analyzes the input image and extracts representative feature maps. These feature maps are further processed by fully connected layers and output layers to finally predict the precise coordinates of the feature points. During the training process, a large number of image data labeled with key feature point coordinates are used. The model adjusts the weights repeatedly to gradually optimize its ability to locate feature points. Finally, the trained model can accurately identify and locate the key feature points on the face under various lighting conditions, expression changes and partial occlusion, ensuring the reliability and stability in practical applications.

[0085] S206, the trained eye detection model, the trained feature point positioning model, and the preset post-processing algorithm are integrated and processed to obtain a privacy protection module, and the privacy protection module is embedded into a standard robot to construct a privacy protection robot.

[0086] In this step, a post-processing algorithm needs to be written, which can comprehensively utilize the output results of the human eye detection model and the feature point positioning model to achieve optimized privacy protection effect. Further, the human eye detection model, the feature point positioning model and the preset post-processing algorithm are integrated into a functional complete privacy protection module. As shown in FIG. 9, first, the eye position in the image is identified by the human eye detection model, the feature point positioning model is used to further accurately position these regions, and then a mask is applied based on the preset post-processing algorithm to cover these sensitive parts. This integrated approach ensures high-precision detection and privacy protection of sensitive areas, effectively preventing unauthorized disclosure of personal information. Finally, the privacy protection module is embedded into the standard robot, enabling the robot to have integrated face image data acquisition and privacy protection functions, thereby forming a privacy protection robot. During the embedding process, the hardware and software interfaces need to be adjusted and optimized to ensure the stability and efficiency of the system. After embedding, the robot can automatically process the privacy-related areas while collecting face image data, ensuring effective protection of user privacy. As shown in FIG. 10, the specific process of embedding the privacy protection module is as follows: first, clean the interfaces in the standard robot to prepare for subsequent interface adjustment. Then, embed the complete privacy protection module into the cleaned standard robot. Finally, adjust the interfaces of the robot embedded with the privacy protection module to ensure that the system can smoothly complete the workflow, ultimately forming a functional complete privacy protection robot.

[0087] The embodiments of the present disclosure integrate the human eye detection model, the feature point positioning model and the preset post-processing algorithm into a whole privacy protection module, which can be embedded into any standard robot without additional adaptation to realize the privacy protection function. The modular design makes the system easy to achieve plug-and-play on different devices and platforms. At the same time, the unified module design reduces the workload of adaptation and development, and reduces the overall development and maintenance cost.

[0088] Optionally, the main task of the preset post-processing algorithm is to analyze and judge the detected eyes and other feature points, so as to determine the areas that need privacy protection. Specifically, the algorithm processes the eye and other facial feature point data output by the model to generate privacy masks or other protection measures to ensure effective protection of sensitive information.

[0089] FIG. 7 is a schematic diagram of the working process of the privacy protection robot and the privacy protection module. In some embodiments, as shown in FIG. 7, the preset post-processing algorithm in the privacy protection module is used to receive feature point data from the feature point positioning model and detection results from the human eye detection model, and the algorithm comprehensively considers these results to accurately determine the coordinates of the eyes in the image. According to the determined eye position coordinates, the post-processing algorithm performs mask operation on the eye area to generate a privacy-protected image.

[0090] S207. Perform a bit error rate test on the privacy protection robot using the test image to obtain a bit error rate test result.

[0091] After the privacy protection robot is completed, a comprehensive test must be performed to ensure that it can accurately identify and protect sensitive information in images, thereby effectively preventing patient privacy leakage. The test will cover the detection ability and stability of the privacy protection robot under different lighting conditions, facial expressions, and postures to verify its performance in various practical application environments. At the same time, the accuracy of the mask needs to be checked, and the corresponding bit error rate is counted to ensure that the robot can accurately mask the sensitive areas. These tests will ensure that the privacy protection robot can work reliably in actual use and effectively protect patient privacy.

[0092] In some embodiments, S207 can include but is not limited to S2071, S2072, S2073:

[0093] S2071. Adjust the output mode of the privacy protection robot to full output mode.

[0094] S2072. Perform privacy protection processing based on the full output mode using the test image to obtain first image data based on the detection result of the human eye detection model, second image data based on the positioning result of the feature point positioning model, and third image data based on the privacy protection processing based on the human eye detection result and the feature point positioning result.

[0095] S2073. Respectively count the bit error rates of the first image data, the second image data, and the third image data to obtain the bit error rate test result. If the bit error rate test result is less than a preset bit error rate threshold, it is determined that the privacy protection robot passes the bit error rate test.

[0096] In S2071-S2073, to further ensure the reliability of the privacy protection function, the output mode of the privacy protection robot is adjusted to full output mode in the bit error rate test, not only the final output image of the privacy protection robot is output, but also the result processing image of the single eye detection model and the result processing image of the single feature point positioning model are output. The test will be carried out under various scenes, different lighting conditions, facial expressions and postures to comprehensively evaluate the performance of the privacy protection robot and ensure the reliability and stability of the privacy protection module in various environments. The test personnel will use the privacy protection robot to collect facial image data and output the facial image after privacy protection processing. Then, by comparing the privacy protected facial image with the expected processing effect of the actual sensitive parts (such as eyes), the accuracy and protection effect of the mask are calculated. Fig. 11 is a comparison diagram of the privacy protection robot and the full output mode of the privacy protection robot, as shown in Fig. 11, detailed bit error rate test is carried out to verify the effect and stability of the privacy protection robot in actual application. During the test, the test case is used to evaluate the privacy protection robot, and the output mode of the privacy protection robot is temporarily adjusted to full output mode. The test not only includes the final processing image (i.e. the third image data) generated by the privacy protection robot, but also outputs the single detection privacy protection image (i.e. the first image data) processed based on the detection result of the single eye detection model, and the single feature point privacy protection image (i.e. the second image data) processed based on the positioning result of the single feature point positioning model. This adjusted full output mode is used for analysis test to ensure the reliability of the privacy protection function.

[0097] S208, obtaining image data to be protected, inputting the image data to be protected into the trained eye detection model and the trained feature point positioning model respectively to obtain eye detection results and feature point positioning results.

[0098] Specifically, the implementation process and principles of S208 and S103 are consistent, and details are not repeated here.

[0099] S209, based on the eye detection results and the feature point positioning results, using a preset post-processing algorithm to perform privacy protection processing on the image data to be protected, to generate image data after privacy protection.

[0100] Specifically, the implementation process and principles of S209 and S104 are consistent, and details are not repeated here.

[0101] The embodiment of the present disclosure collects a large amount of image data containing face images, labels each of the image data, labels the eye bounding box of each image to obtain an eye detection dataset, and divides the eye detection dataset into a training set, a validation set and a test set of the eye detection model based on a preset first proportion. Each of the image data is labeled, and the feature point coordinates of each image are labeled to obtain a feature point positioning dataset, and the feature point positioning dataset is divided into a training set, a validation set and a test set of the feature point positioning model based on a preset second proportion. Further, the eye detection model is trained based on the eye detection dataset, and the model parameters of the eye detection model are updated. If the eye detection result of each image output by the updated eye detection model has converged to the labeled eye bounding box of each image, a trained eye detection model is obtained. The feature point positioning model is trained based on the feature point positioning dataset, and the model parameters of the feature point positioning model are updated. If the feature point positioning result of each image output by the updated feature point positioning model has converged to the labeled feature point coordinates of each image, a trained feature point positioning model is obtained. Next, the trained eye detection model, the trained feature point positioning model, and a preset post-processing algorithm are integrated to obtain a privacy protection module, and the privacy protection module is embedded into a standard robot to construct a privacy protection robot. The privacy protection robot is tested for bit error rate using a test image to obtain a bit error rate test result. Then, the image data to be protected is obtained, and the image data to be protected is input into the trained eye detection model and the trained feature point positioning model to obtain eye detection results and feature point positioning results. Further, based on the eye detection results and the feature point positioning results, a preset post-processing algorithm is used to perform privacy protection processing on the image data to be protected to generate privacy-protected image data. Through the method, the accuracy and robustness of privacy protection can be improved, and the real-time performance and efficiency of privacy protection can be improved.

[0102] FIG. 4 is a flowchart of a face data privacy protection method provided by another embodiment of the present disclosure. As shown in FIG. 4, the method includes the following steps:

[0103] S301, an image dataset is constructed, and the image dataset includes a large amount of image data containing face images.

[0104] Specifically, the implementation process and principles of S301 and S101 are the same, and will not be repeated here.

[0105] S302, the eye detection model and the feature point positioning model are trained based on the image dataset respectively to obtain a trained eye detection model and a trained feature point positioning model.

[0106] Specifically, the implementation process and principles of S302 and S102 are consistent, and details are not repeated here.

[0107] S303, respectively test the trained eye detection model and the trained feature point positioning model using the respective test sets, to obtain the eye detection output results and the feature point positioning output results of the test sets.

[0108] In this step, the trained eye detection model and the trained feature point positioning model are comprehensively evaluated using the test sets in the eye detection dataset and the feature point positioning dataset. This process aims to verify the performance of the models in actual application environments, ensuring that the models can run stably and accurately in diversified scenarios. Through the evaluation of the test sets, the robustness and effectiveness of the models in real scenarios can be detected, thereby ensuring their reliability in actual applications. The embodiments of the present disclosure use images in the test sets to test the two models respectively, to obtain the eye detection output results and the feature point positioning output results of the test sets.

[0109] S304, evaluate the eye detection output results and the feature point positioning output results of the test sets using the preset test indicators, to obtain the test results of the eye detection model and the test results of the feature point positioning model.

[0110] Further, the electronic device evaluates the eye detection output results and the feature point positioning output results of the test sets using the preset test indicators, to obtain the test results of the eye detection model and the test results of the feature point positioning model. Specifically, mAP, Precision, Recall, F1, and Repeatability are used as preset test indicators, and the test results shown in Table 3 are obtained.

[0111] Table 3 Test results of eye detection model and feature point positioning model

[0112] Optionally, mAP combines precision (Precision) and recall (Recall) to comprehensively measure the detection effect of the model at different thresholds, and the higher the mAP, the better the model performance. The calculation formulas of precision and recall are as follows:

[0113] Wherein, TP is the number of correct judgments as positive samples; TN is the number of correct judgments as negative samples; FP is the number of false judgments as positive samples; and FN is the number of false judgments as negative samples.

[0114] The area under the precision-recall curve (AP) can be obtained by averaging the precision value corresponding to each recall rate. The average AP of all classes is the mAP. The calculation formulas of AP and mAP are as follows:

[0115] where R i+1 is the i+1th different recall value; P interp (R n ) is the maximum precision at this recall rate. N is the number of classes; AP i is the average precision of the i-th class.

[0116] F1 Score is a comprehensive measure that considers both precision and recall, making it particularly suitable for imbalanced datasets. It is the harmonic mean of precision and recall, balancing the accuracy and coverage of the model in predicting positive classes. The calculation formula is as follows:

[0117] Repeatability is an important indicator for evaluating the performance of feature point positioning algorithms, mainly reflecting the consistency and stability of the feature points recognized by the algorithm under different conditions (such as changes in viewing angle, changes in lighting, changes in scale, etc.). Feature points with high repeatability can still be correctly identified under different image transformations, thus providing better robustness for subsequent image processing and analysis. The calculation formula is as follows:

[0118] where P is the set of feature points in the annotation; P' is the set of feature points predicted by the feature point positioning model; |P∩P'| represents the number of matched feature point pairs; min(|P|,|P'|) represents the size of the smaller set of feature points, used to normalize the number of matched pairs.

[0119] S305、When the test results of the human eye detection model and the test results of the feature point positioning model both meet the target indicator value, it is determined that the human eye detection model and the feature point positioning model are trained.

[0120] In this step, if the test results of the human eye detection model and the test results of the feature point positioning model both meet the target indicator value, it is determined that the human eye detection model and the feature point positioning model are trained, i.e., the human eye detection model and the feature point positioning model can be used.

[0121] S306、Obtain image data to be protected, input the image data to be protected into the trained human eye detection model and the trained feature point positioning model respectively, and obtain human eye detection results and feature point positioning results.

[0122] In this step, the image data to be protected is collected by an image collection device such as a camera of a robot. Further, the image data to be protected is input into the trained human eye detection model and the trained feature point positioning model respectively to obtain human eye detection results and feature point positioning results.

[0123] S307, coordinate screening is performed on the human eye detection results to determine the top-left corner coordinates and the bottom-right corner coordinates of the first privacy protection area.

[0124] As shown in FIG. 8, after receiving the output results of the human eye detection model, i.e., the human eye detection results, the human eye detection results are subjected to coordinate screening by a preset post-processing algorithm to determine the top-left corner coordinates and the bottom-right corner coordinates of the first privacy protection area, i.e., (x min1 , y min1 , x max1 , y max1 ).

[0125] S308, coordinate screening is performed on the feature point positioning results to determine the top-left corner coordinates and the bottom-right corner coordinates of the second privacy protection area.

[0126] As shown in FIG. 8, after receiving the output results of the feature point positioning model, i.e., the feature point positioning results, the feature point positioning results are subjected to coordinate screening by a preset post-processing algorithm to determine the top-left corner coordinates and the bottom-right corner coordinates of the second privacy protection area, i.e., (x min2 , y min2 , x max2 , y max2 ).

[0127] S309, the coordinates screened from the human eye detection results are fused with the coordinates screened from the feature point positioning results to obtain the top-left corner coordinates and the bottom-right corner coordinates of the target privacy protection area.

[0128] As shown in FIG. 8, the coordinates screened from the human eye detection results are fused with the coordinates screened from the feature point positioning results to obtain the top-left corner coordinates and the bottom-right corner coordinates of the target privacy protection area, i.e., (x min , y min , x max , y max ).

[0129] S310, based on the top-left corner coordinates and the bottom-right corner coordinates of the target privacy protection area, the target privacy protection area is subjected to mask processing to generate image data after privacy protection.

[0130] Further, according to the top-left corner coordinate and the bottom-right corner coordinate of the target privacy protection region, a mask operation is performed on the target privacy protection region, a black mask is applied to the eye position of the image, and a privacy protection image is generated, which can ensure that the eye position is effectively covered, and finally the privacy protection image after the mask processing is output.

[0131] The embodiments of the present disclosure construct an image data set including a large amount of image data containing face images, train a human eye detection model and a feature point positioning model based on the image data set respectively, and obtain a trained human eye detection model and a trained feature point positioning model. Then, the trained human eye detection model and the trained feature point positioning model are tested respectively by using respective test sets to obtain human eye detection output results and feature point positioning output results of the test sets, the human eye detection output results and the feature point positioning output results of the test sets are evaluated by using a preset test index to obtain test results of the human eye detection model and the feature point positioning model, and when the test results of the human eye detection model and the feature point positioning model both satisfy a target index value, it is determined that the human eye detection model and the feature point positioning model are trained. Next, the image data to be protected is obtained, and the image data to be protected is input into the trained human eye detection model and the trained feature point positioning model to obtain human eye detection results and feature point positioning results. The human eye detection results are coordinate screened to determine the top-left corner coordinate and the bottom-right corner coordinate of the first privacy protection region, and the feature point positioning results are coordinate screened to determine the top-left corner coordinate and the bottom-right corner coordinate of the second privacy protection region. Then, the coordinates screened from the human eye detection results and the coordinates screened from the feature point positioning results are fused to obtain the top-left corner coordinate and the bottom-right corner coordinate of the target privacy protection region, and the target privacy protection region is masked based on the top-left corner coordinate and the bottom-right corner coordinate of the target privacy protection region to generate image data after privacy protection. By this method, the accuracy and robustness of privacy protection can be improved, and the real-time performance and efficiency of privacy protection can be improved.

[0132] FIG. 12 is a structural schematic diagram of a face data privacy protection apparatus provided by an embodiment of the present disclosure. The face data privacy protection apparatus can be an electronic device as described in the above embodiments, or the face data privacy protection apparatus can be a component or assembly in the electronic device. The face data privacy protection apparatus provided by an embodiment of the present disclosure can perform the processing flow provided by the face data privacy protection method embodiment. As shown in FIG. 12, the face data privacy protection apparatus 40 includes a constructing unit 41, a training unit 42, an obtaining unit 43, and a generating unit 44. The constructing unit 41 is configured to construct an image data set, where the image data set includes a large amount of image data containing faces. The training unit 42 is configured to train a human eye detection model and a feature point positioning model based on the image data set, respectively, to obtain a trained human eye detection model and a trained feature point positioning model. The obtaining unit 43 is configured to acquire image data to be protected, and input the image data to be protected into the trained human eye detection model and the trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result. The generating unit 44 is configured to perform privacy protection processing on the image data to be protected based on the human eye detection result and the feature point positioning result, using a preset post-processing algorithm, to generate image data after privacy protection.

[0133] Optionally, when the constructing unit 41 constructs the image data set, the constructing unit 41 is specifically configured to: collect a large amount of image data containing faces; label each of the image data to mark an eye bounding box of each image, to obtain a human eye detection data set, and divide the human eye detection data set into a training set, a validation set, and a test set of the human eye detection model based on a preset first proportion; label each of the image data to mark feature point coordinates of each image, to obtain a feature point positioning data set, and divide the feature point positioning data set into a training set, a validation set, and a test set of the feature point positioning model based on a preset second proportion.

[0134] Optionally, the image data set includes a human eye detection data set and a feature point positioning data set. When the training unit 42 trains the human eye detection model and the feature point positioning model based on the image data set, respectively, to obtain a trained human eye detection model and a trained feature point positioning model, the training unit 42 is specifically configured to: train the human eye detection model based on the human eye detection data set, update model parameters of the human eye detection model, and if the human eye detection result of each image output by the updated human eye detection model has converged to the eye bounding box marked for each image, obtain the trained human eye detection model; train the feature point positioning model based on the feature point positioning data set, update model parameters of the feature point positioning model, and if the feature point positioning result of each image output by the updated feature point positioning model has converged to the feature point coordinates marked for each image, obtain the trained feature point positioning model.

[0135] Optionally, after the human eye detection model and the feature point positioning model are trained based on the image data set respectively, the face data privacy protection device 40 further comprises a test unit 45. The test unit 45 is configured to test the trained human eye detection model and the trained feature point positioning model respectively by using respective test sets, to obtain human eye detection output results and feature point positioning output results of the test sets; evaluate the human eye detection output results and the feature point positioning output results of the test sets by using preset test indicators, to obtain test results of the human eye detection model and test results of the feature point positioning model; and determine that the human eye detection model and the feature point positioning model are trained when the test results of the human eye detection model and the test results of the feature point positioning model both satisfy target indicator values.

[0136] Optionally, when the generation unit 44 generates the image data after privacy protection based on the human eye detection results and the feature point positioning results by using the preset post-processing algorithm, the generation unit 44 is specifically configured to: perform coordinate screening on the human eye detection results, to determine the upper left corner coordinate and the lower right corner coordinate of the first privacy protection area; perform coordinate screening on the feature point positioning results, to determine the upper left corner coordinate and the lower right corner coordinate of the second privacy protection area; fuse the coordinates screened from the human eye detection results and the coordinates screened from the feature point positioning results, to obtain the upper left corner coordinate and the lower right corner coordinate of the target privacy protection area; and perform mask processing on the target privacy protection area based on the upper left corner coordinate and the lower right corner coordinate of the target privacy protection area, to generate the image data after privacy protection.

[0137] Optionally, after the human eye detection model and the feature point positioning model are trained based on the image data set respectively, the face data privacy protection device 40 further comprises a construction unit 46. The construction unit 46 is configured to integrate the trained human eye detection model, the trained feature point positioning model, and the preset post-processing algorithm, to obtain a privacy protection module, and embed the privacy protection module into a standard robot, to construct a privacy protection robot; and test the privacy protection robot by using a test image, to obtain a bit error rate test result.

[0138] Optionally, the construction unit 46 performs a bit error rate test on the privacy protection robot using the test image to obtain a bit error rate test result, and specifically adjusts the output mode of the privacy protection robot to a full output mode, performs privacy protection processing on the test image based on the full output mode to obtain first image data subjected to privacy protection processing based on the human eye detection model, second image data subjected to privacy protection processing based on the feature point positioning model, and third image data subjected to privacy protection processing based on the human eye detection result and the feature point positioning result, respectively counts the bit error rates of the first image data, the second image data, and the third image data to obtain the bit error rate test result, and determines that the privacy protection robot passes the bit error rate test if the bit error rate test result is less than a preset bit error rate threshold.

[0139] The face data privacy protection apparatus of the embodiment shown in FIG. 12 can be used to implement the technical solutions of the method embodiments described above, and has similar implementation principles and technical effects, which will not be described here again.

[0140] FIG. 13 is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. The electronic device can be a privacy protection device, specifically a privacy protection robot. Hereinafter, specific reference will be made to FIG. 13, which shows a structural schematic diagram of an electronic device 600 suitable for implementing the electronic device in the embodiments of the present disclosure. The electronic device shown in FIG. 13 is merely an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0141] As shown in FIG. 13, the electronic device 600 can include a processing apparatus (for example, a central processor, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded from a storage apparatus 608 to a random access memory (RAM) 603 to implement the face data privacy protection method of the embodiments as described in the present disclosure. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing apparatus 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0142] In general, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 608 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 609. The communication devices 609 can allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. While FIG. 13 illustrates the electronic device 600 with various devices, it is understood that all of the illustrated devices are not required to be implemented or possessed. More or less devices can be alternatively implemented or possessed.

[0143] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts, thereby implementing the face data privacy protection method as described above. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 609, or installed from the storage devices 608, or installed from the ROM 602. When the computer program is executed by the processing devices 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0144] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the disclosure, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0145] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0146] The computer-readable medium described above can be included in the electronic device; or exist separately from the electronic device, and not be assembled into the electronic device.

[0147] The computer-readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to:

[0148] constructing an image dataset, the image dataset comprising a large amount of image data containing face images;

[0149] training, based on the image dataset, a human eye detection model and a feature point positioning model respectively, to obtain a trained human eye detection model and a trained feature point positioning model;

[0150] obtaining image data to be protected for privacy, inputting the image data to be protected for privacy into the trained human eye detection model and the trained feature point positioning model respectively, to obtain a human eye detection result and a feature point positioning result;

[0151] based on the human eye detection result and the feature point positioning result, performing privacy protection processing on the image data to be protected for privacy by using a preset post-processing algorithm, to generate image data protected for privacy.

[0152] Optionally, when the one or more programs are executed by the electronic device, the electronic device can further perform other steps described in the above embodiments.

[0153] Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as C or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0154] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the method of the first aspect. The computer program product of the first aspect can include a computer-readable medium storing instructions that, when executed, cause one or more processors to perform the operations of the method of the first aspect.

[0155] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0156] The functions described in this document can be implemented in hardware, software, or any combination thereof. In some embodiments, the functions described in this document can be implemented in one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0157] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0158] The above description is merely the preferred embodiments of the present disclosure and the explanation of the principles of the applied technology. It should be understood by those skilled in the art that the disclosed range of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) with similar functions.

[0159] Furthermore, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated, or that they are performed sequentially, as some operations can favorably occur in parallel, concurrently or be performed with partial concurrence. Also, described in the specification are several specific implementations. However, the application should not be construed as limited to the particular implementations described. Rather, the scope of the application includes any implementation that falls within the scope of the appended claims, along with their equivalents. In some embodiments, features from certain embodiments can be combined with features from other embodiments.

[0160] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. Industrial applicability

[0161] The face data privacy protection method provided by the present disclosure can improve the accuracy and robustness of privacy protection, improve the real-time performance and efficiency of privacy protection, and ensure effective protection of sensitive information. The embodiments of the present disclosure fuse the output results of the human eye detection model and the feature point positioning model, accurately position the eye coordinates and range in the image, comprehensively consider the output results of multiple models, reduce the bit error rate, and enhance the accuracy and robustness of the model. Under various face image data acquisition conditions, the accuracy can be maintained, various types of face image data can be processed, the application universality and flexibility in various scenes are improved, and the industrial applicability is very strong.

Claims

1. A face data privacy protection method, the method comprising: constructing an image data set comprising a large amount of image data containing faces; training a human eye detection model and a feature point positioning model based on the image data set, respectively, to obtain a trained human eye detection model and a trained feature point positioning model; obtaining image data to be privacy protected, and inputting the image data to be privacy protected into the trained human eye detection model and the trained feature point positioning model, respectively, to obtain human eye detection results and feature point positioning results; based on the human eye detection results and the feature point positioning results, performing privacy protection processing on the image data to be privacy protected using a preset post-processing algorithm to generate privacy-protected image data; the privacy protection processing based on the human eye detection results and the feature point positioning results, and using the preset post-processing algorithm to generate privacy-protected image data, comprising: performing coordinate screening on the human eye detection results to determine the top-left corner coordinates and the bottom-right corner coordinates of a first privacy protection area; performing coordinate screening on the feature point positioning results to determine the top-left corner coordinates and the bottom-right corner coordinates of a second privacy protection area; fusing the coordinates screened from the human eye detection results and the coordinates screened from the feature point positioning results to obtain the top-left corner coordinates and the bottom-right corner coordinates of a target privacy protection area; based on the top-left corner coordinates and the bottom-right corner coordinates of the target privacy protection area, performing mask processing on the target privacy protection area to generate privacy-protected image data; after the training of the human eye detection model and the feature point positioning model based on the image data set, the method further comprises: integrating the trained human eye detection model, the trained feature point positioning model, and the preset post-processing algorithm to obtain a privacy protection module, and embedding the privacy protection module into a standard robot to construct a privacy protection robot; performing a bit error rate test on the privacy protection robot using a test image to obtain a bit error rate test result.

2. The method of claim 1, wherein the construction of the image data set comprises: collecting a large amount of image data containing faces; annotating each of the image data to mark the eye bounding box of each image to obtain a human eye detection data set, and dividing the human eye detection data set into a training set, a validation set, and a test set for the human eye detection model based on a preset first proportion; annotating each of the image data to mark the feature point coordinates of each image to obtain a feature point positioning data set, and dividing the feature point positioning data set into a training set, a validation set, and a test set for the feature point positioning model based on a preset second proportion.

3. The method of claim 1, wherein the image data set comprises a human eye detection data set and a feature point positioning data set; the training of the human eye detection model and the feature point positioning model based on the image data set, respectively, to obtain a trained human eye detection model and a trained feature point positioning model, comprising: training the human eye detection model based on the human eye detection dataset, updating model parameters of the human eye detection model, and obtaining a trained human eye detection model if the human eye detection result of each image output by the updated human eye detection model converges to the labeled eye bounding box of each image; training the feature point positioning model based on the feature point positioning dataset, updating model parameters of the feature point positioning model, and obtaining a trained feature point positioning model if the feature point positioning result of each image output by the updated feature point positioning model converges to the labeled feature point coordinates of each image.

4. The method of claim 1, after training the human eye detection model and the feature point positioning model based on the image dataset respectively to obtain the trained human eye detection model and the trained feature point positioning model, the method further comprises: testing the trained human eye detection model and the trained feature point positioning model respectively using respective test sets to obtain human eye detection output results and feature point positioning output results of the test sets; evaluating the human eye detection output results and the feature point positioning output results of the test sets using preset test indicators to obtain test results of the human eye detection model and test results of the feature point positioning model; determining that the human eye detection model and the feature point positioning model are trained when the test results of the human eye detection model and the test results of the feature point positioning model both satisfy target indicator values.

5. The method of claim 1, wherein the bit error rate test of the privacy protection robot using the test image comprises: adjusting the output mode of the privacy protection robot to a full output mode; performing privacy protection processing on the test image based on the full output mode to obtain first image data that is privacy-protected based on the detection result of the human eye detection model, second image data that is privacy-protected based on the positioning result of the feature point positioning model, and third image data that is privacy-protected based on the human eye detection result and the feature point positioning result; respectively counting the bit error rates of the first image data, the second image data, and the third image data to obtain bit error rate test results, and determining that the privacy protection robot passes the bit error rate test if the bit error rate test results are less than a preset bit error rate threshold.

6. A facial data privacy protection apparatus, comprising: a construction unit configured to construct an image dataset, the image dataset comprising a large amount of image data containing facial images; a training unit configured to train a human eye detection model and a feature point positioning model based on the image dataset respectively to obtain a trained human eye detection model and a trained feature point positioning model; a obtaining unit configured to obtain image data to be privacy-protected, input the image data to be privacy-protected into the trained human eye detection model and the trained feature point positioning model respectively, and obtain human eye detection results and feature point positioning results. The generation unit is configured to perform privacy protection processing on the image data to be protected by using a preset post-processing algorithm based on the human eye detection result and the feature point positioning result, and generate image data after privacy protection. When the generation unit performs privacy protection processing on the image data to be protected by using a preset post-processing algorithm based on the human eye detection result and the feature point positioning result, and generates image data after privacy protection, the generation unit is specifically configured to: perform coordinate screening on the human eye detection result to determine the top-left corner coordinate and the bottom-right corner coordinate of the first privacy protection area; perform coordinate screening on the feature point positioning result to determine the top-left corner coordinate and the bottom-right corner coordinate of the second privacy protection area; fuse the coordinates screened from the human eye detection result and the coordinates screened from the feature point positioning result to obtain the top-left corner coordinate and the bottom-right corner coordinate of the target privacy protection area; perform mask processing on the target privacy protection area based on the top-left corner coordinate and the bottom-right corner coordinate of the target privacy protection area to generate image data after privacy protection; After the human eye detection model and the feature point positioning model are trained based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model, the face data privacy protection device further includes a construction unit. The construction unit is configured to integrate the trained human eye detection model, the trained feature point positioning model, and the preset post-processing algorithm to obtain a privacy protection module, embed the privacy protection module into a standard robot, and construct a privacy protection robot. The privacy protection robot is tested for bit error rate by using a test image to obtain a bit error rate test result.

7. A privacy protection robot, comprising: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the method of any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method of any one of claims 1-5.

Images