Facial data privacy protection method, device, privacy protection robot and medium

Abstract

The present disclosure relates to a facial data privacy protection method, device, privacy protection robot and medium. An image data set is constructed, and a human eye detection model and a feature point positioning model are trained based on the image data set to obtain a trained human eye detection model and a trained feature point positioning model, and the image data to be protected are obtained. The image data to be protected are input into the trained human eye detection model and the trained feature point positioning model to obtain a human eye detection result and a feature point positioning result. Based on the human eye detection result and the feature point positioning result, a preset post-processing algorithm is used to perform privacy protection processing on the image data to be protected, and generate privacy-protected image data. Compared with the prior art, the embodiments of the present disclosure can improve the accuracy and robustness of privacy protection, improve the real-time and efficiency of privacy protection, and ensure that sensitive information is effectively protected.

Description

Technical Field

[0001] The present disclosure relates to the field of privacy protection technology, and in particular to a facial data privacy protection method, device, privacy protection robot and medium. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and image processing technology, computer vision has been widely used in medical diagnosis, surgical assistance, and patient monitoring. These technologies can help medical staff analyze the health status of patients and improve the accuracy of diagnosis. However, medical data, especially facial image data containing personal information, has extremely high privacy protection requirements. Therefore, in the field of facial image technology, accurate collection of facial image data and privacy protection have always been two important technical challenges. Although traditional privacy protection measures such as data desensitization, anonymization, and data encryption can protect patient privacy to a certain extent, these methods often find it difficult to take into account both the practicality and privacy of data when using facial image data.

[0003] At present, some scientific research institutions have developed some algorithms and systems. These systems mainly rely on a single model to identify and mask the facial features. Commonly used models include detection models, feature point positioning models and feature extraction models. The detection model is an algorithm used in image processing to locate and identify the facial feature areas in an image; the feature point positioning model is used to further accurately locate the key facial feature points in the detected face area, such as eyes, nose, mouth, etc. These feature points provide an accurate reference for subsequent facial analysis and processing; the feature extraction model is used to extract important feature information that can reflect the facial state from facial feature points or areas, such as texture, color, shape, etc. These feature information is crucial for further facial analysis and recognition tasks.

[0004] Although the above three solutions meet the needs of facial feature recognition and positioning to a certain extent, they also have some problems, which makes them difficult to be widely used when collecting facial image data in hospitals:

[0005] 1) The recognition accuracy of the detection model will drop significantly when the background is complex, the lighting conditions are different, the angle changes, and the face is partially blocked (such as wearing glasses, masks, or hair).

[0006] 2) Feature point localization models usually rely on predefined shape models, such as Active Shape Model (ASM) and Active Appearance Model (AAM). This reliance may significantly reduce the positioning accuracy of the model when dealing with facial shapes that are inconsistent with the training dataset. For example, when the shape of the facial features is significantly different from the predefined shape in the model, the model may not be able to accurately match and locate these feature points.

[0007] 3) The feature extraction model aims to extract key features by simplifying the data, but this process may lead to the loss of some important information in the original data. This information loss may affect the performance of the model when dealing with complex scenarios or changing situations, making it impossible for the model to fully capture all important features in the data, thereby affecting the accuracy and effectiveness of subsequent tasks. Summary of the invention

[0008] In order to solve the above technical problems, the present disclosure provides a facial data privacy protection method, device, privacy protection robot and medium.

[0009] In a first aspect, an embodiment of the present disclosure provides a method for protecting facial data privacy, the method comprising:

[0010] Constructing an image dataset, wherein the image dataset includes a large amount of image data including facial images;

[0011] Based on the image data set, the human eye detection model and the feature point positioning model are trained to obtain a trained human eye detection model and a trained feature point positioning model;

[0012] Acquire image data to be protected for privacy, and input the image data to be protected for privacy into a trained human eye detection model and a trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result;

[0013] Based on the human eye detection results and feature point positioning results, a preset post-processing algorithm is used to perform privacy protection processing on the privacy-protected image data to generate privacy-protected image data.

[0014] In a second aspect, an embodiment of the present disclosure provides a facial data privacy protection device, the device comprising:

[0015] A construction unit, configured to construct an image data set, wherein the image data set includes a large amount of image data including facial images;

[0016] A training unit, used to train the human eye detection model and the feature point positioning model based on the image data set to obtain a trained human eye detection model and a trained feature point positioning model;

[0017] An obtaining unit, used for obtaining image data to be protected for privacy, and inputting the image data to be protected for privacy into a trained human eye detection model and a trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result;

[0018] The generation unit is used to perform privacy protection processing on the image data to be protected by privacy using a preset post-processing algorithm based on the human eye detection result and the feature point positioning result, so as to generate the image data after privacy protection.

[0019] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0020] Memory;

[0021] Processor; and

[0022] Computer programs;

[0023] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.

[0024] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in the first aspect.

[0025] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method described in the first aspect is implemented.

[0026] The facial data privacy protection method, device, privacy protection robot and medium provided by the disclosed embodiment are as follows: by constructing an image data set, the image data set includes a large amount of image data containing facial images, training a human eye detection model and a feature point positioning model based on the image data set, obtaining a trained human eye detection model and a trained feature point positioning model, obtaining image data to be protected, inputting the image data to be protected into the trained human eye detection model and the trained feature point positioning model, obtaining a human eye detection result and a feature point positioning result, and based on the human eye detection result and the feature point positioning result, using a preset post-processing algorithm to perform privacy protection processing on the image data to be protected, generating image data after privacy protection. Compared with the prior art, the disclosed embodiment can improve the accuracy and robustness of privacy protection, improve the real-time and efficiency of privacy protection, and ensure that sensitive information is effectively protected. The disclosed embodiments fuse the output results of the human eye detection model and the feature point positioning model to accurately locate the eye coordinates and range in the image, comprehensively consider the output results of multiple models, reduce the bit error rate, enhance the accuracy and robustness of the model, maintain high accuracy under various facial image data acquisition conditions, can process various types of facial image data, and improve the breadth and flexibility of application in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 A flow chart of a facial data privacy protection method provided in an embodiment of the present disclosure;

[0030] Figure 2 A schematic diagram of the overall architecture of the facial data privacy protection method provided in an embodiment of the present disclosure;

[0031] Figure 3 A flow chart of a facial data privacy protection method provided by another embodiment of the present disclosure;

[0032] Figure 4 A flow chart of a facial data privacy protection method provided by another embodiment of the present disclosure;

[0033] Figure 5A diagram of the human eye detection model architecture provided by an embodiment of the present disclosure;

[0034] Figure 6 A diagram of the architecture of a feature point positioning model provided in an embodiment of the present disclosure;

[0035] Figure 7 A schematic diagram of the workflow of the privacy protection robot and the privacy protection module provided in the embodiments of the present disclosure;

[0036] Figure 8 A schematic diagram of the workflow of a preset post-processing algorithm provided in an embodiment of the present disclosure;

[0037] Fig. 9 A schematic diagram of the workflow of the integrated privacy protection module provided in the embodiment of the present disclosure;

[0038] Fig.10 A schematic diagram of a specific process of embedding a privacy protection module provided in an embodiment of the present disclosure;

[0039] Fig.11 A schematic diagram comparing the privacy protection robot provided in the embodiment of the present disclosure and the full output mode of the privacy protection robot;

[0040] Fig.12 A schematic diagram of the structure of a facial data privacy protection device provided in an embodiment of the present disclosure;

[0041] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0043] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0044] All actions to obtain information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the information or data is located, and with the authorization of the owner of the relevant information or data.

[0045] To address this problem, the present disclosure provides a facial data privacy protection method, which is described below in conjunction with a specific embodiment.

[0046] Figure 1This is a flow chart of the facial data privacy protection method provided in the embodiment of the present disclosure. The execution subject of the method is an electronic device, which can be a privacy protection device, specifically a privacy protection robot. The method can be applied to the scenario of privacy protection of facial data, and can improve the accuracy and robustness of privacy protection, and improve the real-time and efficiency of privacy protection. It can be understood that the facial data privacy protection method provided in the embodiment of the present disclosure can also be applied in other scenarios.

[0047] Below Figure 1 The facial data privacy protection method shown in the figure is introduced, and the specific steps of the method are as follows:

[0048] S101. Construct an image dataset, where the image dataset includes a large amount of image data containing facial images.

[0049] like Figure 2 As shown, an image dataset is constructed, and the image dataset includes a large amount of image data containing facial images. Constructing an image dataset is the basis for model training. Through systematic and standardized image data collection, annotation, and dataset production, the model is trained with the support of diverse and high-quality data, thereby providing a solid foundation for subsequent human eye detection and feature point positioning. By constructing an image dataset, the performance and reliability of the model can be effectively improved, ensuring that the model can accurately identify and process facial and human eye feature information in practical applications.

[0050] S102: training a human eye detection model and a feature point positioning model based on the image data set to obtain a trained human eye detection model and a trained feature point positioning model.

[0051] In this step, if Figure 2 As shown in the figure, after constructing the image dataset, the image dataset is used for model training to obtain a trained human eye detection model and a trained feature point positioning model. Optionally, the image dataset includes a training set, a validation set, and a test set. During the training process, the annotation information of the training set will guide the model to learn how to accurately complete the tasks of human eye detection and feature point positioning. The data of the validation set is used to adjust the hyperparameters of the model to ensure the performance of the model outside the training set and avoid overfitting. After the model training is completed, performance evaluation is performed based on the test set to confirm the actual detection capability and robustness of the model.

[0052] S103 , obtaining image data to be protected for privacy, and inputting the image data to be protected for privacy into a trained eye detection model and a trained feature point positioning model, respectively, to obtain eye detection results and feature point positioning results.

[0053] In this step, based on the image data to be protected based on the image acquisition device, the trained eye detection model and the trained feature point positioning model are used to output the eye detection results and the feature point positioning results.

[0054] S104: Based on the human eye detection result and the feature point positioning result, a preset post-processing algorithm is used to perform privacy protection processing on the image data to be privacy protected, to generate privacy-protected image data.

[0055] In this step, in order to ensure that the model can run stably in a variety of scenarios, a post-processing algorithm needs to be written to make full use of the output results of the human eye detection model and the feature point positioning model. The main task of the post-processing algorithm is to analyze and judge the detected eyes and their feature points, so as to determine the areas that need privacy protection. Specifically, the algorithm will process the eye and other facial feature point data output by the model, generate a privacy mask or other protection measures to ensure that sensitive information is effectively protected. The disclosed embodiment automatically identifies and protects sensitive parts while collecting facial images, ensuring that privacy protection is completed in real time. Real-time processing avoids the time consumption of a large amount of image processing in the later stage, significantly improves efficiency, privacy can be protected in real time, and the risk of information leakage is reduced.

[0056] The disclosed embodiment constructs an image data set, which includes a large amount of image data containing facial images, trains a human eye detection model and a feature point positioning model based on the image data set, obtains a trained human eye detection model and a trained feature point positioning model, obtains image data to be protected for privacy, inputs the image data to be protected for privacy into the trained human eye detection model and the trained feature point positioning model, obtains human eye detection results and feature point positioning results, and uses a preset post-processing algorithm to perform privacy protection processing on the image data to be protected for privacy based on the human eye detection results and the feature point positioning results, and generates image data after privacy protection. Compared with the prior art, the disclosed embodiment can improve the accuracy and robustness of privacy protection, improve the real-time and efficiency of privacy protection, and ensure that sensitive information is effectively protected. The disclosed embodiments fuse the output results of the human eye detection model and the feature point positioning model to accurately locate the eye coordinates and range in the image, comprehensively consider the output results of multiple models, reduce the bit error rate, enhance the accuracy and robustness of the model, maintain high accuracy under various facial image data acquisition conditions, can process various types of facial image data, and improve the breadth and flexibility of application in various scenarios.

[0057] Figure 3 A flow chart of a facial data privacy protection method provided by another embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the method includes the following steps:

[0058] S201, collecting a large amount of image data containing facial images.

[0059] In this step, an image acquisition device is used to collect a large amount of image data containing facial images. For example, a standard robot is used to collect facial images in the target environment. The standard robot is equipped with a high-definition camera and a fill light, which can capture high-resolution images under various lighting conditions. Secondly, the standard processing module built into the standard robot performs color calibration on the image to ensure the quality and consistency of the data. The processed image data will be saved in the robot's memory for subsequent annotation and model training.

[0060] Optionally, the main structure and functions of the standard robot are shown in Table 1.

[0061] Table 1 Main structure and function of standard robot

[0062]

[0063] The following points should be noted during data collection:

[0064] a. Use a standard colorimetric card for image calibration: To improve the performance of the image calibration algorithm in the standard processing module, you need to hold the accessory (standard colorimetric card) during each image acquisition and ensure that the colorimetric card is completely within the captured image range.

[0065] b. Diverse collection postures and distances: To improve the stability and versatility of the model, each collector needs to collect data at different distances (0.5m, 0.75m and 1m). At each distance, the collector needs to change different facial postures and angles (such as front, side, looking down, looking up, etc.), and shoot under different lighting conditions (such as bright, dark, backlit, etc.).

[0066] c. Diverse collection staff: To improve the feature extraction performance of the model, the collection staff should be composed of people of different ages, genders, and vision conditions (whether they wear glasses).

[0067] The specific collection strategies include shooting environment arrangement, collection angle adjustment, lighting condition control, etc. See Table 2 for single sample data collection strategy.

[0068] Table 2 Single sample data collection strategy

[0069]

[0070] S202, annotate each of the image data, annotate the eye bounding box of each image, obtain a human eye detection data set, and divide the human eye detection data set into a training set, a verification set, and a test set of the human eye detection model based on a preset first ratio.

[0071] After the image data is collected, the collected image data is annotated. The main tasks of the annotation process include determining the bounding box position and feature point position of the eye in each image. In this step, it is necessary to mark the bounding box of the eye for each image, and these annotation information will provide necessary data support for the training of the human eye detection model. After the collected image data are annotated, the annotated image data are organized into a human eye detection dataset. The human eye detection dataset is divided into the training set, validation set, and test set of the human eye detection model in a ratio of 8:1:1. This division method helps the model to fully learn and adjust during the training process, while performing parameter optimization in the verification stage and performing a comprehensive performance evaluation in the test stage, which can also avoid the problem of model overfitting.

[0072] S203, annotating each of the image data, annotating the feature point coordinates of each image, obtaining a feature point positioning data set, and dividing the feature point positioning data set into a training set, a validation set, and a test set of the feature point positioning model based on a preset second ratio.

[0073] After the image data is collected, the collected image data is annotated. The main tasks of the annotation process include determining the bounding box position and feature point position of the eyes in each image. In this step, the coordinates of key feature points such as eyes and eyebrows need to be annotated for each image. These annotation information will provide necessary data support for the training of the feature point positioning model. After the collected image data is annotated, the annotated image data is organized into a feature point positioning data set. The feature point positioning data set is divided into a training set, a validation set, and a test set of the feature point positioning model in a ratio of 6:2:2. This division method helps to perform effective parameter adjustment and performance evaluation during the model training process, while avoiding the problem of model overfitting. It aims to balance the amount of training data to achieve the stability and accuracy of the model under various data distributions.

[0074] Data annotation plays a key role in this disclosure. It significantly enhances the training effect of the model and the overall performance of the system by providing supervision information, clarifying learning objectives, improving data quality, and enriching data diversity. Data annotation is performed on the collected images, covering two aspects: human eye detection and feature point positioning. This disclosure uses Labelme as a data annotation tool, which is an open source image annotation platform that is widely used in various computer vision tasks. Labelme supports manual annotation of image data, including object areas of types such as rectangular borders, polygons, and control points, for generating training data. In this disclosure, rectangular borders are used to annotate the eyes in the image to clarify the position of the eyes in the image, thereby providing training data for the human eye detection model. At the same time, key feature points are annotated on the eye and eyebrow contours to provide training data for the feature point positioning model.

[0075] It should be noted that the preset first ratio and the preset second ratio may also be other ratio values, and the embodiments of the present disclosure are only for the convenience of explanation and are not specifically limited.

[0076] S204. Train the human eye detection model based on the human eye detection data set, and update the model parameters of the human eye detection model. If the human eye detection result of each image output by the updated human eye detection model has converged to the marked eye boundary box of each image, a trained human eye detection model is obtained.

[0077] In this step, the pre-trained detection model is trained based on the eye detection dataset to teach the model to accurately identify the eye features of faces in images. The model is trained to learn the features and positions of the eyes through a large number of images with eye regions annotated and their bounding box information. During the training process, the model is evaluated and adjusted in real time through the validation set to ensure its robustness and accuracy on different data distributions. The process of eye detection model training mainly uses a large amount of annotated training data to establish a mapping relationship between the input image and the output result. Figure 5 As shown in the figure, the image features are extracted through downsampling operation to generate feature maps, and then the scale of the feature maps is restored through upsampling, and the feature maps are fused into the fully connected layer to finally output the human eye detection results. The downsampling operation uses a convolutional neural network (CNN) to extract image features, and the fully connected layer further processes and combines these features, while the output layer generates the final human eye detection results. This training process enables the model to accurately detect the eye area and its features, thereby improving the detection ability of the system.

[0078] S205. Train the feature point positioning model based on the feature point positioning data set, and update the model parameters of the feature point positioning model. If the feature point positioning results of each image output by the updated feature point positioning model have converged to the marked feature point coordinates of each image, a trained feature point positioning model is obtained.

[0079] In this step, the pre-trained feature point model is trained based on the feature point location dataset to teach the model to accurately identify and locate key feature points on the face (such as eyes and eyebrows). The model learns the distribution and positional relationship of feature points through a large number of facial images with precise feature point locations annotated. During the training process, the validation set is used for real-time evaluation and adjustment to ensure the robustness and accuracy of the model under different facial expressions, postures, and lighting conditions. The feature point location model training aims to accurately locate the key feature points (eyes and eyebrows) on the face. The model uses a convolutional neural network (CNN) as the main feature extraction network, such as Figure 6 As shown in the figure, CNN extracts representative feature maps by analyzing the input image. These feature maps are further processed by the fully connected layer and the output layer to finally predict the precise coordinates of the feature points. During the training process, a large amount of image data with the coordinates of key feature points annotated is used, and the model gradually optimizes its ability to locate feature points by repeatedly adjusting the weights. Ultimately, the trained model can still accurately identify and locate the key feature points of the face under various lighting conditions, expression changes, and partial occlusion, ensuring reliability and stability in practical applications.

[0080] S206: Integrate the trained human eye detection model, the trained feature point positioning model, and the preset post-processing algorithm to obtain a privacy protection module, and embed the privacy protection module into a standard robot to construct a privacy protection robot.

[0081] In this step, you need to write a post-processing algorithm that can comprehensively utilize the output results of the human eye detection model and the feature point positioning model to achieve an optimized privacy protection effect. Furthermore, the human eye detection model, the feature point positioning model and the preset post-processing algorithm are integrated into a fully functional privacy protection module. Fig. 9As shown in the figure, the eye position in the image is first identified through the human eye detection model, and these areas are further accurately located using the feature point positioning model. Then, based on the preset post-processing algorithm, a mask is applied to cover these sensitive areas. This integration method ensures high-precision detection and privacy protection of sensitive areas, and can effectively prevent unauthorized leakage of personal information. Finally, the privacy protection module is embedded in the standard robot, so that the robot has integrated facial image data acquisition and privacy protection functions, thus forming a privacy protection robot. During the embedding process, the hardware and software interfaces need to be adjusted and optimized to ensure the stability and operation efficiency of the system. After the embedding is completed, the robot can automatically process the privacy-related areas while collecting facial image data to ensure that user privacy is effectively protected. As shown in the figure Fig.10 As shown in Figure 1, the specific process of embedding the privacy protection module is as follows: First, clean up the interface in the standard robot to prepare for the subsequent interface adjustment. Then, embed the complete privacy protection module into the cleaned standard robot. Finally, adjust the robot interface with the embedded privacy protection module to ensure that the system can successfully complete the workflow, and finally form a fully functional privacy protection robot.

[0082] The disclosed embodiment integrates the human eye detection model, feature point positioning model and preset post-processing algorithm into an integrated privacy protection module, which can be embedded in any robot that meets the standards and can implement the privacy protection function without additional adaptation. The modular design allows the system to be easily plug-and-play on different devices and platforms. At the same time, the unified module design reduces the workload of adaptation and development, and reduces the overall development and maintenance costs.

[0083] Optionally, the main task of the preset post-processing algorithm is to analyze and judge the detected eyes and other feature points to determine the areas that need privacy protection. Specifically, the algorithm will process the eye and other facial feature point data output by the model to generate a privacy mask or other protection measures to ensure that sensitive information is effectively protected.

[0084] Figure 7 Schematic diagram of the work flow of the privacy protection robot and the privacy protection module. Figure 7 As shown, the preset post-processing algorithm in the privacy protection module is used to receive the feature point data from the feature point positioning model and the detection results of the human eye detection model. The algorithm comprehensively considers these results to accurately determine the position coordinates of the eyes in the image. According to the determined eye position coordinates, the post-processing algorithm performs a mask operation on the eye area to generate a privacy-protected image.

[0085] S207: Perform a bit error rate test on the privacy protection robot using the test image to obtain a bit error rate test result.

[0086] After the privacy protection robot is completed, it must be fully tested to ensure that it can accurately identify and protect sensitive information in the image, thereby effectively preventing the leakage of patient privacy. The test will cover the detection ability and stability of the privacy protection robot under different lighting conditions, facial expressions and postures to verify its performance in various practical application environments. At the same time, it is also necessary to check the accuracy of the mask and calculate the corresponding bit error rate to ensure that the robot can accurately mask sensitive areas. These tests will ensure that the privacy protection robot can work reliably in actual use and effectively protect patient privacy.

[0087] In some embodiments, S207 may include but is not limited to S2071, S2072, S2073:

[0088] S2071. Adjust the output mode of the privacy protection robot to full output mode.

[0089] S2072. Perform privacy protection processing on the test image based on the full output mode to obtain first image data that is processed for privacy protection based on the detection result of the human eye detection model, second image data that is processed for privacy protection based on the positioning result of the feature point positioning model, and third image data that is processed for privacy protection based on the human eye detection result and the feature point positioning result.

[0090] S2073. Count the bit error rates of the first image data, the second image data, and the third image data respectively to obtain a bit error rate test result. If the bit error rate test result is less than a preset bit error rate threshold, it is determined that the privacy protection robot has passed the bit error rate test.

[0091] In S2071~S2073, in order to further ensure the reliability of the privacy protection function, the output mode of the privacy protection robot will be adjusted to the full output mode in the bit error rate test, which will not only output the final output image of the privacy protection robot, but also the result processing image of the single human eye detection model and the result processing image of the single feature point positioning model. The test will be conducted in a variety of scenarios, under different lighting conditions, facial expressions and postures to comprehensively evaluate the performance of the privacy protection robot and ensure the reliability and stability of the privacy protection module in various environments. The tester will use the privacy protection robot to collect facial image data and output the facial image after privacy protection processing. Then, by comparing the expected processing effect of the facial image after privacy protection with the actual sensitive parts (such as eyes), the accuracy and protection effect of the mask are calculated. Fig.11 This is a schematic diagram comparing the privacy protection robot and the privacy protection robot in full output mode. Fig.11As shown, a detailed bit error rate test is performed to verify the effect and stability of the privacy protection robot in practical applications. During the test, the privacy protection robot is evaluated using test cases, and the output mode of the privacy protection robot is temporarily adjusted to the full output mode. The test not only includes the final processed image (i.e., the third image data) generated by the privacy protection robot, but also outputs a single detection privacy protection image (i.e., the first image data) processed based on the detection result of a single human eye detection model, and a single feature point privacy protection image (i.e., the second image data) processed based on the positioning result of a single feature point positioning model. This adjusted full output mode is used for analytical testing to ensure the reliability of the privacy protection function.

[0092] S208 , obtaining image data to be protected for privacy, and inputting the image data to be protected for privacy into a trained eye detection model and a trained feature point positioning model, respectively, to obtain eye detection results and feature point positioning results.

[0093] Specifically, the implementation process and principle of S208 and S103 are the same, and will not be repeated here.

[0094] S209: Based on the human eye detection result and the feature point positioning result, a preset post-processing algorithm is used to perform privacy protection processing on the image data to be privacy protected, to generate privacy-protected image data.

[0095] Specifically, the implementation process and principle of S209 and S104 are the same, and will not be repeated here.

[0096] The disclosed embodiment collects a large amount of image data containing facial images, annotates each of the image data, annotates the eye bounding box of each image, obtains a human eye detection data set, and divides the human eye detection data set into a training set, a validation set, and a test set of the human eye detection model based on a preset first ratio; annotates each of the image data, annotates the feature point coordinates of each image, obtains a feature point positioning data set, and divides the feature point positioning data set into a training set, a validation set, and a test set of the feature point positioning model based on a preset second ratio. Further, the eye detection model is trained based on the eye detection data set, and the model parameters of the eye detection model are updated. If the eye detection results of each image output by the updated eye detection model have converged to the eye boundary box marked in each image, a trained eye detection model is obtained; the feature point positioning model is trained based on the feature point positioning data set, and the model parameters of the feature point positioning model are updated. If the feature point positioning results of each image output by the updated feature point positioning model have converged to the feature point coordinates marked in each image, a trained feature point positioning model is obtained. Next, the trained eye detection model, the trained feature point positioning model, and the preset post-processing algorithm are integrated to obtain a privacy protection module, and the privacy protection module is embedded in a standard robot to construct a privacy protection robot, and the error rate test of the privacy protection robot is performed using a test image to obtain an error rate test result. Then, the image data to be protected is obtained, and the image data to be protected is respectively input into the trained eye detection model and the trained feature point positioning model to obtain the eye detection result and the feature point positioning result. Then, based on the human eye detection results and feature point positioning results, a preset post-processing algorithm is used to perform privacy protection processing on the privacy-protected image data to generate privacy-protected image data. This method can improve the accuracy and robustness of privacy protection, and improve the real-time and efficiency of privacy protection.

[0097] Figure 4 A flow chart of a facial data privacy protection method provided by another embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the method includes the following steps:

[0098] S301: Construct an image dataset, where the image dataset includes a large amount of image data containing facial images.

[0099] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0100] S302: training a human eye detection model and a feature point positioning model based on the image data set to obtain a trained human eye detection model and a trained feature point positioning model.

[0101] Specifically, the implementation process and principle of S302 and S102 are the same, and will not be repeated here.

[0102] S303 , using respective test sets to test the trained human eye detection model and the trained feature point positioning model respectively, to obtain the human eye detection output result and the feature point positioning output result of the test set.

[0103] In this step, the test set in the eye detection dataset and the feature point positioning dataset is used to conduct a comprehensive performance evaluation on the trained eye detection model and the trained feature point positioning model. This process is intended to verify the performance of the model in the actual application environment and ensure that the model can run stably and accurately in a variety of scenarios. By evaluating the test set, the robustness and effectiveness of the model in real scenes can be tested, thereby ensuring its reliability in practical applications. The disclosed embodiment uses the images in the test set to test the two models respectively, and obtains the eye detection output results and feature point positioning output results of the test set.

[0104] S304: Evaluate the human eye detection output result and the feature point positioning output result of the test set using preset test indicators to obtain the test results of the human eye detection model and the test results of the feature point positioning model.

[0105] Further, the electronic device uses preset test indicators to evaluate the human eye detection output results and feature point positioning output results of the test set to obtain the test results of the human eye detection model and the test results of the feature point positioning model. Specifically, mAP, Precision, Recall, F1 and Repeatability are used as preset test indicators to obtain the test results shown in Table 3.

[0106] Table 3 Test results of human eye detection model and feature point positioning model

[0107]

[0108] Optionally, mAP combines precision and recall to comprehensively measure the detection effect of the model at different thresholds. The higher the mAP, the better the model performance. The calculation formulas for precision and recall are as follows:

[0109]

[0110]

[0111] Among them, TP is the number of samples correctly judged as positive; TN is the number of samples correctly judged as negative; FP is the number of samples incorrectly judged as positive; FN is the number of samples incorrectly judged as negative.

[0112] By calculating the average of the precision values ​​corresponding to each recall rate, we can get the area under the precision and recall curves (AP). The average AP value of all categories is mAP. The calculation formulas for AP and mAP are as follows:

[0113]

[0114]

[0115] in, is the i+1th different recall value; is the maximum precision at this recall rate. N is the number of categories; is the average precision of the i-th category.

[0116] F1 Score is a comprehensive measure of precision and recall, and is particularly suitable for classification problems with unbalanced datasets. It is the harmonic mean of precision and recall, and is used to weigh the accuracy and coverage of the model when predicting the positive class. Its calculation formula is as follows:

[0117] .

[0118] Feature point repeatability is an important indicator to measure the performance of feature point positioning algorithms. It mainly reflects the consistency and stability of feature points identified by the algorithm under different conditions (such as changes in viewing angle, illumination, scale, etc.). Feature points with high repeatability can still be correctly identified under different image transformations, thus having good robustness for subsequent image processing and analysis. The calculation formula is as follows:

[0119]

[0120] Among them, P is the set of feature points in the annotation; is the set of feature points predicted by the feature point positioning model; Indicates the number of feature point pairs that are successfully matched; Indicates the size of the set with a smaller number of feature points, used to normalize the number of matching pairs.

[0121] S305: When the test results of the human eye detection model and the test results of the feature point positioning model both meet the target index values, it is determined that the training of the human eye detection model and the feature point positioning model is completed.

[0122] In this step, if the test results of the human eye detection model and the test results of the feature point positioning model both meet the target index values, it is determined that the training of the human eye detection model and the feature point positioning model is completed, that is, the human eye detection model and the feature point positioning model can be used.

[0123] S306: Acquire image data to be protected for privacy, and input the image data to be protected for privacy into a trained human eye detection model and a trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result.

[0124] In this step, the image data to be protected is collected by an image acquisition device such as a robot camera. Further, the image data to be protected is input into a trained eye detection model and a trained feature point positioning model to obtain eye detection results and feature point positioning results.

[0125] S307 , performing coordinate screening on the human eye detection result to determine the coordinates of the upper left corner and the lower right corner of the first privacy protection area.

[0126] like Figure 8 As shown, after receiving the output result of the human eye detection model, that is, the human eye detection result, the human eye detection result is coordinate-screened by a preset post-processing algorithm to determine the upper left corner coordinates and the lower right corner coordinates of the maximized first privacy protection area, that is, (xmin1, ymin1, xmax1, ymax1).

[0127] S308 : Perform coordinate screening on the feature point positioning result to determine the upper left corner coordinates and the lower right corner coordinates of the second privacy protection area.

[0128] like Figure 8 As shown, after receiving the output result of the feature point positioning model, that is, the feature point positioning result, the feature point positioning result is coordinate-screened by a preset post-processing algorithm to determine the upper left corner coordinates and lower right corner coordinates of the maximized second privacy protection area, that is, (xmin2, ymin2, xmax2, ymax2).

[0129] S309: Merge the coordinates selected by the human eye detection result and the coordinates selected by the feature point positioning result to obtain the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area.

[0130] like Figure 8 As shown, the coordinates filtered out by the human eye detection results are fused with the coordinates filtered out by the feature point positioning results to obtain the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area, that is, (xmin, ymin, xmax, ymax).

[0131] S310 , performing mask processing on the target privacy protection area based on the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area to generate privacy-protected image data.

[0132] Further, according to the upper-left corner coordinates and the lower-right corner coordinates of the target privacy protection area, a mask operation is performed on the target privacy protection area, and a black mask is applied to the target privacy protection area, i.e., the eye position of the image, to generate a privacy protection image, which can ensure that the eye position is effectively covered, and finally output the privacy protection image after mask processing.

[0133] The disclosed embodiment constructs an image data set, wherein the image data set includes a large amount of image data containing facial images, and trains a human eye detection model and a feature point positioning model based on the image data set to obtain a trained human eye detection model and a trained feature point positioning model. Then, the trained human eye detection model and the trained feature point positioning model are tested using their respective test sets to obtain the human eye detection output result and the feature point positioning output result of the test set, and the human eye detection output result and the feature point positioning output result of the test set are evaluated using preset test indicators to obtain the test results of the human eye detection model and the test results of the feature point positioning model. When the test results of the human eye detection model and the test results of the feature point positioning model both meet the target indicator values, it is determined that the training of the human eye detection model and the feature point positioning model is completed. Next, obtain the image data to be protected, input the image data to be protected into the trained human eye detection model and the trained feature point positioning model respectively, obtain the human eye detection result and the feature point positioning result, perform coordinate screening on the human eye detection result, determine the upper left corner coordinate and the lower right corner coordinate of the first privacy protection area, perform coordinate screening on the feature point positioning result, determine the upper left corner coordinate and the lower right corner coordinate of the second privacy protection area. Then, merge the coordinates screened by the human eye detection result with the coordinates screened by the feature point positioning result to obtain the upper left corner coordinate and the lower right corner coordinate of the target privacy protection area, perform mask processing on the target privacy protection area based on the upper left corner coordinate and the lower right corner coordinate of the target privacy protection area, and generate the image data after privacy protection. Through this method, the accuracy and robustness of privacy protection can be improved, and the real-time and efficiency of privacy protection can be improved.

[0134] Fig.12 The structure diagram of the facial data privacy protection device provided in the embodiment of the present disclosure is as follows. The facial data privacy protection device may be the electronic device as described in the above embodiment, or the facial data privacy protection device may be a component or assembly in the electronic device. The facial data privacy protection device provided in the embodiment of the present disclosure may execute the processing flow provided in the facial data privacy protection method embodiment, such as Fig.12As shown, the facial data privacy protection device 40 includes: a construction unit 41, a training unit 42, an obtaining unit 43, and a generating unit 44; wherein the construction unit 41 is used to construct an image data set, and the image data set includes a large amount of image data containing facial images; the training unit 42 is used to train the human eye detection model and the feature point positioning model respectively based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model; the obtaining unit 43 is used to obtain the image data to be privacy protected, and input the image data to be privacy protected into the trained human eye detection model and the trained feature point positioning model respectively to obtain the human eye detection result and the feature point positioning result; the generating unit 44 is used to perform privacy protection processing on the image data to be privacy protected based on the human eye detection result and the feature point positioning result by using a preset post-processing algorithm to generate the image data after privacy protection.

[0135] Optionally, when the construction unit 41 constructs an image dataset, it is specifically used to: collect a large amount of image data containing facial images; annotate each of the image data, annotate the eye bounding box of each image, obtain a human eye detection dataset, and divide the human eye detection dataset into a training set, a verification set and a test set of the human eye detection model based on a preset first ratio; annotate each of the image data, annotate the feature point coordinates of each image, obtain a feature point positioning dataset, and divide the feature point positioning dataset into a training set, a verification set and a test set of the feature point positioning model based on a preset second ratio.

[0136] Optionally, the image dataset includes a human eye detection dataset and a feature point positioning dataset; the training unit 42 trains the human eye detection model and the feature point positioning model based on the image dataset respectively to obtain a trained human eye detection model and a trained feature point positioning model, and is specifically used for: training the human eye detection model based on the human eye detection dataset, and updating the model parameters of the human eye detection model, if the human eye detection result of each image output by the updated human eye detection model has converged to the annotated eye boundary box of each image, then the trained human eye detection model is obtained; training the feature point positioning model based on the feature point positioning dataset, and updating the model parameters of the feature point positioning model, if the feature point positioning result of each image output by the updated feature point positioning model has converged to the annotated feature point coordinates of each image, then the trained feature point positioning model is obtained.

[0137] Optionally, after the human eye detection model and the feature point positioning model are trained based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model, the facial data privacy protection device 40 also includes: a testing unit 45; the testing unit 45 is used to test the trained human eye detection model and the trained feature point positioning model using their respective test sets to obtain the human eye detection output results and the feature point positioning output results of the test sets; the human eye detection output results and the feature point positioning output results of the test set are evaluated using preset test indicators to obtain the test results of the human eye detection model and the test results of the feature point positioning model; when the test results of the human eye detection model and the test results of the feature point positioning model both meet the target indicator values, it is determined that the training of the human eye detection model and the feature point positioning model is completed.

[0138] Optionally, the generation unit 44 uses a preset post-processing algorithm to perform privacy protection processing on the image data to be protected based on the human eye detection result and the feature point positioning result, and when generating the image data after privacy protection, it is specifically used to: perform coordinate filtering on the human eye detection result to determine the upper left corner coordinates and the lower right corner coordinates of the first privacy protection area; perform coordinate filtering on the feature point positioning result to determine the upper left corner coordinates and the lower right corner coordinates of the second privacy protection area; merge the coordinates filtered out by the human eye detection result with the coordinates filtered out by the feature point positioning result to obtain the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area; perform mask processing on the target privacy protection area based on the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area to generate the image data after privacy protection.

[0139] Optionally, after the human eye detection model and the feature point positioning model are trained respectively based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model, the facial data privacy protection device 40 also includes: a construction unit 46; the construction unit 46 is used to integrate the trained human eye detection model, the trained feature point positioning model, and a preset post-processing algorithm to obtain a privacy protection module, and embed the privacy protection module into a standard robot to construct a privacy protection robot; use a test image to perform a bit error rate test on the privacy protection robot to obtain a bit error rate test result.

[0140] Optionally, the construction unit 46 uses the test image to perform a bit error rate test on the privacy protection robot, and when the bit error rate test result is obtained, it is specifically used to: adjust the output mode of the privacy protection robot to the full output mode; use the test image to perform privacy protection processing based on the full output mode to obtain first image data that is processed for privacy protection based on the detection result of a human eye detection model, second image data that is processed for privacy protection based on the positioning result of a feature point positioning model, and third image data that is processed for privacy protection based on the human eye detection result and the feature point positioning result; respectively count the bit error rates of the first image data, the second image data, and the third image data to obtain a bit error rate test result; if the bit error rate test result is less than a preset bit error rate threshold, it is determined that the privacy protection robot has passed the bit error rate test.

[0141] Fig.12 The facial data privacy protection device of the illustrated embodiment can be used to implement the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0142] Fig.13 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure, the electronic device may be a privacy protection device, specifically a privacy protection robot. Fig.13 , which shows a structural schematic diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0143] like Fig.13 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the facial data privacy protection method of the embodiment described in the present disclosure. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0144] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.13 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0145] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the facial data privacy protection method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0146] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0147] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0148] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0149] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0150] Constructing an image dataset, wherein the image dataset includes a large amount of image data including facial images;

[0151] Based on the image data set, the human eye detection model and the feature point positioning model are trained to obtain a trained human eye detection model and a trained feature point positioning model;

[0152] Acquire image data to be protected for privacy, and input the image data to be protected for privacy into a trained human eye detection model and a trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result;

[0153] Based on the human eye detection results and feature point positioning results, a preset post-processing algorithm is used to perform privacy protection processing on the privacy-protected image data to generate privacy-protected image data.

[0154] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0155] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0156] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0158] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0159] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0161] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0162] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A facial data privacy protection method, characterized in that: The method comprises: Constructing an image dataset, wherein the image dataset includes a large amount of image data including facial images; Based on the image data set, the human eye detection model and the feature point positioning model are trained to obtain a trained human eye detection model and a trained feature point positioning model; Acquire image data to be protected for privacy, and input the image data to be protected for privacy into a trained human eye detection model and a trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result; Based on the human eye detection results and feature point positioning results, a preset post-processing algorithm is used to perform privacy protection processing on the image data to be protected, and generate privacy-protected image data; The method of performing privacy protection processing on the image data to be protected by privacy protection by using a preset post-processing algorithm based on the human eye detection result and the feature point positioning result to generate the privacy protected image data includes: Performing coordinate screening on the eye detection result to determine the coordinates of the upper left corner and the lower right corner of the first privacy protection area; Perform coordinate screening on the feature point positioning results to determine the upper left corner coordinates and the lower right corner coordinates of the second privacy protection area; The coordinates filtered out by the human eye detection result are merged with the coordinates filtered out by the feature point positioning result to obtain the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area; Based on the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area, mask processing is performed on the target privacy protection area to generate privacy-protected image data; After respectively training the human eye detection model and the feature point positioning model based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model, the method further includes: Integrate the trained human eye detection model, the trained feature point positioning model, and the preset post-processing algorithm to obtain a privacy protection module, and embed the privacy protection module into a standard robot to construct a privacy protection robot; The test image is used to perform a bit error rate test on the privacy protection robot to obtain the bit error rate test result.

2. The method according to claim 1, characterized in that The constructing of the image dataset comprises: Collect a large amount of image data containing facial images; Annotate each of the image data, annotate the eye bounding box of each image, obtain a human eye detection data set, and divide the human eye detection data set into a training set, a validation set, and a test set of the human eye detection model based on a preset first ratio; Each of the image data is annotated to obtain the feature point coordinates of each image to obtain a feature point positioning data set, and the feature point positioning data set is divided into a training set, a validation set and a test set of the feature point positioning model based on a preset second ratio.

3. The method according to claim 1, characterized in that The image data set includes a human eye detection data set and a feature point positioning data set; The step of training the human eye detection model and the feature point positioning model based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model comprises: Training a human eye detection model based on the human eye detection data set, updating model parameters of the human eye detection model, and obtaining a trained human eye detection model if the human eye detection result of each image output by the updated human eye detection model has converged to the marked eye bounding box of each image; The feature point positioning model is trained based on the feature point positioning data set, and the model parameters of the feature point positioning model are updated. If the feature point positioning results of each image output by the updated feature point positioning model have converged to the marked feature point coordinates of each image, a trained feature point positioning model is obtained.

4. The method according to claim 1, characterized in that After respectively training the human eye detection model and the feature point positioning model based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model, the method further includes: Using respective test sets to test the trained eye detection model and the trained feature point positioning model respectively, to obtain the eye detection output result and the feature point positioning output result of the test set; Using preset test indicators to evaluate the human eye detection output results and feature point positioning output results of the test set, to obtain the test results of the human eye detection model and the test results of the feature point positioning model; When the test results of the human eye detection model and the test results of the feature point positioning model both meet the target index values, it is determined that the training of the human eye detection model and the feature point positioning model is completed.

5. The method according to claim 1, characterized in that The bit error rate test of the privacy protection robot is performed using the test image to obtain the bit error rate test result, including: Adjust the output mode of the privacy protection robot to full output mode; Using the test image to perform privacy protection processing based on the full output mode, obtain first image data that performs privacy protection processing based on the detection result of the human eye detection model, second image data that performs privacy protection processing based on the positioning result of the feature point positioning model, and third image data that performs privacy protection processing based on the human eye detection result and the feature point positioning result; The bit error rates of the first image data, the second image data, and the third image data are respectively counted to obtain a bit error rate test result. If the bit error rate test result is less than a preset bit error rate threshold, it is determined that the privacy protection robot passes the bit error rate test.

6. A facial data privacy protection device, characterized in that: include: A construction unit, configured to construct an image data set, wherein the image data set includes a large amount of image data including facial images; A training unit, used to train the human eye detection model and the feature point positioning model based on the image data set to obtain a trained human eye detection model and a trained feature point positioning model; An obtaining unit, used for obtaining image data to be protected for privacy, and inputting the image data to be protected for privacy into a trained human eye detection model and a trained feature point positioning model, respectively, to obtain a human eye detection result and a feature point positioning result; A generating unit, configured to perform privacy protection processing on the image data to be protected by privacy using a preset post-processing algorithm based on the human eye detection result and the feature point positioning result, and generate image data after privacy protection; The generation unit performs privacy protection processing on the image data to be protected by privacy protection using a preset post-processing algorithm based on the human eye detection result and the feature point positioning result, and generates the privacy protected image data, specifically for: Performing coordinate screening on the eye detection result to determine the coordinates of the upper left corner and the lower right corner of the first privacy protection area; Perform coordinate screening on the feature point positioning results to determine the upper left corner coordinates and the lower right corner coordinates of the second privacy protection area; The coordinates filtered out by the human eye detection result are merged with the coordinates filtered out by the feature point positioning result to obtain the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area; Based on the upper left corner coordinates and the lower right corner coordinates of the target privacy protection area, mask processing is performed on the target privacy protection area to generate privacy-protected image data; After respectively training the human eye detection model and the feature point positioning model based on the image data set to obtain the trained human eye detection model and the trained feature point positioning model, the facial data privacy protection device further includes: a construction unit; The construction unit is used to integrate the trained human eye detection model, the trained feature point positioning model, and the preset post-processing algorithm to obtain a privacy protection module, and embed the privacy protection module into a standard robot to construct a privacy protection robot; use the test image to perform a bit error rate test on the privacy protection robot to obtain a bit error rate test result.

7. A privacy protection robot, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images