Methods and devices for protecting facial privacy in images of intelligent driving and intelligent cockpit

Abstract

This application provides a method and apparatus for protecting facial privacy in intelligent driving and intelligent cockpit images, belonging to the technical field of intelligent driving data processing and intelligent cockpit data processing. The method includes: extracting image features from a face image to be desensitized identified from an image to be processed and from a virtual face image set corresponding to the face image to be desensitized; fusing the image features of the corresponding face image to be desensitized and the virtual face image to obtain a fused face image; determining a target fused face image from each fused face image, wherein a first similarity between the target fused face image and the corresponding virtual face image is greater than a first similarity threshold and a second similarity between the target fused face image and the corresponding face image to be desensitized is less than a second similarity threshold; and replacing the corresponding face image to be desensitized in the image to be processed with the target fused face image. This solution can improve the accuracy of image detection and semantic segmentation.

Description

Technical Field

[0001] This application relates to the fields of intelligent driving data processing and intelligent cockpit data processing technology, and in particular to a method and apparatus for protecting facial privacy in intelligent driving and intelligent cockpit images. Background Technology

[0002] Data masking (DM) refers to the process of transforming sensitive information using masking rules to reliably protect sensitive and private data. Data masking includes structured data masking and unstructured data masking.

[0003] Face desensitization falls under the category of unstructured data desensitization, and the main methods currently used are mosaic or blurring. For static images, the common approach is to locate localized regions of the face and then process these regions. Mosaic processing involves dividing the located region into blocks and applying equal-value processing. For example, a located region might be divided into 16 sub-regions, with each sub-region containing only one pixel value, thus achieving a desensitization effect. However, this method results in the greatest information loss. Another approach is to blur the located region, typically using Gaussian blur or mean blur. Blurred areas exhibit a frosted glass appearance, making basic features difficult to discern.

[0004] However, for the training requirements of autonomous driving, mosaicking and blurring can disrupt the distribution of images, leading to a decrease in accuracy in image detection and semantic segmentation tasks. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method, apparatus, and storage medium for protecting facial privacy in images of intelligent driving and intelligent cockpits, which can improve the accuracy of image detection and semantic segmentation.

[0006] According to a first aspect of the embodiments of this application, a method for protecting facial privacy in intelligent driving and intelligent cockpit images is provided. The method includes: identifying at least one face image to be desensitized from an image to be processed; obtaining a virtual face image corresponding to each face image to be desensitized from a set of virtual face images; extracting features from each face image to be desensitized and the virtual face image to obtain image features; fusing the image features of each corresponding set of face images to be desensitized and the virtual face image to obtain a fused face image; determining at least one target fused face image based on each fused face image, wherein a first similarity between the target fused face image and the corresponding virtual face image is greater than a first similarity threshold, and a second similarity between the target fused face image and the corresponding face image to be desensitized is less than a second similarity threshold; and replacing the corresponding face image to be desensitized in the image to be processed with each target fused face image.

[0007] According to a second aspect of the embodiments of this application, a facial privacy protection device for intelligent driving and intelligent cockpit images is provided. The device includes: an acquisition module, configured to identify at least one face image to be desensitized from an image to be processed; a matching module, configured to acquire a virtual face image corresponding to each face image to be desensitized from a set of virtual face images; an extraction module, configured to extract features from each face image to be desensitized and the virtual face image respectively to obtain image features; a fusion module, configured to fuse the image features of each corresponding group of face images to be desensitized and the virtual face image respectively to obtain a fused face image; a filtering module, configured to determine at least one target fused face image based on each fused face image, wherein a first similarity between the target fused face image and the corresponding virtual face image is greater than a first similarity threshold, and a second similarity between the target fused face image and the corresponding face image to be desensitized is less than a second similarity threshold; and a processing module, configured to replace the corresponding face image to be desensitized in the image to be processed with each target fused face image respectively.

[0008] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect.

[0009] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0010] According to a fifth aspect of the present application, a computer program product is provided, which is tangibly stored on a computer-readable medium and includes computer-executable instructions that, when executed, cause at least one processor to perform the method as described in the first aspect.

[0011] The facial privacy protection method for intelligent driving and intelligent cockpit images provided by the above scheme involves identifying one or more face images to be desensitized from the image to be processed, obtaining virtual face images corresponding to the face images to be desensitized from the virtual face image set, extracting features from each face image to be desensitized and each virtual face image, and fusing the features extracted from each set of corresponding face images to be desensitized and virtual face images to obtain a fused face image. One or more target fused face images are determined from each fused face image. The target fused face image needs to satisfy the following conditions: the first similarity between the target fused face image and its corresponding virtual face image is greater than a first similarity threshold, and the second similarity between each target fused face image and its corresponding face image to be desensitized is less than a second similarity threshold. The corresponding face image to be desensitized in the image to be processed is replaced by each target fused face image. Since the target fused face image is obtained by extracting features from the virtual face image and the face image to be desensitized, and then fusing the extracted features, the fused features replace the feature information in the face image to be desensitized with the pixel distribution of the approximate domain in the virtual face image. This is a local processing of the face image to be desensitized, which minimizes the destruction of the image pixel distribution. Therefore, the detection and semantic segmentation tasks of the desensitized face image will not suffer from a decrease in accuracy. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 A flowchart illustrating a method for protecting facial privacy in images of intelligent driving and intelligent cockpits provided in one embodiment of this application;

[0014] Figure 2 A flowchart illustrating a method for protecting facial privacy in images of intelligent driving and intelligent cockpits, provided in another embodiment of this application;

[0015] Figure 3 A schematic diagram of a facial privacy protection device for intelligent driving and intelligent cockpit images provided in one embodiment of this application;

[0016] Figure 4This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0018] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0020] Methods for protecting facial privacy in images of intelligent driving and intelligent cockpits

[0021] like Figure 1 As shown, Figure 1 This is a flowchart of a method for protecting facial privacy in images of intelligent driving and intelligent cockpits, provided in an embodiment of this application. The method includes the following steps 101 to 106:

[0022] Step 101: Identify at least one face image to be desensitized from the image to be processed.

[0023] The core technologies of intelligent driving include perception, decision-making, control, and computing platforms. Among them, the perception layer can be camera-driven.

[0024] The intelligent cockpit concept architecture includes environmental monitoring, driver monitoring, vehicle status monitoring, a human-machine interaction system, and a decision-making and execution system. The human-machine interaction component includes voice, gesture, facial recognition, and identity recognition.

[0025] The images to be processed can be obtained from local databases or cloud databases in intelligent driving and intelligent cockpits of vehicles. Facial recognition algorithms can be used to identify the faces to be de-identified within the images to be processed. Specifically, the images to be processed can be various graphics and images, including not only facial images but also images of animals, landscapes, calligraphy, etc.

[0026] The face image to be desensitized is an image whose privacy needs to be protected; it is a face image in the image to be processed.

[0027] Step 102: Obtain the virtual face image corresponding to each face image to be desensitized from the virtual face image set.

[0028] The virtual face image set comprises multiple virtual face images. For each face image to be de-identified, the corresponding virtual face image can be determined from the virtual face image set. For example, a virtual face image can be randomly sampled from the set as the virtual face image corresponding to the face image to be de-identified. The virtual face image set can be obtained from a local database or a cloud database. Furthermore, the virtual face image set is generated using a neural network model.

[0029] Step 103: Extract features from each face image to be desensitized and each virtual face image to obtain image features.

[0030] Feature extraction can be performed using methods such as HOG (Histogram of Oriented Gradient), Dlib, or convolutional neural networks.

[0031] Image features extracted from desensitized facial images and virtual facial images are identity features, including eyes, nose, mouth, etc.

[0032] Step 104: Perform feature fusion on the image features of each corresponding group of face images to be desensitized and virtual face images to obtain fused face images.

[0033] A deep learning model is trained to perform keypoint localization on features extracted from both the desensitized face image and the virtual face image. Based on the localization results, the two images are fused to obtain a fused face image. Specifically, the fusion algorithm employs adaptive instance normalization.

[0034] Step 105: Determine at least one target fused face image based on each fused face image, wherein the first similarity between the target fused face image and the corresponding virtual face image is greater than a first similarity threshold, and the second similarity between the target fused face image and the corresponding face image to be desensitized is less than a second similarity threshold.

[0035] In order to successfully replace the feature information of the face image to be desensitized, the features of the virtual face image and the features of the face image to be desensitized are fused together. The fused face image should have a higher similarity to the virtual face image. Therefore, the first similarity between the target fused face image and the corresponding virtual face image is greater than the first similarity threshold. Furthermore, the first similarity threshold can be set to any value, for example, the first similarity threshold can be set to 80%.

[0036] The smaller the similarity between the fused face image and the face image to be desensitized, the better. Therefore, the second similarity between the target fused face image and the corresponding face image to be desensitized is less than the second similarity threshold. Furthermore, the second similarity threshold can be set to any value, for example, the second similarity threshold can be set to 20%.

[0037] Specifically, the first and second similarities can be calculated using cosine distance or Euclidean distance.

[0038] Step 106: Replace the corresponding face image to be desensitized in the image to be processed with the fused face image of each target.

[0039] By performing feature point detection, convex hull search, convex hull-based triangular transformation, affine transformation, and seamless cloning on the face image to be desensitized in the image to be processed, the target fused face image is replaced with the face image to be desensitized.

[0040] In this embodiment, features are extracted from both the image to be desensitized and the virtual face image. These features are then fused. At least one target fused face image is determined from the resulting fused face image, ensuring that the similarity between the target fused face image and the corresponding virtual face image and the image to be desensitized meets certain conditions. The fused face image is then used to replace the corresponding image to be desensitized in the image to be processed. Therefore, by fusing the image to be desensitized and the virtual face image to obtain a fused face image, and replacing the corresponding image to be desensitized in the image to be processed with the target fused face image determined from the fused face image, and by using pixels from a domain approximating the virtual face image to replace the original face in the image to be desensitized, local processing of the image to be desensitized is performed without disrupting the distribution of the image to be desensitized, thus avoiding a decrease in accuracy during detection and semantic segmentation tasks.

[0041] In one possible implementation, a neural network model can be trained using the acquired training sample set to obtain an image generation model. This model is then used to generate at least one virtual face image that conforms to the face domain, resulting in a virtual face image set including all generated virtual face images. The training sample set comprises multiple face images.

[0042] The training sample set can be datasets such as CelebA, WIDER Face, and MALF.

[0043] Neural network models can be implemented using neural networks such as StyleGAN and ProGAN.

[0044] Preferably, taking the CelebA dataset as the training sample and the StyleGAN model algorithm as the neural network model as an example, the StyleGAN is trained using the CelebA sample set to obtain a virtual face image generation model. Through model inference, 100 randomly generated virtual face images are designated as the virtual face image set X.

[0045] In this embodiment, the neural network model learns by extracting features from a sample set to obtain an image generation model. This model then generates one or more virtual face images conforming to the face domain, resulting in a virtual face image set. The neural network extracts feature points from the sample set's faces using a generator and uses a discriminator to distinguish between "real" and "fake" feature points, ultimately obtaining the image generation model and generating at least one virtual face image conforming to the face domain. By continuously extracting and generating feature points and discriminating them, the generated virtual face image has high resolution. Taking StyleGAN as an example, it can not only generate high-quality and realistic virtual face images but also better control and understand the generated virtual face images, thus generating credible virtual face images. This lays the foundation for subsequent feature fusion between virtual face images and images to be desensitized.

[0046] In one possible implementation, when extracting features from the face image to be desensitized and the virtual face image, a pre-trained encoder can be used to extract features from each face image to be desensitized, thereby obtaining the image features of each face image to be desensitized. Similarly, the same encoder can be used to extract features from each virtual face image, thereby obtaining the image features of each virtual face image.

[0047] The encoder can be VAE-Encoder (Variational autoEncoder), DCGAN-Encoder (Deep Convolutional Generative Adversarial Networks), VGAE-Encoder (Variational Graph Auto-Encoders), etc.

[0048] Preferably, VAE-Encoder is used as an example. Using a pre-trained VAE-Encoder, convolution and pooling operations are performed on each face image to be de-identified to obtain a set of high-dimensional feature vectors for each face image. Then, using the VAE-Encoder, convolution and pooling operations are performed on each virtual face image to obtain its image features.

[0049] In this embodiment, the same encoder is used to extract features from each face image to be desensitized and each virtual face image. The encoder has the same tendency and ability to extract features from the face image to be desensitized and the virtual face image. The encoder encodes and decodes the feature vectors of the face image to be desensitized and the virtual face image respectively, so that the extracted feature layer is complete, thereby ensuring the stability and consistency of the subsequent fusion steps.

[0050] Figure 2 This is a flowchart of a method for protecting facial privacy in intelligent driving and intelligent cockpit images provided in another embodiment of this application. Figure 2 As shown, Figure 1 Step 105 in the method for protecting facial privacy in intelligent driving and intelligent cockpit images may include the following sub-steps 1051 to 1057:

[0051] Sub-step 1051: For each fused face image, calculate the first similarity between the fused face image and the corresponding virtual face image.

[0052] Sub-step 1052: Determine whether the first similarity corresponding to the fused face image is greater than the preset first similarity threshold. If yes, proceed to sub-step 1053; otherwise, proceed to sub-step 1057.

[0053] A first similarity threshold is preset, for example, the first similarity threshold can be 80%. After obtaining the first similarity of the fused face image, it is determined whether the first similarity is greater than the first similarity threshold. If the first similarity is greater than the first similarity threshold, it means that the fused face image and the virtual face image have a high degree of similarity and the fusion effect is good. The corresponding sub-step 1053 is executed. If the first similarity is less than or equal to the first similarity threshold, it means that the fused face image and the virtual face image have a low degree of similarity. Compared with the face image to be desensitized, the distribution of the fused face image has been severely damaged, which will lead to a decrease in the accuracy of image detection and semantic segmentation tasks. The corresponding sub-step 1057 is executed.

[0054] In this embodiment, for each fused face image, if the first similarity between the fused face image and the corresponding virtual face image is less than or equal to a first similarity threshold, a corresponding virtual face image is re-determined for the fused face image to be desensitized. Feature extraction and feature fusion are then performed on the re-desensitized face image and the corresponding virtual face image to obtain a new fused face image. It can be seen that by discussing the case where the virtual face image and the fused face image are calculated using the algorithm model, the virtual face image corresponding to the fused face image whose first or multiple fusion results do not satisfy the first similarity threshold is re-matched with the virtual face image, ensuring that all faces to be desensitized can be successfully desensitized.

[0055] Sub-step 1053: Adjust the outline of the fused face image to obtain the optimized fused face image.

[0056] To match the contours of the fused face image and the face image to be desensitized, the fused face image and the face image to be desensitized are input into a discriminator pre-trained by a neural network. The discriminator adjusts the contour of the fused face image multiple times, calculating a loss value after each adjustment. If the loss value is less than a preset threshold, an optimized fused face image is obtained. If the loss value is greater than or equal to the preset threshold, the contour of the fused face image is further adjusted based on the loss value until the calculated loss value is less than the preset threshold. Specifically, the contour includes contour attributes of non-feature information of the face, such as hair and lighting in the image.

[0057] The preset threshold can be set according to needs; for example, the preset threshold can be set to 5%.

[0058] Sub-step 1054: Calculate the second similarity between the optimized fused face image and the corresponding face image to be desensitized.

[0059] In this embodiment, for each fused face image, a first similarity is calculated between the fused face image and the corresponding virtual face image. For fused face images with a first similarity greater than a preset first similarity threshold, contour adjustment is performed to make the contour attributes of the fused face image closer to the face to be desensitized. A second similarity is calculated between the fused face image that matches the contour attributes of the face to be desensitized and the face image to be desensitized. The second similarity calculation only requires comparing the similarity of the identity features of the optimized fused face image and the corresponding face image to be desensitized, without being affected by the inconsistency of the contour attributes of the two images in obtaining the second similarity result.

[0060] Sub-step 1055: Determine whether the second similarity corresponding to the optimized and fused face image is less than the preset second similarity threshold. If yes, proceed to sub-step 1056; otherwise, proceed to sub-step 1057.

[0061] After obtaining the optimized and fused face image, it is necessary to determine the second similarity with the corresponding face image to be desensitized. If the second similarity is less than a preset second similarity threshold, for example, the second similarity threshold can be 20%, it means that the similarity between the optimized and fused face image and the face image to be desensitized is low, and the fusion effect is good, so sub-step 1056 is executed. If the second similarity is greater than or equal to the second similarity threshold, it means that the similarity between the optimized and fused face image and the face image to be desensitized is high, indicating a poor fusion effect. The optimized and fused face image has not been successfully desensitized relative to the face image to be desensitized, so sub-step 1057 is executed.

[0062] Sub-step 1056: Determine the optimized fused face image as the target fused face image and execute step 106.

[0063] Sub-step 1057: Re-determine the corresponding virtual face image for the face image to be desensitized corresponding to the fused face image, and execute step 103.

[0064] In this embodiment, for each optimized and fused face image, if the second similarity between the optimized and fused face image and the corresponding face image to be desensitized is greater than or equal to the second similarity threshold, a corresponding virtual face image is re-determined for the face image to be desensitized corresponding to the optimized and fused face image. Feature extraction and feature fusion are then performed on the re-desensitized face image and the corresponding virtual face image to obtain a new fused face image. Therefore, by removing optimized and fused face images that do not meet the second similarity threshold between the optimized and fused face image and the corresponding face image to be desensitized, and re-pairing the corresponding face image to be desensitized with a virtual face, it is ensured that all faces to be desensitized can be successfully desensitized.

[0065] In one possible implementation, when identifying the face image to be desensitized from the image to be processed, a face recognition algorithm can be used to detect faces in the image to be processed. The image regions in the image to be processed with a confidence score greater than a preset confidence score threshold are identified as the face images to be desensitized. Here, the confidence score is used to indicate the probability that the image region includes a face image.

[0066] Face recognition algorithms can include Insightface, Fisherfaces, Haar Cascade, FaceNet, etc.

[0067] Preferably, Insightface is used as an example. First, the images to be processed are resized to a uniform size. Insightface detects faces in the images. If the confidence level of a face in the image is greater than a preset confidence threshold, the region of that face is identified as the face to be desensitized. All detected faces to be desensitized are set to set Y, and the number n is counted. Then, m virtual face images are randomly selected from the virtual face image set X, where m = n. The preset confidence threshold is set to 60%.

[0068] In this embodiment of the application, the face recognition algorithm is used to detect the face in the image to be processed, which can identify the image with a face in the image to be processed and exclude non-human images such as animal images and landscape images from the image to be processed, thereby realizing the detection and localization of the face image to be desensitized.

[0069] In one possible implementation, after identifying at least one face image to be desensitized from the image to be processed, a virtual face image corresponding to each face image to be desensitized is determined through a parallel process, and the corresponding face images to be desensitized and virtual face images are processed through a parallel process.

[0070] The process involves pairing virtual face images (X) with faces in a set of images to be anonymized (Y), and then aligning the paired virtual and anonymized images. For each pair of virtual and anonymized images in sets X and Y, feature extraction is performed to obtain image features. These features are then fused using a fusion algorithm to obtain a fused face image. At least one target fused face image is determined for each pair, and this target fused face image replaces the corresponding anonymized face image in the processing image. Specifically, the face alignment method can employ MTCNN, which detects key points in each pair of paired virtual and anonymized face images, such as the positions of the eyes, nose, and mouth. Based on these key points, affine transformations are used to uniformly calibrate the faces, achieving alignment.

[0071] In this embodiment, each face image to be desensitized and a virtual face image in the image to be processed are paired one-to-one. Alignment is then performed on each paired face image and virtual face image. Features are extracted from the aligned face images and virtual face images, and the extracted features are fused to obtain a fused face image. At least one target fused face image is determined from the fused face image, and each target fused face image replaces the corresponding face image to be desensitized in the image to be processed. Therefore, for each face image and virtual face image to be desensitized, this method for protecting the face privacy of intelligent driving and intelligent cockpit images can be executed in parallel, breaking the limitations of existing technologies that only support a single face and enabling simultaneous replacement of faces in multiple face images and virtual face images to be desensitized.

[0072] Facial privacy protection devices for images in intelligent driving and intelligent cockpits

[0073] like Figure 3 As shown, Figure 3 This is a schematic diagram of a facial privacy protection device for intelligent driving and intelligent cockpit images provided in the application embodiment. The device includes:

[0074] The acquisition module 301 is used to identify at least one face image to be desensitized from the image to be processed;

[0075] Matching module 302 is used to obtain the virtual face image corresponding to each face image to be desensitized from the virtual face image set;

[0076] The extraction module 303 is used to extract features from each face image to be desensitized and the virtual face image to obtain image features;

[0077] The fusion module 304 is used to fuse the image features of each corresponding set of face images to be desensitized and virtual face images to obtain a fused face image;

[0078] The filtering module 305 is used to determine at least one target fused face image based on each fused face image, wherein the first similarity between the target fused face image and the corresponding virtual face image is greater than a first similarity threshold, and the second similarity between the target fused face image and the corresponding face image to be desensitized is less than a second similarity threshold.

[0079] The processing module 306 is used to replace the corresponding face image to be desensitized in the image to be processed with each target fused face image.

[0080] In this embodiment, the acquisition module identifies one or more face images to be desensitized from the image to be processed. The matching module acquires the virtual face image corresponding to each face image to be desensitized from the virtual face image set. The extraction module extracts features from each face image to be desensitized and the virtual face image to obtain image features. The fusion module fuses the features extracted by the extraction module to obtain a fused face image. The filtering module filters at least one target fused face image that meets the conditions from the fused face images in the fusion block. The processing module replaces the corresponding face image to be desensitized in the image to be processed with each target fused face image. It can be seen that the fused face image obtained by fusing the face image to be desensitized and the virtual face image, and the target fused face image determined from the fused face image replacing the corresponding face image to be desensitized in the image to be processed, uses pixels from a domain approximating the virtual face domain to replace the original face in the face image to be desensitized. This is a local processing of the face image to be desensitized, without destroying the distribution of the face image to be desensitized, thus avoiding a decrease in accuracy in detection and semantic segmentation tasks.

[0081] It should be noted that the information interaction and execution process between the modules in the aforementioned intelligent driving and intelligent cockpit image face privacy protection device are based on the same concept as the aforementioned intelligent driving and intelligent cockpit image face privacy protection method embodiments. For details, please refer to the description in the aforementioned intelligent driving and intelligent cockpit image face privacy protection method embodiments, and will not be repeated here.

[0082] electronic devices

[0083] Reference Figure 4 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0084] like Figure 4 As shown, the electronic device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0085] in:

[0086] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0087] Communication interface 404 is used to communicate with other electronic devices or servers.

[0088] The processor 402 is used to execute program 410, which can specifically execute the relevant steps in the above-described image annotation method embodiment.

[0089] Specifically, program 410 may include program code that includes computer operation instructions.

[0090] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0091] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0092] Specifically, program 410 can be used to cause processor 402 to execute the facial privacy protection method for intelligent driving and intelligent cockpit images in any of the foregoing embodiments.

[0093] The specific implementation of each step in procedure 410 can be found in the corresponding steps and units described in any of the foregoing embodiments of the face privacy protection method for intelligent driving and intelligent cockpit images, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0094] The electronic device in this embodiment extracts features from each face image to be desensitized in the image to be processed and the corresponding virtual face image in the virtual face image set. The extracted features are then fused using a fusion algorithm to obtain a fused face image. At least one target fused face image that meets certain conditions is identified from the obtained fused face image, and this target fused face image replaces the corresponding face image to be desensitized in the image to be processed. Therefore, this scheme uses the pixel distribution of the approximate domain of the virtual face to replace the face image to be desensitized, which is a local processing of the face image to be desensitized, thus minimizing damage to the image pixel distribution and preventing accuracy degradation in image detection and semantic segmentation tasks.

[0095] Computer storage media

[0096] This application also provides a computer-readable storage medium storing instructions for causing a machine to execute facial privacy protection methods for intelligent driving and intelligent cockpit images as described herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0097] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of this application.

[0098] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0099] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0100] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0101] Computer program products

[0102] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0103] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0104] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA) for such software processing. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the image annotation methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the image annotation methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the checksum generation method shown herein.

[0105] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0106] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for face privacy protection of intelligent driving and intelligent cockpit images, comprising: identifying at least one face image to be desensitized from an image to be processed; obtaining a virtual face image corresponding to each of the face images to be desensitized from a virtual face image set; respectively extracting features of each of the face images to be desensitized and the virtual face images to obtain image features; respectively performing feature fusion on the image features of each of the face images to be desensitized and the virtual face images corresponding thereto to obtain fused face images; determining at least one target fused face image according to the fused face images, wherein a first similarity between the target fused face image and the virtual face image corresponding thereto is greater than a first similarity threshold, and a second similarity between the target fused face image and the face image to be desensitized corresponding thereto is less than a second similarity threshold; the determining at least one target fused face image according to the fused face images comprises: for each of the fused face images: calculating a first similarity between the fused face image and the virtual face image corresponding thereto; if the first similarity corresponding to the fused face image is greater than a preset first similarity threshold, adjusting a contour of the fused face image to obtain an optimized fused face image, wherein a contour loss value between the optimized fused face image and the face image to be desensitized corresponding thereto is less than a preset loss value threshold, so that a contour attribute of the optimized fused face image is closer to the face image to be desensitized; if the first similarity corresponding to the fused face image is less than or equal to the first similarity threshold, re-determining the virtual face image corresponding to the face image to be desensitized corresponding to the fused face image, and performing feature extraction and feature fusion on the re-determined face image to be desensitized and the virtual face image corresponding thereto to obtain a new fused face image; calculating a second similarity between the optimized fused face image and the face image to be desensitized corresponding thereto; if the second similarity corresponding to the optimized fused face image is less than a preset second similarity threshold, determining the optimized fused face image as the target fused face image; replacing the face image to be desensitized corresponding thereto in the image to be processed by each of the target fused face images. 2.The method of claim 1, wherein the method further comprises: training a neural network model by using a training sample set to obtain an image generation model, wherein the training sample set comprises a plurality of face images; generating at least one virtual face image conforming to a face domain by using the image generation model; and obtaining the virtual face image set comprising the generated virtual face images. 3.The method of claim 1, wherein the respectively extracting features of each of the face images to be desensitized and the virtual face images to obtain image features comprises: respectively extracting features of each of the face images to be desensitized by using a pre-trained encoder to obtain image features of each of the face images to be desensitized. ​ ​ The feature extraction is performed on each of the virtual face images respectively by the encoder to obtain image features of each of the virtual face images.

4. The method of claim 1, wherein, The method further comprises: For each of the tuned fusion face images, if the second similarity corresponding to the tuned fusion face image is greater than or equal to the second similarity threshold, the virtual face image corresponding to the tuned fusion face image is re-determined for the desensitized face image, and the feature extraction and the feature fusion are performed on the re-determined desensitized face image and the corresponding virtual face image to obtain a new fusion face image.

5. The method of claim 1, wherein, The identification of the at least one desensitized face image from the to-be-processed image comprises: The face recognition algorithm is used to detect the face in the to-be-processed image, and the image region corresponding to a confidence greater than a preset confidence threshold in the to-be-processed image is determined as the desensitized face image, wherein the confidence is used to indicate the probability that the image region includes a face image.

6. The method of any one of claims 1-5, wherein, After the at least one desensitized face image is identified from the to-be-processed image, the virtual face image corresponding to each of the desensitized face images is determined through a parallel process, and the corresponding desensitized face image and the virtual face image are processed through a parallel process.

7. A face privacy protection device for intelligent driving and intelligent cockpit images, the face privacy protection device performing the method of claim 1, comprising: An acquisition module configured to identify at least one desensitized face image from a to-be-processed image; A matching module configured to obtain one virtual face image corresponding to each of the desensitized face images from a virtual face image set; An extraction module configured to perform feature extraction on each of the desensitized face images and the virtual face images respectively to obtain image features; A fusion module configured to perform feature fusion on the image features of each of the corresponding desensitized face images and the virtual face images respectively to obtain fusion face images; A screening module configured to determine at least one target fusion face image from the fusion face images, wherein the first similarity between the target fusion face image and the corresponding virtual face image is greater than a first similarity threshold, and the second similarity between the target fusion face image and the corresponding desensitized face image is less than a second similarity threshold; A processing module configured to replace the corresponding desensitized face image in the to-be-processed image with each of the target fusion face images respectively.

8. An electronic device comprising a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface being in communication with each other through the communication bus; The memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform the operations corresponding to the method of any one of claims 1-6.

9. A computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1-6.

10. A computer program product tangibly stored on a computer readable medium and comprising computer executable instructions that, when executed, cause at least one processor to perform the method of any one of claims 1-6.

Images