Physical multi-scene figure information privacy protection method based on image disturbance shielding
By generating robust global perturbations and adding glasses perturbation spoofing in facial recognition technology, the problem of insufficient privacy protection in facial recognition technology is solved, achieving efficient privacy protection in multiple scenarios and ensuring the safety and privacy of the target person.
Patent Information
- Application Number
- CN202511129298.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-13
- Publication Date
- 2025-11-25
AI Technical Summary
In the current technology, facial recognition technology suffers from insufficient privacy protection during its widespread application. In particular, when unauthorized organizations or individuals obtain and use facial images, it may pose a threat to personal safety and property security.
A physical multi-scene privacy protection method for people's information based on image perturbation masking is adopted. By generating robust global perturbations in the scene where the target person's image is published, and adding perturbation deception to the selected glasses in the scene where the target person is not identified, the artificial intelligence model is prevented from correctly identifying the target person.
It achieves efficient, accurate and comprehensive privacy protection in various scenarios, prevents images from being maliciously acquired and identified, ensures that target individuals cannot be correctly identified by artificial intelligence models, and improves the robustness and applicability of privacy protection.
Smart Images
Figure CN121009577A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of artificial intelligence security, and particularly relates to a physical multi-scene character information privacy protection method based on image disturbance masking. BACKGROUND
[0002] Artificial intelligence technology has made great progress in recent years and is widely used in security, finance, retail, and personal devices, among other fields. In many application scenarios, face recognition technology is particularly prominent and has penetrated into people's daily life and work, such as payment, personnel management, video surveillance, etc. With the continuous progress of technology and the increasing demand for security in society, face recognition technology is not limited to basic functions such as identity verification, but has also expanded to more complex scene applications, such as customer behavior analysis in intelligent retail, security warning systems in public places, and automatic photo tagging in social media, etc.
[0003] In modern society, with the widespread use of media channels such as the Internet, magazines, and newspapers, we often need to publish personal images on these platforms. However, this also means that these images may be obtained and collected by unauthorized organizations or individuals. These people may use advanced face recognition technology to mark and identify target individuals, posing a potential threat to the personal safety and property safety of the target individuals. In addition, certain organizations or individuals may also use fixed network cameras and cameras carried by drones to continuously track and monitor specific individuals, which not only infringes on personal privacy but also seriously threatens the personal safety of the target individuals.
[0004] In summary, the development of face recognition technology poses new challenges to privacy protection. SUMMARY
[0005] To solve the defects of face recognition in the prior art, the present application provides a physical multi-scene character information privacy protection method based on image disturbance masking, a character information privacy protection system in a real environment, which comprises two modules: a target character image privacy protection module and a target character anti-recognition module. The target character image privacy protection module generates robust global disturbance according to the target character image publishing scene to prevent the image from being recognized by an artificial intelligence model after being maliciously obtained; and the target character anti-recognition module adds disturbance to the specific glasses selected for the target character to deceive, so that the target character cannot be correctly recognized by an artificial intelligence model.
[0006] To achieve the above purpose, the present application is implemented through the following technical solutions:
[0007] This application discloses a physical multi-scene person information privacy protection method based on image perturbation occlusion. This method is applied to a real physical environment and includes two stages: a target person image privacy protection scene and a target person anti-identification scene. The physical multi-scene person information privacy protection method specifically includes the following steps:
[0008] In scenarios involving the protection of target person images' privacy, robust global adversarial perturbations are generated based on the context in which the target person's images are published. This prevents the target person's images from being maliciously obtained and subsequently identified by artificial intelligence models. Specifically, this includes:
[0009] Step 1.1: Determine the context in which the target person's image will be published;
[0010] Step 1.2: After determining the release scenario, use the particle swarm optimization algorithm to perform a global adversarial perturbation search based on the target person's image. The search target is the features of the target person's image. The global adversarial perturbation search process is modeled as an optimization problem.
[0011] Step 1.3: Add invisible global adversarial perturbations to the target person's image published by the target person, and calculate the printing loss. The printing loss comes from the compression of the target person's image and the conversion of RGB encoding to CMYK format, realizing the mutual conversion between RGB images and the printed images of the publishing channel, and compensating for the distortion of the target person's image caused by the printing process.
[0012] Step 1.4: Determine whether the privacy protection goal has been achieved. If the privacy protection goal has not been achieved, repeat steps 1.2-1.3. If the privacy protection goal has been achieved, output an RGB image containing global adversarial perturbation as the result.
[0013] In scenarios where target person identification is prevented, perturbation deception is added to the glasses patch selected by the target person, so that the target person cannot be correctly identified by the artificial intelligence model. The specific steps include the following:
[0014] Step 2.1: The target person selects the patch type from the eyeglass patch library, uses the facial key point detection module to select three key points: the center of the two brow bones and the bridge of the nose, and aligns the selected eyeglass patch to the target person image initially selected in the anti-identification scene using affine transformation based on the selected key points.
[0015] Step 2.2: After aligning the target person image and the glasses patch initially selected in the anti-identification scenario, search for the perturbation value of the pixels in the frame area of the glasses worn by the target person. After optimization, obtain the RGB encoded image of the glasses patch. The target person can effectively defend against the identification of the face recognition system by wearing glasses with glasses patch.
[0016] Step 2.3: Print the RGB encoded image of the glasses patch obtained from the optimization solution to ensure that the size of the glasses patch is close to the size of the real glasses frame, and that the image size is kept close to the real face in the global anti-perturbation solution, so as to effectively protect the privacy and security of the target person.
[0017] A further improvement of this application is that step 1.2 specifically includes the following steps:
[0018] Step 1.2.1: After determining the publishing channel, perform a global adversarial perturbation search and add global adversarial perturbations to the target person's image published by the target person so that the target person's image published by the target person cannot be recognized normally by the face recognition system.
[0019] Step 1.2.2: Model the global adversarial perturbation search process as an optimization problem;
[0020] Step 1.2.3: Printing loss comes from the compression of the target image and the conversion of RGB encoding to CMYK format. By generating a CycleGAN model, the mutual conversion between RGB images and printed images for distribution channels is realized, thereby reducing printing loss.
[0021] A further improvement in this application is that, in step 1.2.2, the global adversarial perturbation search process is modeled as three loss functions:
[0022] loss1=r(τ1+cos(f(g(x s +δ)),f(x s )))
[0023] loss2=r(τ2-cos(f(g(x s +δ)),f(x t )))
[0024] loss3=||δ|| ∞
[0025] Wherein, loss1 ensures that the cosine similarity between the facial features of the target person image and the facial features of the target person after adding global adversarial perturbation is less than or equal to τ1; loss2 ensures that the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features corresponding to the facial image of the specific release scene is greater than or equal to τ2; loss3 is the infinite norm that minimizes the global adversarial perturbation δ; r is the ReLU activation function, which outputs the input itself if the input value is greater than or equal to zero, and outputs the value if the input value is less than zero; f is the facial feature extractor; g is the print loss elimination function, which includes converting the RGB encoded image to the facial image of the specific release scene and converting the facial image of the specific release scene to the RGB encoded image; δ is the global adversarial perturbation; and x... sFor the target person image, x t For a face image in a specific publishing scenario, τ1 is the upper limit of the similarity threshold with the target face, and τ2 is the lower limit of the similarity threshold with the face incorrectly identified by the face recognition system.
[0026] The final overall loss function is:
[0027] loss = loss1 + loss2 + λ·loss3
[0028] Wherein, λ is the loss importance balancing factor, used to balance the importance of the third loss.
[0029] A further improvement of this application is that step 1.4 specifically includes the following steps:
[0030] Step 1.4.1: The global adversarial perturbation search solution uses an intelligent evolution algorithm. If the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features of the original target person image is greater than a predetermined threshold, or the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features of the facial image of the specific publishing scene is less than a predetermined threshold, it means that the purpose of privacy protection has been achieved.
[0031] Step 1.4.2: After achieving the goal of privacy protection, output an RGB image containing global adversarial perturbation as the result;
[0032] Step 1.4.3: If the privacy protection goal is not achieved, repeat steps 1.2-1.3 until the privacy protection goal is achieved. A further improvement of this application is that the intelligent evolution algorithm mentioned in step 1.4.1 specifically includes the following steps:
[0033] Step 1.4.1.1: Initialize the number of particles to N and the number of iterations to T. Each particle has an initial value of x1 and a velocity of 0.
[0034] Step 1.4.1.2: For each iteration j from 1 to T, and for each particle i from 1 to N, calculate the loss value corresponding to the current particle i based on loss2. If the loss value corresponding to the current particle i Less than or equal to the historical minimum loss value of particle i Then update the historical best position of particle i. and historical minimum loss value If the loss value corresponding to the current particle i Less than or equal to the global history minimum loss value of all particles g Then update the globally optimal position x. g Particle swarm optimization is used.
[0035]
[0036] Where w is the inertia weight, c1 is the individual fitness acceleration factor, c2 is the social fitness acceleration factor, and rand is a random number between [0,1] to update the velocity and position x of particle i. i ←x i +v i x i It is the position of particle i, v i It is the position update amount of particle i;
[0037] Step 1.4.1.3: If the loss value of any particle reaches a preset threshold, loss1 = 0 or loss2 = 0, then stop the iteration and return to the global optimal position x. g This yields the final disturbed image that achieves anti-identification.
[0038] A further improvement of this application is that step 2.1 specifically includes the following steps:
[0039] Step 2.1.1: The target person selects an eyeglasses patch from the patch library. The shape of the eyeglasses patch is consistent with the actual eyeglasses worn by the target person. The eyeglasses patch is printed onto the actual eyeglasses worn by the target person. After printing out the lens patch, it is pasted onto the actual eyeglasses frame to prevent the target person from being identified by the facial recognition system.
[0040] Step 2.1.2: Determine the key points in the target person image initially selected for the anti-identification scene using the face key point detection module. Then, select three key points on the two eyebrows and the bridge of the nose of the target person image initially selected for the anti-identification scene, as well as three key points on the corresponding glasses patch. Then, use the image transformation tool affine transformation, i.e., print and paste to align the target person image initially selected for the anti-identification scene and the glasses patch, i.e., add the glasses patch to the target person's face image.
[0041] A further improvement of this application is that step 2.2 specifically includes the following steps:
[0042] Step 2.2.1: Search for the perturbation value of the pixels in the frame area of the glasses worn by the target person, maximize the distance between the depth features of the original target face image and the depth features of the face image after adding the adversarial glasses patch, and minimize the distance between the depth features after adding the adversarial glasses patch and the depth features of the face image of another selected person, so that the face recognition system cannot recognize the target person normally.
[0043] Step 2.2.2: Consider the quantization loss, namely the printing loss caused by quantization of RGB encoded pixel values and the printing loss caused by the RGB to CMYK conversion of the image, and add the two printing losses to the process of solving the global perturbation resistance.
[0044] A further improvement of this application is that, in step 2.2.1, the perturbation value of the pixels in the search glasses patch area is represented by three losses:
[0045] loss4=r(τ1+cos(f(g(x s ·(1-m)+m·δ),f(x s )))
[0046] loss5=r(τ2-cos(f(g(x s ·(1-m)+m·δ),f(x t )))
[0047] loss3=||δ|| ∞
[0048] Among them, loss4 ensures that the cosine similarity between the facial features of the target person before and after adding local perturbation is less than or equal to τ1, loss5 ensures that the cosine similarity between the facial features of the target person after adding local adversarial perturbation and the facial features of the facial image corresponding to the specific release scene is greater than or equal to τ2, and loss3 minimizes the infinite norm of the global adversarial perturbation δ, where m is the masking matrix, i.e. the eyeglass patch region of the image.
[0049] The beneficial effects of this application are:
[0050] This application achieves high efficiency in facial privacy protection. Through the effective collaboration of the target person image privacy protection module and the target person anti-identification module, privacy protection of image information and facial recognition can be realized in various scenarios. The image privacy protection module prevents images from being maliciously obtained and identified in various publishing scenarios such as the internet and magazines by generating robust global perturbations; the anti-identification module ensures that the target person cannot be correctly identified by the artificial intelligence model through perturbation deception using specific glasses. This application optimizes the comprehensive privacy protection strategy and improves the overall protection capability and concealment of the system.
[0051] This application achieves high precision in facial privacy protection. Through dual protection of a target person image privacy protection module and an anti-identification module, this application can effectively protect the privacy information of the target person in various real-world environments. The image privacy protection module generates global perturbations based on different publishing scenarios, effectively preventing the image from being recognized by artificial intelligence models; the anti-identification module perturbs specific glasses to ensure that the target person cannot be correctly identified when facing an artificial intelligence model. This application significantly improves the accuracy and robustness of privacy protection.
[0052] This application achieves comprehensive facial privacy protection. Through the joint operation of the target person image privacy protection module and the target person anti-identification module, this application can effectively address the risk of privacy leakage in various environments. The image privacy protection module generates robust global perturbations to prevent the image from being identified after being maliciously obtained; the anti-identification module adds perturbations to the target person's glasses to ensure that the target person cannot be correctly identified by the artificial intelligence recognition system. This application significantly enhances the protection scope and applicability of the privacy protection system. Attached Figure Description
[0053] Figure 1 This is a framework diagram of the multi-scenario facial privacy protection system in this application.
[0054] Figure 2 This is a flowchart of the privacy protection module for the target person's image in this application.
[0055] Figure 3 This is a flowchart of the target person anti-identification subsystem module of this application. Detailed Implementation
[0056] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.
[0057] In a facial information privacy protection system, the image information of a target person is protected through a target person image privacy protection module and a target person anti-identification module. The system ensures that the target person's image is not maliciously obtained and identified in different publishing scenarios such as the internet, magazines, and newspapers by generating robust global perturbations and physical masking. Considering the specific requirements of the image publishing scenario, the system perturbs and deceives the target person's glasses to prevent them from being correctly identified by artificial intelligence models. Ultimately, by optimizing the perturbation generation and recognition mechanisms, this application achieves comprehensive protection of the target person's privacy information.
[0058] The functions and tasks of several stakeholders involved in the comprehensive protection of personal information privacy are defined below:
[0059] Face Privacy Protection System (hereinafter referred to as the Privacy Protection System): In the protection of personal information, the Privacy Protection System is mainly responsible for the following two aspects: (1) It is responsible for collecting the image information of the target person and generating global perturbations through the target person image privacy protection module. Combined with physical masking methods, it protects the privacy of the target person's image in different publishing scenarios such as the Internet, magazines, and newspapers. The Privacy Protection System transmits the processed image information to the anti-identification module to ensure that the image cannot be recognized by the artificial intelligence model. (2) It is responsible for anti-identification processing of the target person's image. By adding perturbations to physical objects such as the target person's glasses, it interferes with the artificial intelligence recognition system and prevents the target person from being correctly identified. The Privacy Protection System is the core of the entire image privacy protection process and plays an important role in image processing and anti-identification tasks.
[0060] Target Image Privacy Protection Module (hereinafter referred to as the Image Privacy Protection Module): The Image Privacy Protection Module is mainly responsible for two aspects in the entire privacy protection system: (1) It is responsible for generating robust global perturbations and combining them with physical masking technology to ensure that the image of the target person is not maliciously obtained and artificially identified in different publishing scenarios such as the Internet, magazines, and advertisements. The Image Privacy Protection Module can dynamically adjust the intensity and method of perturbation according to different publishing scenarios and needs, thereby improving the effect of privacy protection. (2) It is responsible for generating personalized perturbation schemes according to the specific content and scenario requirements of the target person's image to ensure that the privacy of the image is effectively protected after publication and to avoid abuse or identification. The Image Privacy Protection Module is the core part of the privacy protection system and undertakes the key tasks of image protection and privacy leakage prevention.
[0061] Target Anti-Recognition Module (hereinafter referred to as the Anti-Recognition Module): In the privacy protection system, the Anti-Recognition Module is mainly responsible for two aspects: (1) By adding perturbations to specific physical characteristics of the target person, such as glasses and hats, it interferes with the operation of the artificial intelligence recognition system, thereby ensuring that the target person cannot be accurately identified. This module can intelligently adjust the perturbation strategy according to different recognition environments and task requirements to achieve the best anti-recognition effect. (2) It is responsible for monitoring and recording the anti-recognition status of the target person, and dynamically adjusting the type and intensity of perturbation according to the real-time situation to ensure that the target person can be effectively prevented from being identified in different scenarios. The Anti-Recognition Module is a key component of the privacy protection system and directly affects the privacy security of the target person in various environments.
[0062] like Figure 1The framework for a multi-scenario facial privacy protection system is shown. This framework comprises two scenarios: target person image privacy protection and target person anti-identification. The target person image privacy protection scenario primarily addresses privacy protection for target person images in various scenarios, such as online, magazine, and newspaper content. In this scenario, robust global perturbations are generated based on the context in which the target person's image is published, preventing malicious acquisition and subsequent identification by AI models. In the target person anti-identification scenario, perturbations are added to the specific glasses selected by the target person to deceive them, thus preventing the AI model from correctly identifying them.
[0063] In scenarios involving the privacy protection of images of individuals, global adversarial perturbations are generated based on the context in which the images are published to prevent malicious acquisition of these images from being identified by artificial intelligence models. Specifically, this includes:
[0064] Step 1: Based on the photos to be published for the target audience, determine the publishing channels, such as the internet, magazines, or newspapers. Different channels have varying degrees of image distortion (print loss), with the internet exhibiting the least print loss, followed by magazines, and newspapers the most. The specific steps include:
[0065] Step 1.1: Determine the publishing channels based on the photos the target person needs to publish, such as the internet, magazines, and newspapers;
[0066] Step 1.2: For each distribution channel, evaluate the degree of image distortion within that channel. Here, all types of image quality degradation are collectively referred to as "print loss," with the print loss increasing progressively from online to magazine to newspaper.
[0067] Step 2: After determining the publishing scenario, based on the target person's image, a global adversarial perturbation search is performed using the particle swarm optimization algorithm. The search target is the features of the target person's image. The global adversarial perturbation search process is modeled as an optimization problem. Invisible adversarial perturbations are added to the target person's image, and printing loss is calculated. The printing loss comes from image compression and RGB encoding to CMYK format conversion. The CycleGAN model is used to realize the mutual conversion between RGB images and newspaper printed images, thereby compensating for image distortion caused by the printing process. Specifically, the following steps are included:
[0068] Step 2.1: After determining the release channel, global adversarial perturbation will be generated. An adversarial perturbation invisible to the naked eye will be added to the entire image of the original target person's face. With almost no impact on image quality, the image of the target person can be made to be unable to be recognized by the face recognition system.
[0069] Step 2.2: Model the global adversarial perturbation search process as an optimization problem: the feature corresponding to the target face image after perturbation should have the largest possible distance from the corresponding face feature before perturbation, and the smallest possible distance from a specific face feature.
[0070] Step 2.3: In the process of searching for global perturbations, printing loss also needs to be considered. Printing loss comes from image compression and RGB encoding to CMYK format. The existence of printing loss will affect the privacy protection effect to a certain extent. By using the generative model CycleGAN to realize the mutual conversion between RGB images and newspaper print images, printing loss can be reduced.
[0071] The global adversarial perturbation search can be modeled as three loss functions:
[0072] (1) loss1=r(τ1+cos(f(g(x)) s +δ)),f(x s The purpose of loss1 is to ensure that the cosine similarity between the facial features of the target person before and after adding global perturbation is less than or equal to τ1; this can be a negative value.
[0073] (2) loss2=r(τ2-cos(f(g(x)) s +δ)),f(x t The purpose of loss2 is to ensure that the cosine similarity between the target person with added global perturbation and the facial features corresponding to a specific face image is greater than or equal to τ2.
[0074] (3) loss3=||δ|| ∞ The purpose of `loss3` is to minimize the infinite norm of the global adversarial perturbation δ. `f` is the face feature extractor, typically a deep neural network. `g` is the print loss elimination function, which includes two sub-functions: one converts the RGB-encoded image to an image of a specific publishing scene, and the other converts the image of the specific publishing scene back to an RGB-encoded image. `r` is the ReLU activation function; if the input value is greater than or equal to zero, it outputs the input itself; otherwise, it outputs the input. s and x t Let be the target person's face image and the specific person's face image, respectively, and δ be the global adversarial perturbation to be solved. The final overall loss function can be summarized as follows: loss = loss1 + loss2 + λ·loss3;
[0075] Step 3: A perturbation search is performed using an intelligent evolutionary algorithm to determine if the privacy protection goal has been achieved. Finally, an RGB image containing the global perturbation is output as the result, such as... Figure 2 As shown, the specific steps include:
[0076] Step 3.1: The perturbation search solution uses an intelligent evolution algorithm. If the cosine similarity between the facial features of the target person and the facial features of the original target person image is greater than a predetermined threshold, or if the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features of the facial image of the specific publishing scene is less than a predetermined threshold, it means that the purpose of privacy protection has been achieved.
[0077] Step 3.2: After achieving the goal of privacy protection, output an RGB image containing global adversarial perturbation as the result.
[0078] The intelligent evolution algorithm specifically includes the following steps:
[0079] (1) Initialize the number of particles to N and the number of iterations to T. Each particle has an initial value of x1 and a velocity of 0.
[0080] (2) For each iteration j from 1 to T, and for each particle i from 1 to N, calculate the loss value corresponding to the current particle i based on loss2. If the current loss value Less than or equal to the historical minimum loss value of particle i Then update the historical best position of particle i. and the corresponding minimum loss value If the current loss value Less than or equal to the historical minimum loss value for all particles g Then update the globally optimal position x. g Using the Particle Swarm Optimization (PSO) formula Where w is the inertia weight, c1 is the individual fitness acceleration factor, c2 is the social fitness acceleration factor, and rand is a random number between [0,1]. Update the velocity and position x of particle i. i ←x i +v i ;x i It is the position of particle i, v i It is the position update amount of particle i;
[0081] (3) If the loss value of a particle reaches the preset threshold, loss1 = 0 or loss2 = 0, then stop the iteration and return the global optimal solution x. g This yields the final disturbed image that achieves anti-identification.
[0082] In scenarios where target person identification is prevented, perturbation deception is added to the glasses patch selected by the target person, so that the target person cannot be correctly identified by the artificial intelligence model. The specific steps include the following:
[0083] Step 4: Select a suitable patch type from the eyeglass patch library based on the target person. Utilize the facial landmark detection module to pinpoint the key points of the initially selected target person image in the anti-identification scenario, and precisely align the eyeglass patch onto the face image. This includes the following steps:
[0084] Step 4.1: The target person selects a suitable glasses-shaped patch from the patch library. The shape of this patch is consistent with the actual glasses and will eventually be printed and pasted onto the real glasses frame to help the wearer prevent being identified by the facial recognition system.
[0085] Step 4.2: Determine the key points in the target person image initially selected for the anti-identification scene using the face key point detection module. Then, select three key points on the two eyebrows and the bridge of the nose of the target person image initially selected for the anti-identification scene, as well as three key points on the corresponding glasses patch. Then, use the image transformation tool affine transformation, i.e., print and paste to align the target person image initially selected for the anti-identification scene and the glasses patch, i.e., add the glasses patch to the target person's face image.
[0086] Step 5: After aligning the target person and the glasses patch, search for the perturbation values of the pixels in the glasses frame area, taking into account the printing loss from the conversion between RGB encoding and CMYK encoding, so that the face recognition system cannot correctly identify the target person. This specifically includes the following steps:
[0087] Step 5.1: Search for the perturbation value of the pixels in the eyeglass frame area, maximize the distance between the original target face image deep feature encoding and the face image deep feature encoding after adding the adversarial glasses patch, and minimize the distance between the deep feature encoding after adding the adversarial glasses patch and the face image deep feature encoding of another selected person, so that the face recognition system cannot recognize the target person normally.
[0088] Step 5.2: Consider printing losses, including quantization losses, i.e., the losses caused by quantization of RGB encoded pixel values and the losses caused by RGB to CMYK conversion of the image. Add the two printing losses to the process of solving adversarial perturbations to reduce the degradation of the protection performance of the solved perturbations in real environment.
[0089] The algorithm for searching the pixel area of the glasses frame can be formally expressed as the following three losses:
[0090] (1) loss4=r(τ1+cos(f(g(x)) s ·(1-m)+m·δ),f(x s The loss 4 ensures that the cosine similarity between the facial features of the target person before and after adding the local perturbation is less than or equal to τ1.
[0091] (2) loss5=r(τ2-cos(f(g(x)) s ·(1-m)+m·δ),f(x t The loss 5 ensures that the cosine similarity between the facial features of the target person with added local adversarial perturbation and the facial features corresponding to the facial image in the specific publishing scene is greater than or equal to τ2.
[0092] (3) loss3=||δ|| ∞ The role of loss3 is to minimize the infinite norm of the global adversarial perturbation δ.
[0093] Step 6: Print the RGB encoded image of the glasses patch obtained from the optimization solution, ensuring that the size of the glasses patch is close to the size of the actual glasses frame, and that the image size remains close to the real face during the adversarial optimization process, thus effectively protecting the privacy and security of the target person. Figure 3 As shown, the specific steps include:
[0094] Step 6.1: Print the RGB-encoded image of the eyeglass patch obtained through the above optimization process. To ensure the best results, the size of the printed patch should be as close as possible to the size of the actual eyeglass frame.
[0095] Step 6.2: During the adversarial optimization process, ensure that the image size matches the real face to reduce the potential impact of interpolation on the protection effect. After printing, the target person can effectively defend against facial recognition systems by wearing glasses containing this special patch.
[0096] This method has the following characteristics:
[0097] 1. Multi-scenario privacy protection: By combining the needs of different publishing scenarios, robust global perturbations are generated to ensure that the target person's image can be effectively prevented from being maliciously obtained and identified in various environments, thus achieving comprehensive privacy protection.
[0098] 2. Flexible Adaptability: By employing specific perturbation and deception strategies on the target person's glasses, the system can quickly adapt to different recognition scenarios without requiring complex adjustments for each scenario, thus enhancing its adaptability in complex environments.
[0099] 3. Highly Effective Anti-Identification: By utilizing a perturbation and deception mechanism, the system can effectively prevent target individuals from being identified by artificial intelligence models in various practical application scenarios, ensuring the security of privacy information.
[0100] 4. Strong scalability: By adjusting the perturbation generation method or optimizing model parameters, the system can adapt to different types of privacy protection needs and supports large-scale privacy protection and anti-identification applications for target information.
[0101] This application presents a privacy protection system for personal information based on image perturbation and physical occlusion. This system boasts advantages such as high-efficiency privacy protection, rapid anti-identification adaptability, powerful image analysis capabilities, and good scalability. The implementation of this method will significantly improve the efficiency and accuracy of privacy protection and anti-identification of target personal images, providing strong technical support and innovative directions for privacy protection and security in facial recognition technology.
[0102] The personal information privacy protection system proposed in this application combines a target person image privacy protection module, a target person anti-identification module, and perturbation generation technology to achieve precise image privacy protection, implementation of dynamic anti-identification strategies, and in-depth optimization of privacy protection effects in multiple scenarios. This system can efficiently and accurately protect and prevent facial information from being identified, providing strong privacy protection, risk prevention, and multi-scenario adaptability for facial recognition technology.
[0103] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.
Claims
1. A method for protecting the privacy of persons in multiple physical scenarios based on image perturbation occlusion, wherein the method is applied to real physical environments: a target person image privacy protection scenario and a target person anti-identification scenario, characterized in that: The method for protecting the privacy of personal information in multiple physical scenarios specifically includes the following steps: In scenarios involving the privacy protection of images of individuals, global adversarial perturbations are generated based on the context in which the images are published to prevent malicious acquisition of these images from being identified by artificial intelligence models. Specifically, this includes: Step 1.1: Determine the context in which the target person's image will be published; Step 1.2: After determining the release scenario, use the particle swarm optimization algorithm to perform a global adversarial perturbation search based on the target person's image. The search target is the features of the target person's image. The global adversarial perturbation search process is modeled as an optimization problem. Step 1.3: Add invisible global adversarial perturbations to the target person's image published by the target person, and calculate the printing loss. The printing loss comes from the compression of the target person's image and the conversion of RGB encoding to CMYK format, realizing the mutual conversion between RGB images and the printed images of the publishing channel, and compensating for the distortion of the target person's image caused by the printing process. Step 1.4: Determine if the privacy protection goal has been achieved. If the privacy protection goal has not been achieved, repeat step 1.
2. Step 1.3: If the privacy protection objective is achieved, output an RGB image containing global adversarial perturbation as the result; In scenarios where target person identification is prevented, perturbation deception is added to the glasses patch selected by the target person, so that the target person cannot be correctly identified by the artificial intelligence model. The specific steps include the following: Step 2.1: The target person selects the patch type from the eyeglass patch library, uses the facial key point detection module to select three key points: the center of the two brow bones and the bridge of the nose, and aligns the selected eyeglass patch to the target person image initially selected in the anti-identification scene using affine transformation based on the selected key points. Step 2.2: After aligning the target person image and the glasses patch initially selected in the anti-identification scenario, search for the perturbation value of the pixels in the frame area of the glasses worn by the target person. After optimization, obtain the RGB encoded image of the glasses patch. The target person can effectively defend against the identification of the face recognition system by wearing glasses with glasses patch. Step 2.3: Print the RGB encoded image of the glasses patch obtained from the optimization solution to ensure that the size of the glasses patch is close to the size of the real glasses frame, and that the image size is kept close to the real face in the global anti-perturbation solution, so as to effectively protect the privacy and security of the target person.
2. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 1, characterized in that: Step 1.2 specifically includes the following steps: Step 1.2.1: After determining the publishing channel, perform a global adversarial perturbation search and add global adversarial perturbations to the target person's image published by the target person so that the target person's image published by the target person cannot be recognized normally by the face recognition system. Step 1.2.2: Model the global adversarial perturbation search process as an optimization problem; Step 1.2.3: Printing loss comes from the compression of the target image and the conversion of RGB encoding to CMYK format. By generating a CycleGAN model, the mutual conversion between RGB images and printed images for distribution channels is realized, thereby reducing printing loss.
3. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 2, characterized in that: In step 1.2.2, the global adversarial perturbation search process is modeled as three loss functions: loss1=r(τ1+cos(f(g(x s +δ)),f(x s ))) loss2=r(τ2-cos(f(g(x s +δ)),f(x t ))) loss3=||δ|| ∞ Where loss1 ensures that the cosine similarity between the facial features of the target person image and the facial features of the target person after adding global adversarial perturbation is less than or equal to τ1; loss2 ensures that the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features corresponding to the facial image of the specific publishing scene is greater than or equal to τ2; loss3 is the infinite norm that minimizes the global adversarial perturbation δ; r is the ReLU activation function; f is the facial feature extractor; g is the print loss elimination function, which includes converting the RGB encoded image to the facial image of the specific publishing scene and converting the facial image of the specific publishing scene back to the RGB encoded image; δ is the global adversarial perturbation; and x... s For the target person image, x t For a specific publishing scenario, τ1 is the upper limit of the similarity threshold with the target task's face, and τ2 is the lower limit of the similarity threshold with the face incorrectly identified by the face recognition system. The final overall loss function is: loss = loss1 + loss2 + λ·loss3 Where λ is the loss importance balancing factor.
4. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 3, characterized in that: Step 1.4 specifically includes the following steps: Step 1.4.1: The global adversarial perturbation search solution uses an intelligent evolution algorithm. If the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features of the original target person image is greater than a predetermined threshold, or the cosine similarity between the facial features of the target person after adding global adversarial perturbation and the facial features of the facial image of the specific publishing scene is less than a predetermined threshold, it means that the purpose of privacy protection has been achieved. Step 1.4.2: After achieving the goal of privacy protection, output an RGB image containing global adversarial perturbation as the result; Step 1.4.3: If the goal of privacy protection is not achieved, repeat steps 1.2-1.3 until the goal of privacy protection is achieved.
5. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 4, characterized in that: The intelligent evolution algorithm mentioned in step 1.4.1 specifically includes the following steps: Step 1.4.1.1: Initialize the number of particles to N and the number of iterations to T. Each particle has an initial value of x1 and a velocity of 0. Step 1.4.1.2: For each iteration j from 1 to T, and for each particle i from 1 to N, calculate the loss value corresponding to the current particle i based on loss2. If the loss value corresponding to the current particle i Less than or equal to the historical minimum loss value of particle i Then update the historical best position of particle i. and historical minimum loss value If the loss value corresponding to the current particle i Less than or equal to the global history minimum loss value of all particles g Then update the globally optimal position x. g Particle swarm optimization is used. Where w is the inertia weight, c1 is the individual fitness acceleration factor, c2 is the social fitness acceleration factor, and rand is a random number between [0,1] to update the velocity and position x of particle i. i ←x i +v i x i It is the position of particle i, v i It is the position update amount of particle i; Step 1.4.1.3: If the loss value of any particle reaches a preset threshold, loss1 = 0 or loss2 = 0, then stop the iteration and return to the global optimal position x. g .
6. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 1, characterized in that: Step 2.1 specifically includes the following steps: Step 2.1.1: The target person selects an eyeglasses patch from the patch library. The shape of the eyeglasses patch is consistent with the actual eyeglasses worn by the target person. The eyeglasses patch is printed onto the actual eyeglasses worn by the target person. After printing out the lens patch, it is pasted onto the actual eyeglasses frame to prevent the target person from being identified by the facial recognition system. Step 2.1.2: Determine the key points in the target person image initially selected for the anti-identification scene using the face key point detection module. Then, select three key points on the two eyebrows and the bridge of the nose of the target person image initially selected for the anti-identification scene, as well as three key points on the corresponding glasses patch. Then, use the image transformation tool affine transformation, i.e., print and paste to align the target person image initially selected for the anti-identification scene and the glasses patch, i.e., add the glasses patch to the target person's face image.
7. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 6, characterized in that: Step 2.2 specifically includes the following steps: Step 2.2.1: Search for the perturbation value of the pixels in the frame area of the glasses worn by the target person, maximize the distance between the depth features of the original target face image and the depth features of the face image after adding the adversarial glasses patch, and minimize the distance between the depth features after adding the adversarial glasses patch and the depth features of the face image of another selected person, so that the face recognition system cannot recognize the target person normally. Step 2.2.2: Consider the quantization loss, namely the printing loss caused by quantization of RGB encoded pixel values and the printing loss caused by the RGB to CMYK conversion of the image, and add the two printing losses to the process of solving the global perturbation resistance.
8. The method for protecting the privacy of physical multi-scene personal information based on image perturbation occlusion according to claim 1, characterized in that: In step 2.2.1, the perturbation value of the pixels in the search glasses patch area is represented by three losses: loss4=r(τ1+cos(f(g(x s ·(1-m)+m·δ)),f(x s ))) loss5=r(τ2-cos(f(g(x s ·(1-m)+m·δ)),f(x t ))) loss3=||δ|| ∞ Among them, loss4 ensures that the cosine similarity between the facial features of the target person before and after adding local perturbation is less than or equal to τ1, loss5 ensures that the cosine similarity between the facial features of the target person after adding local adversarial perturbation and the facial features of the facial image corresponding to the specific release scene is greater than or equal to τ2, loss3 minimizes the infinite norm of the global adversarial perturbation δ, and m is the masking matrix, i.e. the eyeglass patch region of the image.