Physical privacy protection light field modulation device and method

Abstract

The present invention discloses a privacy protection device and method based on physical light field modulation, which belongs to the field of optical devices and data security technology. In view of the problem that the existing privacy protection scheme relies on software algorithms, resulting in the risk of retaining original data, the device selectively attenuates the spatial frequency of the incident light field through an optical modulation unit, and implements irreversible destruction of high-frequency sensitive features (such as facial texture), while retaining low-frequency contour features (such as human body posture) to maintain machine vision functions. Users directly perceive that their privacy is protected through the optical phenomena of the physical appearance of the modulator (atomized surface, diffraction spot) without relying on electronic signal verification. This solution can effectively resist GAN data reconstruction attacks, is compatible with mainstream image sensors, and is suitable for scenarios such as smart security and medical image acquisition that require both privacy and machine vision.

Description

Technical Field

[0001] The present invention belongs to the intersection of optical devices and data security technology, and specifically relates to a physical privacy protection light field modulation device and method. The device is based on an optical modulation unit on the surface or inside of a light-transmitting substrate, and irreversibly attenuates the high-frequency components of the incident light field through single or composite optical effects including reflection, scattering, and diffraction, while retaining the low-frequency contour features, thereby achieving irreversible desensitization of the original image data at the physical optical level. The device can be directly integrated into camera modules, biometric terminals and other equipment, and is suitable for scenarios such as intelligent security systems, autonomous driving environment perception modules, and medical image acquisition terminals that require hardware-level privacy protection and maintain machine vision functions. Background Art

[0002] With the widespread application of image sensing technology in smart security, smart home and other fields, camera equipment has been deeply integrated into social production and family life. However, the current mainstream privacy protection technology solutions have the following systematic technical defects.

[0003] Risk of raw data leakage 1. Software-layer attacks expose flaws: Privacy protection methods based on software algorithms (such as encrypted data transmission and image blurring) cannot eliminate the risk of original data being retained in device memory. Attackers can obtain unprocessed original data through man-in-the-middle attacks and memory injections. The blurred image is reconstructed using a generative adversarial network (GAN), and the peak signal-to-noise ratio (PSNR) of the reconstructed image can reach more than 28dB (close to the 30dB threshold of the original image quality), resulting in a significantly increased risk of leakage of biometric information such as face and iris; 2. Firmware vulnerabilities lead to hardware layer penetration: Undisclosed vulnerabilities (0day vulnerabilities) in device firmware may be exploited to intercept raw video streams. A typical example is the camera firmware vulnerability CVE-2017-79**, where attackers can bypass authentication and obtain high-definition video streams through the camera backdoor.

[0004] Lack of user trust mechanism 1. Trust crisis caused by software dependence: Existing solutions rely on the software implementation and legal compliance supervision of device manufacturers, and users cannot verify the authenticity of the privacy protection status through physically perceptible means; 2. Tamperability of electronic feedback signals: Traditional electronic indicators (such as LED indicators) can tamper with signal output through software and cannot provide irrefutable proof of privacy status.

[0005] Functional synergy contradiction 1. Deterioration of machine vision performance: Software blurring algorithms often over-destruct low-frequency components of images when removing sensitive information. This can lead to a significant drop in the accuracy of machine vision tasks such as fall detection and gesture recognition; 2. Functional deprivation defects of existing hardware solutions: Although commercially available physical shielding privacy protection devices (such as mechanical shutters) can cut off optical signal collection, they will completely deprive the camera of its environmental perception capabilities and cannot meet the continuous operation requirements of smart devices.

[0006] System resource occupation flaw 1. Computing power squeeze and performance degradation: Software desensitization algorithms (such as real-time face blurring and dynamic encryption) require more than 20% of the device's CPU / GPU computing power (based on TensorRT deployment testing), resulting in an increase of 30-100ms in image processing latency, causing video freezes, inaccurate target tracking, and other issues; 2. Energy efficiency degradation and hardware loss: Continuously running the privacy protection algorithm increases device power consumption by 15%-25% (measured on HiSilicon Hi3516DV300 chipset), accelerates battery aging and generates abnormal heat (housing temperature ≥50°C); 3. Insufficient multi-task compatibility: In edge computing devices, privacy algorithms and machine vision tasks (such as behavior recognition) need to share limited computing power, resulting in a decrease in the overall system response speed (refer to the ROS robot platform benchmark).

[0007] Technical scalability limitations 1. Algorithm iteration lag: Software solutions need to be continuously upgraded to deal with new attacks (such as deep learning reconstruction algorithms), but the terminal device firmware update coverage is insufficient, resulting in long-term security vulnerabilities (data from the 2018 IoT Security White Paper); 2. Cross-platform adaptation cost: Different hardware platforms (ARM / x86 / FPGA) require targeted algorithm optimization, and the cross-platform porting development cost of a single privacy protection solution is too high.

[0008] Optical device development is costly and inefficient 1. Long R&D cycle: Usually the development cycle is 6-8 weeks, and a fixed attenuation rate is selected based on experience (such as unified fuzzy parameters); 2. Single simulation verification mechanism: such as Zemax geometric optics simulation, which is not linked with image processing effects; 3. Low efficiency: manual trial and error adjustment, relying on engineer experience and cannot be reproduced; 4. The target path is simple: it relies on manual lookup or finite element simulation, does not consider manufacturing process constraints, and the manufacturing equipment requires more debugging.

[0009] Summary of technical bottlenecks In the existing technology, there is no solution that can simultaneously meet the following core requirements: 1. Realize irreversible desensitization of sensitive data at the hardware optical level (PSNR≤20dB); 2. Providing a self-verification mechanism for privacy status through physically observable features; 3. Maintain the integrity of contour features required for machine vision functions (recognition accuracy ≥ 85%); 4. Eliminate the additional computing load and energy consumption caused by privacy protection; 5. The R&D cost of new scenarios is low, the efficiency is high, and the results are positively promoted; This multi-objective coordination problem has become a key obstacle to the development of high-reliability privacy protection technology. Summary of the invention

[0010] The present invention provides a physical privacy protection light field modulation device and method, which flexibly adapts the privacy protection level of different application scenarios through light field frequency band modulation technology, irreversibly desensitizes sensitive information at the hardware optical level, retains machine vision functions, and establishes a user trust mechanism through physically observable features. At the same time, it provides a research and development method for applying the physical privacy protection light field modulation device to different target fields.

[0011] Technical Solution Physical privacy protection light field modulation device: including a light-transmitting substrate and an optical modulation unit arranged on the surface or inside thereof, and having the following characteristics:

[0012] Optical modulation unit 1. Based on the optical microstructure (including frosting, prism array, metasurface, etc.) on the surface or inside of the light-transmitting substrate (including glass, resin or plastic, etc.), the incident light field is selectively attenuated in spatial frequency by using single or composite effects including reflection, refraction, scattering, diffraction, interference or polarization; 2. Privacy level definition: Multi-level privacy protection is achieved by differentially adjusting the attenuation intensity of high-frequency and low-frequency light field components.

[0013] User verifiable identification 1. The optical characteristics of the modulator surface (such as the degree of atomization and the regularity of the diffraction pattern) are positively correlated with the level of privacy protection: a). The higher the privacy level, the more significant the surface atomization effect and the more disordered the diffraction pattern; b). The lower the privacy level, the stronger the surface transmittance and the more regular the diffraction pattern; 2. Users can directly verify the privacy protection status by observing the surface features with the naked eye without relying on electronic signals.

[0014] Method flow 1. Scenario adaptation design: Select the corresponding optical microstructure type and parameter combination according to the target privacy strength requirements; 2. Light field processing: The incident light field is selectively attenuated in the frequency domain through the optical structure, and data desensitization is completed in the optical propagation stage; 3. Machine vision compatibility: control the attenuation amplitude of low-frequency components and maintain the integrity of the gradient information of contour features; 4. Hardware status self-verification: Based on the correlation between surface optical features and privacy levels, it provides tamper-proof functional verification.

[0015] Advantages of Physical Privacy Protection Light Field Modulation Device Technical Dimension Disadvantages of traditional solutions Innovation of the present invention Privacy level flexibility Can only be completely blocked or software defined Achieving multiple privacy levels through optical structure parameter combination Data Irreversibility Software encryption can be attacked through side-channel attacks High-frequency light field physical destruction (irreversible desensitization) System resource consumption Software algorithms squeeze out the main business computing power and increase power consumption Does not consume hardware system resources and does not increase power consumption Functional compatibility Software blur leads to poor machine vision performance The attenuation rate of low-frequency light field is controlled within the functional threshold User Trust Anchor LED indicators are susceptible to tampering Surface characteristics directly correlate to privacy levels Industrialization cost Increased costs due to the addition of mechanical parts or algorithms Simple structure, little intrusion on the original design, low cost

[0016] Advantages of the physical privacy protection light field modulation device development method 1. Demand-driven parameter setting: Decoupling control of high-frequency / low-frequency attenuation rates to match privacy and functional balance requirements, breaking through the traditional "one-size-fits-all" model and making parameters interpretable; 2. Close the loop of light field simulation, image processing, and machine vision verification to establish a quantitative relationship between optical parameters and AI task performance, and solve the problem of cross-domain effect verification; 3. Automated parameter iteration: Improve R&D efficiency through dynamic convergence of PSNR and accuracy. 4. Based on the optical and process joint derivation of large models, the manufacturable microstructure parameters are output, and the empirical process knowledge is encoded into model parameters, breaking through the traditional design and manufacturing separation problem; 5. Establish multi-dimensional mass production standard inspection of "optical parameters, machine vision performance, and user perception identification". BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings of the embodiment of the present invention: Figure 1 Schematic diagram of a privacy protection light field modulation device. 101 represents an optical modulation unit, and 102 represents incident light. When the incident light passes through the optical modulation unit 101, light field modulation occurs. The modulated outgoing light 103 has completed the privacy desensitization process, and the outgoing light at this time can enter the image acquisition device; Figure 2 A flow chart for developing and creating a light field modulation device. This diagram shows the process of developing and creating a new optical modulation device; Figure 3 The whole body of a human body is sampled at a high privacy protection level, with a resolution of 640*480, to illustrate the actual use effect of the physical privacy protection light field modulation device at a high privacy protection level; Figure 4 This is a close-up sampling of half of the human body at a high privacy protection level, with a resolution of 640*480, to illustrate the actual use effect of the physical privacy protection light field modulation device at a high privacy protection level. DETAILED DESCRIPTION

[0018] The optical modulation device development process of the present invention is as follows Figure 2 As shown in the figure, the core is to integrate optical simulation, machine learning verification and large model parameter derivation to achieve efficient and low-cost device design and manufacturing. The specific implementation steps are as follows:

[0019] Step 1: Demand parameter modeling Define the target optical parameter set: in: 1. : High frequency attenuation rate (@40lp / mm); 2. : Low frequency attenuation rate (@20lp / mm); 3. : Peak signal-to-noise ratio of the reconstructed image (privacy protection strength); 4. : Average accuracy of human posture detection (functional compatibility).

[0020] Step 2: Cross-domain co-simulation Use frequency domain filtering algorithm (based on OpenCV) or ray tracing method (based on MATLAB Optic Toolbox) to construct the light field modulation transfer function:

[0021] in: ; ; ; .

[0022] Step 3: Multi-dimensional effect verification Machine vision verification: Test human posture detection mAP@0.5≥85% and face recognition error rate FAR≥95% on the COCO dataset; Privacy protection verification: Calculate the peak signal-to-noise ratio of the modulated image:

[0023] Step 4: Smart parameter iteration Establish parameter optimization objective function: , adjusted by gradient descent until , convergence.

[0024] Derivation of optical parameters The qualified parameters are input into the optical parameter prediction model (based on the Transformer architecture) and the physical realization parameter set is output: Model training data: historical optical device library (including group process and performance mapping data).

[0025] Process Adaptation Manufacturing 1. According to Select manufacturing process: a). Frosted surface: chemical etching (HF solution concentration 5%-15%, time 30-90s) b). Prism array: nanoimprinting (pressure 10-20MPa, temperature 150-180°C) 2. Execute SPC (Statistical Process Control), key parameter fluctuation range:

[0026] Actual effect verification 1. Use instruments and machines to measure whether the required parameter indicators can be met; 2. Use the device to take a photo and input the photo into the function in step 3 for machine vision verification.

[0027] Closed-loop feedback optimization a). If the device sample test fails to meet the standard, the manufacturing data (process parameters + measured performance) will be returned to the model in step 5 for online fine-tuning: in: is the base model parameter distribution; is the knowledge retention coefficient; b). If the device sample test meets the standards, the optimized parameters will be added to the training set to trigger the incremental learning of the model.

[0028] Example Frosted surface implementation solution Step 1: Optical modulator preparation Substrate selection: polymethyl methacrylate (PMMA) resin plate, thickness 2 mm, refractive index n = 1.49 (@550 nm); Surface treatment: A chemical etching process is used to form a random rough structure on the substrate surface; Roughness parameter: Ra=1.2μm (measured according to ISO4287 standard); Step 2: Optical performance test Spatial frequency response test: Test equipment: ISO12233 resolution test card + IoT low-cost image sensor (GC0308); High frequency attenuation rate (@40lp / mm): 68%±2%; Low frequency attenuation rate (@20lp / mm): 28%±3%; Verification of privacy protection effect: GAN reconstruction attack: Use StyleGAN3 model to reconstruct the processed image, PSNR=19.5dB; Face recognition accuracy (LFW dataset): FRR=99.8%; Machine vision compatibility: Human posture recognition accuracy (OpenPose model): 85.3%±1.5%; Step 3: User perception verification: The surface presents a uniform atomization effect (transmittance ≤ 15%). The results of a blind test of 50 people show that the user's recognition accuracy of the privacy protection status is ≥ 96%.

[0029] Those skilled in the art will understand that: 1. The light-transmitting materials are not limited to those listed in the embodiments; 2. The specific form of the optical modulation unit can be adjusted according to the processing conditions; 3. The optical effects of the logo that users can perceive include but are not limited to atomization, diffraction, and polarization; 4. The parameters in the above research and development method are only used as examples to illustrate how to develop optical modulation devices. The materials, formulas, and parameters are included but not limited to these and can be adjusted according to actual needs.

[0030] It should be noted that the embodiments provided in this specification are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection. Those skilled in the art may select other materials, structures or parameters to achieve the same technical effects without departing from the principles of the present invention; The privacy protection function of the present invention is completely realized by the physical structural characteristics of the optical modulation unit, and has nothing to do with whether other auxiliary electronic components (such as power indicator lights) are included. Any technical solution based on physical structure to achieve selective attenuation of spatial frequency, whether or not it contains non-functional electronic modules, falls within the protection scope of the present invention; The embodiments of the present invention are not limited to the above examples. Any technical solution based on the principle of selective attenuation of spatial frequency and achieving compatibility of privacy protection and machine vision through physical optical modulation falls within the scope of protection of the claims of the present invention.

Claims

1. A physical privacy protection light field modulator, characterized in that include: a). Optical modulation structure: Made of light-transmitting material, including surface or internal optical modulation units, which selectively attenuate the incident light field in spatial frequency by using single or combined effects including reflection, refraction, scattering, diffraction, interference or polarization; b) Detail elimination capability: significantly attenuate high-frequency light field components, making the detail information of sensitive targets in the imaging data irreversibly eliminated; c) Contour preservation capability: Partially preserve the low-frequency light field components so that the machine-recognizable contour features can be maintained; d). User-perceivable identification: The physical appearance of the light field modulator is significantly different from that of a conventional lens, and directly indicates the privacy protection status through optical effects (such as fogging, diffraction, and polarization).

2. A physical privacy protection light field modulator as claimed in claim 1, characterized in that: The optical modulation unit realizes spatial frequency selective attenuation through a differentiated design of high-frequency attenuation and low-frequency retention.

3. A physical privacy protection light field modulator as claimed in claim 1, characterized in that: The light-transmitting material includes glass, resin, plastic or metamaterial.

4. A physical privacy protection light field modulator as claimed in claim 1, characterized in that: The optical modulation structure includes a frosted surface, a prism array, a diffraction grating or a metasurface.

5. A physical privacy protection light field modulator as claimed in claim 1, characterized in that: The high-frequency light field components correspond to sensitive biometric features (such as facial texture and iris details), and the low-frequency light field components correspond to machine-recognizable contours (such as human posture and gestures).

6. A method for developing a physical privacy protection light field modulator, characterized in that The following steps are involved: a). Demand parameter setting: determine the target optical modulation parameters, including high-frequency attenuation rate threshold, low-frequency attenuation rate threshold and machine vision function compatibility index; b) Optical simulation verification: Based on the target optical modulation parameters, the original image is subjected to light field modulation simulation processing to generate a modulated image; c) Verification of privacy protection effect: Calculate whether the irreversible elimination index of sensitive information of the modulated image and the accuracy of the machine vision task meet expectations; d). Parameter iteration optimization: If the irreversible elimination index or the machine vision accuracy does not reach the threshold, adjust the high-frequency / low-frequency attenuation rate parameters and repeat steps bc until the target is reached; e) Optical parameter derivation: Input the qualified optical modulation parameters into the optical parameter derivation model to generate the corresponding set of physical realization parameters, including the type of transparent substrate, surface microstructure process (such as frosting, prism array, metasurface) and microstructure geometric parameters; f). Device preparation and testing: Prepare light field modulator samples according to the physical realization parameters, test their actual spatial frequency response curves, and ensure that the high-frequency attenuation rate and the low-frequency attenuation rate meet expectations; g) Mass production decision: If the measured parameters meet the target threshold, mass production is carried out based on the physical realization parameters, and the parameters are added to the knowledge base; if not, return to step e to optimize the optical parameter derivation model and regenerate the parameters.

7. The research and development method according to claim 8, characterized in that: The light field modulation simulation process in step b is implemented in the following manner: a). Construct an optical transfer function model based on the principle of spatial frequency selective attenuation; b). Use frequency domain filtering algorithm to suppress high frequencies and retain low frequencies of the original image.

8. The research and development method according to claim 8, characterized in that: The machine vision tasks described in step c include human posture recognition, gesture interaction and fall detection, and their accuracy tests are verified based on standard data sets of StyleGAN3, YOLOv5, OpenPose and FaceNet models.

9. The research and development method according to claim 8, characterized in that: The training method of the optical parameter derivation model in step e includes: a). Input layer: high frequency attenuation rate, low frequency attenuation rate, and transmittance requirements; b). Output layer: material refractive index range, surface microstructure type and size parameters; c). Training data: the mapping relationship between the historical optical device parameter library and the measured performance indicators.

10. The research and development method according to claim 8, characterized in that: The mass production decision in step g includes: a). Conduct user-perceivable logo adaptability test on qualified samples to ensure that the visual correlation error of atomization, diffraction pattern and privacy level is ≤5%; b). Establish the tolerance range of mass production process parameters to meet the high-frequency attenuation rate fluctuation ≤±2% and the low-frequency attenuation rate fluctuation ≤±1.5%; c). Once the requirements are met, the optimized parameters are added to the training set to trigger incremental learning of the model.

Images