A camera with privacy protection function and its privacy protection method

Abstract

This invention discloses a camera with privacy protection functions and its privacy protection method. The camera includes an image acquisition module, a parameter storage module, an access control module, and an image post-processing module. The goal of the privacy protection method is to enable the images captured by the camera to inherently possess privacy protection capabilities to combat malicious third-party sniffing of personal privacy, including but not limited to facial recognition. The system deeply participates in the image generation process at multiple levels, influencing various stages including image signal processing, image color adjustment, and image encoding, providing a platform for the research and application of underlying privacy protection algorithms.

Description

Technical Field

[0001] This invention belongs to the field of image privacy protection, and relates to a camera with privacy protection function and a privacy protection method thereof. Background Technology

[0002] With the widespread application of intelligent technologies such as image recognition, cameras are now virtually ubiquitous, used in areas including video surveillance, health monitoring, autonomous driving, and human-computer interaction. For example, the number of surveillance cameras worldwide exceeded 1 billion in 2021, and the market size is projected to reach $57.9 billion by 2024. While image intelligence applications have brought convenience to people's lives, the privacy information captured by cameras could be maliciously used by third parties for criminal purposes, raising concerns about privacy violations related to cameras.

[0003] Most existing methods require passing images containing sensitive privacy information from the camera to a software layer for de-identification. However, the original image may be leaked at various points in the data transmission chain, and the purely software-based processing can be easily bypassed by malicious programs. Therefore, how to reliably protect image privacy is a pressing technical problem that the industry needs to solve. Summary of the Invention

[0004] This invention provides a camera with privacy protection function and its privacy protection method. The system is deeply involved in the image generation process at multiple levels and can affect various aspects including image signal processing, image color adjustment and image encoding process, providing a platform for the application and research and development of underlying privacy protection algorithms.

[0005] This invention is achieved using the following technical solution:

[0006] A privacy-protecting camera includes:

[0007] Image acquisition module: Composed of a lens, a CMOS image sensor, and an ISP (Image Signal Processor), it is used for image capture; privacy protection within the image acquisition module is achieved by adjusting adjustable parameters in the ISP;

[0008] Parameter storage module: Composed of ROM (Read-Only Memory), used to store privacy protection algorithms and privacy protection parameters; the privacy protection algorithms and privacy protection parameters are obtained through computer simulation experiments; the ROM in the parameter storage module is not allowed to be modified by users after the data is burned; the contents of the read-only memory cannot be modified and can only be read by the internal components of the camera;

[0009] Access control module: Composed of MCU (Micro-Controller Unit), used to read privacy protection parameters from the parameter storage module and issue shooting commands to the image acquisition module; the access control module blocks external devices from accessing any internal registers, memory, or storage; only the access control module can communicate with external devices, and only for shooting-related command interaction and image data transmission. External devices cannot obtain the output of the internal CMOS image sensor or other image data that has not been fully privacy protected;

[0010] Image post-processing module: Composed of DSP (Digital Signal Processor) or other high-efficiency image data processors (e.g. GPU (Graphics Processing Unit) and NPU (Neural-Network Processing Unit)), it is used to execute privacy protection algorithms to process the captured images, and then transmit them to the access control module, and finally to external devices.

[0011] The camera only transmits privacy-protected image data to external devices. Here, image data refers to a single image or a video stream composed of multiple images, and the image data is encoded in a common image / video format. The camera can simultaneously run multiple single-module privacy protection algorithms or multiple-module combined privacy protection algorithms.

[0012] This invention provides a privacy protection method for a camera, which is based on the aforementioned camera with privacy protection function and specifically includes the following steps:

[0013] 1) External devices via I 2 The external device sends a shooting request to the access control module via a C or other communication interface. The shooting request includes a series of parameters, including shooting mode selection, image pixel size, and image shooting frame rate. Parameters related to privacy protection are not allowed to be set.

[0014] 2) After receiving a shooting request, the access control module reads privacy protection parameters from the parameter storage module. These privacy protection parameters include existing ISP module parameters in the image acquisition module and DSP processing algorithm parameters in the image post-processing module. These parameters are pre-calculated using the privacy protection algorithm. The access control module sets the ISP parameters in the image acquisition module and the DSP parameters in the image post-processing module via the internal bus. The access control module then enables and sends a shooting command to the image acquisition module.

[0015] 3) After receiving the shooting instruction from the access control module, the image acquisition module performs an image capture, generates the image (photo / video) corresponding to the shooting instruction, and transmits the image data stream to the image post-processing module via the bus;

[0016] 4) The image post-processing module performs privacy protection processing on the received video / photo, which may include pixel value modification at the pixel level, frequency domain processing at the image level, and spatiotemporal domain comprehensive processing at the video level. After JPEG encoding and compression, the data is transmitted to the access control module, which then transmits the image data stream to an external device via MIPI CSI or other image transmission interfaces. The privacy protection algorithm executed by the image post-processing module is characterized by low additional computational load, short processing time, low latency, and no impact on the normal use of external devices.

[0017] The beneficial effects of this invention are:

[0018] (1) This invention participates in the image generation process at multiple levels, providing a platform for the research and development and application of underlying privacy protection algorithms;

[0019] (2) This invention affects the capture of images from the hardware level and can deploy a variety of privacy protection algorithms, such as ISP privacy parameters, digital adversarial samples, image frequency domain watermarking, etc., with wide applicability;

[0020] (3) The present invention burns privacy protection parameters into a read-only memory so that they cannot be changed by external devices. Even if the external system is manipulated by a malicious third party, it cannot obtain the original image, which is highly secure.

[0021] (4) This invention can be used as a complete module to directly replace the unsafe cameras in existing video application systems, and is highly practical. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the camera components in an embodiment of the present invention.

[0023] Figure 2 This is a flowchart of a privacy protection method for a camera according to an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of ISP privacy protection parameters in an embodiment of the present invention.

[0025] Figure 4 This is a flowchart of a privacy protection anti-noise-adding procedure in an embodiment of the present invention.

[0026] Figure 5 This is a flowchart of a privacy-protecting frequency domain watermarking procedure, as exemplified by an embodiment of the present invention. Detailed Implementation

[0027] To facilitate understanding and implementation of this invention by those skilled in the art, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the invention.

[0028] This invention provides a camera with privacy protection function and a method for protecting privacy therewith. Figure 1 A schematic diagram illustrating the camera components of this implementation example is shown. For example... Figure 1 As shown, the privacy-protecting camera consists of four modules, including:

[0029] 1) Image acquisition module: It consists of a lens, a CMOS image sensor and an ISP (Image Signal Processor). Privacy protection within the image acquisition module is achieved by adjusting adjustable parameters in the ISP.

[0030] 2) Parameter storage module: Composed of ROM (Read-Only Memory), it stores the privacy protection algorithm and privacy protection parameters. Once the data is programmed into the ROM of the parameter storage module, it cannot be modified by the user.

[0031] 3) Access Control Module: Composed of an MCU (Micro-Controller Unit), this module reads privacy protection parameters from the parameter storage module and sends shooting commands to the image acquisition module. Only the access control module can communicate with external devices, and this communication is limited to shooting-related command interaction and image data transmission.

[0032] 4) Image post-processing module: Composed of DSP (Digital Signal Processor) or other high-efficiency image data processors (e.g., GPU (Graphics Processing Unit) and NPU (Neural-Network Processing Unit)), it is used to execute image privacy protection algorithms to protect the privacy of the image.

[0033] Figure 2 A flowchart illustrating the privacy protection method for the camera in this implementation example is shown. Figure 2 As shown, the image capture process of a camera device with privacy protection features includes the following steps:

[0034] 1) External devices via I 2The external device sends a shooting request to the system's access control module via a C or other communication interface. The shooting request includes a series of parameters, including shooting mode selection, image pixel size, and image shooting frame rate; parameters related to privacy protection are not allowed to be set.

[0035] 2) After receiving a shooting request, the access control module reads privacy protection parameters from the parameter storage module. These privacy protection parameters include existing ISP module parameters in the image acquisition module and DSP processing algorithm parameters in the image post-processing module. These parameters are pre-calculated using the privacy protection algorithm. The access control module sets the ISP parameters in the image acquisition module and the DSP parameters in the image post-processing module via the internal bus. The access control module then enables and sends a shooting command to the image acquisition module.

[0036] 3) After receiving the shooting instruction from the access control module, the image acquisition module performs an image capture, generates the image (photo or video) corresponding to the shooting instruction, and transmits the image data stream to the image post-processing module via the bus;

[0037] 4) The image post-processing module performs privacy protection processing on the received video / photo, which may include pixel value modification at the pixel level, frequency domain processing at the image level, and spatiotemporal domain comprehensive processing at the video level. After JPEG encoding and compression, the data is transmitted to the access control module, which then transmits the image data stream to an external device via MIPI CSI or other image transmission interfaces. The privacy protection algorithm executed by the image post-processing module is characterized by low additional computational load, short processing time, low latency, and no impact on the normal use of external devices.

[0038] Figure 3 This diagram illustrates the ISP privacy protection parameters in the image acquisition module. For example... Figure 3 As shown, the ISP input is the raw image output from the CMOS image sensor, and the output is the processed image. The ISP contains a series of cascaded image processing flows, including black level correction, lens shading correction, RAW domain noise reduction, RGB interpolation, gamma correction, color correction, and edge enhancement. The ISP's internal image processing flows have adjustable parameters to control key attributes such as degree, direction, and type. By appropriately adjusting these ISP adjustable parameters, privacy protection goals can be partially achieved. The specific values ​​of these adjustable parameters are derived by optimization algorithms for different privacy protection targets and written to the read-only memory of the parameter storage module. Privacy protection parameters are directly set in the ISP via the DMA (Direct Memory Access) channel in the MCU.

[0039] Figure 4The flowchart of the DSP privacy protection and anti-noise enhancement procedure in the image post-processing module is shown. Figure 4 As shown, the DSP first reads adversarial noise from the read-only memory. This adversarial noise is derived by an optimization algorithm for different privacy protection objectives and has a certain degree of universality. Then, adversarial noise is superimposed on the input image data stream, making the image data difficult for the AI ​​algorithm to recognize. Finally, random image noise is superimposed on the image to obfuscate the adversarial noise, and the output image data stream is generated.

[0040] Figure 5 The flowchart of the DSP privacy-preserving frequency domain watermarking program in the image post-processing module is shown. Figure 5 As shown, the DSP first reads the image watermark pattern from the read-only memory. This watermark pattern embeds a device identifier for tracing purposes after a privacy breach. Then, the input image data stream is converted to the frequency domain using a DCT transform. Next, the watermark pattern is added to the high-frequency portion of the image in the frequency domain, and then the watermarked frequency domain image is converted back to the image domain using an inverse DCT transform, outputting the image data stream.

[0041] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0042] It should be understood that the above description of the embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art can make substitutions or modifications under the guidance of this invention without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A camera with privacy protection function, characterized in that: include: Image acquisition module: Composed of a lens, a CMOS image sensor, and an image processing unit (ISP), used for image capture; Parameter storage module: Composed of read-only memory (ROM), used to store privacy protection algorithms and privacy protection parameters; Access control module: Composed of a microcontroller (MCU), used to read privacy protection parameters from the parameter storage module and issue shooting commands to the image acquisition module; the access control module blocks external devices from accessing any internal registers, memory, or storage. Image post-processing module: Composed of digital signal processor (DSP), graphics processor (GPU), and neural network processor (NPU), it is used to execute privacy protection algorithms to perform privacy protection processing on the captured images, and then transmit them to the access control module, and finally to the external device; The camera only transmits privacy-protected image data to external devices; the camera can simultaneously run multiple single-module privacy protection algorithms or multi-module combined privacy protection algorithms; the privacy protection parameters are obtained in advance by the privacy protection algorithm, including the existing module parameters of the ISP in the image acquisition module and the DSP processing algorithm parameters of the image post-processing module.

2. A method for protecting the privacy of a camera, characterized in that, Based on the privacy-protecting camera as described in claim 1, the specific steps include: 1) An external device sends a shooting request to the access control module; 2) After receiving the shooting request, the access control module reads the privacy protection parameters from the parameter storage module; the access control module sets the ISP parameters in the image acquisition module and the DSP parameters in the image post-processing module; the access control module enables and sends a shooting command to the image acquisition module. 3) After receiving the shooting instruction from the access control module, the image acquisition module performs an image capture, generates a video / photo corresponding to the shooting instruction, and transmits it to the image post-processing module; 4) The image post-processing module performs privacy protection processing on the received video / photo, then performs JPEG encoding compression, and transmits it to the access control module. Finally, the access control module transmits the video / photo data stream to the external device.

3. The privacy protection method for a camera according to claim 2, characterized in that, The shooting request from the external device does not include parameters related to privacy protection.

4. The privacy protection method for a camera according to claim 2, characterized in that, The desensitization process of the image post-processing module includes pixel value modification at the pixel level, frequency domain processing at the image level, and spatiotemporal domain comprehensive processing at the video level.

5. The privacy protection method for a camera according to claim 2, characterized in that, Only the access control module can communicate with external devices, and this is limited to command interaction related to shooting and image data transmission.

6. The privacy protection method for a camera according to claim 2, characterized in that, Privacy protection parameters are obtained through computer simulation experiments and are readable only by the internal modules of the camera; they cannot be modified.

Images