Camera privacy protection method, device and storage medium

Abstract

The present invention provides a privacy protection method, device and storage medium for a camera. By installing a mask body at the front end of the camera, the covered camera can only capture the internal image of the mask body and cannot directly capture the surrounding environment, such as scenes such as the living room or bedroom, thereby physically preventing the camera from capturing private information; at the same time, a visual sensor is provided on the outside of the mask body for capturing the image in front of the covered camera to ensure normal camera function; and the video recorded by the visual sensor is partially or completely encrypted and transmitted to the monitoring terminal through the camera's network connection based on visible light communication technology, thereby achieving logical and physical isolation from the camera and its network, thereby effectively protecting the user's privacy information while retaining the original function of the camera to provide a consistent user experience.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security, and in particular to a privacy protection method, device and storage medium for a camera. Background Art

[0002] Today, cameras are commonplace in both public and private spaces. People often worry about whether their sensitive information, such as facial features and behavior, could be captured and misused by the camera's owner or operator. The rich information contained in videos or images is essential for making them "smarter," but it also carries a higher risk of privacy breaches. Smart home devices, such as home security cameras and smart TVs, are particularly equipped with visual sensors for monitoring or improving user experience. Due to the sensitive nature of the home environment, their visual perception capabilities are more susceptible to privacy and security concerns. Summary of the Invention

[0003] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a camera privacy protection method, device and storage medium to reduce the risk of camera privacy leakage.

[0004] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present invention provides a privacy protection method for a camera, comprising: installing a mask body at the front end of the camera so that the camera only captures the image inside the mask body, the mask body being provided with a visual sensor to obtain original monitoring data of the monitoring area; encrypting the original monitoring data to obtain encrypted monitoring data; transmitting the encrypted monitoring data to a monitoring terminal through the camera based on visible light communication technology; the monitoring terminal decrypting the encrypted monitoring data to obtain original monitoring data.

[0005] In some embodiments of the first aspect of the present invention, the method for transmitting the encrypted monitoring data includes: encoding the encrypted monitoring data to generate a visible light communication frame sequence; displaying the visible light communication frame sequence in sequence on a display screen within the mask body; the camera focusing on the display screen through a lens to capture the visible light communication frame sequence on the display screen and transmitting it to the monitoring terminal; the monitoring terminal decoding the received visible light communication frame sequence to obtain the encrypted monitoring data.

[0006] In some embodiments of the first aspect of the present invention, the encryption method of the original monitoring data includes: extracting the area to be encrypted of the video frame based on the configured security policy and the original monitoring data; encrypting the area to be encrypted based on the symmetric key pre-shared by the masking device and the monitoring terminal to obtain the encrypted monitoring data.

[0007] In some embodiments of the first aspect of the present invention, the configuration items of the security policy include: an index item, an area to be encrypted item, an algorithm item, a reversal item, a time item and / or a valid item; wherein, the index item and the area to be encrypted item uniquely identify the security policy; the algorithm item configures the extraction algorithm of the area of ​​interest; the reversal item configures the area to be encrypted to include the extracted area of ​​interest or its opposite supplementary area; the valid item configures whether the security policy is enabled; and the time item configures the enabling time period of the security policy.

[0008] In some embodiments of the first aspect of the present invention, the privacy protection method of the camera includes: when the mask device detects a motion event, the warning LED light or display screen in the mask body simulates a virtual motion event to trigger the motion detection function of the camera.

[0009] In some embodiments of the first aspect of the present invention, the privacy protection method includes: inserting a deviation calibration leading frame before the visible light communication frame to perform color deviation and / or spatial deviation calibration on the visible light communication frame sequence displayed on the display screen.

[0010] In some embodiments of the first aspect of the present invention, the privacy protection method includes: inserting a distortion calibration leading frame before the visible light communication frame to perform image distortion calibration on the visible light communication frame sequence displayed on the display screen.

[0011] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present invention provides a privacy protection device for a camera, comprising: a mask body, arranged at the front end of the camera; a visual sensor, arranged on the mask body, for obtaining original monitoring data of the monitoring area; a processing unit, connected to the visual sensor, for receiving the original monitoring data and encrypting it to obtain encrypted monitoring data; and encoding the encrypted monitoring data based on visible light communication technology to obtain a visible light communication frame sequence; a display screen, arranged in the mask body and connected to the processing unit, for the processing unit to display the visible light communication frame sequence in sequence thereon; a lens, arranged between the display screen and the camera, for focusing the camera on the display screen to capture the visible light communication frame sequence on the display screen and transmit it to the monitoring terminal.

[0012] In some embodiments of the second aspect of the present invention, the privacy protection device includes: a warning LED light, connected to the processing unit, for triggering the motion detection function of the camera; and a message LED light, connected to the processing unit, for displaying the working status of the privacy protection device.

[0013] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the camera privacy protection method when the computer program is executed by a processor.

[0014] As described above, the privacy protection method, device and storage medium of the camera proposed in the present invention have the following beneficial effects: by installing a mask body at the front end of the camera, the covered camera can only capture the internal image of the mask body and cannot directly capture the surrounding environment, such as the living room or bedroom, thereby preventing the camera from capturing privacy information at a physical level; at the same time, a visual sensor is provided on the outside of the mask body to capture the image of the front of the covered camera to ensure normal camera function; and the video recorded by the visual sensor is partially or completely encrypted and transmitted to the monitoring terminal through the network connection of the camera based on visible light communication technology, thereby achieving logical and physical isolation from the camera and its network, thereby effectively protecting the user's privacy information while retaining the original function of the camera to provide a consistent user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Shown is a flowchart of a privacy protection method for a camera according to an embodiment of the present invention.

[0016] Figure 2 Shown is a structural schematic diagram of a privacy protection device for a camera according to an embodiment of the present invention.

[0017] Figure 3 Shown is a schematic diagram of the software architecture of a privacy protection device according to an embodiment of the present invention.

[0018] Figure 4 Shown is a schematic diagram of a visible light communication frame according to an embodiment of the present invention.

[0019] Figure 5 Shown is a schematic diagram of a ROI setting protocol according to an embodiment of the present invention.

[0020] Figure 6 Shown is a schematic diagram of a working scenario of a privacy protection device according to an embodiment of the present invention.

[0021] Figure 7 Shown is a top view of the hardware structure of a privacy protection device according to an embodiment of the present invention.

[0022] Figure 8 Shown is a left side view of the hardware structure of a privacy protection device according to an embodiment of the present invention.

[0023] Figure 9 Shown is a bottom view of the hardware structure of a privacy protection device according to one embodiment of the present invention.

[0024] Figure 10 FIG. 1 is a schematic diagram of ROI-based video encryption according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0026] It should be noted that in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present invention. It should be understood that other embodiments may also be used, and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present invention is limited only by the claims of the published patents. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between one element or feature shown in the figure and another element or feature.

[0027] In the present invention, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," "fixed," and "holding" should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0028] Furthermore, as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition occur only when the combination of elements, functions, or operations is inherently mutually exclusive in some way.

[0029] The present invention proposes a privacy protection method, device and storage medium for a camera to solve the problem that existing cameras are prone to privacy leakage, and is particularly suitable for home environments with high privacy and security requirements.

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the technical solutions in the embodiments of the present invention are further described in detail through the following embodiments and in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0031] Example 1

[0032] like Figure 1 As shown, an embodiment of the present invention provides a flowchart of a privacy protection method for a camera, and the privacy protection method includes the following steps:

[0033] Step S11. A shield body is installed at the front of the camera to limit the camera's view to the interior of the shield body. The shield body is equipped with a visual sensor to obtain raw surveillance data from the monitored area. By installing the shield body at the front of the camera, the covered camera can only capture the image within the shield body and cannot directly capture the surrounding environment, such as the living room or bedroom. This physically prevents the camera from capturing private information. Simultaneously, a visual sensor is installed on the exterior of the shield body to capture the image from the front of the covered camera. This is the camera's surveillance data when the shield body is not installed, which is the raw surveillance data of the monitored area, ensuring normal camera function.

[0034] Furthermore, if the mask device detects a motion event, the warning LED light or display screen within the mask body simulates a virtual motion event to trigger the camera's motion detection function. The warning LED light within the mask body simulates the virtual motion event by flashing. This application maintains the camera's original monitoring function, such as capturing surveillance data that is equivalent to the camera's surveillance data when the mask body is not installed, and retaining the camera's motion detection function.

[0035] Step S12. Encrypt the original monitoring data to obtain encrypted monitoring data. A direct way to protect visual privacy is to blur the entire video stream, but this will result in information loss and reduce the practical value of the video. This embodiment prefers a selective protection method, that is, only sensitive areas are protected and the remaining areas are not affected. The first step of selective protection is to identify the region of interest (ROI), which represents the sensitive area of ​​the video / image, which may usually be the facial area. ROI detection relies on computer vision (CV) technology, such as filter-based detection technology and deep neural network-based detection technology.

[0036] In some examples, ROI detection uses an on-demand processing solution to reduce detection overhead. In other examples, ROI detection uses additional sensors to assist the ROI detection algorithm, for example, using thermal image sensors and biosignal sensors to distinguish facial areas from other areas.

[0037] After obtaining the ROI region through ROI detection, the ROI region is processed to protect privacy. Optional processing methods include blurring, pixelation, object replacement, and so on. In some examples, ROI region processing can be completed before image capture, for example, by processing the ROI region using an optical device. In some examples, the ROI region can be directly protected through encryption, which preserves the original information while allowing full access to authorized entities.

[0038] Step S13: transmitting the encrypted monitoring data to a monitoring terminal via the camera based on visible light communication (VLC) technology.

[0039] In a preferred implementation manner of this embodiment, the method for transmitting the encrypted monitoring data includes: encoding the encrypted monitoring data to generate a visible light communication frame sequence; displaying the visible light communication frame sequence in sequence on a display screen within the mask body; the camera focusing on the display screen through a lens to capture the visible light communication frame sequence on the display screen and transmitting it to the monitoring terminal; the monitoring terminal decoding the received visible light communication frame sequence to obtain the encrypted monitoring data.

[0040] This embodiment transmits data via visible light communication between the display screen and the camera (i.e., screen to camera), establishing a unidirectional data transmission link. Preferably, based on dynamic QR code technology, optical content in the form of a QR code is displayed on the display screen, and the camera captures the data stream corresponding to this optical content. In this embodiment, the unique usage scenario of the QR code gives it a fixed shooting angle, making it simpler than conventional QR code design.

[0041] Furthermore, a deviation calibration leading frame is inserted before the visible light communication frame to perform color deviation and / or spatial deviation calibration on the visible light communication frame sequence displayed on the display screen. A distortion calibration leading frame is inserted before the visible light communication frame to perform image distortion calibration on the visible light communication frame sequence displayed on the display screen.

[0042] Step S14: The monitoring terminal decrypts the encrypted monitoring data to obtain the original monitoring data. The monitoring terminal can be a user's mobile terminal, such as a mobile phone, iPad, smartwatch, or laptop. Furthermore, the monitoring terminal and the encryption terminal (masking device) pre-share a key. For example, the encryption key is set in the device firmware and printed on a disposable sticker included with the new device. The device owner enters the key into the monitoring terminal's application to authorize access to the encrypted information.

[0043] In a preferred embodiment of this embodiment, the encryption method for the raw monitoring data includes: extracting the to-be-encrypted region (i.e., the sensitive region) of the video frame based on the configured security policy and the raw monitoring data; and encrypting the to-be-encrypted region using a symmetric key pre-shared between the masking device and the monitoring terminal to obtain the encrypted monitoring data. Optionally, the symmetric key includes DES (Data Encryption Standard), 3DES (Triple Data Encryption Standard), IDEA, RC5, RC6, etc.

[0044] Specifically, the configuration items of the security policy include: an index item, an area to be encrypted item, an algorithm item, a reversal item, a time item and / or a valid item; wherein, the index item and the area to be encrypted item uniquely identify the security policy; the algorithm item configures the extraction algorithm of the area of ​​interest; the reversal item configures the area to be encrypted to include the extracted area of ​​interest or its opposite supplementary area; the valid item configures whether the security policy is enabled; and the time item configures the enabling time period of the security policy.

[0045] Furthermore, the security policy supports enabling multiple ROI policies at the same time, and the area to be encrypted finally obtained is the sum of the ROI areas extracted by all enabled ROI policies.

[0046] In a preferred implementation of this embodiment, the method for obtaining the encrypted monitoring data includes: the visible light communication frame is cut into multiple tiles, and the tiles to be encrypted that overlap with the area to be encrypted are obtained; the tiles to be encrypted are encrypting the tiles using a key and a randomly generated key stream, and the random number used for encryption is inserted between the frame header and the data part of the network abstraction layer unit, and the data of the entire frame and its timestamp are hashed and appended to the encrypted content to obtain the encrypted monitoring data.

[0047] In some embodiments, the method can be applied to a controller, such as an ARM (Advanced RISC Machines) controller, an FPGA (Field Programmable Gate Array) controller, a SoC (System on Chip) controller, a DSP (Digital Signal Processing) controller, or an MCU (Microcontroller Unit) controller. In some embodiments, the method can also be applied to a computer comprising components such as a memory, a storage controller, one or more processing units (CPUs), peripheral interfaces, RF circuits, audio circuits, speakers, microphones, input / output (I / O) subsystems, displays, other output or control devices, and external ports; the computer includes, but is not limited to, personal computers such as desktop computers, laptop computers, tablet computers, smartphones, smart TVs, and personal digital assistants (PDAs). In other embodiments, the method can also be applied to servers, which can be deployed on one or more physical servers based on various factors such as function and load, or can be composed of distributed or centralized server clusters.

[0048] Example 2

[0049] like Figure 2As shown, an embodiment of the present invention provides a structural diagram of a privacy protection device for a camera. The privacy protection device (mask device) includes: a mask body 21, a visual sensor 22, a processing unit 23, a display screen 24 and a lens 25.

[0050] The mask body 21 is arranged at the front end of the camera (receiving camera) 20. In some examples, the mask body 21 is connected to the camera 20 by an adjustable card sleeve 26, which has the characteristics of convenient connection, beautiful appearance, and adjustable and flexible use. The visual sensor 22 is arranged on the mask body 21, and is used to obtain the original monitoring data of the monitoring area. The processing unit 23 is connected to the visual sensor 22, and is used to receive the original monitoring data and encrypt it to obtain encrypted monitoring data; and encode the encrypted monitoring data based on visible light communication technology to obtain a visible light communication frame sequence. The display screen 24 is arranged in the mask body 21 and is connected to the processing unit 23, so that the processing unit 23 can display the visible light communication frame sequence in sequence thereon. The lens 25 is arranged between the display screen 24 and the camera 20, focusing the camera 20 on the display screen 24 to capture the visible light communication frame sequence on the display screen 24 and transmit it to the monitoring terminal.

[0051] In some examples, visual sensor 22 captures video and stores it in the memory of processing unit 23. Processing unit 23 runs an embedded operating system to manage visual sensor 22 and display screen 24. The masking software within the operating system is responsible for executing image / video processing algorithms, encryption, and communication protocols. Display screen 24 serves as an output interface, transmitting the encrypted video stream from the masking device to camera 20. A suitable lens 25 is positioned in front of camera 20 to adjust the focus of camera 20 to match display screen 24.

[0052] This invention utilizes functional compatibility to preserve the functionality of the obscured camera, primarily including video surveillance preview and motion detection. This is achieved by sharing a video preview clip from the camera's companion application to the mask application (the program used by the monitoring terminal for decryption and decoding). The mask application then decodes the video clip to produce a surveillance preview. For motion detection, the mask device (privacy protection device) first analyzes the captured video stream to detect motion. Then, a simulated motion event triggers the camera, generating a motion detection alert that is sent to the camera's companion application.

[0053] In a preferred implementation of this embodiment, since the VLC stream displayed on the screen (display screen 24) may also be regarded as motion and cause a false alarm, the detection area setting function of the camera is used to exclude the screen displaying the VLC stream from the motion detection area.

[0054] In a preferred embodiment of this embodiment, the privacy protection device further includes a warning LED 27 connected to the processing unit 23 for triggering the motion detection function of the camera. The warning LED 27 is disposed within the mask body 21 and flashes to simulate a virtual motion event, thereby triggering the motion detection function of the camera 20.

[0055] In a preferred embodiment of this embodiment, the privacy protection device further includes a message LED 28 connected to the processing unit 23 for displaying the operating status of the privacy protection device. This message LED 28 is located on the exterior of the mask body 21 and is used to notify the owner of the mask's operating status, such as whether the face ROI is enabled. This mechanism is used to prevent potential privacy vulnerabilities caused by the owner's lack of knowledge, such as if the owner forgets to enable the ROI.

[0056] The processing unit 23 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0057] like Figure 3 As shown, an embodiment of the present invention proposes a software architecture diagram of a mask device (privacy protection device), which can be expressed as follows: First, ROI-based privacy protection is implemented on the captured video stream: a standard computer vision module is used to detect the ROI area of ​​the original video; while compressing the original video, the encryption module encrypts the video content belonging to the ROI area. Then, the encrypted and compressed video stream is encoded into VLC frames (visible light communication frames), which are filled into the frame buffer and displayed on the display screen inside the mask body. The VLC frames are captured by the camera (i.e., the receiving camera) and transmitted to the mask application via the network. Finally, the owner uses the mask application with a key to retrieve, decode and decrypt the captured original video stream to obtain a monitoring preview video.

[0058] The embodiment of the present invention utilizes the visual sensing capability of the camera to construct a one-way data transmission path (one-way VLC data path) from the mask device to the camera, and uses the camera to relay the video stream to the Internet, and then transmits the video stream to the user, thereby isolating the mask device from the network and ensuring the privacy and security of the data transmission process.

[0059] like Figure 4 As shown, an embodiment of the present invention provides a schematic diagram of a visible light communication frame. This VLC frame looks like a colorful QR code. The mask device encodes the captured video stream into a sequence of VLC frames. The VLC frames are displayed sequentially on a display screen within the mask and captured by a camera. The monitoring terminal decodes the captured VLC frames to extract the video captured by the mask device.

[0060] To describe the VLC design, this embodiment of the present invention defines the VLC frame for transmission as a pixel matrix txFR, where each matrix element represents the color of a pixel. rxFR, meanwhile, represents the VLC frame captured by the camera. Each VLC frame uses a 10x10 pixel block to represent a matrix element. Pixels in the same block have the same color. In a preferred implementation of this embodiment, an eight-color scheme is used to obtain the VLC frame sequence, thereby improving the throughput of the data path. The mapping relationship of this eight-color scheme is shown in Table 1, i.e., each matrix element modulates 3 bits.

[0061] Table 1. Mapping relationship of eight-color system

[0062]

[0063] The lens design introduces severe distortion, causing the camera's captured image to appear barrel-shaped and bulging. Another issue with the lens is chromatic aberration, with different colors exhibiting varying degrees of distortion. This is because the lens refracts light at varying ratios for different colors. The farther from the center, the greater the observed shift in color, making it more difficult to precisely locate the block. This is because the different color components of the data block are positioned differently when displaying different colors.

[0064] In a preferred implementation of this embodiment, in order to solve the impact of lens distortion, the embodiment of the present invention calibrates the distortion by inserting a distortion calibration leading frame before the VLC data frame, such as Figure 4As shown. The distortion calibration leading frame includes a black and white checkerboard pattern with a grid size equal to the data block size. Computer vision methods are used to accurately locate each black and white block in rxFR and record their positions. In order to deal with chromatic distortion, the above process is performed independently in the red, green and blue channels. The acquired position information will then be used to index the blocks in the VLC data frame to extract the color and thus decode the data bits. It should be noted that since the mask device is fixed to the camera, the position of each data block is stable, so the distortion calibration leading frame only needs to be used during the initialization process after the device is started, and does not need to be repeatedly inserted during data transmission.

[0065] Furthermore, the receiving camera uses various non-configurable image processing schemes to automatically adjust the contrast, brightness, saturation, and other aspects of the captured video. This introduces color bias in the rxFR. Furthermore, some smart security cameras exhibit a vignetting effect, causing the center of the rxFR image to appear brighter than the periphery, resulting in spatially inconsistent color in the data blocks.

[0066] In a preferred implementation of this embodiment, the color deviation caused by automatic adjustment and / or vignetting effect is calibrated by introducing a deviation calibration leading frame, such as Figure 4 As shown in the figure, this deviation calibration preamble frame includes a color palette (palette). This deviation calibration preamble frame is composed of the same palette used to display the colors used by txFR. These palettes serve as a reference for decoded colors. That is, during decoding, the closest color in the palette is selected as the color of the block in rxFR. Because the palette is stacked across the entire VLC frame, spatial deviations caused by vignetting effects can also be compensated.

[0067] This embodiment of the present invention uses digital encryption technology to protect information between the cover device and authorized parties (such as the cover application). It employs selective encryption based on the region of interest (ROI), rather than encrypting the entire frame. This allows users to flexibly specify the areas they wish to protect. Furthermore, by implementing different security policies, privacy and security are ensured while maintaining camera functionality.

[0068] In a preferred embodiment of the present invention, the user (device owner) can flexibly configure the device's security policy and specify the ROI-based encryption mode. Table 2 (Security Policy Configuration Table, ROI Setting Table) shows an example of a security policy entry in a masked device (similar to the access control list rules in a firewall).

[0069] Table 2. Security policy configuration table

[0070]

[0071] 1 haar cascades

[0072] 2 histograms of oriented gradients

[0073] 3 scene text detection

[0074] Each row in Table 2 defines the protection behavior of the mask device for the corresponding ROI. The "Index" item and the "ROI" item uniquely identify the policy for that ROI (where index 1 represents the haar cascades classifier, index 2 represents the histograms of oriented gradients, and index 3 represents scene text detection). ROI detection can be performed in a variety of ways, which are specified by the "Algorithm" item. The "Inversion" item specifies that the protection area is the detected ROI or the corresponding supplementary area, such as the body or the background outside the body. The "Time" item and the "Valid" item determine when and whether this policy entry should be enabled. The policy supports enabling multiple ROI policies at the same time, and the final area to be encrypted is the sum of the ROI areas extracted by all enabled ROI policies.

[0075] In a preferred embodiment of the present invention, to allow the mask device owner to configure the ROI policy, the present invention maintains a single input interface for receiving messages from the owner. This configuration is achieved by utilizing the mask device's visual sensor to scan a QR code containing configuration information, which is then shared between the mask application and the mask device. The owner uses the mask application to generate a QR code containing the configuration information and displays it to the mask device. An example configuration message for the "Time" item of the ROI policy is as follows:

[0076]

[0077] Furthermore, since the configuration interface involves sensitive processing, the present invention uses Figure 5The ROI setup protocol shown protects it from spoofing and replay attacks. The masked device uses a typical challenge and response protocol to authenticate the application displaying the QR code. When starting configuration, the masked application first displays a QR code containing a HELLO message. After the masked device receives HELLO, it sends a challenge message REQ containing a random number nonce to the paired application via the VLC data path. Once the application receives the challenge, it replies to the masked device with another QR code called RES, which contains the hash of the random number and the shared key. The masked device can verify the legitimacy of the application by verifying the hash, because the key is only shared with authorized masked applications. Once authentication is successfully completed, the masked device will accept the encrypted configuration message, which is the configuration information ConfigMsg attached to RES.

[0078] Example 3

[0079] like Figure 6 As shown in the figure, an embodiment of the present invention provides a schematic diagram of a privacy protection device operating scenario. The privacy protection device (camera mask, indicated by dashed lines) uses its mask body to shield the camera and uses its own visual sensor to capture the real image to ensure monitoring capabilities. It captures video through an encrypted channel and transmits it to the monitoring terminal via the camera, the internet, and a cloud server. The monitoring terminal decrypts the video data using the installed camera mask application to enable remote monitoring and preview. The camera mask device is physically and logically isolated from the network and infected cameras. Therefore, as long as its factory firmware is trusted, it can ensure its credibility. Even if the camera is completely controlled by malicious programs, the mask system can protect the privacy of the video content.

[0080] like Figures 7 to 9 As shown in FIG, an embodiment of the present invention proposes a hardware structure diagram of a privacy protection device, wherein: Figure 7 is a top view of the device, Figure 8 This is the left side view of the device. Figure 9This is a bottom view of the device. This embodiment uses commercial products and a 3D-printed housing to implement the hardware of the mask device. Specifically, the processing unit 71 uses the Nvidia Jetson Nano development board; the visual sensor 72 uses a Raspberry Camera V2.1, 60-fps @ 720p, and is directly connected to the CMOS serial port of the Nano development board 71; the display screen 73 is a SHARP LS029B3SX021 440x1440 @ 60fps, driven by an HDMI driver board 74; and a lens 75 with a focal length of 60-mm is used to focus the receiving camera 76 onto the display screen 73. Two LEDs (a message LED 77 and a warning LED 78) are connected to the GPIO pins of the Nano development board 71. These components are attached to a 3D-printed housing 79. Furthermore, the display screen is placed in a removable slot so that it can be adjusted to fit the focal plane of the smart camera.

[0081] In some examples, the mask device implements video ROI encryption and transmission by utilizing a feature of modern video codecs (e.g., HEVC or H.265) that can divide a video frame into uniform rectangular regions, i.e., tiles, and package the tiles into NAL units (Network Abstraction Layer Units) for general network transmission.

[0082] Based on this feature, the mask device packs multiple HEVC compressed video frames into a VLC frame, each video frame contains multiple encrypted and unencrypted tiles; any tile overlapping with the ROI is encrypted with a key and a randomly generated key stream, and the random number used for encryption is inserted between the NALU header and the NALU data; ordinary HEVC players that do not know the key can still render the encrypted tile area, but the picture information will be lost (such as Figure 10 The figure shows an example of ROI-based video encryption. Unencrypted tiles can be decoded normally by the player, ensuring visibility of non-privacy areas. The data of the entire frame and its timestamp are hashed and appended to the encrypted content to ensure the integrity and authenticity of the data frame.

[0083] Example 4

[0084] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the camera privacy protection method is implemented.

[0085] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0086] In summary, the camera privacy protection method, device, and storage medium proposed in the present invention install a mask body at the front end of the camera, so that the covered camera can only capture the internal image of the mask body and cannot directly capture the surrounding environment, such as scenes such as the living room or bedroom, thereby physically preventing the camera from capturing private information; at the same time, a visual sensor is provided on the outside of the mask body to capture the image of the front of the covered camera, ensuring normal camera function; and the video recorded by the visual sensor is partially or completely encrypted and transmitted to the monitoring terminal through the camera's network connection based on visible light communication technology, achieving logical and physical isolation from the camera and its network, thereby effectively protecting the user's privacy information and realizing a privacy protection method for a camera using functional replication and physical isolation. The present invention designs a camera privacy protection device (camera mask system) to protect the privacy security of the camera, which can be directly applied to commercial home smart security cameras and retains its key intelligent functions, such as monitoring image preview and motion detection, to provide users with a consistent user experience. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has high industrial utilization value.

[0087] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A camera privacy protection method, characterized in that: include: A mask body is installed at the front end of the camera so that the camera only captures the image inside the mask body. The mask body is provided with a visual sensor to obtain the original monitoring data of the monitoring area; a display screen is provided inside the mask body; and a lens is provided between the display screen and the camera. encrypting the original monitoring data to obtain encrypted monitoring data; Transmitting the encrypted monitoring data to a monitoring terminal via the camera based on visible light communication technology; The monitoring terminal decrypts the encrypted monitoring data to obtain original monitoring data; The encrypted monitoring data transmission method includes: encoding the encrypted monitoring data to generate a visible light communication frame sequence; Displaying the visible light communication frame sequence in sequence on a display screen within the mask body; The camera is focused on the display screen via a lens to capture a visible light communication frame sequence on the display screen and transmit the frame sequence to the monitoring terminal; The monitoring terminal decodes the received visible light communication frame sequence to obtain the encrypted monitoring data.

2. The camera privacy protection method according to claim 1, characterized in that: The encryption method of the original monitoring data includes: Extracting the area to be encrypted of the video frame based on the configured security policy and the original monitoring data; The area to be encrypted is encrypted based on a symmetric key pre-shared by the masking device and the monitoring terminal to obtain the encrypted monitoring data.

3. The camera privacy protection method according to claim 2, characterized in that: The configuration items of the security policy include: index item, area to be encrypted item, algorithm item, reversal item, time item and / or validity item; Among them, the index item and the area to be encrypted item uniquely identify the security policy; the algorithm item configures the extraction algorithm of the area of ​​interest; the inversion item configures the area to be encrypted to include the extracted area of ​​interest or its opposite supplementary area; the validity item configures whether the security policy is enabled; and the time item configures the enabling time period of the security policy.

4. The camera privacy protection method according to claim 1, wherein: include: When the mask device detects a motion event, the warning LED light or display screen in the mask body simulates a virtual motion event to trigger the motion detection function of the camera.

5. The camera privacy protection method according to claim 1, wherein: include: A deviation calibration leading frame is inserted before the visible light communication frame to perform color deviation and / or spatial deviation calibration on the visible light communication frame sequence displayed on the display screen.

6. The camera privacy protection method according to claim 1, characterized in that: include: A distortion calibration leading frame is inserted before the visible light communication frame to perform image distortion calibration on the visible light communication frame sequence displayed on the display screen.

7. A privacy protection device for a camera, characterized in that: include: A mask body, arranged at the front end of the camera; A visual sensor, provided on the mask body, for acquiring raw monitoring data of the monitoring area; a processing unit connected to the visual sensor, configured to receive the raw monitoring data and encrypt the raw monitoring data to obtain encrypted monitoring data; and encoding the encrypted monitoring data based on visible light communication technology to obtain a visible light communication frame sequence; a display screen, disposed in the mask body and connected to the processing unit, for the processing unit to display the visible light communication frame sequence in sequence thereon; The lens is arranged between the display screen and the camera, and focuses the camera on the display screen to capture the visible light communication frame sequence on the display screen and transmit it to the monitoring terminal.

8. The privacy protection device for a camera according to claim 7, characterized in that: include: a warning LED light, connected to the processing unit, for triggering the motion detection function of the camera; A message LED light is connected to the processing unit and is used to display the working status of the privacy protection device.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the privacy protection method for a camera according to any one of claims 1 to 6 is implemented.

Images