Video privacy protection method, video playback method and related devices

Abstract

The present application provides a video privacy protection method, a video playback method and related devices. Among them, the video privacy protection method includes: obtaining original encryption parameters and original video; for each original video frame, determining the privacy area of ​​the original video frame, and using a preset blurring algorithm to perform image blurring processing on the privacy area of ​​the original video frame to obtain a blurred video frame; using the blurred video frame as the input data of the base layer in the target encoder, and using the original video frame as the input data of the enhancement layer in the target encoder, using the target encoder to perform video encoding, and obtain first video encoding data; encrypting the enhanced data in the first video encoding data according to the original encryption parameters to obtain second video encoding data; generating a privacy-protected video based on the second video encoding data. The present application combines the scalability and cryptographic technology of H.265 encoding technology to take into account both high efficiency and reversible restoration.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a video privacy protection method, a video playback method, and related devices. Background Art

[0002] The widespread use of video surveillance systems has brought security and convenience to public places (such as stations, shopping malls, and streets) and homes. To support live viewing and AI (artificial intelligence) analysis, surveillance videos captured by video surveillance systems are typically uploaded to cloud servers for storage and access. However, with increasing demands for data security and privacy protection, ensuring video privacy has become a key concern.

[0003] Currently, existing technologies mainly provide video privacy protection through the following methods:

[0004] (1) Encrypting the entire video file does not allow for AI analysis and application on the encrypted data file, which greatly affects the value release of the video. At the same time, the video file needs to be fully decrypted before playback, and the decrypted video no longer has privacy protection. This "all or nothing" encryption is not suitable for the actual needs of both playback and privacy protection.

[0005] (2) Locally encrypt the privacy area in the video, that is, only encrypt the designated privacy area in the video, and the rest of the area is visible during playback. Although this method can present the visual effect of privacy protection during video playback, it requires accurate recording of the coordinates of the privacy area of ​​each video frame, and the encryption algorithm needs to be organically integrated with the entropy coding algorithm in video encoding, so you can only selectively encrypt some coding elements. At the same time, due to the complexity of the entropy coding algorithm and the difficulty of algorithm integration, this implementation method has the disadvantages of low encryption efficiency and low flexibility.

[0006] (3) Anonymize the video, that is, use blur algorithms such as Gaussian blur, pixelation, and mosaic to make the image content corresponding to the private area in the video unrecognizable, thereby achieving privacy protection. However, video privacy protection processing modes such as Gaussian blur and mosaic will cause the loss of image information, resulting in the playback device being unable to accurately restore the original video from the encrypted video.

[0007] (4) Generative adversarial networks are used to replace the image content corresponding to the private area in the video, so as not to leak the true information of the original video. Compared with the mosaic algorithm, the method of using generative adversarial networks to replace image content has better visual acceptance, but in terms of video restoration, even if key content features are additionally preserved, it still cannot support accurate restoration.

[0008] This shows that the existing video privacy protection scheme cannot take into account both high efficiency and reversible restoration. Summary of the Invention

[0009] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the technical deficiency in the prior art that it is impossible to achieve both high efficiency and reversible reduction.

[0010] In a first aspect, an embodiment of the present application provides a video privacy protection method, which is applied to a video encryption device, comprising:

[0011] Obtaining original encryption parameters and original video; wherein the original video includes multiple original video frames;

[0012] For each of the original video frames, determining a privacy area of ​​the original video frame, and performing image blurring processing on the privacy area of ​​the original video frame using a preset blurring algorithm to obtain a blurred video frame;

[0013] Using the blurred video frame as input data of a base layer of a target encoder, using the original video frame as input data of an enhancement layer of the target encoder, and using the target encoder to perform video encoding to obtain first video encoded data; wherein the target encoder is an encoder supporting H.265 scalable coding;

[0014] Encrypting the enhanced data in the first video encoding data according to the original encryption parameter to obtain second video encoding data; wherein the enhanced data is data output by the enhancement layer;

[0015] A privacy-preserving video is generated based on the second video encoding data.

[0016] In some embodiments, encrypting the enhanced data in the first video encoding data according to the original encryption parameters includes:

[0017] Determine a unit identifier corresponding to target NAL unit data; wherein the target NAL unit data is any first NAL unit data included in the enhanced data, and the unit identifier is used to uniquely identify the first NAL unit data;

[0018] generating target encryption parameters according to the unit identifier and the original encryption parameters;

[0019] The payload data of the target NAL unit data is encrypted according to the target encryption parameter.

[0020] In some embodiments, the original encryption parameters include an original initialization vector and an original encryption key, and the target encryption parameters include a target initialization vector and a target encryption key;

[0021] Generating target encryption parameters according to the unit identifier and the original encryption parameters includes:

[0022] Using a preset hash algorithm, performing a hash calculation on the unit identifier, the original initialization vector, and the original encryption key to obtain a target hash value;

[0023] According to a preset division rule, the target Hash value is divided into the target initialization vector and the target encryption key.

[0024] In some embodiments, encrypting the payload data of the target NAL unit data according to the target encryption parameter includes:

[0025] According to the target encryption parameters, the payload data of the target NAL unit data is encrypted using the SM4-CTR mode encryption algorithm.

[0026] In some embodiments, the enhancement data in the second video coded data includes at least one SEI data and at least one second NAL unit data, and the second NAL unit data includes a type identification value;

[0027] The generating of the privacy-preserving video based on the second video encoding data includes:

[0028] For each second NAL unit data, the type identification value of the second NAL unit data is written into the corresponding SEI data, and the type identification value of the second NAL unit data is modified to an undefined identification value.

[0029] In some embodiments, the original encryption parameters include an original initialization vector and an original encryption key;

[0030] The step of generating a privacy-preserving video based on the second video encoding data further includes:

[0031] Encrypting the original encryption key using a preset password to obtain a ciphertext key;

[0032] The ciphertext key and the original initialization vector are written into the first SEI data of the second video encoding data.

[0033] In a second aspect, an embodiment of the present application provides a video playback method, applied to a video playback device, comprising:

[0034] Obtaining a privacy-preserving video; wherein the privacy-preserving video is generated by the video privacy protection method described in any of the above embodiments;

[0035] Obtaining original encryption parameters corresponding to the privacy-preserving video;

[0036] The enhanced data in the privacy-preserving video is decrypted according to the original encryption parameters, and the original video is obtained based on the decryption result.

[0037] In a third aspect, an embodiment of the present application provides a video privacy protection device, which is applied to a video encryption device, including:

[0038] An original data acquisition module, configured to acquire original encryption parameters and original video; wherein the original video includes a plurality of original video frames;

[0039] a blur processing module, configured to determine, for each of the original video frames, a privacy region of the original video frame, and perform image blur processing on the privacy region of the original video frame using a preset blur algorithm to obtain a blurred video frame;

[0040] a video encoding module, configured to use the blurred video frame as input data of a base layer of a target encoder, use the original video frame as input data of an enhancement layer of the target encoder, perform video encoding using the target encoder, and obtain first video encoded data; wherein the target encoder is an encoder supporting H.265 scalable coding;

[0041] a video encryption module, configured to encrypt enhanced data in the first video encoding data according to the original encryption parameters to obtain second video encoding data; wherein the enhanced data is data output by the enhancement layer;

[0042] The privacy protection module is used to generate a privacy protection video based on the second video encoding data.

[0043] In a fourth aspect, an embodiment of the present application provides a video playback apparatus, which is applied to a video playback device, including:

[0044] A video acquisition module, configured to acquire a privacy-preserving video; wherein the privacy-preserving video is generated based on the video privacy protection method described in any of the above embodiments;

[0045] An encryption parameter acquisition module, used to obtain the original encryption parameters corresponding to the privacy protection video;

[0046] A decryption module is used to decrypt the enhanced data in the privacy-protected video according to the original encryption parameters, and obtain the original video based on the decryption result.

[0047] In a fifth aspect, an embodiment of the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video privacy protection method described in any of the above embodiments, and / or execute the steps of the video playback method described in any of the above embodiments.

[0048] In the video privacy protection method, video playback method, and related devices provided in some embodiments of the present application, the video encryption device can use any blurring algorithm to perform image blurring on the privacy area of ​​the original video frame to obtain a blurred video frame that can protect privacy information, thereby improving the flexibility of video privacy protection. The video encryption device utilizes the scalability of the H.265 encoding technology to encode the blurred video frame through the base layer of the target encoder, and to encode the original video frame through the enhancement layer of the target encoder. Since the scalable coding uses inter-layer predictive coding optimization technology, that is, the enhancement layer only stores additional information that is different from the base layer, it can improve the efficiency of blurring and encoding, thereby realizing an efficient video privacy protection solution.

[0049] Furthermore, the video encryption device can encrypt the enhanced data output by the enhancement layer and generate a privacy-preserving video based on the second video coded data obtained through the encryption process. Without knowing the encryption parameters, the user cannot access the plaintext of the enhanced data and can only view blurred video frames with privacy-preserving effects, without being able to view the original clear video. Users who can obtain the encryption parameters can exploit the reciprocity of encryption and decryption to decrypt the enhanced data, thereby obtaining the plaintext of the enhanced data. This allows accurate decryption and restoration of the original video, achieving reversible restoration.

[0050] It can be seen that this application utilizes the scalability of H.265 encoding technology to achieve flexible and efficient video privacy protection, and combines the encryption and decryption reciprocity of cryptographic technology to achieve reversible restoration function, thereby taking into account both high efficiency and reversible restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1FIG1 is an application environment diagram of a video privacy protection method and a video playback method in one embodiment;

[0053] Figure 2 FIG1 is a schematic diagram of a frame of an original video frame in one embodiment;

[0054] Figure 3 Schematic diagram of a frame image processed by an AI recognition algorithm in one embodiment;

[0055] Figure 4 Schematic diagram of an encrypted frame image in one embodiment;

[0056] Figure 5 Schematic diagram of a decrypted frame image in one embodiment;

[0057] Figure 6 1 is a flow chart of a video privacy protection method in one embodiment;

[0058] Figure 7 A schematic flow chart of a video playback method in one embodiment;

[0059] Figure 8 1 is a schematic structural diagram of a video privacy protection device in one embodiment;

[0060] Figure 9 1 is a structural diagram of a video playback device in one embodiment;

[0061] Figure 10 FIG. 1 is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] The terms involved in this application are explained below.

[0064] HEVC (High Efficiency Video Coding): High-efficiency video coding, also known as H.265 encoding.

[0065] SHVC (Scalable HEVC): HEVC / H.265 scalable coding.

[0066] BL (Base Layer): The base layer of SHVC encoding.

[0067] EL (Enhancement Layer): SHVC encoding enhancement layer.

[0068] ROP (Region of Privacy): Privacy area.

[0069] YOLO (You Only Look Once): A deep learning object detection model.

[0070] IV (Initialization Vector): Initialization vector of the block encryption algorithm.

[0071] CTR (Counter): Counter mode of block cipher algorithm.

[0072] AI (Artificial Intelligence): Artificial Intelligence.

[0073] SEI (Supplemental Enhancement Information): Supplemental enhancement information.

[0074] NAL (Network Abstraction Layer): Network abstraction layer for video.

[0075] In some embodiments, the present application provides a video privacy protection method and a video playback method that can be applied to Figure 1 In the application environment shown. Among them, the acquisition device refers to a video acquisition device, which can be installed in public and / or private places in a legal and compliant manner to capture original video. For example, the acquisition device can be a surveillance camera. It should be noted that the image / video acquisition, storage, use, processing, transmission, provision and disclosure involved in this application are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0076] A video encryption device is used to encrypt original videos to generate privacy-protected videos. A video playback device is used to decrypt privacy-protected videos to retrieve the original videos and play them back. Cloud storage can be implemented through cloud servers, meaning that privacy-protected videos can be uploaded to cloud servers for storage.

[0077] It is understood that the video encryption device and the video playback device can be implemented using any device, and this document does not impose any specific restrictions. In one example, the video encryption device and the video playback device can be local computing devices or security-certified computing devices. In another example, the video encryption device and the capture device can be integrated into one, and the resulting capture device can have both video capture and video encryption functions. In yet another example, the video playback device can be an authorized and authenticated electronic terminal.

[0078] See also Figure 1 Some embodiments of this application utilize H.265 scalable coding, AI recognition models, and cryptographic technologies to provide a more flexible and efficient video privacy protection method and video playback method. The video privacy protection method may include steps such as identifying privacy regions in the video, protecting the privacy regions, encrypting data, and generating a privacy-protected video. The video playback method may also include steps such as decryption and restoration using a key.

[0079] Specifically, the acquisition device can acquire the original video and obtain the video file of the original video. The original video includes one or more original video frames, and at least one original video frame includes privacy information. The video encryption device can use an AI recognition algorithm to automatically identify the privacy area of ​​each original video frame, and use a blur algorithm such as a Gaussian blur algorithm to perform privacy protection processing on the privacy area of ​​the original video frame, and obtain a blurred video frame. In one example, the original video frame can be as follows: Figure 2 As shown, the privacy area identified by AI can be Figure 3 As shown in the rectangular box, the blurred video frame can be Figure 4 shown.

[0080] After obtaining each blurred video frame, the video encryption device can use a target encoder supporting H.265 scalable coding to encode the privacy-protected image frame (i.e., the blurred video frame) into a base layer data unit and the original clear version of the video image frame (i.e., the original video frame) into an enhancement layer data unit, thereby obtaining a plaintext scalable coded video file (i.e., the first video coded data). For the enhanced data output by the enhancement layer, the video encryption device can encrypt the NAL unit data corresponding to the enhanced data using the original encryption parameters and record the original encryption parameters to form the final privacy-protected video. The video encryption device can then upload the video coded data corresponding to the privacy-protected video to a cloud server for cloud storage.

[0081] Without knowing the encryption parameters, the video playback effect of the privacy-protected video can be as follows: Figure 4As shown, a blurred visual effect is presented. This demonstrates that this application effectively protects privacy and is applicable to surveillance video privacy protection in both public and private locations. Even if the video is stored in the cloud, cloud storage administrators or unauthorized attackers can only see the privacy-protected video.

[0082] In the case of having encryption parameters, the video playback device can use the encryption parameters to decrypt the correct enhancement layer data and play the video based on the decrypted data. After decryption, the video presents the original clear visual effect, such as Figure 5 Furthermore, in some examples, during online playback, the video playback device can implement decrypted playback by decrypting and playing simultaneously. In other examples, during offline playback, the video playback device can play the video after it is completely decrypted into plaintext.

[0083] It should be noted that Figure 2 、 Figure 3 and Figure 5 The fuzzy processing of the face area is to meet the relevant regulations of patent examination. In actual application scenarios, Figure 2 The original video frame shown, Figure 3 The privacy area recognition results shown and Figure 5 The decoded video frames shown are all clear, non-blurred image frames, thereby providing a technical solution for video viewing by legally authorized persons or legal electronic evidence collection.

[0084] It can be seen that the solutions provided by some embodiments of the present application have at least one of the following advantages:

[0085] (1) It can effectively protect privacy and is applicable to surveillance video privacy protection in public and private places. Even if the video is stored in the cloud, the cloud storage administrator or attacker can only see the privacy-protected video. Only the key owner can decrypt and watch the original clear video.

[0086] (2) It has great flexibility in terms of ROP designation and privacy protection processing of privacy areas. First, ROP can flexibly provide privacy areas that need to be protected based on AI or manual selection. Second, the processing of base layer privacy areas is not limited to Gaussian blurring. Privacy area anonymization technologies such as pixel blurring, ROP encryption, and generative adversarial networks can also be used. The method provided in this application does not require the precise recording of the coordinates of the ROP area, because this application essentially uses enhanced layer data to save clear version of video frame data.

[0087] (3) Secure and efficient privacy-preserving encryption capabilities. This application encrypts the enhanced layer data units after scalable encoding. The encryption algorithm can use a secure national secret block encryption algorithm. Cloud storage administrators or attackers without the key cannot restore the original video. The block encryption algorithm has high computational efficiency and does not require recording the coordinates of the encrypted area or considering the selective encryption of complex encoding parameters, as is required for ROP encryption.

[0088] (4) The video is completely reversible. When necessary, the correct key can be used to decrypt the video to produce a video that is exactly the same as the original video, meeting the needs of authorized viewing or judicial digital evidence collection.

[0089] The following describes the video privacy protection method provided by this application.

[0090] In some embodiments, the present application provides a video privacy protection method, and the following embodiments are described using the method applied to a video encryption device as an example. Figure 6 As shown, the video privacy protection method provided by this application includes the following steps:

[0091] S602: Obtain original encryption parameters and original video.

[0092] The original video may be video that has not been processed for privacy protection, such as original surveillance video captured by a data acquisition device. The original video may include multiple original video frames, where the original video frames are video frames that have not been processed for privacy protection. The original encryption parameters refer to the information used to encrypt the video encoding data. It is understood that the method for obtaining the original video and original encryption parameters can be determined based on actual circumstances and is not specifically limited herein.

[0093] S604: For each original video frame, determine the privacy area of ​​the original video frame, and use a preset blurring algorithm to perform image blurring processing on the privacy area of ​​the original video frame to obtain a blurred video frame.

[0094] The privacy area can be the image area corresponding to the private information in the original video frame. The preset blur algorithm refers to an image processing algorithm capable of anonymizing image data, and may include, but is not limited to, Gaussian blur algorithms, pixelated blur algorithms (such as mosaic algorithms), ROP encryption algorithms, generative adversarial networks, and other algorithms.

[0095] For each frame of the original video, the video encryption device may first determine the target subject and target area in the original video frame that require privacy protection, and then determine the privacy area of ​​the original video frame.

[0096] In one example, a video encryption device can use an AI recognition algorithm to automatically identify privacy areas within raw video frames. Furthermore, the AI ​​recognition algorithm can be implemented based on the YOLO model. A trained YOLO model can support the recognition of typical privacy targets, including faces and license plates. The recognition results can include the target subject and the coordinates of the privacy area in each video frame, and the privacy area can be identified using a rectangular area.

[0097] In another example, privacy zones can be manually selected by the user. Furthermore, manually specifying privacy zones requires support from a specific video viewing front-end. Users can use a mouse or gestures to mark out privacy zones. For example, when shooting with a fixed camera in the living room, the user can set the bathroom and bedroom as privacy zones.

[0098] For each original video frame, after determining the privacy region of the original video frame, the video encryption device can use a preset blurring algorithm to process the image data corresponding to the privacy region, so that the resulting blurred video frame meets the visual anonymization and achieves privacy protection. In this application, by blurring and anonymizing the designated privacy region, the video-based AI visual analysis and application capabilities (such as people / vehicle counting and human posture determination) are not affected.

[0099] S606: Using the blurred video frame as input data of a base layer in a target encoder, using the original video frame as input data of an enhancement layer in the target encoder, performing video encoding using the target encoder, and obtaining first video encoding data.

[0100] The target encoder is an encoder that supports H.265 scalable encoding.

[0101] In this step, the video encryption device can use the blurred video frame as input data for the target encoder's base layer (BL), and the original clear video frame as input data for the target encoder's enhancement layer (EL). The target encoder can then process these two inputs in accordance with H.265 scalable coding and complete the encoding operation of the video image frame, thereby generating the first video encoded data.

[0102] It can be understood that the first video encoding data includes basic data and enhanced data. Basic data refers to the H.265 encoded data output by the base layer, which records the image frame data corresponding to the blurred video frame. Enhanced data refers to the H.265 encoded data output by the enhancement layer, which records the image frame data corresponding to the original video frame. Since scalable coding uses inter-layer predictive coding optimization technology, that is, the enhancement layer only stores additional information that is different from the base layer, the first video encoding data only increases the data volume by about 10% to 15% compared to the encoded data obtained by the existing encoding method, which will not cause too much additional burden on data storage.

[0103] S608: Encrypt the enhanced data in the first video encoding data according to the original encryption parameter to obtain second video encoding data.

[0104] The enhanced data is the data output by the enhancement layer.

[0105] Specifically, in the first video encoding data, the enhanced data is plaintext data. If the video is played directly based on the first video encoding data, the encoder can parse and render the original video frames, presenting a clear version of the original video. This shows that the first video encoding data does not have a high privacy protection capability.

[0106] To achieve effective privacy protection, the video encryption device can encrypt the enhanced data in the first video encoding data and obtain the second video encoding data after encryption. It can be understood that compared with the first video encoding data, the basic data of the second video encoding data remains unchanged, and the enhanced data of the second video encoding data is the ciphertext data obtained by encrypting the enhanced data of the first video encoding data.

[0107] For example, if the first video encoding data includes multiple plaintext base data and multiple plaintext enhancement data, the video encryption device can perform encryption calculations on the multiple plaintext enhancement data to obtain ciphertext enhancement data, and thereby obtain the second video encoding data. The second video encoding data can also include multiple plaintext base data and multiple ciphertext enhancement data.

[0108] S610: Generate a privacy-preserving video based on the second video encoding data.

[0109] In this step, the video encryption device can form the entire privacy-protected scalable coded video based on the second video coding data, and obtain the privacy-protected video.

[0110] In this case, if a player is used to play the privacy-protected video, it can only decode the base layer data and present a blurred visual effect in the private area, thus achieving the privacy protection of the video. Without the key, the player cannot decrypt the enhancement layer data and cannot restore the clear version of the original video image encoded in the enhancement layer.

[0111] In this application, the scalable capabilities of H.265 encoding technology can be used to achieve flexible and efficient video privacy protection, and the reversibility of encryption and decryption of cryptographic technology can be combined to achieve reversible restoration function, thereby taking into account both high efficiency and reversible restoration.

[0112] In some embodiments, encrypting the enhanced data in the first video encoded data according to the original encryption parameters includes:

[0113] Step A1: Determine a unit identifier corresponding to target NAL unit data; wherein the target NAL unit data is any first NAL unit data included in the enhanced data, and the unit identifier is used to uniquely identify the first NAL unit data;

[0114] Step A2: Generate target encryption parameters based on the unit identifier and the original encryption parameters;

[0115] Step A3: Encrypt the payload data of the target NAL unit data according to the target encryption parameter.

[0116] In this embodiment, by introducing a unit identifier to perform data encryption, it can be ensured that each NAL unit data corresponding to the enhanced data is encrypted using a different encryption parameter, thereby further improving information security.

[0117] Specifically, the first encoded data includes a plurality of NAL unit data, including NAL unit data corresponding to the base data and NAL unit data corresponding to the enhanced data. For each NAL unit data corresponding to the enhanced data, the video encryption device determines a unit identifier of the NAL unit data and encrypts a payload of the NAL unit data according to the unit identifier and the original encryption parameters.

[0118] In some examples, the unit identifier may be an arrangement sequence number of the NAL unit data in the first video coded data. The video encryption device may read each NAL unit data of the enhancement layer, set the number of the current NAL unit data as NAL_id (id starts counting from 1), and the video encryption device may determine the target encryption parameter for encrypting the current NAL unit data based on the number of the current NAL unit data and the original encryption parameter, and perform encryption accordingly.

[0119] In some embodiments, the original encryption parameters include an original initialization vector and an original encryption key, and the target encryption parameters include a target initialization vector and a target encryption key. The initialization vector refers to an initialization vector in a block cipher, used for video encryption. The encryption key refers to key information used for video encryption.

[0120] In one example, to improve data security and privacy protection, the original initialization vector and the original encryption key can be randomly generated using a random algorithm. Furthermore, the data length of the original initialization vector and the original encryption key is greater than or equal to 128 bits to meet security strength requirements and further improve security.

[0121] Generate target encryption parameters based on the unit identifier and original encryption parameters, including:

[0122] Step A21: Using a preset hash algorithm, perform hash calculation on the unit identifier, the original initialization vector, and the original encryption key to obtain a target hash value;

[0123] Step A22: Divide the target Hash value into a target initialization vector and a target encryption key according to a preset division rule.

[0124] In this embodiment, in the process of generating target encryption parameters corresponding to the target NAL data unit, the video encryption device may use a preset hash algorithm to perform a hash calculation on the initialization vector, the original encryption key, and the unit identifier corresponding to the target NAL data unit, and divide the target hash value obtained by the hash calculation according to a preset division rule to obtain a target initialization vector and a target encryption key. The video encryption device may use the target initialization vector and the target encryption key to perform an encryption calculation on the payload data of the target NAL unit data.

[0125] In one example, the video encryption device may use the commercially available hash algorithm SM3, key (original encryption key), IV (original initialization vector), and the ID value corresponding to the target NAL unit data to calculate the key stream SM3(id, key, IV), where id = 1, 2, 3...M, where M is the total number of first NAL unit data. The first 128 bits of SM3(id, key, IV) are used as the target encryption key key_id, and the last 128 bits are used as the target initialization vector IV_id.

[0126] In some embodiments, encrypting the payload data of the target NAL unit data according to the target encryption parameters includes encrypting the payload data of the target NAL unit data using the SM4-CTR encryption algorithm according to the target encryption parameters. Since CTR is a stream cipher mode, the length of the ciphertext and the plaintext can be guaranteed to be consistent, and the data length will not be increased after encryption, thereby minimizing the data size of the second video encoded data.

[0127] In the above example, the video encryption device may encrypt the payload data payload of the target NAL unit data using the CTR encryption mode of the commercial secret algorithm SM4 based on key_id and IV_id, specifically:

[0128] Cipher_payload=SM4_CTR_Encrypt(payload,key_id,IV_id)

[0129] Cipher_payload is the ciphertext data corresponding to the payload.

[0130] In some embodiments, the enhancement data in the second video encoding data includes at least one SEI data and at least one second NAL unit data. The SEI data is used to record the SEI of the supplemental enhancement information. The second NAL unit data is NAL unit data obtained by performing an encryption calculation on the payload data of the first NAL unit data. The second NAL unit data includes a type identification value, which is used to reflect the data type of the data recorded by the second NAL unit data.

[0131] Generate a privacy protection video based on the second video encoding data, including: for each second NAL unit data, write the type identification value of the second NAL unit data into the corresponding SEI data, and modify the type identification value of the second NAL unit data to an undefined identification value.

[0132] Specifically, the video encryption device may set the type of the NAL unit data corresponding to the enhancement layer in the second video coded data to an undefined value specified in the coding specification (for example, it may be set to 63), thereby instructing the decoder to ignore this NAL unit data during decoding, thereby ensuring the correctness of decoding. In addition, the video encryption device may retain the correct type identification value of the second NAL unit data in a custom field in the SEI preceding the NAL, so as to facilitate the recovery of the correct NAL type when providing decryption.

[0133] In some embodiments, the original encryption parameters include an original initialization vector and an original encryption key. For relevant descriptions of the original initialization vector and the original encryption key, please refer to the above embodiments and will not be repeated here.

[0134] Generating a privacy-preserving video based on the second video encoding data further includes:

[0135] Encrypt the original encryption key using the preset password information to obtain the ciphertext key;

[0136] The ciphertext key and the original initialization vector are written into the first SEI data of the second video encoding data.

[0137] The preset password information may be pre-set and used to encrypt the frame encryption information and obtain encryption parameter data.

[0138] Since the original encryption key is difficult to remember, a preset password can be used to encrypt the original encryption key. This allows the original encryption key to be restored using the preset password during decryption, making it easier to remember. It also improves security by preventing the original encryption key from being leaked.

[0139] In this embodiment, the video encryption device can use preset password information to encrypt the original encryption key to obtain a ciphertext key, and write the ciphertext key and the plaintext data of the original initialization vector into the first SEI data of the second video encoding data, so that the second video encoding data carries frame encryption information for decryption.

[0140] The video playback method provided by this application is described below.

[0141] In some embodiments, the present application provides a video playback method, and the following embodiments are described using the method applied to a video playback device as an example. Figure 7 As shown, the video playback method provided by this application includes the following steps:

[0142] S702: Obtain a privacy-preserving video; wherein the privacy-preserving video is generated based on the video privacy protection method of any of the above embodiments;

[0143] S704: Obtaining original encryption parameters corresponding to the privacy-protected video;

[0144] S706: Decrypt the enhanced data in the privacy-preserving video according to the original encryption parameters, and obtain the original video based on the decryption result.

[0145] In this embodiment, the video playback device can decrypt and restore the enhanced data in the privacy-preserving video using the original encryption parameters. Because the enhanced data in the privacy-preserving video is generated based on the encryption algorithm and the encoded data corresponding to the original video frames, after decryption, the video playback device can obtain the plaintext encoded data corresponding to the original video frames, thereby obtaining a clear original video. This allows for authorized viewing and meets legal evidence collection requirements.

[0146] In some examples, when a video playback device obtains preset password information, it can perform decryption and restoration according to the following steps:

[0147] Step B1: Read the first SEI data from the video file of the privacy-preserving video, and read the original initialization vector and ciphertext key from the first SEI data.

[0148] Step B2: Decrypt the ciphertext key using the preset password information and obtain the original encryption key.

[0149] Step B3: Using the original encryption key and original initialization vector, calculate the target key and target initialization vector corresponding to each second NAL unit data. For example, the video playback device can use the commercially available hash algorithm SM3, key (original encryption key), IV (original initialization vector), and the ID value corresponding to the second NAL unit data to calculate the key stream SM3(id, key, IV), where id = 1, 2, 3...M, where M is the total number of second NAL units. The first 128 bits of SM3(id, key, IV) are used as the target encryption key key_id, and the last 128 bits are used as the target initialization vector IV_id.

[0150] Step B4: Decrypt the payload data of each second NAL unit data using the target key and target initialization vector corresponding to each second NAL unit data, and obtain the plaintext payload. In one example, the decryption method can be performed according to the following expression:

[0151] payload=SM4_CTR_Decrypt(Cipher_payload,key_id,IV_id)

[0152] Step B5: For each second NAL unit data, read the correct type value from the SEI data preceding the second NAL unit data, and correct the type identification value of the second NAL unit data from 63 (ie, undefined value) to the correct type value.

[0153] This allows the entire video file to be restored to its scalable, encoded plaintext form. Playback now presents the original, clear visual experience, without privacy protection. Due to the complete reversibility of encryption and decryption, the original encoded plaintext and the decrypted video are identical in data, meeting key-authorized video viewing and forensic evidence collection requirements.

[0154] The video privacy protection device provided in an embodiment of the present application is described below. The video privacy protection device described below and the video privacy protection method described above can be referenced to each other.

[0155] In some embodiments, as Figure 8 As shown, the present application provides a video privacy protection device 800, which is applied to a video encryption device, including:

[0156] The original data acquisition module 802 is used to obtain original encryption parameters and original video; wherein the original video includes multiple original video frames;

[0157] A blur processing module 804 is configured to determine, for each original video frame, a privacy region of the original video frame, and perform image blur processing on the privacy region of the original video frame using a preset blur algorithm to obtain a blurred video frame;

[0158] a video encoding module 806 configured to use the blurred video frame as input data of a base layer of a target encoder and the original video frame as input data of an enhancement layer of the target encoder, perform video encoding using the target encoder, and obtain first video encoded data; wherein the target encoder is an encoder supporting H.265 scalable coding;

[0159] a video encryption module 808, configured to encrypt the enhanced data in the first video encoding data according to the original encryption parameters to obtain second video encoding data; wherein the enhanced data is data output by the enhancement layer;

[0160] The privacy protection module 810 is configured to generate a privacy-protected video based on the second video encoding data.

[0161] In some embodiments, the video encryption module 808 of the present application includes:

[0162] An identifier determining unit, configured to determine a unit identifier corresponding to target NAL unit data; wherein the target NAL unit data is any first NAL unit data included in the enhanced data, and the unit identifier is used to uniquely identify the first NAL unit data;

[0163] a target encryption parameter generating unit, configured to generate a target encryption parameter according to the unit identifier and the original encryption parameter;

[0164] A payload encryption unit is configured to encrypt the effective payload data of the target NAL unit data according to the target encryption parameter.

[0165] In some embodiments, the original encryption parameters include an original initialization vector and an original encryption key, and the target encryption parameters include a target initialization vector and a target encryption key. The target encryption parameter generation unit of the present application includes:

[0166] A hash calculation unit, configured to perform a hash calculation on the unit identifier, the original initialization vector, and the original encryption key using a preset hash algorithm to obtain a target hash value;

[0167] The division unit is configured to divide the target Hash value into the target initialization vector and the target encryption key according to a preset division rule.

[0168] In some embodiments, the payload encryption unit of the present application includes:

[0169] The CTR encryption unit is used to encrypt the payload data of the target NAL unit data using the SM4-CTR encryption algorithm according to the target encryption parameter.

[0170] In some embodiments, the enhanced data in the second video encoding data includes at least one SEI data and at least one second NAL unit data, and the second NAL unit data includes a type identification value. The privacy protection module 810 of the present application includes:

[0171] The type identification value modification unit is used to write the type identification value of each second NAL unit data into the corresponding SEI data, and modify the type identification value of the second NAL unit data to an undefined identification value.

[0172] In some embodiments, the original encryption parameters include an original initialization vector and an original encryption key. The privacy protection module 810 of the present application also includes:

[0173] A key encryption unit, configured to encrypt the original encryption key using a preset password to obtain a ciphertext key;

[0174] A data writing unit is used to write the ciphertext key and the original initialization vector into the first SEI data of the second video encoding data.

[0175] The video playback device provided in an embodiment of the present application is described below. The video playback device described below and the video playback method described above can refer to each other.

[0176] In some embodiments, as Figure 9 As shown, the present application provides a video playback device 820, which is applied to a video playback device, including:

[0177] A video acquisition module 822 is configured to acquire a privacy-preserving video; wherein the privacy-preserving video is generated based on the video privacy protection method described in any of the above embodiments;

[0178] An encryption parameter acquisition module 824 is used to obtain the original encryption parameters corresponding to the privacy protection video;

[0179] The decryption module 826 is configured to decrypt the enhanced data in the privacy-protected video according to the original encryption parameters, and obtain the original video based on the decryption result.

[0180] In one embodiment, the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video privacy protection method in any embodiment and / or execute the steps of the video playback method in any embodiment.

[0181] In one embodiment, the present application also provides a video encryption device, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video privacy protection method in any embodiment.

[0182] In one embodiment, the present application further provides a video playback device, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the video playback method in any embodiment.

[0183] In some embodiments, the video encryption device and video playback device of the present application can both be computer devices. Schematically, Figure 10 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. Figure 10 Computer device 900 includes a processing component 902, which further includes one or more processors, and a memory resource represented by memory 901 for storing instructions executable by processing component 902, such as an application. The application stored in memory 901 may include one or more modules, each corresponding to a set of instructions. In addition, processing component 902 is configured to execute the instructions to perform the steps of the method described in any of the above embodiments.

[0184] The computer device 900 may further include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate based on an operating system stored in the memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0185] Those skilled in the art will understand that the internal structure of the computer device shown in the present application is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0186] Finally, it should be noted that, in this article, relational terms such as first and second are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further restriction, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element. Herein, "one," "said," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. A plurality refers to at least two, such as 2, 3, 5, or 8. "And / or" includes any and all combinations of the relevant listed items.

[0187] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0188] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video privacy protection method, characterized in that: Applicable to video encryption equipment, including: Obtaining original encryption parameters and original video; wherein the original video includes multiple original video frames; For each of the original video frames, determining a privacy area of ​​the original video frame, and performing image blurring processing on the privacy area of ​​the original video frame using a preset blurring algorithm to obtain a blurred video frame; Using the blurred video frame as input data of a base layer of a target encoder, using the original video frame as input data of an enhancement layer of the target encoder, and using the target encoder to perform video encoding to obtain first video encoded data; wherein the target encoder is an encoder supporting H.265 scalable coding; Encrypting the enhanced data in the first video coded data according to the original encryption parameter to obtain second video coded data; wherein the enhanced data is data output by the enhancement layer, the enhanced data in the second video coded data includes at least one SEI data and at least one second NAL unit data, and the second NAL unit data includes a type identification value; generating a privacy-preserving video based on the second video encoding data; The step of encrypting the enhanced data in the first video encoding data according to the original encryption parameter includes: Determine a unit identifier corresponding to target NAL unit data; wherein the target NAL unit data is any first NAL unit data included in the enhanced data, and the unit identifier is used to uniquely identify the first NAL unit data; generating target encryption parameters according to the unit identifier and the original encryption parameters; Encrypting the payload data of the target NAL unit data according to the target encryption parameter; The generating of the privacy-preserving video based on the second video encoding data includes: For each second NAL unit data, the type identification value of the second NAL unit data is written into the corresponding SEI data, and the type identification value of the second NAL unit data is modified to an undefined identification value.

2. The method according to claim 1, characterized in that The original encryption parameters include an original initialization vector and an original encryption key, and the target encryption parameters include a target initialization vector and a target encryption key; Generating target encryption parameters according to the unit identifier and the original encryption parameters includes: Using a preset hash algorithm, performing a hash calculation on the unit identifier, the original initialization vector, and the original encryption key to obtain a target hash value; According to a preset division rule, the target Hash value is divided into the target initialization vector and the target encryption key.

3. The method according to claim 1, characterized in that The encrypting the payload data of the target NAL unit data according to the target encryption parameter includes: According to the target encryption parameters, the payload data of the target NAL unit data is encrypted using the SM4-CTR mode encryption algorithm.

4. The method according to claim 1, wherein The original encryption parameters include an original initialization vector and an original encryption key; The step of generating a privacy-preserving video based on the second video encoding data further includes: Encrypting the original encryption key using a preset password to obtain a ciphertext key; The ciphertext key and the original initialization vector are written into the first SEI data of the second video encoding data.

5. A video playback method, characterized in that: Applicable to video playback devices, including: Obtaining a privacy-preserving video; wherein the privacy-preserving video is generated based on the video privacy protection method according to any one of claims 1 to 4; Obtaining original encryption parameters corresponding to the privacy-preserving video; The enhanced data in the privacy-preserving video is decrypted according to the original encryption parameters, and the original video is obtained based on the decryption result.

6. A video privacy protection device, characterized in that: Applicable to video encryption equipment, including: An original data acquisition module, configured to acquire original encryption parameters and original video; wherein the original video includes a plurality of original video frames; a blur processing module, configured to determine, for each of the original video frames, a privacy region of the original video frame, and perform image blur processing on the privacy region of the original video frame using a preset blur algorithm to obtain a blurred video frame; a video encoding module, configured to use the blurred video frame as input data of a base layer of a target encoder, use the original video frame as input data of an enhancement layer of the target encoder, perform video encoding using the target encoder, and obtain first video encoded data; wherein the target encoder is an encoder supporting H.265 scalable coding; a video encryption module, configured to encrypt the enhanced data in the first video encoding data according to the original encryption parameters to obtain second video encoding data; wherein the enhanced data is data output by the enhancement layer, the enhanced data in the second video encoding data includes at least one SEI data and at least one second NAL unit data, and the second NAL unit data includes a type identification value; a privacy protection module, configured to generate a privacy protection video based on the second video encoding data; Wherein: the video encryption module includes: An identifier determining unit, configured to determine a unit identifier corresponding to target NAL unit data; wherein the target NAL unit data is any first NAL unit data included in the enhanced data, and the unit identifier is used to uniquely identify the first NAL unit data; a target encryption parameter generating unit, configured to generate a target encryption parameter according to the unit identifier and the original encryption parameter; A payload encryption unit, configured to encrypt the payload data of the target NAL unit data according to the target encryption parameter; The privacy protection module includes: The type identification value modification unit is used to write the type identification value of each second NAL unit data into the corresponding SEI data, and modify the type identification value of the second NAL unit data to an undefined identification value.

7. A video playback device, characterized in that: Applicable to video playback devices, including: A video acquisition module, configured to acquire a privacy-preserving video; wherein the privacy-preserving video is generated based on the video privacy protection method according to any one of claims 1 to 4; An encryption parameter acquisition module, used to obtain the original encryption parameters corresponding to the privacy protection video; A decryption module is used to decrypt the enhanced data in the privacy-protected video according to the original encryption parameters, and obtain the original video based on the decryption result.

8. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to execute the steps of the video privacy protection method according to any one of claims 1 to 4, and / or execute the steps of the video playback method according to claim 5.

Images