Video coding method and device for privacy protection and computer readable medium

Abstract

The embodiment of the invention discloses a video coding method and device for privacy protection and a computer readable medium. A specific embodiment of the method comprises the following steps: carrying out video frame sampling on a video to obtain a video frame image set; performing the following coding steps on each video frame image to obtain an entropy coding data set: performing video coding on the video frame image to obtain a quantization coefficient matrix set; performing encryption transformation on a quantization coefficient matrix corresponding to the high-authority image block matrix to obtain an encrypted quantization coefficient matrix set; obtaining a low-permission quantization coefficient matrix set; entropy coding is carried out on the low-permission quantization coefficient matrix set and the encrypted quantization coefficient matrix set to obtain entropy coding data; encrypting the transformation matrix to obtain encrypted transformation data; and generating code stream data corresponding to the video, and transmitting the code stream data to a receiving end for video decoding. The embodiment can be used for coding the privacy-containing video, and the waste of video transmission resources is reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of video coding, and in particular to a video coding method, device, and computer-readable medium for privacy protection. Background Art

[0002] During video transmission, since video files usually have a large amount of data, they need to be encoded to reduce the amount of data, thereby achieving efficient transmission. For videos containing private content (such as license plate numbers, faces, etc.), these private contents must be protected during transmission to prevent unauthorized access and privacy leakage. At present, when encoding private videos, the commonly used method is: based on a two-stream method, the privacy blocks in the entire video or video image are encrypted, and the encrypted high-authority code stream and the unencrypted low-authority code stream are obtained through video encoding, and the two code streams are transmitted separately to achieve the purpose of privacy protection.

[0003] However, when encoding private videos in the above manner, the following technical problems often occur:

[0004] First, the two-stream method increases the bit rate of video transmission because it needs to transmit two copies of data. Moreover, when the privacy block in the video image is encrypted separately, the privacy block and the non-privacy block need to be encoded separately, resulting in the need to manage two sets of decoding information during the encoding and decoding process, which increases the amount of information and wastes video transmission resources.

[0005] Second, the recognition accuracy of privacy areas in the video is low, and some privacy areas may be missed, or non-privacy areas may be determined as privacy areas, thereby reducing the effectiveness of video encoding for privacy protection.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a video encoding method, an apparatus, and a computer-readable medium for privacy protection to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a video encoding method for privacy protection, the method comprising: sampling video frames of a video captured by a camera to obtain a video frame image set; performing the following encoding steps on each video frame image in the video frame image set to generate entropy coded data to obtain an entropy coded data set: performing video encoding on the video frame image to obtain a quantization coefficient matrix set; performing an encryption transformation on the quantization coefficient matrix corresponding to each high-authority image block matrix in the quantization coefficient matrix set according to a preset transformation matrix to generate an encrypted quantization coefficient matrix to obtain an encrypted quantization coefficient matrix set, wherein the transformation matrix is ​​a square matrix of an integer type; determining each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to each high-authority image block matrix as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set; performing entropy encoding on the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set to obtain entropy coded data; encrypting the transformation matrix to obtain encrypted transformation data; generating code stream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmitting the code stream data to a receiving end for video decoding.

[0010] In a second aspect, some embodiments of the present disclosure provide a video encoding device for privacy protection, the device comprising: a video frame sampling unit, configured to perform video frame sampling on a video captured by a camera to obtain a video frame image set; an encoding unit, configured to perform the following encoding steps on each video frame image in the above video frame image set to generate entropy coded data, and obtain an entropy coded data set: perform video encoding on the video frame image to obtain a quantization coefficient matrix set; according to a preset transformation matrix, perform an encryption transformation on the quantization coefficient matrix corresponding to each high-authority image block matrix in the above quantization coefficient matrix set and a preset high-authority image block matrix set to generate an encrypted quantization coefficient matrix, and obtain an encrypted A quantization coefficient matrix set, wherein the transformation matrix is ​​a square matrix of an integer data type; each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to each high-authority image block matrix is ​​determined as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set; entropy coding is performed on the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set to obtain entropy coded data; an encryption unit is configured to encrypt the transformation matrix to obtain encrypted transformation data; a generation and transmission unit is configured to generate bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmit the bitstream data to a receiving end for video decoding.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0013] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the video encoding method for privacy protection of some embodiments of the present disclosure, the waste of video transmission resources can be reduced. Specifically, the reason for the waste of video transmission resources is that the method based on two streams needs to transmit two copies of data, which will increase the bit rate of video transmission, and when the privacy block in the video image is encrypted separately, the privacy block and the non-privacy block need to be encoded separately, resulting in the need to manage two sets of decoding information during the encoding and decoding process, resulting in an increase in the volume of information. Based on this, the video encoding method for privacy protection of some embodiments of the present disclosure, first, the video captured by the camera is sampled to obtain a video frame image set. Secondly, for each video frame image in the above video frame image set, the following encoding steps are performed to generate entropy coded data to obtain an entropy coded data set: the video frame image is video encoded to obtain a quantization coefficient matrix set. According to a preset transformation matrix, the quantization coefficient matrix corresponding to each high-authority image block matrix in the above quantization coefficient matrix set and the preset high-authority image block matrix set is encrypted to generate an encrypted quantization coefficient matrix to obtain an encrypted quantization coefficient matrix set. Wherein, the above transformation matrix is ​​a square matrix of integer data type. Thus, the quantization coefficient matrix corresponding to the privacy area is encrypted, thereby ensuring that the video image in the privacy area is protected during the encoding and transmission process. Each quantization coefficient matrix in the above quantization coefficient matrix set that does not correspond to each high-authority image block matrix is ​​determined as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set. The above low-authority quantization coefficient matrix set and the above encrypted quantization coefficient matrix set are entropy encoded to obtain entropy encoded data. By entropy encoding low-authority data and encrypted data at the same time, it is possible to avoid managing additional decoding information, thereby reducing the complexity of video encoding and decoding. Then, the above transformation matrix is ​​encrypted to obtain encrypted transformation data. Thus, the security of privacy content protection in the video is improved. Finally, the bitstream data corresponding to the above video is generated according to the above entropy encoding data set and the above encrypted transformation data, and the above bitstream data is transmitted to the receiving end for video decoding. By realizing privacy protection in video encoding through one bitstream, the waste of video transmission resources can be effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flowchart of some embodiments of a video encoding method for privacy protection according to the present disclosure;

[0016] Figure 2 is a schematic structural diagram of some embodiments of a video encoding device for privacy protection according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1A process 100 of some embodiments of a video encoding method for privacy protection according to the present disclosure is shown. The video encoding method for privacy protection comprises the following steps:

[0025] Step 101: perform video frame sampling on the video captured by the camera to obtain a video frame image set.

[0026] In some embodiments, the execution subject of the video encoding method for privacy protection may perform video frame sampling on the video captured by the camera to obtain a video frame image set. Wherein, the video frame sampling may be performed on the video according to the video frame rate of the video to obtain the video frame image set. Here, the video frame rate is the number of image frames displayed per second in the video, which may be 30 frames per second (30 frames per second, 30 FPS) or 120 frames per second (120 FPS).

[0027] Step 102, for each video frame image in the video frame image set, the following encoding steps are performed to generate entropy coded data, thereby obtaining an entropy coded data set:

[0028] Step 1021, perform video encoding on the video frame image to obtain a quantization coefficient matrix set.

[0029] In some embodiments, the execution subject may perform video encoding on the video frame image to obtain a quantization coefficient matrix set. In particular, the video frame image may be encoded by a video encoding technology to obtain a quantization coefficient matrix set.

[0030] As an example, the above video coding technology may include but is not limited to at least one of the following: H.264 / AVC (Advanced Video Coding) and H.265 / HEVC (High Efficiency Video Coding).

[0031] In some optional implementations of some embodiments, the execution subject performs video encoding on the video frame image to obtain a quantization coefficient matrix set, which may include the following steps:

[0032] In the first step, the video frame image is divided into privacy regions to obtain a privacy region set. The privacy region is the image region in the video frame image that needs to be protected. For example, the region corresponding to the face and license plate number in the video frame image can be determined as the privacy region.

[0033] Optionally, the execution subject divides the video frame image into privacy areas to obtain a privacy area set, including the following steps:

[0034] Sub-step one, performing multi-level feature extraction on the above-mentioned video frame image to obtain a multi-scale feature atlas. The multi-scale feature atlas contains feature maps of different spatial resolutions. First, the initial privacy semantic segmentation model can be trained using a labeled sample image training data set to obtain a privacy semantic segmentation model. Here, the sample image training data contains images with privacy content and semantic segmentation labels that annotate the above-mentioned privacy content. The privacy semantic segmentation model can include a multi-level feature extraction module. Here, the multi-level feature extraction module is composed of multiple pooling transformation modules with different sampling rates.

[0035] Specifically, the above-mentioned video frame images can be subjected to multi-level feature extraction in sequence through pooling transformation modules with different sampling rates, and a first scale feature map, a second scale feature map, a third scale feature map and a fourth scale feature map can be obtained respectively. Afterwards, the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are respectively determined as multi-scale feature maps to obtain a multi-scale feature map set. Here, the above-mentioned pooling transformation module can be composed of a convolutional layer, a flattened layer (Flattened Layer), a pooling transformation network block (PoolFomer), and an effective channel attention block (Efficient ChannelAttention, ECA).

[0036] As an example, if the height of the video frame image is 256 and the width is 256. First, the first layer feature extraction can be performed on the video frame image through a pooling transformation module with a 4-fold downsampling rate to obtain a first scale feature map. The parameters of the convolution layer corresponding to the pooling transformation module with a 4-fold downsampling rate can be set to: the convolution kernel size is 5, the convolution step is 4, and the convolution padding is 1. Correspondingly, the size of the first scale feature map is 64×64. Afterwards, the second layer feature extraction can be performed on the first scale feature map through a pooling transformation module with a 2-fold downsampling rate to obtain a second scale feature map. The size of the second scale feature map can be 32×32. Then, the third layer feature extraction can be performed on the second scale feature map through a pooling transformation module with a 2-fold downsampling rate to obtain a third scale feature map. The size of the third scale feature map can be 16×16. Finally, the fourth layer feature extraction can be performed on the third scale feature map through a pooling transformation module with a 2-fold downsampling rate to obtain a fourth scale feature map. The size of the fourth scale feature map may be 8×8.

[0037] Sub-step 2, performing a global pooling operation on the multi-scale feature map with the lowest spatial resolution in the multi-scale feature map set to obtain a pooled feature map. The privacy semantic segmentation model also includes a pyramid pooling module. Here, the multi-scale feature map with the lowest spatial resolution in the multi-scale feature map set is the fourth-scale feature map. The fourth-scale feature map can be globally pooled by a pyramid pooling module (PPM) to obtain a pooled feature map. For example, the size of the pooled feature map can be 8×8.

[0038] Sub-step three, according to the above-mentioned pooled feature map, feature fusion is performed on each multi-scale feature map in the above-mentioned multi-scale feature map set to obtain a multi-scale fused feature map set. The above-mentioned privacy semantic segmentation model may also include a feature fusion module. Here, through the above-mentioned feature fusion module, feature fusion can be performed on each multi-scale feature map in the above-mentioned multi-scale feature map set according to the above-mentioned pooled feature map to obtain a multi-scale fused feature map set.

[0039] Specifically, first, the pooling feature map can be upsampled by bilinear interpolation to obtain an upsampled pooling feature map. Here, the size of the upsampled pooling feature map can be 16×16. Secondly, the upsampled pooling feature map and the third scale feature map can be element-by-element added to obtain a third fused feature map. After that, the third fused feature map can be upsampled by bilinear interpolation to obtain an upsampled third fused feature map. Here, the size of the upsampled third fused feature map can be 32×32. Secondly, the upsampled third fused feature map and the second scale feature map can be element-by-element added to obtain a second fused feature map. Then, the second fused feature map can be upsampled by bilinear interpolation to obtain an upsampled second fused feature map. Here, the size of the upsampled second fused feature map can be 64×64. Secondly, the upsampled second fused feature map and the first scale feature map are element-by-element added to obtain a first fused feature map. Finally, the pooled feature map, the third fused feature map, the second fused feature map and the first fused feature map may be respectively determined as multi-scale fused feature maps to obtain a multi-scale fused feature map set.

[0040] Sub-step 4: splicing each multi-scale fusion feature map in the multi-scale fusion feature map set to obtain a spliced ​​feature map. The privacy semantic segmentation model further includes a splicing module. Here, through the splicing module, each multi-scale fusion feature map in the multi-scale fusion feature map set can be spliced ​​to obtain a spliced ​​feature map.

[0041] Specifically, first, each multi-scale fused feature map in the multi-scale fused feature map set can be upsampled by bilinear interpolation to obtain a fused feature map of uniform size to generate a fused feature map set. For example, the size of each fused feature map can be 256×256. Secondly, each fused feature map in the fused feature map set can be spliced ​​in the channel dimension of the feature map to obtain a spliced ​​feature map.

[0042] Sub-step five, semantic segmentation is performed on the above-mentioned spliced ​​feature map to obtain a segmentation result label set. The above-mentioned privacy semantic segmentation model also includes a detection head. First, the above-mentioned detection head can be used to perform semantic segmentation on the above-mentioned spliced ​​feature map to obtain a segmentation result label set. Here, the above-mentioned detection head includes a convolution layer with a convolution kernel size of 3 and a Softmax classifier. Secondly, each segmentation result label corresponds to a segmentation area, and the segmentation area is a combination of pixels in the video frame image. For example, the segmentation result label set may include segmentation result labels such as "face", "license plate number", "tree", and "ground". "Face" corresponds to the segmentation area containing a face image in the video frame image.

[0043] Sub-step 6: determine the segmentation result labels in the above segmentation result label set that match each privacy object in the preset privacy object set as privacy labels to obtain a privacy label set. The above privacy object set may include privacy objects such as "face" and "license plate number".

[0044] Sub-step 7: determine the segmented area corresponding to each privacy label in the above privacy label set as a privacy area to obtain a privacy area set. For example, the segmented area containing the face or license plate image in the video frame image corresponding to the privacy labels such as "face" and "license plate number" can be used as the privacy area to obtain the privacy area set.

[0045] Optionally, a different permission identifier can be assigned to each privacy tag in the above privacy tag set. For example, the privacy tag set includes "house number", "license plate number", "face", etc., among which "house number" can correspond to permission identifier level 1, "license plate number" can correspond to permission identifier level 2, and "face" can correspond to permission identifier level 3. The number of levels of the above permission identifiers is not limited here. Correspondingly, each privacy area in the above privacy area set corresponds to a different permission identifier. For example, the above privacy area set can include a level 1 privacy area subset, a level 2 privacy area subset, a level 3 privacy area subset, etc.

[0046] The above sub-steps 1 to 7 and their related contents are an inventive point of an embodiment of the present disclosure, which solves the above technical problem of "reducing the effectiveness of video encoding for privacy protection". The factors that lead to the above technical problems are often as follows: the recognition accuracy of the privacy area in the video is low, and some privacy areas may be missed or non-privacy areas may be determined as privacy areas. If the above factors are solved, the effectiveness of privacy protection can be improved. In order to achieve this effect, first, multi-level feature extraction is performed on the above video frame image through pooling transformation modules with different sampling rates. In this way, the performance of semantic segmentation can be maintained while reducing the computational cost. Moreover, by simultaneously retaining the high-dimensional features and low-dimensional features of the video frame image, the loss of detail information in the feature extraction process is avoided. Secondly, the pyramid pooling module can better obtain the global information of the entire video frame image. Thirdly, through the feature fusion module and the splicing module, feature maps of different scales can be combined, while retaining the detail features and global features, thereby improving the accuracy of semantic segmentation. Finally, the segmentation result label matching the privacy object is used as a privacy label, and each privacy label corresponds to a privacy area. In this way, the accurate division of the privacy area is achieved, thereby improving the effectiveness of privacy protection.

[0047] The second step is to divide the video frame image into image blocks to obtain an image block matrix set. The video frame image can be divided into a plurality of image blocks of equal size according to a preset image block size to obtain an image block matrix set. Here, the preset image block size can be 16×16 pixels or 8×8 pixels.

[0048] The third step is to determine the image block matrix corresponding to the privacy area in the above image block matrix set as a high-authority image block matrix to obtain a high-authority image block matrix set. A privacy area can contain multiple image block matrices.

[0049] Optionally, each high-authority image block matrix in the above-mentioned high-authority image block matrix set corresponds to a different authority identifier. According to the authority identifier corresponding to each privacy area in the above-mentioned privacy area set, the image block matrix corresponding to the n-level privacy area in the above-mentioned image block matrix set can be determined as the n-level authority image block matrix, and the n-level authority image block matrix set is obtained. Wherein, n is the authority identifier. For example, the image block matrix corresponding to the level 1 privacy area in the above-mentioned image block matrix set can be determined as the level 1 authority image block matrix, and the level 1 authority image block matrix set is obtained.

[0050] The fourth step is to perform video coding prediction on each image block matrix in the above image block matrix set to generate a predicted matrix and obtain a predicted matrix set. In particular, a preset video coding prediction algorithm can be used to perform video coding prediction on each image block matrix in the above image block matrix set to generate a predicted matrix and obtain a predicted matrix set.

[0051] As an example, the video coding prediction algorithm may include but is not limited to at least one of the following: motion compensation, intra-frame prediction, bidirectional prediction, etc.

[0052] The fifth step is to perform a video coding transformation on each predicted matrix in the predicted matrix set to generate a transformed matrix and obtain a transformed matrix set. In particular, a video coding transformation can be performed on each image block matrix in the image block matrix set to generate a transformed matrix and obtain a transformed matrix set by using a preset video coding transformation algorithm.

[0053] As an example, the video coding transformation algorithm may include but is not limited to at least one of the following: discrete cosine transform (DCT), wavelet transform, etc.

[0054] In the sixth step, each transformed matrix in the transformed matrix set is subjected to video coding quantization to generate a quantization coefficient matrix, thereby obtaining a quantization coefficient matrix set. In particular, each transformed matrix in the transformed matrix set is subjected to video coding quantization to generate a quantization coefficient matrix, thereby obtaining a quantization coefficient matrix set, through a preset video coding quantization algorithm.

[0055] As an example, the video encoding quantization algorithm may include but is not limited to at least one of the following: uniform quantization (Uniform Quantization), adaptive quantization (Adaptive Quantization), etc.

[0056] Step 1022, according to a preset transformation matrix, encrypt the quantization coefficient matrix corresponding to each high-authority image block matrix in the quantization coefficient matrix set and the preset high-authority image block matrix set to generate an encrypted quantization coefficient matrix and obtain an encrypted quantization coefficient matrix set.

[0057] In some embodiments, the above-mentioned execution entity can perform an encryption transformation on the quantization coefficient matrix corresponding to each high-authority image block matrix in the above-mentioned quantization coefficient matrix set and the preset high-authority image block matrix set according to a preset transformation matrix to generate an encrypted quantization coefficient matrix and obtain an encrypted quantization coefficient matrix set.

[0058] In some optional implementations of some embodiments, the execution subject performs an encryption transformation on the quantization coefficient matrix corresponding to each high-authority image block matrix in the quantization coefficient matrix set according to a preset transformation matrix to generate an encrypted quantization coefficient matrix, and obtaining the encrypted quantization coefficient matrix set may include the following steps:

[0059] The first step is to determine the quantization coefficient matrix corresponding to each high-authority image block matrix in the above-mentioned quantization coefficient matrix set and the high-authority image block matrix set as a high-authority quantization coefficient matrix, and obtain a high-authority quantization coefficient matrix set. Among them, one high-authority image block matrix corresponds to one quantization coefficient matrix. Here, the high-authority quantization coefficient matrix is ​​a quantization coefficient matrix that is only visible to the receiving end whose authority identifier corresponds to the high-authority identifier. Among them, the above-mentioned authority identifier includes a high-authority identifier and a low-authority identifier.

[0060] Optionally, each high-authority quantization coefficient matrix in the high-authority quantization coefficient matrix set corresponds to a different authority identifier. For example, the quantization coefficient matrix corresponding to each level 1 authority image block matrix in the above-mentioned quantization coefficient matrix set and the level 1 authority image block matrix set can be determined as a level 1 authority quantization coefficient matrix to obtain a level 1 authority quantization coefficient matrix set.

[0061] In the second step, each high-authority quantization coefficient matrix in the above-mentioned high-authority quantization coefficient matrix set is mapped with a preset transformation matrix to obtain a mapping transformation matrix to generate a mapping transformation matrix set. Among them, the above-mentioned transformation matrix can be a square matrix of integer data type, and its value can be a random number. The size of the above-mentioned transformation matrix is ​​greater than or equal to the size of the above-mentioned high-authority quantization coefficient matrix. For example, the size of the transformation matrix can be 128×128, and the value can be a random number of int type (between 0 and 4294967295). First, the width (W) and height (H) of the high-authority quantization coefficient matrix can be obtained. Secondly, the values ​​of the first H rows and the first W columns in the above-mentioned transformation matrix can be used as the values ​​of the corresponding positions of the mapping transformation matrix to obtain a mapping transformation matrix to generate a mapping transformation matrix set. Among them, the size of the above-mentioned mapping transformation matrix is ​​the same as that of the high-authority quantization coefficient matrix, and its value is the value of the corresponding position in the above-mentioned transformation matrix. Moreover, one high-authority quantization coefficient matrix corresponds to one mapping transformation matrix. Here, each mapping transformation matrix in the mapping transformation matrix set is the same.

[0062] Optionally, a transformation matrix set may be preset, and each transformation matrix in the above transformation matrix set corresponds to a different permission identifier. For example, each level 1 permission quantization coefficient matrix in the level 1 permission quantization coefficient matrix set corresponds to a level 1 transformation matrix in the preset transformation matrix set. A level 1 permission quantization coefficient matrix corresponds to a mapping level 1 transformation matrix.

[0063] In addition, according to the width (W) and height (H) of each high-authority quantization coefficient matrix, the values ​​between the i-th row to the i+H-th row and the j-th column to the j+W-th column in the above transformation matrix can be randomly obtained as the values ​​of the mapping transformation matrix to obtain the mapping transformation matrix to generate a mapping transformation matrix set. Among them, i+H is less than the height of the above transformation matrix, and j+W is less than the width of the above transformation matrix. Here, each mapping transformation matrix in the mapping transformation matrix set is different.

[0064] The third step is to determine the difference between each high-authority quantization coefficient matrix in the above-mentioned high-authority quantization coefficient matrix set and the corresponding mapping transformation matrix in the above-mentioned mapping transformation matrix set as an encrypted quantization coefficient matrix to obtain an encrypted quantization coefficient matrix set. Among them, the above-mentioned encrypted quantization coefficient matrix is ​​determined by the difference between the high-authority quantization coefficient matrix and the corresponding mapping transformation matrix. Here, one encrypted quantization coefficient matrix corresponds to one high-authority quantization coefficient matrix. Secondly, the size of the encrypted quantization coefficient matrix is ​​the same as the size of the high-authority quantization coefficient matrix.

[0065] Optionally, the authority identifier of each encrypted quantization coefficient matrix in the encrypted quantization coefficient matrix can be determined according to the authority identifier corresponding to each high-authority quantization coefficient matrix in the above-mentioned high-authority quantization coefficient matrix set. For example, the difference between each level 1 authority quantization coefficient matrix in the above-mentioned level 1 authority quantization coefficient matrix set and the corresponding mapping level 1 transformation matrix in the mapping level 1 transformation matrix set can be determined as the level 1 authority encrypted quantization coefficient matrix to obtain the level 1 authority encrypted quantization coefficient matrix set.

[0066] Step 1023: determine each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to any high-authority image block matrix as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set.

[0067] In some embodiments, the execution entity may determine each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to each high-authority image block matrix as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set. Here, the low-authority quantization coefficient matrix is ​​a quantization coefficient matrix visible to all receiving ends.

[0068] Optionally, each low-authority quantization coefficient matrix in the low-authority quantization coefficient matrix set may correspond to a level 0 authority identifier.

[0069] Step 1024, entropy coding is performed on the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set to obtain entropy coded data.

[0070] In some embodiments, the execution subject may perform entropy coding on the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set by a preset entropy coding algorithm to obtain entropy coded data. One video frame image corresponds to one entropy coded data. Here, one entropy coded data corresponds to the union of the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set.

[0071] As an example, the entropy coding algorithm may include but is not limited to at least one of the following: Huffman Coding, Arithmetic Coding, Context-Adaptive Coding, etc.

[0072] Step 103, encrypt the transformation matrix to obtain encrypted transformation data.

[0073] In some embodiments, the execution entity may encrypt the transformation matrix to obtain encrypted transformation data.

[0074] In some optional implementations of some embodiments, the execution subject encrypts the transformation matrix to obtain encrypted transformation data, which may include the following steps:

[0075] The first step is to perform one-dimensional processing on the above transformation matrix to obtain a one-dimensional array. The length of the above one-dimensional array is the product of the number of rows and the number of columns of the transformation matrix. Here, the above transformation matrix can be rearranged into a one-dimensional array in the order of row priority or column priority.

[0076] The second step is to perform integrity verification on the one-dimensional array to obtain encrypted verification data. The one-dimensional array can be integrity verified by a preset integrity verification algorithm to obtain encrypted verification data.

[0077] As an example, the integrity verification algorithm may include, but is not limited to, at least one of the following: Secure Hash Algorithm (SHA), Message Authentication Code (MAC), and Key-based Hash Message Authentication Code (HMAC).

[0078] The third step is to divide the one-dimensional array into subarrays based on the preset subarray length to obtain a subarray set. The subarray length is less than the length of the one-dimensional array. Here, the preset subarray length can be set according to the data block length that can be processed by the encryption algorithm. First, starting from the first element of the one-dimensional array, a subarray can be intercepted every preset subarray length; second, if the number of remaining elements of the one-dimensional array is less than the subarray length, the remaining elements are taken as a subarray to obtain a subarray set.

[0079] In the fourth step, each subarray in the subarray set is encrypted according to a preset key to generate an encrypted subarray, thereby obtaining an encrypted subarray set. The key can be generated by a preset encryption algorithm. Here, each subarray in the subarray set can be encrypted according to the key by the encryption algorithm to generate an encrypted subarray, thereby obtaining an encrypted subarray set.

[0080] As an example, the encryption algorithm may include but is not limited to at least one of the following: Advanced Encryption Standard (AES), Data Encryption Standard (DES), etc.

[0081] The fifth step is to combine the encrypted sub-arrays in the encrypted sub-array set to obtain an encrypted array, and determine the encrypted array as encrypted transformation data.

[0082] Optionally, the execution subject may perform the encryption steps of the first to fifth steps on each transformation matrix corresponding to a different authority identifier in the preset transformation matrix set to obtain an encrypted transformation data set, wherein each encrypted transformation data corresponds to a different authority identifier and a key corresponding to a different authority identifier.

[0083] Step 104, generating bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmitting the bitstream data to a receiving end for video decoding.

[0084] In some embodiments, the execution subject may generate bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmit the bitstream data to a receiving end for video decoding.

[0085] In some optional implementations of some embodiments, the execution subject generates bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmits the bitstream data to the receiving end for video decoding, which may include the following steps:

[0086] The first step is to add the encrypted transformation data to the preamble information to obtain the encrypted preamble information. The preamble information may be a sequence parameter set (SPS). The preamble information includes parameter information for encoding the video and is set at the start position of the bitstream data.

[0087] The second step is to sequentially append each entropy coded data in the entropy coded data set to the encrypted preamble information to obtain the initial code stream data. First, an empty byte stream can be initialized. Secondly, the encrypted preamble information and each entropy coded data in the entropy coded data set are sequentially added to the empty byte stream to obtain a code stream byte stream. Finally, the code stream byte stream is used as the initial code stream data.

[0088] The third step is to encapsulate the above-mentioned initial bitstream data to obtain the bitstream data corresponding to the above-mentioned video. Among them, a media container format, such as MP4, MKV, etc., can be used to package the above-mentioned initial bitstream data with the audio or other data corresponding to the above-mentioned video to obtain the bitstream data corresponding to the above-mentioned video. Here, streaming media protocols can also be used: such as HTTP Live Streaming (HLS) and HTTP dynamic adaptive streaming (DASH) to encapsulate the above-mentioned initial bitstream data and obtain the bitstream data corresponding to the above-mentioned video, which can be used for network streaming media transmission.

[0089] The fourth step is to transmit the bit stream data to the receiving end for video decoding. The bit stream data can be transmitted to the receiving end for video decoding in a wired or wireless manner.

[0090] Optionally, after step 104, the execution subject may further perform video decoding on the bit stream data, including the following steps:

[0091] The first step is to entropy decode each entropy coded data in the entropy coded data set corresponding to the above-mentioned code stream data in response to the receiving end receiving the above-mentioned code stream data, and obtain an entropy decoded data set. Among them, the above-mentioned entropy decoded data includes an encrypted quantization coefficient matrix set and a low-authority quantization coefficient matrix set. First, according to the encapsulation method in the third step of the above-mentioned step 104, the above-mentioned code stream data can be decapsulated to obtain decapsulated initial code stream data. Here, the above-mentioned decapsulated initial code stream data corresponds to the above-mentioned initial code stream data. Secondly, the corresponding entropy coded data set and encrypted transformation data can be obtained from the above-mentioned decapsulated initial code stream data. Afterwards, according to the entropy coding algorithm in the above-mentioned step 1024, each entropy coded data in the entropy coded data set can be entropy decoded to obtain an entropy decoded data set. I will not go into details here.

[0092] In the second step, in response to determining that the receiving end corresponds to the high-authority identifier, the encrypted transformation data corresponding to the code stream data is decrypted to obtain a decryption transformation matrix.

[0093] Optionally, the above-mentioned receiving end may correspond to different levels of permission identifiers. According to the level of the permission identifier corresponding to the receiving end, each encrypted transformation data in the encrypted transformation data set that is less than or equal to the level of the permission identifier can be decrypted separately to obtain a decryption transformation matrix set. Among them, one decryption transformation matrix corresponds to one transformation matrix. Specifically, if the level of the permission identifier corresponding to the receiving end is lower than the permission identifier level of the encrypted transformation data, the encrypted transformation data cannot be decrypted, and the corresponding decryption transformation matrix cannot be obtained. For example, if the receiving end corresponds to a level 2 permission identifier, the level 1 encrypted transformation data and the level 2 encrypted transformation data in the encrypted transformation data set in the code stream data can be decrypted separately to obtain a level 1 decryption transformation matrix and a level 2 decryption transformation matrix. Among them, the level 1 decryption transformation matrix is ​​the same as the level 1 transformation matrix.

[0094] Optionally, the execution subject, in response to determining that the receiving end corresponds to a high-authority identifier, decrypts the encrypted transformation data corresponding to the code stream data to obtain a decryption transformation matrix, which may include the following steps:

[0095] Sub-step 1, in response to determining that the receiving end corresponds to a high-authority identifier, obtaining a receiving key of the receiving end. Each receiving end corresponds to an authority identifier. Here, the receiving key is a key sent in advance to the receiving end corresponding to the high-authority identifier in a secure manner.

[0096] Optionally, according to the level of the authority identifier corresponding to the receiving end, the keys of the authority identifiers less than or equal to the level can be pre-sent to the receiving end in a secure manner to obtain a receiving key set. For example, for a receiving end with a level 3 authority identifier, a level 1 key, a level 2 key and a level 3 key are pre-sent to the receiving end to obtain the corresponding level 1 receiving key, level 2 receiving key and level 3 receiving key.

[0097] As an example, the above security method may include but is not limited to at least one of the following: a method of encrypting keys through asymmetric encryption technology, and a method of sharing keys through a key exchange protocol.

[0098] Sub-step 2: In response to determining that the received key is the same as the key, the encrypted transformed data is sub-array segmented based on a preset sub-array length to obtain a segmented sub-array set. If it is determined that the received key is the same as the key, the encrypted transformed data can be sub-array segmented according to the sub-array length by the sub-array segmentation method of step 103 to obtain a segmented sub-array set. No further details will be given here.

[0099] Sub-step three, decrypting each segmentation sub-array in the segmentation sub-array set according to the received key to generate a decrypted sub-array, and obtaining a decrypted sub-array set. Wherein, the decryption sub-array set can be obtained by decrypting each segmentation sub-array in the segmentation sub-array set according to the received key through the inverse step of the encryption algorithm in the fourth step of step 103, and obtaining a decrypted sub-array according to the received key. No further details will be given here.

[0100] Sub-step four, combining the decryption sub-arrays in the above decryption sub-array set to obtain a decryption array.

[0101] Sub-step 5: integrity verification is performed on the decrypted array to obtain decrypted verification data. The decrypted array can be integrity verified by the same integrity verification algorithm as the second step of step 103 to obtain decrypted verification data.

[0102] Sub-step 6: In response to determining that the decryption verification data is the same as the encryption verification data, converting the decryption array into a decryption transformation matrix. If it is determined that the decryption verification data is the same as the encryption verification data, the decryption array can be converted into a decryption transformation matrix in the same row-first or column-first order as in the first step of step 103. The decryption transformation matrix is ​​the same as the transformation matrix.

[0103] Sub-step 7: in response to determining that the decrypted verification data is different from the encrypted verification data, sending a retransmission video instruction to the sending end. If it is determined that the decrypted verification data is different from the encrypted verification data, a retransmission video instruction may be sent to the sending end.

[0104] In practice, if the decrypted verification data after decryption at the receiving end is inconsistent with the encrypted verification data before encryption at the sending end, it is determined that an error has occurred in the transmission of the bitstream data corresponding to the above video, for example, it may be maliciously attacked during the transmission. At this time, the receiving end can send a notification of integrity verification failure to the sending end and request retransmission of the above video. After that, the sending end can verify the sent data, check the security of the encryption and transmission links, and then re-encode the video according to the above steps 101-104, and transmit it to the receiving end.

[0105] The above step 103 and the above sub-steps 1 to 7 in the subsequent optional implementation and their related contents are an inventive point of the embodiment of the present disclosure, which solves the technical problem that "the security of privacy-containing videos during encoding and transmission is low". The factors that lead to the above technical problems are often as follows: during the encoding and transmission of videos, network attacks, network failures, software and hardware errors, etc. often occur, resulting in the loss or tampering of the transmitted video encoding data. If the above factors are solved, the security of video encoding and transmission can be improved. In order to achieve this effect, first, the preset transformation matrix is ​​converted into a one-dimensional array. After that, the integrity of the above one-dimensional array is verified to ensure that the one-dimensional array is complete before encryption. Then, the above one-dimensional array is divided and each sub-array is encrypted in parallel. In this way, the speed and efficiency of encryption can be improved. Secondly, only the receiving end with high authority can obtain the key, which can avoid unauthorized access. Thirdly, the integrity of the decrypted data is verified to ensure that the data has not been tampered with or damaged during transmission. Finally, if the results of the integrity verification are consistent, the data recovery is continued; if the results of the integrity verification are inconsistent, a retransmission instruction can be sent to the sending end. In this way, not only can the fault tolerance of encryption and decryption be improved, but also the confidentiality, integrity and availability of data can be ensured, thereby improving the security of privacy-containing videos during the encoding and transmission process.

[0106] The third step is to perform the following decoding steps on each entropy decoded data in the above entropy decoded data set to generate a decoded video frame image, thereby obtaining a decoded video frame image set:

[0107] The first sub-step is to perform an inverse encryption transformation on each encrypted quantization coefficient matrix in the encrypted quantization coefficient matrix set according to the decryption transformation matrix to obtain a corresponding decryption quantization coefficient matrix set. Among them, first, the decoding mapping transformation matrix corresponding to each encrypted quantization coefficient matrix in the encrypted quantization coefficient matrix set can be determined according to the decryption transformation matrix through the same mapping scheme as the second step of step 1022 to obtain a decoding mapping transformation matrix set. Then, each encrypted quantization coefficient matrix in the encrypted quantization coefficient matrix set can be element-by-element added to the corresponding decoding mapping transformation matrix to obtain a decryption quantization coefficient matrix to generate a decryption quantization coefficient matrix set. Among them, the decryption quantization coefficient matrix set is the same as the high-authority quantization coefficient matrix set.

[0108] Optionally, according to the decryption transformation matrix of different permission identifiers, each encrypted quantization coefficient matrix corresponding to the permission identifier in the above encrypted quantization coefficient matrix set can be inversely encrypted to obtain the corresponding decryption quantization coefficient matrix set. For example, if the above steps obtain a level 1 decryption transformation matrix, the corresponding level 1 permission encrypted quantization coefficient matrix in the encrypted quantization coefficient matrix set can be inversely encrypted to obtain the corresponding level 1 decryption quantization coefficient matrix set. Among them, the level 1 decryption quantization coefficient matrix corresponds to the level 1 permission quantization coefficient matrix.

[0109] The second sub-step is to determine the low-authority quantization coefficient matrix set and the decryption quantization coefficient matrix set as a decoding quantization coefficient matrix set.

[0110] In practice, a receiver with a lower level of permission identification cannot obtain a decryption transformation matrix greater than the permission identification level. Furthermore, it is impossible to perform an inverse encryption transformation on an encrypted quantization coefficient matrix greater than the permission identification level to obtain the corresponding decryption quantization coefficient matrix. The encrypted quantization coefficient matrix that has not undergone an inverse encryption transformation can be used as a decryption quantization coefficient matrix. Since the encrypted quantization coefficient matrix is ​​not the correct high-authority quantization coefficient matrix, the subsequent video decoding step cannot obtain the correct high-authority image block matrix, achieving the purpose of preventing users with low permission identification levels from viewing image blocks with high permission identification levels.

[0111] The third sub-step is to determine the decoded video frame image according to the above decoded quantization coefficient matrix set. The corresponding decoded video frame image can be determined according to the above decoded quantization coefficient matrix set by the inverse step of the above video encoding technology.

[0112] Optionally, the above-mentioned determining the decoded video frame image according to the above-mentioned decoded quantization coefficient matrix set includes the following steps:

[0113] Sub-step 1, inverse quantization is performed on each decoded quantization coefficient matrix in the above decoded quantization coefficient matrix set to generate an inverse quantization matrix, and an inverse quantization matrix set is obtained. Wherein, each decoded quantization coefficient matrix in the above decoded quantization coefficient matrix set can be inversely quantized by the inverse step of the video coding quantization algorithm in step 1021, and an inverse quantization matrix set is obtained. No further details are given here.

[0114] Sub-step 2: inversely transform each inverse quantization matrix in the inverse quantization matrix set to generate an inverse transformation matrix, thereby obtaining an inverse transformation matrix set. In this case, each inverse quantization matrix in the inverse quantization matrix set can be inversely transformed by the inverse step of the video coding transformation algorithm in step 1021 to obtain an inverse transformation matrix set.

[0115] Sub-step three, predictive decoding is performed on each inverse transform matrix in the inverse transform matrix set to generate a predictive decoding matrix, and a predictive decoding matrix set is obtained. Wherein, predictive decoding can be performed on each inverse transform matrix in the inverse transform matrix set to generate a predictive decoding matrix and a predictive decoding matrix set can be obtained by the inverse step of the video coding prediction algorithm in the fourth step of step 1021.

[0116] Sub-step four: performing matrix block combination on each prediction decoding matrix in the above prediction decoding matrix set to obtain a decoded video frame image.

[0117] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the video encoding method for privacy protection of some embodiments of the present disclosure, the waste of video transmission resources can be reduced. Specifically, the reason for the waste of video transmission resources is that the method based on two streams needs to transmit two copies of data, which will increase the bit rate of video transmission, and when the privacy block in the video image is encrypted separately, the privacy block and the non-privacy block need to be encoded separately, resulting in the need to manage two sets of decoding information during the encoding and decoding process, resulting in an increase in the volume of information. Based on this, the video encoding method for privacy protection of some embodiments of the present disclosure, first, the video captured by the camera is sampled to obtain a video frame image set. Secondly, for each video frame image in the above video frame image set, the following encoding steps are performed to generate entropy coded data to obtain an entropy coded data set: the video frame image is video encoded to obtain a quantization coefficient matrix set. According to a preset transformation matrix, the quantization coefficient matrix corresponding to each high-authority image block matrix in the above quantization coefficient matrix set and the preset high-authority image block matrix set is encrypted to generate an encrypted quantization coefficient matrix to obtain an encrypted quantization coefficient matrix set. Wherein, the above transformation matrix is ​​a square matrix of integer data type. Thus, the quantization coefficient matrix corresponding to the privacy area is encrypted, thereby ensuring that the video image in the privacy area is protected during the encoding and transmission process. Each quantization coefficient matrix in the above quantization coefficient matrix set that does not correspond to each high-authority image block matrix is ​​determined as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set. The above low-authority quantization coefficient matrix set and the above encrypted quantization coefficient matrix set are entropy encoded to obtain entropy encoded data. By entropy encoding low-authority data and encrypted data at the same time, it is possible to avoid managing additional decoding information, thereby reducing the complexity of video encoding and decoding. Then, the above transformation matrix is ​​encrypted to obtain encrypted transformation data. Thus, the security of privacy content protection in the video is improved. Finally, the bitstream data corresponding to the above video is generated according to the above entropy encoding data set and the above encrypted transformation data, and the above bitstream data is transmitted to the receiving end for video decoding. By realizing privacy protection in video encoding through one bitstream, the waste of video transmission resources can be effectively reduced.

[0118] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a video encoding device for privacy protection. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0119] like Figure 2 As shown, in some embodiments, the video encoding device 200 for privacy protection includes: a video frame sampling unit 201, an encoding unit 202, an encryption unit 203, and a generation and transmission unit 204. The video frame sampling unit 201 is configured to perform video frame sampling on the video captured by the camera to obtain a video frame image set; the encoding unit 202 is configured to perform the following encoding steps on each video frame image in the above video frame image set to generate entropy coded data and obtain an entropy coded data set: perform video encoding on the video frame image to obtain a quantization coefficient matrix set; according to a preset transformation matrix, perform an encryption transformation on the quantization coefficient matrix corresponding to each high-authority image block matrix in the preset high-authority image block matrix set in the above quantization coefficient matrix set to generate an encrypted quantization coefficient matrix and obtain an encrypted quantization coefficient matrix set, wherein the above transformation matrix It is a square matrix of integer data type; each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to each high-authority image block matrix is ​​determined as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set; the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set are entropy encoded to obtain entropy encoded data; the encryption unit 203 is configured to encrypt the transformation matrix to obtain encrypted transformation data; the generation and transmission unit 204 is configured to generate the bitstream data corresponding to the video according to the entropy encoded data set and the encrypted transformation data, and transmit the bitstream data to the receiving end for video decoding.

[0120] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be described in detail here.

[0121] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0123] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0124] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0125] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0126] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0127] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: performs video frame sampling on the video captured by the camera to obtain a video frame image set; for each video frame image in the above-mentioned video frame image set, performs the following encoding steps to generate entropy coded data to obtain an entropy coded data set: performs video encoding on the video frame image to obtain a quantization coefficient matrix set; according to a preset transformation matrix, performs an encryption transformation on the quantization coefficient matrix corresponding to each high-authority image block matrix in the above-mentioned quantization coefficient matrix set and a preset high-authority image block matrix set to generate an encrypted quantization coefficient matrix. Matrix, obtain an encrypted quantization coefficient matrix set, wherein the above-mentioned transformation matrix is ​​a square matrix of integer data type; each quantization coefficient matrix in the above-mentioned quantization coefficient matrix set that does not correspond to each high-authority image block matrix is ​​determined as a low-authority quantization coefficient matrix, to obtain a low-authority quantization coefficient matrix set; entropy coding the above-mentioned low-authority quantization coefficient matrix set and the above-mentioned encrypted quantization coefficient matrix set is performed to obtain entropy coded data; the above-mentioned transformation matrix is ​​encrypted to obtain encrypted transformation data; according to the above-mentioned entropy coded data set and the above-mentioned encrypted transformation data, the bitstream data corresponding to the above-mentioned video is generated, and the above-mentioned bitstream data is transmitted to the receiving end for video decoding.

[0128] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a video frame sampling unit, an encoding unit, an encryption unit, a generation and transmission unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the video frame sampling unit may also be described as a "unit for sampling video frames of videos captured by a camera".

[0131] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A video encoding method for privacy protection, comprising: Sampling video frames of the video captured by the camera to obtain a video frame image set; For each video frame image in the video frame image set, the following encoding steps are performed to generate entropy coded data, thereby obtaining an entropy coded data set: Perform video encoding on the video frame image to obtain a quantization coefficient matrix set; According to a preset transformation matrix, an encryption transformation is performed on the quantization coefficient matrix corresponding to each high-authority image block matrix in the preset high-authority image block matrix set to generate an encrypted quantization coefficient matrix, thereby obtaining an encrypted quantization coefficient matrix set, wherein the transformation matrix is ​​a square matrix of an integer type; Determine each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to each high-authority image block matrix as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set; Performing entropy coding on the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set to obtain entropy coded data; Encrypting the transformation matrix to obtain encrypted transformation data; Generate bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmit the bitstream data to a receiving end for video decoding.

2. The method according to claim 1, wherein: The method further comprises: In response to the receiving end receiving the bitstream data, entropy decoding is performed on each entropy coded data in the entropy coded data set corresponding to the bitstream data to obtain an entropy decoded data set, wherein the entropy decoded data includes an encrypted quantization coefficient matrix set and a low-authority quantization coefficient matrix set; In response to determining that the receiving end corresponds to a high-authority identifier, decrypting the encrypted transformation data corresponding to the code stream data to obtain a decryption transformation matrix; For each entropy decoded data in the entropy decoded data set, the following decoding steps are performed to generate a decoded video frame image, thereby obtaining a decoded video frame image set: According to the decryption transformation matrix, performing an inverse encryption transformation on each encrypted quantization coefficient matrix in the encrypted quantization coefficient matrix set to obtain a corresponding decryption quantization coefficient matrix set; Determine the low-authority quantization coefficient matrix set and the decryption quantization coefficient matrix set as a decoding quantization coefficient matrix set; A decoded video frame image is determined according to the decoded quantization coefficient matrix set.

3. The method according to claim 1, wherein: The video frame image is subjected to video encoding to obtain a quantization coefficient matrix set, including: Dividing the video frame image into privacy regions to obtain a privacy region set, wherein the privacy region is an image region in the video frame image that needs to be protected; Dividing the video frame image into image blocks to obtain an image block matrix set; Determine the image block matrix corresponding to the privacy area in the image block matrix set as a high-authority image block matrix to obtain a high-authority image block matrix set; Performing video coding prediction on each image block matrix in the image block matrix set to generate a predicted matrix, thereby obtaining a predicted matrix set; Performing video coding transformation on each predicted matrix in the predicted matrix set to generate a transformed matrix, thereby obtaining a transformed matrix set; Video coding quantization is performed on each transformed matrix in the transformed matrix set to generate a quantization coefficient matrix, thereby obtaining a quantization coefficient matrix set.

4. The method according to claim 3, wherein: The method of performing encryption transformation on the quantization coefficient matrix corresponding to each high authority image block matrix in the quantization coefficient matrix set and the preset high authority image block matrix set according to the preset transformation matrix to generate an encrypted quantization coefficient matrix to obtain an encrypted quantization coefficient matrix set includes: Determine the quantization coefficient matrix corresponding to each high-authority image block matrix in the high-authority image block matrix set in the quantization coefficient matrix set as a high-authority quantization coefficient matrix to obtain a high-authority quantization coefficient matrix set; Mapping each high-authority quantization coefficient matrix in the high-authority quantization coefficient matrix set with a preset transformation matrix to obtain a mapping transformation matrix, so as to generate a mapping transformation matrix set; The difference between each high-authority quantization coefficient matrix in the high-authority quantization coefficient matrix set and the corresponding mapping transformation matrix in the mapping transformation matrix set is determined as an encrypted quantization coefficient matrix to obtain an encrypted quantization coefficient matrix set.

5. The method according to claim 2, wherein: The step of determining a decoded video frame image according to the decoded quantization coefficient matrix set comprises: Dequantizing each decoded quantization coefficient matrix in the decoded quantization coefficient matrix set to generate a dequantization matrix, thereby obtaining a dequantization matrix set; Performing an inverse transformation on each inverse quantization matrix in the inverse quantization matrix set to generate an inverse transformation matrix, thereby obtaining an inverse transformation matrix set; Performing predictive decoding on each inverse transform matrix in the inverse transform matrix set to generate a predictive decoding matrix, thereby obtaining a predictive decoding matrix set; Matrix blocks are combined for each prediction decoding matrix in the prediction decoding matrix set to obtain a decoded video frame image.

6. The method according to claim 1, wherein: The step of generating bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformed data, and transmitting the bitstream data to a receiving end for video decoding includes: Adding the encrypted transformation data to the preamble information to obtain the encrypted preamble information, wherein the preamble information includes parameter information for encoding the video and is set at the start position of the bit stream; Sequentially appending each entropy coded data in the entropy coded data set to the encrypted preamble information to obtain initial bitstream data; Encapsulating the initial bitstream data to obtain bitstream data corresponding to the video; The code stream data is transmitted to a receiving end for video decoding.

7. A video encoding device for privacy protection, comprising: A video frame sampling unit is configured to perform video frame sampling on the video captured by the camera to obtain a video frame image set; The encoding unit is configured to perform the following encoding steps on each video frame image in the video frame image set to generate entropy coded data, thereby obtaining an entropy coded data set: Perform video encoding on the video frame image to obtain a quantization coefficient matrix set; According to a preset transformation matrix, an encryption transformation is performed on the quantization coefficient matrix corresponding to each high-authority image block matrix in the preset high-authority image block matrix set to generate an encrypted quantization coefficient matrix, thereby obtaining an encrypted quantization coefficient matrix set, wherein the transformation matrix is ​​a square matrix of an integer type; Determine each quantization coefficient matrix in the quantization coefficient matrix set that does not correspond to each high-authority image block matrix as a low-authority quantization coefficient matrix to obtain a low-authority quantization coefficient matrix set; Performing entropy coding on the low-authority quantization coefficient matrix set and the encrypted quantization coefficient matrix set to obtain entropy coded data; an encryption unit, configured to encrypt the transformation matrix to obtain encrypted transformation data; The generating and transmitting unit is configured to generate bitstream data corresponding to the video according to the entropy coded data set and the encrypted transformation data, and transmit the bitstream data to a receiving end for video decoding.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images