Video coding method and decoding method based on compressed sensing and corresponding devices

Abstract

The invention discloses a video coding method and decoding method based on compressed sensing and a corresponding device, and belongs to the technical field of coding and decoding, and the method comprises the following steps: S1, preprocessing: grouping collected video data, selecting the size of a macro block, and dividing each frame of data into the same and non-overlapping macro blocks; and S2, target object screening and data compression: compression processing is performed on the key frame and the non-key frame. According to the scheme, user privacy can be effectively protected.

Description

Technical Field

[0001] The present invention relates to the field of encoding and decoding technologies, and more specifically, to a video encoding method, a decoding method, and corresponding devices based on compressive sensing. Background Art

[0002] With the progress of technology, audio and video conferencing, video live streaming, and short videos have become the most widespread and popular applications in people's work and life. Currently, most commonly used monitoring devices basically adopt the H.26X or MPEG standards. Video files or traffic generated based on these compression standards are easily intercepted and restored during transmission, and used to smear and publicize the monitored objects. In terms of video storage, many people like to record and store critical moments in life or work on personal hard drives or network disks, which are easily copied by network attackers and then made public or tampered with and made public, affecting the daily work and life of relevant objects in the video. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a video encoding method, a decoding method, and corresponding devices based on compressive sensing, which can effectively protect user privacy.

[0004] The purpose of the present invention is achieved through the following solutions:

[0005] A video encoding method based on compressive sensing includes the following steps:

[0006] S1, preprocessing: Group the collected video data and select the macroblock size, and then divide each frame of data into identical and non-overlapping macroblocks;

[0007] S2, target object screening and data compression: Perform compression processing on key frames and non-key frames respectively.

[0008] Further, in step S1, the grouping of the collected video data and the selection of the macroblock size, and then dividing each frame of data into identical and non-overlapping macroblocks specifically include sub-steps:

[0009] S11, according to the user-set parameters, group the collected video YUV data, group the collected video YUV data according to the set parameter G, G>1;

[0010] S12, select the macroblock size according to the video picture resolution, and the macroblock size includes 16X16, 32X32, or 64X64;

[0011] S13. Select the first frame of each group of video YUV data as the reference key frame, and the remaining G - 1 frames as non - key frames. Then divide each frame of data into N macro - blocks of the same size and non - overlapping.

[0012] Further, in step S2, the following steps are performed for the key frame:

[0013] S201. Compressively sample the first macro - block using observation matrix 1, and then use H.26X compression to achieve quantization and entropy coding to obtain the encoded data.

[0014] S202. Use entropy decoding, inverse quantization, and compressive sensing reconstruction on the encoded macro - block data to obtain the decoded video macro - block.

[0015] S203. Perform intra - frame prediction on other macro - blocks except the first encoded macro - block in step S202 to obtain the intra - frame prediction mode.

[0016] S204. Determine whether the macro - block is a directly - encoded macro - block or an intra - frame - predicted macro - block and perform corresponding processing: If the macro - block is a directly - encoded macro - block, compressively sample the macro - block using observation matrix 1, and then use H.26X compression to achieve quantization and entropy coding to obtain the encoded data; if the macro - block is an intra - frame - predicted macro - block, encode it according to the following steps: First, decode the encoded macro - block data and use intra - frame prediction for the current macro - block to obtain the predicted residual data; then compressively sample the residual data of the macro - block using observation matrix 2 to obtain the compressed sampling observation value; finally, perform quantization and entropy coding based on H.26X on the difference of the observation value to obtain the compressed data.

[0017] Further, in step S2, the following steps are performed for the non - key frame:

[0018] S211. Perform intra - frame and inter - frame prediction on each macro - block to obtain the motion estimation vector.

[0019] S212. If the macro - block is a directly - encoded macro - block, compressively sample it using observation matrix 1, and perform quantization and entropy coding on the sampled data to obtain the compressed data.

[0020] If the macro - block is an intra - frame - predicted macro - block, encode it according to the following steps: First, use intra - frame prediction to obtain the predicted residual data; then compressively sample the residual data of the macro - block using observation matrix 2 to obtain the compressed sampling observation value; finally, perform quantization and entropy coding based on H.26X on the difference of the observation value to obtain the compressed data.

[0021] If the macroblock is an inter - prediction macroblock, the encoding is performed according to the following steps: First, using the motion vector and the reference macroblock, based on entropy decoding, inverse quantization, and compressive sensing reconstruction, the decoded macroblock is obtained, and the loss value between this macroblock and the reference macroblock is obtained; then, the minimum loss value is compared with the reference value P1. If it is greater than P1, this macroblock directly performs compressive sampling using the observation matrix 3; if it is less than P1, this macroblock directly performs compressive sampling using the observation matrix 2; then, the motion vector with the minimum loss value is selected as the final motion estimation vector; then, the residual data between this macroblock and the reference macroblock under the optimal motion estimation vector is measured by compressive sensing using the observation matrix 2 to obtain the observed value; finally, quantization and entropy coding based on H.26X are performed on the difference of this observed value to obtain the compressed data.

[0022] Further, the matrix uses a wavelet transform matrix.

[0023] Further, the loss value is calculated using the mean - square error function.

[0024] A video decoding method based on compressive sensing performs the following steps:

[0025] S3, Entropy decoding and inverse quantization: Receive the compressed data obtained by the encoding method described in any one of the above, and then perform entropy decoding and inverse quantization operations according to the H.26X codec standard to obtain the observed value data;

[0026] S4, CS reconstruction: Reconstruct the data obtained after being processed in step S3 using the OSL0 algorithm;

[0027] S5, Observed value reconstruction: Obtain the true observed value of this macroblock through the reconstructed data, the motion estimation vector, and the observed value of the reference macroblock;

[0028] S6, Data reconstruction: Obtain the restored data according to the true observed value and the motion - estimated macroblock.

[0029] A video encoding device based on compressive sensing includes a processor and a memory. The memory stores a computer program, and when the computer program is loaded and executed by the processor, it performs the encoding method described in any one of the above.

[0030] A video decoding device based on compressive sensing includes a processor and a memory. The memory stores a computer program, and when the computer program is loaded and executed by the processor, it performs the decoding method described above.

[0031] A video codec device based on compressive sensing includes a processor and a memory. The memory stores a computer program, and when the computer program is loaded and executed by the processor, it performs the encoding method described above and the decoding method described.

[0032] The beneficial effects of the present invention include:

[0033] The video encoding and decoding solution of the present invention is in the H.26X standard form in terms of data structure, etc., but only the real data can be obtained by decoding through the decoding method provided by the present invention. Using an ordinary H.26X decoder can decode normally, but the data content after decoding is incorrect error data. Therefore, the method of the present invention can effectively protect user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0035] Figure 1 is the encoding and decoding architecture in the embodiment of the present invention;

[0036] Figure 2 is the decoding effect diagram of a standard H264 decoder;

[0037] Figure 3 is the decoding effect diagram of the dedicated decoder provided by the method in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] All the features disclosed in all the embodiments in this specification, or all the steps in the methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or extended and replaced in any way.

[0039] In view of the current situation in the background, especially in an environment where network threats occur frequently, the present invention aims to address the problem that the transmission of surveillance videos and the storage of video files in daily life are easily intercepted and tampered with and made public. Based on the H.26X encoding and decoding standard, a new video encoding and decoding solution based on compressive sensing is proposed to implement video data encoding and decoding. During the video transmission and storage process, it is consistent with the video data form based on the H.26X encoding and decoding standard, but cannot be restored using existing decoders. Therefore, it can effectively protect user privacy and also maintain a compression ratio similar to the H.26X encoding and decoding standard. Figure 1 is the encoding and decoding architecture in the embodiment of the present invention. Figure 2 is the decoding effect diagram of a standard H264 decoder, Figure 3 is the decoding effect diagram of the dedicated decoder provided by the method in the embodiment of the present invention.

[0040] Furthermore, in a preferred embodiment, the specific implementation process of the present invention is as follows:

[0041] (1) Encoding process

[0042] (1) Preprocessing

[0043] 1) Group the collected video YUV data according to the user - set parameters, and group the collected video YUV data according to the set parameter G (G > 1).

[0044] 2) Select the macro - block size according to the video frame resolution, which can be 16X16, 32X32 or 64X64.

[0045] 3) Select the first frame of each group of video YUV data as the reference key frame, and the remaining G - 1 frames as non - key frames; then divide each frame of data into N macro - blocks of the same size and non - overlapping.

[0046] (2) Target object screening and data compression

[0047] For key frames:

[0048] 1) Compressively sample the first macro - block using observation matrix 1, and then use H.26X compression technology to achieve quantization, entropy coding, etc., to obtain the encoded data.

[0049] 2) Use entropy decoding, inverse quantization, and compressive sensing reconstruction on the first encoded macro - block data to obtain the decoded video macro - block.

[0050] 3) Perform intra - frame prediction on other macro - blocks to obtain the intra - frame prediction mode.

[0051] 4) If the macro - block is a directly - encoded macro - block, compressively sample the macro - block using observation matrix 1, and then use H.26X compression technology to achieve quantization, entropy coding, etc., to obtain the encoded data.

[0052] 5) If the macro - block is an intra - frame prediction macro - block, encode it according to the following steps:

[0053] a) Use intra - frame prediction technology to obtain the predicted residual data.

[0054] b) Compressively sample the residual data of the macro - block using observation matrix 2 to obtain the compressed sampling observation value.

[0055] c) Perform quantization and entropy coding based on H.26X on the difference of the observation value to obtain the compressed data.

[0056] For non - key frames:

[0057] 1) Perform intra - frame and inter - frame prediction on each macro - block to obtain the motion estimation vector.

[0058] 2) If the macroblock is a directly coded macroblock, compressed sampling is performed using observation matrix 1, and the sampled data is quantized and entropy encoded to obtain compressed data;

[0059] 3) If the macroblock is an intra-predicted macroblock, the encoding is performed according to the following steps:

[0060] a) Decode using the data of the already encoded macroblocks and obtain the predicted residual data for the current macroblock using intra-prediction technology;

[0061] b) Perform compressed sampling on the residual data of the macroblock using observation matrix 2 to obtain compressed sampling observations;

[0062] c) Perform quantization and entropy encoding based on H.26X on the difference of the observations to obtain compressed data.

[0063] 4) If the macroblock is a predicted macroblock, the encoding is performed according to the following steps:

[0064] a) Use the motion vector and the reference macroblock to obtain the decoded video macroblock through entropy decoding, inverse quantization, and compressive sensing reconstruction, and obtain the loss value between this macroblock and the reference macroblock;

[0065] b) Compare the minimum loss value with the reference value P1. If it is greater than P1, the macroblock directly performs compressed sampling using observation matrix 3; if it is less than P1, the macroblock directly performs compressed sampling using observation matrix 2;

[0066] c) Select the motion vector with the minimum loss value as the final motion estimation vector;

[0067] d) Calculate the residual data between this macroblock and the reference macroblock under the optimal motion estimation vector and perform compressive sensing measurement using observation matrix 2 to obtain observations;

[0068] e) Perform quantization and entropy encoding based on H.26X on the difference of the observations to obtain compressed data.

[0069] (3) Loss value function

[0070] The loss value function uses the mean square error function.

[0071] (4) Sparse matrix

[0072] The sparse matrix uses the wavelet transform matrix.

[0073] (2) Decoding process

[0074] (1) Entropy decoding and inverse quantization

[0075] Perform operations such as entropy decoding and inverse quantization on the received compressed data according to the H.26X encoding and decoding standard to obtain observations;

[0076] (2) CS Reconstruction

[0077] Reconstruct the data using the OSL0 algorithm.

[0078] (3) Observation Value Reconstruction

[0079] Obtain the true observation value of the macroblock by reconstructing the data, running the estimated vector, and the observation value of the reference macroblock.

[0080] (4) Data Reconstruction

[0081] Obtain the restored data based on the true observation value and the motion - estimated macroblock.

[0082] It should be noted that within the scope of protection defined in the claims of the present invention, the following embodiments can be combined and / or extended and replaced in any logical manner from the above - mentioned specific implementation manners, such as the disclosed technical principles, disclosed technical features, or implicitly disclosed technical features, etc.

[0083] Embodiment 1

[0084] A video coding method based on compressive sensing, comprising the following steps:

[0085] S1, Pre - processing: Group the collected video data and select the macroblock size, and then divide each frame of data into identical and non - overlapping macroblocks;

[0086] S2, Target Object Screening and Data Compression: Perform compression processing on key frames and non - key frames respectively.

[0087] Embodiment 2

[0088] On the basis of Embodiment 1, in step S1, the operation of grouping the collected video data and selecting the macroblock size, and then dividing each frame of data into identical and non - overlapping macroblocks specifically includes sub - steps:

[0089] S11, According to the user - set parameters, group the collected video YUV data, group the collected video YUV data according to the set parameter G, where G>1;

[0090] S12, Select the macroblock size according to the video picture resolution, and the macroblock size includes 16X16, 32X32, or 64X64;

[0091] S13, Select the first frame of each group of video YUV data as the reference key frame, and the remaining G - 1 frames as non - key frames, and then divide each frame of data into N macroblocks of the same size and non - overlapping.

[0092] Embodiment 3

[0093] Based on Embodiment 1, in step S2, the following steps are performed for key frames:

[0094] S201, perform compressive sampling on the first macroblock using observation matrix 1, and then use H.26X compression to achieve quantization and entropy coding to obtain the encoded data;

[0095] S202, perform entropy decoding, inverse quantization, and compressive sensing reconstruction on the encoded macroblock data to obtain the decoded video macroblock;

[0096] S203, perform intra-frame prediction on other macroblocks except the first encoded macroblock in step S202 to obtain the intra-frame prediction mode;

[0097] S204, determine whether the macroblock is a directly encoded macroblock or an intra-frame predicted macroblock, and perform corresponding processing: if the macroblock is a directly encoded macroblock, perform compressive sampling on the macroblock using observation matrix 1, and then use H.26X compression to achieve quantization and entropy coding to obtain the encoded data; if the macroblock is an intra-frame predicted macroblock, perform encoding according to the following steps: first, use the encoded macroblock data for decoding and the current macroblock for intra-frame prediction to obtain the predicted residual data; then perform compressive sampling on the residual data of the macroblock using observation matrix 2 to obtain the compressive sampling observation value; finally, perform quantization and entropy coding based on H.26X on the difference of the observation value to obtain the compressed data.

[0098] Embodiment 4

[0099] Based on Embodiment 1, in step S2, the following steps are performed for non-key frames:

[0100] S211, perform intra-frame and inter-frame prediction on each macroblock to obtain the motion estimation vector;

[0101] S212, if the macroblock is a directly encoded macroblock, perform compressive sampling using observation matrix 1, and perform quantization and entropy coding on the sampled data to obtain the compressed data;

[0102] If the macroblock is an intra-frame predicted macroblock, perform encoding according to the following steps: first, use intra-frame prediction to obtain the predicted residual data; then perform compressive sampling on the residual data of the macroblock using observation matrix 2 to obtain the compressive sampling observation value; finally, perform quantization and entropy coding based on H.26X on the difference of the observation value to obtain the compressed data;

[0103] If the macroblock is an inter - prediction macroblock, the encoding is performed according to the following steps: First, using the motion vector and the reference macroblock, based on entropy decoding, inverse quantization, and compressive sensing reconstruction, the decoded macroblock is obtained, and the loss value between this macroblock and the reference macroblock is obtained; then, the minimum loss value is compared with the reference value P1. If it is greater than P1, this macroblock directly performs compressive sampling using the observation matrix 3; if it is less than P1, this macroblock directly performs compressive sampling using the observation matrix 2; then, the motion vector with the minimum loss value is selected as the final motion estimation vector; then, the residual data between this macroblock and the reference macroblock under the optimal motion estimation vector is measured by compressive sensing using the observation matrix 2 to obtain the observed value; finally, the difference of this observed value is quantized and entropy - encoded based on H.26X to obtain the compressed data.

[0104] Example 5

[0105] Based on Example 4, the matrix uses a wavelet transform matrix.

[0106] Example 6

[0107] Based on Example 4, the loss value is calculated using the mean - square error function.

[0108] Example 7

[0109] A video decoding method based on compressive sensing performs the following steps:

[0110] S3, Entropy decoding and inverse quantization: Receive the compressed data obtained by the encoding method described in any one of Examples 1 - 4, and then perform entropy decoding and inverse quantization operations according to the H.26X codec standard to obtain the observed value data;

[0111] S4, CS reconstruction: Use the OSL0 algorithm to reconstruct the data obtained after being processed in step S3;

[0112] S5, Observed value reconstruction: Through the reconstructed data, the observed value of the motion estimation vector and the reference macroblock, obtain the true observed value of this macroblock;

[0113] S6, Data reconstruction: Obtain the restored data according to the true observed value and the motion - estimated macroblock.

[0114] Example 8

[0115] A video encoding device based on compressive sensing includes a processor and a memory. The memory stores a computer program, and when the computer program is loaded and executed by the processor, it performs the method described in any one of Examples 1 - 4.

[0116] Example 9

[0117] A video decoding device based on compressive sensing, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded and executed by the processor, the method described in Embodiment 7 is performed.

[0118] Embodiment 10

[0119] A video encoding and decoding device based on compressive sensing, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded and executed by the processor, the encoding method described in any one of Embodiments 1 to 4 is performed, and the decoding method described in Embodiment 7 is performed.

[0120] The units involved in the embodiments of the present invention can be implemented in software or in hardware, and the described units can also be provided in the processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0121] According to one aspect of the embodiments of the present invention, there is provided a computer program product or a computer program, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various alternative implementation manners.

[0122] As another aspect, the embodiments of the present invention further provide a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist alone without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device implements the methods described in the above embodiments.

Claims

1. A video encoding method based on compressed sensing, characterized in that: The following steps are involved: S1， Preprocessing: Group the collected video data and select the macroblock size, and then divide each frame of data into identical and non-overlapping macroblocks; S2, target object screening and data compression: compression processing is performed on key frames and non-key frames respectively.

2. The video encoding method based on compressed sensing according to claim 1, characterized in that: In step S1, the collected video data is grouped and the macroblock size is selected, and then each frame of data is divided into identical and non-overlapping macroblocks, which specifically includes the following sub-steps: S11, grouping the collected video YUV data according to the user set parameters, and grouping the collected video YUV data according to the set parameter G, G>1; S12, selecting a macroblock size according to the video picture resolution, wherein the macroblock size includes 16X16, 32X32 or 64X64; S13, selecting the first frame of each group of video YUV data as a reference key frame, and the remaining G-1 frames as non-key frames, and then dividing each frame of data into N macroblocks of the same size and non-overlapping.

3. The video encoding method based on compressed sensing according to claim 1, characterized in that: In step S2, the following steps are performed for the key frame: S201, compressing and sampling the first macroblock using observation matrix 1, and then implementing quantization and entropy coding using H.26X compression to obtain coded data; S202, reconstructing the encoded macroblock data by entropy decoding, inverse quantization, and compressed sensing to obtain a decoded video macroblock; S203, performing intra-frame prediction on the macroblocks other than the first macroblock encoded in step S202 to obtain an intra-frame prediction mode; S204, determine whether the macroblock is a directly encoded macroblock or an intra-frame predicted macroblock, and perform corresponding processing: if the macroblock is a directly encoded macroblock, the macroblock is compressed and sampled using observation matrix 1, and then quantization and entropy coding are implemented using H.26X compression to obtain encoded data; if the macroblock is an intra-frame predicted macroblock, the encoding is performed according to the following steps: first, the encoded macroblock data is decoded and the current macroblock is intra-frame predicted to obtain the predicted residual data; then the residual data of the macroblock is compressed and sampled using observation matrix 2 to obtain compressed sampled observation values; finally, the difference of the observation values ​​is quantized and entropy coded based on H.26X to obtain compressed data.

4. The video encoding method based on compressed sensing according to claim 1, characterized in that: In step S2, the following steps are performed for non-key frames: S211, performing intra-frame and inter-frame prediction on each macroblock to obtain a motion estimation vector; S212, if the macroblock is a directly coded macroblock, compression sampling is performed using observation matrix 1, and the sampled data is quantized and entropy coded to obtain compressed data; If the macroblock is an intra-frame prediction macroblock, the encoding is performed according to the following steps: first, the predicted residual data is obtained by using intra-frame prediction; then, the residual data of the macroblock is compressed and sampled using observation matrix 2 to obtain compressed sampled observation values; finally, the difference of the observation values ​​is quantized and entropy encoded based on H.26X to obtain compressed data; If the macroblock is an inter-frame prediction macroblock, the encoding is performed according to the following steps: first, the motion vector and the reference macroblock are used to obtain the decoded macroblock based on entropy decoding, inverse quantization, and compressed sensing reconstruction to obtain the loss value of the macroblock and the reference macroblock; then the minimum loss value is compared with the reference value P1. If it is greater than P1, the macroblock is directly compressed and sampled using the observation matrix 3; If it is less than P1, the macroblock directly uses observation matrix 2 for compression sampling; Then select the motion vector with the smallest loss value as the final motion estimation vector; calculate the residual data of the macroblock and the reference macroblock under the optimal motion estimation vector, and use observation matrix 2 for compressed sensing measurement to obtain the observation value; finally, perform quantization and entropy coding based on H.26X on the difference of the observation value to obtain compressed data.

5. The video encoding method based on compressed sensing according to claim 4, characterized in that: The matrix adopts a wavelet transform matrix.

6. The video encoding method based on compressed sensing according to claim 4, characterized in that: The loss value is calculated using a mean square error function.

7. A video decoding method based on compressed sensing, characterized in that: Follow these steps: S3, entropy decoding and dequantization: receiving the compressed data obtained by the encoding method according to any one of claims 1 to 4, and then performing entropy decoding and dequantization operations according to the H.26X encoding and decoding standard to obtain observation value data; S4, CS reconstruction: reconstruct the data obtained after processing in step S3 using the OSL0 algorithm; S5, observation value reconstruction: obtaining the real observation value of the macroblock through the reconstructed data, the running estimation vector and the observation value of the reference macroblock; S6, data reconstruction: obtaining restored data according to the real observation value and the motion estimation macroblock.

8. A video encoding device based on compressed sensing, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to any one of claims 1 to 4 is executed.

9. A video decoding device based on compressed sensing, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to claim 7 is executed.

10. A video encoding and decoding device based on compressed sensing, characterized in that: The invention comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the encoding method according to any one of claims 1 to 4 and the decoding method according to claim 7 are executed.

Images