Video compression method and system based on neural network

Abstract

The invention discloses a video compression method and system based on a neural network, and the method comprises the steps: carrying out the key frame extraction of video data, which comprises the steps: dividing a moving target key video data segment detected through a difference method into a plurality of sub-segments through sub-segment segmentation, calculating the mutual information amount between the sub-segments, carrying out the clustering according to the mutual information, and carrying out the key frame extraction of the video data; extracting a key frame according to a clustering result; carrying out key frame compression on the extracted key frame; performing inter-frame prediction according to the extracted key frame data, including performing motion estimation and prediction by using a current frame, a reference frame and an earlier-stage prediction frame, and then performing compressed encoding on a motion vector; reconstructing the motion compensation frame to obtain a predicted image frame, and comparing the predicted frame with the current frame to calculate an estimated residual error; encoding a residual between the original frame and the predicted frame using a residual auto-encoder network; and compensating the prediction frame by using the reconstruction residual error to obtain a code frame, and caching the code frame in a decoding frame buffer for next motion prediction.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a video compression method and system based on neural network. Background Art

[0002] Video data is large in volume. In order to save transmission bandwidth and storage space, it needs to be compressed before transmission and storage. Existing video compression standards such as the H.26x series are lossy compression. Although the compression rate is high, there is a loss of image quality.

[0003] Quickly detect key video segments in the video and capture key frames of video data to reduce video redundant data: In video compression, key frames are also reference data for predicting other frames, so they need to be lossless or almost lossless. Therefore, key data frame extraction, key frame lossless / near lossless compression, and inter-frame prediction based on key frames are technical issues that need to be solved. Summary of the invention

[0004] In view of this, the present invention provides a video compression method based on a neural network, which comprises:

[0005] S1: extract key frames from video data. This step includes: firstly dividing the key visual data segments of the moving target detected by the differential method into several sub-segments by sub-segment segmentation, then calculating the mutual information between the sub-segments, and then clustering according to the mutual information, and extracting key frames according to the clustering results.

[0006] S2: Perform key frame compression on the extracted key frames.

[0007] S3: Perform inter-frame prediction based on the extracted key frame data; this step includes: using the current frame, reference frame and previous predicted frame to perform motion estimation and prediction, and then compressing and encoding the motion vector; obtaining the predicted image frame after motion compensation frame reconstruction, and comparing the predicted frame with the current frame to calculate the estimated residual; using the residual autoencoder network to encode the residual between the original frame and the predicted frame; using the reconstructed residual to compensate the predicted frame to obtain a code frame, and caching it in the decoding frame buffer for the next step of motion prediction.

[0008] Preferably, in step S2, the JPEG-LS image coding algorithm is used to compress key frames; the true pixel value and the corrected error are quantized and then encoded using a distortion quantization parameter, so that the average code length of the Golomb code is reduced, thereby improving the compression rate.

[0009] Preferably, in step S2, key frame compression is performed based on a neural network, which includes: first compressing and reconstructing the image lossily; then calculating the lossy compression reconstruction residual and uniformly quantizing it to meet a given error bound; jointly calculating an entropy parameter of the lossy reconstructed image and the residual, and then performing entropy coding.

[0010] In addition, the present invention also provides a video compression system based on a neural network, which includes a key frame extraction module, a key frame compression module and an inter-frame prediction module, wherein the key frame compression module is arranged between the key frame extraction module and the inter-frame prediction module. The key frame extraction module is used to complete the detection of video data segments with correlation and the extraction of key retrieval frames in the data segments; the key frame compression module is used to compress the extracted key frames; and the inter-frame prediction module is used to perform segmented prediction according to the results of key frame extraction.

[0011] The present invention has the following beneficial effects:

[0012] In criminal investigation or other applications, it is often necessary to search for security surveillance videos. Manual sequential search is time-consuming and labor-intensive, and the efficiency is extremely low. It is necessary to extract key frames as indexes. Key frames often contain the start and end of a highly correlated video. The present invention considers combining key frame extraction with video compression to compress videos with strong temporal correlation in segments, thereby obtaining higher compression efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the process flow of the video compression method based on neural network of the present invention;

[0014] Figure 2 A schematic diagram of the key frame extraction process in the present invention;

[0015] Figure 3 It is a schematic diagram of the process of near-lossless compression in the present invention;

[0016] Figure 4 It is a flowchart diagram of the inter-frame prediction method in the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0018] In criminal investigation or other applications, it is often necessary to search for security surveillance videos. Manual sequential search is time-consuming and labor-intensive, and the efficiency is extremely low. It is necessary to extract key frames as indexes. Key frames often contain the start and end of a highly relevant video. Drawing on this idea, this project considers combining key frame extraction with video compression to compress videos with strong temporal correlation in segments in order to achieve higher compression efficiency.

[0019] like Figure 1As shown, the video compression method based on neural network of the present invention mainly includes three parts: key frame extraction, key frame compression and inter-frame prediction. The key frame extraction module is used to complete the detection of video data segments with strong correlation and the extraction of key search frames in the data segments; the key frame compression is used to compress the extracted key frames losslessly / nearly losslessly; the inter-frame prediction module is used to perform segmented prediction based on the results of key frame extraction.

[0020] There are many similar adjacent frames in a video, resulting in a large amount of redundant information. Extracting key frames from the video for storage or compression can greatly simplify the video information. The present invention intends to use the differential method to detect moving target frames in surveillance videos, and complete the extraction of key frames such as the starting frame and the ending frame of the moving target for the key video segments where the moving target appears.

[0021] like Figure 2 As shown, the key frame extraction includes: firstly dividing the key visual data segment of the moving target detected by the differential method into several sub-segments by sub-segment segmentation, then calculating the mutual information between the sub-segments, and then clustering according to the mutual information, and extracting the key frame according to the clustering result. In addition, optionally, part or all of the key frame extraction process can be adopted in the form of a deep neural network, and CNN, LSTM, autoencoder, generative adversarial network or hybrid network are all possible methods.

[0022] The key frame compression is described in detail below. The key frame contains important information of the video and is the reference data for inter-frame prediction. In situations where the video quality is required to be relatively high, the key frame needs to be losslessly compressed or nearly losslessly compressed. The present invention can use JPEG-L and a neural network-based method to compress the key frame data.

[0023] Key frame compression scheme based on JPEG-LS: JPEG-LS is an image coding algorithm used for lossless / near-lossless compression of static continuous tone images in ISO / ITU standards. The JPEG-LS algorithm has excellent performance and low algorithm complexity, but its output bit rate fluctuations in lossless compression mode are large and uncontrollable, and its application scenarios are relatively limited. In near-lossless mode, the distortion quantization parameter is used to quantize and encode the true pixel value and the corrected error, so that the average code length of Golomb coding is reduced and the compression rate is improved. By utilizing the characteristics of human visual perception and the prediction scheme in JPEG-LS, the information loss is distributed to multiple levels, and the final compression ratio is controlled to be close to the target compression ratio expected by the end user.

[0024] Key frame compression scheme based on neural network: Inspired by the traditional "lossy plus residual" coding scheme, lossy image compression and residual compression are combined to obtain a nearly lossless compression reconstruction of the original image. Figure 3As shown in the figure, specifically, the image is firstly compressed and reconstructed with lossy compression; then the lossy compression reconstruction residual is calculated and uniformly quantized to meet the given error bound. The error bound size represents the image compression loss size. If the error bound is zero, it is lossless image compression; the entropy parameter is calculated by combining the lossy reconstructed image and the residual, and then entropy coding is performed.

[0025] like Figure 4 The figure shows a flow chart of the inter-frame prediction method. Inter-frame prediction mainly includes modules such as motion estimation and prediction, motion field compression coding, motion field compensation, frame reconstruction residual compression and residual compensation. Specifically, first, the current frame, reference frame and previous prediction frame are used for motion estimation and prediction, and then the motion vector is compressed and encoded; the predicted image frame is obtained after motion compensation frame reconstruction, and the predicted frame is compared with the current frame to calculate the estimated residual; the residual autoencoder network is used to encode the residual between the original frame and the predicted frame; the predicted frame is compensated with the reconstructed residual to obtain a code frame, and it is cached in the decoding frame buffer for the next step of motion prediction.

[0026] In addition, the present invention also provides a video compression system based on a neural network, which includes a key frame extraction module, a key frame compression module and an inter-frame prediction module, wherein the key frame compression module is arranged between the key frame extraction module and the inter-frame prediction module. The key frame extraction module is used to complete the detection of video data segments with correlation and the extraction of key retrieval frames in the data segments; the key frame compression module is used to compress the extracted key frames; and the inter-frame prediction module is used to perform segmented prediction according to the results of key frame extraction.

[0027] Those skilled in the art will appreciate that, although specific functions are discussed above with reference to specific modules, it should be noted that the functions of the various modules discussed herein may be divided into multiple modules, and / or at least some functions of multiple modules may be combined into a single module. In addition, the specific module discussed herein performing an action includes the specific module itself performing the action, or alternatively the specific module calling or otherwise accessing another component or module that performs the action (or performing the action in conjunction with the specific module). Therefore, the specific module that performs an action may include the specific module itself that performs the action and / or another module that the specific module that performs the action calls or otherwise accesses.

[0028] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0029] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0030] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.

Claims

1. A video compression method based on neural network, characterized in that: The following steps are involved: S1: extracting key frames from video data, which includes: firstly dividing the key visual data segment of the moving target detected by the difference method into several sub-segments by sub-segment segmentation, then calculating the mutual information between the sub-segments, and then clustering according to the mutual information, and extracting key frames according to the clustering results; S2: perform key frame compression on the extracted key frames; S3: Perform inter-frame prediction based on the extracted key frame data; this step includes: using the current frame, reference frame and previous predicted frame to perform motion estimation and prediction, and then compressing and encoding the motion vector; obtaining the predicted image frame after motion compensation frame reconstruction, and comparing the predicted frame with the current frame to calculate the estimated residual; using the residual autoencoder network to encode the residual between the original frame and the predicted frame; using the reconstructed residual to compensate the predicted frame to obtain a code frame, and caching it in the decoding frame buffer for the next step of motion prediction.

2. The method according to claim 1, characterized in that In step S2, the JPEG-LS image coding algorithm is used to compress the key frames; the true pixel value and the corrected error are quantized and then encoded using the distortion quantization parameter, so that the average code length of the Golomb code is reduced, thereby improving the compression rate.

3. The method according to claim 2, characterized in that In step S2, key frame compression is performed based on a neural network, which includes: first compressing and reconstructing the image lossily; then calculating the lossy compression reconstruction residual and uniformly quantizing it to meet a given error limit; jointly calculating an entropy parameter of the lossy reconstructed image and the residual, and then performing entropy coding.

4. A video compression system based on a neural network, characterized in that: The video compression method according to claim 1 is adopted, and the video compression system based on the neural network includes a key frame extraction module, a key frame compression module and an inter-frame prediction module, and the key frame compression module is arranged between the key frame extraction module and the inter-frame prediction module, wherein: The key frame extraction module is used to complete the detection of relevant video data segments and the extraction of key retrieval frames in the data segments; the key frame compression module is used to compress the extracted key frames; and the inter-frame prediction module is used to perform segmented prediction based on the results of key frame extraction.

Images