Video compression method and device, computer readable medium and electronic equipment

Abstract

The invention relates to the field of video compression, and discloses a video compression method and device, a computer readable medium and electronic equipment. The method comprises the steps that hierarchical compression is conducted on an original video frame through network modules of multiple hierarchies in a video compression system, the network module of the first hierarchy is used for compressing the original video frame, and when compression of each hierarchy after the first hierarchy is conducted, the network module of the second hierarchy is used for compressing the original video frame; performing compression of the current level at least based on the compressed video frame output by the network module of the previous level, wherein the network module of at least one level processes motion among multiple frames by using single motion mapping; and fusing the compressed video frames output by the network modules of all levels through a fusion module in the video compression system to obtain compressed video frames corresponding to the original video frames. According to the method, on the premise that the bit rate is not increased, the quality of the compressed video is remarkably improved, and the use of a memory can be reduced.

Description

Background Art

[0002] With the development of information technology, video compression becomes more and more important.

[0003] However, the quality of compressed videos produced by current video compression schemes is often not high, and the compression rate is often not high, and the bit rate requirement is high. In addition, the consumption of memory resources is also relatively large, which makes it difficult to meet the needs of business scenarios. Summary of the invention

[0004] In the field of video compression technology, in order to solve the above technical problems, the purpose of this application is to provide a video compression method, device, computer-readable medium and electronic device.

[0005] According to one aspect of an embodiment of the present application, a video compression method is provided, the method comprising:

[0006] The original video frames are hierarchically compressed by network modules of multiple levels in the video compression system, wherein the network modules of the first level are used to compress the original video frames, and when each level of compression after the first level is performed, the current level of compression is performed based on at least the compressed video frames output by the network modules of the previous level, and the network modules of at least one level use a single motion map to process motion between multiple frames;

[0007] The compressed video frames output by the network modules at each level are fused through the fusion module in the video compression system to obtain compressed video frames corresponding to the original video frames.

[0008] According to one aspect of an embodiment of the present application, a video compression device is provided, the device comprising:

[0009] A hierarchical compression module, used for hierarchical compression of original video frames through network modules of multiple layers in a video compression system, wherein the network modules of the first layer are used for compressing the original video frames, and when performing compression of each layer after the first layer, the compression of the current layer is performed based on at least the compressed video frames output by the network modules of the previous layer, and the network modules of at least one layer use a single motion map to process motion between multiple frames;

[0010] The video frame fusion module is used to fuse the compressed video frames output by the network modules at each level through the fusion module in the video compression system to obtain the compressed video frames corresponding to the original video frames.

[0011] In some embodiments of the present application, based on the above solution, the hierarchical compression module is configured as follows:

[0012] Performing high-quality compression on the original video frame to obtain a high-quality compressed frame having a bit rate lower than that of the original video frame;

[0013] Using the original video frame and the high-quality compressed frame as bidirectional reference video frames, bidirectionally deeply compressing the bidirectional reference video frame to obtain a medium-quality compressed frame having a smaller bit rate than the high-quality compressed frame;

[0014] Performing single motion deep compression on the medium quality compressed frame to process motion between multiple frames using a single motion map to obtain a low quality compressed frame having a lower bit rate than the medium quality compressed frame;

[0015] The video frame fusion module is configured as follows:

[0016] The high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are merged to obtain a compressed video frame corresponding to the original video frame.

[0017] In some embodiments of the present application, based on the above solution, the video frame fusion module is configured as follows:

[0018] Determine weights corresponding to the high-quality compressed frame, the medium-quality compressed frame, and the low-quality compressed frame, respectively;

[0019] According to the weights, weighted processing is performed on the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame to obtain a compressed video frame corresponding to the original video frame.

[0020] In some embodiments of the present application, based on the above solution, the video frame fusion module is configured as follows:

[0021] Obtaining compression quality and bit rate of a compressed video frame generated corresponding to a target video frame preceding the original video frame;

[0022] constructing a quality feature of the original video frame according to the compression quality and the bit rate;

[0023] The weights respectively corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are determined according to the quality characteristics of the original video frame.

[0024] In some embodiments of the present application, based on the aforementioned scheme, the fusion of the high-quality compressed frames, the medium-quality compressed frames and the low-quality compressed frames is completed through a weighted cyclic quality enhancement network module, and the weighted cyclic quality enhancement network module is constructed based on the QG-ConvLSTM network structure. The weighted processing of the high-quality compressed frames, the medium-quality compressed frames and the low-quality compressed frames is completed through a quality gating unit in the weighted cyclic quality enhancement network module, and the quality gating unit is used to utilize the correlation between multiple compressed frames to perform video frame fusion.

[0025] In some embodiments of the present application, based on the aforementioned scheme, the bidirectional deep compression is accomplished through a bidirectional deep compression network module, and the bidirectional deep compression network module includes at least one of the following sub-networks: motion estimation sub-network, motion compression sub-network, motion preprocessing sub-network, and residual compression sub-network.

[0026] In some embodiments of the present application, based on the aforementioned scheme, the single motion deep compression is accomplished through a single motion deep compression network module, and the single motion deep compression network module includes at least one of the following subnetworks: motion estimation subnetwork, motion compression subnetwork, motion preprocessing subnetwork, and residual compression subnetwork.

[0027] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0028] According to one aspect of an embodiment of the present application, there is provided an electronic device, including:

[0029] one or more processors;

[0030] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the above embodiments.

[0031] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0032] The video compression method provided in the embodiment of the present application includes the following steps: hierarchical compression of original video frames through network modules of multiple levels in a video compression system, wherein the network module of the first level is used to compress the original video frames, and when performing compression at each level after the first level, compression of the current level is performed based on at least the compressed video frames output by the network module of the previous level, and the network module of at least one level uses a single motion map to process the motion between multiple frames; and the compressed video frames output by the network modules of each level are fused through a fusion module in the video compression system to obtain compressed video frames corresponding to the original video frames.

[0033] Under this method, since the layered compression of the original video frames is realized, and after the compressed video frames output by different network modules are obtained, the compressed video frames corresponding to the original video frames are obtained by fusing the compressed video frames output by different network modules, the quality of the compressed video is significantly improved without increasing the bit rate; and since the network modules of at least one layer can use a single motion map to process the motion between multiple frames, the use of memory is significantly reduced, which can meet the needs of business scenarios and improve the user experience.

[0034] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] Figure 1 is a flowchart of a video compression method according to an exemplary embodiment;

[0037] Figure 2 According to an exemplary embodiment, Figure 1 A flowchart of details of step 110 and step 120 in the embodiment;

[0038] Figure 3 is a block diagram showing a principle of video compression according to an exemplary embodiment;

[0039] Figure 4 is a flowchart showing a method of determining weights respectively corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame according to an exemplary embodiment;

[0040] Figure 5 is a schematic diagram of an overall process according to an exemplary embodiment;

[0041] Figure 6is a block diagram of a video compression device according to an exemplary embodiment;

[0042] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0044] In addition, the drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0045] With the rapid development of fields such as the Internet of Things and autonomous driving that rely on massive amounts of video data, efficient video compression technology has become crucial. Traditional video compression methods usually rely on a single loss function for model training and apply the same compression strategy to all video frames. That is, the structure of the traditional compression model is to directly generate compressed video frames from the original video frames. Although this method simplifies the compression process, it results in low quality of compressed videos, which is difficult to meet the needs of high-quality video storage and transmission.

[0046] In addition, traditional video compression methods usually use only one model to compress all video frames, which requires a high resolution of the input original video frames. Although such high-resolution video frames can improve video quality, they require longer rendering time and larger memory resources during the compression process, thus limiting the performance and efficiency of the system.

[0047] At the same time, existing video compression technologies usually adopt a single-layer compression strategy, which cannot fully utilize the correlation between video frames. This results in the failure to effectively save bandwidth and storage resources during the compression process, and has high requirements for high-quality video transmission and storage, which makes it difficult to meet the actual needs of modern visual networking and autonomous driving.

[0048] To this end, the present application first provides a video compression method, which proposes a hierarchical quality layer video compression scheme, which significantly improves the quality of the compressed video without increasing the bit rate by compressing the video frames in layers and utilizing the correlation between adjacent frames. In addition, the present application also proposes a single-motion deep compression network module, which can significantly reduce the use of memory while ensuring video quality, and provides strong technical support for efficient video storage and transmission. These innovative improvements provide efficient and economical solutions for application scenarios that rely on massive videos, such as visual networking and autonomous driving.

[0049] Figure 1 FIG. 1 is a flowchart of a video compression method according to an exemplary embodiment. The video compression method can be executed by any device with computing and processing functions. Figure 1 As shown, the video compression method may include the following steps:

[0050] Step 110, hierarchical compression is performed on the original video frames through multiple layers of network modules in the video compression system, wherein the network modules of the first layer are used to compress the original video frames, and when compression is performed on each layer after the first layer, compression of the current layer is performed based on at least the compressed video frames output by the network modules of the previous layer, and the network modules of at least one layer use a single motion map to process the motion between multiple frames.

[0051] The multiple layers of network modules in the video compression system are multiple network modules connected in sequence, and each network module is a layer. The number of network modules included in the video compression system can be arbitrary, for example, the number of network modules included in the video compression system can be 3, 5, 8, etc.

[0052] The deeper the level of the network module (ie, the larger the level or the later the level), the lower the bit rate and quality of the compressed video frame outputted by the network module can be.

[0053] Figure 2 According to an exemplary embodiment, Figure 1 Flow chart of details of step 110 and step 120 in the embodiment. Figure 2 As shown, hierarchical compression of the original video frame by multiple layers of network modules in the video compression system may include the following steps:

[0054] Step 111 , high-quality compression is performed on the original video frame to obtain a high-quality compressed frame whose bit rate is smaller than that of the original video frame.

[0055] The original video frame is a certain video frame of the target video. The original video frame can be any video frame in the target video. Of course, the original video frame can be any video frame after the second video frame in the target video.

[0056] The original video frame may also be a video frame to be compressed collected by an external device. The original video frame may be compressed with high quality by an image compression module.

[0057] Figure 3 is a block diagram showing the principle of video compression according to an exemplary embodiment. Figure 3 As shown in the figure, after the original video frames are input into the video compression system, they are first transmitted to the image compression module, which is responsible for efficiently compressing the input original video frames into high-quality compressed frames. As the part with the most concentrated bit rate consumption in the entire system, this module not only ensures the high fidelity of the compressed video, but also lays the foundation for the compression and quality enhancement of subsequent frames. The module can be constructed by the structure of a convolutional neural network (CNN) to achieve efficient feature extraction and compression. The module automatically identifies key information in the video through a deep learning algorithm and uses advanced coding technology to ensure that the compressed video frames significantly reduce the bit rate while maintaining high fidelity. The quality of high-quality compressed frames can be lower than that of the original video frames.

[0058] Step 112: using the original video frame and the high-quality compressed frame as bidirectional reference video frames, bidirectional deep compression is performed on the bidirectional reference video frame to obtain a medium-quality compressed frame whose bit rate is smaller than that of the high-quality compressed frame.

[0059] In one embodiment of the present application, the bidirectional deep compression is accomplished through a bidirectional deep compression network module, which includes at least one of the following sub-networks: a motion estimation sub-network, a motion compression sub-network, a motion preprocessing sub-network, and a residual compression sub-network.

[0060] The bidirectional deep compression network module may include one or more of the above-mentioned sub-networks, or even all of the above-mentioned sub-networks, that is, the bidirectional deep compression network module may be composed of a motion estimation (ME) sub-network, a motion compression (MC) sub-network, a motion preprocessing (MP) sub-network, and a residual compression (RC) sub-network, and these sub-networks may all be constructed by a CNN network, ensuring the efficiency and accuracy of the compression process. The relationship between the sub-networks in the bidirectional deep compression network module may be set and adaptively adjusted as needed, and the sub-networks may be serial or parallel.

[0061] The original video frames and the high-quality compressed frames are used as bidirectional references for generating subsequent medium-quality compressed layers and subsequent low-quality compressed frames.

[0062] Please continue to see Figure 3 As shown, the original video frame and the high-quality compressed frame are input into the bidirectional deep compression network module. The bidirectional deep compression network module uses these two frames as bidirectional references to obtain the medium-quality compressed frame output by the bidirectional deep compression network module. The compression quality of the medium-quality compressed frame can be lower than the compression quality of the high-quality compressed frame. This step not only improves the compression efficiency, but also provides important information for the compression and enhancement of the subsequent third-layer low-quality compression layer.

[0063] Therefore, the bidirectional deep compression network module is responsible for processing the input high-quality compression layer and the original video frames obtained from the external device to generate the medium-quality compression layer. Through a clever bidirectional reference mechanism, the module uses the original video frame and the high-quality compressed frame as a bidirectional reference, providing valuable information for the quality enhancement of the low-quality compressed frames in the third layer. This strategy allows the module to optimize the use of bit rate without sacrificing video quality, achieving a balance between compression efficiency and video quality.

[0064] Step 113: Perform single motion deep compression on the medium quality compressed frame to process motion between multiple frames using a single motion map to obtain a low quality compressed frame having a lower bit rate than the medium quality compressed frame.

[0065] In one embodiment of the present application, the single motion deep compression is accomplished by a single motion deep compression network module, and the single motion deep compression network module includes at least one of the following sub-networks: motion estimation sub-network, motion compression sub-network, motion preprocessing sub-network, and residual compression sub-network.

[0066] The single motion deep compression network module may include one or more of the above-mentioned sub-networks, and may even include all of the above-mentioned sub-networks, that is, the single motion deep compression network module may be composed of a motion estimation (ME) sub-network, a motion compression (MC) sub-network, a motion preprocessing (MP) sub-network, and a residual compression (RC) sub-network, and these sub-networks may all be constructed by a CNN network, thereby ensuring the efficiency and accuracy of the compression process. The sub-networks included in the single motion deep compression network module are similar to the sub-networks included in the bidirectional deep compression network module. The connection relationship between the sub-networks in the single motion deep compression network module may be the same as or different from the connection relationship between the sub-networks in the bidirectional deep compression network module. The relationship between the sub-networks in the single motion deep compression network module may be set and adaptively adjusted as needed, and the sub-networks may be serial or parallel.

[0067] The medium-quality compressed frames can be used as input to the single-motion deep compression network module.

[0068] Please continue to see Figure 3 As shown, the single motion deep compression network module is used to calculate and process the medium-quality compressed frames output by the bidirectional deep compression network module. The single motion deep compression network module uses the information of the high-quality compressed frames and medium-quality compressed frames generated by the image compression module and the bidirectional deep compression network module, and uses a single motion map to process the motion between multiple frames, and compresses the medium-quality compressed frames into low-quality compressed frames. The compression quality of the low-quality compressed frames can be less than the compression quality of the medium-quality compressed frames. The low-quality compressed frames output by the single motion deep compression network module will be used as input for the subsequent weight cycle quality enhancement network module.

[0069] Therefore, the role of the single motion deep compression network module is to efficiently convert medium-quality compressed frames into low-quality compressed frames. By adopting a single motion map to capture the motion changes between multiple frames of video, this module significantly reduces the bit rate requirement while maintaining compression efficiency.

[0070] Step 120, the compressed video frames output by the network modules at each level are merged by a fusion module in the video compression system to obtain compressed video frames corresponding to the original video frames.

[0071] The fusion module can be Figure 3 The weight cycle quality enhancement network module in .

[0072] Please continue to see Figure 2 As shown, the compressed video frames output by the network modules at each level are fused by the fusion module in the video compression system to obtain the compressed video frames corresponding to the original video frames, which may specifically include the following steps:

[0073] Step 120': merge the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame to obtain a compressed video frame corresponding to the original video frame.

[0074] Fusion of the high-quality compressed frames, the medium-quality compressed frames, and the low-quality compressed frames may refer to the following operations: intelligently analyzing and classifying the video content in an automated manner, and distinguishing important and non-important segments of the video without manual intervention. Specifically, it includes automatic video content analysis, video segmentation, compression strategy selection, time tagging and encapsulation. Automatic video content analysis: Analyze the video content using a deep learning algorithm to automatically identify key frames and non-key frames in the video. Video segment segmentation: According to the analysis results, the video is segmented into key video segments and non-key video segments. Compression strategy selection: Use a lossless compression algorithm for key video segments to ensure the integrity of key information; use a lossy compression algorithm for non-key video segments to optimize storage and transmission efficiency. Time tagging and encapsulation: Time tag the compressed key video segments to ensure the continuity and synchronization of the video; encapsulate all compressed segments in the same file in chronological order to complete the video compression.

[0075] In one embodiment of the present application, the fusion of the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame is completed by a weighted cyclic quality enhancement network module, and the weighted cyclic quality enhancement network module is constructed based on a QG-ConvLSTM network structure. The weighted cyclic quality enhancement network module can enhance the compression quality by using multi-frame information.

[0076] The high-quality compressed frames generated in the first stage, the medium-quality compressed frames in the second stage, and the low-quality compressed frames in the third stage can be input into the weighted cyclic quality enhancement network module.

[0077] The QG-ConvLSTM (Quality-Gated Convolutional Long Short-Term Memory) network is a deep learning model that combines convolution operations and long short-term memory (LSTM) units. It is particularly suitable for enhancing the quality of compressed videos. It aims to enhance the quality of compressed videos through deep learning models and spatial and temporal features, especially for videos with high compression ratios. QG-ConvLSTM can specifically include three components: convolution operations (Convolution), long short-term memory units (LSTM), and quality gating units. Convolution operations can capture local spatial features in video frames and extract information such as edges and textures of images. Long short-term memory units (LSTM) are a special type of recurrent neural network (RNN) that can effectively process long-term dependencies in sequence data.

[0078] Please continue to see Figure 3 As shown, the obtained high-quality compressed frames, medium-quality compressed frames and low-quality compressed frames are input into the weighted cycle quality enhancement network module to output compressed video frames.

[0079] In one embodiment of the present application, the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are fused to obtain a compressed video frame corresponding to the original video frame, which may specifically include the following steps: determining weights corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame, respectively; and performing weighted processing on the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame according to each of the weights to obtain a compressed video frame corresponding to the original video frame.

[0080] The weights respectively corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame may be determined by a weighted cyclic quality enhancement network module.

[0081] Figure 4 is a flowchart showing a method for determining weights corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame, respectively, according to an exemplary embodiment. Figure 4 As shown, determining the weights corresponding to the high-quality compressed frame, the medium-quality compressed frame, and the low-quality compressed frame respectively may specifically include the following steps:

[0082] Step 410: Obtain the compression quality and bit rate of the compressed video frame generated corresponding to the target video frame before the original video frame.

[0083] Since the compression is encoded in the bitstream, the compression quality Qi and bit rate Bi corresponding to video frame i can be accessed from the bitstream.

[0084] The target video frame before the original video frame may be any video frame whose frame number is smaller than the original video frame. The target video frame may be adjacent to the original video frame or may not be adjacent to the original video frame.

[0085] The target video frame may be a video frame in the target video whose frame number is 2 less than that of the original video frame.

[0086] Step 420: construct a quality feature of the original video frame according to the compression quality and the bit rate.

[0087] F i = {Q j ,B j} i+2 j=i-2 is used as the quality feature, where i+2 and j=i-2 both represent that i is 2 greater than j, i is the frame number of the original video frame, and j is the frame number of the target video frame that is 2 less than the original video frame. Q j is the compression quality of the target video frame j, B j is the bit rate of the target video frame j, F iis the quality feature of the original video frame.

[0088] Step 430: Determine weights corresponding to the high-quality compressed frame, the medium-quality compressed frame, and the low-quality compressed frame, respectively, according to the quality characteristics of the original video frame.

[0089] In the weighted cycle quality enhancement network module, a weight generator can be learned to transform the quality feature F of the original video frame i By inputting into the weight generator, the generated weight corresponding to each compressed frame can be obtained.

[0090] The weighted cyclic quality enhancement network module can fully and reasonably utilize multi-frame information to improve the compression quality of the entire video sequence by weighting each compressed frame. Through weighted processing, the model compresses different frames at different ratios according to their criticality. It can give high weights to key frames and low weights to non-key frames to reduce quality loss during compression.

[0091] In one embodiment of the present application, weighted processing of the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame is completed by a quality gating unit in the weighted cyclic quality enhancement network module, and the quality gating unit is used to utilize the correlation between multiple compressed frames to perform video frame fusion.

[0092] In one embodiment of the present application, the weighted cyclic quality enhancement network module includes a residual block, and the residual block uses a skip connection.

[0093] Therefore, the weighted recurrent quality enhancement network module is designed based on the QG-ConvLSTM method, which has a spatiotemporal structure and uses quality gating units to exploit multi-frame correlations. Different from the QG-ConvLSTM method, residual blocks are used in the spatial feature extraction and reconstruction network, and skip connections are used to improve the enhancement performance.

[0094] The recurrent units of the weighted recurrent quality enhancement network module are weighted according to the quality features to reasonably utilize the multi-frame information. In particular, the quality of the third layer can be significantly improved by utilizing the high-quality and medium-quality information of the first and second layers. On low-quality compressed frames, bit rate can be saved because no additional information needs to be stored to improve quality.

[0095] Figure 5 is a schematic diagram of the overall process according to an exemplary embodiment. Figure 5 As shown, the overall process of the embodiment of the present application may include the following steps:

[0096] Step S1-1: Input a video image frame.

[0097] Step S1-2: Construct an image compression module.

[0098] Step S1-3: Obtain high-quality compressed frames.

[0099] Step S2-1: Construct a bidirectional deep compression network module.

[0100] Step S2-2: Obtain a bidirectional reference.

[0101] Step S2-3: Obtain medium quality compressed frames.

[0102] Step S3-1: Construct a single-motion deep compression network module.

[0103] Step S3-2: Obtain low-quality compressed frames.

[0104] Step S4-1: Construct a weighted cycle quality enhancement network module.

[0105] Step S4-2: Output compressed video frames.

[0106] In summary, according to the video compression method provided in the embodiment of the present application, the concept based on hierarchical image compression is proposed for the first time, and the three-layer quality layer and the cyclic network structure are innovatively introduced, which significantly improves the quality of video compression without increasing the bit rate. It is particularly worth mentioning that the present application creatively designs a bidirectional deep compression network module, which not only uses the original video frame and the high-quality compressed frame of the first layer as a bidirectional reference, but also enhances the compression effect of the low-quality frame of the third layer through information interaction between multiple frames. In addition, the single deep compression network module captures the motion changes between multiple frames through a single motion mapping. This design cleverly reduces the bit rate requirement while maintaining the compression efficiency. The present application can achieve high-quality compression of video frames only through software algorithms without relying on special hardware devices. This not only saves video storage resources, but also reduces the bandwidth required for video transmission. For application fields such as visual networking and autonomous driving that require processing of large amounts of video data, the present application provides strong technical support.

[0107] In summary, the solution of the embodiment of the present application can be used to compress video frames in video, audio, entertainment and other related application scenarios. The video image frames are compressed, transmitted and stored, and the compressed video can be used in applications such as DVD and Blu-ray. Video compression based on the solution of the embodiment of the present application reduces the redundancy in the original data, reduces the requirements for transmission bandwidth and storage space, and can achieve higher quality video services on limited bandwidth.

[0108] According to the second aspect of the present application, the present application also provides a video compression device. The following is an embodiment of the device of the present application.

[0109] Figure 6 FIG. 1 is a block diagram of a video compression device according to an exemplary embodiment. Figure 6 As shown, the video compression device 600 includes:

[0110] A hierarchical compression module 610 is used to perform hierarchical compression on the original video frame through multiple layers of network modules in the video compression system, wherein the network module of the first layer is used to compress the original video frame, and when performing compression on each layer after the first layer, the compression of the current layer is performed based on at least the compressed video frame output by the network module of the previous layer, and the network module of at least one layer uses a single motion map to process the motion between multiple frames;

[0111] The video frame fusion module 620 is used to fuse the compressed video frames output by the network modules at each level through the fusion module in the video compression system to obtain the compressed video frames corresponding to the original video frames.

[0112] In some embodiments of the present application, based on the above solution, the hierarchical compression module 610 is configured as follows:

[0113] Performing high-quality compression on the original video frame to obtain a high-quality compressed frame having a bit rate lower than that of the original video frame;

[0114] Using the original video frame and the high-quality compressed frame as bidirectional reference video frames, bidirectionally deeply compressing the bidirectional reference video frame to obtain a medium-quality compressed frame having a smaller bit rate than the high-quality compressed frame;

[0115] Performing single motion deep compression on the medium quality compressed frame to process motion between multiple frames using a single motion map to obtain a low quality compressed frame having a lower bit rate than the medium quality compressed frame;

[0116] The video frame fusion module 620 is configured as follows:

[0117] The high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are merged to obtain a compressed video frame corresponding to the original video frame.

[0118] In some embodiments of the present application, based on the above solution, the video frame fusion module 620 is configured as follows:

[0119] Determine weights corresponding to the high-quality compressed frame, the medium-quality compressed frame, and the low-quality compressed frame, respectively;

[0120] According to the weights, weighted processing is performed on the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame to obtain a compressed video frame corresponding to the original video frame.

[0121] In some embodiments of the present application, based on the above solution, the video frame fusion module 620 is configured as follows:

[0122] Obtaining compression quality and bit rate of a compressed video frame generated corresponding to a target video frame preceding the original video frame;

[0123] constructing a quality feature of the original video frame according to the compression quality and the bit rate;

[0124] The weights respectively corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are determined according to the quality characteristics of the original video frame.

[0125] In some embodiments of the present application, based on the aforementioned scheme, the fusion of the high-quality compressed frames, the medium-quality compressed frames and the low-quality compressed frames is completed through a weighted cyclic quality enhancement network module, and the weighted cyclic quality enhancement network module is constructed based on the QG-ConvLSTM network structure. The weighted processing of the high-quality compressed frames, the medium-quality compressed frames and the low-quality compressed frames is completed through a quality gating unit in the weighted cyclic quality enhancement network module, and the quality gating unit is used to utilize the correlation between multiple compressed frames to perform video frame fusion.

[0126] In some embodiments of the present application, based on the aforementioned scheme, the bidirectional deep compression is accomplished through a bidirectional deep compression network module, and the bidirectional deep compression network module includes at least one of the following sub-networks: motion estimation sub-network, motion compression sub-network, motion preprocessing sub-network, and residual compression sub-network.

[0127] In some embodiments of the present application, based on the aforementioned scheme, the single motion deep compression is accomplished through a single motion deep compression network module, and the single motion deep compression network module includes at least one of the following subnetworks: motion estimation subnetwork, motion compression subnetwork, motion preprocessing subnetwork, and residual compression subnetwork.

[0128] According to another aspect of the present application, an electronic device capable of implementing the above method is also provided.

[0129] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0130] Figure 7A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0131] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0132] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703, such as executing the method described in the above embodiment. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, ROM 702 and RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0133] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0134] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 709, and / or installed from a removable medium 711. When the computer program is executed by a central processing unit (CPU) 701, various functions defined in the system of the present application are executed.

[0135] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0136] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0138] As one aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0139] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0140] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0141] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0142] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A video compression method, characterized in that: The method comprises: The original video frames are hierarchically compressed by network modules of multiple levels in the video compression system, wherein the network modules of the first level are used to compress the original video frames, and when each level of compression after the first level is performed, the current level of compression is performed based on at least the compressed video frames output by the network modules of the previous level, and the network modules of at least one level use a single motion map to process motion between multiple frames; The compressed video frames output by the network modules at each level are fused through the fusion module in the video compression system to obtain compressed video frames corresponding to the original video frames.

2. The video compression method according to claim 1, characterized in that: The hierarchical compression of the original video frame by the network modules of multiple levels in the video compression system includes: Performing high-quality compression on the original video frame to obtain a high-quality compressed frame having a bit rate lower than that of the original video frame; Using the original video frame and the high-quality compressed frame as bidirectional reference video frames, bidirectionally deeply compressing the bidirectional reference video frame to obtain a medium-quality compressed frame having a smaller bit rate than the high-quality compressed frame; Performing single motion deep compression on the medium quality compressed frame to process motion between multiple frames using a single motion map to obtain a low quality compressed frame having a lower bit rate than the medium quality compressed frame; The step of fusing the compressed video frames output by the network modules at each level through the fusion module in the video compression system to obtain the compressed video frames corresponding to the original video frames includes: The high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are merged to obtain a compressed video frame corresponding to the original video frame.

3. The video compression method according to claim 2, characterized in that: The fusing the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame to obtain a compressed video frame corresponding to the original video frame includes: Determine weights corresponding to the high-quality compressed frame, the medium-quality compressed frame, and the low-quality compressed frame, respectively; According to the weights, weighted processing is performed on the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame to obtain a compressed video frame corresponding to the original video frame.

4. The video compression method according to claim 3, characterized in that: The determining of the weights respectively corresponding to the high-quality compressed frame, the medium-quality compressed frame, and the low-quality compressed frame comprises: Obtaining compression quality and bit rate of a compressed video frame generated corresponding to a target video frame preceding the original video frame; constructing a quality feature of the original video frame according to the compression quality and the bit rate; The weights respectively corresponding to the high-quality compressed frame, the medium-quality compressed frame and the low-quality compressed frame are determined according to the quality characteristics of the original video frame.

5. The video compression method according to claim 3, characterized in that: The fusion of the high-quality compressed frames, the medium-quality compressed frames and the low-quality compressed frames is completed through a weighted cyclic quality enhancement network module, and the weighted cyclic quality enhancement network module is constructed based on the QG-ConvLSTM network structure. The weighted processing of the high-quality compressed frames, the medium-quality compressed frames and the low-quality compressed frames is completed through a quality gating unit in the weighted cyclic quality enhancement network module, and the quality gating unit is used to utilize the correlation between multiple compressed frames to perform video frame fusion.

6. The video compression method according to claim 2, characterized in that: The bidirectional deep compression is accomplished through a bidirectional deep compression network module, which includes at least one of the following subnetworks: a motion estimation subnetwork, a motion compression subnetwork, a motion preprocessing subnetwork, and a residual compression subnetwork.

7. The video compression method according to claim 2, characterized in that: The single motion deep compression is accomplished through a single motion deep compression network module, which includes at least one of the following subnetworks: a motion estimation subnetwork, a motion compression subnetwork, a motion preprocessing subnetwork, and a residual compression subnetwork.

8. A video compression device, characterized in that: The device comprises: A hierarchical compression module, used for hierarchical compression of original video frames through network modules of multiple layers in a video compression system, wherein the network modules of the first layer are used for compressing the original video frames, and when performing compression of each layer after the first layer, the compression of the current layer is performed based on at least the compressed video frames output by the network modules of the previous layer, and the network modules of at least one layer use a single motion map to process motion between multiple frames; The video frame fusion module is used to fuse the compressed video frames output by the network modules at each level through the fusion module in the video compression system to obtain the compressed video frames corresponding to the original video frames.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method according to any one of claims 1 to 7.

Images