Video secondary compression method, device and network equipment based on residual coding

Abstract

An embodiment of the present application provides a video secondary compression method, apparatus, and network device based on residual coding, the method comprising: obtaining a first video and performing lossy compression on the first video to obtain a second video; determining residual data of the first video and the second video; and determining residual features and contextual features of the residual features based on the residual data; inputting the residual features and the contextual data of the residual features into an entropy model to determine a probability mass function, processing the residual data in an arithmetic coding manner, and determining a reconstructed residual; merging the bitstream of the second video and the bitstream of the reconstructed residual, and sending the merged bitstream; this solution can analyze the residual between the original video and the compressed video, and then encode and compress the residual, and subsequently restore the video by recovering the residual and based on the residual and the compressed video to improve the video quality.

Description

Technical Field

[0001] The present application relates to the field of computer information technology, and in particular to a video secondary compression method, apparatus and network equipment based on residual coding. Background Art

[0002] With the rapid development of computer hardware, people have higher requirements for image clarity and resolution. In the process of video transmission, video compression is usually adopted and the compressed code stream is transmitted.

[0003] Most traditional video compression methods and neural network video compression methods adopt lossy compression methods, which will cause a large loss of video quality and lead to poor video quality. Summary of the Invention

[0004] The embodiments of the present application provide a video secondary compression method, apparatus and network device based on residual coding. By analyzing the residual between the original video and the compressed video, and then selecting a corresponding coding compression method for the residual to perform coding compression, the video can be restored subsequently through the residual and the compressed video, thereby improving the video quality.

[0005] The technical solution is as follows:

[0006] In a first aspect, the present application provides a video secondary compression method based on residual coding, the method comprising: obtaining a first video and performing lossy compression on the first video to obtain a second video; determining residual data of the first video and the second video; and determining residual features and context features of the residual features based on the residual data; inputting the residual features and the context data of the residual features into an entropy model, determining a probability mass function, processing the residual data in an arithmetic coding manner, and determining a reconstructed residual; merging the bitstream of the second video and the bitstream of the reconstructed residual, and sending the merged bitstream.

[0007] Furthermore, the step of determining residual features based on the residual data includes: inputting the residual data into a feature extraction network to perform feature extraction to obtain residual features.

[0008] Furthermore, the step of determining the context data includes: inputting the residual data into a masked convolutional layer having a target receptive field, and using the target receptive field of the masked convolutional layer as the context data.

[0009] Furthermore, the inputting of the residual data into a masked convolution layer with a target receptive field includes: obtaining an error range and quantizing the residual data based on the error range; inputting the quantized residual data into a masked convolution layer with a target receptive field; the inputting of the residual features and the context data of the residual features into an entropy model to determine a probability mass function includes: inputting the residual features, the error range and the context data of the residual features into a conditional entropy model to determine a probability mass function.

[0010] Furthermore, the inputting of the residual data into a masked convolutional layer having a target receptive field includes: obtaining multiple residual levels and level error ranges corresponding to different residual levels; transforming the residual data to a target level based on the residual level and the level error range; inputting the target level into a masked convolutional layer having a target receptive field; the inputting of the residual features and the context data of the residual features into an entropy model to determine a probability mass function includes: inputting the residual features, the level error range, and the context data of the target level of the residual data into a conditional entropy model to determine a probability mass function.

[0011] Furthermore, the step of determining the reconstructed residual of the target frame includes: obtaining adjacent frames of the target frame and obtaining adjacent residual distributions corresponding to the residual data of the adjacent frames; and determining the reconstructed residual corresponding to the residual data of the target frame based on the adjacent residual distributions.

[0012] Furthermore, the method further includes: obtaining a merged code stream, and determining a second video and a reconstructed residual based on the merged code stream; and determining a reconstructed video based on the second video and the reconstructed residual.

[0013] In the second aspect, the present application provides a video secondary compression device based on residual coding, the device comprising: a second video acquisition module, used to acquire a first video and perform lossy compression on the first video to obtain a second video; a residual data acquisition module, used to determine the residual data of the first video and the second video; and determine the residual features and the context features of the residual features based on the residual data; a reconstructed residual acquisition module, used to input the residual features and the context data of the residual features into an entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual; a merged code stream generation module, used to merge the code stream of the second video and the code stream of the reconstructed residual, and send the merged code stream.

[0014] In a third aspect, the present application provides a network device comprising: a memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; and the processor is used to read the computer program in the memory and execute the method described in the first aspect.

[0015] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, which implements the method described in the first aspect when the computer program is executed by a processor.

[0016] The beneficial effects of the technical solution provided by this application are:

[0017] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. Subsequently, the residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. The receiver reconstructs the video based on the video stream after receiving the video stream containing the compressed video and the residual. The sending end can obtain the first video and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This solution analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored using the residual and the compressed video later, which can improve the video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0019] Figure 1 This is a schematic diagram of the processing steps of a video secondary compression method based on residual coding according to an embodiment of the present application;

[0020] Figure 2 This is a schematic diagram of the residual data processing flow in one embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the residual data processing flow of another embodiment of the present application;

[0022] Figure 4 This is a schematic diagram of the residual data processing flow of another embodiment of the present application;

[0023] Figure 5 This is a flowchart of a video secondary compression method based on residual coding according to an embodiment of the present application;

[0024] Figure 6 This is a structural diagram of a video secondary compression device based on residual coding according to an embodiment of the present application;

[0025] Figure 7 This is a structural block diagram of a network device according to an embodiment of the present application;

[0026] Figure 8 This is a structural block diagram of a user device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0028] Those skilled in the art will understand that, unless otherwise specified, the singular forms "a," "an," "the," and "the" used herein may also include the plural forms, and "a plurality" refers to two or more, and other quantifiers are similar. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connection or wireless coupling. The term "and / or" used herein describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0029] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. Subsequently, the residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. The receiver reconstructs the video based on the video stream after receiving the video stream containing the compressed video and the residual.

[0030] In some scenarios, traditional video compression methods are primarily based on fixed-rule coding standards, such as Advanced Video Coding (H.264), High Efficiency Video Coding (H.265), and Versatile Video Codec (VVC). Academic and industrial communities at home and abroad have conducted extensive research in this area. H.264 has become the dominant standard for HD streaming, while H.265 significantly improves compression ratios through more efficient coding techniques and is widely used for 4K video streaming. The latest VVC further optimizes compression efficiency in complex scenarios, particularly for 8K video and future video transmission requirements, providing better bandwidth utilization. The Audio and Video Standard (AVS) in China, developed by the Audio and Video Codec Standards Working Group, provides efficient compression standards for applications such as HDTV, streaming media, and video conferencing. AVS3 (Audio and Video Standard, Third Generation), its latest coding standard, similar to VVC, improves compression ratios while reducing decoding complexity. Some researchers are focusing on fine-tuning traditional codec frameworks and optimizing compression frameworks. Other approaches are narrowing the problem down to image compression and interpolation. These traditional compression methods mostly rely on feature extraction and coding strategies designed by human engineers, such as block matching motion estimation, intra-frame and inter-frame prediction, quantization, and entropy coding.

[0031] With the widespread application of deep learning, neural network video compression methods have gradually attracted the attention of researchers both domestically and internationally. These methods adaptively extract spatiotemporal features from videos through end-to-end learning, demonstrating superior representational capabilities compared to traditional methods. A series of international research projects represent cutting-edge research in neural network-based video compression. Chen et al. proposed NeRV (Neural Representation of Video), a neural representation learning method that uses video frames as input to a neural network and outputs feature representations of the video sequence. Unlike traditional video coding methods, NeRV uses implicit neural networks to learn spatiotemporal representations of videos, eliminating the need for traditional intra-frame and inter-frame prediction. This approach enables compact representation and efficient decoding in video coding, significantly improving the quality of video reconstruction. HNeRV (Hybrid Neural Representation of Video), subsequently proposed by Chen et al., further improves NeRV's performance. Its hierarchical structure and multi-resolution feature representation enable better processing of video information at different resolutions. VQ-NeRV (Vector Quantized Neural Network Representation of Video) introduces a vector quantization (VQ) mechanism, which significantly improves video compression ratios and reduces network complexity by quantizing continuous feature spaces into discrete representations. Subsequently, Li et al. proposed E-NeRV (Efficient Neural Video Representation with Disentangled Spatiotemporal Context) and Zhao et al. proposed DNeRV (Modeling Intrinsic Dynamics via Differential Neural Representation of Video), which further improved the performance of video regression. Bai et al. proposed deep lossy residual coding, which achieved advanced compression performance at competitive coding speeds.

[0032] Both traditional video compression methods and neural network video compression methods result in a loss of video quality. Neural network compression methods, in particular, suffer from poor controllability over compressed video quality. Using neural network compression, video quality can only be roughly controlled through model parameters, making it difficult to adjust video clarity. Traditional and neural network video compression methods suffer from quality loss, and high-quality video compression efficiency is low. During the neural network video compression process, it's difficult to accurately control the quality of the compressed video, often falling into only a few broad grades, without the ability to perform gradual quality control.

[0033] This proposal proposes a controllable video compression post-processing method based on residual coding. Taking into account the quality losses associated with different video compression methods, this approach combines efficient residual coding with residual distribution learning to address issues such as post-compression video quality loss affecting the visual experience, uncontrollable post-compression video quality, and low post-compression video quality. By introducing efficient residual coding into the video compression task, this approach further reduces video compression quality losses and achieves control over video quality.

[0034] This solution introduces residual coding into video compression tasks. Regardless of whether traditional or neural network video compression methods are used, quality loss occurs. The residual between the compressed video and the original video can be encoded. The quality of the video compression loss is then controlled by restoring the encoded residual, achieving lossless compression and restoring the video to the same quality as the original. This invention, as a controllable video compression post-processing method based on residual coding, is applicable to different video compression methods and has good generalization properties.

[0035] This proposal investigates a controllable residual coding method for video secondary compression based on datasets of varying sizes and types. This method can be combined with various video coding models to encode the residual of the compressed video, achieving quality control and lossless compression. Autoregressive context modeling of the residual is used to improve lossless compression performance. In a near-lossless mode, the original residual is quantized to meet a given error bound. A scalable near-lossless compression scheme suitable for variable error bounds is proposed. Without training multiple networks, a probabilistic model for quantized residuals is derived from a lossless compression probability model of the original residuals. The context mismatch introduced by residual quantization is then corrected through probabilistic model bias correction, resulting in a lossless and near-lossless controllable residual coding framework. The main contents include: quantizing the video residual to train a probabilistic model for lossless compression; achieving near-lossless residual coding within a given error bound; and introducing a residual transform to further improve near-lossless video compression performance through adaptive residual bound spacing.

[0036] Lossy video compression results in a certain loss in rendering quality. To improve rendering quality and manage compression losses, we are considering learning the residual of the compressed video. This involves encoding the residual of the compressed video through residual coding, combining the compressed video bitstream with the bitstream of the encoded residual into a new bitstream. This effectively controls video compression losses and recovers any quality loss caused by compression. This solution aims to improve the accuracy of residual recovery and reduce the size of the residual encoding bitstream during the encoding and decoding process.

[0037] like Figure 1 As shown, this solution first uses a video compression method to compress the video, and obtains the compressed video stream and the compressed reconstructed video. Subtract the original video x, the residual Thus, the residuals r1, r2, r3, ..., rn of each frame of the compressed video are obtained. A probability model of lossless compression is introduced to estimate the probability mass function (PMF) of the residual r, and the residual r is compressed using arithmetic coding.

[0038] like Figure 2As shown in the figure, this scheme can perform lossless compression on the residual and introduce features to learn the distribution of the residual r. u is the feature of the residual r given the original video x. Feature u has the same height and width as r and has 256 channels. The residual r in the pixel domain has spatial redundancy and cannot be fully utilized by feature u alone. Therefore, it is considered to further introduce the autoregressive model into the statistical modeling of r, use a masked convolution layer with a specific receptive field to implement the spatial autoregressive model, and use the receptive field of the masked convolution layer as the context Cr. Therefore, the probability mass function (PMF) of the residual r is modeled using discrete logistic mixture likelihood. In the constructed entropy model, the input is feature u and context Cr, and the output is the likelihood parameter corresponding to the estimated probability distribution of r.

[0039] After lossless compression is achieved, Figure 3 As shown, by introducing quantized residual coding to achieve nearly lossless video compression, the original residual is r and the reconstructed residual is The error range of the loss is τ. Setting different error values ​​for τ allows for controllable quality loss. To achieve controllable near-lossless video compression, the probability mass function (PMF) of the residual r is obtained using the feature u and the original residual r. The original residual is quantized with a given τ to obtain the quantized residual r. This yields the probability mass function (PMF) of the quantized residual r. By setting different τ values, video quality can be controlled.

[0040] The residual error range of the approximate lossless residual coding is determined by a τ value. In order to achieve an adaptive error range interval, the original residual value is transformed, and the residual of the rendered image is transformed according to a certain loss range. Then, the distribution of the transformed residual is learned. Figure 4 As shown in the figure, the original residual r is divided into n levels along different error ranges, with each level corresponding to a level value ri, that is, the true residual is transformed into different level values ​​in the residual distribution R. The residual distribution R is obtained and the cross entropy model is used to learn the residual distribution R. The residual value is restored according to the level value through the learned residual distribution. The error range of the restored residual is related to the range of τ. The residual transformation is based on the lossy residual distribution learning of τ. The restored residual value is combined with the compressed video to form a new reconstructed video. When learning the residual distribution, the larger the original residual value, the larger the bitrate required for learning the distribution. Since adjacent frame images are similar, using the learned residual distribution of adjacent frames to assist in learning the residual distribution of the current frame can reduce the bitrate required for residual distribution learning.

[0041] The main steps of this solution are as follows: compress the video Subtract the original video x, the residual The residuals r1, r2, r3, ..., rn of each frame of the compressed video are obtained. The probability model of lossless compression is introduced to estimate the probability mass function (PMF) of the residual r, and the residual r is compressed using arithmetic coding. The feature u is introduced to realize the distribution learning of the residual r. The probability mass function (PMF) of the residual r can be obtained by the feature u and the original residual r. The original residual and the given τ are quantized to obtain the quantized residual Then we can get the quantized residual The probability mass function (PMF) is used to control video quality by setting different τ values. The original residual value is transformed, and the residual value of the rendered image is transformed according to a certain loss range. The transformed residual distribution is then learned. The residual distribution learned in adjacent frames is used to assist in learning the residual distribution of the current frame, reducing the bitrate required for residual distribution learning.

[0042] The probability mass function (PMF) estimation step of the original residual of the video includes: compressing the video Subtract the original video x from the original video to obtain the residual r1, r2, r3, ..., rn of each frame of the compressed video; each frame of the compressed video passes through the feature extraction network to extract the residual feature ui of each frame; a masked convolutional layer with a specific receptive field is used to implement a spatial autoregressive model for the residual, and the receptive field of the masked convolutional layer is used as the context Cr; the feature u and the context Cr are input into the entropy model for training. When the model converges, the probability mass function (PMF) of the residual r is estimated, and the likelihood parameter corresponding to the probability distribution is obtained.

[0043] The original residual and the given τ are quantized to obtain the quantized residual, and the steps of obtaining the probability mass function (PMF) of the quantized residual include: the original residual and the given τ are quantized to obtain the quantized residual Quantized residual Use masked convolutional layers with specific receptive fields to obtain context for the quantized residuals The context of feature u, set error range τ, and quantized residual The conditional entropy model is input to estimate the probability mass function (PMF) of the quantized residual.

[0044] The original residual value is transformed, different error ranges are set respectively, and the steps of reducing the bitstream required for learning by combining the residual distribution of adjacent frames include: the original residual is transformed into the corresponding residual level value R according to a certain residual value range; the residual level value R is passed through the mask convolution layer to obtain the residual level context C R ; Set the feature u, set the error range τ, and the context C of the residual level R The conditional entropy model is input to estimate the probability mass function (PMF) of the quantized residual.

[0045] Use the residual distribution learned by adjacent frames to assist the residual distribution learning of the current frame, including: the residual of the video frame t-1 and the frame t+1, estimate its probability mass function (PMF), and then reconstruct the residual of the frame t-1 and the frame t+1; combine the image of the frame t-1 with the residual to obtain The t+1 frame image is combined with the residual to obtain Video frame t Replace with and The combination of makes the residual smaller, that is, The context of feature u, set error range τ, and quantized residual The conditional entropy model is input to estimate the probability mass function (PMF) of the quantized residual.

[0046] On the basis of the above embodiments, the present application also provides a video secondary compression method based on residual coding, such as Figure 5 As shown, the method includes:

[0047] Step 102: Obtain a first video, and perform lossy compression on the first video to obtain a second video.

[0048] Step 104: Determine residual data of the first video and the second video; and determine residual features and contextual features of the residual features based on the residual data.

[0049] Step 106: Input the residual features and the context data of the residual features into the entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual.

[0050] Step 108: Merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream.

[0051] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. Subsequently, the residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. The receiver reconstructs the video based on the video stream after receiving the video stream containing the compressed video and the residual. The sending end can obtain the first video and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This solution analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored using the residual and the compressed video later, which can improve the video quality.

[0052] This solution can use a feature extraction network to convert residual data into residual features and then analyze them. Specifically, as an optional embodiment, the step of determining residual features based on residual data includes: inputting the residual data into a feature extraction network for feature extraction to obtain residual features.

[0053] In addition to considering the residual features of the target frame, this solution also considers the contextual data of the residual features of the target frame, such as the residual features of one or more frames before and / or after the target frame, for fusion and comprehensive analysis to better perform residual encoding. This solution can use a convolutional layer with a pre-set receptive field to determine the context. Specifically, as an optional embodiment, the step of determining the contextual data includes: inputting the residual data into a masked convolutional layer with a target receptive field, and using the target receptive field of the masked convolutional layer as the contextual data.

[0054] The above encoding process is a lossless encoding process. This solution can also set a smaller error range to achieve near-lossless encoding. Specifically, as an optional embodiment, the inputting of residual data into a masked convolution layer with a target receptive field includes: obtaining an error range and quantizing the residual data based on the error range; inputting the quantized residual data into a masked convolution layer with a target receptive field; the inputting of residual features and the context data of the residual features into an entropy model to determine the probability mass function includes: inputting the residual features, the error range, and the context data of the residual features into a conditional entropy model to determine the probability mass function. This solution can also set multiple levels, each corresponding to its own level error, so that the residual can be transformed into a fixed level to determine the corresponding PMF for encoding. Specifically, as an optional embodiment, the inputting of residual data into a masked convolutional layer with a target receptive field includes: obtaining multiple residual levels and level error ranges corresponding to different residual levels; transforming the residual data to a target level based on the residual level and the level error range; inputting the target level into a masked convolutional layer with a target receptive field; the inputting of residual features and context data of residual features into an entropy model to determine a probability mass function includes: inputting residual features, level error ranges, and context data of the target level of residual data into a conditional entropy model to determine a probability mass function.

[0055] During the encoding of the residual data of a target frame, this solution can utilize the determined residual distribution information of adjacent frames to determine the residual distribution of the target frame, thereby reducing the amount of analysis. Specifically, as an optional embodiment, the step of determining the reconstructed residual of the target frame includes: obtaining adjacent frames of the target frame and obtaining adjacent residual distributions corresponding to the residual data of the adjacent frames; and determining the reconstructed residual corresponding to the residual data of the target frame based on the adjacent residual distributions.

[0056] After determining the compressed video and the encoded residual, the data can be transmitted to the receiving end, which then performs data restoration to determine the reconstructed video. Specifically, as an optional embodiment, the method further includes: obtaining a merged code stream, and determining a second video and a reconstructed residual based on the merged code stream; and determining the reconstructed video based on the second video and the reconstructed residual.

[0057] On the basis of the above embodiments, the embodiment of the present application further provides a video secondary compression device based on residual coding, such as Figure 6 As shown, the device includes:

[0058] The second video acquisition module 202 is configured to acquire the first video and perform lossy compression on the first video to obtain the second video.

[0059] The residual data acquisition module 204 is configured to determine residual data of the first video and the second video; and determine residual features and context features of the residual features based on the residual data.

[0060] The reconstructed residual acquisition module 206 is used to input the residual features and the context data of the residual features into the entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual.

[0061] The merged code stream generating module 208 is configured to merge the code stream of the second video and the code stream of the reconstructed residual, and send the merged code stream.

[0062] The implementation of the embodiment of the present application is similar to that of the above embodiment. The specific implementation can refer to the specific implementation of the above embodiment and will not be repeated here.

[0063] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. Subsequently, the residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. The receiver reconstructs the video based on the video stream after receiving the video stream containing the compressed video and the residual. The sending end can obtain the first video and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This solution analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored using the residual and the compressed video later, which can improve the video quality.

[0064] It should be noted that the division of units and / or modules in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, there may be other division methods. In addition, the functional units and / or modules in the various embodiments of the present application may be integrated into one processing unit and / or module, or each unit and / or module may exist physically alone, or two or more units and / or modules may be integrated into one unit and / or module. The above-mentioned integrated units and / or modules may be implemented in the form of hardware or in the form of software functional units and / or modules.

[0065] If the integrated units and / or modules are implemented in the form of software functional units and / or modules and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0066] In addition, the data transmission device and data transmission method provided in the above embodiments are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0067] Figure 7 A structural block diagram of a network device is shown according to an exemplary embodiment.

[0068] like Figure 7 As shown, the network device 1100 includes at least: a processor 1110 , a memory 1120 and a transceiver 1130 .

[0069] The transceiver 1130 is used to receive and send data under the control of the processor 1110 .

[0070] exist Figure 7In the embodiment of the present invention, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1110 and memory represented by memory 1120. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described herein. The bus interface provides an interface. The transceiver 1130 may be a plurality of components, i.e., a transmitter and a receiver, providing units and / or modules for communicating with various other devices over a transmission medium, such as a wireless channel, a wired channel, an optical cable, or the like.

[0071] The processor 1110 is responsible for managing the bus architecture and general processing, and the memory 1120 can store data used by the processor 1110 when performing operations.

[0072] Optionally, the processor 1110 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor 1110 may also employ a multi-core architecture. The processor 1110 and the memory 1120 may also be physically separated.

[0073] The processor 1110 calls the computer program stored in the memory 1120 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0074] Figure 8 A structural block diagram of a user equipment is shown according to an exemplary embodiment.

[0075] like Figure 8 As shown, the user equipment 1300 includes at least: a processor 1310 , a memory 1320 and a transceiver 1330 .

[0076] The transceiver 1330 is used to receive and send data under the control of the processor 1310.

[0077] exist Figure 8In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1310 and memory represented by memory 1320. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 1330 may be a plurality of components, i.e., a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, such as wireless channels, wired channels, optical cables, and other transmission media. For different user devices, the user interface 1340 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0078] The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 can store data used by the processor 1310 when performing operations.

[0079] Optionally, the processor 1310 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a CPLD (Complex Programmable Logic Device). The processor 1310 may also employ a multi-core architecture. The processor 1310 and the memory 1320 may also be physically separated.

[0080] The processor 1310 calls the computer program stored in the memory 1320 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0081] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0082] In addition, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the data transmission method of each of the above embodiments. The storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as a floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0083] In an embodiment of the present application, a program product is provided. For example, the program product is an FPGA chip or a DSP chip. The program product includes executable instructions stored in a storage medium. A processor reads the executable instructions from the storage medium, so that when the executable instructions are executed by the processor, the data transmission method described in each of the above embodiments is implemented.

[0084] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. Subsequently, the residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. The receiver reconstructs the video based on the video stream after receiving the video stream containing the compressed video and the residual. The sending end can obtain the first video and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This solution analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored using the residual and the compressed video later, which can improve the video quality.

[0085] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0086] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0089] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0090] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be included in the scope of protection of the present application.

Claims

1. A video secondary compression method based on residual coding, characterized in that: The method comprises: Obtain a first video, and perform lossy compression on the first video to obtain a second video; Determine residual data of the first video and the second video; and determine residual features and context features of the residual features based on the residual data; The residual features and the context data of the residual features are input into the entropy model to determine the probability mass function, and the residual data is processed in an arithmetic coding manner to determine the reconstructed residual; The bitstream of the second video and the bitstream of the reconstructed residual are merged, and the merged bitstream is sent.

2. The method according to claim 1, wherein The steps of determining residual features based on residual data include: The residual data is input into the feature extraction network for feature extraction to obtain residual features.

3. The method according to claim 2, wherein Steps to determine contextual data include: The residual data is input into the masked convolution layer with the target receptive field, and the target receptive field of the masked convolution layer is used as the context data.

4. The method according to claim 3, wherein The residual data is input into the mask convolution layer with the target receptive field, including: Obtain the error range and quantify the residual data based on the error range; The quantized residual data is input into the masked convolution layer with the target receptive field; The residual features and the context data of the residual features are input into the entropy model to determine the probability mass function, including: The residual features, error ranges, and contextual data of the residual features are input into the conditional entropy model to determine the probability mass function.

5. The method according to claim 3, wherein The residual data is input into the mask convolution layer with the target receptive field, including: Get multiple residual levels and the level error ranges corresponding to different residual levels; Transform the residual data to the target level based on the residual level and level error range; The object level is input to a masked convolutional layer with the target receptive field; The residual features and the context data of the residual features are input into the entropy model to determine the probability mass function, including: The residual features, the level error range, and the contextual data of the target level of the residual data are input into the conditional entropy model to determine the probability mass function.

6. The method according to claim 1, wherein The step of determining the reconstructed residual of the target frame comprises: Obtain adjacent frames of the target frame, and obtain adjacent residual distributions corresponding to the residual data of the adjacent frames; Based on the distribution of adjacent residuals, the reconstructed residual corresponding to the residual data of the target frame is determined.

7. The method according to claim 1, wherein The method further comprises: Obtaining a combined code stream, and determining a second video and a reconstructed residual based on the combined code stream; A reconstructed video is determined based on the second video and the reconstructed residual.

8. A video secondary compression device based on residual coding, characterized in that: The device comprises: A second video acquisition module is used to acquire the first video and perform lossy compression on the first video to obtain a second video; A residual data acquisition module, configured to determine residual data of the first video and the second video; and determine residual features and contextual features of the residual features based on the residual data; The reconstructed residual acquisition module is used to input the residual features and the context data of the residual features into the entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual; The combined code stream generation module is used to combine the code stream of the second video and the code stream of the reconstructed residual, and send the combined code stream.

9. A network device, characterized in that: include: A memory, a transceiver, and a processor; wherein the memory is used to store computer programs; the transceiver is used to send and receive data under the control of the processor; The processor is configured to read the computer program in the memory and execute the method according to claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claims 1 to 7 is implemented.

Images