Video secondary compression method and device based on residual coding and network equipment

Abstract

The embodiment of the invention provides a video secondary compression method and device based on residual coding and network equipment, and the method comprises the steps: obtaining a first video, and carrying out the lossy compression of the first video, and obtaining a second video; determining residual data of the first video and the second video; determining a residual feature and a context feature of the residual feature based on the residual data; inputting the residual error characteristics and context data of the residual error characteristics into an entropy model, determining a probability quality function, processing the residual error data by adopting an arithmetic coding mode, and determining a reconstruction residual error; combining the code stream of the second video and the code stream of the reconstructed residual error, and sending the combined code stream; according to the scheme, the residual error of the original video and the compressed video can be analyzed, then the residual error is coded and compressed, and the video is restored subsequently by restoring the residual error and according to the residual error and the compressed video, so that the video quality is improved.

Description

Technical Field

[0001] The present application relates to the field of computer information technology, and in particular to a video secondary compression method, device and network equipment based on residual coding. Background Art

[0002] With the rapid development of computer hardware, people have higher requirements for image clarity and resolution. In the process of video transmission, video compression is usually adopted, and the compressed code stream is transmitted.

[0003] Most traditional video compression methods and neural network video compression methods adopt lossy compression methods, which will cause a large loss of video quality and lead to poor video quality. Summary of the invention

[0004] The embodiments of the present application provide a video secondary compression method, device and network device based on residual coding. By analyzing the residual between the original video and the compressed video, and then selecting a corresponding coding compression method for the residual to perform coding compression, the video can be restored through the residual and the compressed video subsequently, thereby improving the video quality.

[0005] The technical solution is as follows:

[0006] In a first aspect, the present application provides a video secondary compression method based on residual coding, the method comprising: obtaining a first video, and performing lossy compression on the first video to obtain a second video; determining residual data of the first video and the second video; and determining residual features and context features of the residual features based on the residual data; inputting the residual features and the context data of the residual features into an entropy model, determining a probability mass function, processing the residual data in an arithmetic coding manner, and determining a reconstructed residual; merging the bitstream of the second video and the bitstream of the reconstructed residual, and sending the merged bitstream.

[0007] Furthermore, the step of determining residual features based on the residual data includes: inputting the residual data into a feature extraction network to perform feature extraction to obtain residual features.

[0008] Furthermore, the step of determining context data includes: inputting the residual data into a masked convolutional layer having a target receptive field, and using the target receptive field of the masked convolutional layer as the context data.

[0009] Furthermore, the step of inputting the residual data into a masked convolutional layer having a target receptive field includes: obtaining an error range and quantizing the residual data based on the error range; inputting the quantized residual data into a masked convolutional layer having a target receptive field; the step of inputting the residual features and the context data of the residual features into an entropy model to determine a probability mass function includes: inputting the residual features, the error range, and the context data of the residual features into a conditional entropy model to determine a probability mass function.

[0010] Furthermore, the step of inputting the residual data into a masked convolutional layer having a target receptive field includes: obtaining multiple residual levels and level error ranges corresponding to different residual levels; transforming the residual data to a target level based on the residual level and the level error range; inputting the target level into a masked convolutional layer having a target receptive field; the step of inputting the residual features and the context data of the residual features into an entropy model to determine a probability mass function includes: inputting the residual features, the level error range, and the context data of the target level of the residual data into a conditional entropy model to determine a probability mass function.

[0011] Furthermore, the step of determining the reconstructed residual of the target frame includes: obtaining adjacent frames of the target frame, and obtaining adjacent residual distributions corresponding to residual data of the adjacent frames; and determining the reconstructed residual corresponding to the residual data of the target frame based on the adjacent residual distributions.

[0012] Furthermore, the method also includes: obtaining a merged code stream, and determining a second video and a reconstructed residual based on the merged code stream; and determining a reconstructed video based on the second video and the reconstructed residual.

[0013] In the second aspect, the present application provides a video secondary compression device based on residual coding, the device comprising: a second video acquisition module, used to acquire a first video, and lossily compress the first video to obtain a second video; a residual data acquisition module, used to determine the residual data of the first video and the second video; and determine the residual features and the context features of the residual features based on the residual data; a reconstructed residual acquisition module, used to input the residual features and the context data of the residual features into an entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual; a merged code stream generation module, used to merge the code stream of the second video and the code stream of the reconstructed residual, and send the merged code stream.

[0014] In a third aspect, the present application provides a network device comprising: a memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; and the processor is used to read the computer program in the memory and execute the method described in the first aspect.

[0015] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0016] The beneficial effects of the technical solution provided by this application are:

[0017] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. The residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. After receiving the video stream containing the compressed video and the residual, the receiver reconstructs the video based on the video stream. The sending end can obtain the first video, and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This scheme analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored by the residual and the compressed video later, which can improve the video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0019] Figure 1 It is a schematic diagram of the processing steps of a video secondary compression method based on residual coding according to an embodiment of the present application;

[0020] Figure 2 This is a schematic diagram of the residual data processing flow of an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a processing flow of residual data according to another embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a processing flow of residual data in yet another embodiment of the present application;

[0023] Figure 5 It is a flowchart of a video secondary compression method based on residual coding according to an embodiment of the present application;

[0024] Figure 6 It is a structural schematic diagram of a video secondary compression device based on residual coding according to an embodiment of the present application;

[0025] Figure 7 It is a structural block diagram of a network device according to an embodiment of the present application;

[0026] Figure 8 This is a structural block diagram of a user device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0028] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms, and "multiple" refers to two or more, and other quantifiers are similar. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or wireless coupling. The term "and / or" used here describes the association relationship of associated objects, indicating that there may be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0029] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. The residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. After receiving the video stream containing the compressed video and the residual, the receiver reconstructs the video based on the video stream.

[0030] In some schemes, traditional video compression methods are mainly based on fixed-rule coding standards, such as Advanced Video Coding (H.264), High Efficiency Video Coding (H.265), and Versatile Codec (VVC). Domestic and foreign academic and industrial circles have conducted a lot of research in this field. H.264 has become the main standard for high-definition streaming media, while H.265 has significantly improved the compression ratio through more efficient coding technology and is widely used in the transmission of 4K video streams. The latest VVC further optimizes the compression efficiency in complex scenes, especially for 8K video and future video transmission needs, providing better bandwidth utilization. The domestic audio and video standard (AVS) is formulated by the Audio and Video Codec Standard Working Group (AVS Working Group), which provides efficient compression standards for application scenarios such as HDTV, streaming media, and video conferencing. AVS3 (Audio and Video Standard Third Generation) as its latest coding standard, similar to VVC, reduces decoding complexity while improving compression ratio. Some people focus on fine-tuning the traditional codec framework and optimizing the compression framework. There are also some methods that narrow the problem to image compression and interpolation. These traditional compression methods mostly rely on feature extraction and coding strategies designed by human engineers, such as block matching motion estimation, intra-frame and inter-frame prediction, quantization, and entropy coding.

[0031] With the widespread application of deep learning, neural network video compression methods have gradually attracted the attention of researchers at home and abroad. These methods adaptively extract spatiotemporal features in videos through end-to-end learning, and have stronger expression capabilities than traditional methods. A series of research works abroad represent the cutting-edge direction of video compression research based on neural networks. Chen et al. proposed NeRV (neural representation of video) as a neural representation learning method, which takes video frames as input of neural networks and outputs feature representations of video sequences. Unlike traditional video coding methods, NeRV uses implicit neural networks to learn spatiotemporal representations of videos, without the need for traditional intra-frame and inter-frame predictions. It can achieve compact representation and efficient decoding in video coding, significantly improving the quality of video reconstruction. HNeRV (hybrid neural representation of video) proposed by Chen et al. further improved the performance of NeRV. Through hierarchical structure and multi-resolution feature representation, it can better process video information at different resolutions. VQ-NeRV (vector quantized neural network representation of video) introduced the mechanism of vector quantization (VQ). This technology significantly improves the video compression ratio and reduces the complexity of the network by quantizing the continuous feature space into discrete representation. Subsequently, Li et al. proposed E-NeRV (Easy Neural Video Representation with Disentangled Spatiotemporal Context) and Zhao et al. proposed DNeRV (Modeling Intrinsic Dynamics through Differential Neural Representation of Video), which further improved the performance of video regression. Bai et al. proposed Deep Lossy Plus Residual Coding, which achieved advanced compression performance at competitive coding speeds.

[0032] Both traditional video compression methods and neural network video compression methods will lead to loss of video quality, especially the neural network compression method has poor controllability over the quality of compressed videos. When compressing videos through neural networks, the video quality can only be roughly controlled by model parameters, which is not convenient for adjusting the clarity of the video. Traditional video compression methods and neural network video compression methods have quality loss, and the compression efficiency of high-definition and high-quality videos is not high. In the compression process of the neural network video compression method, it is difficult to accurately control the quality of the compressed video. Often the quality of the compressed video can only be divided into several broad levels, and the quality cannot be controlled step by step.

[0033] This scheme proposes a controllable video compression post-processing method based on residual coding. Considering the quality loss caused by different video compression methods, combining efficient residual coding with residual distribution learning, to solve the problems of compressed video quality loss affecting visual experience, uncontrollable compressed video quality, and low compressed video quality. By introducing efficient residual coding into the video compression task, the quality loss of video compression can be further reduced, and the video quality can be controlled.

[0034] This solution introduces residual coding into the video compression task. No matter whether it is a traditional video compression method or a neural network video compression method, quality loss will occur after use. The residual between the compressed video and the original video can be encoded. The quality of the video compression loss is subsequently controlled by restoring the encoded residual, and lossless compression can be achieved at best, restoring it to the same as the original video. As a controllable video compression post-processing method based on residual coding, the present invention can be used for different video compression methods and has good generalization.

[0035] This scheme selects data sets of different sizes and types to study a controllable residual coding method for video secondary compression. It can be used in combination with different video coding models to encode the residual of the compressed video to control the quality of the compressed video and achieve lossless compression. The autoregressive context modeling of the residual is used to improve the lossless compression performance. In the near-lossless mode, the original residual is quantized to meet the given error range, and a scalable near-lossless compression scheme is proposed. The scheme is suitable for variable error ranges. Without training multiple networks, the probability model of the quantized residual is derived by quantizing the lossless compression probability model of the original residual, and the context mismatch caused by the residual quantization is optimized by the probability model deviation correction, forming a lossless and nearly lossless controllable residual coding framework. The main contents include: quantizing the video residual and training the probability model for lossless compression; achieving nearly lossless residual coding through a given error range; introducing residual transform, and further improving the nearly lossless video compression performance through adaptive residual range interval.

[0036] Lossy video compression will cause a certain loss in rendering quality. In order to improve the rendering quality and achieve controllability of compression loss, it is considered to learn the residual of the compressed video. That is, the residual of the compressed video is encoded by residual coding, and the bitstream of the compressed video and the bitstream of the encoded residual are merged into a new bitstream, thereby achieving control of the video compression loss and recovery of the quality loss caused by compression. This solution needs to improve the accuracy of residual recovery and reduce the bitstream size of residual coding during the encoding and decoding process.

[0037] like Figure 1 As shown, this scheme first uses a video compression method to compress the video to obtain a compressed video code stream and a compressed reconstructed video. Subtract the original video x, the residual Thus, the residuals r1, r2, r3, ..., rn of each frame of the compressed video are obtained, and a probability model of lossless compression is introduced to estimate the probability mass function (PMF) of the residual r, and the residual r is compressed using arithmetic coding.

[0038] like Figure 2As shown in the figure, this scheme can perform lossless compression on the residual and introduce features to realize the distribution learning of the residual r. u is the feature of the residual r of the given original video x. The feature u has the same height and width as r and has 256 channels. The residual r in the pixel domain has spatial redundancy, which cannot be fully utilized by feature u alone. Therefore, it is considered to further introduce the autoregressive model into the statistical modeling of r, use the masked convolution layer with a specific receptive field to realize the spatial autoregressive model, and use the receptive field of the masked convolution layer as the context Cr. Therefore, the probability mass function (PMF) of the residual r is modeled using discrete logical mixed likelihood. The input of the constructed entropy model is feature u and context Cr, and the output is the likelihood parameter corresponding to the estimated probability distribution of r.

[0039] After lossless compression is achieved, Figure 3 As shown in the figure, approximately lossless video compression is achieved by introducing quantized residual coding. The original residual is r and the reconstructed residual is The error range of the loss is τ. Different error τ values ​​are set to control the quality loss. In order to achieve controllable lossless video compression from lossless to approximate lossless, the probability mass function (PMF) of the residual r can be obtained through the feature u and the original residual r. The original residual and the given τ are quantized to obtain the quantized residual r, and then the probability mass function (PMF) of the quantized residual r can be obtained. By setting different τ values, the video quality can be controlled.

[0040] The residual error range of the approximate lossless residual coding is determined by a τ value. In order to achieve an adaptive error range interval, the original residual value is transformed, the residual of the rendered image is transformed according to a certain loss range, and then the distribution of the transformed residual is learned. Figure 4 As shown in the figure, the original residual r is divided into n levels along different error ranges, and each level corresponds to a level value ri, that is, the real residual is transformed into different level values ​​in the residual distribution R. The residual distribution R is obtained and the cross entropy model is used to learn the residual distribution R. The residual value is restored according to the level value through the learned residual distribution. The error range of the restored residual is related to the range of τ. The residual transformation is based on the lossy residual distribution learning of τ. The restored residual value is combined with the compressed video to form a new reconstructed video. When learning the residual distribution, the larger the original residual value, the larger the bitrate required for learning the distribution. There is similarity between adjacent frame images. Using the residual distribution learned by the adjacent frame to assist the residual distribution learning of the current frame can reduce the bitrate required for residual distribution learning.

[0041] The main steps of this solution are as follows: compress the video Subtract the original video x, the residual Thus, the residual r1, r2, r3, ..., rn of each frame of the compressed video is obtained. The probability model of lossless compression is introduced to estimate the probability mass function (PMF) of the residual r, and the residual r is compressed using arithmetic coding. The feature u is introduced to realize the distribution learning of the residual r. The probability mass function (PMF) of the residual r can be obtained through the feature u and the original residual r. The original residual and the given τ are quantized to obtain the quantized residual Then we can get the quantized residual The probability mass function (PMF) of the image is used to control the video quality by setting different τ values. The original residual value is transformed, and the residual value of the rendered image is transformed according to a certain loss range, and then the transformed residual distribution is learned. The residual distribution learned in the adjacent frame is used to assist the residual distribution learning of the current frame to reduce the bit rate required for residual distribution learning.

[0042] The probability mass function (PMF) estimation step of the original residual of the video includes: compressing the video Subtract the original video x to obtain the residual r1, r2, r3, …, rn of each frame of the compressed video; each frame of the compressed video passes through the feature extraction network to extract the residual feature ui of each frame; a masked convolutional layer with a specific receptive field is used to implement the spatial autoregressive model for the residual, and the receptive field of the masked convolutional layer is used as the context Cr; the feature u and the context Cr are input into the entropy model for training, and the training is completed until the model converges, then the probability mass function (PMF) of the residual r is estimated, and the likelihood parameter corresponding to the probability distribution is obtained.

[0043] The original residual and the given τ are quantized to obtain a quantized residual, and then the probability mass function (PMF) of the quantized residual can be obtained. The steps include: the original residual and the given τ are quantized to obtain a quantized residual Quantized residual Use a masked convolutional layer with a specific receptive field to obtain the context of the quantized residual The feature u, the error range τ, and the context of the quantized residual The conditional entropy model is input to estimate the probability mass function (PMF) of the quantized residual.

[0044] The steps of transforming the original residual value, setting different error ranges respectively, and combining the residual distribution of adjacent frames to reduce the bitstream required for learning include: transforming the original residual into a corresponding residual level value R according to a certain residual value range; the residual level value R is passed through a mask convolution layer to obtain a residual level context C R ; Set the feature u, set the error range τ, and the context C of the residual level R The conditional entropy model is input to estimate the probability mass function (PMF) of the quantized residual.

[0045] Use the residual distribution learned in adjacent frames to assist the residual distribution learning of the current frame, including: estimating the probability mass function (PMF) of the residuals of the video's t-1th frame and t+1th frame, and then reconstructing the residuals of the t-1th frame and t+1th frame; combining the t-1th frame image with the residual to obtain The t+1th frame image is combined with the residual to obtain Video frame t Replace with and The combination of makes the residual smaller, that is, The feature u, the error range τ, and the context of the quantized residual The conditional entropy model is input to estimate the probability mass function (PMF) of the quantized residual.

[0046] Based on the above embodiments, the present application also provides a video secondary compression method based on residual coding, such as Figure 5 As shown, the method includes:

[0047] Step 102: Obtain a first video, and perform lossy compression on the first video to obtain a second video.

[0048] Step 104: determine residual data of the first video and the second video; and determine residual features and context features of the residual features based on the residual data.

[0049] Step 106: Input the residual features and the context data of the residual features into the entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual.

[0050] Step 108: merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream.

[0051] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. The residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. After receiving the video stream containing the compressed video and the residual, the receiver reconstructs the video based on the video stream. The sending end can obtain the first video, and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This scheme analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored by the residual and the compressed video later, which can improve the video quality.

[0052] This solution can use a feature extraction network to convert residual data into residual features and then analyze them. Specifically, as an optional embodiment, the step of determining residual features based on residual data includes: inputting the residual data into a feature extraction network to perform feature extraction to obtain residual features.

[0053] In addition to considering the residual features of the target frame, this solution also considers the context data of the residual features of the target frame, such as fusing and comprehensively analyzing the residual features of one or more frames before and / or after the target frame, so as to better perform residual encoding. This solution can use a convolutional layer with a pre-set receptive field to determine the context. Specifically, as an optional embodiment, the step of determining the context data includes: inputting the residual data into a masked convolutional layer with a target receptive field, and using the target receptive field of the masked convolutional layer as the context data.

[0054] The above encoding process is a lossless encoding process. This solution can also set a smaller error range to achieve approximately lossless encoding. Specifically, as an optional embodiment, the residual data is input into a mask convolution layer with a target receptive field, including: obtaining an error range, and quantizing the residual data based on the error range; the quantized residual data is input into a mask convolution layer with a target receptive field; the residual features and the context data of the residual features are input into an entropy model to determine the probability mass function, including: inputting the residual features, the error range and the context data of the residual features into a conditional entropy model to determine the probability mass function. This solution can also set multiple levels, each level corresponding to its own level error, so that the residual can be transformed into a fixed level to determine the corresponding PMF for encoding. Specifically, as an optional embodiment, the inputting of residual data into a masked convolutional layer with a target receptive field includes: obtaining multiple residual levels and level error ranges corresponding to different residual levels; transforming the residual data to a target level based on the residual level and the level error range; inputting the target level into a masked convolutional layer with a target receptive field; the inputting of residual features and context data of residual features into an entropy model to determine a probability mass function includes: inputting residual features, level error ranges, and context data of the target level of residual data into a conditional entropy model to determine a probability mass function.

[0055] In the process of encoding the residual data of the target frame, the present solution can use the determined residual distribution information of the adjacent frames to determine the residual distribution of the target frame to reduce the amount of analysis. Specifically, as an optional embodiment, the step of determining the reconstructed residual of the target frame includes: obtaining adjacent frames of the target frame, and obtaining adjacent residual distributions corresponding to the residual data of the adjacent frames; based on the adjacent residual distributions, determining the reconstructed residual corresponding to the residual data of the target frame.

[0056] After determining the compressed video and the encoded residual, the data can be transmitted to the receiving end, and the receiving end performs data restoration after receiving it to determine the reconstructed video. Specifically, as an optional embodiment, the method also includes: obtaining a merged code stream, and determining a second video and a reconstructed residual based on the merged code stream; determining the reconstructed video based on the second video and the reconstructed residual.

[0057] Based on the above embodiments, the present application also provides a video secondary compression device based on residual coding, such as Figure 6 As shown, the device comprises:

[0058] The second video acquisition module 202 is used to acquire the first video and perform lossy compression on the first video to obtain the second video.

[0059] The residual data acquisition module 204 is used to determine the residual data of the first video and the second video; and determine the residual features and the context features of the residual features based on the residual data.

[0060] The reconstructed residual acquisition module 206 is used to input the residual features and the context data of the residual features into the entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual.

[0061] The merged bitstream generating module 208 is configured to merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream.

[0062] The implementation of the embodiment of the present application is similar to the implementation of the above embodiment. The specific implementation can refer to the specific implementation of the above embodiment, which will not be repeated here.

[0063] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. The residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. After receiving the video stream containing the compressed video and the residual, the receiver reconstructs the video based on the video stream. The sending end can obtain the first video, and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This scheme analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored by the residual and the compressed video later, which can improve the video quality.

[0064] It should be noted that the division of units and / or modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit and / or module in each embodiment of the present application may be integrated into a processing unit and / or module, or each unit and / or module may exist physically separately, or two or more units and / or modules may be integrated into one unit and / or module. The above-mentioned integrated units and / or modules may be implemented in the form of hardware or in the form of software functional units and / or modules.

[0065] If the integrated units and / or modules are implemented in the form of software functional units and / or modules and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0066] In addition, the data transmission device and the data transmission method provided in the above embodiments are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0067] Figure 7 A structural block diagram of a network device is shown according to an exemplary embodiment.

[0068] like Figure 7 As shown, the network device 1100 includes at least: a processor 1110 , a memory 1120 , and a transceiver 1130 .

[0069] The transceiver 1130 is used to receive and send data under the control of the processor 1110 .

[0070] exist Figure 7In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1110 and memory represented by memory 1120. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1130 may be a plurality of components, namely, a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, which may include a wireless channel, a wired channel, an optical cable, and other transmission media.

[0071] The processor 1110 is responsible for managing the bus architecture and general processing, and the memory 1120 can store data used by the processor 1110 when performing operations.

[0072] Optionally, the processor 1110 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor 1110 may also adopt a multi-core architecture. The processor 1110 and the memory 1120 may also be physically arranged separately.

[0073] The processor 1110 calls the computer program stored in the memory 1120 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0074] Figure 8 A structural block diagram of a user equipment is shown according to an exemplary embodiment.

[0075] like Figure 8 As shown, the user equipment 1300 includes at least: a processor 1310 , a memory 1320 , and a transceiver 1330 .

[0076] The transceiver 1330 is used to receive and send data under the control of the processor 1310 .

[0077] exist Figure 8In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 1310 and various circuits of memory represented by memory 1320 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1330 may be a plurality of components, namely, a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, including transmission media such as wireless channels, wired channels, and optical cables. For different user devices, the user interface 1340 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0078] The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 can store data used by the processor 1310 when performing operations.

[0079] Optionally, the processor 1310 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor 1310 may also adopt a multi-core architecture. The processor 1310 and the memory 1320 may also be physically arranged separately.

[0080] The processor 1310 calls the computer program stored in the memory 1320 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0081] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0082] In addition, a storage medium is provided in an embodiment of the present application, on which a computer program is stored, and when the computer program is executed by the processor, the data transmission method in the above embodiments is implemented. The storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (such as floppy disk, hard disk, tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid state drive (SSD)), etc.

[0083] In an embodiment of the present application, a program product is provided, for example, the program product is an FPGA chip or a DSP chip, and the program product includes executable instructions, and the executable instructions are stored in a storage medium. The processor reads the executable instructions from the storage medium, so that when the executable instructions are executed by the processor, the data transmission method in each of the above embodiments is implemented.

[0084] The solution of the present application can be applied in video compression scenarios. The video can be compressed using a lossy compression method. After that, the residual between the original video and the compressed video can be analyzed, and then the residual can be encoded and compressed. The residual can be restored and the video can be restored based on the residual and the compressed video to improve the video quality. This solution designs a variety of ways to encode and compress the residual. For example, the residual can be encoded using a lossless compression method, or the residual can be approximately lossless encoded within the error range. Different levels and level errors can also be set to transform the residual into a corresponding level for residual encoding. This solution can involve a transmitter and a receiver. The transmitter compresses the video and transmits it to the receiver. After receiving the video stream containing the compressed video and the residual, the receiver reconstructs the video based on the video stream. The sending end can obtain the first video, and perform lossy compression on the first video to obtain the second video; determine the residual data of the first video and the second video; and determine the residual features and the contextual features of the residual features based on the residual data; input the residual features and the contextual data of the residual features into the entropy model to determine the probability mass function, and process the residual data in an arithmetic coding manner to determine the reconstructed residual; merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream to the receiving end. The receiving end obtains the merged bitstream, and determines the second video and the reconstructed residual based on the merged bitstream; determines the reconstructed video based on the second video and the reconstructed residual. This scheme analyzes the residual between the original video and the compressed video, and then selects the corresponding encoding compression method for the residual to perform encoding compression. The video can be restored by the residual and the compressed video later, which can improve the video quality.

[0085] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0086] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0088] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0089] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0090] The above is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be included in the scope of protection of the present application.

Claims

1. A video secondary compression method based on residual coding, characterized in that: The method comprises: Obtain a first video, and perform lossy compression on the first video to obtain a second video; Determine residual data of the first video and the second video; and determine residual features and context features of the residual features based on the residual data; The residual features and the context data of the residual features are input into the entropy model to determine the probability mass function, and the residual data is processed by arithmetic coding to determine the reconstructed residual; The bitstream of the second video and the bitstream of the reconstructed residual are combined, and the combined bitstream is sent.

2. The method according to claim 1, characterized in that The steps of determining residual features based on residual data include: The residual data is input into the feature extraction network for feature extraction to obtain residual features.

3. The method according to claim 2, characterized in that Steps to determine contextual data include: The residual data is input into a masked convolutional layer with a target receptive field, and the target receptive field of the masked convolutional layer is used as context data.

4. The method according to claim 3, characterized in that The residual data is input into a mask convolution layer having a target receptive field, comprising: Obtain the error range and quantify the residual data based on the error range; The quantized residual data is input into the masked convolution layer with the target receptive field; The step of inputting the residual features and the context data of the residual features into the entropy model to determine the probability mass function includes: The residual features, error ranges, and contextual data of the residual features are input into the conditional entropy model to determine the probability mass function.

5. The method according to claim 3, characterized in that The residual data is input into a mask convolution layer having a target receptive field, comprising: Obtain multiple residual levels and the level error ranges corresponding to different residual levels; Transform the residual data to the target level based on the residual level and level error range; The object level is input to a masked convolutional layer with the object receptive field; The step of inputting the residual features and the context data of the residual features into the entropy model to determine the probability mass function includes: The residual features, the level error range, and the contextual data of the target level of the residual data are input into the conditional entropy model to determine the probability mass function.

6. The method according to claim 1, characterized in that The step of determining a reconstructed residual of a target frame comprises: Obtain adjacent frames of the target frame, and obtain adjacent residual distributions corresponding to residual data of the adjacent frames; Based on the adjacent residual distribution, the reconstructed residual corresponding to the residual data of the target frame is determined.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining a merged bitstream, and determining a second video and a reconstructed residual according to the merged bitstream; A reconstructed video is determined based on the second video and the reconstructed residual.

8. A video secondary compression device based on residual coding, characterized in that: The device comprises: A second video acquisition module is used to acquire the first video and perform lossy compression on the first video to obtain a second video; A residual data acquisition module, used to determine residual data of the first video and the second video; and determine residual features and context features of the residual features based on the residual data; A reconstruction residual acquisition module is used to input the residual features and the context data of the residual features into the entropy model, determine the probability mass function, process the residual data in an arithmetic coding manner, and determine the reconstructed residual; The merged bitstream generation module is used to merge the bitstream of the second video and the bitstream of the reconstructed residual, and send the merged bitstream.

9. A network device, characterized in that: include: A memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; The processor is used to read the computer program in the memory and execute the method according to claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claims 1-7 is implemented.

Images