36 results about "Rate distortion" patented technology

The invention discloses a VVC code rate control algorithm based on deep reinforcement learning, and the algorithm comprises the following steps: importing a video sequence into an encoder, and enabling the video sequence to enter initial frame coding; after the encoder completes the default encoding of the first two frames, the subsequent frame prediction firstly extracts the encoding state information of the previous prediction frame, and a greedy strategy is adopted to perform action selection; an overall reward value is obtained through CTU-level code rate control and an actual coding process in sequence; observing the next state and the last reward, performing TD iteration on the Q value, and adding the Q value into the Q value network after TD iteration; setting an experience playback pool; after the capacity of the experience playback pool reaches a threshold value, randomly sampling from the experience playback pool in batches; resetting the Q value network; and carrying out coding test based on the Q value network obtained by training. The VVC code rate control algorithm based on deep reinforcement learning has a good capability of guiding code rate control coding, and compared with standard code rate control in VTM13.0, the VVC code rate control algorithm based on deep reinforcement learning can bring high rate distortion performance and improve code control precision.

The invention provides a video coding method, which comprises the following steps of: inputting a coding tree unit to be coded, and extracting texture features of a coding block, the texture features comprising local binary pattern similarity of all pixel points in the coding block and direction dispersion of each pixel point; inputting the texture features into the trained multi-type division prediction architecture, and sequentially judging whether to execute quadtree division, horizontal or vertical division, horizontal binary tree or ternary tree division and vertical binary tree or ternary tree division or not through a binary classification problem method; and calculating the rate distortion cost under each division mode to obtain an optimal division mode and a suboptimal division mode of the coding block, and outputting an optimal mode or a combination of the optimal mode and the suboptimal mode. According to the invention, the accuracy of the prediction mode in coding is improved.

Systems and methods directed to a rate-distortion optimization process. One example is directed to a video encoding device including an electronic processor. The electronic processor is configured to obtain motion information associated with a video block and determine, as a function of the motion information, a subset of coding tools from a plurality of coding tools for rate-distortion performance evaluation of the video block.

A better rate distortion ratio is achieved by making interrelationships between coding parameters of different planes available for exploitation for the aim of redundancy reduction despite the additional overhead resulting from the need to signal the inter-plane prediction information to the decoder. In particular, the decision to use inter plane prediction or not may be performed for a plurality of planes individually. Additionally or alternatively, the decision may be done on a block basis considering one secondary plane.

The application discloses a video compression transmission method and related equipment, the method comprises the following steps: obtaining a video sequence to be compressed, taking the first frame and the last frame as conditional frames; inputting the video sequence into the encoder of a pre-trained variational autoencoder to obtain a target latent space representation; processing the target latent space representation through the downsampling module of the compressor to generate an extreme compression representation; transmitting the conditional frames and the extreme compression representation to the receiving end to enable the receiving end to reconstruct the video through the upsampling module of the compressor, the generation model and the decoder of the variational autoencoder to obtain a reconstructed video sequence; the application provides time sequence boundary information through the conditional frames, and in combination with the reconstruction capability of the generation model, can effectively reduce the block effect, blur and high-frequency detail loss commonly seen in traditional methods; the conditional frames and the extreme compression representation significantly reduce the transmission code rate and bandwidth demand, can significantly improve the rate distortion performance, and can be widely applied to the technical field of video compression.

This invention discloses deep learning-based video compression using optimized rate distortion. Systems and methods are provided for enhancing image patches of frames in a video to be compressed. These systems and methods enhance image patches to improve the compression of the video. In at least one embodiment, systems and methods are provided for enhancing video content using a learnable pre-filtering network trained with a joint loss function to improve compression.

Apparatuses and methods are disclosed for encoding mesh data. Techniques disclosed include receiving a sequence of frames, each of which includes mesh data. For a frame in the sequence, techniques disclosed for encoding the mesh data of the frame according to a static path and according to a motion path of a multipath encoder, computing a static path cost of the encoding according to the static path and a motion path cost of the encoding according to the motion path, where the costs are computed by optimizing a rate-distortion cost function, and selecting, based on the computed motion path cost and static path cost, a bitstream generated by the encoding according to the motion path or a bitstream generated by the encoding according to the static path.

Techniques are described for storing (302) BC7 texture blocks on disk with lower precision index data and transcoding (304) back to the GPU native BC7 format when loading data. And transcoding back to the standard BC7 is lossless.

A kind of machine vision rate distortion encoding method of license plate recognition task, by obtaining license plate image data set, extracting saliency feature, modulation adaptive quantization parameter, rate distortion optimization and fast division, license plate recognition step composition.The present application extracts background, vehicle, license plate three-level area feature using target detection network, calculates normalized perception intensity, uses hyperbolic tangent function to dynamically adjust the quantization parameter of coding unit, and optimizes the coding division process by combining subsection depth constraint strategy.Solves the technical problems of license plate character target blur and machine recognition rate decline under the conditions of application of video coding on unmanned aerial vehicle and limited code rate, reduces the encoding code rate and computational complexity, preserves the high-frequency microscopic details of license plate characters, improves the recognition accuracy and robustness of machine vision.The present application has the advantages of balancing code rate, low computational complexity, high license plate recognition accuracy, and can be used for unmanned aerial vehicle long-distance license plate recognition.

A method and apparatus for predicting motion vector for inter-frame encoding are provided. An implementation scheme of the method includes: acquiring first and second sets of motion vectors; dividing, in response to the number of valid adjacent PUs being greater than or equal to a preset number, the second set of motion vectors into at least one motion vector subset; calculating a correlation between the first set of motion vectors and each motion vector subset respectively, to obtain a priority of each adjacent PU; calculating, sequentially according to the priority in descending order, a rate distortion based on a motion vector of each adjacent PU, and stop calculating until a rate distortion smaller than a predetermined threshold is obtained; and determining a motion vector of an adjacent PU used when the rate distortion smaller than the predetermined threshold is obtained as the motion vector of the current PU.

The embodiment of the invention discloses a video coding mode determination method and device, a coder, a medium and a product, and the method comprises the steps: determining a preset condition corresponding to a coding stage in a video coding process; wherein the nth preset condition corresponds to the nth coding stage, and is matched with the characteristics of the nth coding stage; n is smaller than or equal to N; n is the total number of the coding stages; under the condition that the coding parameter of the to-be-processed coding tree unit meets a target preset condition in the preset conditions, determining a coding mode of the to-be-processed coding tree unit as a coding mode corresponding to the target preset condition, and executing a skipping operation based on the coding mode corresponding to the target preset condition; the coding parameter comprises coding cost and / or a motion vector; the coding stage at least comprises any one of the following stages: a down-sampling motion estimation stage, an integer pixel motion estimation stage, a sub-pixel motion estimation stage and a rate distortion optimization stage.

The embodiment of the invention provides a three-dimensional video compact representation method and device, equipment and a storage medium, and relates to the technical field of video processing. The method comprises the following steps: determining a plurality of anchor point primitives according to a key frame sequence, carrying out quantization coding on the anchor point primitives to obtain a key frame coding code stream which comprises corresponding decoding anchor point primitives, selecting non-key frame sequences one by one as a processing frame sequence, obtaining a reference frame sequence according to a previous acquisition moment of the processing frame sequence, and carrying out decoding on the reference frame sequence according to a corresponding decoding anchor point primitive. Performing rate distortion optimization on the decoding anchor point primitive corresponding to the reference frame sequence to generate a coding motion parameter, determining a coding residual error anchor point primitive according to the decoding motion parameter corresponding to the coding motion parameter, and obtaining a decoding anchor point primitive corresponding to the processing frame sequence according to the decoding residual error anchor point primitive corresponding to the coding residual error anchor point primitive, and sending the coding code streams of all the frames to a decoding end for decoding operation. The video coding efficiency and expression compactness can be improved, and the overall coding complexity can be reduced.

The embodiment of the invention discloses a coding and decoding method, a code stream, a coder, a decoder and a storage medium, and the method comprises the steps: at a decoding end, the decoder determines a prediction mode corresponding to a current point; when the prediction mode is the first mode, determining a neighbor point corresponding to the current point in the current frame, and determining a reference point corresponding to the current point in a reference frame corresponding to the current frame; and determining a predicted value of the current point according to the first predicted value of the reference point and the second predicted value of the neighbor point. At an encoding end, an encoder determines a prediction mode corresponding to a current point according to a rate distortion optimization algorithm; or determining the prediction mode corresponding to the current point according to the correlation between the current point and the reference point corresponding to the current point in the reference frame; wherein the prediction mode comprises a first mode and a second mode.

Systems, methods, and instrumentalities are disclosed for selecting an inter-layer mode as the coding mode for encoding a portion of a video block. A plurality of candidate modes may be determined for encoding a portion of a video block. For each candidate mode of the plurality of candidate modes, a first rate-distortion cost may be calculated using at least a first rate-distortion multiplicator. It may be determined whether an inter-layer mode among the plurality of candidate modes has the lowest rate-distortion cost. The inter-layer mode may be selected as the coding mode, if the inter-layer mode has the lowest rate-distortion cost. An indication of the selected coding mode and encode the portion of the video block may be coded according to the selected coding mode.

The embodiments of the present application relate to the technical field of computer vision. Provided are a three-dimensional scene reconstruction method and apparatus, and a device and a storage medium. The method comprises: acquiring an anchor-based initial three-dimensional representation model corresponding to a target scene; for each anchor, performing compression encoding on anchor attributes to obtain an encoded bitstream corresponding to the target scene; performing entropy decoding on the encoded bitstream to obtain a decoded feature attribute, a decoded size attribute and a decoded offset attribute; on the basis of the decoded feature attributes and a feature attribute mean value, performing weighted prediction to obtain weighted feature attributes, and performing channel-wise grouping on the weighted feature attributes to obtain rendered feature attributes; and finally, on the basis of the rendered feature attributes, the decoded size attributes and the decoded offset attributes, generating a three-dimensional representation model of the target scene. By means of a weighted prediction process and a channel-wise grouping process, feature attributes of each anchor are simplified, so as to improve the rate distortion performance of an initial three-dimensional representation model while reducing a storage space.

A method is provided for performing an entropy coding scheme for a signal enhancement filter. The method includes estimating rate distortion enhancement for the current frame and one or more subsequent frames, and optimizing the coding of parameters based on the estimated rate distortion costs.

The invention provides a face image compression method and system for a 3D video conference, a medium and equipment, and the method comprises the steps: carrying out the model compression of a pre-trained personalized face model, and completing the transmission in a 3D conference initialization stage; at the coding end, adopting a preset face parameter tracker to extract attitude parameters and expression parameters from the 3D face image of the 3D conference; performing quantization processing and zero-order exponential Golomb coding on the attitude parameters and the expression parameters in sequence, and then transmitting the attitude parameters and the expression parameters; at a decoding end, performing exponential-Golomb decoding and inverse quantization processing on the real-time transmission code stream; carrying out decompression processing on the model compression representation by adopting an LZ77 algorithm; and according to the recovered attitude parameters and the recovered expression parameters, a pre-trained personalized face model is adopted to render and generate a reconstructed 3D face image through a 3D Gaussian sputtering technology. According to the invention, the fine geometry and appearance details of the 3D face are reserved at an extremely low bit rate, and the rate distortion performance is improved.

A method for learned image compression implemented in an autoencoder includes: a) extracting from an image a latent space by the learnable encoder; b) quantizing the latent space by a quantizer to obtain a quantized latent space; c) entropy coding the quantized latent space by an entropy encoder to obtain a bitstream, wherein an entropy model used to encode the latent space is represented by a probability distribution; d) entropy decoding the bitstream by an entropy decoder to obtain an entropy decoded bitstream; e) feeding the entropy decoded bitstream to the decoder; f) recover a reconstructed image by the decoder; g) training the autoencoder via standard gradient descent of the backpropagated error gradient by finding learnable parameters of the learnable encoder and of the decoder that minimize a rate distortion cost function, wherein the entropy encoder is based on a differentiable formulation of a soft frequency counter.

The application relates to the technical field of image compression, and discloses an image compression and reconstruction method based on vector quantization, which compresses and reconstructs an input image x through an image coding and decoding model to obtain a reconstructed image, the image coding and decoding model comprises a multilayer structure; each layer structure comprises a down-sampling layer, an up-sampling layer and a plurality of quantization layers; the down-sampling layers and the quantization layers in the layer structures are sequentially arranged to form a compression path; the up-sampling layers and the quantization layers in the layer structures are sequentially arranged to form a reconstruction path; compared with a traditional coding architecture, the upper bound of rate distortion performance is higher, the problem of excessively high coding and decoding complexity of vector quantization is solved, and the practicability of the whole framework is improved.

Decoding using rate distortion optimization including an inverse transform including transform size adaptive directional nonlinear filtering includes generating decoded block data by decoding a current block of a current frame. Decoding the current block comprises: obtaining quantized transform coefficients of the current block from the encoded bitstream; obtaining transform data of the current block from the encoded bitstream, wherein the transform data indicates a transform type and a transform size; obtaining dequantized transform block data by dequantizing the quantized transform coefficient; and obtaining decoded residual block data by inversely transforming the dequantized transform block data from the transform data. Inverse transform of the dequantized transform block data includes: obtaining intermediate decoded block data by combining prediction block data of a current block and decoded residual block data; and obtaining decoded block data by filtering the intermediate decoded block data using a transform size adaptive directional nonlinear filter.

The invention provides a rate distortion optimization quantization method, a rate distortion optimization quantization device, electronic equipment, a chip, a storage medium and a computer program product. The rate distortion optimization quantization method comprises the following steps: performing column scanning on a transformation unit (TU), and performing first processing on each column of coefficient group (CG) obtained by scanning in parallel by taking the CG as a unit until all columns of the TU are scanned, thereby obtaining an optimal quantization coefficient of the TU; the first processing comprises pre-quantization processing, level estimation processing and all-zero coefficient group detection processing; and determining the position of the optimal last significant coefficient (LSC) based on the optimal quantization coefficient of the TU.

The invention relates to the technical field of machine learning, and discloses an H.266 coding macro block division method based on machine learning, which comprises the following steps: step 1, multi-dimensional feature extraction: extracting content feature features, boundary and position features and division history and rule features of a current coding block; step 2, intelligent division decision making: the multi-dimensional features are input into a double-branch Transform network for processing, the double-branch Transform network comprises a content branch and a constraint branch, the features are fused through a cross attention mechanism, and the size of macro block division and probability distribution of a division mode are output; and step 3, multi-stage check optimization: carrying out candidate set screening based on the probability distribution, carrying out rate distortion check on screened candidate partitions, and selecting the partition with the minimum rate distortion cost as a final macro block partition result. According to the method, intelligent decision-making of the size and the division mode of the macro block can be achieved, the VVC standard is accurately adapted, and the accuracy and the coding efficiency of macro block division in H.266 / VVC coding can be improved.

Disclosed in the embodiments of the present application are an encoding method, a decoding method, a bit stream, an encoder, a decoder and a storage medium. The decoding method comprises: at a decoding end, decoding a bit stream to determine first identification information; when, on the basis of the first identification information, it is determined to use a CCALF to perform filtering processing on a current color component of a current image, determining a scale factor and a filtering coefficient corresponding to the current color component; and on the basis of the scale factor and the filtering coefficient, determining a reconstructed block of a current block. The encoding method comprises: at an encoding end, determining a scale factor corresponding to a current color component of a current image; and on the basis of a rate distortion cost corresponding to the scale factor, determining first identification information, and writing the first identification information into a bit stream, wherein the first identification information is used for indicating whether to use a CCALF to perform filtering processing on the current color component of the current image.Fig. 6

To provide a multiple perspectives signal codec which enables a higher compression rate or a better rate / distortion ratio.SOLUTION: The encoder 20 achieves a higher compression rate or a better rate / distortion ratio by adopting or predicting the second encoding parameters to be used for encoding the second view 122 of the multiple perspectives signal from the first encoding parameters 461, 481, 501 used when encoding the first view 121 of the multiple perspectives signal 10. The redundancy between the viewpoints of the multiple perspectives signal is not limited to the viewpoints themselves such as the video information of these viewpoints, but is a point that the coding parameters when coding these viewpoints in parallel also indicate similarity, and if this point is used, the coding rate can be further improved.SELECTED DRAWING: Figure 1

The invention discloses an initial QP selection optimization algorithm based on hierarchical reinforcement learning. According to the algorithm, action QP value selection is carried out from a rough target range to a fine target range by adopting a hierarchical reinforcement learning strategy; a two-layer decision selection architecture is adopted, and an enhanced Q learning algorithm is adopted; a reward value, namely a Q value, is obtained through CTU-level code rate control and an actual coding process in sequence; observing the next state and the last reward, performing TD iteration on the Q value, and updating the Q value to a corresponding Q table; performing an initial QP coding test based on the Q table obtained by training; the current state information is extracted, the Q table is inquired according to the current state, and the Q table gives a preferred action, namely, a preferred QP value; and then carrying out subsequent operation according to a coding process. According to the initial QP selection optimization algorithm based on hierarchical reinforcement learning, a long-term rate distortion target and a current coding target of video sequence coding are considered at the same time, and the method has a good capability of guiding code rate control optimization coding.

A video encoding system is configured to determine, based on image data, a distortion metric for a prediction mode candidate based on 1) gradient similarity between a prediction block of the image data and a reconstructed prediction block of reconstructed image data generated based on the image data and 2) motion between two frames of the image data. The video encoding system is also configured to select, from a plurality of prediction mode candidates, the prediction mode candidate as a prediction mode based at least in part on the distortion metric. Additionally, the video encoding system is configured to encode the image data based on the prediction mode.

The invention relates to a video subjective quality optimization method based on face area rate distortion adjustment, and belongs to the technical field of video coding. The method specifically comprises the following steps: triggering a human face region detection and coding optimization whole process through a human face optimization enabling control mechanism; performing skin color region multi-dimensional detection based on the enabling activation identifier, screening skin color pixels through a chrominance component preset judgment interval, synchronously calculating an absolute error and a texture complexity parameter of an image block, and generating a coding unit classification identifier set; in combination with a matching relationship between the frame-level quantization parameter and a preset trigger threshold, performing a differential optimization process on different types of coding units to generate a coding configuration adjustment result; and finally, counting a skin color related coding unit proportion and matching the skin color related coding unit proportion with a preset threshold, and generating a next frame coding intervention configuration instruction. According to the invention, targeted coding optimization is carried out on the skin color region related to the face, so that the subjective quality of the face region in the video is improved on the premise of ensuring the coding efficiency.

It is described a method for learned image compression implemented in an autoencoder (1') comprising an image reconstruction encoder (fa) and an image reconstruction decoder (fs), said method comprising the steps of: a) extracting from an image (x) a main latent representation (y) by means of said image reconstruction encoder (fa); b) quantizing said main latent representation (y) by means of a quantizer (2) to obtain a discrete main latent representation (ŷ); c) entropy coding said discrete main latent representation (ŷ) by means of an entropy encoder (4) to obtain a bitstream (5); d) entropy decoding said bitstream (5) by means of an entropy decoder (6) to obtain an entropy decoded bitstream; e) feeding said entropy decoded bitstream to said image reconstruction decoder (fs); f) recovering a reconstructed image (x ̂) by means of said image reconstruction decoder (fs); g) training said autoencoder (3) via standard gradient descent of the backpropagated error gradient by minimizing a rate distortion cost function L through relaxing said discrete main latent representation (ŷ) by means of a soft-quantized main latent representation (ỹ), wherein said quantizer (2) is implemented as a parameterized activation function (I) wherein L is a number of sought quantization levels, β is an inverse temperature parameter regulating a slope of said quantization levels, and w and b are parameters respectively determining reconstruction levels and width of quantization intervals of said soft-quantized main latent representation (ỹ).

In some embodiments, a method includes inputting a proxy encoding configuration into an encoder. This encoder determines proxy quality values from an actual encoding of a portion of a video. These proxy quality values are for a set of bitrates and a resolution. The method also determines target feature values for the same portion of the video. The method inputs both the target feature values and a target configuration into a prediction network. This network generates a plurality of quality offset values for the set of bitrates and the resolution. The prediction network is configured to predict an offset to the proxy quality values. This offset is determined based on a difference between the proxy encoding configuration and the target configuration. The method then generates a plurality of quality values. The generation uses the plurality of proxy quality values and the corresponding plurality of quality offset values.

The present disclosure relates to a video encoder, a video encoding method, an electronic device, a storage medium and a computer program product, the video encoder comprising: a rate distortion optimization module for receiving an ith probability model sent by an entropy encoding module when an (N + i) th coding tree unit CTU in a to-be-encoded video frame needs to be processed, where i is a positive integer greater than or equal to 1, and N is a positive integer greater than or equal to 1; n is a positive integer greater than 1; and the rate distortion optimization module is used for carrying out rate distortion optimization on the (N + i) th CTU by utilizing the ith probability model, and determining and sending coding information of the (N + i) th CTU to the entropy coding module. According to the embodiment of the invention, the calculation time for executing the code rate estimation in the rate distortion optimization process can be effectively reduced, and the overall calculation performance of the video encoder is effectively improved.