196 results about "Rate distortion" patented technology

The invention provides an image coding and decoding method and a compression method for machine and human vision, and the method mainly comprises the steps: obtaining a first feature map and a second feature of a to-be-coded image, and obtaining the super-prior information of the first feature map; obtaining quantization step sizes of the to-be-coded image under different machine vision tasks, and performing quantization to obtain a corresponding feature map; modeling the feature map into Gaussian distribution, and predicting a corresponding mean value and variance through an entropy model; coding layer by layer according to the distribution and probability of each layer of feature map to obtain a feature map binary code stream; and combining the super-prior information binary code stream and the feature map binary code stream to obtain a compressed image binary code rate. According to the method, the quantization step size of feature map quantization is adaptively adjusted, transmission feature code streams for human vision and different machine vision tasks are formed, information in the feature map can be more efficiently utilized, the transmission code rate is reduced, meanwhile, the performance of the machine vision tasks is guaranteed, and meanwhile, the rate distortion performance oriented to human vision is balanced.

The invention discloses a video compression intelligent preprocessing method, system and device based on frequency domain perception optimization and a medium. The method comprises the following steps: constructing a preprocessing model; preprocessing the input coding frame and the reference frame; enabling the preprocessed coding frame and reference frame to pass through a virtual encoder to obtain a frequency domain coefficient of a residual frame, and calculating a code rate of the coding frame; enabling the frequency domain coefficient of the residual frame to pass through a virtual decoder to obtain a decoded frame; training a preprocessing model; testing the effect; the system, the equipment and the medium are used for implementing the method. According to the method, a frequency domain sensing module is introduced into a virtual encoder in a training stage, modeling is carried out on a frequency domain information context, and statistical redundancy between transformation units is eliminated; an optical flow collaboration module is introduced into a virtual encoder, local distortion of a prediction frame is repaired in a feature domain by using high-level semantic information, and the difference between training and actual deployment is effectively reduced; through the optimization, the preprocessing model can better learn the rate distortion characteristic of the standard encoder, and the quality of the preprocessed video is remarkably improved.

The invention provides a communication segmentation learning system and method for adaptive channel compression, and a medium, and relates to the technical field of segmentation learning and communication compression. The system comprises a plurality of clients, a server side and a channel compression device arranged between the clients and the server side, the channel compression device comprises a channel sensitivity modeling module, a rate distortion adaptive compression module and a cross-client fair coordination module. In a channel sensitivity modeling module, channel importance is dynamically evaluated through intermediate layer activation value fusion; differentiated quantization and compression strategies are designed on the basis of sensitivity scores in a rate-distortion self-adaptive compression module, so that key information is reserved while communication overhead is reduced; and meanwhile, the cross-client fair coordination module realizes balanced distribution of communication resources among multiple clients through a fairness regularization and dual optimization mechanism, so that the influence of excessive compression on global convergence is avoided. According to the method, the communication efficiency and the training stability of segmentation learning in a complex heterogeneous environment are remarkably improved.

The invention provides an image coding method and device based on deep learning, and belongs to the field of image coding, and the image coding method comprises the steps: a coding end uses nonlinear transformation to project an original image to a hidden space; the coding end performs probability distribution estimation by using a causal context; the encoding end encodes the hidden variables by using an entropy encoding technology; the decoding end performs probability distribution estimation by using a causal context; the decoding end decodes the hidden variables by using an entropy coding technology; the coding end reconstructs the hidden variables into an original image by using nonlinear transformation; and performing end-to-end reverse gradient propagation optimization on an image coding and decoding process by using a distortion loss function and a causal context adjustment loss function. By using a better causal context to adjust a loss function and a more efficient network structure, better image compression capability and higher rate distortion performance are obtained.

The embodiment of the invention provides a Gaussian point cloud rendering method and device, equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: acquiring pixel points of foreground targets in two corresponding key frames, generating a double-view-angle Gaussian point cloud based on the pixel points, quantifying double-view-angle Gaussian parameters of the double-view-angle Gaussian point cloud based on different rate-distortion coefficients to obtain quantized Gaussian parameters corresponding to each rate-distortion coefficient, and obtaining the target foreground target in the two key frames according to the quantized Gaussian parameters of the double-view-angle Gaussian point cloud and the quantized Gaussian parameters of the double-view-angle Gaussian point cloud. At least entropy coding is carried out on the quantized Gaussian parameter to obtain a coded Gaussian parameter; and determining a target rate-distortion coefficient according to the definition requirement, selecting a coding Gaussian parameter corresponding to the target rate-distortion coefficient as a target coding Gaussian parameter, and sending at least one target coding Gaussian parameter corresponding to each dual-view sequence to a decoding end. The frame extraction operation is performed on the video frame, so that the data volume can be reduced. Secondly, quantitative Gaussian parameters corresponding to different definitions are obtained according to different rate distortion coefficients, and balance of data transmission and rendering quality is achieved.

Given a sequence of images in a first codeword representation, methods, processes, and systems are presented for image reshaping using rate distortion optimization, wherein reshaping allows the images to be coded in a second codeword representation which allows more efficient compression than using the first codeword representation. Syntax methods for signaling reshaping parameters are also presented.

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is proposed. The method comprises: determining, for a conversion between the current video block and a bitstream of the video, a target number for a set of motion candidates, the target number being smaller than or equal to an upper limit value; determining, based on the target number and a fast rate distortion optimization (RDO) process, the set of motion candidates from a plurality of motion candidates for an intra block copy (IBC) based mode for coding the current video block; and performing the conversion based on the set of motion candidates.

Disclosed in the present application are a rate-distortion optimized quantization method, system and device for video coding, and a storage medium. The technical solution provided in the present application comprises: during entropy coding bit estimation of rate-distortion optimized quantization, according to the fact that a set first fixed value represents first bit consumption information required for coding the current pixel point when a coefficient to be coded is greater than 1, calculating an optimized quantization coefficient of the current pixel point on the basis of the first fixed value (S110); and according to the fact that a set second fixed value represents second bit consumption information required for marking the current pixel point as the last non-zero bit of the current transform unit, calculating, on the basis of the second fixed value, a rate distortion cost for coding the current pixel point as the last non-zero coefficient (S120). By using the technical means, the time consumption of rate-distortion optimized quantization can be shortened while ensuring the coding quality, and the optimization of a rate-distortion optimized quantization process is realized, thereby improving the video coding efficiency.

The present disclosure relates to picture encoding and decoding of image regions on tile-basis. In particular, multiple components of an input tensor including a first and second component in spatial dimensions is processed within multiple pipelines. The processing of the first component includes dividing the first component in the spatial dimensions into a first plurality of tiles. Likewise, the processing of the second component includes dividing the second component in the spatial dimensions into a second plurality of tiles. The respective first and second plurality of tiles are then processed each separately. Among the first and second plurality of tiles there are at least two respective collocated tiles differing in size. In case of compression, the processing of the first and / or second component includes picture encoding, rate distortion optimization quantization, and picture filtering. In case of decompression, the processing includes picture decoding and picture filtering.

Methods, apparatus, systems, and articles of manufacture are disclosed to improve video encoding. An example apparatus includes at least one memory, instructions, and processor circuitry to generate a pool of clipping index set candidates by executing a machine learning model, select a clipping index set from the pool of the clipping index set candidates based on a rate distortion cost associated with the clipping index set, the clipping index set including clipping coefficients, and filter a video frame based on the clipping coefficients.

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is proposed. The method comprises: determining, for a conversion between a video unit of a video and a bitstream of the video unit, whether to apply at least one neural network (NN) filter model or determine a rate distortion cost during a rate distortion optimization (RDO) process of the video unit based on at least one of: a distortion without NN filter model, a distortion with n-th NN filter model, a combination of distortions of a plurality of NN filter models, or coding statistics of the video unit, and wherein n is an integer number; determining a coding mode of the video unit based on a rate distortion optimization (RDO) criterion in the RDO process; and performing the conversion based on the coding mode.

The embodiment of the invention provides an end-to-end image compression method and system based on window local attention and generalized chessboard space channel context, and belongs to the technical field of image processing. The method comprises the following steps: constructing a transformation network based on an attention module and a stacked residual block; the transformation network based on the attention module and the stacked residual block is used for executing adaptive transformation of contents through dynamic representation and neighborhood information embedding to obtain potential features; establishing a generalized chessboard space channel context model; the generalized chessboard space channel context model is used for carrying out entropy coding on the potential features; and obtaining image compression data according to the transformation network based on the attention module and the stacked residual block and the generalized chessboard space channel context model. According to the method, redundancy can be eliminated to the maximum extent, excellent rate distortion performance is achieved, and meanwhile high-throughput parallel computing efficiency is ensured.

The invention discloses a joint optimization method for a joint photographic experts group (JPEG) quantization table and a truncation position based on rate distortion, relates to the technical field of JPEG coding optimization, and solves the technical problems that in a JPEG compression scheme, improvement of coding efficiency caused by joint optimization by using the quantization table and the truncation position is neglected, and the optimal quantization table and the optimal truncation position are difficult to find. According to the method, joint optimization is carried out on a quantization table and a truncation position of the JPEG by combining a JPEG compression method from the perspective of evolutionary computation, a strategy that the truncation position is fixed when the quantization table is optimized, a strategy that the quantization table is unchanged when the truncation position is optimized, an estimation method for designing fitness evaluation and a method based on rate-distortion optimization are respectively carried out, and the quantization table and the truncation position are optimized. The optimal quantization table and the truncation position can be effectively found, and the corresponding truncation position is optimized while multiple groups of quantization tables are generated in the optimization process; through joint optimization of the quantization table and the truncation position, image data can be compressed more effectively, redundant information is reduced, and therefore coding efficiency is improved.

The invention belongs to the technical field of video compression coding and decoding, and particularly relates to a fast VVC intra-frame coding method for machine video coding. According to the method, a neural network technology is adopted, and encoders and decoders of a convolutional neural network (CNN) block and a deep former block are connected in series; and efficient and accurate coding is realized, so that the segmentation judgment of the coding block is more accurate. According to the method, block division is guided by retraining the CNN network, the complexity of a rate distortion optimization (RDO) process is reduced, and the method is particularly suitable for a new generation of universal video coding standard VVC. According to the method, the coding complexity can be reduced, and meanwhile, the video coding and decoding performance is remarkably improved, especially in a machine-oriented application scene.

The invention provides a code rate estimation method suitable for hardware implementation. The code rate estimation method comprises the following steps: S1, optimizing a context probability model; s2, independently designing a bit processing unit (BPU) for updating and storing a context model value; and S3, carrying out hardware design on the entropy coding process realized by hardware. According to the invention, a'context modeling of binarized bin 'module in the CABAC entropy coding process is optimized, and an algorithm and a hardware pipeline structure are provided, so that the time for estimating the consumed bit digital rate by using an entropy coding algorithm is shortened, the area and the power consumption of a hardware product are reduced, hardware implementation is facilitated, and the cost is reduced. According to the method, the dependency that context model values need to be updated bin by bin in each bin of entropy coding in the rate distortion optimization (RDO) process is eliminated, hardware parallel pipeline is facilitated, meanwhile, the key path of updating the context model values bin by bin is reduced, the calculation complexity is reduced, and thus the hardware power consumption is reduced.

The invention discloses a video coding method based on multi-domain perception feature fusion, and belongs to the technical field of image communication. The method comprises the following steps: extracting perception features of a space domain, a time domain and a frequency domain from an input video frame sequence; normalizing the multi-domain sensing features, and generating a sensing importance factor for each coding unit through a dynamic fusion weight mechanism; constructing perceptual distortion evaluation based on perceptual importance factors, replacing a traditional rate distortion objective function, and guiding code rate distribution of coding units; and according to the perception importance factor, performing nonlinear mapping by using a hyperbolic tangent function, calculating to obtain a coding unit level quantization parameter adjustment amount, and obtaining a final quantization parameter in combination with the basic quantization parameter, thereby realizing adaptive rate distortion optimization. According to the method, the visual sensitive area can be accurately identified, and the code rate is remarkably reduced while the subjective visual quality is guaranteed.

The invention provides a video coding method and device, electronic equipment, a storage medium, a program product and a bit stream generation method, and the video coding method comprises the steps: traversing a plurality of preset coding modes for a current block, and obtaining the rate distortion cost in each preset coding mode; selecting a coding mode of the current block from a plurality of preset coding modes based on the rate distortion cost in each preset coding mode; encoding the current block based on the encoding mode of the current block; wherein the step of acquiring the rate distortion cost in each preset coding mode comprises the following steps: acquiring a subjective quality index corresponding to the current coding mode; a penalty coefficient associated with the subjective quality index is calculated, and the penalty coefficient is in positive correlation with the possibility of the subjective quality problem indicated by the subjective quality index; and based on the penalty coefficient, amplifying the original rate-distortion cost of the current coding mode to obtain the rate-distortion cost of the current coding mode. The method can improve the subjective quality of the coded video.

The invention discloses a VVC code rate control algorithm based on deep reinforcement learning, and the algorithm comprises the following steps: importing a video sequence into an encoder, and enabling the video sequence to enter initial frame coding; after the encoder completes the default encoding of the first two frames, the subsequent frame prediction firstly extracts the encoding state information of the previous prediction frame, and a greedy strategy is adopted to perform action selection; an overall reward value is obtained through CTU-level code rate control and an actual coding process in sequence; observing the next state and the last reward, performing TD iteration on the Q value, and adding the Q value into the Q value network after TD iteration; setting an experience playback pool; after the capacity of the experience playback pool reaches a threshold value, randomly sampling from the experience playback pool in batches; resetting the Q value network; and carrying out coding test based on the Q value network obtained by training. The VVC code rate control algorithm based on deep reinforcement learning has a good capability of guiding code rate control coding, and compared with standard code rate control in VTM13.0, the VVC code rate control algorithm based on deep reinforcement learning can bring high rate distortion performance and improve code control precision.

The invention relates to a three-dimensional point cloud prediction geometric coding method based on deep learning, and the method comprises the steps: 1, forming a prediction tree through points collected by each laser transmitter, and converting an original laser radar point cloud LPC into a plurality of prediction trees for representation; 2, respectively designing different predictors and entropy encoders for each component of the coordinates of the points, and compressing each component of the coordinates of the points by applying different quantization step lengths; 3, selecting a quantization step size for quantifying each component by adopting a quantization step size selection strategy; 4, adopting an entropy model to model the probability distribution of the residual error of each component so as to encode the residual error entropy; and step 5, decoding is realized through a reverse process from the step 1 to the step 4. Compared with other methods, the method provided by the invention obtains the best rate distortion performance.

The invention discloses a video coding method based on conditional probability and reconstruction distortion propagation optimization, which comprises the following steps of: constructing a prospective image set, deciding a coding structure, constructing a block-level time domain reference relation chain, calculating accumulated distortion offset and determining time domain quantization parameter offset, calculating the prediction cost of each image block corresponding to each frame of image in the look-ahead image set based on the reference frame list, determining a reference block according to the prediction cost so as to construct a block-level time domain reference relation chain, and constructing a distortion dependency item based on the block-level time domain reference relation chain, and calculating an accumulated distortion offset based on the distortion dependency so as to calculate a time domain quantization parameter offset of each image block, and coding the to-be-coded reference frame based on the time domain quantization parameter offset. A reference frame list consistent with an actual coding process is set, so that a distortion propagation path of time domain distortion on a reference relation chain in the actual coding process is simulated, an expression of a distortion dependency item is reconstructed, and the rate distortion performance is improved.

The embodiment of the invention discloses a coding parameter determination method and device, equipment, a storage medium and a program product. The method comprises the steps that the complexity of a target image is acquired; inputting the complexity of the target image into a constant rate factor adjustment model to obtain a target constant rate factor corresponding to the target image; wherein the constant rate factor adjustment model is used for predicting a target slope of the target image in the rate distortion curve according to the complexity of the target image, and obtaining a target constant rate factor corresponding to the target image according to the target slope; if the complexity of the target image is less than or equal to the preset complexity, the target constant rate factor is greater than the initial constant rate factor; and coding the target image according to the target constant rate factor to obtain a coded target image corresponding to the target constant rate factor. According to the embodiment of the invention, the image coding rate can be reduced.

The invention provides a video coding method, which comprises the following steps of: inputting a coding tree unit to be coded, and extracting texture features of a coding block, the texture features comprising local binary pattern similarity of all pixel points in the coding block and direction dispersion of each pixel point; inputting the texture features into the trained multi-type division prediction architecture, and sequentially judging whether to execute quadtree division, horizontal or vertical division, horizontal binary tree or ternary tree division and vertical binary tree or ternary tree division or not through a binary classification problem method; and calculating the rate distortion cost under each division mode to obtain an optimal division mode and a suboptimal division mode of the coding block, and outputting an optimal mode or a combination of the optimal mode and the suboptimal mode. According to the invention, the accuracy of the prediction mode in coding is improved.

The present invention relates to image encoding and decoding of image regions in units of blocks. Specifically, a plurality of components of an input tensor, including a first component and a second component, are processed in a spatial dimension in a plurality of pipelines. The processing of the first component includes dividing the first component into a first plurality of tiles in the spatial dimension. Similarly, the processing of the second component includes dividing the second component into a second plurality of tiles in the spatial dimension. The respective first plurality of blocks and the second plurality of blocks are then processed, respectively. And at least two corresponding juxtaposed blocks with different sizes are arranged in the plurality of first blocks and the plurality of second blocks. In the case of compression, the processing of the first component and / or the second component includes image encoding, rate distortion optimized quantization, and image filtering. In the case of decompression, the processing includes image decoding and image filtering.

The present invention belongs to the field of image processing technology, and specifically relates to a zero-delay panoramic video rate control method considering the propagation of time domain distortion. The method of the present invention is mainly to optimize the target bit allocation and global rate distortion optimization coding, including rate control at the coding tree unit (CTU) level and time domain global rate distortion optimization. CTU-level rate control mainly includes optimizing target bit allocation and updating rate control parameters; time domain global rate distortion optimization mainly uses the reconstruction error and motion compensation prediction error information of the encoded frame to estimate the time domain dependency of each CTU in the current encoded frame and adjust the rate distortion optimization parameters of the CTU accordingly, and at the same time, according to the process of projecting the panoramic spherical graphic onto the 2D plane (ERP projection), the characteristics of oversampling and increasing pixel redundancy will be generated at the two poles, and the coding parameters are adjusted according to the projection area change ratio. The present invention can effectively shorten the encoding time and improve the video encoding quality under the condition of a given bandwidth.

Systems and methods directed to a rate-distortion optimization process. One example is directed to a video encoding device including an electronic processor. The electronic processor is configured to obtain motion information associated with a video block and determine, as a function of the motion information, a subset of coding tools from a plurality of coding tools for rate-distortion performance evaluation of the video block.

The invention relates to application-based rate distortion optimization in video compression. Systems and methods herein for a video encoder associated with an interface for receiving, from an application, at least one metric associated with a quality preference of video compression to be performed by the video encoder, a weight map is provided for the video encoder and for providing the weight map to enable the video encoder to perform rate distortion optimization (RDO) on frames received from the application using the weight map to weight one or more first blocks associated with a single one of the frames larger than one or more second blocks associated with the single one of the frames.

The invention discloses a standing wave angular rate error compensation method and device, equipment and a storage medium, and relates to the technical field of error compensation, and the method comprises the steps: obtaining a two-dimensional vibration equation corresponding to a harmonic oscillator; obtaining a target standing wave angular rate error formula corresponding to the harmonic oscillator based on the two-dimensional vibration equation, the driving electrode error corresponding to the harmonic oscillator and the detection electrode error; controlling the harmonic oscillator to rotate at different target rotating speeds, acquiring a change relation of a standing wave angular rate error corresponding to the harmonic oscillator along with each target rotating speed in the rotating process, and performing error compensation on the standing wave angular rate according to a target standing wave angular rate error formula and the change relation; wherein the standing wave angular rate error comprises an error generated when the harmonic oscillator rotates under the condition that the rotating speed is not smaller than the target rotating speed. The error generated when the harmonic oscillator rotates under the condition that the rotating speed is not smaller than the target rotating speed is compensated, so that the problem of distortion of the output angle or the angular rate is solved.

Processing circuitry determines to encode one or more blocks in a current picture using cross component sample offset (CCSO), and begins to search for combinations of values of filter parameters of the CCSO, each combination of values of filter parameters comprising a value for each filter parameter, respectively. The processing circuitry determines that a first rate distortion cost associated with at least a first combination of values satisfies a condition; in response to the first rate distortion cost satisfying a condition, terminating searching for at least a second combination of values of filter parameters of the CCSO; and selecting a value of a filter parameter for performing CCSO to encode one or more blocks in the current picture based on a value combination obtained from the search, the value combination obtained from the search not including a second combination of values.

A better rate distortion ratio is achieved by making interrelationships between coding parameters of different planes available for exploitation for the aim of redundancy reduction despite the additional overhead resulting from the need to signal the inter-plane prediction information to the decoder. In particular, the decision to use inter plane prediction or not may be performed for a plurality of planes individually. Additionally or alternatively, the decision may be done on a block basis considering one secondary plane.

Systems and methods for event and object detection and annotation in the video streams may include extracting a plurality of features in a picture in a video frame, grouping at least a portion of the plurality of features into at least one object, determining a region for the at least one object, assigning object identifiers to the at least one object and encoding the object identifiers into the bitstream. Feature-based rate distortion optimization may be employed for video coding including extracting a set of features from a picture in the video, generating a relevance map for the extracted features, determining a relevance score for portions of the picture using the relevance map, and encoding the portion of the picture with a bit rate determined at least in part by the relevance score.