6592 results about "Video encoding" patented technology

Systems and methods are described for video coding using generalized bi-prediction. In an exemplary embodiment, to code a current block of a video in a bitstream, a first reference block is selected from a first reference picture and a second reference block is selected from a second reference picture. Each reference block is associated with a weight, where the weight may be an arbitrary weight ranging, e.g., between 0 and 1. The current block is predicted using a weighted sum of the reference blocks. The weights may be selected from among a plurality of candidate weights. Candidate weights may be signaled in the bitstream or may be derived implicitly based on a template. Candidate weights may be pruned to avoid out-of-range or substantially duplicate candidate weights. Generalized bi-prediction may additionally be used in frame rate up conversion.

A method implemented by a video coding apparatus. The method includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample. The NN filter includes an NN filter model generated based on partitioning information of the video unit. Usage of the NN filter is indicated by one or more syntax elements in a bitstream. A conversion is performed between a video media file and a bitstream based on the filtered sample.

A method of encoding or decoding video data comprises: for each respective intra prediction mode of a plurality of intra prediction modes in a most-probable mode (MPM) list: generating, based on reference samples for a template region and using the respective intra prediction mode, prediction samples for the template region; and determining a cost for the respective intra prediction mode; determining a first intra prediction mode and a second intra prediction mode in the MPM list having lowest costs; determining a preliminary prediction block for the first intra prediction mode and a preliminary prediction block for the second intra prediction mode; generating a prediction block based on a fusion of the preliminary prediction blocks weighted according to a weight for the first intra prediction mode and a weight for the second intra prediction mode.

The invention discloses an e-commerce live broadcast real-time interaction quality evaluation system based on edge calculation, and relates to the technical field of e-commerce live broadcast, and the system comprises a multi-modal interaction data collection module which collects multi-modal interaction data of a live broadcast stream in real time through a distributed edge calculation node cluster, and constructs a multi-dimensional quality feature vector, the multi-modal interaction data comprises a video coding parameter, an audio quality index, user interaction behavior data and network transmission state data; according to the invention, the distributed edge computing node cluster collects the multi-modal interaction data of the live stream in real time, the lightweight space-time attention neural network model carries out data fusion processing, the deep reinforcement learning network generates a quality optimization scheme, and the adaptive fuzzy inference system corrects the optimization scheme in real time. And dynamic parameter adjustment is carried out in combination with network bandwidth fluctuation and a terminal device resource state, so that the effect of accurately and comprehensively evaluating the e-commerce live broadcast interaction quality in real time is achieved.

The invention relates to the technical field of video coding, in particular to a video coding method and system based on multi-channel concurrent software and hardware mixing, and the method comprises the following steps: analyzing brightness distribution and edge structures, screening effective frames, calculating frame priorities, dividing image blocks and combining processing units, counting the number of frame streams, and analyzing frequency concentration. And evaluating image coherence and a load state, and outputting a processing channel adjustment record. According to the invention, through screening of image content features, occupation of redundant frames on coding resources is reduced, through dynamic configuration of frame priorities, timeliness and accuracy of target frame scheduling are improved, an aggregation strategy of a block structure is combined, picture consistency after image processing is enhanced, and task frequency trend identification is utilized, so that image processing efficiency is improved. According to the method, accurate resource matching of a high-load flow section is achieved, joint judgment of image boundary continuity and structure hopping frequency is adopted, an allocation strategy of a processing channel is optimized, an image structure and channel scheduling form linkage, and stability and adaptive capacity are improved.

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.

The embodiment of the invention discloses a method and a device for generating a voice-driven digital human video, which are used for improving the generation quality of the voice-driven digital human video. The method comprises the following steps: acquiring driving voice and a figure reference image; the driving voice and the figure reference image are input into a digital human video generation model, continuous video frames are obtained, and the digital human video generation model is obtained through training of a training sample constructed based on the figure reference image and audio of a figure video and a training label constructed based on the continuous video frames of the figure video; and performing audio and video coding on the driving voice and the continuous video frames to obtain a digital human video.

The invention discloses an MVPS video coding method and device based on deep learning, and a medium, and relates to the technical field of video coding. The method comprises the following steps: acquiring a video stream, performing time domain and space domain feature extraction on the video stream through a feature extraction unit, and generating a joint feature map through an adaptive weight fusion layer; carrying out bidirectional feature interaction between hierarchies on the combined feature map to obtain a multi-scale feature set; generating a frame-level quantization parameter initial value according to the multi-scale feature set in combination with current network bandwidth feedback; performing time sequence consistency correction on the quantization parameter initial value to generate a target quantization parameter; and performing block coding on the current frame according to the target quantization parameter to generate a coded data stream. According to the method, through dynamic parameter adjustment, hardware algorithm collaborative optimization, cross-frame consistency enhancement and network awareness, the problems of low efficiency, insufficient real-time performance, unstable quality and poor adaptability of a traditional scheme are solved.

A method comprising: receiving an image block unit of a frame, the image block unit comprising samples in color channels comprising at least one chrominance channel and one luminance channel (900); reconstructing samples of said luminance channel of the image block unit (902); determining a reference area for predicting samples of at least one color channel of the image block unit, wherein said reference area comprises one or more of reference samples in a neighboring block in current color channel / frame, in the neighboring of a co-located block in reference color channel / frame; and / or inside the co-located block in reference color channel / frame (904); determining filter coefficients of a filter for said predicting based on the reference samples and a shape of the filter (906); and predicting samples of at least one color channel of the image block unit by performing a convolution between the samples of said luminance channel and the filter coefficients (908).

Embodiments of the present disclosure relate to the field of video coding and decoding, and provide a picture decoding method, a picture coding method, and corresponding apparatuses. According to an example of the picture decoding method, a quantization parameter (QP) value of at least one pixel point is determined for any at least one pixel point indicating one pixel point or a plurality of parallel dequantized pixel points in the current coding block, the QP values of at least two pixel points among pixel points in the current coding block being different. The at least one pixel point is then dequantized according to the QP value of the at least one pixel point. The QP value of each pixel point is determined for the coding block; therefore, each pixel point can be dequantized according to the QP value of each pixel point.

Systems, methods, and instrumentalities are disclosed for template based cross component linear model / multimode linear model (CCLM / MMLM) adjustment. In an example, a device, such as a video decoding device, or a video encoding device, may obtain a prediction model for predicting a coding block. The device may select an adjustment model, from multiple adjustment models, for adjusting the prediction model. The device may adjust the prediction model based on the selected adjustment model. The device may process (e.g., encode and / or decode) the coding block based on the adjusted prediction model.

The invention relates to the technical field of video coding and decoding, and discloses a remote desktop video coding and decoding method based on a neural network multi-head attention mechanism. Comprising the following steps: dividing a screen of a remote desktop into a plurality of areas by utilizing a multi-head attention mechanism; performing grading identification according to the activeness and the importance degree of each area in the screen; the encoder performs encoding sorting on each area according to the grading identifier, formulates an encoding strategy, and dynamically adjusts the encoding strategy based on a feedback mechanism; generating a coded video stream according to a formulated coding strategy, and extracting attached information, related to original coded data, of each region; synchronously sending the generated coded video stream and the attached information to a receiving end; and the decoder decodes each area according to the coding sequence, original coding data is recovered, and the remote desktop picture is recovered. The multi-head attention mechanism is utilized to perform intelligent grading and associated coding on the screen area, and the problems of low efficiency, redundant transmission and high bandwidth occupation in a traditional coding and decoding method are solved.

The invention provides an image processing method, processing equipment and a storage medium. The image processing method comprises the following steps: determining or generating a target image block according to a reference parameter of at least one image block and a filtering lookup table; according to the technical scheme, when the filtering lookup table is adopted for filtering processing, the complexity of video coding and / or decoding can be reduced through the technical scheme, and then the efficiency of video coding and / or decoding is improved.

The invention discloses an extreme compressed video coding method based on an intelligent reference frame, comprising the following steps: S1, acquiring an original video stream, dividing the original video stream into frame groups according to a time sequence, each group comprising a current frame and a candidate reference frame; s2, carrying out region positioning on the candidate reference frame, extracting motion, edge and background regions, and generating a mapping graph; s3, calculating a quality score based on the mapping graph according to a motion vector, gradient change and a region overlapping rate; s4, selecting three frames with highest scores to form a reference set, and establishing an index structure; s5, predicting the current frame by using the reference set to generate a predicted frame and a residual error; s6, multi-path coding cost is calculated, and a path with the minimum cost is selected; and S7, entropy coding is carried out on the reference index, the motion vector, the residual error and the control parameter, a code stream is output, and calling information is recorded. According to the method, the compression ratio and prediction precision of video coding are improved, the image quality is maintained while the code rate is reduced, and the method is suitable for efficient transmission and storage of high-resolution videos.

The embodiment of the invention provides a video generation method and device, equipment and a medium. The method comprises the following steps: acquiring a source video, a camera track parameter and a video description text; converting the source video into point cloud data, performing point cloud rendering processing on the point cloud data according to the camera track parameters to obtain a point cloud rendering result, and encoding the point cloud rendering result into view encoding features; encoding a source video into a reference encoding feature, and encoding a video description text into a text encoding feature; converting the view coding feature into a first self-attention feature, and converting the text coding feature into a second self-attention feature; performing cross attention processing on the first self-attention feature and the reference coding feature to obtain a video interaction feature, and obtaining a multi-modal coding feature according to the video interaction feature and the second self-attention feature; and carrying out decoding processing on the multi-modal coding features to generate a target video. By implementing the embodiment of the invention, the generation quality of the target video can be improved.

A video coding system is provided that performs intra prediction in a mode using a neural network for block of only a set of specific block sizes. The signaling of this mode is designed to be efficient in terms of rate-distortion under this constraint. Different transformations of the context of a block and the neural network prediction of this block are introduced in order to use one single neural network for predicting blocks of several sizes, as well as the corresponding signaling. The neural network-based prediction mode considers both luminance blocks and chrominance blocks. The video coding system comprises encoder and decoder apparatuses, encoding, decoding and signal generation methods and a signal carrying information corresponding to the described coding mode.

An example device for decoding video data includes a memory configured to store video data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: generate a prediction block for a current block of video data using a neural network; determine an equivalent intra mode for the prediction block from a set of available intra-prediction modes using decoder-side intra mode derivation (DIMD), the equivalent intra mode representing one of the intra-prediction modes that would generate an intra-prediction block that would best match the prediction block generated using the neural network; decode a residual block for the current block of the video data based on the equivalent intra mode; and combine the prediction block with the residual block to decode the current block of the video data.

A method comprising: receiving an image block unit of a frame, the image block unit comprising samples in color channels comprising at least one chrominance channel and one luminance channel; reconstructing samples of said luminance channel of the image block unit; determining a reference area for predicting target samples of at least one color channel of the image block unit, wherein said reference area comprises one or more reference samples in a neighboring block in current color channel / frame, in the neighboring of a co-located block in reference color channel / frame; and / or inside the co-located block in reference color channel / frame; determining weights for predicting said target samples based on a ratio between a normalized luminance histogram in said reference area and a normalized luminance histogram co-locating said target samples; determining filter coefficients of a filter for said predicting based on the weights, the reference samples and a shape of the filter; and predicting samples of at least one color channel of the image block unit based on the samples of said luminance channel and the filter coefficients.

A video encoder and video decoder may determine to code a block of video data using a non-directional intra mode. The video encoder and video decoder may determine a secondary transform from a secondary transform set for the block, and perform a transform process on the block of video data using the secondary transform. For non-directional intra prediction modes, the secondary transform set has a first number of transform kernels that is less than a second number of transform kernels in the secondary transform set for other blocks coded using a directional intra mode.

A method for video decoding is provided. The method includes obtaining, from a bitstream, a coding unit in a current picture, the coding unit including a luma block and a chroma block; obtaining a reconstructed luma sample in the luma block; determining one or more cross-component prediction models based upon a block size, the block size being a size of the luma block or a size of a reconstructed neighbouring block located on a top of or a left of the luma block; and applying at least one of the one or more cross-component prediction models to at least the reconstructed luma sample to predict a chroma sample in the chroma block.

There is provided a method for video encoding and decoding based on a picture of a video and a supplementary enhancement information (SEI) message which is associated with the at least one picture and indicates one or more parameters of an optical correction to be applied to the at least one picture.

A method and an apparatus are disclosed for video coding using an improved inloop filter for chroma components. In the disclosed embodiments, a video decoding device performs obtaining a reconstructed frame that includes luma samples and chroma samples, subsequently obtaining a scale value representing a resolution difference between the reconstructed frame and an original frame. Responsive to the reconstructed frame having a resolution that is based on the scale value and is less than a resolution of the original frame, the video decoding device generates an upsampled chroma frame by inputting the luma samples and the chroma samples into a cross component resampling inloop filter (CC-RIF).

The application provides a signal transmission method, a video encoder and a computer readable storage medium. A computer-implemented signal transmission method performed by an encoder comprises the following steps: a processor transmits a bitstream comprising weight information for a prediction coding unit (CU) to a video decoder, the weight information indicating that if weighted prediction is enabled for a bi-prediction mode of the CU, weighted averaging for the bi-prediction mode is disabled, and the weight information indicating that if weighted prediction is enabled for at least one of a luma component and a chroma component of a reference picture of the CU, weighted averaging for the bi-prediction mode is disabled, wherein the bitstream comprises a flag indicating whether weighted prediction is enabled for at least one of the luma component and the chroma component of the reference picture, and the flag comprises a flag luma_weight_lx_flag[i] transmitted for an i-th reference picture in a reference picture list Lx, wherein x is 0 or 1.

A non-uniform video encoder system for generating a multi-depth encoding data for a scene is provided. The non-uniform video encoder system is configured to receive a sequence of video frames of a video of the scene and transform the sequence of video frames into series input data. The series input data is analyzed to identify changes in the evolution of the scene, by partitioning the series input data into a sequence of non-uniform segments. Each segment in the sequence of non-uniform segments is encoded by an encoder of an autoencoder architecture with non-uniform unrolling recursion to produce multi-depth encoding of the series input data. To encode a current segment at a current iteration to produce a current encoding, the non-uniform unrolling recursion combines the current segment with a previous encoding produced at a previous iteration and encodes the combination with the encoder.

The invention relates to the technical field of video monitoring communication, in particular to a 5G high-definition video monitoring method and system for a smart city, and a medium, and the method specifically comprises the steps: obtaining a 5G high-definition video based on the moving features of each feature point of a moving target between each frame and a previous frame image in a monitoring video, and the gray gradient change condition in each channel image of an RGB image of each video frame; constructing an information rich characteristic value of each video frame; calculating a bandwidth influence coefficient of each detection time period based on a data fluctuation abnormal condition of the network bandwidth data in each preset detection time period; constructing a video transmission quality coefficient by combining the features, optimizing a quantization parameter of a video coding algorithm in the current detection time period, carrying out coding transmission on the monitoring video in the current detection time period, and carrying out intelligent analysis on the transmitted monitoring video; the problem of frame loss or low video quality of video content with relatively high potential information value is reduced; and the interference on subsequent video monitoring intelligent analysis and target detection can be reduced.

The invention provides a cloud rendering technology-based cloud CAD online cooperation system and method. The cloud rendering technology-based cloud CAD online cooperation system comprises a front end, a rear end and a network communication module, the rear end comprises a storage layer, a CAD core layer and a view layer, and the CAD core layer processes and updates CAD file data in the storage layer in real time based on any front-end data and renders the CAD file data to a video memory cache. The view layer comprises view sets in one-to-one correspondence with the front ends, any view set comprises a viewport layout module and a video coding module, and the CAD core layer realizes real-time synchronous operation of any front end on the same CAD file data and real-time synchronous operation of any front end on the same CAD file data video memory cache data by sharing the video memory cache. And the information is pushed to all front ends through the network communication module. The method has the advantages that multiple users are supported to edit the same CAD file online at the same time, and therefore interactive free modeling and parametric modeling are completed in a multi-user online cooperation scene.

A video encoding method, a video decoding method and apparatus, and a storage medium are provided. The decoding method includes that: angular accuracy corresponding to the intra prediction mode candidate list is determined, the first angular accuracy is used for indicating a search range of an angular prediction mode in the intra-frame prediction mode candidate list; the intra prediction mode candidate list is constructed on the basis of the first angular accuracy; and the prediction for the current block is performed on the basis of the constructed intra prediction mode candidate list.

Devices, systems, and methods for digital video coding, which includes matrix-based intra prediction methods for video coding, are described. In a representative aspect, a method for video processing includes performing a conversion between a current video block of a video and a bitstream representation of the current video block according to a rule, where the rule specifies a relationship between applicability of a matrix based intra prediction (MIP) mode or a transform mode during the conversion, where the MIP mode includes determining a prediction block of the current video block by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation, and where the transform mode specifies use of a transform operation for the determining the prediction block for the current video block.

The method includes receiving an image block unit of a frame, the image block unit including samples in a first chrominance channel, a second chrominance channel, and one luminance channel; defining a co-located reference area across a luminance component of the luminance channel, a chrominance component of the first chrominance channel, and a chrominance component of the second chrominance channel; performing motion compensation prediction of the luminance component, the first chrominance component, and the second chrominance component using the reference frame and a motion vector; obtaining a first mapping function that maps the luminance component to the first chrominance component and a second mapping function that maps the luminance component to the second chrominance component using the co-located luminance prediction and chrominance prediction; reconstructing a luminance residual; and obtaining a first chrominance prediction by using the first function and the luminance residual and obtaining a second chrominance prediction by using the second function and the luminance residual.

A video encoding or decoding apparatus and method perform existing inter prediction on a current block to generate a motion vector and predicted samples. The video encoding or decoding apparatus and method generate enhanced predicted samples for the current block using a deep learning-based video prediction network (VPN) on the basis of the motion vector, the reference samples, the predicted samples, and the like to improve encoding efficiency.