Intelligently skipping coding of video blocks to reduce latency

Abstract

Techniques are described for training (302) a machine learning (ML) model (402) to determine whether to encode a video frame, e.g., based on bit rate capacity. If a frame is discarded according to the output of the ML model (508), the next frame may be encoded with a higher quality than normal using the computational savings obtained from discarding the first frame. Compressed domain information may be used to cancel portions of a frame from encoding, but not the entire frame.

Description

Intelligently skip encoding of video blocks to reduce latency. Technical Field

[0001] This application relates to a technically inventive and unconventional solution that is necessarily rooted in computer technology and produces specific technical improvements, and more specifically relates to intelligently skipping the encoding of video blocks to reduce latency. Background Technology

[0002] Computer games typically take place over a network, where the game itself can be streamed, and multiple players, potentially geographically separated, can play the same game. Streaming games presents technical challenges related to energy usage and network limitations. For example, it may impose stringent energy demands on network computing, limiting the network's data-carrying capacity. Furthermore, the massive amounts of data that may be attempted to be sent over the network can overwhelm its capacity, especially in the so-called "last mile." These limitations generate several problems, including reduced bandwidth and increased latency. Summary of the Invention

[0003] As understood in this article, data transmission load can be reduced under certain conditions by sending fewer keyframes (e.g., I-frames or portions thereof) than the standard number (where network conditions may dictate this).

[0004] Therefore, in a first aspect, an apparatus includes at least one processor component configured to input at least a first frame of a video into at least one machine learning (ML) model, and to not encode the first frame in response to an output indication from the ML model that the first frame should not be encoded. The processor component is configured to encode the first frame in response to an output indication from the ML model for transmission to a receiver.

[0005] The video can include computer games, and the first frame can include a keyframe.

[0006] In an example embodiment, the processor component may be configured to encode a first frame in response to an output indication from the ML model, encode the first frame at a first quality for transmission to the receiver, and, in response to an output indication from the ML model, not encode the first frame, and encode a second frame at a second quality higher than the first quality.

[0007] In an example implementation, the decoder component may include a decoder ML model configured to receive frames from the decoder of the decoder component and insert reconstructed frames between the frames received from the decoder. The decoder ML model may be trained on a training set comprising a sequence of video frames at a first frame rate and missing ground truth frames in the frame sequence.

[0008] If needed, the processor component can be configured to use compression domain information in the video to remove portions of frames from the encoding.

[0009] In another aspect, the apparatus includes at least one computer medium that is not a transient signal, and further includes instructions executable by at least one processor component to input at least one first frame of video into at least one machine learning (ML) model. The instructions are executable to encode a first pixel region of the first frame in response to the output from the ML model, and not to encode a second pixel region of the first frame. The instructions may be executable to transmit the first portion after encoding, and not to encode the second portion or not to transmit the second portion to a receiver.

[0010] In another approach, one method involves invoking at least one machine learning (ML) model to process the video frame before encoding it. This method includes providing the ML model with one or both of pixel domain and / or compression domain information. The method also includes, at least partially based on the output of the ML model, not encoding at least a first frame, and / or encoding only a first pixel region of a second frame without encoding a second pixel region of the second frame, for provision to at least one receiver via a network.

[0011] The details of this disclosure regarding its structure and operation can be best understood with reference to the accompanying drawings, wherein like reference numerals denote like parts, and wherein: Attached Figure Description

[0012] Figure 1 is a block diagram of an example system that includes examples consistent with this principle;

[0013] Figure 2 illustrates an example encoder-decoder system;

[0014] Figure 3 shows an example training logic in an example flowchart format for an encoder-side ML model;

[0015] Figure 4 shows an example system for intelligently discarding frames from the encoding using an ML model trained according to Figure 3;

[0016] Figure 5 shows an example training logic in the same example flowchart format as Figure 4;

[0017] Figure 6 shows an example training logic in an example flowchart format for a decoder-side ML model;

[0018] Figure 7 shows an example system for reconstructing lost frames using an ML model trained according to Figure 6;

[0019] Figure 8 shows an alternative example training logic in the format of an example flowchart of an encoder-side ML model using compressed domain information;

[0020] Figure 9 shows a frame consistent with Figure 8;

[0021] Figure 10 shows an alternative example training logic in an example flowchart format for a decoder-side ML model using compressed domain information; and

[0022] Figure 11 shows an example system for reconstructing discarded portions of a frame using an ML model trained according to Figure 10. Detailed Implementation

[0023] This disclosure generally relates to the computer ecosystem, including various aspects of consumer electronics (CE) device networks, such as, but not limited to, computer gaming networks. Systems described herein may include server and client components that can be networked, enabling the exchange of data between the client and server components. Client components may include one or more computing devices, including game consoles such as the Sony PlayStation® or game consoles manufactured by Microsoft, Nintendo, or other manufacturers; extended reality (XR) headsets such as virtual reality (VR) headsets; augmented reality (AR) headsets; portable televisions (e.g., smart TVs, internet-enabled televisions); portable computers such as laptops and tablets; and other mobile devices including smartphones and additional examples discussed below. These client devices may operate with a variety of operating environments. For example, by way of example, some client computers may run on a Linux operating system, an operating system from Microsoft, or a Unix operating system, or an operating system manufactured by Apple, Inc., or Google, or a Berkeley Software distribution or Berkeley Standard Distribution (BSD) OS (including descendants of BSD). These operating environments can be used to execute one or more browsing programs, such as browsers made by Microsoft, Google, or Mozilla, or other browser programs that can access websites hosted by the internet servers discussed below. Furthermore, operating environments based on this principle can be used to execute one or more computer game programs.

[0024] A server and / or gateway may be used, which may include one or more processors executing instructions that configure the server to receive and send data over a network such as the Internet. Alternatively, the client and server may connect via a local intranet or virtual private network. The server or controller may be instantiated from a game console such as a Sony PlayStation®, a personal computer, etc.

[0025] Information can be exchanged between clients and servers over a network. For this purpose, and for security, servers and / or clients may include firewalls, load balancers, temporary storage, and proxies, as well as other network infrastructure for reliability and security. One or more servers can form a means of implementing methods to provide network members with a secure community, such as an online social networking site or a gaming network.

[0026] A processor can be a single-chip or multi-chip processor that executes logic via various lines such as address lines, data lines, and control lines, as well as registers and shift registers. A processor that includes a digital signal processor (DSP) can be an embodiment of a circuit. A processor assembly can include one or more processors.

[0027] Components included in one embodiment may be used in other embodiments in any suitable combination. For example, any of the various components described herein and / or depicted in the accompanying drawings may be combined, interchanged, or excluded from other embodiments.

[0028] "A system having at least one of A, B and C" (similarly, "a system having at least one of A, B or C" and "a system having at least one of A, B, and C") includes systems having only A, only B, only C, A and B together, A and C together, B and C together and / or A, B and C together.

[0029] Referring now to Figure 1, an example system 10 is shown, which may include one or more of the example devices mentioned above and further described below according to this principle. A first example device among the example devices included in system 10 is a consumer electronics (CE) device, such as an audio-visual device (AVD) 12, such as, but not limited to, a projector-based theater display system, or an internet-enabled TV with a TV tuner (equivalently, a set-top box controlling the TV). The AVD 12 may alternatively also be a computerized internet-enabled (“smart”) phone, tablet computer, laptop computer, head-mounted device (HMD) and / or headset (such as smart glasses or VR headsets), another wearable computerized device, a computerized internet-enabled music player, a computerized internet-enabled headset, a computerized internet-enabled implantable device (such as an implantable skin device), etc. In any case, it should be understood that the AVD 12 is configured to take this principle (e.g., communicate with other CE devices to take this principle, perform the logic described herein, and perform any other functions and / or operations described herein).

[0030] Therefore, to implement this principle, the AVD 12 can be constructed from some or all of the components shown. For example, the AVD 12 may include one or more touch-enabled displays 14, which may be implemented using a high-definition or ultra-high-definition "4K" or higher flat panel screen. The touch-enabled displays 14 may include, for example, a capacitive or resistive touch sensing layer having an electrode grid for touch sensing consistent with this principle.

[0031] AVD 12 may also include one or more speakers 16 for outputting audio according to these principles, and at least one additional input device 18, such as an audio receiver / microphone, for inputting audible commands to control AVD 12. The exemplary AVD 12 may also include one or more network interfaces 20 for communicating over at least one network 22 (such as the Internet, WAN, LAN, etc.) under the control of one or more processors 24. Thus, interface 20 may be, but is not limited to, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that processor 24 controls AVD 12 to perform these principles, including other elements of AVD 12 described herein, such as controlling display 14 to present images thereon and receive input therefrom. Furthermore, note that network interface 20 may be a wired or wireless modem or router, or other suitable interface, such as a wireless telephone transceiver or a Wi-Fi transceiver as described above.

[0032] In addition to the above, the AVD 12 may also include one or more input and / or output ports 26, such as an HDMI port or a USB port, for physical connection to another CE device and / or a headphone port for connecting headphones to the AVD 12 to present audio from the AVD 12 to the user via headphones. For example, input port 26 may be connected via a cable or satellite source 26a to audio / video content, either wired or wirelessly. Therefore, source 26a may be a standalone or integrated set-top box or satellite receiver. Alternatively, source 26a may be a game console or disc player containing content. When implemented as a game console, source 26a may include some or all of the components described below with respect to CE device 48.

[0033] AVD 12 may also include one or more computer memory / computer-readable storage media 28, such as non-transient signal disk storage or solid-state storage, which in some cases are embodied in the chassis of the AVD as a standalone device or personal video recording device (PVR) or video disk player, located inside or outside the AVD chassis for playing AV programs, or as a removable storage medium or a server as described below. Furthermore, in some embodiments, AVD 12 may include a location or place receiver, such as, but not limited to, a mobile phone receiver, a GPS receiver, and / or an altimeter 30, configured to receive geolocation information from a satellite or mobile phone base station and provide that information to the processor 24 and / or determine the height at which the AVD 12 is placed with the processor 24.

[0034] Continuing the description of AVD 12, in some embodiments, AVD 12 may include one or more cameras 32, which may be thermal imaging cameras, digital cameras such as webcams, IR sensors, event-based sensors, and / or cameras integrated into AVD 12 and controllable by processor 24 to collect pictures / images and / or videos in accordance with these principles. AVD 12 may also include a Bluetooth® transceiver 34 and other near-field communication (NFC) elements 36 for communicating with other devices using Bluetooth and / or NFC technologies, respectively. An example NFC element may be a radio frequency identification (RFID) element.

[0035] Furthermore, the AVD 12 may include one or more auxiliary sensors 38 that provide input to the processor 24. For example, one or more of the auxiliary sensors 38 may include one or more pressure sensors forming a layer of the touch-enabled display 14 itself, and may be, but are not limited to, piezoelectric pressure sensors, capacitive pressure sensors, piezoresistive strain gauges, optical pressure sensors, electromagnetic pressure sensors, etc. Other sensor examples include pressure sensors, motion sensors (such as accelerometers, gyroscopes, tachometers) or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or rhythm sensors, event-based sensors, and gesture sensors (e.g., for sensing gesture commands). Thus, the sensors 38 may be implemented by one or more motion sensors, such as individual accelerometers, gyroscopes, and magnetometers, and / or typically include an inertial measurement unit (IMU) comprising a combination of accelerometers, gyroscopes, and magnetometers, to determine the position and orientation of the AVD 12 in three dimensions, or by event-based sensors, such as event detection sensors (EDS). Consistent with this disclosure, the EDS provides an output indicating changes in light intensity sensed by at least one pixel of the light-sensing array. For example, if the light sensed by the pixel is decreasing, the output of EDS can be -1; if it is increasing, the output of EDS can be +1. No change in light intensity below a certain threshold can be indicated by an output binary signal of 0.

[0036] The AVD 12 may also include an over-the-air (OTA) television broadcast port 40 for receiving OTA television broadcasts that provide input to the processor 24. In addition to the foregoing, it should be noted that the AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided to power the AVD 12, such as a kinetic energy harvester that can convert kinetic energy into electrical energy to charge the battery and / or power the AVD 12. A graphics processing unit (GPU) 44 and a field-programmable gate array (FPGA) 46 may also be included. One or more tactile / vibration generators 47 may be provided to generate tactile signals that can be felt by a person holding or touching the device. Therefore, the haptic generator 47 can use an electric motor to vibrate all or part of the AVD 12, which is connected to an eccentric and / or unbalanced weight via a rotatable shaft, such that the shaft can be rotated under the control of the motor (which can in turn be controlled by a processor (e.g., processor 24)) to generate vibrations of various frequencies and / or amplitudes as well as force simulations of various directions.

[0037] It may also include a light source such as a projector (such as an infrared (IR) projector).

[0038] In addition to AVD 12, System 10 may include one or more other CE device types. In one example, the first CE device 48 may be a computer game console that can be used to send computer game audio and video to AVD 12 via commands sent directly to AVD 12 and / or via a server described below, while the second CE device 50 may include components similar to the first CE device 48. In the example shown, the second CE device 50 may be configured as a computer game controller operated by a player or a head-mounted display (HMD) worn by a player. The HMD may include a heads-up transparent or opaque display for presenting AR / MR content or VR content (more generally, extended reality (XR) content), respectively. The HMD may be configured as a glasses-type display sold by a computer game device manufacturer or a larger VR-type display.

[0039] In the example shown, only two CE devices are illustrated; it should be understood that fewer or more devices may be used. The devices described herein may implement some or all of the components shown for AVD 12. Any components shown in the following figures may be combined with some or all of the components shown in the case of AVD 12.

[0040] Referring now to at least one server 52 described above, it includes at least one server processor 54, at least one tangible computer-readable storage medium 56 (such as a disk-based or solid-state storage device), and at least one network interface 58, which, under the control of the server processor 54, allows communication with other illustrated devices via network 22 and, in practice, facilitates communication between server and client devices according to this principle. Note that the network interface 58 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface, such as a wireless telephone transceiver.

[0041] Therefore, in some embodiments, server 52 may be an internet server or an entire server "farm" and may include and perform "cloud" functionality, enabling devices of system 10 to access a "cloud" environment via server 52 in example embodiments for, for example, online gaming applications. Alternatively, server 52 may be implemented by one or more game consoles or other computers in the same room as other devices shown or nearby.

[0042] The components shown in the following figures may include some or all of the components shown herein. Any user interface (UI) described herein may be combined and / or extended, and UI elements may be mixed and matched between UIs.

[0043] This principle can be applied to various machine learning models, including deep learning models. Machine learning models consistent with this principle can be trained using a variety of algorithms, including supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms that can be implemented by computer circuits include one or more neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and RNN types known as long short-term memory (LSTM) networks. Generative pre-trained transformers (GPTTs) can also be used. Support vector machines (SVMs) and Bayesian networks can also be considered examples of machine learning models. In addition to the network types described above, the models in this paper can be implemented using classifiers.

[0044] As understood in this paper, performing machine learning can therefore involve accessing and then training a model on training data so that the model can process further data to make inferences. Thus, an artificial neural network / AI model trained via machine learning can include an input layer, an output layer, and multiple hidden layers in between, which are configured and weighted to make inferences about the appropriate output.

[0045] Figure 2 illustrates a system including a video encoder 200 for encoding / compressing video 202. A video decoder 204 can receive encoded video and decode / decompress it into output video 206.

[0046] Figure 3 illustrates an example technique for training a machine learning (ML) model consistent with the disclosures herein. Box 300 indicates that some data is fed into the ML model at box 302 to train the ML model. The training data includes multiple videos, such as computer game videos, each labeled with a ground truth classification. For example, one video could be labeled "shooting game," another "driving game," and a third "e-sports." The videos can be labeled using internal feature classifications such as "high motion," "low motion," "large color distribution," and "small color distribution."

[0047] In addition, accompanying the video may be indications of the encoding parameters used to encode the video, including bitrate, frame rate, and resolution. Training data may include ground truth indicators based on subjective metrics such as whether frames can be dropped from the encoding, representing whether (or not) an average human viewer can notice differences in bitrate and / or frame rate and / or resolution variations in the video. One such index that can be used is the Mean Opinion Score (MOS), which can be considered a subjective measure of video quality.

[0048] An example goal of training an ML model is to train the model to decide whether to drop frames (or portions thereof) from the encoding before reaching a minimum perceptible difference (JND) threshold at which a human would notice a degradation in video quality. In this way, power consumption and bandwidth consumption are reduced without noticeable degradation when watching the video.

[0049] Referring now to Figure 4. The input frames of video 400 (such as frames of a computer game video to be streamed over a network to a receiving player's system) are analyzed by ML model 402 (such as an ML model trained according to Figure 3) to determine whether to discard (404) frames from encoding or to pass the frames to encoder 406 for encoding. The frames analyzed by ML model 402 may be only keyframes of the video or all frames (including P-frames). Therefore, given that ML model 402 will select some frames to be eliminated from encoding, only some (not all) keyframes of the video may be encoded and provided to transmitter 408 for transmission over network 410 to receiver 412 for display on display 414.

[0050] Therefore, when encoder 406 encounters, for example, insufficient bitrate capacity, it can invoke ML model 402 to selectively discard frames from the encoding. This may result in lower and / or variable frame rates. For example, if the input frame rate is 60 FPS, the frame rate may be reduced to 48 FPS or 30 FPS after processing by the ML model, where the end user viewing display 414 will hardly notice a discernible difference.

[0051] Figure 5 illustrates the same logic as Figure 4. Starting at state 500, video frames, such as keyframes (e.g., I-frames), are input into the ML model. At state 502, the ML model determines whether the frame should be encoded based on the model's training. If the frame is to be encoded, it is encoded at state 504 and sent to the receiver at state 506.

[0052] However, if the frame is not encoded, it is discarded in state 508, thus saving bandwidth. Alternatively, the next frame to be encoded can be encoded at a higher quality than if the bandwidth and processing savings from discarding the previous frame were used. For example, a 2K resolution frame can be encoded at 4K resolution using the bandwidth and processing savings from discarding the previous frame.

[0053] On the receiver side, the decoder can use machine learning to regenerate the lost frames, or simply reduce the decoding to the lower frame rate of the received frames. Figures 6 and 7 illustrate the former technique.

[0054] By inputting the training dataset into the model at state 600 in Figure 6 and training the model on the training set at state 602, the ML model is trained to regenerate the frames discarded by the encoder in Figures 4 and 5. The training set may include video frame sequences with relatively low frame rates as well as ground truth frames missing from the missing frame sequences, which have been removed from the encoding, consistent with the principles of this paper.

[0055] Figure 7 illustrates that after training the decoder ML model, video 700, such as computer game video, can be received and decoded by decoder 702. The decoded video is passed to ML model 704 for reconstructing frames discarded by the encoder. The frame sequence decoded by decoder 702 and the frame sequence reconstructed using ML model 704 and inserted into the video stream can be displayed on display 706.

[0056] The techniques used to remove video content from the encoding shown in Figures 3 through 5 are pixel-based. Now turn to Figure 8 to understand additional and / or alternative techniques for removing video content from the encoding using compression domain information (e.g., motion vectors) in the video to save bandwidth.

[0057] Starting at state 800, the training dataset is fed into the ML model to train the model at state 802. The training data may include complete frames with accompanying compressed domain information such as motion vectors, as well as ground truth frames that can be output based on accompanying compressed domain information such as motion vectors. In this way, for example, portions of frames characterized by motion vectors indicating little or no motion can be omitted from the encoding of the ML model.

[0058] Figure 9 shows an example output of the ML model trained according to Figure 8. Pixel domain information 900 of the received frame is used. Based on the frame-associated compression domain information 902, the first region 904 of the frame's pixel domain is marked as encoded, while the second region 906 of the frame's pixel domain is marked as discarded from encoding. Therefore, only a small portion (less than 100% but greater than 0%) of the frame, obtained by dividing the number of pixels in the encoded region 904 by the total number of pixels in the encoded region 904 and the non-coded region 906, is encoded.

[0059] Figures 10 and 11 illustrate decoder-side features consistent with Figures 8 and 9. Starting at state 1000, the training dataset is fed into the ML model to train the model at state 1002. The training data may include incomplete frames of pixel data with accompanying compressed domain information such as motion vectors, as well as ground truth complete frames that can be output based on the accompanying compressed domain information. In this way, for example, the omitted parts of the frame can be reconstructed after decoding other parts of the frame using the ML model and adding them to the decoded parts.

[0060] Figure 11 shows that a portion of the video frame is received at state 1100 and decoded by decoder 1102. The ML model 1104 trained according to Figure 10 can reconstruct the missing portion of the decoded frame using the accompanying compression domain information based on the training of the ML model. Then, the two parts of the frame (decoded by decoder 1102 and reconstructed by ML model 1104) can be presented together on display 1106.

[0061] While specific techniques are shown and described in detail herein, it should be understood that the subject matter covered by this application is limited only by the claims.

Claims

1. An apparatus comprising: At least one processor component is configured to: input at least a first frame of the video into at least one machine learning (ML) model; In response to an output indication from the ML model that the first frame should not be encoded, the first frame is not encoded. And in response to an output indication from the ML model that the first frame should be encoded, the first frame is encoded for transmission to the receiver.

2. The apparatus according to claim 1, wherein, The video includes at least one computer game.

3. The apparatus according to claim 1, wherein, The processor component is configured to: in response to an output indication from the ML model that the first frame should be encoded, encode the first frame with a first quality for transmission to the receiver; And in response to an output indication from the ML model that the first frame should not be encoded, the first frame is not encoded, and the second frame is encoded with a second quality higher than the first quality.

4. The apparatus according to claim 1, wherein, The first frame includes a keyframe.

5. The apparatus of claim 1, comprising at least one decoder component, the at least one decoder component including at least one decoder ML model, the at least one decoder ML model being configured to receive frames from a decoder of the decoder component and insert reconstructed frames between frames received from the decoder.

6. The apparatus according to claim 5, wherein, The decoder ML model is trained on a training set, which includes a sequence of video frames at a first frame rate and missing ground truth frames in the frame sequence.

7. The apparatus according to claim 1, wherein, The processor component is configured to use compression domain information in the video to remove portions of frames from the encoding.

8. An apparatus comprising: At least one computer medium, which is not a transient signal and includes instructions executable by at least one processor component to: input at least a first frame of a video into at least one machine learning (ML) model; and encode a first pixel region of the first frame in response to an output from the ML model, without encoding a second pixel region of the first frame. The first part is transmitted after being encoded, and the second part is neither encoded nor transmitted to the receiver.

9. The apparatus according to claim 8, wherein, The video includes at least one computer game.

10. The apparatus according to claim 8, wherein, The first frame includes a keyframe.

11. The apparatus according to claim 8, wherein, The instruction can be executed to input the compressed domain information associated with the first frame into the ML model along with the first frame.

12. The apparatus according to claim 8, wherein, The ML model is trained on a training set, which includes complete data frames with accompanying compressed domain information and truth part frames that can be output based on the accompanying compressed domain information.

13. The apparatus according to claim 8, wherein, The amount of the encoded first frame is equal to the number of pixels in the first region divided by the total number of pixels in both the first and second regions, each region including at least one pixel.

14. The apparatus of claim 8, further comprising at least one decoder component configured to: receive the first region of the first frame; decode the first region; reconstruct the second region; and present the first region and the second region together on at least one display.

15. The apparatus according to claim 14, wherein, The decoder component includes at least one decoder ML model to reconstruct the second region, the decoder ML model being trained on a training set comprising incomplete frames and ground truth frames of pixel data with accompanying compressed domain information.

16. A method comprising: Invoke at least one machine learning (ML) model to process the frame before encoding the video frame; Provide one or both of pixel domain and / or compressed domain information to the ML model; based at least in part on the output of the ML model, not encode at least a first frame, and / or only encode a first pixel region of a second frame and not encode a second pixel region of the second frame, to provide to at least one receiver via a network.

17. The method of claim 16, comprising: Provide pixel domain information to the ML model; The at least first frame is not encoded, at least in part, based on the output of the ML model.

18. The method of claim 16, comprising: Provide compression domain information to the ML model; At least in part based on the output of the ML model, only the first pixel region of the second frame is encoded, while the second pixel region of the second frame is not encoded.

19. The method of claim 16, wherein, The videos include computer game videos.

20. The method according to claim 18, wherein, The compressed domain information includes motion vectors.

Images