Reinforcement learning for improved mode selection in video codecs

Abstract

Systems and methods for improving video coding. Systems and methods are provided for performing video coding using a dynamically determined weighting parameter that balances rate and distortion tradeoffs. In at least one embodiment, a neural network is trained using reinforcement learning to dynamically determine the weighting parameter that balances rate and distortion tradeoffs.

Description

AREA The present disclosure relates to video coding, and in particular video coding using a dynamically determined weighting parameter that balances rate and distortion tradeoffs. In at least one embodiment, a neural network is trained using reinforcement learning to dynamically determine the weighting parameter that balances rate and distortion tradeoffs. BACKGROUND Video compression reduces the size of digital video files while limiting distortion to maintain visual quality. In recent years, on-demand video playback and video streaming services have proliferated due to advancements in compression, computing, and network and data transport infrastructure technologies. Video content now accounts for more than 80% of all consumer internet traffic—a figure expected to increase in the coming years. Bandwidth reduction achieved through video compression has therefore become crucial for reducing network traffic to meet consumer demand. The video compression process typically begins by receiving input in the form of a video (consisting of a sequence of images or frames) and dividing each frame into a multitude of smaller blocks. Algorithms identify redundancies between different frames (to provide temporal compression) and within individual frames (to provide spatial compression). Once identified, temporal and spatial redundancies can be removed or simplified, allowing the remaining data to be encoded efficiently. To exploit temporal and spatial redundancies for compression purposes, many techniques rely on both inter- and intra-frame prediction. In inter-frame prediction, a predictor estimates frame content based on a previous frame or frames, and a corrector determines the difference between the estimated frame content and the actual frame content—that is, frame correction values. In intra-frame prediction, a predictor estimates pixels based on, for example, neighboring pixels, and a corrector determines the difference between the estimated pixels and the actual pixel values—that is, pixel correction values. Compression is achieved by encoding only the frame correction values ​​(for inter-frame prediction) and the pixel correction values ​​(for intra-frame prediction).The original video can then be reconstructed by estimating frames and pixels in the same way and then correcting them with decoded frame correction values ​​and pixel correction values. To determine a prediction mode for individual frames and / or blocks thereof, video coding techniques consult a joint rate-distortion cost function to maintain a balance between rate and distortion. The joint rate-distortion cost function is represented as J = D + λR, where D represents distortion, R represents rate, and λ is a Lagrange multiplier (i.e., a weighting parameter) that maintains a balance between rate and distortion. BRIEF DESCRIPTION OF THE DRAWINGS The present systems and methods for video acquisition and processing are described in detail below with reference to the accompanying drawings, wherein: Fig. 1A is a block diagram illustrating a system according to one embodiment for encoding video content using a dynamically determined weighting parameter; Fig. 1B is a flowchart illustrating a process according to one embodiment for selecting, using a dynamically determined weighting parameter, a candidate prediction image block for use in a downstream encoding process; Fig. 2A is a block diagram illustrating a system for training a neural network to dynamically determine a weighting parameter that balances rate and distortion during video encoding; Fig.2B is a flowchart illustrating a process according to one embodiment for training a neural network to dynamically determine a weighting parameter that balances rate and distortion during video coding; Fig. 3 is a block diagram illustrating an amplification learning framework for training a neural network; Fig. 4 is a conceptual diagram of a processing system implemented using a PPU suitable for use in implementing some embodiments of the present disclosure; Fig. 5A illustrates an exemplary system in which the various architectures and / or functionality of the various preceding embodiments can be implemented; Fig. 5B illustrates components of an exemplary system that can be used for training and utilizing machine learning in at least one embodiment; and Fig.6 illustrates an exemplary streaming system suitable for use in implementing some embodiments of the present disclosure. DETAILED DESCRIPTION The present disclosure provides systems and methods for improving video coding. These systems and methods are provided for performing video coding using a dynamically determined weighting parameter that balances rate and distortion tradeoffs. In at least one embodiment, a neural network is trained using reinforcement learning to dynamically determine the weighting parameter that balances rate and distortion tradeoffs. In at least one embodiment, the weighting parameter is determined based on coding one or more previous frames and / or based on video content to be coded. In at least one embodiment, the dynamically determined weighting parameter dynamically adjusts a common rate-distortion cost function during video coding to improve video compression quality (i.e.,(providing improved quality at a fixed bitrate or reduced bitrate at a fixed video quality). The following description details various embodiments. For illustrative purposes, specific configurations and details are presented to provide a thorough understanding of these embodiments. However, it is also apparent to those skilled in the art that the embodiments can be implemented without these specific details. Furthermore, known features may be omitted or simplified so as not to obscure the described embodiment. The systems and procedures described here can be used, without restriction, by non-autonomous vehicles, semi-autonomous vehicles (e.g., in one or more advanced driver assistance systems (ADAS)), controlled and uncontrolled robots or robotic platforms, warehouse vehicles, all-terrain vehicles, vehicles coupled with one or more trailers, flying ships, boats, shuttles, emergency vehicles, motorcycles, electric or motorized bicycles, aircraft, construction vehicles, trains, underwater vehicles, remotely controlled vehicles such as drones and / or other vehicle types.Furthermore, the systems and methods described here can be used for a variety of purposes, including but not limited to machine control, machine locomotion, machine driving, synthetic data generation, model training or updating, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twinning, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twinning, data center processing, converse AI, generative AI, light transport simulation (e.g., ray tracing, path tracking, etc.), collaborative content creation for 3D assets, cloud computing, and / or any other suitable applications. Disclosed embodiments can be exhibited by a wide variety of different systems, such as automotive systems (e.g., a control system for an autonomous or semi-autonomous machine, a perception system for an autonomous or semi-autonomous machine), systems implemented using a robot, aerial systems, medical systems, boat systems, intelligent area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using an edge device, systems containing one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, and systems for performing conversational AI operations.Systems for performing generative AI operations, systems implemented using large language models (LLMs), systems implemented using vision language models (VLMs), systems for performing light transport simulation, systems for performing collaborative content creation for 3D assets, systems implemented at least partially using cloud computing resources, and / or other types of systems. In some examples, the machine learning model(s) described here (e.g., deep neural networks, language models, LLMs, VLMs, multimodal language models, perceptual models, tracking models, fusion models, transformer models, diffusion models, encoder-only models, decoder-only models, encoder-decoder models, neural rendering field (NERF) models, etc.) may be packaged as a microservice—such as an inference microservice (e.g., NVIDIA NIMs)—which may contain a container (e.g., an operating system (OS)-level virtualization package) that may include an application programming interface (API) layer, a server layer, a runtime layer, and / or at least one model “engine.” In some cases, such as when the machine learning model(s) is small enough (e.g., has a small number of parameters), the model(s) can be contained within the container itself.In other examples—such as when the model(s) is / are large—the model(s) may be hosted / stored in the cloud (e.g., in a data center) and / or may be hosted on-premises and / or at the edge (e.g., on a local server or computing device, but outside the container). In such embodiments, the model(s) may be accessible via one or more APIs, such as REST APIs. Thus, and in some embodiments, the machine learning model(s) described here may be used as an inference microservice to accelerate the deployment of a model(s) on any cloud, data center, or edge computing system, while ensuring data security. For example, the inference microservice can include one or more APIs, a pre-configured container for simplified deployment, an optimized inference engine (e.g.,built using a standardized AI model deployment, execution software such as NVIDIA's Triton inference server, and / or one or more APIs for high-performance deep learning inference, which may include inference runtime and model optimizations that deliver low latency and high throughput for production applications (such as NVIDIA's TensorRT), and / or enterprise management data for telemetry (e.g., including identity, metrics, integrity checks, and / or monitoring). The machine learning model(s) described here can be included as part of the microservice, along with accelerated infrastructure capable of single-command deployment and / or orchestration and automatic scaling using a container orchestration system on accelerated infrastructure (e.g., from a single device to data center scale). Thus, the inference microservice can include the machine learning model(s) (optimized, for example, for high-performance inference), inference runtime software for executing the machine learning model(s) and providing outputs / responses to inputs (e.g., user queries, prompts, etc.), and enterprise management software for providing integrity checks, identity verification, and / or other monitoring. In some embodiments, the inference microservice can include software for performing on-premises replacement and / or updates of the machine learning model(s).During replacement or update, the software performing the replacement / update may maintain user configurations of the inference runtime software and the enterprise management software. The systems and methods provided by this disclosure offer novel techniques for video coding. In at least one embodiment, this disclosure provides a learned neural network, trained via reinforcement learning, for dynamically determining, for one or more frames (e.g., each frame) of a video to be encoded, a weighting parameter that balances rate and distortion during video encoding. In at least one embodiment, this disclosure provides training a neural network using reinforcement learning to dynamically determine, for one or more frames (e.g., each frame) of a video to be encoded, a weighting parameter that balances rate and distortion during video encoding.In at least one embodiment, the present disclosure provides video content encoding using a weighting parameter that balances rate and distortion, wherein the weighting parameter is dynamically determined based on encoding previous frames and / or based on the video content. The use of a dynamically determined weighting parameter (e.g., as determined by a trained RL agent from state information corresponding to encoding a previous frame) for video encoding has demonstrated a substantial improvement in the bitrate (i.e., Bjontegaard Delta Rate, a measure of the bitrate reduction achieved while maintaining the same quality) compared to the use of a static Lagrange multiplier (e.g., as empirically determined for a single commercially available video codec) for video encoding.According to at least one embodiment, a BD rate improvement of more than 15% was demonstrated through the use of the dynamically determined weighting parameter described herein. A procedure for coding a video is provided herein. The procedure involves determining a weighting parameter for a frame of the video by a trained reinforcement learning (RL) agent. The weighting parameter is determined based on the results of coding one or more previous frames of the video. The procedure further involves generating, for a reference block of a frame of the video, a plurality of candidate prediction blocks and selecting, using rate bias optimization (RDO) based on the determined weighting parameter to balance rate and bias components of a common loss function, one prediction block from the plurality of candidate prediction blocks. According to one embodiment, the method further includes calculating remainders using the selected prediction block and the reference block, and encoding the remainders in a code stream. According to one embodiment of the method, the joint loss function J = D + (λf∗ λ) ∗ R, where D represents distortion, R represents rate, λf is the weighting parameter determined by the trained RL agent, and λ is an empirically determined Lagrange multiplier. According to an alternative embodiment of the method, the joint loss function J = D + λfR, where D represents distortion, R represents rate, and λf is the weighting parameter determined by the trained RL agent. According to one embodiment of the method, encoding one or more previous frames of the video has encoding statistics that correspond to encoding one or more previously encoded video frames.According to one embodiment, the coding statistics include one or more of: a bitstream size, an average quantization parameter (QP) value for one or more frames, a count of intra-coded macroblocks, a count of inter-coded macroblocks, a number of slices, an image type, a sum of absolute transformed differences (SATD) for one or more frames, an average motion vector (MV) x-component, an average motion vector (MV) y-component, a count of 32x32 coding units (CUs) encoded using intra-prediction, a count of 32x32 CUs encoded using inter-prediction, a count of 16x16 CUs encoded using intra-prediction, and a count of 16x16 CUs encoded using inter-prediction. In one embodiment of the method, the trained RL agent is a trained neural network comprising one or more layers of long-short-term memory (LSTM) units. In another embodiment, the trained RL agent is a neural network trained via proximal policy optimization (PPO). In yet another embodiment, the RL agent is a neural network trained via reinforcement learning to maximize a reward based on coding performance. In yet another embodiment, the reward is calculated using an advantage function that measures the benefit gained by coding the frame using the weighting parameter compared to coding the frame without using the weighting parameter. According to one embodiment of the method, the plurality of candidate prediction blocks is generated by an intra-frame predictor and an inter-frame predictor, at least one of the plurality of candidate prediction blocks is predicted according to a combination of one or more prediction parameters, and the combination of one or more prediction parameters includes at least one of: a prediction mode, a partition mode, a coding mode, a skipping mode, a reference frame selection, a chroma subsampling mode, or a strength of one or more filters. According to one embodiment of the method, the encoding is performed using a commercially available video codec, which is one of: H.264, HEVC, AV1, AV2 or VVC. A system for encoding a video is provided herein. The system includes processing circuitry configured to determine a weighting parameter for a frame of the video using a trained reinforcement learning (RL) agent. The weighting parameter is determined based on the results of encoding one or more previous frames of the video. The processing circuitry is further configured to generate, for a reference block of a frame of the video, a plurality of candidate prediction blocks and, using rate bias optimization (RDO) based on the determined weighting parameter to balance the rate and bias components of a common loss function, select a prediction block from the plurality of candidate prediction blocks. The system also includes memory configured to store the frame and the plurality of candidate prediction blocks. According to one embodiment of the system, the processing circuitry is further configured to calculate, using the selected prediction block and reference block, remainders and encode the remainders in a code stream. According to one embodiment of the system, the joint loss function J = D + (λf∗ λ) ∗ R, where D represents distortion, R represents rate, λf is the weighting parameter determined by the trained RL agent, and λ is an empirically determined Lagrange multiplier. According to an alternative embodiment of the system, the joint loss function J = D + λfR, where D represents distortion, R represents rate, and λf is the weighting parameter determined by the trained RL agent. According to one embodiment of the system, the results of encoding one or more previous frames of the video exhibit encoding statistics that correspond to encoding one or more previously encoded video frames.According to one embodiment, the coding statistics include one or more of: a bitstream size, an average quantization parameter (QP) value for one or more frames, a count of intra-coded macroblocks, a count of inter-coded macroblocks, a number of slices, an image type, a sum of absolute transformed differences (SATD) for one or more frames, an average motion vector (MV) x-component, an average motion vector (MV) y-component, a count of 32x32 coding units (CUs) encoded using intra-prediction, a count of 32x32 CUs encoded using inter-prediction, a count of 16x16 CUs encoded using intra-prediction, and / or a count of 16x16 CUs encoded using inter-prediction. are, etc. In one embodiment of the system, the trained RL agent is a trained neural network comprising one or more layers of long-short-term memory (LSTM) units. In another embodiment, the trained RL agent is a neural network trained via proximal policy optimization (PPO). In yet another embodiment, the RL agent is a neural network trained via reinforcement learning to maximize a reward based on coding performance. In yet another embodiment, the reward is calculated using an advantage function that measures the benefit gained by coding the frame using the weighting parameter compared to coding the frame without using the weighting parameter. According to one embodiment of the system, the plurality of candidate prediction blocks is generated by an intra-frame predictor and an inter-frame predictor, wherein at least one of the plurality of candidate prediction blocks is predicted according to a combination of one or more prediction parameters, and the combination of one or more prediction parameters includes at least one of: a prediction mode, a partition mode, a coding mode, a skipping mode, a reference frame selection, a chroma subsampling mode, or a strength of one or more filters. According to one embodiment of the system, encoding is performed using a commercially available video codec, which is one of: VP8, VP9, ​​AVC (h.264), HEVC (h.265), AV1, AV2 or VVC (h.266). A system is provided herein which includes one or more processing units for determining a weighting parameter for a frame of a video, wherein the weighting parameter is determined based on results of coding from one or more previous frames of the video, for generating, for a reference block of a frame of the video, a plurality of candidate prediction blocks, and for selecting, using rate bias optimization (RDO) based on the determined weighting parameter to balance rate and bias components of a common loss function, one prediction block from the plurality of candidate prediction blocks. According to one embodiment of the system, the one or more processing units are located in at least one of: a system for performing simulation operations; a system for performing simulation operations for testing or validating autonomous machine applications; a system for performing digital twin operations; a system for performing light transport simulation; a system for rendering graphical output; a system for performing deep learning operations; a system implemented using an edge device; a system for generating or presenting virtual reality (VR) content; a system for generating or presenting augmented reality (AR) content; a system for generating or presenting mixed reality (MR) content; a system containing one or more virtual machines (VMs); or a system that is at least partially implemented in a data center.a system for performing hardware tests using simulation; a system for generating synthetic data; a system for performing generative AI operations; a system implemented using one or more large language models (LLMs); a system implemented using one or more vision language models (VLMs); a collaborative content creation platform for 3D assets; or a system implemented at least partially using cloud computing resources. Fig. 1A provides a block diagram of an exemplary system 100 according to one embodiment. It is understood that this and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, groupings of functions, etc.) may be used in addition to or instead of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in conjunction with other components, and in any suitable combination and position. Various functions described herein as being performed by entities may be executed by hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in memory.Furthermore, those skilled in the art will understand that any system performing the operations of exemplary system 100 is within the scope and concept of embodiments of the present disclosure. System 100 encodes video content using Rate Distortion Optimization (RDO), a dynamically determined weighting parameter that balances rate and distortion. The weighting parameter is dynamically determined based on encoding previous frames and / or the video content being encoded. System 100 includes a predictor 102, a λ-factor agent 104, a Rate Distortion Optimization (RDO) mode selector 106, a corrector 108, and a residual encoder 110. Predictor 102 receives an input video 101A and performs preprocessing of the input video 101A to generate candidate frame blocks 103. Predictor 102 subdivides one or more frames (e.g., each frame) in the input video 101A into a plurality of macroblocks. The subdivision is performed according to a commercially available video codec (e.g., VP8, VP9, ​​AVC (h.264), HEVC (h.265), AV1, AV2, or VVC (h.266), etc.). Predictor 102 further provides, for one or more respective macroblocks (e.g., each macroblock), a plurality of candidate prediction frame blocks for use in a downstream encoding process. At least one (e.g., each) of the multitude of candidate prediction image blocks is predicted according to a unique combination of prediction parameters that includes a prediction mode (i.e., intra-prediction or inter-prediction).For candidate prediction image blocks specifying the intra-prediction mode, additional specified prediction parameters may include, for example, a macroblock partition mode and a direction mode. For candidate image blocks specifying the inter-prediction mode, one or more additional specified prediction parameters may include a macroblock partition mode, motion vector candidates, and frame reference indices. In at least one embodiment, the combination of prediction parameters includes parameters specifying one or more of: a prediction mode, a partition mode, an encoding mode, a skip mode, a frame reference selection, a chroma subsampling mode, and the strength of one or more filters. The candidate image blocks are output as candidate image blocks 103 and provided as input for the RDO mode selector 106. In one or more embodiments, the λ-factor agent 104 receives coding statistics 101B and, based on these, determines a λ-factor that is used during video coding. In at least one embodiment, the coding statistics 101B include, for example, frame statistics, mode selection information, a combination of prediction parameters, etc., corresponding to the coding of one or more previously encoded video frames. In at least one embodiment, coding statistics 101B include one or more of a bitstream size (e.g., bitstreamSizeInBytes), an average quantization parameter (QP) value for one or more frames (e.g., frameAvgQP), a count of intra-coded macroblocks (e.g., intraMBCount), a count of inter-coded macroblocks (e.g., interMBCount), a number of slices (numSlices), a picture type (e.g., pictureType), a sum of absolute transformed differences (SATD) for one or more frames (e.g.,frameSatd), an average motion vector (MV) x-component (e.g., averageMVX), an average motion vector (MV) y-component (e.g., averageMVY), a count of 32x32 coding units (CUs) coded using intra-prediction (e.g., intra_cu32x32_count), a count of 32x32 CUs coded using inter-prediction (e.g., inter_cu32x32_count), a count of 16x16 CUs coded using intra-prediction (e.g., intra_cu16x16_count), and a count of 16x16 CUs coded using inter-prediction (e.g., inter_cu16x16_count). In at least one embodiment, λ-factor agent 104 is a trained neural network (e.g., a neural network trained via the system illustrated in Fig. 2A, the process illustrated in Fig. 2B, and / or the framework illustrated in Fig. 3).In at least one embodiment, λ-factor agent 104 is a neural network trained via reinforcement learning. In at least one embodiment, the neural network is trained via proximal policy optimization (PPO). In at least one embodiment, the neural network is trained to predict a λ-factor that minimizes distortion for a given bitrate. In at least one embodiment, the neural network is trained to predict a λ-factor that minimizes bitrate for a given video playback quality. In at least one embodiment, the trained neural network includes one or more layers of long-short-term memory (LSTM) units. In at least one embodiment, the trained neural network consists of three layers of LSTM units, each containing 64 neurons.LSTM units are able to capture temporal dependencies and are therefore suitable for video data that includes temporal sequences of frames. In one or more embodiments, the RDO mode selector 106 selects, for one or more macroblocks (e.g., each macroblock), a single candidate image block (specifying a unique combination of prediction parameters) from the plurality of candidate image blocks 103. The selection made by the RDO mode selector 106 is based on the λ-factor provided by the λ-factor agent 104, which balances rate and distortion during video encoding to improve video compression quality (i.e., to provide improved quality at a fixed bitrate or to provide reduced bitrate at a fixed video quality). In at least one embodiment, rate and distortion are balanced during video encoding by minimizing a joint rate-distortion cost function J = D + (λf∗ λ)R, where D represents distortion, R represents rate, and λ is a Lagrange multiplier (i.e.,a weighting parameter) that maintains a balance between rate and distortion, and λf is the λ-factor determined by the λ-factor agent 104. The product λf*λ is therefore a dynamically determined (based on encoding previous frames and / or the video content to be encoded) weighting parameter that balances rate and distortion in the RDO mode selection performed by the RDO mode selector 106. In at least one such embodiment, the static weighting parameter λ is determined based on a quantization parameter of a commercial video codec. For example, in at least one embodiment, the static weighting parameter λ is selected from a fixed table of empirically derived values ​​corresponding to selected quantization parameters of a single commercial video codec.In at least one alternative embodiment, rate and distortion are balanced during video encoding by minimizing a common rate-distortion cost function J = D + λfR, where D is a value of the distortion measure, R is a value of the bit rate measure, and λfR is a dynamically determined λ-factor determined by the λ-factor agent 104. In at least one embodiment, the RDO mode selector 106 selects the single prediction image block for the macroblock via the process illustrated in Fig. 1B. Corrector 108 calculates, using the prediction image block selected by the RDO mode selector 106 and the corresponding macroblock of the frame of the input video 101A, remainders (i.e., correction values) between the prediction image block and the corresponding macroblock. Corrector 108 outputs the calculated remainders, which are provided as input to the remainder encoder 110. The remainder encoder 110 encodes the remainders into codestream 111 (e.g., an encoded bitstream), which can then be transmitted over a network to a user device and / or stored in memory. In addition to the encoded remainders, the codestream 111 includes a specification of a combination of prediction parameters (i.e., the prediction parameters according to which the selected prediction image block is predicted) to enable a decoder to reconstruct the macroblock. Figure 1B is a flowchart illustrating a process 150 according to one embodiment for selecting, using a dynamically determined weighting parameter, a candidate prediction image block for use in a downstream coding process. Each block of process 150 described herein has a computational process that can be performed using any combination of hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in memory. The process can also be embodied as computer-usable instructions stored on computer storage media.The process can be provided by a standalone application, a service, a hosted service (alone or in combination with another hosted service), or a plug-in for another product, to name just a few. Additionally, process 150 is described by way of example with respect to the system of Fig. 1A. However, this process can additionally or alternatively be executed by any system or any combination of systems, including, but not limited to, those described herein. Furthermore, those skilled in the art will understand that any system performing process 150 falls within the scope and intent of embodiments of this disclosure. Process 150 is a process for selecting, by an RDO mode selector (e.g., RDO mode selector 106) for use in a downstream coding process (e.g., the coding process performed by corrector 108 and residual encoder 110), a single candidate prediction image block (e.g., one of the candidate prediction image blocks 103) for a macroblock of a frame of an input video (e.g., a macroblock of a frame of input video 101A). At 152, the process receives a plurality of candidate prediction image blocks that can be used to encode a macroblock of a frame from an input video. One or more (e.g., each) of the plurality of candidate prediction image blocks have been predicted according to a unique combination of prediction parameters, which may include—for example, and without limitation—a prediction mode (i.e., intra-prediction or inter-prediction), a partition mode, an encoding mode (e.g., I-frame, P-frame, or B-frame), a skip mode, a reference frame selection, and the strength of one or more filters, such as a denoising filter, a sharpening filter, and / or an unblocking filter. In at least one embodiment, the denoising and / or sharpening filters include one or more of a sample adaptive offset (SAO) filter, an unblocking filter, and / or a Weiner filter.At 152, the process also initializes the minimum value Jmin of a common loss function J by setting Jmin equal to positive infinity. At 154, the process selects a single candidate prediction image block from the plurality of candidate prediction image blocks received at 152. At 156, the process computes the value of the joint loss function J for the selected candidate prediction image block. In at least one embodiment, the process computes the value of the joint loss function J at 156 by (i) computing remainders (i.e., correction values) between the candidate prediction image block and the corresponding macroblock of the input, (iii) transforming the remainders into the frequency domain, (iv) quantizing the transformed remainders, and (v) encoding the quantized transformed remainders. The process then uses (vi) the encoded remainders to measure a distortion and bit rate corresponding to the encoding of the selected candidate prediction image block and (vii) computes the value of the joint loss function J. In at least one embodiment, the joint loss function J is calculated according to J = D + (λf∗ λ) ∗ R, where D is a value of the distortion measure, R is a value of the bitrate measure, and λ is a static weighting parameter (i.e., a Lagrange multiplier) that controls the relative magnitude of the distortion measure D and the bitrate measure R, and λfein is a dynamically determined λ-factor determined based on encoding previous frames and / or the content of the video to be encoded (e.g., the λ-factor determined by the λ-factor agent 104 of Fig. 1A). The product λf∗ λ is therefore a dynamically determined weighting parameter that balances rate and distortion in RDO mode selection. In at least one such embodiment, the static weighting parameter λ is determined based on a quantization parameter of a commercially available video codec.For example, in at least one embodiment, the static weighting parameter λ is selected from a fixed table of empirically derived values ​​corresponding to selected quantization parameters of a single commercially available video codec. In at least one alternative embodiment, the joint loss function J is calculated according to J = D + λfR, where D is a value of the distortion measure, R is a value of the bit rate measure, and λfein is a dynamically determined λ-factor based on encoding previous frames and / or the content of the video to be encoded (e.g., the λ-factor determined by the λ-factor agent 104 of Fig. 1A).Smaller values ​​of the joint loss function J indicate better quality because (i) for a constant bit rate, lower distortion and therefore lower J indicate better quality or better compression, and (ii) for a constant distortion, a lower bit rate and therefore lower J indicate a smaller file size / reduced network bandwidth. At step 158, the process compares the value of the joint loss function J, computed for the selected candidate prediction image block, with the stored value of Jmin. If the computed value of J is less than Jmin, the process updates the value of Jmin at step 160, stores an identifier for the selected candidate prediction image block corresponding to the value of Jmin, and proceeds to step 162. If the computed value of J is less than Jmin, the process proceeds directly to step 162. At step 162, the process determines whether the candidate prediction image block selected at step 154 ​​was the last candidate prediction image block from the multitude of candidate image blocks received at step 152. If the process determines at step 162 that additional candidate prediction image blocks remain, the process returns to step 154, where an additional candidate prediction image block is selected.Alternatively, if the process determines at 162 that all candidate prediction image blocks received at 152 have been processed, the process proceeds to 164. At 164, the process outputs the candidate prediction image block identified as matching the Jminentgleich. In this way, the process compares the compression results achieved with one or more (e.g., each) from the multitude of candidate prediction image blocks and selects the single candidate prediction image block that provides the best results—by optimizing the common rate compression loss function—for the downstream coding process. Fig. 2A provides a block diagram of an exemplary system 200 according to one embodiment. It is understood that this and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, groupings of functions, etc.) may be used in addition to or instead of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in conjunction with other components, and in any suitable combination and position. Various functions described herein as being performed by entities may be executed by hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in memory.Furthermore, those skilled in the art will understand that any system performing the operations of exemplary system 200 is within the scope and concept of embodiments of the present disclosure. System 200 includes neural network 210, encoder 220, memory 230 and processing circuitry 240. Neural network 210 is configured to receive, as input, coding state information 232 (i.e., corresponding to a previously encoded frame) from memory 230 and to generate, as output, a λ-factor to compensate for rate and distortion during video coding performed by encoder 220. Encoder 220 is configured to receive, as input, a frame of stored video 231 from memory 230 and the λ-factor generated by neural network 210, and to generate, as output, coding state information 232 (which is stored in memory 230 and serves as input for neural network 210 to determine a λ-factor for a subsequent frame) and a first encoded frame, which is encoded using the λ-factor generated by neural network 210.In at least one embodiment, encoder 220 is further configured to generate, as output, a second coded frame, which is coded without using the λ-factor generated by the neural network. Processing circuitry 240 is configured to receive one or more coded frames 233 and to calculate, based on one or more of the coded frames (and, in at least one embodiment, also based on the stored video 231), performance metrics 234. Processing circuitry 240 is further configured to calculate, based on the performance metrics 234, a reward for the λ-factor generated by the neural network 210 and an accumulated reward 235, which is accumulated for generating the λ-factors over multiple frames.The processing circuit 240 is additionally configured to determine updated network parameters 236 based on the accumulated reward 235 and to update the parameters of the neural network 210 accordingly. In at least one embodiment, the processing circuit 240 is configured to execute a PPO algorithm to update the parameters of the neural network 210. Figure 2B is a flowchart illustrating a Process 250 for training a neural network (e.g., λ-factor agent 104 from Figure 1A) using reinforcement learning to dynamically determine a λ-factor for modifying a weighting factor that balances rate and distortion during video encoding. Each block of Process 250 described herein has a computational process that can be performed using any combination of hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in memory. The process can also be embodied as computer-usable instructions stored on computer storage media.The process can be provided by a standalone application, a service, a hosted service (alone or in combination with another hosted service), or a plug-in for another product, to name just a few. Additionally, process 250 is described by way of example with respect to the system of Fig. 2A. However, this process can additionally or alternatively be executed by any system or any combination of systems, including, but not limited to, those described herein. Furthermore, those skilled in the art will understand that any system performing process 250 falls within the scope and intent of embodiments of this disclosure. Process 250 receives a batch of video frames to be encoded (e.g., a batch of video frames comprising a video clip) and encodes the video frames on a frame-by-frame basis. At 251, Process 250 receives a video frame to be encoded. At 252, the neural network (e.g., λ-factor agent 104 of Fig. 1A), based on information corresponding to the encoding of previous video frames, determines a λ-factor to modify a weighting factor that balances rate and distortion during video encoding. At 254, an encoder (e.g., an encoder formed by predictor 102, RDO mode selector 106, corrector 108, and residual encoder 110 from Fig. 1A) encodes the video frame to be encoded using the λ-factor determined at 252 (e.g., by selecting prediction parameters using the process of Fig. 1B). At 256, the process outputs an encoding performance metric (e.g.,a PSNR value for encoding the video frame to be coded using the λ-factor). At 258, the process outputs state information (e.g., frame statistics, mode selection information, prediction parameters, etc.) corresponding to the encoding of the video frame to be coded (which is provided to the neural network for use in determining the λ-factor for subsequent iterations). At 260, an encoder (e.g., the encoder formed by predictor 102, RDO mode selector 106, corrector 108, and residual encoder 110 of Fig. 1A) encodes the video frame to be encoded using a static weighting parameter λ (e.g., an empirically derived value) that compensates for rate and distortion during video encoding. In at least one such embodiment, the static weighting parameter λ is determined based on a quantization parameter of a commercial video codec. For example, in at least one embodiment, the static weighting parameter λ is selected from a fixed table of empirically derived values ​​corresponding to selected quantization parameters of a single commercial video codec. At 262, the process outputs an encoding performance metric (e.g., a PSNR value for encoding using the static weighting factor).At 264, the process calculates a reward based on the coding performance metrics received at 256 and 262. In at least one embodiment, the reward is calculated using an advantage function that provides a measure of the relative improvement achieved through coding performed using the λ-factor determined at 252. At 266, the process updates an accumulated reward by adding the reward calculated at 264 to rewards calculated for coding previous video frames in the batch of video frames. At 268, the process determines whether any additional frames remain in the batch. The process repeats until all frames in the batch have been coded, at which point, at 270, the process updates the neural network parameters. The entire process can then be restarted with a new batch of video frames to be coded (e.g.,(a new video clip) are repeated to further refine the parameters of the neural network. Figure 3 is a block diagram illustrating a reinforcement learning framework for training Agent 302, provided as a neural network (e.g., λ-factor Agent 104 of Figure 1A), to dynamically determine a λ-factor for modifying a weighting factor that balances rate and distortion during video encoding. In at least one embodiment, the trained neural network includes one or more layers of long-short-term memory (LSTM) units. In at least one embodiment, the trained neural network consists of three layers of LSTM units, each containing 64 neurons. LSTM units are capable of capturing temporal dependencies and are therefore suitable for video data containing temporal sequences of frames. In the reinforcement learning framework illustrated in Fig. 3, the λ-factor determined by Agent 302 (i.e., the action At selected by Agent 302) is communicated to Environment 304 (provided in the form of a video encoder), which encodes a frame using the λ-factor and provides feedback to the Agent in the form of a state St (e.g., frame statistics, mode selection information, prediction parameters, etc. – as described at 208 of Fig. 2A) and a reward Rt (e.g., a PSNR). In at least one embodiment, the neural network is trained via proximal policy optimization (PPO). Therefore, in the reinforcement learning framework illustrated in Fig. 3, Agent 302 is responsible for predicting a λ-factor and transmitting it to Environment 304 (i.e., the encoder).In at least one embodiment, the λ-factor is transmitted to the encoder via interprocess communication (IPC), which acts as a bridge between Agent 302 and the encoder. Environment 304 receives the λ-factor and uses it to encode a frame, after which it generates feedback and returns it to Agent 302 in the form of state Stz. In at least one embodiment, state Stz includes, for example, frame statistics and training progress. Environment 304 also returns a reward Rt, which—in at least one embodiment—is measured as a distortion in terms of peak signal-to-noise ratio (PSNR). The feedback loop provided by the reinforcement learning framework is crucial because it allows Agent 302 to update its internal state based on the received information, thereby facilitating continuous learning and adaptation.The entire process is summarized in a feedback loop, in which Agent 302's predictions are refined over time, resulting in improved coding performance. Further illustrative information will now be presented regarding various optional architectures and features with which the aforementioned framework can be implemented according to the user's requirements. It should be expressly noted that the following information is presented for illustrative purposes and should in no way be interpreted as restrictive. Each of the following features can be included optionally, with or without excluding other described features. EXEMPLARY COMPUTING SYSTEM Systems with multiple GPUs and CPUs are used in a wide variety of industries as developers unlock and leverage greater parallelism in applications such as artificial intelligence. High-performance GPU-accelerated systems with tens to many thousands of compute nodes are deployed in data centers, research facilities, and supercomputers to solve increasingly complex problems. As the number of processing units within these high-performance systems grows, communication and data transmission mechanisms must scale to support the increased bandwidth. Fig. 4 is a conceptual diagram of a processing system 500 implemented using multiple PPUs 400, according to one embodiment. The exemplary system 500 can be used as a single node—or part thereof—in the multi-node computing systems described above. In addition to the multiple PPUs 400, the processing system 500 includes a CPU 530, a switch 510, and respective memory 404s for the PPUs 400. Each Parallel Processing Unit (PPU) 400 can contain hundreds or thousands of cores capable of handling hundreds or thousands of software threads simultaneously. The PPUs 400 can generate pixel data for output images in response to render commands (e.g., render commands from the CPU(s) 530 received via a host interface). The PPUs 400 can include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPU data. The display memory can be included as part of the memory 404. The PPUs 400 can contain two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using NVLINK 410) or connect them via a switch (e.g., using Switch 510).When combined, each PPU 400 can generate pixel data or GPGPU data for different portions of an output or for different outputs (e.g., a first PPU for a first image and a second PPU for a second image). Each PPU 400 can have its own dedicated memory or can share memory with other PPUs 400. The PPUs 400 can all include one or more processing cores and / or components thereof, such as Tensor Cores (TCs), Tensor Processing Units (TPUs), Visual Pixel Cores (PVCs), Vision Processing Units (VPUs), Graphics Processing Clusters (GPCs), Texture Processing Clusters (TPCs), Streaming Multiprocessors (SMs), Tree Traversal Units (TTUs), Artificial Intelligence Accelerators (AIAs), Deep Learning Accelerators (DLAs), Arithmetic Logic Units (ALUs), Application-Specific Integrated Circuits (ASICs), Floating-Point Units (FPUs), Input / Output (I / O) elements, Peripheral Component Interconnect (PCI) or Peripheral Component Interconnect Express (PCIe) elements, and / or the like. The NVLink 410 provides high-speed communication links between each of the PPUs 400. Although a single number of NVLink 410 and Interconnect 402 links is illustrated in Fig. 4, the number of links to each PPU 400 and the CPU 530 can vary. The switch 510 provides an interface between the Interconnect 402 and the CPU 530. The PPUs 400, memory 404, and NVLinks 410 can be located on a single semiconductor platform to form a parallel processing module 525. In one embodiment, the switch 510 supports two or more protocols for interface between different links and / or interconnects. In another embodiment (not shown), the NVLink 410 provides one or more high-speed communication links between each of the PPUs 400 and the CPU 530, and the switch 510 provides an interface between the interconnect 402 and each of the PPUs 400. The PPUs 400, memory 404, and interconnect 402 can be located on a single semiconductor platform to form a parallel processing module 525. In yet another embodiment (not shown), the interconnect 402 provides one or more communication links between each of the PPUs 400 and the CPU 530, and the switch 510 provides an interface between each of the PPUs 400 using the NVLink 410 to provide one or more high-speed communication links between the PPUs 400.In another embodiment (not shown), the NVLink 410 provides one or more high-speed communication links between the PPUs 400 and the CPU 530 via the switch 510. In yet another embodiment (not shown), the interconnect 402 provides one or more communication links directly between each of the PPUs 400. One or more of the high-speed communication links of the NVLink 410 can be implemented as a physical NVLink interconnect or as either an on-chip or on-die interconnect using the same protocol as the NVLink 410. In the context of this description, a single semiconductor platform can refer to a single, unified semiconductor-based integrated circuit fabricated on a die or chip. It should be noted that the term "single semiconductor platform" can also refer to multi-chip modules with enhanced connectivity that simulate on-chip operation and offer significant improvements over the use of a conventional bus implementation. Naturally, the various circuits or devices can also be arranged separately or in various combinations of semiconductor platforms, according to the user's requirements. Alternatively, the parallel processing module 525 can be implemented as a printed circuit board substrate, and each of the PPUs 400 and / or memory 404 can be a packed device.In one embodiment, the CPU 530, the switch 510 and the parallel processing module 525 are located on a single semiconductor platform. In one embodiment, the signaling rate of each NVLink 410 is 20 to 25 gigabits per second, and each PPU 400 includes six NVLink 410 interfaces (as shown in Fig. 4, five NVLink 410 interfaces are included for each PPU 400). Each NVLink 410 provides a data transfer rate of 25 gigabits per second in each direction, with six links providing 400 gigabytes per second. The NVLinks 410 can be used exclusively for PPU-to-PPU communication, as shown in Fig. 4, or for a combination of PPU-to-PPU and PPU-to-CPU communication if the CPU 530 also includes one or more NVLink 410 interfaces. In one embodiment, the NVLink 410 enables direct load / store / atomic access from the CPU 530 to the memory 404 of each PPU 400. In another embodiment, the NVLink 410 supports coherence operations, allowing data read from the memory 404 to be stored in the cache hierarchy of the CPU 530, thus reducing cache access latency for the CPU 530. In yet another embodiment, the NVLink 410 includes support for Address Translation Services (ATS), enabling the PPU 400 to directly access page tables within the CPU 530. One or more of the NVLinks 410 can also be configured to operate in a low-power mode. Fig. 5A illustrates an exemplary system 565 in which the different architecture and / or functionality of the various preceding embodiments can be implemented. The exemplary system 565 can be configured to implement the method 300 shown in Fig. 3. As shown, a system 565 is provided which includes at least one central processing unit 530 connected to a communication bus 575. The communication bus 575 can directly or indirectly couple one or more of the following devices: main memory 540, network interface 535, CPU(s) 530, display device(s) 545, input device(s) 560, switch(es) 510, and parallel processing system 525. The communication bus 575 can be implemented using any suitable protocol and can represent one or more links or buses, such as an address bus, a data bus, a control bus, or a combination thereof.The 575 communication bus can include one or more bus or interconnect types, such as an Industry Standard Architecture (ISA) bus, an Extended Industry Standard Architecture (EISA) bus, a Video Electronics Standard Association (VESA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, HyperTransport, and / or another type of bus or interconnect. In some embodiments, there are direct connections between components. For example, the 530 CPU(s) can be directly connected to the 540 main memory. Furthermore, the 530 CPU(s) can be directly connected to the 525 parallel processing system. When there is a direct or point-to-point connection between components, the 575 communication bus can include a PCIe interconnect to perform the connection. In these examples, a PCI bus need not be included in the 565 system. Although the various blocks of Fig. 5A are shown connected by lines via the communication bus 575, this is not intended to be restrictive and is for illustrative purposes only. For example, in some embodiments, a presentation component, such as display device(s) 545, may be considered an I / O component, such as input device(s) 560 (e.g., if the display is a touchscreen). As another example, the CPU(s) 530 and / or the parallel processing system 525 may include memory (e.g., the main memory 540 may be representative of a storage device in addition to the parallel processing system 525, the CPUs 530, and / or other components). In other words, the computing device of Fig. 5A is merely illustrative.No distinction is made between categories such as "workstation", "server", "laptop", "desktop", "tablet", "client device", "mobile device", "handheld device", "game console", "electronic control unit (ECU)", "virtual reality system" and / or other device or system types, as all are considered within the scope of the computing device of Fig. 5A. The System 565 also includes a main memory 540. Control logic (software) and data are stored in the main memory 540, which can take the form of a variety of computer-readable media. These computer-readable media can be any available media accessible to the System 565. They can include volatile and non-volatile media, as well as removable and non-removable media. For example, and without limitation, computer-readable media can include computer storage media and communication media. Computer storage media can include both volatile and non-volatile media, and / or removable and non-removable media, implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules, and / or other data types. For example, main memory can store 540 computer-readable instructions (which represent, for example, a program and / or program element, such as an operating system). Computer storage media can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, Digital Versatile Discs (DVDs) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the System 565.As used herein, computer storage media do not inherently possess signals. Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal, such as a carrier wave or other transport mechanism, and include any information transmission media. The term "modulated data signal" can refer to a signal in which one or more of its properties are set or modified in such a way that information is encoded in the signal. For example, and without limitation, computer storage media can include wired media, such as a wired network or a direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of any of the foregoing should also be included within the scope of computer-readable media. Computer programs, when executed, enable the System 565 to perform various functions. The CPU(s) 530 can be configured to execute at least some of the computer-readable instructions, to control one or more components of the System 565, and to perform one or more of the procedures and / or processes described herein. The CPU(s) 530 can each contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling a large number of software threads simultaneously. The CPU(s) 530 can contain any type of processor and may contain different types of processors depending on the type of System 565 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers).For example, depending on the System 565 type, the processor could be an Advanced RISC Machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC), or an x86 processor implemented using Complex Instruction Set Computing (CISC). The System 565 could include one or more CPUs 530 in addition to one or more microprocessors or supplementary coprocessors, such as math coprocessors. In addition to or as an alternative to the CPU(s) 530, the parallel processing module 525 can be configured to execute at least some of the computer-readable instructions to control one or more components of the System 565 to perform one or more of the procedures and / or processes described herein. The parallel processing module 525 can be used by the System 565 to render graphics (e.g., 3D graphics) or to perform general-purpose computations. For example, the parallel processing module 525 can be used for general-purpose computing on GPUs (GPGPU). In embodiments, the CPU(s) 530 and / or the parallel processing module 525 can perform any combination of the procedures, processes, and / or portions thereof, either discretely or jointly. The System 565 also includes input device(s) 560, the parallel processing system 525, and display device(s) 545. The display device(s) 545 may include a display (e.g., a monitor, a touchscreen, a television screen, a head-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The display device(s) 545 may receive data from other components (e.g., the parallel processing system 525, the CPU(s) 530, etc.) and output the data (e.g., as an image, video, sound, etc.). The network interface 535 enables the system 565 to be logically coupled with other devices, including the input devices 560, the display device(s) 545, and / or other components, some of which may be built into (e.g., integrated with) the system 565. Illustrative input devices 560 include a microphone, mouse, keyboard, joystick, gamepad, game controller, satellite dish, scanner, printer, wireless device, etc. The input devices 560 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some cases, inputs can be transferred to a suitable network element for further processing.A NUI can implement any combination of speech recognition, pen recognition, facial recognition, biometric recognition, gesture recognition (both on-screen and off-screen), air gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with a System 565 display. The System 565 can incorporate depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof for gesture detection and recognition. Additionally, the System 565 can incorporate accelerometers or gyroscopes (e.g., as part of an inertial measurement unit (IMU)) to enable motion detection. In some examples, the output from the accelerometers or gyroscopes can be used by the System 565 to render immersive augmented reality or virtual reality. Furthermore, the System 565 can be connected to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the internet, a peer-to-peer network, a cable network, or the like) via a network interface 535 for communication purposes. The System 565 can be included in a distributed network and / or a cloud computing environment. The Network Interface 535 can include one or more receivers, transmitters, and / or transceivers that enable the System 565 to communicate with other computing devices over an electronic communications network, including wired and / or wireless communications. The Network Interface 535 can be implemented as a Network Interface Controller (NIC) that includes one or more Data Processing Units (DPUs) to perform operations such as (but not limited to) packet parsing and accelerating network processing and communication. The Network Interface 535 can include components and functionality to enable communication over any number of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., Ethernet ...Communicating via Ethernet or InfiniBand), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.) and / or the Internet. The System 565 may also include secondary storage (not shown). Secondary storage may include, for example, a hard disk drive and / or a removable storage drive, such as a floppy disk drive, magnetic tape drive, compact disc drive, digital versatile disk (DVD) drive, recording device, or universal serial bus (USB) flash memory. The removable storage drive reads from and / or writes to a removable storage device in a known manner. The System 565 may also include a hardwired power supply, a battery power supply, or a combination thereof (not shown). The power supply can provide power to the System 565 to enable its components to operate. Each of the foregoing modules and / or devices can even be located on a single semiconductor platform to form the System 565. Alternatively, the various modules can also be arranged separately or in various combinations of semiconductor platforms according to the user's requirements. While various embodiments have been described above, it is understood that these are only exemplary and not limiting. Therefore, the breadth and scope of a preferred embodiment should not be limited by any of the exemplary embodiments described above, but should only be defined according to the following claims and their equivalents. EXEMPLARY NETWORK ENVIRONMENTS Network environments suitable for use in implementing embodiments of the disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be implemented on one or more instances of the processing system 500 of Fig. 4 and / or the exemplary system 565 of Fig. 5A—e.g., each device may include similar components, features, and / or functionality to the processing system 500 and / or the exemplary system 565. Components of a network environment can communicate with each other over a network, which can be wired, wireless, or both. The network can include multiple networks or a network of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks, such as the internet and / or a public switched telephone network (PSTN), and / or one or more private networks. If the network includes a wireless telecommunications network, components such as a base station, a communications mast, or even access points (as well as other components) can provide wireless connectivity. Compatible network environments can include one or more peer-to-peer network environments—in which case a server may not be included in a network environment—and one or more client-server network environments—in which case one or more servers may be included in a network environment. In peer-to-peer network environments, the functionality described herein can be implemented with respect to a server on any number of client devices. In at least one embodiment, a network environment can include one or more cloud-based network environments, a distributed computing environment, a combination thereof, etc. A cloud-based network environment can include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. A framework layer can include a framework for supporting software of a software layer and / or one or more application(s) of an application layer. The software or application(s) can each include web-based service software or applications. In embodiments, one or more of the client devices can use the web-based service software or applications (e.g.,by accessing the service software and / or applications via one or more application programming interfaces (APIs). The framework layer can, but is not limited to, be a type of free and open-source software web application framework, such as one that can use a distributed file system for large-scale data processing (e.g., "Big Data"). A cloud-based network environment can provide cloud computing and / or cloud storage, performing any combination of the computing and / or data storage functions described here (or one or more portions thereof). Each of these different functions can be distributed across multiple locations of central or core servers (e.g., one or more data centers, which may be distributed across a state, region, country, the globe, etc.). If a connection to a user (e.g., a client device) is relatively close to an edge server(s), a core server may designate at least some of the functionality for the edge server(s). A cloud-based network environment can be private (e.g., restricted to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment). The client device(s) may include at least some of the components, features and functionality of the exemplary processing system 500 of Fig. 4 and / or the exemplary system 565 of Fig. 5A.For example, and without limitation, a client device may be embodied as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, portable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, video camera, surveillance device or system, vehicle, boat, flying vessel, virtual machine, drone, robot, handheld communication device, hospital device, gaming device or system, entertainment system, vehicle computer system, embedded control panel, remote control, device, consumer electronics device, workstation, edge device, any combination of these outlined devices, or any other suitable device. MACHINE LEARNINGDeep neural networks (DNNs) developed on processors such as the PPU 400 have been used for a variety of applications, from self-driving cars to accelerated drug development, from automatic image labeling in online image databases to intelligent real-time speech translation in video chat applications. Deep learning is a technique that models the neural learning process of the human brain, continuously learning, becoming increasingly intelligent, and delivering more accurate results faster over time. A child is initially taught by an adult to correctly identify and classify different shapes, eventually becoming able to identify shapes without coaching. Similarly, a deep learning or neural learning system needs to be trained in object recognition and classification to become more intelligent and efficient at identifying basic objects, hidden objects, and so on.to identify objects, while also assigning context to them. At its simplest level, neurons in the human brain consider various inputs they receive, assign importance levels to each of these inputs, and pass the output on to other neurons to respond to. An artificial neuron is the most basic model of a neural network. For example, a neuron might receive one or more inputs representing different features of an object that the neuron is trained to recognize and classify. Each of these features is assigned a weight based on its importance in defining the object's shape. A deep neural network (DNN) model comprises multiple layers of many interconnected nodes (e.g., neurons, Boltzmann machines, radial basis functions, convolutional layers, etc.) that can be trained on enormous amounts of input data to quickly solve complex problems with high accuracy. In one example, the first layer of the DNN model decomposes an input image of a car into different sections and looks for basic patterns such as lines and angles. The second layer reassembles the lines to search for higher-level patterns such as wheels, windshields, and mirrors. The next layer identifies the vehicle type, and the final few layers generate a label for the input image that identifies the model of a specific car brand. Once the DNN is trained, it can be deployed and used to identify and classify objects or patterns in a process known as inference. Examples of inference (the process by which a DNN extracts useful information from a given input) include identifying handwritten numbers on checks deposited in ATMs, identifying pictures of friends in photos, providing movie recommendations to over fifty million users, identifying and classifying different types of automobiles, pedestrians, and road hazards in self-driving cars, and translating human speech in real time. During training, data flows through the DNN in a forward propagation phase until a prediction is generated that displays a label corresponding to the input. If the neural network does not label the input correctly, then errors between the correct label and the predicted label are analyzed, and the weights for each feature are adjusted during a backward propagation phase until the DNN correctly labels the input and other inputs in a training dataset. Training complex neural networks requires massive amounts of parallel computing power, including floating-point multiplication and addition, which is supported by the PPU 400.Inference is less computationally intensive than training because it is a latency-sensitive process in which a trained neural network is applied to new inputs it has not seen before to classify images, detect emotions, identify recommendations, recognize and translate speech, and generally infer new information. Neural networks rely heavily on matrix math operations, and complex multilayer networks require enormous amounts of floating-point performance and bandwidth for both efficiency and speed. With thousands of processing cores optimized for matrix math operations and delivering tens to hundreds of TFLOPS of performance, the PPU 400 is a computing platform capable of delivering the performance required for deep neural network-based artificial intelligence and machine learning applications. Furthermore, images generated using one or more of the techniques disclosed herein can be used to train, test, or certify DNNs used to recognize objects and environments in the real world. Such images can include scenes of streets, factories, buildings, urban environments, rural environments, people, animals, and any other physical object or real-world situation. Such images can be used to train, test, or certify DNNs used in machines or robots to manipulate, handle, or modify physical objects in the real world. Furthermore, such images can be used to train, test, or certify DNNs used in autonomous vehicles to navigate and move the vehicles through the real world.Additionally, images produced using one or more of the techniques disclosed herein can be used to transmit information to users of such machines, robots and vehicles. Figure 5B illustrates components of an exemplary system 555 that can be used for training and utilizing machine learning according to at least one embodiment. As discussed, different components can be provided by different combinations of computing devices and resources, or by a single computing system that may be under the control of a single entity or multiple entities. Furthermore, aspects can be triggered, initiated, or requested by different entities. In at least one embodiment, the training of a neural network could be instructed by a provider associated with a provider environment 506, while in at least one embodiment, the training could be requested by a client or other user who has access to a provider environment via a client device 502 or other such resource.In at least one embodiment, training data (or data to be analyzed by a trained neural network) can be provided by a provider, a user, or a third-party content provider 524. In at least one embodiment, client device 502 can be a vehicle or object that, for example, is to be navigated on behalf of a user, which can submit requests and / or receive instructions that assist in navigating a device. In at least one embodiment, requests can be submitted via at least one network 504 to be received by a provider environment 506. In at least one embodiment, a client device can be any suitable electronic and / or computing device that enables a user to generate and send such requests, such as, but not limited to, desktop computers, notebook computers, computer servers, smartphones, tablet computers, game consoles (portable or otherwise), computer processors, computing logic, and set-top boxes. Network(s) 504 can include any suitable network for transmitting a request or other such data, such as the Internet, an intranet, an Ethernet, a cellular network, a local area network (LAN), a wide area network (WAN), a personal area network (PAN), an ad-hoc network of direct wireless links between peers, and so on. In at least one embodiment, requests can be received at an interface layer 508, which in this example can forward data to a training and inference manager 532. The training and inference manager 532 can be a system or service that includes hardware and software for managing requests and service-related data or content. In at least one embodiment, the training and inference manager 532 can receive a request to train a neural network and provide data for a request to a training module 512. In at least one embodiment, the training module 512 can select a suitable model or neural network to use if it is not specified by the request and can train a model using relevant training data.In at least one embodiment, training data can be a batch of data stored in a training data repository 514, received by the client device 502, or obtained from a third-party provider 524. In at least one embodiment, the training module 512 can be responsible for providing the training data. A neural network can be any suitable network, such as a recurrent neural network (RNN) or a convolutional neural network (CNN). Once a neural network is trained and successfully evaluated, a trained neural network can be stored, for example, in a model repository 516, which can store different models or networks for users, applications, or services, etc. In at least one embodiment, there can be multiple models for a single application or entity, as they can be used based on a number of different factors. In at least one embodiment, at a subsequent time, a request can be received from client device 502 (or another such device) for content (e.g., path determinations) or data that is at least partially determined or influenced by a trained neural network. This request can, for example, include input data to be processed using a neural network to obtain one or more inferences or other output values, classifications, or predictions. Alternatively, in at least one embodiment, input data can be received through interface layer 508 and routed to inference module 518, although another system or service may also be used.In at least one embodiment, Inference Module 518 can retrieve a suitable trained network, such as a trained Deep Neural Network (DNN) as discussed herein, from Model Repository 516 if it is not already stored locally in Inference Module 518. Inference Module 518 can provide data as input for a trained network, which can then generate one or more inferences as output. This can, for example, include classifying an instance of input data. In at least one embodiment, inferences can then be transmitted to Client Device 502 for display or other communication with a user.In at least one embodiment, context data for a user can also be stored in a user context data repository 522, which may contain data about a user that can be useful as input to a network when generating inferences or determining data to return to a user after receiving instances. In at least one embodiment, relevant data, which may include at least some input or inference data, can also be stored in a local database 534 for processing future requests. In at least one embodiment, a user can use account information or other information to access resources or functionality of a provider environment.In at least one embodiment, if permitted and available, user data can also be collected and used to further train models to provide more accurate inferences for future requirements. In at least one embodiment, requests can be received through a user interface to a machine learning application 526 running on a client device 502, and results can be displayed through the same interface. A client device can include resources such as a processor 528 and memory 562 for generating a request and processing results or a response, as well as at least one data storage element 552 for storing data for the machine learning application 526. In at least one embodiment, a Processor 528 (or a processor from Training Module 512 or Inference Module 518) will be a central processing unit (CPU). As mentioned, however, resources in such environments can use GPUs to process data for at least certain types of requests. With thousands of cores, GPUs, such as the PPU 400, are designed to handle substantial parallel workloads and have thus become popular in deep learning for training neural networks and generating predictions. While using GPUs for offline builds has enabled faster training of larger and more complex models, generating predictions offline implies that either request-time input features cannot be used or predictions for all permutations of features must be generated and stored in a lookup table to serve real-time requests.If a deep learning framework supports a CPU mode and a model is small and simple enough to perform feed-forward on a CPU with reasonable latency, then a service could host the model on a CPU instance. In this case, training can be done offline on a GPU and inference can be performed in real time on a CPU. If a CPU-based approach is not practical, then a service can run on a GPU instance. However, since GPUs have different performance and cost characteristics than CPUs, running a service that offloads a runtime algorithm to a GPU may require a different design than a CPU-based service. In at least one embodiment, video data from client device 502 can be provided to provider environment 506 for enhancement. In at least one embodiment, video data can be processed on client device 502 for enhancement. In at least one embodiment, video data can be streamed from a third-party content provider 524 and enhanced by the third-party content provider 524, provider environment 506, or client device 502. In at least one embodiment, video data from client device 502 can be provided to provider environment 506 for use as training data. In at least one embodiment, supervised and / or unsupervised training can be performed by the client device 502 and / or the provider environment 506. In at least one embodiment, a set of training data 514 (e.g., classified or labeled data) is provided as input to serve as training data. In at least one embodiment, training data can include instances of at least one object type for which a neural network is to be trained, as well as information that identifies this object type. In at least one embodiment, training data can include a set of images, each representing an object type, with each image also including a label, metadata, classification, or other information that identifies, is associated with, or identifies an object type represented in that image. Various other data types can also be used as training data, such as text data, audio data, video data, and so on. In at least one embodiment, training data 514 is provided as training input for a training module 512. In at least one embodiment, the training module 512 can be a system or service that includes hardware and software, such as...one or more computing devices that execute a training application for training a neural network (or another model or algorithm, etc.). In at least one embodiment, training module 512 receives an instruction or request specifying a type of model to be used for training. In at least one embodiment, a model can be any suitable statistical model, network, or algorithm useful for such purposes, such as an artificial neural network, deep learning algorithm, learning classifier, Bayesian network, and so on. In at least one embodiment, training module 512 can select an initial model or other untrained model from a suitable repository 516 and use training data 514 to train a model, thereby creating a trained model (e.g.,a trained deep neural network) is generated, which can be used to classify similar data types or to generate other such inferences. In at least one embodiment where no training data is used, a suitable initial model for training input data can still be selected for each training module 512. In at least one embodiment, a model can be trained in a number of different ways, some of which may depend on the type of model selected. In at least one embodiment, a machine learning algorithm can be provided with a set of training data, where a model is a model artifact generated by a training process. In at least one embodiment, each instance of training data contains a correct response (e.g., classification), which can be referred to as a target or target attribute. In at least one embodiment, a learning algorithm finds patterns in training data that map input data attributes to a target, a predictable response, and outputs a machine learning model that captures these patterns. In at least one embodiment, a machine learning model can then be used to make predictions about new data for which a target is not specified. In at least one embodiment, the training and inference manager 532 can select from a set of machine learning models, including binary classification, multiclass classification, generative and regression models. In at least one embodiment, the type of model to be used can depend at least partially on the type of target to be predicted. GRAPHICS PROCESSING PIPELINE In one embodiment, the PPU 400 includes a graphics processing unit (GPU). The PPU 400 is configured to receive instructions that specify shader programs for processing graphics data. Graphics data can be defined as a set of primitives, such as points, lines, triangles, quadrilaterals, triangular strips, and the like. Typically, a primitive includes data specifying a number of vertices for the primitive (e.g., in a model space coordinate system) as well as attributes associated with each vertex of the primitive. The PPU 400 can be configured to process the graphics primitives to generate a frame buffer (e.g., pixel data for each pixel of the display). An application writes model data for a scene (e.g., a collection of vertices and attributes) to memory, such as system memory or memory 404. The model data defines each of the objects that can be visible on a display. The application then makes an API call to the driver kernel, requesting that the model data be rendered and displayed. The driver kernel reads the model data and writes commands to one or more streams to perform operations on the model data. These commands can reference various shader programs to be implemented on the processing units within the PPU 400, including one or more vertex shaders, envelope shaders, domain shaders, geometry shaders, and pixel shaders.For example, one or more of the processing units can be configured to run a vertex shader program that processes a number of vertices defined by the model data. In one embodiment, the different processing units can be configured to run different shader programs concurrently. For example, a first subset of processing units can be configured to run a vertex shader program, while a second subset of processing units can be configured to run a pixel shader program. The first subset of processing units processes vertex data to generate processed vertex data and writes the processed vertex data to the L2 cache and / or memory location 404. After the processed vertex data is rasterized (e.g.,After three-dimensional data has been transformed into two-dimensional data in screen space to produce fragment data, the second subset of processing units executes a pixel shader to produce processed fragment data. This fragment data is then mixed with other processed fragment data and written to the frame buffer in memory 404. The vertex shader program and the pixel shader program can run concurrently, processing different data from the same scene in a pipelined manner until all model data for the scene has been rendered into the frame buffer. The frame buffer contents are then transferred to a display controller for display on a display device. Images generated using one or more of the techniques disclosed herein can be displayed on a monitor or other display device. In some embodiments, the display device can be directly coupled to the system or processor that generates or renders the images. In other embodiments, the display device can be indirectly coupled to the system or processor, such as via a network. Examples of such networks include the internet, mobile telecommunications networks, a Wi-Fi network, and any other wired and / or wireless network system. When the display device is indirectly coupled, the images generated by the system or processor can be streamed to the display device via the network.Such streaming allows, for example, video games or other applications that render images to run on a server, data center, or cloud-based computing environment, and the rendered images to be transmitted and displayed on one or more user devices (such as a computer, video game console, smartphone, other mobile device, etc.) that are physically separate from the server or data center. Thus, the techniques disclosed here can be used to enhance the images being streamed and to improve services that stream images, such as NVIDIA GeForce Now (GFN), Google Stadia, and the like. EXEMPLARY STREAMING SYSTEM Fig. 6 is an exemplary system diagram for a streaming system 605 according to some embodiments of the present disclosure. Fig. 6 includes one or more servers 603 (which may include similar components, features, and / or functionality to the exemplary processing system 500 of Fig. 4 and / or the exemplary system 565 of Fig. 5A), one or more client devices 604 (which may include similar components, features, and / or functionality to the exemplary processing system 500 of Fig. 4 and / or the exemplary system 565 of Fig. 5A), and one or more networks 606 (which may be similar to the network(s) described herein). In some embodiments of the present disclosure, the system 605 may be implemented. In one embodiment, the streaming system 605 is a game streaming system and the server(s) 603 are one or more game servers. In the system 605, the client device(s) 604 for a game session can only receive input data in response to inputs to the input device(s) 626, transmit the input data to the server(s) 603, receive encoded display data from the server(s) 603, and display the display data on the display 624. As such, the more computationally intensive calculation and processing is offloaded to the server(s) 603 (e.g., rendering—in particular, ray tracing or path tracing—for graphical output of the game session is performed by the GPU(s) 615 of the server(s) 603). In other words, the game session is streamed from server(s) 603 to client device(s) 604, thereby reducing the graphics processing and rendering requirements of client device(s) 604. For example, with respect to instantiating a game session, a client device 604 can display a frame of the game session on the display 624 based on receiving display data from the server(s) 603. The client device 604 can receive input at one of the input devices 626 and generate input data in response. The client device 604 can transmit the input data to the server(s) 603 via the communication interface 621 and via the network(s) 606 (e.g., the Internet), and the server(s) 603 can receive the input data via the communication interface 618. The CPU(s) 608 can receive the input data, process the input data, and transmit data to the GPU(s) 615, which causes the GPU(s) 615 to render the game session.For example, the input data can be representative of a user character's movement in a game, firing a weapon, reloading, passing a ball, turning a vehicle, etc. The render component 612 can render the game session (e.g., representative of the result of the input data), and the render capture component 614 can capture the game session's rendering as display data (e.g., as image data capturing the rendered frame of the game session). The game session rendering can include ray- or path-traced lighting and / or shadow effects, calculated using one or more parallel processing units—such as GPUs, which may further utilize one or more dedicated hardware accelerators or processing cores to perform ray- or path-tracing techniques—of the server(s) 603.The encoder 616 can then encode the display data to generate coded display data, and the coded display data can be transmitted via the network(s) 606 to the client device 604 via the communication interface 618. The client device 604 can receive the coded display data via the communication interface 621, and the decoder 622 can decode the coded display data to generate the display data. The client device 604 can then display the display data via the display 624. It is noted that the techniques described herein may be embodied in executable instructions contained on a computer-readable medium for use by or in conjunction with a processor-based instruction execution machine, system, device, or apparatus. The person skilled in the art will recognize that for some embodiments, various types of computer-readable media for storing data may be included. As used herein, a “computer-readable medium” includes one or more suitable media for storing the executable instructions of a computer program, such that the instruction execution machine, system, device, or apparatus can read (or retrieve) the instructions from the computer-readable medium and execute the instructions to perform the described embodiments. Suitable storage formats include one or more of an electronic, magnetic, optical, and electromagnetic format.A non-exhaustive list of conventional examples of computer-readable media includes: a portable computer disk; a random access memory (RAM); a read-only memory (ROM); a erasable programmable read-only memory (EPROM); a flash memory device; and optical storage devices, including a portable compact disc (CD), a portable digital video disc (DVD), and the like. The arrangement of components illustrated in the accompanying figures serves for illustrative purposes, and other arrangements are possible. For example, one or more of the elements described herein may be realized wholly or partially as an electronic hardware component. Other elements may be implemented in software, hardware, or a combination of both. Furthermore, some or all of these other elements may be combined, some may be omitted entirely, and additional components may be added while still achieving the functionality described herein. Thus, the subject matter described herein may be embodied in many different variations, and it is considered that all such variations are within the scope of the claims. To facilitate understanding of the subject matter described herein, many aspects relating to sequences of actions are described. Various actions may be performed by specialized circuits or switching technology, by program instructions executed by one or more processors, or by a combination of both. The description of any sequence of actions herein is not intended to imply that the specific order described for performing that sequence must be followed. All procedures described herein may be performed in any suitable order unless otherwise specified herein or unless clearly contradicted by context. The use of the terms "a" and "an" and "the" and similar references in the context of describing the subject matter (particularly in the context of the following claims) shall be interpreted as covering both the singular and the plural, unless otherwise specified herein or clearly contradicted by context. The use of the term "at least one" followed by a list of one or more elements (for example, "at least one of A and B") shall be interpreted as meaning an element selected from the listed elements (A or B) or any combination of two or more of the listed elements (A and B), unless otherwise specified herein or clearly contradicted by context.Furthermore, the foregoing description serves only for illustrative purposes and not for limitation, since the scope of protection is defined by the claims, as set forth below, together with any equivalents thereof. The use of any and all examples or illustrative language (e.g., "such as") provided herein is intended only to better illustrate the subject matter and does not constitute a limitation of the scope of the subject matter unless otherwise claimed. The use of the term "based on" and other similar phrases indicating a condition for achieving a result, both in the claims and in the written description, is not intended to exclude any other conditions that bring about that result.No language in the patent specification should be interpreted as indicating any unclaimed element as essential for the execution of the claimed invention.

Claims

A method for encoding a video, wherein the method comprises: determining a weight parameter for a frame of the video by a trained reinforcement learning (RL) agent, wherein the weight parameter is determined based on encoding one or more previous frames of the video; generating, for a reference block of a frame of the video, a plurality of candidate prediction blocks; and selecting, using rate bias optimization (RDO) based on the determined weight parameter, a prediction block from the plurality of candidate prediction blocks. The method according to claim 1, further comprising: calculating, using the selected prediction block and the reference block, one or more remainders; and encoding the one or more remainders in a code stream. The method according to claim 1, wherein the RDO uses the determined weighting parameter to balance one or more rate and distortion components of a common loss function, and wherein the common loss function is represented using at least one of: J = D + (λf∗ λ) ∗ R, where D represents distortion, R represents rate, λf is the weighting parameter determined by the trained RL agent, and λ is an empirically determined Lagrange multiplier, or J = D + λfR, where D represents distortion, R represents rate, and λf is the weighting parameter determined by the trained RL agent. The method according to claim 1, wherein the results of encoding one or more previous frames of the video have encoding statistics that correspond to encoding one or more previously encoded video frames. The method according to claim 4, wherein the coding statistics include one or more of: a bitstream size, an average quantization parameter (QP) value for one or more frames, a count of intra-coded macroblocks, a count of inter-coded macroblocks, a number of slices, an image type, a sum of absolute transformed differences (SATD) for one or more frames, an average motion vector (MV) x-component, an average motion vector (MV) y-component, a count of 32x32 coding units (CUs) encoded using intra-prediction, a count of 32x32 CUs encoded using inter-prediction, a count of 16x16 CUs encoded using intra-prediction, or a count of 16x16 CUs encoded using Inter-prediction are encoded. The method according to claim 1, wherein the trained RL agent is a trained neural network comprising one or more layers of Long-Short-Term-Memory (LSTM) units. The method according to claim 1, wherein the trained RL agent is a neural network trained via proximal policy optimization (PPO). The method according to claim 1, wherein the RL agent is a neural network trained via reinforcement learning to maximize a reward based on coding performance. The method according to claim 8, wherein the reward is calculated using an advantage function that measures an advantage achieved by coding the frame using the weighting parameter compared to coding the frame without using the weighting parameter. The method according to claim 1, wherein the plurality of candidate prediction blocks is generated by an intra-frame predictor and an inter-frame predictor, wherein at least one of the plurality of candidate prediction blocks is predicted according to a combination of one or more prediction parameters, and wherein the combination of one or more prediction parameters includes at least one of the following: a prediction mode, a partition mode, a coding mode, a skipping mode, a reference frame selection, a chroma subsampling mode, or a strength of one or more filters. The method according to claim 1, wherein the encoding is performed using a standards-compliant video codec selected from the group of video codec standards consisting of: VP8, VP9, ​​AVC (h.264), HEVC (h.265), AV1, AV2 or VVC (h.266). A system for encoding a video, comprising: processing circuitry configured to: determine a weighting parameter for a frame of the video by a trained reinforcement learning (RL) agent, wherein the weighting parameter is determined based on results from encoding one or more previous frames of the video; generate, for a reference block of a frame of the video, a plurality of candidate prediction blocks; and select, using rate bias optimization (RDO) based on the determined weighting parameter, a prediction block from the plurality of candidate prediction blocks; and memory configured to store the frame and the plurality of candidate prediction blocks. The system according to claim 12, wherein the processing circuitry is further configured to: calculate, using the selected prediction block and the reference block, one or more remainders; and encode the one or more remainders in a code stream. The system according to claim 12, wherein the RDO uses the determined weighting parameter to balance one or more rate and distortion components of a common loss function, and wherein the common loss function is represented using at least one of: J = D + (λf∗ λ) ∗ R, where D represents distortion, R represents rate, λf is the weighting parameter determined by the trained RL agent, and λ is an empirically determined Lagrange multiplier, or J = D + λfR, where D represents distortion, R represents rate, and λf is the weighting parameter determined by the trained RL agent. The system according to claim 12, wherein the results of encoding from one or more previous frames of the video have encoding statistics that correspond to encoding from one or more previously encoded video frames. The system according to claim 15, wherein the coding statistics include one or more of: a bitstream size, an average quantization parameter (QP) value for one or more frames, a count of intra-coded macroblocks, a count of inter-coded macroblocks, a number of slices, an image type, a sum of absolute transformed differences (SATD) for one or more frames, an average motion vector (MV) x-component, an average motion vector (MV) y-component, a count of 32x32 coding units (CUs) encoded using intra-prediction, a count of 32x32 CUs encoded using inter-prediction, a count of 16x16 CUs encoded using intra-prediction, or a count of 16x16 CUs encoded using Inter-prediction are encoded. The system according to claim 12, wherein the trained RL agent is a neural network comprising one or more layers of long-short-term memory (LSTM) units and trained via reinforcement learning to maximize a reward based on coding performance. The system according to claim 17, wherein the reward is calculated using an advantage function that measures an advantage achieved by coding the frame using the weighting parameter compared to coding the frame without using the weighting parameter. A system comprising: one or more processing units for: determining a weighting parameter for a frame of a video, wherein the weighting parameter is determined based on results of coding from one or more previous frames of the video; generating, for a reference block of a frame of the video, a plurality of candidate prediction blocks; and selecting, using rate bias optimization (RDO) based on the determined weighting parameter, one prediction block from the plurality of candidate prediction blocks. The system according to claim 19, wherein the one or more processing units are included in at least one of the following: a system for performing simulation operations; a system for performing simulation operations for testing or validating autonomous machine applications; a system for performing digital twin operations; a system for performing light transport simulation; a system for rendering graphical output; a system for performing deep learning operations; a system implemented using an edge device; a system for generating or displaying virtual reality (VR) content; a system for generating or presenting augmented reality (AR) content; a system for generating or presenting mixed reality (MR) content; a system containing one or more virtual machines (VMs); a system that is at least partially implemented in a data center;a system for performing hardware tests using simulation; a system for generating synthetic data; a system for performing generative AI operations; a system implemented using one or more large language models (LLMs); a system implemented using one or more vision language models (VLMs); a collaborative content creation platform for 3D assets; or a system implemented at least partially using cloud computing resources.

Images