A reuse-centric client–server framework for real-time path tracing

Abstract

A method implemented by a server, comprising performing path tracing to generate radiance values for a subset of pixels of a frame; generating a binary mask identifying newly shaded pixels of the frame from the subset of pixels; determining, for the newly shaded pixels, delta-radiance values relative to a motion-compensated prior-frame radiance; encoding the binary mask and the delta-radiance values into a payload; and transmitting the payload to a client.

Description

Atty. Docket: 4502-90001 (6000809PCT02)A Reuse-Centric Client-Server Framework for Real-Time Path TracingCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U. S. Provisional Application No. 63 / 939,484 filed on December 12, 2025 and U. S. Provisional Application No. 63 / 822,767 filed on June 12, 2025, which are hereby incorporated by reference.TECHNICAL FIELD

[0002] The present disclosure describes techniques for real-time photorealistic rendering. More specifically, this disclosure is related to hybrid sample and shading reuse techniques and distributed sample reuse and shading reuse across heterogeneous computing devices for accelerating path tracing in dynamic scenes.BACKGROUND

[0003] Physically based path tracing is gaining prominence in photorealistic rendering due to its ability to generate images with high accuracy and realism. However, its intensive computation burden makes it challenging for real-time and interactive applications.SUMMARY

[0004] The disclosure relates to computer graphics rendering systems, including methods, systems, and computer-readable media for performing real-time physically based path tracing using spatiotemporal sample reuse and adaptive shading reuse. More specifically, the disclosure pertains to a hybrid sample and shading reuse (HyShare) algorithm that integrates Reservoir-based Spatiotemporal Importance Resampling (ReSTIR)- style path sample reuse with adaptive shading reuse across spatial and temporal domains, including scheduling, validation, and motion- sensitive control mechanisms implemented in graphics processing unit (GPU)-based rendering pipelines.

[0005] In some embodiments, the disclosure further pertains to a distributed, reuse-centric client-server framework for real-time path tracing, referred to as distributed HyShare (D-HyShare). D-HyShare extends the hybrid sample-shading reuse formulation of HyShare to a distributed architecture that leverages both ReSTIR-based sample reuse and temporally validated shading reuse across heterogeneous machines. The system fundamentally addresses the bandwidthAtty. Docket: 4502-90001 (6000809PCT02)bottleneck of client / server photorealistic rendering by reducing the total pixel transmission from a full-resolution frame to a sparse subset of newly sampled pixels.

[0006] A first aspect relates to a method implemented by a server, comprising performing path tracing to generate radiance values for a subset of pixels of a frame; generating a binary mask identifying newly shaded pixels of the frame from the subset of pixels; determining, for the newly shaded pixels, delta-radiance values relative to a motion-compensated prior-frame radiance; encoding the binary mask and the delta-radiance values into a payload; and transmitting the payload to a client.

[0007] Optionally, in any of the preceding aspects, another implementation of the aspect provides performing the path tracing comprises maintaining per-pixel sample reservoirs across successive frames to generate the radiance values.

[0008] Optionally, in any of the preceding aspects, another implementation of the aspect provides performing the path tracing comprises executing reservoir-based spatiotemporal importance resampling.

[0009] Optionally, in any of the preceding aspects, another implementation of the aspect provides determining the subset of pixels based on a cycling shading reuse schedule, wherein the cycling shading reuse schedule selects one pixel in each 2x2 pixel group per frame for path tracing and designates remaining pixels for shading reuse, and wherein at keyframes occurring at periodic intervals, the server performs the path tracing for all pixels of the frame.

[0010] Optionally, in any of the preceding aspects, another implementation of the aspect provides validating pixels designated for shading reuse based on reuse eligibility criteria, wherein the reuse eligibility criteria comprise at least one of an occlusion condition, a radiance change condition, or a motion condition; and reclassifying pixels failing the reuse eligibility criteria as newly shaded pixels.

[0011] Optionally, in any of the preceding aspects, another implementation of the aspect provides determining the delta-radiance values comprises determining the motion-compensated prior-frame radiance by back-projecting pixel locations to a prior frame; and subtracting the motion-compensated prior-frame radiance from corresponding current radiance values of the newly shaded pixels.

[0012] Optionally, in any of the preceding aspects, another implementation of the aspect provides encoding the binary mask and the delta-radiance values comprises quantizing the delta-Atty. Docket: 4502-90001 (6000809PCT02)radiance values to obtain quantized delta-radiance values; packing the quantized delta-radiance values corresponding to the newly shaded pixels into a one-dimensional buffer; and compressing the one-dimensional buffer and the binary mask to form the payload.

[0013] Optionally, in any of the preceding aspects, another implementation of the aspect provides quantizing the delta-radiance values comprises applying a uniform linear quantizer mapping the delta-radiance values into a signed 8-bit range.

[0014] Optionally, in any of the preceding aspects, another implementation of the aspect provides receiving, from the client, a camera state packet comprising a frame index, a timestamp, a camera pose, and projection parameters.

[0015] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the binary mask is arranged for raster-order traversal.

[0016] Optionally, in any of the preceding aspects, another implementation of the aspect provides performing the path tracing, encoding, and transmitting are executed asynchronously using a plurality of threads.

[0017] Optionally, in any of the preceding aspects, another implementation of the aspect provides operating a rendering thread, an encoding thread, and a transmission thread connected via queues in a producer-consumer model.

[0018] Optionally, in any of the preceding aspects, another implementation of the aspect provides selecting, for processing, a most recent camera state packet received from the client and discarding one or more earlier unprocessed camera state packets.

[0019] A second aspect relates to a method implemented by a client, comprising receiving, from a server, a payload comprising a binary mask and delta-radiance values associated with a subset of pixels of a frame; decoding the payload to recover the binary mask and the delta-radiance values; determining motion-compensated prior- frame radiance for pixels of the frame; reconstructing a full-resolution frame by applying the delta-radiance values to pixels identified as newly shaded based on the binary mask and reusing the motion-compensated prior-frame radiance for pixels not identified as newly shaded; and displaying the full-resolution frame.

[0020] Optionally, in any of the preceding aspects, another implementation of the aspect provides reconstructing the frame comprises, for each pixel identified as newly shaded, adding a corresponding delta-radiance value to the motion-compensated prior-frame radiance.Atty. Docket: 4502-90001 (6000809PCT02)

[0021] Optionally, in any of the preceding aspects, another implementation of the aspect provides reconstructing the frame comprises, for each pixel not identified as newly shaded, setting a radiance value of that pixel equal to the motion-compensated prior-frame radiance.

[0022] Optionally, in any of the preceding aspects, another implementation of the aspect provides determining the motion-compensated prior-frame radiance comprises back-projecting a pixel location to a prior frame using a motion vector; and performing bilinear interpolation using neighboring pixel samples in the prior frame.

[0023] Optionally, in any of the preceding aspects, another implementation of the aspect provides storing prior-frame radiance, motion vectors, depth, and normal information computed locally from synchronized camera for use in motion-compensated reconstruction.

[0024] Optionally, in any of the preceding aspects, another implementation of the aspect provides performing temporal accumulation across successive reconstructed frames.

[0025] Optionally, in any of the preceding aspects, another implementation of the aspect provides generating a predicted frame using motion-compensated reprojection in response to delayed receipt of the payload.

[0026] Optionally, in any of the preceding aspects, another implementation of the aspect provides updating a locally maintained velocity buffer using received synchronization data.

[0027] Optionally, in any of the preceding aspects, another implementation of the aspect provides transmitting, to the server, a camera state packet comprising a frame index, a timestamp, a camera pose, and projection parameters.

[0028] Optionally, in any of the preceding aspects, another implementation of the aspect provides decoding and reconstructing are performed asynchronously using a plurality of threads.

[0029] Optionally, in any of the preceding aspects, another implementation of the aspect provides operating a receive thread, a decode thread, a frame reconstruction thread, and a display thread connected via queues.

[0030] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the binary mask indicates newly shaded pixels using a first value and shading-reused pixels using a second value.

[0031] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the delta-radiance values are stored in a one-dimensional buffer that is sequentially consumed in correspondence with raster-order traversal of the binary mask.Atty. Docket: 4502-90001 (6000809PCT02)

[0032] A third aspect relates to a server comprising one or more processors and a memory storing instructions that, when executed, cause the server to perform path tracing to generate radiance values for a subset of pixels of a frame: generate a binary mask identifying newly shaded pixels of the frame from the subset of pixels; determine, for the newly shaded pixels, delta-radiance values relative to a motion-compensated prior-frame radiance; encode the binary mask and the delta-radiance values into a payload; and transmit the payload over a network; and a client device communicatively coupled to the server and comprising one or more processors and a memory storing instructions that, when executed, cause the client device to receive the payload; decode the binary mask and the delta-radiance values; determine a motion-compensated prior-frame radiance for pixels of the frame; reconstruct a full-resolution frame by applying the delta-radiance values to pixels identified as newly shaded based on the binary mask and reusing the motion-compensated prior-frame radiance for pixels not identified as newly shaded; and display the full-resolution frame.

[0033] A fourth aspect relates to a system comprising a memory configured to store instructions; and one or more processors coupled to the memory and configured to execute the instructions to cause the system to perform the method in any of the disclosed embodiments.

[0034] A fifth aspect relates to a computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computerexecutable instructions when executed by one or more processors of a system, cause the system to perform the method in any of the disclosed embodiments.

[0035] A sixth aspect relates to a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a system to perform operations in any of the disclosed embodiments.

[0036] For the purpose of clarity, any one of the foregoing embodiments may be combined with any one or more of the other foregoing embodiments to create a new embodiment within the scope of the present disclosure.

[0037] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.

[0038] For the purpose of clarity, any one of the foregoing embodiments may be combined with any one or more of the other foregoing embodiments to create a new embodiment within the scope of the present disclosure.Atty. Docket: 4502-90001 (6000809PCT02)

[0039] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0040] For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0041] FIG. 1 is a block diagram of the HyShare rendering pipeline according to an embodiment of the present disclosure.

[0042] FIG. 2 is a diagram illustrating spatial and temporal sample reuse invalidation scenarios according to an embodiment of the present disclosure.

[0043] FIG. 3 is a diagram illustrating a two-level reuse scheduling strategy according to an embodiment of the present disclosure.

[0044] FIG. 4 is a diagram illustrating a shading reuse history check mechanism according to an embodiment of the present disclosure.

[0045] FIG. 5 is a diagram illustrating an occlusion scenario in a dynamic scene according to an embodiment of the present disclosure.

[0046] FIG. 6 is a diagram illustrating floating-point motion vector backprojection and bilinear interpolation for shading reuse according to an embodiment of the present disclosure.

[0047] FIG. 7 is a diagram illustrating side-by-side visual comparison of hybrid reuse method relative to reservoir-based baselines according to an embodiment of the present disclosure.

[0048] FIG. 8 is a diagram illustrating a comparative image set of the importance and effectiveness of the shading reuse history check according to an embodiment of the present disclosure.

[0049] FIG. 9 is a diagram illustrating a comparative image set of the effectiveness of occlusion check according to an embodiment of the present disclosure.

[0050] FIG. 10 is a diagram illustrating a comparative image set of the impact of applying radiance and color variation check according to an embodiment of the present disclosure.

[0051] FIGS. 11 A- 11C are diagrams illustrating the impact of a weighing factor values on image quality according to an embodiment of the present disclosure.

[0052] FIG. 12 is a diagram illustrating a visual comparison of dynamically rendered framesAtty. Docket: 4502-90001 (6000809PCT02)from HyShare and corresponding ReSTIR for direct illumination (DI) baseline according to an embodiment of the present disclosure.

[0053] FIG. 13 is a diagram illustrating a visual comparison of dynamically rendered frames from HyShare and corresponding baseline ReSTIR for path tracing (PT) baseline according to an embodiment of the present disclosure.

[0054] FIG. 14 is a diagram illustrating a visual comparison of dynamically rendered frames from HyShare and corresponding baseline method area ReSTIR baseline according to an embodiment of the present disclosure.

[0055] FIGS. 15A-15C are diagrams illustrating a set of plots comparing rendering speed in frames per second (FPS) over frame number between HyShare and the baseline approaches according to an embodiment of the present disclosure.

[0056] FIGS. 16A-16D are diagrams illustrating a convergence assessment showing accumulated static-scene renderings and mean absolute percentage error (MAPE) plots according to an embodiment of the present disclosure.

[0057] FIG. 17 is a diagram illustrating a bar chart comparing peak GPU memory usage according to an embodiment of the present disclosure.

[0058] FIG. 18 is a flowchart according to an embodiment of the present disclosure.

[0059] FIG. 19 is a block diagram of distributed client-server rendering framework according to an embodiment of the present disclosure.

[0060] FIG. 20 is a diagram illustrating data communication between client and server framework according to an embodiment of the present disclosure.

[0061] FIG. 21 is a diagram illustrating packing of newly shaded pixels using a binary mask and a one-dimensional radiance buffer according to an embodiment of the present disclosure.

[0062] FIG. 22 is a diagram illustrating motion-compensated prediction and delta-radiance generation according to an embodiment of the present disclosure.

[0063] FIG. 23 is a diagram illustrating asynchronous multi-threaded execution of the distributed rendering pipeline according to an embodiment of the present disclosure.

[0064] FIG. 24 is a diagram illustrating a visual comparison of server- and client- rendered frames using D-HyShare according to an embodiment of the present disclosure.

[0065] FIGS. 25A-25F are diagrams illustrating a set of per-frame FPS and bandwidth traces for six scenes according to an embodiment of the present disclosure.Atty. Docket: 4502-90001 (6000809PCT02)

[0066] FIGS. 26A-26F are diagrams illustrating a set of per-frame network latency for six test scenes according to an embodiment of the present disclosure.

[0067] FIGS. 27A-27B are diagrams illustrating a chart showing prediction loss trend for multiple consecutive frames according to an embodiment of the present disclosure.

[0068] FIG. 28 is a diagram illustrating a qualitative panel showing predicted frames and corresponding error maps according to an embodiment of the present disclosure.

[0069] FIG. 29 is a flowchart according to an embodiment of the present disclosure.

[0070] FIG. 30 is a flowchart according to an embodiment of the present disclosure.

[0071] FIG. 31 is a diagram illustrating an apparatus according to an embodiment of the present disclosure.DETAILED DESCRIPTION

[0072] It should be understood at the outset that although an illustrative implementation of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

[0073] The disclosure relates to computer graphics rendering systems, including methods, systems, and computer-readable media for performing real-time physically based path tracing using spatiotemporal sample reuse and adaptive shading reuse. More specifically, the disclosure pertains to a HyShare algorithm that integrates ReSTIR-style path sample reuse with adaptive shading reuse across spatial and temporal domains, including scheduling, validation, and motionsensitive control mechanisms implemented in GPU-based rendering pipelines.

[0074] The disclosed framework provides a technical solution to a technical problem via unified architecture that implements a hybrid sample and shading reuse algorithm, a sample reservoir dormancy mechanism, and / or a per-pixel motion- sensitive shading control mechanism. As a practical application, the unified framework is configured to address the increased complexity associated with hybrid reuse, reduce visual artifacts, enhance robustness, and optimize reuse scheduling via one or more of the embodiments discussed in detail herein.Atty. Docket: 4502-90001 (6000809PCT02)

[0075] Hyshare Framework and related work

[0076] Physically based path tracing is gaining prominence in photorealistic rendering due to its ability to generate images with high accuracy and realism. However, its intensive computational burden makes it challenging to deploy in real-time and interactive applications. Recent advances in spatiotemporal reuse techniques, such as ReSTIR and related extended methods, have significantly improved sampling efficiency for complex real-time lighting scenarios. Nevertheless, despite these advances, existing approaches often focus exclusively on either path-tracing sample reuse or shading reuse, thereby limiting the ability to fully exploit reusable information within the rendering process.

[0077] In the category of sample reuse, ReSTIR introduced spatiotemporal reuse for direct illumination based on reservoir sampling and demonstrated a significant reduction in rendering error through unbiased reuse of nearby samples. However, this approach relies on screen-space correlations and is primarily limited to single-bounce lighting, which may degrade image quality in regions with disocclusions. lighting discontinuities, or complex geometry.

[0078] More recent developments extend the reservoir-based concept to indirect lighting and multi-bounce paths. For example, approaches such as ReSTIR for global illumination (GI) improve rendering performance in indoor scenes, but their effectiveness may diminish under conditions such as rapid camera motion or the presence of glossy surfaces, which can introduce noise and sample correlation artifacts. Additional techniques such as ReSTIR PT provide efficient multi-bounce global illumination and enable correlated sample reuse. However, these methods may still suffer from reservoir staleness in dynamic scenes. Other extensions, including approaches that incorporate sub-pixel sampling for anti-aliasing, improve certain aspects of image quality but remain spatially limited during reuse and may not address color channel variance.

[0079] Although these techniques provide improvements in sampling efficiency, they share a common limitation. In particular, shading operations are typically performed for every sample accepted by the reuse process, even when shading results themselves could potentially be reused to avoid redundant computations. Consequently, the number of samples that can be allocated per pixel remains limited, particularly in real-time or high-performance rendering environments where shading costs represent a primary computational bottleneck. This limitation may result in degraded image quality, such as increased noise or rendering artifacts.Atty. Docket: 4502-90001 (6000809PCT02)

[0080] Shading reuse techniques have been explored in rasterization-based rendering pipelines; however, such techniques remain difficult to integrate directly with reservoir-based sampling approaches and Monte Carlo path tracing methods. Path tracing approaches generally provide higher rendering fidelity than rasterization-based techniques, but their computational requirements remain significant.

[0081] In this disclosure, a hybrid sample and shading reuse algorithm is disclosed, referred to herein as HyShare. The proposed approach combines the advantages of both sample reuse and shading reuse techniques. In particular, the method employs ReSTIR-style sample reuse while selectively and adaptively applying shading reuse across both spatial and temporal dimensions.

[0082] One challenge associated with hybrid reuse is the reservoir dormancy problem. When shading results are reused without corresponding updates to the associated reservoirs, the reservoirs may become stale and may no longer receive updated samples, which can degrade reuse quality. To address this issue, mechanisms are introduced to identify dormant reservoirs, avoid reuse from inactive pixels, and periodically refresh shading and sample reservoirs using hybrid scheduling strategies designed to balance efficiency and rendering quality.

[0083] Furthermore, the proposed method incorporates motion-sensitive mechanisms, including radiance-change detection and depth-change detection, to selectively pause shading reuse in highly dynamic regions. This mechanism helps prevent visual artifacts such as ghosting and edge aliasing. Compared with temporally adaptive approaches, the proposed method provides more granular control by detecting radiance variation and depth change within pixel groups and selectively halting shading reuse to better handle motion-induced artifacts.

[0084] Experimental evaluations indicate that the proposed hybrid reuse approach can achieve significant improvements in rendering efficiency while also improving image quality, including reductions in noise, aliasing, and temporal artifacts.

[0085] The main contributions of the disclosed techniques include: (1) A hybrid reuse approach that combines ReSTIR-style sample reuse with adaptive shading reuse across spatial and temporal dimensions on-the-fly, without waiting for completion of shading for the current frame, thereby improving rendering efficiency and image quality, (2) A mechanism for addressing sample reservoir dormancy caused by shading reuse, including two-level reuse scheduling and per-pixel tagging strategies to validate reusable reservoirs, and (3) A per-pixel motion- sensitive shadingAtty. Docket: 4502-90001 (6000809PCT02)control mechanism based on radiance change and depth change detection to improve robustness against motion-induced artifacts.

[0086] ReSTIR and related Extensions

[0087] Reservoir-based spatiotemporal importance resampling techniques have emerged as promising approaches for improving the efficiency of real-time rendering. One example of such techniques introduces the use of per-pixel reservoirs to resample light contributions across both spatial and temporal dimensions based on importance metrics. This approach enables reuse of samples generated in previous frames or neighboring pixels, resulting in substantial reductions in variance for direct illumination computations. While highly efficient, such approaches are typically limited to single-bounce lighting and rely on screen-space reservoirs. Consequently, rendering quality may degrade in scenarios involving disocclusions, scene motion, or complex geometry.

[0088] Subsequent techniques extend the reservoir-based concept to support more complex light transport. For example, certain approaches incorporate multi-bounce indirect illumination into the sampling framework, thereby enabling improved global illumination results in indoor or enclosed environments. However, these extensions may experience increased noise under conditions such as rapid camera motion and may struggle with specular or glossy surfaces. Such limitations may arise from sample correlation and path incoherence during reuse.

[0089] Further developments introduce generalized resampled importance sampling frameworks that formalize reuse of entire light paths using shift mappings. In such approaches, path tracing frameworks are extended to enable reuse of path segments across both spatial and temporal domains. For example, some techniques utilize a single path per pixel and reuse path segments from neighboring pixels or previous frames, thereby improving rendering efficiency while maintaining unbiased sampling. These approaches demonstrate that full path reuse can be applied even for complex multi-bounce illumination scenarios, enabling high-quality rendering at interactive frame rates. However, such techniques may remain susceptible to sample correlation artifacts and visible noise, and often require careful management of path reuse, particularly in dynamic scenes or scenes containing glossy materials.

[0090] Other extensions expand the spatial sampling domain by incorporating sub-pixel sampling and integrating ray differences originating from the lens and sensor. Such techniques support rendering effects including anti-aliasing and depth of field. While these methods enrichAtty. Docket: 4502-90001 (6000809PCT02)the spatial sampling domain, they typically apply filtering primarily in temporal reuse stages and may overlook per-channel sampling variance, which may result in color noise.

[0091] Additional variants introduce conditional sampling techniques that enable unbiased multi-bounce path resampling by performing reuse operations within conditional probability spaces. Such approaches may mitigate sample correlation artifacts by delaying reuse along light paths; however, they may introduce additional computational overhead due to operations such as spatial searches over hierarchical acceleration structures.

[0092] Other methods incorporate photon mapping or related light transport techniques into reservoir-based reuse frameworks to improve rendering of caustics. Additional improvements have been proposed to reduce energy loss during resampling through optimized weighting strategies or visibility-aware sampling. Some techniques also attempt to reduce bias by restricting reuse to samples that exhibit similar probability density functions.

[0093] Despite the effectiveness of these techniques, many reservoir-based resampling approaches share a common limitation. In particular, shading computations are typically performed for every sample accepted during the reuse process. As a result, shading operations often become the dominant computational cost in real-time rendering systems, particularly when large numbers of samples must be evaluated per pixel.

[0094] Shading Reuse

[0095] Shading reuse techniques aim to reduce redundant shader invocations by reusing previously computed radiance values, often across multiple frames or across neighboring pixels. In some implementations, previously computed shading values may be reused for special effects such as anti-aliasing, motion blur, or post-processing operations including denoising. Such approaches exploit temporal coherence in rendered images to reduce the number of shading computations required for real-time rendering.

[0096] In rasterization-based rendering pipelines, certain techniques utilize shading gradients, motion vectors, or luminance changes to guide reuse decisions. These approaches can significantly reduce the number of shader invocations while maintaining perceptual image quality. In ray tracing systems, other techniques decouple shading computations from geometric sampling operations. For example, shading grids or nearest-neighbor searches in object space may be used to reuse shading results across visibility samples or anti-aliasing samples. Additionally, multi-view rayAtty. Docket: 4502-90001 (6000809PCT02)tracing techniques may detect silhouette edges or discontinuities and share shading computations across multiple views along a camera path while maintaining visibility constraints.

[0097] However, integrating shading reuse with advanced sampling reuse frameworks, such as reservoir-based sampling techniques used in path tracing, presents several challenges. In particular, maintaining unbiased sampling while ensuring that sample reservoirs remain representative of the current scene state can be difficult when shading results are reused. For example, reuse of shading results may prevent reservoirs from receiving updated samples, resulting in stale sample sets that degrade the quality of spatiotemporal sample reuse. Such conditions may introduce visual artifacts including ghosting, incorrect illumination, or temporal instability, particularly in dynamic scenes.

[0098] Hybrid Reuse Approaches

[0099] While sample reuse techniques and shading reuse techniques have been studied independently, their integration into a unified rendering framework remains relatively limited. Combining shading reuse with reservoir-based sample reuse can be challenging due to dependencies between sampling validity, shading freshness, and reuse eligibility. For example, reuse of shading results may compromise the statistical validity of sample reservoirs, thereby requiring mechanisms to manage reservoir activity, shading update frequency, and motion handling in dynamic scenes. Some hybrid rendering approaches employ alternative strategies, such as combining neural-network-based shading approximations with sampling reuse techniques for dynamic scene elements. However, such approaches often rely on substantially different mechanisms and may not directly address the interaction between reservoir-based sampling and shading reuse.

[0100] Accordingly, embodiments described herein provide a hybrid reuse framework that jointly leverages spatiotemporal sample reuse and selective shading reuse. In some implementations, mechanisms such as reservoir tagging, motion- sensitive shading control, and periodic shading updates through scheduling strategies may be used to maintain reservoir freshness while reducing shading costs. These mechanisms enable adaptive trade-offs between rendering performance and image quality.

[0101] Through the combination of adaptive sample reuse and selective shading reuse, the disclosed framework can significantly reduce shading overhead while preserving rendering fidelity.Atty. Docket: 4502-90001 (6000809PCT02)Such techniques may be particularly advantageous in real-time rendering environments, including interactive graphics applications such as games, simulations, and virtual reality systems.

[0102] Hybrid Sample And Shading Reuse Framework

[0103] The disclosure describes the design and implementation of the proposed HyShare approach, which recognizes and unifies sample reuse with shading reuse across both spatial and temporal dimensions. The objective of the approach is to further improve real-time path tracing performance while preserving high image quality. The disclosed framework provides a unified architecture together with several mechanisms configured to address the increased complexity associated with hybrid reuse, reduce visual artifacts, enhance robustness, and optimize reuse scheduling.

[0104] Hybrid reuse presents several challenges because sample reuse and shading reuse interact with and influence each other in non-trivial ways. Simply combining these two reuse strategies may lead to problems such as reservoir staleness, strong sample correlations, redundant sampling, or visual artifacts. These issues may become particularly significant in scenes containing dynamic elements, such as motion of objects or camera movement.

[0105] To address these challenges, the disclosed framework provides a unified system that explicitly tracks reuse eligibility at a pixel level, monitors reuse state information, and coordinates reuse scheduling across pixels. In some embodiments, the framework maintains per-pixel metadata that indicates whether shading reuse or sample reuse should be applied during a given rendering iteration. In some embodiments, reuse is adaptively disabled in motion-sensitive regions based on radiance and geometric change detection. Experimental results on complex dynamic scenes demonstrate that HyShare outperforms state-of-the-art methods, including ReSTIR DI, ReSTIR PT, and Area ReSTIR, achieving a 37.4% improvement in rendering speed along with enhanced image quality (PSNR +1.8 dB, SSIM +0.17).

[0106] A fundamental concept of the HyShare framework is that, for pixels determined to be eligible for shading reuse, shading reuse is applied instead of sample reuse, since reusing shading results can provide a more computationally efficient method of reducing rendering cost. For pixels that are not eligible for shading reuse, the framework performs path-tracing sample reuse by leveraging reservoir-based sampling techniques. Such techniques may include resampled importance sampling approaches similar to those used in prior reservoir-based sampling frameworks.Atty. Docket: 4502-90001 (6000809PCT02)

[0107] By selectively applying shading reuse or sample reuse based on per-pixel eligibility, the framework enables efficient reuse of both shading results and sampling information while mitigating issues such as reservoir staleness and sampling artifacts. This unified approach enables improved rendering efficiency while maintaining high-quality path-traced images.

[0108] FIG. 1 is a block diagram of a Hy Share rendering pipeline 100 according to an embodiment of the present disclosure. The HyShare rendering pipeline 100 integrates shading reuse into a reservoir-based path tracing framework. In some embodiments, the pipeline extends the ReSTIR framework by incorporating mechanisms that selectively reuse historical shading data for a subset of pixels while maintaining parallel execution with sample reuse pixels. As shown in FIG. 1, the HyShare rendering pipeline 100 begins at start module 102, which represents initialization of a rendering process for a current frame. At this stage, frame-level resources such as pixel buffers, motion vector buffers, and shading history buffers may be prepared.

[0109] The HyShare rendering pipeline 100 further proceeds to a pixel group scheduler 104 configured to divide pixels of the current frame into two processing groups according to a scheduling strategy (e.g., the 3-out-of-4 shading reuse strategy). In some embodiments, the pixel group scheduler 104 partitions the pixels into two groups: pixels designated for path tracing and sample reuse, and pixels designated for shading reuse. This scheduling mechanism ensures that all pixels periodically execute path tracing operations to refresh their corresponding sample reservoirs.

[0110] The HyShare rendering pipeline 100 further proceeds to a reuse eligibility check module 106 configured to process the pixels scheduled for shading reuse to determine whether shading reuse is valid for a given pixel. The reuse eligibility check module 106 may evaluate one or more reuse conditions including motion magnitude, depth variation, radiance variation, or other scene-change indicators. If the reuse conditions are satisfied, the pixel proceeds through the shading reuse processing path. If the reuse conditions are not satisfied, the pixel may instead be redirected to the path tracing and sample reuse path to ensure accurate shading computation.

[0111] The HyShare rendering pipeline 100 further proceeds to a path tracing and sample reuse module 108 configured to process the pixels selected for rendering operations. In this stage, light transport paths are generated and evaluated using path tracing techniques. Candidate lighting samples may be processed using resampled importance sampling mechanisms, including resampled importance sampling (RIS) and weighted reservoir sampling (WRS). These operations maintain per-pixel reservoirs of candidate light samples and allow reuse of samples across spatialAtty. Docket: 4502-90001 (6000809PCT02)and temporal dimensions. As a result, new lighting contributions are accumulated and the corresponding sample reservoirs are refreshed.

[0112] The HyShare rendering pipeline 100 further proceeds to a shading reuse and accumulation module 110 configured to process the pixels selected for shading reuse. In this module, path tracing operations are bypassed and previously computed shading values are reused. The reused shading values may be accumulated using temporal filtering techniques. In some embodiments, an adaptive exponential smoothing process is applied to combine historical shading values with accumulated values to maintain temporal consistency and reduce flickering artifacts.

[0113] The HyShare rendering pipeline 100 further proceeds to a motion compensation module 112. To obtain the appropriate historical shading values, the motion compensation module 112 determines the corresponding pixel position in a previous frame. Motion vectors associated with the current pixel may be used to compute a back-projected pixel location. In some implementations, subpixel- accurate motion vectors are used to improve reprojection accuracy.

[0114] The HyShare rendering pipeline 100 further proceeds to a shading history buffer 114. The motion-compensated pixel location is used to retrieve previously computed shading data from the shading history buffer 114. The shading history buffer stores radiance values or other shading outputs generated in previous frames. These stored values may be used as inputs to the shading reuse and accumulation module.

[0115] The HyShare rendering pipeline 100 further proceeds to a reuse output merger 116 configured to combine outputs from the path tracing and sample reuse module 108 and the shading reuse and accumulation module 110. The reuse output merger 116 integrates shading-reused pixels and sample-reused pixels into a unified full-resolution image.

[0116] The HyShare rendering pipeline 100 further proceeds to a display module 118 configured to display the merged frame and output the rendered image to a display device or to subsequent rendering stages.

[0117] Through the coordinated operation of modules 102-118, the disclosed HyShare rendering pipeline 100 selectively applies shading reuse while preserving the sampling and resampling mechanisms of the underlying path tracing framework. This hybrid reuse strategy improves rendering efficiency by reducing expensive shading computations while maintaining high rendering quality and robustness in dynamic scenes.

[0118] FIG. 2 is a diagram 200 illustrating spatial and temporal sample reuse invalidationAtty. Docket: 4502-90001 (6000809PCT02)scenarios according to an embodiment of the present disclosure. In the left portion of FIG. 2, spatial sample reuse 202 among neighboring pixels within the same frame is illustrated. A pixel p may attempt to reuse samples from a neighboring pixel p'. However, if the neighboring pixel is designated for shading reuse rather than path tracing sample generation, the reservoir associated with pixel is not updated in the current frame. In the right portion of FIG. 2, temporal sample reuse 204 from pixels in a previous frame is illustrated. When a pixel p was previously designated for shading reuse in the preceding frame, the reservoir associated with that pixel was not refreshed through path tracing. As a result, samples stored in the reservoir of that pixel become stale and cannot be validly reused in a subsequent frame. These situations illustrate the problem of reservoir dormancy in a hybrid reuse framework.

[0119] Reservoir dormancy occurs when a pixel repeatedly performs shading reuse without refreshing its ReSTIR reservoir through path tracing, thereby preventing the reservoir from receiving updated samples. Over time, this results in outdated or invalid sample data that can degrade rendering quality. Accordingly, a reuse scheduler is employed to mitigate reservoir dormancy by managing when pixels perform sample reuse and when reservoirs are refreshed. Unlike another ReSTIR-based rendering approaches in which each pixel’s reservoir is refreshed during every frame through path tracing, the disclosed hybrid reuse framework selectively schedules path tracing and shading reuse operations to maintain reservoir validity while improving computational efficiency.

[0120] FIG. 3 is a diagram 300 illustrating a two-level reuse scheduling strategy according to an embodiment of the present disclosure. In some embodiments, to mitigate reservoir dormancy, a two-level reuse scheduling strategy is employed to ensure that each pixel periodically receives new path-traced samples to prevent reservoir staleness.

[0121] Key Frame Scheduling

[0122] In some embodiments, specific key frames are designated in which all pixels perform sample reuse through complete ReSTIR path tracing. The key frames occur at regular intervals (for example, every 4N + Iframes). These key frames serve as periodic reset points that force a global refresh of reservoir states, thereby mitigating sample staleness.

[0123] Non-Key Frame Scheduling

[0124] In some embodiments, between key frames, 4N non-key frames follow a cyclic shading reuse schedule. In one cycle (shown by the dashed pattern in FIG. 3), one pixel in eachAtty. Docket: 4502-90001 (6000809PCT02)2 x 2 pixel group performs sample reuse, while the remaining three pixels perform shading reuse. Across four consecutive non-key frames, all four pixels within the group are updated once via sample reuse. With this scheduling scheme, every pixel is refreshed at least once per cycle. The number of cycles is determined by N. In experimental implementations, N may be empirically set within the range of [3. 8] for most dynamic scenes. Let St(p) indicate if pixel p performs sample reuse at frame t as:1, if t mod (41V + 1) = 0St (p) = 1, if selected in its 2 x 2 groupto, otherwise.

[0125] In some embodiments, a pixel is refreshed at least once every four frames, ensuring bounded dormancy.

[0126] Even Spatial Distribution

[0127] In some embodiments, to avoid block-like visual artifacts, sample reuse pixels and shading reuse pixels are evenly distributed across the image. Unlike other approaches that apply reuse to spatially contiguous blocks of pixels, the present framework ensures that reuse decisions are spatially decorrelated, thereby preserving fine image details and temporal stability. As shown in FIG. 3, the reuse scheduler operates in two levels: 1) Key Frames 302 (e.g. frame if. All pixels perform sample reuse via path tracing, and 2) Non-key Frames 304 (frame i -I- 1 to i + 41V): Only one pixel in each 2x2 group performs sample reuse per frame (e.g. Ptin frame i + 1, P2in frame i + 2, etc.), rotating through non-key frames. While the other three pixels are selected as candidates for shading reuse, they will undergo further reuse validation control as described in the following sections and, if required, may be forced to refresh via sample reuse.

[0128] Tagging and Invalidation

[0129] Prior to invoking ReSTIR, a buffer having a size corresponding to the frame resolution is initialized to tag pixels for either sample reuse (value = 0) or shading reuse (value f 0). In non-key frames, pixels are tagged using an interlaced pattern to ensure periodic refreshing of ReSTIR reservoirs.

[0130] During key frames, all buffer values are set to zero, forcing a full ReSTIR update for all pixels. During non-key frames, the buffer stores a count representing the number of shading reuse operations performed for each pixel. Pixels that perform shading reuse are tagged as having dormant reservoirs. These non-zero tags are used to exclude those pixels from contributing toAtty. Docket: 4502-90001 (6000809PCT02)spatial or temporal sample reuse. Otherwise, outdated information from dormant reservoirs may propagate and degrade the quality of sample reuse.

[0131] FIG. 4 is a diagram 400 illustrating a shading reuse history check mechanism according to an embodiment of the present disclosure. In some embodiments, to avoid excessive shading reuse in dynamic scenes involving camera motion or data staleness, a shading reuse history is tracked for each pixel. After a pixel is tagged for shading reuse, the number of consecutive frames in which shading reuse is applied is recorded. A motion vector is used to locate the corresponding position of the pixel in a previous frame in order to evaluate the shading reuse history. For a pixel pat frame t, a corresponding location p'in a previous frame is determined using motion-compensated backprojection:p' = p — m(p)where m(p)denotes a motion vector associated with the pixel. A shading reuse counter Ht(p)is maintained for each pixel and updated based on the reuse history of the backprojected pixel p' + 1, if shading reuse is appliedHtp) =otherwisewhere Tis a predefined threshold representing a maximum allowable number of consecutive shading reuse operations. When the reuse counter satisfies Ht-1(p') > T, shading reuse is disabled and the pixel is instead processed using path tracing to refresh its associated sample reservoir. Otherwise, shading reuse is permitted. For example, if shading reuse has occurred fewer than three consecutive times, the reuse count is incremented by one. If shading reuse has already occurred three consecutive times, the tag is set to zero, thereby invalidating shading reuse for the current frame and forcing sample reuse to refresh the reservoir through path tracing. This mechanism caps shading reuse at three consecutive frames, ensuring that pixels undergoing rapid changes (for example, due to camera or object motion) are refreshed sufficiently often to maintain accuracy. Although this mechanism may slightly reduce computational efficiency, it prevents long-term reuse of outdated shading information and improves visual consistency. This mechanism ensures that each pixel periodically receives updated shading information, thereby preventing the accumulation and propagation of stale data. By constraining reuse in this manner, the approach mitigates temporal artifacts such as ghosting and improves consistency in regions affected by motion or dynamic lighting changes.Atty. Docket: 4502-90001 (6000809PCT02)

[0132] FIG. 5 is a diagram 500 illustrating an occlusion scenario in a dynamic scene according to an embodiment of the present disclosure. In some embodiments, in dynamic scenes, occlusion may occur due to changes in view angle or object motion as shown in FIG. 5. Image quality may decrease significantly in such occluded regions, which are sometimes referred to as motion-unreliable regions. An occlusion check is performed for both shading reuse and sample reuse by comparing depth and surface normals with the corresponding values from the previous frame. If a pixel tagged for shading reuse fails, this check indicates that the pixel corresponds to a newly visible object region in the current frame reusing shading from the previous frame may introduce significant radiance discrepancies and ghosting artifacts. In such cases, the pixel is redirected to the sample reuse path, and its reservoir is updated through path tracing. This update prevents increased noise in subsequent frames and maintains rendering quality. To handle newly visible regions that were previously occluded, a per-pixel geometric occlusion check is performed. Pixels tagged for shading reuse are compared with the previous frame using depth and normal information. If a pixel fails the occlusion check, shading reuse is disabled for that pixel in the current frame. This ensures that newly uncovered surfaces receive updated path-traced samples through sample reuse. In an example, for a pixel p at frame t, a corresponding location p'in the previous frame is obtained via motion-compensated backprojection:p' = p — m(pwhere m(p)denotes the motion vector. Geometric consistency is evaluated by comparing depth and surface normal information between the current frame and the previous frame. A depth consistency condition is defined as:I Dt(p) - Dt- p' l< tDwhere the adaptive depth threshold tDis given by:tD= Ci + c2• max (Dt(p), £t-i(p'))and c2and c2are predefined constants. A normal consistency condition is defined as:Nt(p ■ Nt- p') > cos (0W)where Nt(p) and N^^p') denote surface normals, and 0Nis a maximum allowable angular deviation, shading reuse is permitted only if both the depth and normal consistency conditions are satisfied. Otherwise, the pixel is classified as belonging to an occluded or disoccluded region, and shading reuse is disabled. In such cases, the pixel is processed using path tracing and sample reuse to obtain updated shading information. In some implementations, the geometric consistencyAtty. Docket: 4502-90001 (6000809PCT02)evaluation is performed across multiple neighboring pixels in the previous frame to account for subpixel motion and thin geometry. If any comparison fails, shading reuse is rejected. This geometric validation mechanism ensures that shading reuse is restricted to regions with consistent scene geometry, thereby preventing the propagation of outdated shading data and reducing artifacts such as ghosting, edge leakage, and temporal instability in dynamic scenes.

[0133] Radiance-Based Motion Artifact Reduction

[0134] Some pixels may experience radiance inconsistencies caused by rapid lighting changes or edge regions that are not detected by the depth- and normal-based occlusion checks because the geometric properties remain similar. To address this issue, an additional radiance-based validation strategy is applied. Radiance and color differences between corresponding pixels across frames are evaluated. Pixels exhibiting substantial radiance variation are forced to perform sample reuse rather than shading reuse in order to prevent smearing and ghosting artifacts. This radiance-based validation complements the depth-based occlusion checks and improves robustness under rapid lighting or material changes.

[0135] FIG. 6 is a diagram 600 illustrating floating-point motion vector backprojection and bilinear interpolation for shading reuse according to an embodiment of the present disclosure. In dynamic scenes with camera or object motion, direct shading reuse may suffer from misalignment due to sub-pixel shifts between frames. In some embodiments, to improve motion compensation accuracy, floating-point motion vectors are used rather than integer-based motion mapping that aligns only to pixel centers. As illustrated in FIG. 6, each pixel in a current frame 602 is backprojected to a corresponding location in a previous frame 604 using floating-point motion vectors. In an example, for a pixel p in a current frame t, a corresponding location in the previous frame is determined using a motion mapping function M(-), yielding a generally non-integer coordinatep' = M(p)Because p'may not align with a discrete pixel grid, it typically overlaps a 2 x 2 region of four neighboring pixels, denoted as A, B, C, and D. The shading value at the projected location is computed using bilinear interpolation of the shading values of these four pixels:St-i = WAX^A) + WsXt-^B) + WcX^^C) + wDxt-1D')Atty. Docket: 4502-90001 (6000809PCT02)where xt-1represents pixel shading values from frame t — 1, and w represents bilinear weights derived from the overlapped area between the projected location and the four neighboring pixels. The weights satisfy the normalization condition:wa+ wb+ wc+ wd= 1The interpolated shading value is then assigned to the current pixel:t(p) = St-i( (p))

[0136] The interpolated shading value st_iis reused in the current frame by setting st=This approach produces smoother transitions and reduces artifacts such as aliasing and ghosting, particularly along moving object edges. By leveraging bilinear interpolation with floating-point motion compensation, it maintains visual consistency in dynamic scenes, enhancing the temporal stability of shading reuse while preserving the integrity of the original ReSTIR pipeline.

[0137] Adaptive Accumulation from Temporal Reuse

[0138] In some embodiments, to enhance temporal stability for pixels rendered through sample reuse, an adaptive accumulation mechanism selectively combines current and historical shading values. This mechanism is applied only to sample reuse pixels to reduce temporal noise. For each pixel p, the current shading value xt(p)is combined with a motion-projected shading value from the previous frame st-1(M(p)):st(p) = axt(p) + (1 - a)st_!(M(p))where M(p)denotes the motion-mapped position of pixel p in the previous frame. The parameter acontrols the trade-off between responsiveness and temporal smoothing. Higher values of a emphasize the current frame, improving responsiveness to scene changes, while lower values emphasize historical data and reduce noise. Based on empirical analysis, a value such as a = 0.4 may be used to balance noise suppression and detail preservation. The HyShare framework also supports per-pixel adaptive weighting based on radiance variation. Letand ^represent the mean and standard deviation of the four shading values used in interpolation. A radiance range is defined as:Atty. Docket: 4502-90001 (6000809PCT02)If the current shading value lies within this range, the region is considered stable and a lower value of a is applied. Otherwise, a larger a value is used to preserve rapid changes. The adaptive weighting is defined as:where abaseis a base smoothing weight, / cis a sensitivity factor, and ris a threshold parameter. This adaptive accumulation suppresses flickering in stable regions while preserving edge sharpness and temporal consistency in high-contrast regions.

[0139] Experiments and results implementation

[0140] To validate the general applicability of the disclosed approach, the method was integrated into three existing rendering frameworks, including ReSTIR DI, ReSTIR PT, and Area ReSTIR. The resulting unified frameworks combine sample reuse and adaptive shading reuse across both spatial and temporal dimensions. The unified frameworks incorporate mechanisms including a per-pixel shading reuse history check, a depth-based occlusion check, and radiance and color variation checks, as well as tagging and refresh strategies to address reservoir dormancy. In addition, a motion-sensitive shading control mechanism is incorporated to improve robustness against motion-induced artifacts. Based on ablation studies, a value of a = 0.4 is selected to achieve a balanced trade-off between noise reduction and preservation of fine details. A shading reuse scheduling strategy is adopted to provide a balance between speed performance and image quality. No additional post-processing operations, such as anti-aliasing or smoothing, are applied.

[0141] Datasets

[0142] To evaluate the disclosed approach, six scene datasets from the NVIDIA Open Research Content Archive (ORCA) were used, including Bistro, Emerald Square, Sun Temple, and Zero Day. The Bistro scene includes several variations, including Bistro Day, Bistro Night, and Bistro Wine. All images were rendered at a resolution of 1920 x 1080 using an NVIDIA GeForce RTX 4090 graphics processing unit with 16 gigabytes (GB) of graphics memory.

[0143] Evaluation Metrics

[0144] The disclosed approach is evaluated by comparing the unified frameworks with the corresponding baseline methods (ReSTIR DI, ReSTIR PT, and Area ReSTIR) as well as with ground- truth rendering results, focusing on both performance and image quality in dynamic rendering across the six scene datasets. Qualitative evaluation focuses on visual perceptionAtty. Docket: 4502-90001 (6000809PCT02)characteristics including noise level, fidelity, and artifact reduction. For quantitative analysis, two standard metrics are used: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), which evaluate image noise and perceptual similarity.

[0145] FIG. 7 is a diagram 700 illustrating side-by-side visual comparison of hybrid reuse method relative to reservoir-based baselines according to an embodiment of the present disclosure. As shown in FIG. 7, that by combining ReSTIR-style sample reuse with adaptive shading reuse across both spatial and temporal domains, the proposed method, HyShare addresses the high shading costs and artifact risks that limit existing approaches. Compared with Area ReSTIR, taken as an example baseline, the disclosed method achieves up to 35.5% average performance improvement while simultaneously improving image quality by 1.6 dB in PSNR and 0.18 in SSIM in tests performed on the Zero Day scene.

[0146] FIG. 8 is a diagram 800 illustrating a comparative image set of effectiveness of the shading reuse history check according to an embodiment of the present disclosure. As discussed earlier, an eligibility criterion in the disclosed approach is the per-pixel shading reuse history check. This mechanism helps prevent excessive consecutive shading reuse by forcing the mechanism back to path tracing sample reuse instead to generate updated data, if the history indicates that the mechanism has reused shading for multiple consecutive times (e.g. 3 times in the experiments). This validation check can reduce the risk of misusing outdated shading data, especially in regions with significant motion or dynamic lighting changes. To evaluate its importance and effectiveness, the results have been compared with and without this check. As shown in FIG. 8, hybrid reuse without the per-pixel shading reuse history check (e.g., image 802) results in visible artifacts, which is caused by mis-reuse of shading from the neighboring black region due to horizontal motion in this example. When the check is applied (e.g., image 804), the artifacts are significantly reduced and the results closely match the reference image in visual quality.

[0147] FIG. 9 is a diagram 900 illustrating a comparative image set of the effectiveness of occlusion check according to an embodiment of the present disclosure. As discussed above, another key eligibility criterion in the disclosed framework is the per-pixel depth-based occlusion check, designed to detect regions or objects that were occluded in previous frames but become visible due to scene or camera motion. This validation check ensures that such newly visible regions use fresh path tracing sample reuse instead of shading reuse to obtain updated data. As shown in FIG. 9, image 902 shows the reference image rendered from the corresponding staticAtty. Docket: 4502-90001 (6000809PCT02)scene, and image 904 highlights the detected occlusion regions. Without this check in the dynamic scene, artifacts caused by outdated shading reuse appear clearly in the occlusion regions (e.g., image 906). In contrast, with this check, these artifacts are significantly reduced (e.g., image 908).

[0148] FIG. 10 is a diagram 1000 illustrating a comparative image set of the impact of applying radiance and color variation check according to an embodiment of the present disclosure. As discussed above, validation check based on changes in radiance amount and color serves as another key eligibility criterion to detect regions that should be refreshed with path tracing sample reuse due to the changes in lighting conditions, otherwise not detectable by depth change only. In a similar way as for the shading reuse history check, the effectiveness of the radiance and color variation check have been evaluated by comparing the results with and without this check. As shown in FIG. 10. hybrid reuse without this check (e.g., image 1002) results in visible artifacts, highlighted by the arrows. When the check is applied (e.g., image 1004), the artifacts disappear and the results preserve color fidelity of the reference image (e.g., image 1006) in visual quality.

[0149] FIGS. 11 A- 11C are diagrams 1100A-1100C illustrating the impact of a weighing factor values on image quality according to an embodiment of the present disclosure. As discussed above, the weighing factor a controls sensitivity of shading reuse to rapid scene changes. Although a does not affect rendering performance, it plays a critical role in image quality. Lower a values increase dependence on previously computed shading data, which reduces noise through temporal smoothing and propagation, but may blur fine details. On contrast, higher values reduce blurring and preserve details at the cost of increased noise. In an embodiment, multiple scenes have been evaluated using abasevalues ranging from 0.1 to 0.5. FIG. 11 A shows representative results 1100A for zoom-in image display for a = 0.1, 0.2, 0.3, 0.4, 0.5. As can be seen in FIG. 11A, a = 0.1 produces smoother but more noticeable blurred images, while a = 0.5 preserves better sharpness and more details, but introduces visible increased noise. FIG. 11B shows corresponding PSNR metric plot over frame number. FIG. 11C corresponding SSIM metric plot over frame number. Quantitatively, the PSNR and SSIM scores are relatively stable across the a range. For an optimal trade-off between noise reduction and sharpness preservation, a = 0.4 have been selected in all subsequent evaluation experiments.

[0150] To evaluate image quality under dynamic conditions, sequences of frames are rendered from the six diverse scenes with continuous camera motion for approximately 5-10 minutes per scene. This setup mimics challenges in real-time dynamic scenarios, such as motion, occlusion,Atty. Docket: 4502-90001 (6000809PCT02)lighting and shadow changes, and allows us to assess both visual quality and temporal stability. For each scene, results have been generated using proposed HyShare frameworks and the corresponding baseline methods: ReSTIR DI, ReSTIR PT and area ReSTIR.

[0151] FIG. 12 is a diagram 1200 illustrating a visual comparison of dynamically rendered frames from HyShare and corresponding ReSTIR DI baseline according to an embodiment of the present disclosure. The visual comparison includes only two out of the six tested scenes, i.e., the bistro wine scene and the sun temple scene, illustrating the key visual perceptual difference between HyShare and ReSTIR DI baseline.

[0152] FIG. 13 is a diagram illustrating a visual comparison of dynamically rendered frames from HyShare and corresponding baseline ReSTIR PT baseline according to an embodiment of the present disclosure. The visual comparison includes only two out of the six tested scenes, i.e., the bistro night scene and the emerald square scene, illustrating the key visual perceptual difference between HyShare and ReSTIR PT baseline.

[0153] FIG. 14 is a diagram illustrating a visual comparison of dynamically rendered frames from HyShare and corresponding baseline method area ReSTIR baseline according to an embodiment of the present disclosure. The visual comparison includes only two out of the six tested scenes, i.e., the bistro day scene and the zero day scene, illustrating the key visual perceptual difference between HyShare and area ReSTIR baseline.

[0154] As shown in FIG. 12, FIG. 13, and FIG. 14, visual inspection of the results consistently shows that HyShare improves image quality over its baselines. Further, HyShare reduces noise without over- smoothing partially due to the balanced a values, maintains high-frequency details, and avoids ghosting artifacts in dynamic and occluded regions, and matches the ground truth more closely. It is also worth noting that although HyShare is designed for on-the-fly processing without requiring full-frame completion as in post-rendering techniques for aliasing reduction, it still effectively reduces aliasing artifacts compared to baseline methods. As shown in FIG. 14, HyShare produces smoother and cleaner edges with less jagged artifacts. This quality improvement is primarily attributed to the use of bilinear interpolation when applying floating-point motion vectors, rather than simply using integer-based mappings, in shading reuse.

[0155] Quantitative evaluation: In an embodiment, to complement the qualitative analysis described above, the quantitative scores of the PSNR and SSIM metrics may been calculated by comparing the rendered frames with the ground truth images obtained through a long accumulationAtty. Docket: 4502-90001 (6000809PCT02)of path tracing. Table 1 and Table 2 summarize the PSNR and SSIM results, respectively, for all six scenes. Each row pair compares HyShare to its corresponding baseline. The rightmost column in both Table 1 and Table 2 is the average score in all scenes. The quantitative results demonstrate that HyShare consistently outperforms the baselines in both metrics with only one exception of the PSNR of HyShare vs. ReSTIR PT in the Bistro Wine scene. On average, PSNR is improved by 1.3-2.5 decibels (dB) and SSIM is improved by 0.13-0.19 in all scenes tested. These improvements confirm that HyShare not only reduces noise and artifacts but also better preserves color fidelity, structural content, and fine details across a variety of dynamic environments of rendering, which matches the observations in the visual inspection above.Table 1. PSNR metrics across six dynamic rendering scenesBistro Bistro Bistro Emerald Sun Zero Across All Day Night Wine Square Temple Day 6 Scenes ReSTIR DI 16.4 15.4 18.9 15.9 20.6 16.1 17.2 HyShare 17.3 18.7 19.3 17.3 21.2 17.0 18.5 ReSTIR PT 15.3 16.0 16.6 14.4 15.0 13.7 15.2 HyShare 16.5 19.2 15.2 16.3 17.4 16.4 16.8Area ReSTIR 18.5 17.6 23.1 16.6 25.7 18.3 20.0 HyShare 20.3 23.9 25.2 18.3 27.4 19.9 22.5Table 2. SSIM metrics across six dynamic rendering scenesBistro Bistro Bistro Emerald Sun Zero Across All Day Night Wine Square Temple Day 6 Scenes ReSTIR DI 0.46 0.36 0.61 0.52 0.52 0.33 0.46 HyShare 0.58 0.71 0.69 0.59 0.58 0.53 0.61 ReSTIR PT 0.38 0.26 0.50 0.44 0.52 0.18 0.38 HyShare 0.58 0.49 0.62 0.54 0.58 0.43 0.54Area ReSTIR 0.64 0.36 0.60 0.59 0.73 0.48 0.57 HyShare 0.77 0.71 0.85 0.71 0.90 0.66 0.77

[0156] FIGS. 15A-15C are diagrams 1500A-1500C illustrating a set of plots comparing rendering speed in fps over frame number between HyShare and the baseline approaches according to an embodiment of the present disclosure. In addition to image quality, the rendering speed of proposed HyShare framework may be evaluated against the corresponding baseline methods, focusing on the average frame rate for the six dynamic scenes. FIG. 15A shows HyShare vs. ReSTIR DI baseline comparison of fps plot over frames (or time) on bistro day scene. FIG. 15BAtty. Docket: 4502-90001 (6000809PCT02)shows HyShare vs. ReSTIR PT baseline comparison of fps plot over frames (or time) on sun temple day scene. FIG. 15C shows HyShare vs. area ReSTIR baseline comparison of fps plot over frames (or time) on bistro night scene. For each method and for each scene, the fps score is averaged over the plot. As shown below, Table 3 summarizes the results of the comparison of the averaged fps following the similar arrangement used in the above quantitative image quality metrics.Table 3. Average rendering speed (fps) across six dynamic scenesBistro Bistro Bistro Emerald Sun Zero Across All Day Night Wine Square Temple Day 6 Scenes ReSTIR DI 210.0 186.9 280.4 135.1 348.5 89.9 208.5 HyShare 279.5 266.1 396.3 204.6 431.7 95.9 279.0 Improvement 33.1% 42.4% 41.3% 51.5% 23.9% 6.6% 33.8% ReSTIR PT 22.5 22.5 38.1 19.4 42.5 21.6 27.8 HyShare 29.4 29.4 52.1 30.5 56.9 29.0 37.9 Improvement 30.7% 30.4% 36.6% 56.9% 33.7% 34.1% 36.3% Area ReSTIR 24.3 23.5 43.6 25.9 50.2 29.5 32.8 HyShare 32.7 31.4 67.7 34.2 73.7 40.0 46.6 Improvement 34.4% 33.9% 55.1% 31.0% 46.9% 35.5% 42.0%

[0157] For all scenes tested. HyShare shows a consistent and substantial improvement in rendering speed. On average, HyShare achieves an increase of 33.8% over ReSTIR DI, 36.3% over ReSTIR PT, 42.0% over Area ReSTIR, respectively, and 37.4% in all cases. In high complex scenes such as Emerald Square and Bistro Wine, the performance improvements are more significant, reaching 56.9% and 55.1%, respectively. These improvements demonstrate the effectiveness of the proposed hybrid reuse approach in reducing the cost of shading and improving rendering efficiency. Even in more challenging or motion-heavy scenes like Zero Day or Sun Temple (vs. ReSTIR DI), HyShare still maintains meaningful gains, which indicates its robustness and general applicability across diverse environments and rendering methods. These results confirm that HyShare not only enhances image quality, but also provides significant real-time performance improvements, which means good potential for practical use in high-fidelity realtime or interactive dynamic rendering systems.

[0158] Convergence: The baseline methods have demonstrated their properties of convergence to ground truth in prior work. To evaluate whether the proposed HyShare framework preserves these properties, convergence tests may be conducted using static scenes with extended sampleAtty. Docket: 4502-90001 (6000809PCT02)accumulation to produce low-noise high-quality images.

[0159] FIGS. 16A-16D are diagrams illustrating a convergence assessment showing accumulated static-scene renderings and mean absolute percentage error (MAPE) plots over time comparing HyShare to each baseline according to an embodiment of the present disclosure. FIG.16A shows a visual comparison of accumulated images between baseline methods for ReSTIR DI, ReSTIR PT and area ReSTIR and a corresponding HyShare image column using two representative static scenes (e.g., a bistro day scene and a bistro night scene). FIG. 16B shows MAPE vs. time plot for ReSTIR DI with bistro day and bistro night curves. FIG. 16C shows MAPE vs. time plot for ReSTIR PT with bistro day and bistro night curves. FIG. 16D shows MAPE vs. time plots for Area ReSTIR with bistro day and bistro night curves. As can be seen from FIGS.16B-16D, the MAPE result plots show a consistent decrease in error over accumulated time, with all final MAPE values falling within a few percentage points. These results demonstrate that the proposed hybrid reuse strategy in HyShare maintains the convergence behaviors of the baseline methods.

[0160] FIG. 17 is a diagram illustrating a bar chart comparing peak GPU memory usage according to an embodiment of the present disclosure. HyShare introduces additional buffers, including a tagging buffer for pixel states and multiple history buffers for radiance, normals, and motion vectors. FIG. 17 shows that the overall GPU memory usage remains lower than the original ReSTIR. The baseline ReSTIR pipeline requires two per-pixel reservoir buffers and retraces new paths for every pixel in both spatial and temporal sample reuse passes. In contrast, HyShare optimizes memory by allowing pixels tagged for shading reuse to bypass path retracing and sample reuse, directly leveraging cached shading results. These pixels only require lightweight buffers, reducing dynamic memory overhead and improving scalability. The experimental results show HyShare reduces peak memory usage by 0.9, 0.5, and 1.3 GB (23.7%, 11.6%, and 7.3%) for DI, PT, and area settings, respectively, while sustaining real-time quality and performance, demonstrating that the HyShare algorithm is more memory-efficient than the original ReSTIR.

[0161] Ablation Studies

[0162] Selection of Shading Reuse Scheduling Strategies: To evaluate the impact of different shading reuse scheduling strategies used in non-key frames, an ablation study comparing three configuration choices may be conducted that operate on different proportions of pixels using pathtracing sample reuse versus shading reuse within 2x2 pixel tiles. Strategy A (1:3): The default oneAtty. Docket: 4502-90001 (6000809PCT02)used in our experiments: 1 pixel perform sample reuse and 3 pixels for shading reuse. Strategy B (2:2): 2 pixels for sample reuse and 2 pixels for shading reuse, Strategy C (3: 1): 3 pixels for sample reuse and 1 pixel for shading reuse. These configurations represent a trade-off between speed performance and image quality. Strategy A emphasizes performance by reducing the frequency of path-tracing computation, while strategies B and C emphasize quality at the cost of increased computation. Table. 4 summarizes the HyShare with ReSTIR PT results of PSNR, SSIM and FPS metrics measured on two representative scenes (Bistro Day and Bistro Night). The results show that image quality (PSNR and SSIM) remains comparable for the three strategies, but strategy A achieves notably higher FPS scores. This study supports our choice of Strategy A in the experiments, which results in a preferred balance between speed and fidelity for real-time rendering applications.Table 4. Comparisons of the three shading reuse scheduling strategiesBistro Day Bistro NightPSNR SSIM FPS PSNR SSIM FPSStrategy A 18.5 0.58 29.4 21.5 0.56 29.4 Strategy B 18.4 0.57 27.3 21.4 0.55 27.2 Strategy C 17.1 0.53 24.8 20.9 0.53 24.4

[0163] FIG. 18 is a flowchart 1800 according to an embodiment of the disclosure. In block 1802, the method comprises partitioning pixels of a current frame into a first group assigned to sample reuse and a second group assigned to shading reuse. In one embodiment, partitioning the pixels comprises partitioning the pixels based on a scheduling strategy, and wherein the scheduling strategy comprises: designating key frames occurring at intervals of 4N+1 frames, wherein N is an integer; assigning all pixels to the first group during each key frame to perform sample reuse; and applying, for non-key frames between the key frames, a cyclic rotation pattern within 2x2 pixel groups such that one pixel in each 2x2 pixel group is assigned to the first group and remaining pixels in each 2x2 pixel group are assigned to the second group.

[0164] In block 1804, the method further comprises performing, for each pixel in the first group, reservoir-based path tracing sample reuse to generate a first shading value and to update a corresponding pixel sample reservoir.

[0165] In block 1806. the method further comprises evaluating, for each pixel in the second group, one or more reuse eligibility criteria to determine whether shading reuse is permitted. InAtty. Docket: 4502-90001 (6000809PCT02)one embodiment, valuating the one or more reuse eligibility criteria comprises: performing, for each pixel in the second group, a shading reuse history check that tracks a number of consecutive frames during which shading reuse has been applied to the pixel; and redirecting that pixel to the first group when the number reaches a predetermined threshold.

[0166] In block 1808, the method further comprises determining, for each pixel in the second group that satisfies the one or more reuse eligibility criteria, a corresponding pixel location in a previous frame using a motion vector associated with that pixel.

[0167] In block 1810, the method further comprises retrieving, from a shading history buffer, a second shading value based on the corresponding pixel location in the previous frame. In one embodiment, retrieving the second shading value from the shading history buffer comprises back-projecting each pixel in the second group in the current frame to the corresponding pixel location in the previous frame using the motion vector, wherein the motion vector is a floating-point motion vector; identifying four neighboring pixels in a 2x2 region in the previous frame that surround the corresponding pixel location; and computing the second shading value using bilinear interpolation of shading values of the four neighboring pixels.

[0168] In block 1812, the method further comprises merging the first shading value and the second shading value to generate a rendered frame.

[0169] In block 1814, the method further comprises outputting the rendered frame to a display device.

[0170] In this disclosure, a hybrid sample and shading reuse method on-the-fly is disclosed that simultaneously improves the efficiency and quality of real-time path tracing rendering. By combining ReSTIR-style sample reuse with adaptive shading reuse across both spatial and temporal domains, our method addresses the high shading costs and artifacts risks that limit existing approaches isolated in individual domain. Key mechanisms such as reservoir tagging, sharing pause, and periodic refresh ensure reuse validity and robustness in dynamic scenes. The experiments show that the proposed method achieves an improvement in speed performance of more than 35% on average while simultaneously enhancing image quality in PSNR and SSIM through reduced noise, aliasing artifacts, and temporal instability. This demonstrates the effectiveness of incorporating sample and shading reuse in a unified system for real-time photorealistic rendering.

[0171] Physically based path tracing (PT) has become a widely adopted approach forAtty. Docket: 4502-90001 (6000809PCT02)photorealistic rendering due to its ability to solve the rendering equation using Monte Carlo integration. However, multi-bounce light transport typically requires a large number of samples per pixel to achieve convergence, which makes real-time PT computationally expensive, particularly for thin or mobile devices with limited graphics processing resources.

[0172] Recent advances in resampled importance sampling have improved sampling efficiency for real-time ray tracing. For example, ReSTIR DI introduced spatiotemporal reservoir resampling for direct illumination, and subsequent techniques have extended this framework to multi-bounce transport, including generalized reservoir-based resampling and area-based resampling approaches for subpixel sampling. These techniques reuse path candidates across pixels and frames to reduce variance at a fixed sampling budget, enabling near real-time global illumination on high-performance graphics hardware.

[0173] Hybrid and distributed rendering systems have been developed to reduce client-side computational cost by offloading computationally intensive operations to remote devices. For example, hybrid remote rendering architectures and distributed hybrid rendering (DHR) pipelines combine local rasterization with remotely computed ray-traced effects, which are composited using depth-aware blending or Red, Green, Blue + Depth (RGB-D) streaming. Mobile edge computing (MEC)-assisted and cloud-based extended reality (XR) streaming frameworks further reduce latency by placing compute resources closer to users. However, such systems typically rely on video-based or full-frame image streaming, which remains bandwidth-intensive, particularly for high-resolution XR applications.

[0174] D-HyShare Framework and related work

[0175] In some embodiments, the present disclosure further pertains to a distributed, reusecentric client-server framework for real-time path tracing, referred to as distributed HyShare (D-HyShare). The D-HyShare framework extends a hybrid sample and shading reuse formulation to a distributed architecture that leverages reservoir-based spatiotemporal importance resampling at a server and temporally validated shading reuse at a client across heterogeneous computing devices. The framework addresses bandwidth constraints in client-server photorealistic rendering by reducing transmitted pixel data from a full-resolution frame to a subset of pixels corresponding to newly shaded regions.

[0176] In some embodiments, the D-HyShare framework includes multiple technical features. In a first aspect, reuse decomposition is performed across devices, wherein the server executesAtty. Docket: 4502-90001 (6000809PCT02)path tracing with reservoir-based sampling and maintains per-pixel sample reservoirs, while the client reconstructs a full-resolution frame using motion-compensated shading reuse. In a second aspect, delta-radiance encoding is performed, wherein the server transmits data corresponding to pixels requiring updated shading as a delta-compressed radiance stream. In a third aspect, latency-resilient frame generation is performed, wherein the client maintains temporal continuity by generating intermediate frames using motion-compensated reprojection based on locally stored shading history during network delays.

[0177] In some embodiments, the D-HyShare framework achieves consistent bandwidth savings exceeding 96% relative to raw RGB streaming across a range of scenes and resolutions, delivers interactive frame rates in excess of 31 frames per second on thin-client hardware at 1920x1080 resolution, and reduces client GPU memory footprint by more than 4x compared to local path tracing baselines. The framework is compatible with multiple ReSTIR variants (ReSTIR DI, ReSTIR PT, Area ReSTIR) and scales with server hardware improvements.

[0178] ReSTIR and Resampled Importance Sampling

[0179] Modem real-time path tracing relies on variance reduction techniques to improve rendering efficiency. Resampled importance sampling has been applied to Monte Carlo rendering to improve sampling effectiveness in high-dimensional light transport integration. In path tracing, outgoing radiance is estimated using a Monte Carlo formulation that requires sampling a highdimensional integral. Achieving a target signal-to-noise ratio typically requires a large number of samples per pixel, resulting in substantial computational cost. The total computation time T is proportional to the number of samples N and a per-sample processing cost Tray, such that:T = N · Traywhere Trayis the time to trace and shade a single ray. In real-time applications, the number of samples N is constrained by performance budgets, which motivates techniques that reuse samples across pixels and frames to improve effective sampling efficiency without proportionally increasing computation. The evolution of real-time path tracing has been driven by advancements in resampled importance sampling (RIS) techniques, particularly the ReSTIR family, which enables efficient rendering of dynamic direct lighting and multi-bounce global illumination. ReSTIR DI introduced reservoir-based spatiotemporal resampling for dynamic direct lighting, enabling substantial variance reduction at low sample counts by reusing and resampling candidateAtty. Docket: 4502-90001 (6000809PCT02)light samples across pixels and frames. Building upon this idea, ReSTIR PT provides the mathematical basis for multi-bounce path reuse and underpins, enabling principled resampling of full light transport paths rather than isolated light samples. Subsequent work explored extensions of ReSTIR to broader sampling domains, such as conditional ReSTIR improves robustness and flexibility when naive spatiotemporal reuse would otherwise introduce bias, and Area ReSTIR, which extends reservoir resampling from light-sample space to per-pixel ray space, including film and lens dimensions. These developments illustrate a clear progression of ReSTIR from direct lighting to indirect illumination, full path reuse, conditional formulations, and ray-space resampling. While these methods significantly reduce sampling variance, they typically perform both sample resampling and shading each frame locally, motivating the exploration of distributed architectures that decouple sampling and shading reuse across heterogeneous devices.

[0180] Shading Reuse and Temporal Reconstruction

[0181] To further reduce computation, shading reuse techniques exploit temporal coherence to avoid redundant shading when scene attributes remain stable. Temporal Anti-Aliasing (TAA) blends motion-compensated history to reduce aliasing and improve stability, and spatiotemporal filters such as Spatiotemporal Variance-Guided Filtering (SVGF) accumulate radiance over time to reconstruct global illumination from sparse samples. Other methods explore adaptive or decoupled shading to reduce the frequency of shading updates, guided by motion, shading gradients, or luminance changes. In some approaches, shading reuse is selectively applied based on reuse eligibility criteria to avoid temporal artifacts. These approaches typically rely on motion vectors and locally stored history buffers to determine whether previously computed radiance values remain valid. However, such techniques are generally developed independently of sample reuse mechanisms and are not integrated with reservoir-based sampling approaches. HyShare directly addresses this limitation by unifying sample reuse (Rs) and selective shading reuse (Rsh 'i within a single framework. By decoupling the scheduling of sampling and shading updates, HyShare improves efficiency while preserving temporal stability, but its execution remains confined to a single-node architecture.

[0182] Client / Server Hybrid Rendering

[0183] Distributed rendering systems offload computationally intensive operations to remote devices while maintaining interactive responsiveness at a client device. Hybrid rendering pipelines may combine client-side rasterization with server-side ray tracing using RGB-D compositing.Atty. Docket: 4502-90001 (6000809PCT02)Distributed hybrid ray tracing approaches further partition rendering workloads to support high-fidelity rendering on resource-constrained clients. Cloud-based and mobile edge computing (MEC)-assisted extended reality (XR) systems reduce latency by placing compute resources closer to end users. However, such systems often rely on full-frame image transmission or video encoding, which can result in high bandwidth usage, particularly for high-resolution applications. Other approaches reduce transmission requirements by transmitting lower-resolution frames and reconstructing higher-resolution outputs at the client, for example using super-resolution techniques or spatiotemporal interpolation. These approaches typically rely on dense image representations or learned models. In contrast, the D-HyShare framework transmits sparsely updated radiance data corresponding to selected pixels and reconstructs a full-resolution frame at the client using motion-compensated shading reuse.

[0184] The disclosed D-HyShare framework extends a hybrid sample- shading reuse formulation of HyShare to a distributed client-server architecture. In this architecture, computationally expensive multi-bounce path sampling is executed on a remote server, while shading reuse, image reconstruction, and temporal accumulation are performed on a thin client. In some embodiments, HyShare employs a deterministic cycling shading reuse schedule in which only one pixel in each 2x2 group is shaded from a new sample, while the remaining 75% reuse shading results from prior frames after motion-compensated validation. This schedule guarantees that every pixel is refreshed once every four frames, significantly reducing redundant shading computation. To mitigate artifacts caused by fast camera or object motion, HyShare periodically inserts keyframes after 4k non-key frames. At each keyframe, full path tracing is performed for all pixels, thereby fully refreshing all reservoirs and preventing long-term reuse staleness. The D-HyShare framework leverages this sparse shading structure to reduce network bandwidth.

[0185] FIG. 19 is a block diagram of distributed client-server rendering framework 1900 according to an embodiment of the present disclosure. The framework 1900 comprises a server 1902, a network communication layer such as Internet 1904, and a client 1906. In some embodiments, the server 1902 is configured to perform ReSTIR-based spatiotemporal reservoir resampling for multi-bounce path tracing, while the client 1906 is configured to perform shading reuse, image reconstruction, temporal accumulation, and display of a full-resolution frame. In some embodiments, the network communication layer provides bidirectional data exchange between the server 1902 and the client 1906 over a network such as the Internet 1904.Atty. Docket: 4502-90001 (6000809PCT02)

[0186] The server 1902 comprises a start module 1908 configured to initiate execution of the rendering framework 1900 and transitions control to a waiting module 1910, which is configured to idle or synchronize based on incoming client requests or frame timing constraints. The start module 1908 is configured to receive a compact state packet Ct from the client 1906 once per frame. The compact state packet comprises a frame index, timestamp, camera pose, and projection parameters. By sharing the compact state packet Ct, both the server 1902 and the client 1906 render from an identical viewpoint at each frame. Using the most recent camera state packet, the server 1902 executes reservoir-based path sampling over a subset of pixels to generate updated radiance values and to maintain per-pixel sample reservoirs across successive frames.

[0187] In some embodiments, the server 1902 further comprises a shading reuse module 1912 configured to determine, on a per-pixel basis, an initial shading assignment indicating whether a pixel is to be newly shaded or designated for shading reuse. The shading reuse module 1912 configured to perform an initial per-pixel classification according to a deterministic cycling shading reuse schedule. Under this schedule, for each frame, a subset of pixels is selected for fresh shading while remaining pixels are designated for shading reuse. In one implementation, one pixel within each 2x2 pixel group is selected for path tracing, while remaining pixels are designated as candidates for shading reuse. This schedule ensures that all pixels are refreshed over successive frames, thereby reducing redundant shading computations.

[0188] In some embodiments, to mitigate error accumulation and visual artifacts caused by rapid camera or scene motion, the shading reuse module 1912 further incorporates periodic keyframes inserted after a predetermined number of non-key frames. During a keyframe, the cycling shading reuse schedule is overridden, and all pixels are refreshed via full path tracing, thereby resetting per-pixel reservoirs and preventing long-term reuse staleness.

[0189] In some embodiments, the server 1902 further comprises a sample reuse module 1914 configured to perform validation and execution based on the initial shading assignment determined by the shading reuse scheduling module 1912. In response to determining that shading reuse is not applicable for a pixel, the sample reuse module 1914 performs reservoir-based spatiotemporal importance resampling to generate updated shading samples. In some embodiments, in response to determining that shading reuse is applicable, the sample reuse module 1914 bypasses the sample reuse module 1914 and reuses prior- frame radiance for the pixel, thereby reducing computation.Atty. Docket: 4502-90001 (6000809PCT02)

[0190] In some embodiments, the sample reuse module 1914 is configured to refine the initial shading assignment by evaluating per-pixel reuse eligibility according to one or more criteria, including occlusion consistency, radiance variation, and motion validity. Pixels that fail the reuse eligibility criteria are reclassified for refresh and processed via path tracing, while pixels that satisfy the criteria reuse prior-frame radiance via motion-compensated reprojection. Accordingly, the sample reuse module 1914 enforces correctness and selectively triggers shading updates, such that only pixels requiring updated radiance, including newly shaded or insufficiently converged pixels, are processed for further shading.

[0191] In some embodiments, the server 1902 further comprises a current shading module 1916 configured to generate transmission-ready output data corresponding to newly shaded pixels. For pixels identified as newly shaded, the current shading module 1916 computes delta-radiance values relative to motion-compensated prior-frame radiance. The current shading module 1916 further generates a frame-wide binary mask indicating, for each pixel, whether the pixel has been newly shaded or designated for shading reuse. The delta-radiance values corresponding to the newly shaded pixels are quantized and packed into a one-dimensional radiance stream, and the one-dimensional radiance stream and the binary mask are encoded to form a payload for transmission to the client 1906, such that only pixels identified as newly shaded are transmitted.

[0192] In some embodiments, the Internet 1904 transmits shading data from the current shading module 1916 to the client 1906, and transmits rendering requests and / or feedback signals from the client 1906 to the serve 1902.

[0193] In some embodiments, the client 1906 is configured to perform shading reuse and final image synthesis. The client comprises a request module 1918 configured to generate and transmit shading requests to the server 1902 via the Internet 1904, and receives corresponding shading data. The client 1906 further comprises a history shading module 1920 configured to store shading data from one or more previously rendered frames, enabling temporal reuse of shading information.

[0194] In some embodiments, the client 1906 further comprises a spatiotemporal shading reuse module 1922 configured to reconstruct radiance for pixels not updated in a current frame using shading data stored in the history shading module 1920. For such pixels, the spatiotemporal shading reuse module 1922 retrieves prior-frame radiance using motion-compensated reprojection. In some embodiments, spatial reuse is additionally performed by incorporating radiance contributions from neighboring pixels exhibiting similar geometric and material properties.Atty. Docket: 4502-90001 (6000809PCT02)

[0195] In some embodiments, the client 1906 further comprises a temporal shading reuse module 1924 configured to apply received shading updates based on a decoded server payload. For pixels identified as newly shaded by the binary mask, the temporal shading reuse module 1924 applies decoded delta-radiance values to corresponding motion-compensated radiance. For remaining pixels, the temporal shading reuse module 1924 retains the motion-compensated radiance.

[0196] In some embodiments, the client 1906 further comprises a merge frame module 1926 configured to generate a full-resolution frame by combining updated radiance values with reused historical radiance. The merge frame module 1926 performs per-pixel integration and, in some embodiments, adaptive temporal accumulation to improve visual stability and reduce noise.

[0197] In some embodiments, the client 1906 further comprises a display module 1928 configured to output the reconstructed full-resolution frame for presentation to a user.

[0198] In some embodiments, the operations of the spatiotemporal shading reuse module 1922, the temporal shading reuse module 1924, and the merge frame module 1926 collectively reconstruct the full-resolution frame. After decoding the payload, the client 1906 updates or reconstructs a locally maintained vertex buffer (V-buffer) based on synchronized camera state, applies radiance updates for newly shaded pixels, reconstructs remaining pixels via motion-compensated shading reuse, and generates the full-resolution frame for display. Because the server 1902 and the client 1906 share consistent scene geometry, motion, and material data, the V-buffer remains aligned across devices, thereby supporting consistent reprojection and temporal reconstruction.

[0199] The disclosed architecture decouples server-side path tracing from client-side reconstruction, enabling efficient distributed rendering in which computationally intensive shading is performed at the server, while the client performs reprojection, reconstruction, and blending operations to generate a displayable frame.

[0200] FIG. 20 is a diagram 2000 illustrating data communication between a client-server framework according to an embodiment of the present disclosure. The data communication flow is established between a remote device 2002 (e.g., a server) and a local device 2004 (e.g., a client) over one or more data channels. As shown in FIG. 20, the data exchange includes two primary channels: a synchronization (sync) data channel 2006 and an image data channel 2008. The sync data channel 2006 is configured to transmit per-frame compact state packets Ct from the localAtty. Docket: 4502-90001 (6000809PCT02)device 2004 to the remote device 2002 at a high frequency. Each compact state packet includes camera-related parameters comprising frame index, timestamp, camera pose, and projection parameters, thereby enabling maintenance of a consistent rendering viewpoint between the remote device 2002 and the local device 2004. The image data channel 2008 conveys compressed image payloads from the remote device 2002 to the local device 2004.

[0201] In some embodiments, for keyframes, the remote device 2002 transmits a fullresolution image in which all pixels are newly shaded, thereby refreshing accumulated sampling state and mitigating artifacts associated with prolonged reuse. In some embodiments, for nonkeyframes, the remote device 2002 transmits only a sparse subset of newly shaded pixels. Pixel states are indicated using a binary mask, wherein a first value (e.g., 0) identifies a newly shaded pixel for which a corresponding delta-radiance value is transmitted, and a second value (e.g., 1) identifies a pixel designated for shading reuse from a prior frame. The shading reuse is performed according to a deterministic cycling schedule in which one pixel within each predefined pixel group is updated using a new sample while remaining pixels reuse prior shading results subject to motion-compensated validation, thereby reducing bandwidth relative to full-frame transmission.

[0202] In some embodiments, the local device 2004 reconstructs a current frame ftby combining the received sparse payload with a locally stored prior frame ^that includes radiance, motion vectors, depth, and normal information. Reconstruction is performed using motion-compensated reprojection to obtain a prior-frame radiance estimate and applying decoded deltaradiance updates to pixels identified as newly shaded, thereby generating a full-resolution output frame.

[0203] In some embodiments, stability under network latency (τ) is achieved using latencyhiding techniques, including client-side motion-compensated reprojection to predict a current view. In such embodiments, a previous pixel position xt-1is mapped to a predicted position x based on a motion vector vt(x), according to:x't = Xt-l + Vt (x)

[0204] While motion-compensated reprojection may be used in other systems for latency compensation or image quality enhancement, the disclosed framework integrates motion-compensated reprojection directly into a primary frame reconstruction pipeline, wherein predicted or reprojected radiance values are used as a basis for both shading reuse and incorporation of received radiance updates.Atty. Docket: 4502-90001 (6000809PCT02)

[0205] FIG. 21 is a diagram 2100 illustrating generation and traversal of a binary mask during server-side encoding for sparse radiance updates according to an embodiment of the present disclosure. In particular, FIG. 21 depicts a frame 2102 processed in raster order, wherein a singlebit cursor sequentially traverses a binary mask (Mt) 2104 that records which pixels have been newly shaded. For each pixel x. a determination is made by an encoder as to whether the pixel has been updated in a current frame by a sampling scheduler, such as a ReSTIR-based scheduler. In some embodiments, responsive to determining that the pixel is newly shaded, a corresponding mask value is set to a first value (e.g., 0), whereas responsive to determining that the pixel is not newly shaded, the mask value is set to a second value (e.g., 1).

[0206] In some embodiments, for pixels identified as newly shaded, a motion-compensated historical radiance valueis computed by back-projecting the pixel location to a prior frame and performing bilinear interpolation over neighboring pixel samples. In some embodiments, the motion-compensated historical radiance is computed as:where xk' denotes neighboring pixel locations in the prior frame corresponding to a back-projected position of pixel x, where wkare normalized interpolation weights derived from the overlapped area between the projected location of the current pixel and the four surrounding pixels such that Σk=1wk=1. A delta radiance ALt(x) is then determined as a difference between a current radiance value Lt(x) and the motion-compensated historical radianceThe delta-radiance values corresponding to newly shaded pixels are then quantized using a uniform quantization process and packed into a one-dimensional radiance buffer 2106 for subsequent encoding and transmission.

[0207] FIG. 22 is a diagram 2200 illustrating motion-compensated prediction and deltaradiance generation according to an embodiment of the present disclosure. FIG. 22 illustrates that, following motion-compensated prediction, delta radiance values ALt(x) tend to have reduced magnitude due to temporal coherence between successive frames. In some embodiments, for each pixel identified as newly shaded, a corresponding radiance value Lt(x) is stored in 8-bit channels and represented using a fixed-precision format, and a delta-radiance value ΔLt(x) is computed relative to a motion-compensated prior-frame radiance and falls within [-255, 255]. The deltaradiance values are then quantized using a linear quantization function having a step size Q(typically Q E [2, 8]), such that the quantized delta-radiance values are mapped to a signedAtty. Docket: 4502-90001 (6000809PCT02)integer range [-128, 127]. In some embodiments, the signed integer range corresponds to an 8-bit representation.

[0208] In some embodiments, the quantization step size Q is selected to map delta-radiance values within a predefined range to the signed 8-bit interval [—128,127]. The use of delta-radiance values, which are reduced in magnitude due to temporal coherence, enables efficient representation and subsequent compression of radiance data. This quantization is highly effective because most delta are small and cluster near zero, allowing aggressive entropy compression while preserving perceptual fidelity. In some embodiments, for pixels not identified as newly shaded according to a corresponding binary mask, no delta-radiance value is generated, and the encoding process advances without appending data to a delta-radiance buffer. In practice, this scheme yields a compact delta- radiance buffer of variable but usually small size. In some embodiments, the quantized delta-radiance values corresponding to newly shaded pixels are stored in a onedimensional buffer. The binary mask Mtand the one-dimensional buffer are combined and encoded, including by applying entropy compression (e.g., using LZ4 or Zstd), to generate a transmission payload Ptfor communication to a client device.

[0209] Client-Side Shading Reuse and Reconstruction

[0210] In some embodiments, the client stores a previous-frame reconstructed radiance Lt-i along with motion vectors, depth, and normals information computed locally from a synchronized camera state. After receiving the server payload Pt for frame t, the client decompresses the binary mask Mt and the ID delta-radiance buffer ABt, then apply the inverse quantization using the same quantization step size Q to the delta radiance before reconstructs a full-resolution radiance Lt(x) in raster order. For each pixel x, the client computes a motion-compensated shading candidate by back- projecting the pixel location to a prior frame and performing bilinear interpolation, consistent with the corresponding server-side computation. The client then performs per-pixel reconstruction as follows.

[0211] If Mt (x) = 0, indicating that the pixel is newly shaded, a corresponding delta-radiance value is obtained from ABt. A reconstructed radiance value for the current frame is computed as:Lt(x) = Q -t[i] + Ht-1(x)where i indexes the delta-radiance stream.

[0212] If Mt (x) = 1, indicating that the pixel is designated for shading reuse, no delta-radiance value is consumed from ABt, and the reconstructed radiance is set as:Atty. Docket: 4502-90001 (6000809PCT02)

[0213] This per-pixel reconstruction procedure ensures that newly shaded pixels incorporate server-computed radiance updates through application of decoded delta-radiance values to motion-compensated prior-frame radiance, while pixels designated for reuse rely on temporally reprojected historical radiance. The use of motion-compensated back-projection provides consistent temporal correspondence across frames, even when a number and spatial distribution of newly shaded pixels varies between successive frames.

[0214] FIG. 23 is a diagram illustrating asynchronous multi-threaded execution of the distributed rendering pipeline according to an embodiment of the present disclosure. To achieve efficient and smooth real-time rendering in a client-server setting, D-HyShare employs an asynchronous distributed pipeline that decouples rendering, encoding, and data transmission operations. In some embodiments, rendering, encoding, and network input / output (I / O) are executed in parallel using a producer-consumer model, thereby enabling a rendering thread to proceed without blocking on GPU-CPU data transfers, compression, or network latency.

[0215] As illustrated in FIG. 23, both a server 2302 (a remote device) and a client 2304 (a local device) implement multi-threaded pipelines in which rendering, encoding or decoding, and network communication are performed by independent threads communicatively coupled via queues. On the server 2302, camera state updates received from the client 2304 are consumed by a rendering thread to update the V-buffer and to initiate path tracing. Output data generated by the rendering thread are provided to an encoding thread and a transmission thread, which operate asynchronously to encode and transmit payloads to the client 2304.

[0216] On the client 2304, received payloads are processed by a receive thread and a decode thread, and decoded data are integrated into a local frame buffer using shading reuse and motion-compensated reconstruction. This asynchronous processing enables continuous frame generation and display despite variability in network conditions.

[0217] In some embodiments, the client 2304 continuously transmits camera state packets to the server 2302 to support interactive rendering. The server 2302 selectively processes camera state packets by prioritizing a most recently received packet and discarding one or more earlier unprocessed packets, thereby maintaining rendering consistency and reducing network congestion associated with excessive request handling. For example, in FIG. 23, only the requests 1, 3, 5, 8 from the client 2304 are responded to by the server 2302 to render frames f1, f2, f3, f4. In someAtty. Docket: 4502-90001 (6000809PCT02)embodiments, a first time interval Atxrepresents an end-to-end system latency measured from transmission of a camera state packet by the client 2304 to display of a corresponding reconstructed frame. A second time interval At2represents a frame response interval corresponding to a rate at which frames are generated by the system. In some embodiments, a system frame rate is determined based on A t2rather than At1.

[0218] In some embodiments, At2is less than At1as the server 2302 initiates rendering of a subsequent frame upon completion of path tracing for a current frame, while encoding and transmission operations are performed asynchronously in parallel threads. This decoupled execution enables increased rendering throughput and improved responsiveness in distributed rendering environments.

[0219] Experiments Results

[0220] In some embodiments, the disclosed framework D-HyShare is implemented based on an Area ReSTIR rendering framework and evaluated across six representative scenes, including Bistro Day, Bistro Night, Bistro Wine, Emerald Square, Sun Temple, and Zero Day. These scenes encompass diverse conditions, including indoor and outdoor environments, varying material properties such as glossy and diffuse materials, and different levels of geometric and illumination complexity. For evaluation, a baseline configuration is established by executing Area ReSTIR locally on the client device. The disclosed D-HyShare Area approach is then evaluated under a distributed server-client configuration using identical scenes and rendering parameters, thereby enabling direct performance and quality comparisons.

[0221] In some embodiments, the experiments are performed at a predetermined image resolution of 1920 x 1080 pixels. A remote device (server) comprises a high-performance graphics processing unit (GPU), such as an RTX 5090, while a local device (client) comprises a lower-power GPU, such as an Intel Arc 140V. Both devices load identical scene assets and operate under synchronized camera states throughout the evaluation. In some embodiments, motion vectors are independently computed at both the client and server using the shared synchronized camera parameters, thereby ensuring consistent motion-compensated reprojection under dynamic viewpoint variations.

[0222] In addition, a full-frame Area ReSTIR rendering is generated at the server and designated as the reference ground truth for image quality assessment. This enables pixel-aligned comparisons between the server-generated reference frames and the reconstructed framesAtty. Docket: 4502-90001 (6000809PCT02)produced at the client. Furthermore, frame indices corresponding to keyframes and non-keyframes are explicitly logged to support subsequent analysis of bandwidth consumption, temporal update patterns, and shading refresh behavior.

[0223] Rendering Performance Across Scenes

[0224] In some embodiments, frame rate performance is evaluated by comparing a client device executing a baseline rendering pipeline locally against the same client operating within the D-HyShare distributed framework. The results across six diverse scenes, summarized in Table 5, which compares the frame rate of local baseline rendering on the client against the D-HyShare pipeline across six representative scenes. The results indicate that the baseline implementation achieves only 1-3 frames per second (fps) on the client due to the computational expense associated with path sampling and shading operations. In contrast, the D-HyShare Area configuration consistently achieves substantially higher performance, delivering approximately 31-54 fps across all evaluated scenes.Table 5. Frame rate (fps↑) comparison across six scenesBistro Bistro Bistro Emerald Sun Zero AverageDay Night Wine Square Temple Day of All Baseline Area 1.7 1.1 2.1 1.0 2.5 1.3 1.62 D-HyShare Area 36.4 35.5 54.7 31.9 50.3 38.5 41.2

[0225] On average, the D-HyShare Area framework achieves 41.2 fps on the client device, whereas the baseline achieves only 1.6 fps, corresponding to an average performance improvement of about 25x. This improvement demonstrates the effectiveness of offloading sampling and shading computations to a remote device, thereby significantly reducing the computational burden on the client and enabling real-time rendering on resource-constrained hardware. Variations in achieved frame rates across scenes reflect differences in scene characteristics, including geometric complexity and shading density; however, the disclosed approach maintains interactive performance levels across all tested scenarios.

[0226] Compute Load Redistribution

[0227] The observed performance improvement is attributable to effective redistribution of computational workload from the resource-constrained client device to the remote server. As reflected in Table 6, client-side resource utilization is significantly reduced when operating underAtty. Docket: 4502-90001 (6000809PCT02)the D-HyShare framework, after accounting for idle system overhead. In a baseline configuration executing Area ReSTIR locally, the client incurs substantial resource consumption, including an average of approximately 7.6 GB of GPU memory and 10.8 GB of CPU memory, while driving GPU utilization to near- saturation levels (e.g., approximately 96-98%). Despite this high utilization, the achieved frame rate remains limited to approximately 1-3 fps, indicating that the client is bottlenecked by the computational cost of path sampling and shading operations.Table 6. Client-side resource utilizationBistro Bistro Bistro Emerald Sun Zero Average Day Night Wine Square Temple Day of All CPU Memory GB)Baseline 9.8 10.6 9.3 10.7 12.3 12.0 10.8 D-HyShare 3.6 3.4 3.6 3.3 3.7 3.2 3.5 CPU Usage (%)Baseline 48 45 45 49 47 48 47D-HyShare 37 37 45 35 45 37 39 GPU Memory GB)Baseline 7.7 8.8 6.9 8.1 7.4 6.7 7.6D-HyShare 1.6 1.6 1.6 1.6 1.6 1.5 1.6 GPU Usage (%)Baseline 97 98 96 93 97 98 97D-HyShare 98 97 97 96 99 97 97

[0228] As reflected in Table 6, in contrast, the D-HyShare framework significantly reduces the client-side memory footprint while simultaneously enabling real-time rendering performance. Specifically, average GPU memory usage is reduced to approximately 1.6 GB and CPU memory usage to approximately 3.5 GB, corresponding to reductions of greater than 4x and 3x, respectively.

[0229] Under the baseline configuration, the client GPU is nearly fully saturated (96-98% utilization) yet achieves only 1-2 fps, indicating that performance is constrained by the computational intensity of path sampling and shading. In contrast, D-HyShare achieves real-time performance under comparable GPU utilization conditions by offloading computationally expensive sampling and shading operations to the server. On the client side, GPU execution is primarily dedicated to lightweight reconstruction and reuse-based shading, enabling substantially higher frame rates on the same hardware.

[0230] This redistribution of computation accounts for the significant performance gap between the two approaches. Although both configurations may exhibit high GPU utilization, D-HyShare relocates the most computationally intensive path tracing operations to the server,Atty. Docket: 4502-90001 (6000809PCT02)allowing the client to achieve frame rates exceeding 30 fps at 1920 x 1080 resolution while operating under a substantially reduced memory budget.

[0231] Overall, these results demonstrate that D-HyShare does not merely reduce resource utilization, but instead fundamentally redefines the client-side computational role. By offloading the heavy path tracing workload to the server, the client is transformed from a resource-constrained ray tracing system into an efficient frame composition and reconstruction pipeline, thereby enabling real-time rendering on hardware that is otherwise incapable of supporting interactive path tracing.

[0232] Data Encoding and Bitrate Saving

[0233] In some embodiments, D-HyShare employs a cyclic shading reuse strategy in which one out of every four pixels is refreshed per frame, while the remaining approximately 75% are marked for shading reuse. After applying validation criteria including occlusion testing, radiance variation detection, and motion consistency checks, the observed effective reuse ratio is reduced to an average of approximately 69% across all evaluated scenes. Through sparse pixel updates combined with quantized delta transmission, the proposed method substantially reduces per-frame data transmission requirements.

[0234] In some embodiments, bandwidth reduction is evaluated by measuring the resulting bitrate under varying quantization settings. As shown in Table 7, the joint encoding strategy comprising pixel masking, motion-compensated prediction, and delta quantization provides a controllable trade-off between bandwidth consumption and reconstructed image quality. Increasing the quantization step size Q from 2 to 8 improves the overall compression ratio from 91.97% to 96.76%. This level of compression corresponds to an estimated minimum bitrate of approximately 48.37 Mbps for a 1080p stream at 30 fps (for Q = 8). Importantly, even under aggressive quantization, high perceptual quality is preserved, with structural similarity (SSIM) remaining approximately 0.9715 and peak signal-to-noise ratio (PSNR) reaching approximately 36.63 dB.Table 7. Analysis of bandwidth-quality trade-offQ Savings (%) f MAE RMSE | PSNR(dB) j SSIM f 2 91.97 1.2289 3.7633 37.14 0.9733 3 92.73 1.2578 3.7704 37.08 0.9732 4 94.77 1.3068 3.7980 36.90 0.9726 6 96.03 1.3541 3.8286 36.75 0.9720Atty. Docket: 4502-90001 (6000809PCT02)7 96.44 1.3689 3.8391 36.70 0.9718 8 96.76 1.3878 3.8567 36.63 0.9715

[0235] FIG. 24 is a diagram illustrating a visual comparison of server- and client- rendered frames using D-HyShare according to an embodiment of the present disclosure. Qualitative assessment further demonstrates strong perceptual similarity between the client-rendered output and the server-rendered reference frames.

[0236] FIGS. 25A-25F are diagrams illustrating a set of per- frame FPS and bandwidth traces for six scenes according to an embodiment of the present disclosure. In some embodiments, end-to-end measured bitrates are further reported under the full D-HyShare pipeline with a quantization setting of Q = 8, including periodic key-frame insertion every 33 frames. Table 8 presents a comparative evaluation of bandwidth consumption between the proposed framework and raw RGB video streaming at a resolution of 1920 x 1080 with 24-bit color depth. In Table 8. the raw bitrate (Braw) is analytically computed as Braw = 1920 ■ 1080 ■ 3 • 8 • FPS, representing the theoretical data rate required for uncompressed RGB frame transmission at the observed frame rates. The compressed bitrate (Bcomp) corresponds to the empirically measured transmission rate achieved under the full D-HyShare pipeline, including masking, motion-compensated prediction, and delta quantization.

[0237] As shown in Table 8, the average compressed bitrate ranges from 51.44 Mbps of Emerald Square to 106.88 Mbps of Sun Temple, while the corresponding raw RGB bitrate computed based on observed frame rates, ranges from 1389.88 Mbps to 2948.76 Mbps. Overall, D-HyShare achieves consistent savings above 96% for all six scenes (96.30%-96.38%), with an average saving of 96.34%. These results demonstrate that the proposed mask-scan packing and delta quantization remain highly effective under realistic interactive workloads. Furthermore, the periodic insertion of key frames introduces only marginal overhead relative to the substantial savings achieved through sparse updates and residual coding.Table 8. Comparative evaluation of bandwidth consumption Scene FPS BComP(Mbps) Braw(Mbps) Saving (%) BistroDay 37.552 68.896 1868.828 96.31 BistroNight 36.154 66.394 1799.254 96.31 BistroWine 56.645 102.621 2819.018 96.36 EmeraldSquare 27.928 51.437 1389.876 96.30 SunTemple 59.252 106.875 2948.759 96.38Atty. Docket: 4502-90001 (6000809PCT02)ZeroDay 43.161 78.093 2147.968 96.36Avg↑ 43.449 79.053 2162.284 96.34

[0238] Rendering Latency

[0239] D-HyShare achieves an average rendering performance of 41.2 fps on the client (Table 5), corresponding to 24.3 ms per frame. In comparison, standalone Area ReSTIR executed on the server achieves an average performance of 62.4 fps, corresponding to a frame interval At2 of approximately 16.0 ms as illustrated in FIG. 23. The resulting difference of approximately 8.3 ms represents the additional overhead introduced by the distributed rendering pipeline beyond the core rendering computation. To characterize end-to-end frame latency in D-HyShare, the total elapsed time is measured from the moment the client issues a rendering request to the time the corresponding decoded frame is displayed on the client device. This end-to-end latency, denoted as Ati in FIG. 23, includes all stages of the pipeline, including server-side rendering, GPU-CPU readback, delta encoding, network transmission, client-side decoding, and final frame display.

[0240] FIGS. 26A-26F are diagrams illustrating a set of per- frame network latency for six test scenes according to an embodiment of the present disclosure. The latency profiles exhibit two consistent characteristics: first, latency is primarily governed by a relatively stable baseline with intermittent short-duration spikes; and second, both the frequency and magnitude of these spikes vary across scenes, reflecting differences in the number of sparsely updated pixels and the corresponding packet sizes generated by the delta-encoding scheme. Table 9 summarizes per-scene latency statistics. Across all evaluated scenes, the mean network latency ranges from approximately 24.1 ms to 55.5 ms, with standard deviations between approximately 5.8 ms and 14.8 ms. Notwithstanding this variability, D-HyShare consistently maintains real-time rendering performance at the client device. This robustness is facilitated by an asynchronous pipeline architecture in which the client continuously transmits updated camera states, the server renders based on the most recent request, and the client concurrently decodes and displays previously received frames, thereby effectively overlapping computation and communication stages.Table 9. End-to-end frame latency (ms) A ti from server to client (mean ± std).Atty. Docket: 4502-90001 (6000809PCT02)

[0241] Network Jitter Compensation via Frame Prediction

[0242] The observed client-side frame rate jitter, as illustrated in FIGS. 25A-25F, may result in perceptible display stutter. Variations in network transmission latency directly affect the arrival time of updated shading data, thereby causing the effective client frame rate to be more sensitive to network conditions than to encoding overhead. This indicates that temporal stability in D-HyShare is primarily constrained by network variability rather than computational performance. To mitigate the impact of such fluctuations and maintain interactive responsiveness, D-HyShare employs a motion-compensated frame prediction mechanism as a latency-hiding technique. In this approach, the client utilizes a locally maintained history buffer and performs motion-compensated reprojection to synthesize a predicted frame without waiting for newly received shading updates. The prediction mechanism is selectively activated only when an update packet Pt from the server is delayed beyond an expected interval, thereby ensuring that predictive reconstruction is invoked adaptively and only when necessary to preserve visual continuity.

[0243] FIGS. 27A-27B are diagrams illustrating prediction loss trends over multiple consecutive frames, according to an embodiment of the present disclosure. To evaluate the longterm robustness of the proposed prediction mechanism, network delay is artificially introduced for 20 consecutive frames, thereby forcing the client to rely exclusively on predicted frames without receiving updated shading data from the server.

[0244] The results, as shown in FIGS. 27A-27B, assess prediction quality using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Peak Signal-to-Noise Ratio (PSNR) for each predicted frame. The analysis indicates a gradual degradation in prediction accuracy over time, with PSNR decreasing from an initial value of approximately 35.89 dB to approximately 26.26 dB after 20 consecutive predicted frames, reflecting the accumulation of temporal error in the absence of refreshed data.

[0245] FIG. 28 is a diagram illustrating a qualitative panel showing predicted frames and corresponding error maps according to an embodiment of the present disclosure. The observed visual errors predominantly arise from dynamic scene components, including moving objects and newly exposed regions resulting from occlusion changes (e.g., geometry revealed due to rapidAtty. Docket: 4502-90001 (6000809PCT02)camera motion), which cannot be accurately reconstructed by the client in the absence of updated server-side information. These observations indicate that while the forward projection-based prediction method is effective in mitigating short-term network disruptions, its accuracy degrades under prolonged reliance on prediction. The resulting errors remain largely localized to dynamically changing regions, while temporally stable areas continue to be reconstructed with comparatively high fidelity. Once the server’s next frame arrives, the binary mask forces updates in these regions, and the reconstruction quickly converges to the correct shading over 1-2 frames.

[0246] Ablation Study

[0247] To further characterize the performance of the proposed D-HyShare framework, a series of ablation studies are conducted to isolate the effects of algorithmic complexity, server hardware capability, and image resolution on overall system behavior.

[0248] Algorithmic Complexity: To evaluate the flexibility of D-HyShare with respect to different rendering backends, experiments are performed using multiple server-side sampling techniques, including Area ReSTIR, ReSTIR DI, and ReSTIR PT. As summarized in Table 10, D-HyShare DI consistently achieves the highest throughput across all evaluated scenes, reaching frame rates in the range of approximately 47.1-69.9 fps, with an average of 57.58 fps. In comparison, D-HyShare PT achieves lower performance, ranging from approximately 29.1-38.1 fps, with an average of 34.20 fps. These results indicate that reduced path depth and lower sampling complexity in DI directly translate into higher update rates at the server. Accordingly, D-HyShare enables a tunable trade-off between rendering fidelity and performance by selecting the underlying sampling method, and is inherently compatible with future improvements in sampling algorithms and hardware acceleration.Table 10. Frame rate (fps) of D-HyShare PT and D-HyShare DI, with GPU 5090 Bistro Bistro Bistro Emerald Sun Zero AvgDay Night Wine Square Temple DayD-HyShare PT 31.2 35.1 36.3 29.1 38.1 35.4 34.20 D-HyShare DI 47.1 53.3 69.0 54.7 69.9 51.5 57.58

[0249] Server Hardware Scaling: To assess the impact of server-side computational resources, D-HyShare performance is evaluated across different GPU generations. As shown in Table 7, an RTX 4090-based server achieves average frame rates of approximately 16.15 fps (ReSTIR PT),Atty. Docket: 4502-90001 (6000809PCT02)35.92 fps (ReSTIR DI), and 19.25 fps (Area ReSTIR) across the evaluated scenes. When upgrading to an RTX 5090, performance increases substantially, with ReSTIR PT improving from 16.15 fps to 34.20 fps (approximately 2.12x), and ReSTIR DI improving from 35.92 fps to 57.58 fps (approximately 1.60x), as also reflected in Table 11. These results confirm that D-HyShare scales effectively with increased server- side compute capability, directly translating hardware improvements into higher rendering throughput and enabling either increased update frequency or more complex lighting configurations within the same client-side constraints.Table 11. Frame rate (fps) of D-HyShare with GPU 4090Bistro Bistro Bistro Emerald Sun Zero AvgDay Night Wine Square Temple DayD-HyShare PT 16.4 15.5 16.3 15.1 18.1 15.5 16.15 D-HyShare DI 37.1 33.3 39.0 34.7 39.9 31.5 35.92 D-HyShare Area21.2 17.8 20.7 16.2 21.1 18.5 19.25

[0250] Resolution Scaling: The scalability of D-HyShare with respect to image resolution is evaluated by repeating experiments at a reduced resolution of 1280 x 720. As reported in Table 12, standalone Area ReSTIR remains non-interactive on the client, achieving only approximately 1.7-5.1 fps across scenes. In contrast, D-HyShare consistently maintains real-time performance, achieving frame rates in the range of approximately 55.1-83.3 fps, with an average of 66.5 fps. These results demonstrate that the advantages of distributed sampling and shading reuse persist across resolutions, allowing resource-constrained client devices to effectively leverage increased server-side throughput.Table 12. Frame rate (fps) at 1280 x 720 resolutionBistro Bistro Bistro Bistro Sun ZeroDay Night Wine Square Temple Day Avg Baseline Area 2.9 3.1 4.2 1.7 5.1 2.3 3.2D-HyShare Area 55.1 60.6 83.3 65.8 80.4 53.8 66.5

[0251] Collectively, these ablation studies demonstrate that D-HyShare is not limited to a specific resolution, rendering algorithm, or hardware configuration. Instead, the framework robustly propagates server-side performance gains to the client, enabling consistent real-time photorealistic rendering across a wide range of system conditions.

[0252] In summary, the disclosed Distributed HyShare framework provides a client-server architecture that decouples sample reuse from shading reuse to enable efficient real-timeAtty. Docket: 4502-90001 (6000809PCT02)photorealistic rendering in interactive applications, including virtual reality and metaverse environments. By leveraging pixel-level motion vectors and transmitting only differential updates for newly rendered pixels, the framework significantly reduces bandwidth requirements compared to full-frame transmission approaches. Under adverse network conditions, such as increased latency or jitter, the client reconstructs intermediate frames using temporally reprojected history buffers, thereby maintaining visual continuity until updated data is received. This design enables real-time physically based rendering on thin-client devices and establishes a scalable foundation for future distributed rendering systems that are network-aware, resource-efficient, and capable of delivering high-quality interactive graphics.

[0253] FIG. 29 is a method 2900 implemented by a server according to an embodiment of the disclosure. In block 2902, the method comprises performing path tracing to generate radiance values for a subset of pixels of a frame. In some embodiments, performing the path tracing comprises maintaining per-pixel sample reservoirs across successive frames to accumulate and reuse candidate samples. In some embodiments, performing the path tracing comprises executing reservoir-based spatiotemporal importance resampling to reuse and resample candidate light transport paths across pixels and frames, thereby generating updated radiance values for the subset of pixels.

[0254] In block 2904, the method further comprises generating a binary mask identifying newly shaded pixels of the frame from the subset of pixels, wherein the newly shaded pixels correspond to pixels for which radiance values are generated in the subset of pixels. In some embodiments, the binary mask is organized for raster-order traversal such that each mask value corresponds to a respective pixel location in the frame.

[0255] In block 2906, the method further comprises determining, for the newly shaded pixels, delta-radiance values relative to a motion-compensated prior-frame radiance. In some embodiments, determining the delta-radiance values comprises determining the motion-compensated prior-frame radiance by back-projecting pixel locations of the newly shaded pixels to a prior frame based on motion vectors; and determining the delta-radiance values as differences between current radiance values of the newly shaded pixels and the motion-compensated priorframe radiance.

[0256] In block 2908, the method further comprises encoding the binary mask and the deltaradiance values into a payload. In some embodiments, the encoding comprises quantizing theAtty. Docket: 4502-90001 (6000809PCT02)delta-radiance values to obtain quantized delta-radiance values; packing the quantized deltaradiance values corresponding to the newly shaded pixels into a one-dimensional buffer; and compressing the one-dimensional buffer and the binary mask to form the payload

[0257] In block 2910, the method further comprises transmitting the payload to a client for reconstruction of a full-resolution frame.

[0258] FIG. 30 is a method 3000 implemented by a client according to an embodiment of the disclosure. In block 3002, the method comprises receiving, from a server, a payload comprising a binary mask and delta-radiance values associated with a subset of pixels of a frame. In some embodiments, the binary mask identifies newly shaded pixels of the frame and a one-dimensional buffer comprises the delta-radiance values corresponding to the newly shaded pixels, wherein the delta-radiance values are generated at the server based on path tracing performed for the subset of pixels.

[0259] In block 3004, the method further comprises decoding the payload to recover the binary mask and the delta-radiance values. In some embodiments, the client decompresses the payload and reconstructs the one-dimensional delta-radiance buffer in association with the binary mask.

[0260] In block 3006, the method further comprises determining motion-compensated priorframe radiance for pixels of the frame. In some embodiments, determining the motion-compensated prior-frame radiance comprises: back-projecting a pixel location to a prior frame using a motion vector; and performing bilinear interpolation using neighboring pixel samples in the prior frame.

[0261] In block 3008, the method further comprises reconstructing a full-resolution frame by applying the delta-radiance values to pixels identified as newly shaded based on the binary mask and reusing the motion-compensated prior-frame radiance for pixels not identified as newly shaded. In some embodiments, reconstructing the frame comprises, for each pixel identified as newly shaded, adding a corresponding delta-radiance value to the motion-compensated priorframe radiance, and for each pixel not identified as newly shaded, setting a radiance value of that pixel equal to the motion-compensated prior-frame radiance.

[0262] In block 3010, the method further comprises displaying the full-resolution frame.

[0263] FIG. 31 is a schematic diagram of a computing device 3100 (e.g., a personal computer, smart phone, smart tablet, handheld gaming device, etc.) according to an embodiment of the disclosure. The computing device 3100 is suitable for implementing the disclosed embodimentsAtty. Docket: 4502-90001 (6000809PCT02)as described herein. The computing device 3100 comprises ingress ports / ingress means 3110 (a.k.a., upstream ports) and receiver units (Rx) / receiving means 3120 for receiving data; a processor, logic unit, or central processing unit (CPU) / processing means 3130 to process the data; transmitter units (Tx) / transmitting means 3140 and egress ports / egress means 3150 (a.k.a., downstream ports) for transmitting the data; and a memory / memory means 3160 for storing the data. In an embodiment, the receiver units (Rx) / receiving means 3120 comprise a discrete circuit, integrated circuit, chip set, package, hardware module, electronic device, or other structure capable of receiving signals. In an embodiment, the transmitter units (Tx) / transmitting means 3140 comprise a discrete circuit, integrated circuit, chip set, package, hardware module, electronic device, or other structure capable of transmitting signals. The computing device 3100 may also comprise optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the ingress ports / ingress means 3110, the receiver units / receiving means 3120, the transmitter units / transmitting means 3140, and the egress ports / egress means 3150 for egress or ingress of optical or electrical signals.

[0264] The processor / processing means 3130 is implemented by hardware and software. The processor / processing means 3130 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor / processing means 3130 is in communication with the ingress ports / ingress means 3110, receiver units / receiving means 3120, transmitter units / transmitting means 3140, egress ports / egress means 3150, and memory / memory means 3160. The processor / processing means 3130 comprises a HyShare / D-Hyshare module 3170. The HyShare / D-Hyshare module 3170 is able to implement the methods disclosed herein. The inclusion of the HyShare / D-Hyshare module 3170 therefore provides a substantial improvement to the functionality of the computing device 3100 and effects a transformation of the computing device 3100 to a different state. Alternatively, the HyShare / D-Hyshare 3170 is implemented as instructions stored in the memory / memory means 3160 and executed by the processor / processing means 3130.

[0265] The computing device 3100 may also include input and / or output (I / O) devices or I / O means 3180 for communicating data to and from a user. The I / O devices or I / O means 3180 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. The I / O devices or I / O means 3180 may also include input devices, such as a keyboard,Atty. Docket: 4502-90001 (6000809PCT02)mouse, trackball, etc., and / or corresponding interfaces for interacting with such output devices. The memory / memory means 3160 comprises one or more disks, tape drives, and solid-state drives and may be used as an over-flow data storage device, to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. The memory / memory means 3160 may be volatile and / or non-volatile and may be read-only memory (ROM), random-access memory (RAM), ternary content-addressable memory (TCAM), and / or static RAM (SRAM).

[0266] While several embodiments have been provided in the present disclosure, it may be understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.

[0267] In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Claims

Atty. Docket: 4502-90001 (6000809PCT02)CLAIMSWhat is claimed is:

1. A method implemented by a server, comprising:performing path tracing to generate radiance values for a subset of pixels of a frame; generating a binary mask identifying newly shaded pixels of the frame from the subset of pixels;determining, for the newly shaded pixels, delta-radiance values relative to a motion- compensated prior-frame radiance;encoding the binary mask and the delta-radiance values into a payload; and transmitting the payload to a client.

2. The method of claim 1, wherein performing the path tracing comprises maintaining per- pixel sample reservoirs across successive frames to generate the radiance values.

3. The method of any of claims 1-2, wherein performing the path tracing comprises executing reservoir-based spatiotemporal importance resampling.

4. The method of any of claims 1-3, further comprising determining the subset of pixels based on a cycling shading reuse schedule, wherein the cycling shading reuse schedule selects one pixel in each 2x2 pixel group per frame for path tracing and designates remaining pixels for shading reuse, and wherein at keyframes occurring at periodic intervals, the server performs the path tracing for all pixels of the frame.

5. The method of claim 4, further comprising:validating pixels designated for shading reuse based on reuse eligibility criteria, wherein the reuse eligibility criteria comprise at least one of an occlusion condition, a radiance change condition, or a motion condition; andreclassifying pixels failing the reuse eligibility criteria as newly shaded pixels.Atty. Docket: 4502-90001 (6000809PCT02)6. The method of any of claims 1 -5, wherein determining the delta-radiance values comprises:determining the motion-compensated prior-frame radiance by back-projecting pixel locations to a prior frame; andsubtracting the motion-compensated prior-frame radiance from corresponding current radiance values of the newly shaded pixels.

7. The method of any of claims 1-6, wherein the encoding comprises:quantizing the delta- radiance values to obtain quantized delta-radiance values; packing the quantized delta-radiance values corresponding to the newly shaded pixels into a one-dimensional buffer; andcompressing the one-dimensional buffer and the binary mask to form the payload.

8. The method of any of claims 1-7, wherein quantizing the delta-radiance values comprises applying a uniform linear quantizer mapping the delta-radiance values into a signed 8-bit range.

9. The method of any of claims 1-8, further comprising receiving, from the client, a camera state packet comprising a frame index, a timestamp, a camera pose, and projection parameters.

10. The method of any of claims 1-9, wherein the binary mask is arranged for raster-order traversal.

11. The method of any of claims 1-10, wherein performing the path tracing, encoding, and transmitting are executed asynchronously using a plurality of threads.

12. The method of any of claims 1-11, further comprising operating a rendering thread, an encoding thread, and a transmission thread connected via queues in a producer-consumer model.

13. The method of any of claims 1-12, further comprising selecting, for processing, a most recent camera state packet received from the client and discarding one or more earlier unprocessed camera state packets.Atty. Docket: 4502-90001 (6000809PCT02)14. A method implemented by a client, the method comprising:receiving, from a server, a payload comprising a binary mask and delta-radiance values associated with a subset of pixels of a frame;decoding the payload to recover the binary mask and the delta-radiance values; determining motion-compensated prior-frame radiance for pixels of the frame; reconstructing a full-resolution frame by applying the delta-radiance values to pixels identified as newly shaded based on the binary mask and reusing the motion-compensated priorframe radiance for pixels not identified as newly shaded; anddisplaying the full-resolution frame.

15. The method of claim 14, wherein reconstructing the frame comprises, for each pixel identified as newly shaded, adding a corresponding delta-radiance value to the motion- compensated prior-frame radiance.

16. The method of any of claims 14-15, wherein reconstructing the frame comprises, for each pixel not identified as newly shaded, setting a radiance value of that pixel equal to the motion- compensated prior-frame radiance.

17. The method of any of claims 14-16, wherein determining the motion-compensated priorframe radiance comprises:back-projecting a pixel location to a prior frame using a motion vector; and performing bilinear interpolation using neighboring pixel samples in the prior frame.

18. The method of any of claims 14-17, further comprising storing prior-frame radiance, motion vectors, depth, and normal information computed locally from synchronized camera for use in motion-compensated reconstruction.

19. The method of any of claims 14-18, further comprising performing temporal accumulation across successive reconstructed frames.Atty. Docket: 4502-90001 (6000809PCT02)20. The method of any of claims 14-19, further comprising generating a predicted frame using motion-compensated reprojection in response to delayed receipt of the payload.

21. The method of any of claims 14-20, further comprising updating a locally maintained velocity buffer using received synchronization data.

22. The method of any of claims 14-21, further comprising transmitting, to the server, a camera state packet comprising a frame index, a timestamp, a camera pose, and projection parameters.

23. The method of any of claims 14-22, wherein decoding and reconstructing are performed asynchronously using a plurality of threads.

24. The method of any of claims 14-23, further comprising operating a receive thread, a decode thread, a frame reconstruction thread, and a display thread connected via queues.

25. The method of any of claims 14-24, wherein the binary mask indicates newly shaded pixels using a first value and shading-reused pixels using a second value.

26. The method of any of claims 14-25, wherein the delta-radiance values are stored in a onedimensional buffer that is sequentially consumed in correspondence with raster-order traversal of the binary mask.

27. A system, comprising:a server comprising one or more processors and a memory storing instructions that, when executed, cause the server to:perform path tracing to generate radiance values for a subset of pixels of a frame; generate a binary mask identifying newly shaded pixels of the frame from the subset of pixels;determine, for the newly shaded pixels, delta-radiance values relative to a motion- compensated prior-frame radiance;encode the binary mask and the delta-radiance values into a payload; andAtty. Docket: 4502-90001 (6000809PCT02)transmit the payload over a network; anda client device communicatively coupled to the server and comprising one or more processors and a memory storing instructions that, when executed, cause the client device to:receive the payload;decode the binary mask and the delta-radiance values;determine a motion-compensated prior-frame radiance for pixels of the frame; reconstruct a full-resolution frame by applying the delta-radiance values to pixels identified as newly shaded based on the binary mask and reusing the motion-compensated prior-frame radiance for pixels not identified as newly shaded; anddisplay the full-resolution frame.

28. A system comprising:a memory configured to store instructions; andone or more processors coupled to the memory and configured to execute the instructions to cause the system to perform a method according to any of claims 1-13.

29. A system, comprising:a storage means configured to store instructions; andone or more processing means coupled to the storage means and configured to execute the instructions to the system to perform a method according to any of claims 1-13.

30. A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer-executable instructions when executed by one or more processors of a system, cause the system to perform a method according to any of claims 1-13.

31. A non-transitory computer- readable medium storing instructions that, when executed by one or more processors, cause a system to perform operations according to any of claims 1-13.

32. A system comprising:a memory configured to store instructions; andAtty. Docket: 4502-90001 (6000809PCT02)one or more processors coupled to the memory and configured to execute the instructions to cause the system to perform a method according to any of claims 14-26.

33. A system, comprising:a storage means configured to store instructions; andone or more processing means coupled to the storage means and configured to execute the instructions to the system to perform a method according to any of claims 14-26.

34. A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer-executable instructions when executed by one or more processors of a system, cause the system to perform a method according to any of claims 14-26.

35. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a system to perform operations according to any of claims 14-26.

Images