Mobile terminal real-time ray tracing noise reduction method and system based on edge perception

Abstract

The invention discloses a mobile terminal real-time ray tracing noise reduction method and system based on edge perception, and the method comprises the steps: carrying out the multi-channel edge detection of the G-buffer information of an input frame, and extracting an image edge; generating a complete edge density map of the current extraction frame; dynamically dividing the pixels into calculation groups with different kernel sizes according to the edge density value; performing pixel clustering in each kernel size calculation group by using local depth gradient information; for each sub-cluster, brightness (obtained by albedo conversion), surface normal and depth information in each pixel G-buffer in the sub-cluster are obtained, a filter weight corresponding to each sub-cluster is obtained by using a lightweight multilayer perceptron network, and a global weight lookup table is generated; and quantizing the continuous feature space of the frame to be detected into a discrete index, realizing weight retrieval by using the global weight lookup table, and finally executing spatial filtering according to the allocated kernel size and weight to output a noise-reduced image. The real-time performance and the quality are both considered, and the noise reduction efficiency is improved on the premise that the noise reduction effect is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of real-time rendering technology, and in particular to a method and system for real-time ray tracing noise reduction on mobile devices based on edge awareness. Background Art

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the evolution of mobile GPU architecture, mobile platforms (SoCs) such as the Qualcomm Snapdragon 8 Gen3 and MediaTek Dimensity 9300 have integrated hardware ray tracing units, making real-time ray tracing possible on mobile devices. However, due to the power consumption and heat dissipation constraints of mobile devices, the sampling rate is usually limited to less than 1 sample / pixel (1spp), resulting in significant Monte Carlo noise and temporal artifacts (such as flickering and ghosting) in the rendered frames.

[0004] The current mainstream noise reduction solution, Spatiotemporal Variance Guided Filtering (SVGF), achieves efficient noise suppression on desktop platforms by fusing information from multiple frames and combining filtering guided by a geometric buffer (G-buffer). However, its core reliance on a fixed-size filter kernel and pixel-by-pixel floating-point variance estimation results in significant computational overhead when deployed on mobile GPUs.

[0005] With the rise of machine learning, neural network-based denoising methods have emerged, focusing on end-to-end mapping learning from low-sampled inputs to high-quality outputs. However, existing data-driven methods face an inherent trade-off between model complexity and real-time performance on mobile devices, and have limited optimization for mobile GPU architectures. Although subsequent achievements, such as the Real-Time Denoising Network (RTDNet), employ the U-Net architecture with depthwise separable convolutions to minimize parameters, they still cannot meet the real-time requirements of mobile devices.

[0006] To adapt to mobile platforms, existing technologies have proposed several optimization solutions. For example, Qualcomm's shadow noise denoiser achieves a 4x speedup by utilizing pre-computed light field features, but it only supports specific light source types and lacks generalization. OPPO's hybrid rendering engine combines rasterization and ray tracing noise reduction, which can reduce power consumption by 15%, but it results in a 15% loss of material reflection accuracy, affecting the quality of rendered images. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a mobile real-time ray tracing denoising method and system based on edge awareness. Through edge adaptive filtering, data-driven weight prediction, and motion-aware reuse, it achieves efficient real-time denoising for 1spp ray tracing, making it particularly suitable for denoising in real-time rendering of mobile images.

[0008] In some implementations, the following technical solutions are adopted: A real-time ray tracing noise reduction method for mobile devices based on edge awareness, comprising: Multi-channel edge detection is performed on the input frame G-buffer information to extract image edges; The edge density values ​​of historical frames are reprojected using motion vectors, and the edge density values ​​of the currently extracted frame are fused using a hole-filling strategy to generate a complete edge density map of the currently extracted frame, thereby quantifying the local edge concentration. Pixels are dynamically divided into computation groups with different kernel sizes based on edge density values; Within each kernel size calculation group, pixel clustering is performed using local depth gradient information, and each kernel group is further divided into several sub-clusters with similar geometric continuity and surface orientation. For each sub-cluster, the brightness (derived from albedo), surface normal, and depth information of each pixel's G-buffer are obtained. A lightweight multilayer perceptron network is used to obtain the filter weights corresponding to each sub-cluster, and then a global weight lookup table is generated. The continuous feature space of the frame to be tested is quantized into a discrete index, and a global weight lookup table is used to perform weight retrieval. Finally, spatial filtering is performed according to the allocated kernel size and weight to output a denoised image.

[0009] As a further solution, the G-buffer information includes three channels: depth, normal, and albedo. The directional gradient on each channel is calculated, normalized, and then weighted and fused. The fused gradient value is compared with an adaptive threshold to obtain a binary mask of the image edge, thereby obtaining the image edge region.

[0010] As a further solution, the adaptive threshold is specifically as follows: ; in, Represents total pixels. It is a set constant value used to balance edges and noise; For adaptive threshold, Represents pixels The gradient values ​​after fusion; This represents the mean of the gradient magnitude, reflecting the overall gradient strength. The standard deviation of the gradient magnitude measures the degree of dispersion of the gradient distribution.

[0011] As a further solution, for each pixel its edge density Defined as local neighborhood Ratio of inner edge pixels: ; in, It is based on pixels Centered window, Indicate its area; Represents pixels Edge binarization mask.

[0012] As a further solution, for the current extracted frame t Pixels Its frame The corresponding position in is ; If pixels If the edge density value is within the valid range of the previous frame, then the edge density value of the previous frame is directly reused. As the current extracted frame t Pixels Edge density values; If pixels If the location is not within the valid range of the previous frame, it is marked as a hole, and the edge density value at that location is recalculated. The current extracted frame t The edge density values ​​of all pixels in the image are fused to generate the current extracted frame. t The complete edge density map.

[0013] As a further approach, pixel clustering is performed within each kernel size calculation group using local depth gradient information. Each kernel group is further divided into several sub-clusters with similar surfaces, making them exhibit similar geometric continuity and surface orientation. Specifically: For each pixel within a kernel-sized computational group, a feature vector is constructed. ,in Indicates the depth gradient magnitude. Indicates the direction of the normalized gradient; K-means clustering is used to divide pixels within each kernel group into clusters based on minimizing the within-cluster variance. Sub-clustering.

[0014] As a further approach, the continuous feature space of the frame to be tested is quantized into a discrete index, specifically: The continuous feature space includes the brightness (derived from albedo), surface normals, and depth features in the G-buffer; Specifically, the brightness L is quantized into a discrete index, as follows: , Indicates the brightness quantization index. Indicates the number of brightness quantization intervals; Quantizing surface normals into discrete indices specifically involves converting the normals into spherical coordinates. ,in, Polar angle, The azimuth angle is then uniformly quantized to... Each box: ; in, and They represent polar angles respectively. and azimuth Quantitative indicators This indicates the number of normal quantization intervals, ensuring isotropic sampling on a unit sphere. Depth is quantized into discrete indices, specifically: depth values. After logarithmic scaling and quantization, Each interval: ; in and To define the depth range of the scene, Indicates a deep quantization index. Indicates the number of deep quantization intervals.

[0015] In other embodiments, the following technical solutions are adopted: A real-time ray tracing noise reduction system for mobile devices based on edge awareness, comprising: The edge extraction module is configured to perform multi-channel edge detection on the input frame G-buffer information to extract image edges; The edge density calculation module is configured to use motion vector reprojection of the edge density values ​​of historical frames, fuse the edge density values ​​of the currently extracted frame through a hole-filling strategy, and generate a complete edge density map of the currently extracted frame to quantify the local edge concentration. The pixel partitioning module is configured to dynamically partition pixels into computational groups with different kernel sizes based on edge density values; within each kernel size computational group, pixel clustering is performed using local depth gradient information, and each kernel group is further divided into several sub-clusters with similar geometric continuity and surface orientation; The weight calculation module is configured to obtain the brightness, surface normal, and depth features of each pixel in the G-buffer for each sub-cluster, and use a lightweight multilayer perceptron network to obtain the filter weights corresponding to each sub-cluster; and then generate a global weight lookup table. The image denoising module is configured to quantize the continuous feature space of the frame under test into a discrete index, use a global weight lookup table to perform weight retrieval, and finally perform spatial filtering according to the assigned kernel size and weights to output a denoised image.

[0016] In other embodiments, the following technical solutions are adopted: A terminal device includes a processor and a memory, wherein the processor is used to implement instructions; and the memory is used to store multiple instructions adapted to be loaded and executed by the processor to perform the aforementioned edge-aware real-time ray tracing noise reduction method for mobile devices.

[0017] In other embodiments, the following technical solutions are adopted: A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned edge-aware mobile real-time ray tracing noise reduction method.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention dynamically divides pixels into computational groups with different kernel sizes based on edge density values. Then, within each kernel-sized computational group, pixel clustering is performed using local depth gradient information. Each kernel group is further divided into several sub-clusters with similar surfaces, making them exhibit similar geometric continuity and surface orientation. Then, a lightweight multilayer perceptron network is used to calculate the filter weights corresponding to each sub-cluster. The filter kernel size is dynamically determined based on edge density values. Small kernels are used to preserve details in high-complexity regions, while large kernels are used to improve noise suppression efficiency in low-complexity regions. Balancing real-time performance and quality, this invention improves noise reduction efficiency while ensuring noise reduction effect, and solves the dilemma of real-time noise reduction on mobile devices without significantly increasing computational overhead.

[0019] (2) To amplify edge features and suppress noise, this invention designs an adaptive threshold driven by global gradient statistics. By applying the adaptive threshold to binarize the fused edge image, it overcomes the limitation of traditional fixed threshold methods in adapting to varying gradient distributions under different lighting and material conditions. Specifically, the mean... Reflecting the overall gradient intensity, the threshold is automatically adjusted according to the light intensity: lowering the threshold in low light to avoid missed detections, and raising the threshold in strong light to prevent oversensitivity. Standard deviation To address material variations, the threshold is automatically increased in complex areas (such as the boundary between metal and diffuse reflection) to reduce false positives. (Empirical coefficient) It provides flexible adjustment space, through adjustment The value strikes a balance between edge detection and noise reduction; ultimately it can robustly segment perceptually significant edges while suppressing Monte Carlo noise and adapting to gradient distributions in scenes with complex lighting (such as contrast between indoor low light and outdoor high light) and mixed materials (such as the interplay of metal and diffuse surfaces).

[0020] (3) This invention utilizes motion vectors to reproject the edge density map of the previous frame and uses hole filling technology to ensure consistency across time steps; it can reduce the edge density calculation frequency and enhance time stability; and finally realizes time coherent filtering, saving the overhead of recalculating the entire frame. (4) This invention uses a lightweight multilayer perceptron network to predict the filter weights of each pixel based on a concise feature set, thus effectively serving as a learning alternative function for the SVGF spatial weighting function. The predicted weights are stored in a compact lookup table, enabling efficient inference through table indexing during the rendering process.

[0021] Other features and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] Figure 1 This is a flowchart of the real-time ray tracing noise reduction method for mobile devices based on edge awareness, as described in this embodiment of the invention. Figure 2 This is a schematic diagram of the real-time ray tracing noise reduction process for mobile devices based on edge awareness in an embodiment of the present invention; Figure 3 This is a diagram illustrating the execution effect in an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0025] Example 1 In one or more embodiments, a real-time ray tracing noise reduction method for mobile devices based on edge awareness is disclosed, combined with... Figure 1 and Figure 2 Specifically, it includes the following process: S101: Perform multi-channel edge detection on the input frame G-buffer information to extract image edges.

[0026] Typically, a scene consists of multiple input frame images. In this embodiment, multiple input frame images under different scenes are obtained, and the edges of each input frame image are extracted.

[0027] In this embodiment, the G-buffer information includes depth, normal, and albedo, and image edges are extracted through multi-channel edge detection.

[0028] Specifically, the Sobel operator (a computationally efficient edge detector that strikes a balance between gradient sensitivity and noise resilience) is used to calculate the directional gradients in the depth, normal, and albedo channels of each pixel's G-buffer. For each channel... Using a 3×3 Sobel kernel, the levels ( ) and vertical ( The gradient, based on the horizontal and vertical gradients, is calculated for the gradient magnitude of each channel, as follows: ; in, and Used to encode changes in directional intensity, these quantities encode different edge semantics: depth gradients highlight geometric discontinuities (e.g., object contours), normal gradients capture changes in surface curvature (e.g., tessellation artifacts), and albedo gradients detect abrupt material transitions (e.g., the boundary between metal and non-metal).

[0029] To synthesize robust edge priors, these quantities were normalized to And it is fused through weighted combination: ; Among them, 0.5, 0.3 and 0.2 are the weights of albedo, depth and normal channel, respectively. The weight design prioritizes material edges while suppressing false depth noise. , and These represent the normalized albedo, depth, and normal channel gradient magnitudes in the G-buffer for each pixel. This represents the gradient magnitude after fusion.

[0030] According to gradient calculations, edges exhibit high-amplitude gradients, while non-edge regions exhibit low-amplitude gradients. Existing techniques typically set a fixed threshold, identifying regions with gradient amplitudes greater than this threshold as edge regions; however, a fixed threshold cannot adapt to varying gradient distributions under different lighting and material conditions, leading to errors in edge determination.

[0031] Based on this, this embodiment proposes an adaptive thresholding strategy driven by global gradient statistics, which uses an adaptive threshold T to adjust the gradient magnitude after fusion. Binarization can amplify edge features and suppress noise, enabling accurate extraction of edge regions.

[0032] Specifically, the adaptive threshold T is calculated as follows: ; in, This represents the total number of pixels, and k is a constant value used to balance edges and noise. In this embodiment, k=1.2. For adaptive threshold, Represents pixels The gradient value after fusion. This represents the mean of the gradient magnitude, reflecting the overall gradient strength. The standard deviation of the gradient magnitude measures the dispersion of the gradient distribution. By statistically analyzing the global gradient distribution (mean and variance), the threshold is dynamically adjusted to avoid the failure of a fixed threshold in low-light (generally small gradient magnitude) or high-noise (largely discrete gradient magnitude) scenarios.

[0033] Edge masking using an adaptive threshold T Binarization is performed as follows: ; edge mask A value of 1 indicates an edge region, while a value of 0 indicates a non-edge region.

[0034] This statistically driven approach can robustly segment perceptually salient edges while suppressing Monte Carlo noise and adapting to gradient distributions in scenes with complex lighting (such as contrast between indoor low light and outdoor high light) and mixed materials (such as the interplay of metal and diffuse surfaces).

[0035] S102: The edge density values ​​of historical frames are reprojected using motion vectors, and the edge density values ​​of the currently extracted frames are fused using a hole-filling strategy to generate a complete edge density map of the currently extracted frames, thereby quantifying the local edge concentration.

[0036] In this embodiment, after the edge region is extracted, the edge density is calculated to quantify the local edge concentration, which can serve as a key indicator for adaptive kernel size in the subsequent denoising stage.

[0037] For each pixel its edge density Defined as local neighborhood Ratio of inner edge pixels: ; in, Represents pixels Edge binarization mask; It is based on pixels Centered The window, r, is the side length of a rectangular window centered on this pixel used to calculate edge density. A larger r contains more pixels. The value of r is chosen from 9, 17, and 33, starting with a 9x9 pixel area for edge density calculation. Does it satisfy the condition of being greater than or equal to 0.5? If so, then r = 9. If not, then calculate the edge density using 17*17. Determine whether the condition is greater than or equal to 0.3. If it is, then r = 17; if not, then r = 33.

[0038] It represents its area; this density measure quantifies the edge sparsity, where Indicates edge clustering, This indicates edge homogeneity.

[0039] To improve the computational efficiency of edge density estimation, this embodiment utilizes motion vectors in temporal reprojection to reuse edge information from the previous frame in static or rigid motion regions.

[0040] Specifically, for the current extracted frame t Pixels Its frame The corresponding position in is : ; in, It is a motion vector.

[0041] If pixels If the edge density value is within the valid range of the previous frame, then the edge density value of the previous frame is directly reused. As the current extracted frame t Pixels The edge density value is used to avoid redundant calculations.

[0042] If pixels If a location is not within the valid range of the previous frame, it is marked as a hole, and the edge density value at that location is recalculated.

[0043] Finally, extract the current frame t The edge density values ​​of all pixels in the image are fused to generate the current extracted frame. t A complete edge density map ensures spatiotemporal consistency.

[0044] Through the above process, the edge density map of all input frames can be obtained.

[0045] The edge density calculation strategy in this embodiment significantly maintains high fidelity while improving computational efficiency in real-time rendering scenarios with high requirements for temporal coherence.

[0046] S103: Dynamically divide pixels into calculation groups with different kernel sizes based on edge density values.

[0047] In this embodiment, the filter kernel size is dynamically determined based on the edge density value. Small kernels are used to preserve details in high-complexity regions, while large kernels are used to improve noise suppression efficiency in low-complexity regions.

[0048] Specifically, an edge density-based grouping strategy is adopted, which determines the grouping based on the edge density value. All pixels in all input frame images are dynamically divided into computational groups with different kernel sizes. This adaptive allocation ensures that structurally complex regions can be processed finely, while enabling efficient large-scale filtering of regions with similar geometric continuity and surface orientation.

[0049] In this embodiment, the kernel size of each group The following is confirmed: ; This embodiment optimizes the threshold (0.5, 0.3) on the dataset through grid search to maximize the harmonic mean of denoising quality (PSNR) and computational efficiency (FPS).

[0050] By edge density The pixels are divided into three different processing groups (with 9×9, 17×17 or 33×33 kernels) and then processed independently through subsequent feature clustering and weight prediction stages, thereby establishing a tightly integrated edge-aware filtering pipeline.

[0051] S104: Within each kernel size calculation group, pixel clustering is performed using local depth gradient information, further dividing each kernel group into several sub-clusters with similar geometric continuity and surface orientation.

[0052] In this embodiment, pixel clustering is performed within each kernel group using local depth gradient information. This divides the pixels into multiple clusters, assuming that the filter weights within each cluster have similar characteristics. Then, the weight distribution of pixels in each cluster can be efficiently retrieved using a compact lookup table containing only a small number of entries.

[0053] Specifically, for each pixel within each kernel group, a corresponding feature vector is constructed. ,in This represents the depth gradient magnitude, used to quantify the intensity of surface discontinuities. This indicates the normalized gradient direction, used to capture the consistency of the encoding direction.

[0054] Then, K-means clustering is used to partition each kernel group based on minimizing the within-cluster variance. Sub-clustering.

[0055] Each generated cluster guides the dynamic weight allocation in subsequent prediction networks. This clustering method enables more accurate weight allocation in prediction networks, concentrating computational resources on perceiving key regions within each large kernel group; through dynamic weight allocation, filters are contracted in geometrically complex regions and expanded in flatter regions; this local kernel adjustment reduces unnecessary computation in uniform regions while preserving structural details at the edges.

[0056] S105: For each sub-cluster, obtain the brightness (derived from albedo), surface normal, and depth features of each pixel's G-buffer, and use a lightweight multilayer perceptron network to obtain the filter weights corresponding to each sub-cluster; generate a global weight lookup table.

[0057] In this embodiment, the pixels of all input frames are divided into different sub-clusters, and each sub-cluster corresponds to a trained lightweight multilayer perceptron network. For each sub-cluster, the brightness, surface normal, and depth features of each pixel in the sub-cluster are obtained and input into the corresponding trained lightweight multilayer perceptron network to obtain the filter weights corresponding to the pixels of that sub-cluster. In this way, the filter weights corresponding to each sub-cluster can be obtained, thereby constructing a global weight lookup table. The predicted filter weights are stored in a compact lookup table, thereby enabling efficient inference through table indexing during the rendering process.

[0058] The global weight lookup table stores multiple arrays consisting of sub-cluster classes, the central features of the corresponding cluster centers, and the corresponding filter weights; through the global weight lookup table, the weights of each sub-cluster can be assigned. The search process involves performing spatial filtering based on the assigned kernel size and weights to output a denoised image.

[0059] In this embodiment, a lightweight multilayer perceptron (MLP) network is used for real-time denoising on mobile ray tracing hardware. Compared with the SVGF method, which relies on computationally intensive variance and weight estimation, the lightweight multilayer perceptron (MLP) network in this embodiment directly utilizes G-buffer features and predicts denoising parameters by learning correlations, thereby eliminating complex variance and weight calculations and improving denoising efficiency.

[0060] Specifically, the lightweight multilayer perceptron (MLP) network in this embodiment includes one input layer, one output layer, and three hidden layers; all layers use ReLU as the activation function, and the network directly accepts a 5-channel input tensor from the G-buffer, which are as follows: (1) Brightness: through A single channel derived from RGB is used to capture luminous intensity, which is crucial for noise perception.

[0061] (2) Surface normal: Three channels (XYZ components) in tangent space, used to encode local geometric directions.

[0062] (3) Depth: Represents a single channel of the Z value in the view space, which has been normalized to [0,1] for numerical stability.

[0063] This embodiment uses a lightweight multilayer perceptron (MLP) network with a three-hidden-layer structure, balancing expressive power and latency, specifically: First hidden layer (128 neurons, ReLU): via Extracting high dimensions Feature intersection terms. Capturing brightness changes ( ), surface normal discontinuity ( ) and depth changes ( Nonlinear coupling between )

[0064] Second hidden layer (64 neurons, ReLU): via Feature compression. Features are compressed by 50% to filter out noise-sensitive components, while salient patterns are preserved through hierarchical abstraction.

[0065] Third hidden layer (32 neurons, ReLU): using Project the features into the latent weight space.

[0066] The output layer generates the original filter weights. ,in, This corresponds to the dynamically selected kernel size.

[0067] Loss function: The network training uses mean squared error (MSE) to approximate the reference weights of SVGF. ; in, This represents the normalized weights of the network prediction. This represents the reference weights generated by the traditional SVGF algorithm. Compared to L1 loss, MSE loss amplifies the contribution of larger error terms through its squared term, forcing the model to prioritize correcting regions where the weight distribution is significantly biased. This mechanism can more effectively guide the model to approximate the global distribution of the target weights.

[0068] It should be noted that the training of each lightweight multilayer perceptron (MLP) network also involves first extracting edge density from the dataset images, dividing them into computation groups with different kernel sizes, clustering each computation group into multiple sub-clusters, and then training an MLP for each sub-cluster separately; this is something that those skilled in the art can do, and the specific process will not be described in detail.

[0069] S106: Quantize the continuous feature space of the frame to be tested into discrete indices. For each discrete index, determine which cluster center it is closest to, and then determine which cluster class the discrete index belongs to. Then, use a global weight lookup table to perform weight retrieval. Finally, perform spatial filtering according to the allocated kernel size and weights to output a denoised image.

[0070] In this embodiment, to completely eliminate real-time inference overhead, a hardware-friendly LUT architecture is proposed for instantaneous weight prediction on mobile GPUs; the core innovation lies in using a continuous 5D feature space (surface normals) ,depth and brightness ) is quantized into a discrete index, which allows weights to be retrieved directly from a pre-computed table.

[0071] Specifically, the method for quantizing surface normals into discrete indices is to convert the normals into spherical coordinates. ,in and Then uniformly quantized into Each box: ; in, and They represent polar angles ( ) and azimuth ( Quantitative indicators of ) This indicates the number of normal quantization intervals, ensuring isotropic sampling on a unit sphere.

[0072] The method for quantizing depth into discrete indices is: depth value After logarithmic scaling and quantization, Each interval is prioritized for near-field accuracy. ; in, and Define the depth range of the scene. Indicates a deep quantization index. This indicates the number of depth quantization intervals. This aligns with human depth perception and maintains the accuracy of occlusion boundaries.

[0073] The method for quantizing brightness into a discrete index is as follows: Brightness Linear quantization is performed after hue mapping to [0,1]. ; in, Indicates the brightness quantization index. This indicates the number of brightness quantization intervals. The choice should aim to differentiate key brightness ranges (such as shadows and midtones) while remaining consistent with the color gamut constraints of mobile displays.

[0074] The above quantization range , and The specific number of bins was obtained through experimental verification. Different bin sizes will achieve different rendering effects. Fine-grained bins have high precision and good image quality but require more storage space, while coarse-grained bins have the opposite effect. In this embodiment, the number of bins for all quantization bins is set to 16 by default, which can achieve the best balance between noise reduction quality and efficiency.

[0075] Index metrics Mapped to a pre-computed five-dimensional tensor (i.e., a global weight lookup table), this tensor stores all kernel sizes. The filter weights, where This indicates the results of grouping and clustering.

[0076] Weights are inferred from all combinations of quantized features during offline MLP generation: ; in, Representing an interval The centroid eigenvector.

[0077] The complete model maintains a pre-computed tensor. The total memory consumption was 2.3GB. During runtime, the weights were... Retrieval within a short time frame eliminates the need for real-time computation.

[0078] This embodiment is applicable to real-time ray tracing in mobile games and AR / VR applications, facilitating the transition of next-generation mobile graphics from rasterization to physically based rendering. The method in this embodiment is compared and analyzed with the classic SVGF and BMFR algorithms on five benchmark scenes (Tavern 1-3, Day Zero scene, and Kitchen scene), such as... Figure 3 As shown. Figure 3 Selected frames from these animations are shown for detailed comparison, with insets highlighting key differences in the results. The results demonstrate that the method described in this embodiment achieves sustained improvements in both quantitative metrics and visual quality.

[0079] From a quantitative perspective, the method in this embodiment achieves lower RMSE values ​​in all test cases, even compared to the BMFR algorithm, which outperforms SVGF. From a visual perspective, this method effectively eliminates Monte Carlo noise without introducing the global blurring characteristic of SVGF. The method exhibits unique advantages in complex lighting environments, such as excellent handling of mixed lighting and multiple reflections in day-zero scenes, and robust noise reduction for various surface materials in kitchen scenes.

[0080] On the Qualcomm Snapdragon 768G platform, this embodiment's method, compared to SVGF, achieves an average frame rate improvement of 67.42% and a power consumption reduction of 16.80%, while maintaining excellent visual quality. Furthermore, compared to the more real-time BMFR algorithm, the method still achieves an average frame rate improvement of 10.86% and a power consumption reduction of 5.45%.

[0081] Compared to traditional mobile noise reduction techniques, this invention significantly improves efficiency while maintaining good visual quality through dynamic adaptive filtering and data-driven weight prediction. In numerous complex scenes, its RMSE is significantly better than benchmark methods, effectively suppressing global blur and halo artifacts, providing a high-efficiency real-time ray tracing solution for mobile games and AR / VR.

[0082] Example 2 In one or more embodiments, a mobile real-time ray tracing noise reduction system based on edge awareness is disclosed, characterized in that it includes: The edge extraction module is configured to perform multi-channel edge detection on the input frame G-buffer information to extract image edges; The edge density calculation module is configured to use motion vector reprojection of the edge density values ​​of historical frames, fuse the edge density values ​​of the currently extracted frame through a hole-filling strategy, and generate a complete edge density map of the currently extracted frame to quantify the local edge concentration. The pixel partitioning module is configured to dynamically partition pixels into computational groups with different kernel sizes based on edge density values; within each kernel size computational group, pixel clustering is performed using local depth gradient information, and each kernel group is further divided into several sub-clusters with similar surfaces; The weight calculation module is configured to, for each sub-cluster, obtain the brightness (derived from albedo), surface normal, and depth features of each pixel in the G-buffer, use a lightweight multilayer perceptron network to obtain the filter weights corresponding to each sub-cluster, and generate a global weight lookup table. The image denoising module is configured to quantize the continuous feature space of the frame under test into a discrete index, use a global weight lookup table to perform weight retrieval, and finally perform spatial filtering according to the assigned kernel size and weights to output a denoised image.

[0083] The specific implementation methods of the above modules are exactly the same as those in Example 1, and will not be described in detail again.

[0084] Example 3 In one or more embodiments, a terminal device is disclosed, comprising a processor and a memory, wherein the processor is used to implement instructions; and the memory is used to store multiple instructions adapted to be loaded by the processor and executed by the processor to perform the edge-aware mobile real-time ray tracing noise reduction method described in Embodiment 1.

[0085] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0086] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0087] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0088] Example 4 In one or more embodiments, a computer-readable storage medium is disclosed, wherein a plurality of instructions are stored, the instructions being adapted to be loaded by a processor of a terminal device and executed by the edge-aware mobile real-time ray tracing noise reduction method described in Embodiment 1.

[0089] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A real-time ray tracing noise reduction method for mobile devices based on edge awareness, characterized in that, include: Multi-channel edge detection is performed on the input frame G-buffer information to extract image edges; The edge density values ​​of historical frames are reprojected using motion vectors, and the edge density values ​​of the currently extracted frame are fused using a hole-filling strategy to generate a complete edge density map of the currently extracted frame, thereby quantifying the local edge concentration. Pixels are dynamically divided into computation groups with different kernel sizes based on edge density values; Within each kernel size calculation group, pixel clustering is performed using local depth gradient information, and each kernel group is further divided into several sub-clusters with similar geometric continuity and surface orientation. For each sub-cluster, the brightness, surface normal, and depth information of each pixel in the G-buffer are obtained. A lightweight multilayer perceptron network is used to obtain the filter weights corresponding to each sub-cluster and generate a global weight lookup table. The continuous feature space of the frame to be tested is quantized into a discrete index, and a global weight lookup table is used to perform weight retrieval. Finally, spatial filtering is performed according to the allocated kernel size and weight to output a denoised image.

2. The edge-aware real-time ray tracing noise reduction method for mobile devices as described in claim 1, characterized in that, The G-buffer information includes three channels: depth, normal, and albedo. The directional gradient on each channel is calculated, normalized, and then weighted and fused. The fused gradient value is compared with an adaptive threshold to obtain a binary mask of the image edge, and thus the image edge region is obtained.

3. The edge-aware real-time ray tracing noise reduction method for mobile devices as described in claim 2, characterized in that, The adaptive threshold is specifically: ； in, Represents total pixels. It is a set constant value used to balance edges and noise; For adaptive threshold, Represents pixels The gradient values ​​after fusion; This represents the mean of the gradient magnitude, reflecting the overall gradient strength. The standard deviation of the gradient magnitude measures the degree of dispersion of the gradient distribution.

4. The edge-aware real-time ray tracing noise reduction method for mobile devices as described in claim 1, characterized in that, For each pixel its edge density Defined as local neighborhood Ratio of inner edge pixels: ； in, It is based on pixels Centered window, Indicate its area; Represents pixels Edge binarization mask.

5. A real-time ray tracing noise reduction method for mobile devices based on edge awareness as described in claim 1 or 4, characterized in that, For the current extracted frame t Pixels Its frame The corresponding position in is ; If pixels If the edge density value is within the valid range of the previous frame, then the edge density value of the previous frame is directly reused. As the current extracted frame t Pixels Edge density values; If pixels If the location is not within the valid range of the previous frame, it is marked as a hole, and the edge density value at that location is recalculated. The current extracted frame t The edge density values ​​of all pixels in the image are fused to generate the current extracted frame. t The complete edge density map.

6. The edge-aware real-time ray tracing noise reduction method for mobile devices as described in claim 1, characterized in that, Within each kernel size calculation group, pixel clustering is performed using local depth gradient information. Each kernel group is further divided into several sub-clusters with similar surfaces, making them exhibit similar geometric continuity and surface orientation. Specifically: For each pixel within a kernel-sized computational group, a feature vector is constructed. ,in Indicates the depth gradient magnitude. Indicates the direction of the normalized gradient; K-means clustering is used to divide pixels within each kernel group into clusters based on minimizing the within-cluster variance. Sub-clustering.

7. The edge-aware real-time ray tracing noise reduction method for mobile devices as described in claim 1, characterized in that, The continuous feature space of the frame under test is quantized into a discrete index, specifically: The continuous feature space includes the brightness (derived from albedo), surface normals, and depth features in the G-buffer; Specifically, the brightness L is quantized into a discrete index, as follows: , Indicates the brightness quantization index. Indicates the number of brightness quantization intervals; Quantizing surface normals into discrete indices specifically involves converting the normals into spherical coordinates. ,in, Polar angle, The azimuth angle is then uniformly quantized to... Each box: ； in, and They represent polar angles respectively. and azimuth Quantitative indicators This indicates the number of normal quantization intervals, ensuring isotropic sampling on a unit sphere. Depth is quantized into discrete indices, specifically: depth values. After logarithmic scaling and quantization, Each interval: ; in and To define the depth range of the scene, Indicates a deep quantization index. Indicates the number of deep quantization intervals.

8. A mobile real-time ray tracing noise reduction system based on edge awareness, characterized in that, include: The edge extraction module is configured to perform multi-channel edge detection on the input frame G-buffer information to extract image edges; The edge density calculation module is configured to use motion vector reprojection of the edge density values ​​of historical frames, fuse the edge density values ​​of the currently extracted frame through a hole-filling strategy, and generate a complete edge density map of the currently extracted frame to quantify the local edge concentration. The pixel partitioning module is configured to dynamically partition pixels into computational groups with different kernel sizes based on edge density values; within each kernel-size computational group, pixel clustering is performed using local depth gradient information, and each kernel group is further divided into several sub-clusters with similar geometric continuity and surface orientation; The weight calculation module is configured to obtain the brightness (derived from albedo), surface normal, and depth features of the central pixel G-buffer for each sub-cluster, and use a lightweight multilayer perceptron network to obtain the filter weights corresponding to each sub-cluster. Generate a global weight lookup table; The image denoising module is configured to quantize the continuous feature space of the frame under test into a discrete index, use a global weight lookup table to perform weight retrieval, and finally perform spatial filtering according to the assigned kernel size and weights to output a denoised image.

9. A terminal device comprising a processor and a memory, the processor for implementing instructions; the memory for storing multiple instructions, characterized in that, The instructions are adapted to be loaded by a processor and executed as described in any one of claims 1-7: the edge-aware real-time ray tracing noise reduction method for mobile devices.

10. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are adapted to be loaded by the processor of the terminal device and executed as described in any one of claims 1-7, the edge-aware mobile real-time ray tracing noise reduction method.

Images