Dynamic lod and ray tracing adaptive energy-saving rendering method and device based on dynamic lod and ray tracing adaptive energy-saving rendering method and device

Abstract

This invention discloses an adaptive energy-saving rendering method, device, and medium based on dynamic LOD and ray tracing, belonging to the field of 3D real-time rendering technology. It constructs a unified visual perception loss function constrained by the maximum visual perception loss imperceptible to the human eye, and establishes a linked coupling mapping model for coverage distance, visual perception threshold, LOD level, and ray tracing sampling rate to determine the rendering parameters of the 3D model. A pre-trained dual-branch lightweight LSTM model collaboratively completes user perspective prediction and rendering resource loading priority output, achieving precise resource preloading. This invention significantly reduces 3D rendering power consumption without sacrificing core visual experience, and significantly improves the adaptability and operational stability of ray tracing rendering on mobile terminals.

Description

Technical Field

[0001] This invention discloses an adaptive energy-saving rendering method, device, and medium based on dynamic LOD and ray tracing, which relates to the field of 3D real-time rendering technology. Background Technology

[0002] With the rapid development of 3D real-time rendering technology, ray tracing technology, with its high-fidelity lighting and shadow rendering effects, has been widely used in scenarios such as mobile terminal XR interaction, 3D digital twins, and open-world games. However, the high computing power and high power consumption of ray tracing rendering create a significant contradiction with the limited computing power and battery life of mobile and embedded terminals, becoming a core bottleneck restricting the large-scale deployment of ray tracing technology on mobile devices.

[0003] In existing 3D rendering engines, Level of Detail (LOD) adjustment and ray tracing sampling rate control are mostly independent functional modules, unable to achieve deep integration and adaptation. They often resort to fixed reductions in rendering precision to optimize power consumption, easily resulting in a significant loss of core visual experience; or maintaining high-quality rendering leads to excessive power consumption, making stable operation on low-to-mid-range mobile devices impossible. Meanwhile, existing rendering resource preloading solutions mostly rely on fixed scene rules or general viewpoint prediction models, lacking scene adaptation flexibility and having low preloading hit rates, easily causing ineffective bandwidth and memory resource usage, further exacerbating the power consumption pressure on the device. Currently, there is no effective solution that can simultaneously meet the adaptation requirements of high visual fidelity and low rendering power consumption. Summary of the Invention

[0004] The main objective of this invention is to provide an adaptive energy-saving rendering method based on dynamic LOD and ray tracing, which significantly reduces the power consumption of 3D rendering without sacrificing the core visual experience, and significantly improves the adaptability and operational stability of ray tracing rendering on mobile terminals.

[0005] To achieve the above objectives, this invention provides an adaptive energy-saving rendering method based on dynamic LOD and ray tracing, comprising the following steps: During the real-time rendering of a 3D scene, the rendering parameters and view prediction feature data of the current frame are collected frame by frame. Using the relative distances of the various object models in the 3D scene as input, a pre-built linkage coupling mapping model is called, and the relative distances of the various object models are input into the linkage coupling mapping model. The model processing method of the linkage coupling mapping model is as follows: check the first LOD level corresponding to each object model and the target ray tracing sampling rate, and render each object model in conjunction with the current frame rendering parameters, the target LOD level and the target ray tracing sampling rate, and suppress the total visual perception loss value after rendering to be less than or equal to the constraint threshold. The view prediction feature data is input into a pre-trained dual-branch lightweight LSTM model. The model processing method of the dual-branch lightweight LSTM model is as follows: output the user's view vector in the three-dimensional scene through the view prediction feature data, simultaneously output the second LOD level and ray tracing rendering data within the predicted field of view corresponding to the view vector, load the priority of the LOD level and ray tracing rendering data, and preload the rendering resources of the corresponding three-dimensional model into the terminal memory according to the priority. Based on the target LOD level and target ray tracing sampling rate of the verified 3D model, the corresponding rendering resources are called to perform ray tracing rendering on the 3D scene and output the current rendering frame.

[0006] Furthermore, the step of collecting two types of core data frame by frame during the real-time rendering process of a 3D scene—the current frame rendering parameters and the viewpoint prediction feature data—includes: Data acquisition is performed during the real-time rendering of the 3D scene, and data is acquired from the current frame rendering parameter acquisition channel and the view prediction feature data acquisition channel after initialization. The core parameters required to adapt to the linkage coupling mapping model are collected through the current frame rendering parameter acquisition channel. After the acquisition is completed, the parameters are cached in the current frame rendering parameter dedicated cache area. The multi-dimensional feature data required to adapt the dual-branch lightweight LSTM model is collected through the perspective prediction feature data acquisition channel. After the acquisition is completed, the time series data is timestamped and cached in the dedicated perspective prediction cache area.

[0007] Furthermore, the step of verifying the first LOD level and target ray tracing sampling rate corresponding to each object model, and rendering each object model in conjunction with the current frame rendering parameters, target LOD level, and target ray tracing sampling rate, includes: Read the spatial position, orientation, and view frustum range parameters of the user's viewpoint in the current frame from the dedicated buffer of the current frame rendering parameters, and perform visibility pre-filtering on all object models in the 3D scene; The relative distance between the current frame rendering parameter and the user's view camera is read from the dedicated buffer. The linkage coupling mapping model is called to calibrate the preliminary parameter combination of the target LOD level and the target ray tracing sampling rate of the only visible model among all object models. By combining the real-time computing power load and battery power status parameters of the terminal device read from the dedicated cache of the current frame rendering parameters, dynamic fine-tuning is performed on the initial parameter combination to obtain the implementation parameter combination; The unified visual perception loss function is invoked to perform a fast secondary verification on the implementation parameter combination to verify that the total visual perception loss after rendering the only visible model does not exceed the global constraint threshold. Each of the aforementioned visible-only models is rendered according to the combination of implementation parameters.

[0008] Furthermore, the step of dynamically fine-tuning the initial parameter combination by combining the terminal device's real-time computing load and battery power status parameters read from the dedicated buffer of the current frame rendering parameters to obtain the implementation parameter combination includes: A hard constraint is set that the total visual perception loss after rendering does not exceed the global constraint threshold, thus defining an unadjustable boundary. When the real-time computing load of the terminal device exceeds the preset load threshold, or the battery power is lower than the preset low power threshold, the low power adaptation mode is triggered, and the LOD level and ray tracing sampling rate are synchronously reduced for visible models outside the visual focus area. During the synchronous downscaling process, the LOD level of a single model is reduced by no more than 1 level in a single instance, and the reduction in the ray tracing sampling rate does not exceed 50% of the current value; After adjusting and integrating the parameter combinations of all models, the final implementation parameter combination is obtained.

[0009] Furthermore, the step of outputting the user's viewpoint vector in the 3D scene through the viewpoint prediction feature data, and simultaneously outputting the second LOD level and ray tracing rendering data corresponding to the viewpoint vector within the predicted field of view, includes: The view prediction feature data, which has been validated and aligned with timestamps, is read from the view prediction dedicated cache, and after standardization preprocessing, it is input into the pre-trained dual-branch lightweight LSTM model. The model predicts the user's viewpoint switching direction within a preset time period by outputting the viewpoint prediction branch, generating the corresponding predicted view frustum range, and locking the preloaded effective field of view boundary. Using the effective field of view boundary as a spatial constraint, the resource priority prediction branch of the model is combined with the parameter matching rules of the linkage coupling mapping model to synchronously output the LOD and ray tracing rendering resource loading priority of each 3D model within the range; A threshold check is performed on the loading priority, and non-core models below the preset minimum threshold are eliminated. The corresponding rendering resources are preloaded into the frame rendering dedicated cache area in the terminal memory according to the checked priority.

[0010] Furthermore, the step of synchronously outputting the LOD and ray tracing rendering resource loading priority of each 3D model within the range, using the effective view boundary as a spatial constraint, through the resource priority prediction branch of the model, and combined with the parameter matching rules of the linkage coupling mapping model, includes: Filter the 3D models within the effective field of view boundary and remove irrelevant models that exceed the spatial constraints; By combining the relative distance between each 3D model and the predicted viewpoint, the corresponding LOD level and ray tracing sampling rate are matched through the linkage coupling mapping model. Then, the real-time computing power load of the terminal device, the basic state parameters of the battery power, and the visual focus area features of the screen are integrated to sort the resource priority of the 3D models after filtering. The loading weights are divided into multiple levels according to the sorting results, and the rendering resource loading standards corresponding to the core visual area, secondary auxiliary area, and distant weakened area are distinguished.

[0011] Furthermore, the step of calling the corresponding rendering resources, performing ray tracing rendering on the 3D scene, and outputting the current rendering frame based on the target LOD level and target ray tracing sampling rate of the verified 3D model includes: Read the target LOD level, target ray tracing sampling rate, and user view frustum parameters of each visible 3D model after secondary verification from the dedicated buffer of rendering parameters for the current frame; Based on the read rendering parameters, the corresponding 3D model rendering resources that have been preloaded in the dedicated frame rendering cache are called first, and the resources that have not been preloaded are retrieved synchronously from the terminal's local storage. Load the 3D model geometry based on the target LOD level, perform ray tracing rendering calculations at the matching target ray tracing sampling rate, and generate the original rendered image of the current frame. Perform image quality compliance checks on the original rendered image. After verifying that the total visual perception loss of the entire image does not exceed the global constraint threshold, output the final current rendered frame.

[0012] This invention also provides an adaptive energy-saving rendering device based on dynamic LOD and ray tracing, which executes the adaptive energy-saving rendering method based on dynamic LOD and ray tracing described above. The adaptive energy-saving rendering device based on dynamic LOD and ray tracing includes: The model pre-building module is used to build a unified visual perception loss function and linkage coupling mapping model, as well as a pre-trained dual-branch lightweight LSTM model; The data acquisition module is used to collect the rendering parameters and view prediction feature data of the current frame in parallel frame by frame during real-time rendering, and to complete data verification and caching. The rendering parameter linkage determination module is used to call the linkage coupling mapping model to determine and verify the target LOD level and target ray tracing sampling rate of the 3D model. The viewpoint prediction and resource preloading module is used to output the viewpoint switching direction and rendering resource loading priority through a dual-branch lightweight LSTM model to complete the corresponding resource preloading. The rendering execution and output module is used to call the corresponding resources based on the determined rendering parameters, perform ray tracing rendering, and output the rendering frame.

[0013] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the adaptive energy-saving rendering method based on dynamic LOD and ray tracing described above.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described adaptive energy-saving rendering method based on dynamic LOD and ray tracing. Attached Figure Description

[0015] Figure 1 This is a flowchart of an adaptive energy-saving rendering method based on dynamic LOD and ray tracing for file upload in one embodiment of the present invention; Figure 2 This is a structural block diagram of an adaptive energy-saving rendering device based on dynamic LOD and ray tracing for file uploading in one embodiment of the present invention; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] Reference Figure 1 This is a flowchart illustrating an adaptive energy-saving rendering method based on dynamic LOD and ray tracing proposed in this invention, including the following steps: S1, during the real-time rendering of the 3D scene, collects the rendering parameters and view prediction feature data of the current frame frame by frame; S2, taking the relative distance of each object model in the three-dimensional scene as input, calling the pre-built linkage coupling mapping model, and inputting the relative distance of each object model into the linkage coupling mapping model, the model processing method of the linkage coupling mapping model is: checking the first LOD level corresponding to each object model and the target ray tracing sampling rate, rendering each object model in conjunction with the current frame rendering parameters, target LOD level and target ray tracing sampling rate, and suppressing the total visual perception loss value after rendering to be less than or equal to the constraint threshold; S3, input the view prediction feature data into the pre-trained dual-branch lightweight LSTM model. The model processing method of the dual-branch lightweight LSTM model is as follows: output the user's view vector in the three-dimensional scene through the view prediction feature data, simultaneously output the second LOD level and ray tracing rendering data within the predicted field of view corresponding to the view vector, load the priority of the LOD level and ray tracing rendering data, and preload the rendering resources of the corresponding three-dimensional model into the terminal memory according to the priority. S4. Based on the target LOD level and target ray tracing sampling rate of the verified 3D model, call the corresponding rendering resources, perform ray tracing rendering on the 3D scene, and output the current rendering frame.

[0018] A unified visual perception loss function was constructed, and the maximum visual perception loss imperceptible to the human eye was set as the global constraint threshold. Based on the human eye contrast sensitivity function and combined with the visual characteristics of the human eye in real-time 3D rendering, the thresholds at which the human eye can perceive changes in geometric details and lighting details of 3D models at different viewing distances were calibrated. Combining objective image quality indicators such as structural similarity and peak signal-to-noise ratio with the results of subjective evaluation experiments by the human eye, the final calibration of the constraint threshold was completed.

[0019] Separate quantization terms for geometric detail loss corresponding to the LOD level and illumination detail loss corresponding to the ray tracing sampling rate were constructed. Dynamic correlation weights were assigned to these two types of quantization terms based on human eye sensitivity at different distances, and a weighted sum was formed to create a unified visual perception loss function. Based on this function, a linked coupling mapping model was built to establish the correspondence between the relative distance between the user's viewpoint and the 3D model, and the combination of LOD level and ray tracing sampling rate parameters. With visual perception loss not exceeding a constraint threshold as a hard constraint and minimizing rendering power consumption as the optimization objective, the mapping model was constructed and validated.

[0020] A lightweight LSTM model with two branches for simultaneous pre-training of LOD and ray tracing-linked rendering is constructed. A collaborative network structure including a view prediction branch and a resource priority prediction branch is built. View trajectory, scene element distribution, and device performance data in multiple scenes are collected to build a training set. The training, quantization, and optimization of the model are completed to enable the model to meet the low-latency inference requirements of real-time rendering.

[0021] As described in step S1 above, after the 3D scene enters real-time rendering, a data acquisition action is triggered during the vertical synchronization interval of each rendering frame. Two parallel acquisition channels are initialized to avoid the acquisition operation occupying the main rendering thread resources and affecting the smoothness of the image. One channel is responsible for acquiring the rendering parameters of the current frame, specifically including the spatial position, orientation, and frustum range of the user's viewpoint in the current frame, the relative distance between each 3D model within the frustum range and the user's viewpoint camera, as well as basic status parameters such as the real-time computing load and battery level of the terminal device. After acquisition, all parameters are written to the dedicated buffer for rendering parameters of the current frame for direct use in the rendering parameter calculation process. The other channel is responsible for acquiring viewpoint prediction feature data, specifically including the temporal trajectory data such as the angular velocity, linear velocity, and dwell time of the user's viewpoint movement within a preset historical time period, the spatial coordinate data of core elements in the 3D scene, the user's visual focus area data in the current frame, and the real-time performance status data of the terminal device. After acquisition, all temporal data undergoes timestamp alignment processing to ensure data temporal consistency and is written to the dedicated buffer for viewpoint prediction. After completing the acquisition of two data streams, the validity of all acquired data is verified, and abnormal data and duplicate redundant data that exceed the preset reasonable value range are removed. If core parameters are found to be missing during the verification process, the valid verification data of the previous frame is directly reused to complete the missing data, ensuring that the input data of the rendering and prediction stages are complete and continuous.

[0022] As described in step S2 above, the spatial position, orientation, and frustum range parameters of the user's viewpoint in the current frame are read from the dedicated buffer for rendering parameters of the current frame. A frustum visibility pre-screening is performed on all 3D models in the 3D scene, eliminating invisible models completely outside the frustum range and retaining only visible models within the frustum range for processing, reducing computational and power consumption losses caused by invalid calculations. For each retained visible model, the relative distance between the model and the user's viewpoint camera is read from the dedicated buffer for rendering parameters of the current frame. This relative distance is used as input to call a pre-built linkage coupling mapping model, initially determining the preliminary parameter combination of the target LOD level and target ray tracing sampling rate corresponding to the visible model. Combined with the real-time computing load and battery power status parameters of the terminal device read from the dedicated buffer, the preliminary parameter combination is dynamically fine-tuned. Without exceeding the global visual perception loss constraint threshold, the rendering parameters of visible models in non-visual focus areas are progressively reduced to further optimize rendering power consumption. After completing the parameter fine-tuning, the pre-built unified visual perception loss function is called to perform a fast secondary verification on the fine-tuned parameter combination, calculate the total visual perception loss after model rendering under the parameter combination, ensure that the loss value does not exceed the preset global constraint threshold, and write the finally verified target LOD level and target ray tracing sampling rate into the dedicated buffer area for rendering parameters of the current frame.

[0023] As described in step S3 above, the view prediction feature data, after validity verification and timestamp alignment, is read from the dedicated view prediction cache. Standardized preprocessing is performed on all input data, including normalization of temporal trajectory data, camera coordinate system transformation of the spatial coordinates of core scene elements, and generation of corresponding masks for the visual focus area. This ensures the preprocessed data fully matches the input requirements of the dual-branch lightweight LSTM model. The preprocessed feature data is then synchronously input into the pre-trained dual-branch lightweight LSTM model. The model's view prediction branch extracts temporal features of view movement and spatial distribution features of core scene elements, outputting the user's view switching direction within a preset timeframe. Simultaneously, the predicted view frustum spatial range corresponding to this view switching direction is generated, locking the effective field of view boundary for the preloading action. Using the locked predicted field of view range as a spatial constraint, all 3D models within this range are selected as preloading candidates. Through the model's resource priority prediction branch, combined with visual focus area data, real-time performance status data of the terminal device, and parameter matching rules of the linkage coupling mapping model, all preloading candidates are prioritized, and the loading priority of the corresponding rendering resources is output synchronously. The loading priority of the output is checked by a threshold. Non-core models with a priority lower than the preset minimum threshold are removed to avoid invalid memory occupation and bandwidth loss. Preloading tasks are generated according to the priority after verification and sent to the terminal resource scheduling module to preload the corresponding rendering resources into the frame rendering dedicated buffer area in the terminal memory.

[0024] As described in step S4 above, the target LOD level, target ray tracing sampling rate, and view frustum parameters of the user's perspective for each visible 3D model, after secondary verification, are read from the dedicated rendering parameter cache for the current frame, completing the parameter preparation work before rendering. Based on the read rendering parameters, the pre-loaded corresponding 3D model rendering resources in the dedicated rendering cache are prioritized. For rendering resources that have not been pre-loaded, they are synchronously retrieved from the terminal's local storage to ensure smooth operation of the rendering pipeline and avoid screen stuttering caused by resource retrieval delays. The corresponding model geometry is loaded based on the target LOD level of each 3D model, and ray tracing rendering calculations are performed at the matched target ray tracing sampling rate to generate and composite the lighting and shadow effects within the scene, obtaining the original rendered image of the current frame. Image quality compliance verification is performed on the generated original rendered image, calculating the total visual perception loss of the entire image and confirming that the total visual perception loss does not exceed the preset global constraint threshold, ensuring that the visual experience of the rendered image meets the preset requirements. After completing the image quality verification, the final rendered image is output to the display terminal, completing the entire process of the current rendering frame and entering the loop process of the next rendering frame.

[0025] In the specific implementation process, the human visual perception threshold calibration and unified visual perception loss function construction stages relied on standardized subjective evaluation experiments to complete the threshold calibration. The experiments were adapted to two mainstream application scenarios: mobile handheld terminals and XR headsets. Two basic test environments were set: a standard handheld viewing distance of 30 cm and a near-eye viewing distance of 15 cm for the headset. The test terminals used mainstream 2K resolution 6.7-inch handheld screens and single-eye 2K resolution XR headset screens. Twenty subjects with normal corrected vision, aged 18 to 45, were recruited. Imperceptible threshold calibration was conducted for two visual dimensions: geometric detail and illumination detail. For geometric detail calibration, high-precision car and architectural models with the same structure were used as test samples. The highest precision level model had 100,000 facets. The number of facets was progressively reduced to generate six levels of detail. Through a forced choice experiment, the critical value of the number of facets at which subjects could not distinguish adjacent levels of the model at different viewing distances was calibrated, obtaining the imperceptible threshold for geometric detail changes at the corresponding viewing distance. The illumination detail calibration used the same 3D scene as the test sample. The baseline ray tracing sampling rate was 64 times per pixel. Multiple sets of rendered images were generated by progressively decreasing the sampling rate. Through the same subjective evaluation experiment, the critical sampling rate value at which subjects could not distinguish the differences in the images at different viewing distances was calibrated, thus obtaining the imperceptible threshold of illumination detail changes at the corresponding viewing distances. During the experiment, objective image quality indicators such as structural similarity and peak signal-to-noise ratio were collected simultaneously and matched with the subjective evaluation results. The structural similarity corresponding to the image imperceptible to the human eye was not less than 0.99, and the peak signal-to-noise ratio was not less than 45 dB. Based on this, the maximum visual perception loss imperceptible to the human eye was determined as the global constraint threshold. In the process of constructing the unified visual perception loss function, geometric detail loss quantization terms and illumination detail loss quantization terms were built separately. The geometric detail loss quantization term was based on the ratio of the number of faces of the current LOD level to the highest LOD level, combined with the imperceptible threshold of geometric details at the corresponding viewing distance to complete normalized quantization. The illumination detail loss quantization term was based on the ratio of the current ray tracing sampling rate to the baseline sampling rate, combined with the imperceptible threshold of illumination details at the corresponding viewing distance to complete normalized quantization. Based on the changes in human eye sensitivity at different viewing distances, dynamic correlation weights are assigned to two types of quantization terms. As the viewing distance increases, the human eye’s sensitivity to geometric details decreases synchronously, and the weight ratio of the geometric detail loss quantization term is adjusted downward accordingly. As the viewing distance decreases, the weight ratio of the geometric detail loss quantization term is adjusted upward accordingly. The weighted sum of the two types of quantization terms yields a unified visual perception loss function, and the output value of the function is the total visual perception loss after rendering a single model.

[0026] In the stage of building and optimizing the linkage coupling mapping model, the relative distance between the user's perspective and the 3D model is used as the sole input variable. The hard constraint is that the total visual perception loss calculated by the unified visual perception loss function does not exceed the global constraint threshold. The optimization objective is to minimize the total power consumption of single-model rendering. Before building the model, a full parameter combination sample library is constructed. The sample library covers the normal view distance range of 0.1 meters to 10 meters across the entire scene, the LOD level covers 6 levels of fixed detail, and the ray tracing sampling rate covers the full range from 2 to 64 times per pixel. All possible view distance and parameter combination samples are generated through permutations and combinations. For each set of samples, the corresponding total visual perception loss is calculated using the unified visual perception loss function, and the rendering power consumption corresponding to that set of parameters is calculated simultaneously. The power consumption calculation is based on the measured performance data of mobile GPUs. For every level reduction in LOD level, the model geometry processing power consumption decreases by 15%, and for every halving of the ray tracing sampling rate, the ray tracing calculation power consumption decreases by 40%. Among all parameter combinations at the same view distance, the parameter combination with the lowest total visual perception loss (not exceeding the global constraint threshold) and lowest rendering power consumption is selected as the optimal output parameter for that view distance. A one-to-one correspondence is established between view distance and the optimal LOD level and ray tracing sampling rate combination, completing the initial construction of the linkage coupling mapping model. To address the parameter jump problem at adjacent boundaries within the view distance range, linear smooth transition processing is performed on the parameter combinations in the boundary range to avoid screen flickering caused by drastic jumps in rendering parameters when the view distance changes slightly. After the model is built, it is validated through three typical scenarios: an open-world game city scene, a digital twin park scene, and an XR interactive scene. The validation results show that the parameter combination output by the model can ensure that there is no visually perceptible loss in the rendered image, while reducing the average rendering power consumption across all scenes by more than 35%.

[0027] The dual-branch lightweight LSTM model design and pre-training phase are tailored for LOD and ray tracing-linked rendering scenarios. It includes a collaborative viewpoint prediction branch and a resource priority prediction branch, both sharing a common underlying temporal feature extraction layer, reducing the overall model's parameter and computational load. The viewpoint prediction branch employs a two-layer LSTM structure with a hidden layer dimension of 64. Its core function is to extract temporal features of viewpoint movement and output the future viewpoint switching direction. The resource priority prediction branch also uses a two-layer LSTM structure with a hidden layer dimension of 64. Its core function is to combine the predicted field of view to output the loading priority of rendering resources. The model's input features include multi-dimensional temporal data, specifically viewpoint movement trajectory data from 10 consecutive historical frames, spatial coordinate data of core interactive elements within the scene, position data of the user's visual focus area, and real-time performance status data of the terminal device. This differs from conventional viewpoint prediction models that only input viewpoint trajectory data. Model training was conducted using a self-built multi-scene temporal dataset. This dataset collected viewpoint operation data from 20 subjects across three typical rendering scenarios, totaling 500,000 frames of valid temporal data. During the data collection process, model resource distribution and device performance status data for each scenario were recorded simultaneously. Data augmentation was performed on the dataset, including temporal axis scaling and Gaussian noise addition, to improve the model's generalization ability. Model training employed a joint loss function for optimization, consisting of a weighted average of angle mean squared error and resource loading priority matching loss. Mini-batch gradient descent with a batch size of 32 was used, and the Adam algorithm was employed as the optimizer. The initial learning rate was set to 0.001, and the total training epochs were 50. After training, INT8 quantization and depthwise separable convolution optimization were performed on the model to reduce inference time.

[0028] In one embodiment, step S1, which involves collecting rendering parameters and viewpoint prediction feature data frame by frame during real-time rendering of a 3D scene, includes: S11, Frame period acquisition trigger and parallel acquisition channel initialization; S12, dual-channel core data directional acquisition and preprocessing; S13, Data validity verification and hierarchical caching.

[0029] In the specific implementation, the acquisition action is triggered during the vertical synchronization interval of each rendering frame. At a rendering frame rate of 30 frames per second, the frame period is 33.3 milliseconds. The trigger window is fixed at the end of the last 2 milliseconds of the frame period, avoiding peak periods of geometry processing and lighting calculations in the main rendering thread, thus preventing image stuttering or frame rate fluctuations caused by the acquisition operation. When the acquisition action starts, two parallel independent acquisition channels are initialized simultaneously. These two channels run on the low-power small cores of the terminal CPU and the GPU auxiliary computing unit, respectively, without consuming the computing resources of the main rendering thread. One channel is responsible for acquiring and processing the rendering parameters of the current frame, while the other channel is responsible for acquiring and processing the viewpoint prediction feature data. For near-eye rendering scenarios with XR headsets, the trigger frequency of the acquisition channels is synchronously increased to 90 frames per second, and the underlying acquisition frequency of head posture data is locked at 1000 Hz to ensure the real-time performance and prediction accuracy of the viewpoint data.

[0030] The current frame rendering parameter acquisition channel acquires all the input parameters required by the linked coupling mapping model, adapting to the frame synchronization requirements of real-time rendering. The acquired content includes the spatial position, orientation, and frustum range parameters of the user's viewpoint camera in the current frame. In handheld terminal rendering scenarios, the near clipping plane of the frustum is fixed at 0.1 meters, and the far clipping plane is fixed at 1000 meters. In XR near-eye rendering scenarios, the near clipping plane of the frustum is adjusted to 0.01 meters, and the far clipping plane is adjusted to 500 meters, perfectly matching the visual characteristics of the two scenarios. The relative distance between each 3D model and the optical center of the viewpoint camera within the frustum range is acquired synchronously. Distance calculation is based on the center of the model's axial bounding box, rather than the model's surface vertices, ensuring the stability of the distance values ​​and avoiding parameter jumps caused by changes in model vertices.

[0031] In the digital twin park rendering scenario, when the user switches perspectives to view the power distribution room, cooling tower, and inspection robot models within the park, the acquisition channel updates the relative distances of the static building models frame by frame and refreshes the real-time relative distances of the dynamic inspection robot models every millisecond. Simultaneously, it collects device status parameters such as the terminal's current GPU utilization, CPU large core load, remaining battery power, and real-time discharge power. All collected rendering parameters for the current frame are temporarily written to a temporary buffer in the register, awaiting subsequent verification processing.

[0032] The viewpoint prediction feature data acquisition channel acquires multi-dimensional temporal features required by the dual-branch lightweight LSTM model, perfectly matching the model's input dimension requirements. The acquired content covers viewpoint movement temporal data from 10 consecutive historical frames, corresponding to a rendering frame rate of 30 frames per second with a temporal window length of 333 milliseconds. Specifically, it includes angular velocity, linear velocity, and single-region dwell time of viewpoint rotation. In handheld terminal scenarios, it acquires viewpoint change data from touchscreen swiping and joystick control; in XR scenarios, it acquires head six-degree-of-freedom pose change data. All temporal data are accompanied by microsecond-level timestamps. Simultaneously, it acquires the world space coordinates of core elements within the 3D scene. Core elements include NPCs and quest objective points in game scenes, key monitoring equipment in digital twin scenes, and operable buttons and exhibit models in XR interactive scenes. It also acquires the user's visual focus area data for the current frame. Terminals with eye-tracking capabilities directly acquire a 20-degree field of view around the user's gaze point; terminals without eye-tracking capabilities default to using a 30% field of view area at the center of the screen. The acquisition process synchronously obtains performance status data such as terminal memory usage, system bandwidth margin, and storage read / write speed. After acquisition, timestamp alignment is performed on all time-series data to uniformly align the raw data from different acquisition frequencies to the start time point of each frame, ensuring the continuity and consistency of time-series data.

[0033] After initial processing of the raw data from the two acquisition channels, a unified full validity check and hierarchical caching process are performed. The verification process sets fixed, reasonable value ranges: the valid range for relative distance is 0.01 meters to 1000 meters, and the valid range for viewpoint rotation angular velocity is 0 degrees per second to 360 degrees per second. Values ​​outside these ranges are directly identified as abnormal data. Simultaneously, redundant static data is removed. The coordinates of core scene elements that remain unchanged between frames do not need to be stored repeatedly, reducing computational power and memory usage. If core parameters are found to be missing during verification, the valid verification data from the previous frame is reused to complete the missing parameters. Continuous completion operations are limited to no more than three frames; exceeding this limit triggers a full data re-acquisition to avoid rendering errors or prediction failures caused by invalid data. The verified valid data undergoes hierarchical caching. The current frame's rendering parameters are written to the on-chip L2 cache directly accessible to the GPU, ensuring low-latency calls in the rendering parameter matching process. Viewpoint prediction feature data is written to a dedicated continuous cache area in system memory, adapting to the high-speed reading requirements of the neural network inference unit and providing stable input support for subsequent viewpoint prediction and resource preloading processes.

[0034] In one embodiment, the step S2, which uses the relative distances of the various object models in the 3D scene as input, calls a pre-built linkage coupling mapping model and inputs the relative distances of the various object models into the linkage coupling mapping model, wherein the model processing method of the linkage coupling mapping model is: checking the first LOD level corresponding to each object model and the target ray tracing sampling rate, rendering each object model in conjunction with the current frame rendering parameters, the target LOD level, and the target ray tracing sampling rate, and suppressing the total visual perception loss value after rendering to be less than or equal to the constraint threshold, includes: S21, 3D model visibility pre-screening; S22, Preliminary matching of rendering parameters based on the linkage coupling mapping model; S23, Parameter fine-tuning for device status adaptation and compliance verification of visual loss.

[0035] In the specific implementation process, the view frustum parameters of the user's perspective in the current frame are read from the GPU's on-chip cache. Visibility pre-screening is performed on all 3D models in the scene to remove invalid models that do not need to participate in the rendering of the current frame, reducing the computational power consumption of subsequent steps. The pre-screening first performs coarse view frustum culling. Based on the axial bounding box pre-calculated for each 3D model, an intersection test is performed with the six clipping faces of the view frustum. Models completely outside the view frustum are directly culled and do not enter the subsequent processing flow. For the city scene of the open-world game, the entire scene contains more than 100,000 3D models, covering various elements such as buildings, vegetation, vehicles, and props. When the user's view is facing the commercial street in the city center, the suburban mountains outside the view frustum and the residential area models behind the view are all culled. The number of remaining visible models to be processed is reduced to less than 8,000, and the computational power consumption is reduced by more than 90%. In XR near-eye rendering scenarios, simultaneous culling is performed on the left and right visual cones to ensure consistency of rendered content for both eyes. The axial bounding boxes of static models are pre-generated and stored locally, eliminating the need for recalculation every frame. For dynamically moving models such as vehicles and NPCs, the bounding box data is updated synchronously every frame to ensure the accuracy of the culling results. After pre-screening, only the visible models within the visual cone area are retained for subsequent parameter matching.

[0036] For each retained visible-only model, the relative distance between the model and the optical center of the user's viewpoint camera is read from a dedicated cache. Using the relative distance as the sole input variable, a pre-built linkage coupling mapping model is invoked to complete the initial matching of rendering parameters. The linkage coupling mapping model is pre-stored in the GPU cache in the form of a one-dimensional lookup table, supporting batch queries of the corresponding parameters of all visible-only models in a single frame. The time taken for a single batch query is consistently less than 1 millisecond, fully adapting to the frame cycle requirements of real-time rendering. In the digital twin park scene, when the user's viewpoint is 0.5 meters away from the high-voltage distribution cabinet model, it is considered a close-range detailed observation scene. The mapping model directly outputs the highest level of LOD, Level 0, corresponding to 120,000 model faces and a ray tracing sampling rate of 32 times per pixel. When the user moves back to 5 meters away from the model, the mapping model outputs Level 2 LOD, corresponding to 30,000 model faces and a ray tracing sampling rate of 8 times per pixel. When the relative distance between the user and the model exceeds 20 meters, the mapping model outputs Level 4 LOD, corresponding to 5,000 model faces and a ray tracing sampling rate of 2 times per pixel. The initial LOD level and ray tracing sampling rate combination has met the basic requirement that visual perception loss does not exceed the global constraint threshold, ensuring the compliance of parameter matching. Real-time computing load, remaining battery power, and real-time discharge power parameters of the terminal device are read from the system cache, and the initially matched parameter combination is dynamically fine-tuned to further optimize terminal power consumption and rendering smoothness. When the terminal GPU real-time utilization exceeds 85% or the remaining battery power is below 20%, it automatically enters low-power adaptation mode. Without exceeding the global constraint threshold, it prioritizes the visible models in non-visual focus areas, gradually lowering rendering parameters, with each adjustment not exceeding one LOD level and the ray tracing sampling rate adjustment not exceeding 50%, to avoid drastic parameter changes causing screen flickering.

[0037] In open-world game scenes, the user's visual focus is on the NPC model in front, at which point the terminal's GPU utilization reaches 90%. After entering low-power mode, the LOD level of background vegetation and distant building models outside the focus area is lowered from level 2 to level 3, and the ray tracing sampling rate is reduced from 8 times per pixel to 4 times per pixel. The parameters of the NPC model within the focus area remain unchanged to ensure that the core visual experience is not affected. After the parameter fine-tuning is completed, a pre-built unified visual perception loss function is called to perform batch secondary verification on the parameter combinations of all visible models. The verification process is executed in parallel in the GPU shader unit, and the verification time per frame is short. The total visual perception loss after fine-tuning of each model is calculated to ensure that the value does not exceed the preset global constraint threshold. If the loss value of a model is found to exceed the threshold during the verification process, it is automatically redirected to the previous level parameter combination and the verification is re-executed until compliance is achieved. Finally, the target LOD level and ray tracing sampling rate that pass the verification are written to a dedicated cache that can be directly accessed by the GPU, providing direct parameter basis for subsequent rendering execution.

[0038] In one embodiment, the step S3, which involves inputting the viewpoint prediction feature data into a pre-trained dual-branch lightweight LSTM model, and processing the dual-branch lightweight LSTM model, comprises: outputting the user's viewpoint vector in the 3D scene through the viewpoint prediction feature data; simultaneously outputting the second LOD level and ray tracing rendering data within the predicted field of view corresponding to the viewpoint vector; prioritizing the loading of the LOD level and ray tracing rendering data; and preloading the rendering resources of the corresponding 3D model into the terminal memory according to the priority, includes: S31, Standardization and preprocessing of predictive feature data; S32, Predictive view switching direction and preloaded field of view locking; S33, output of rendering resource loading priority and preloading execution.

[0039] In the specific implementation process, a dedicated cache area is predicted from the perspective of system memory. Multi-dimensional feature data, after validity verification and timestamp alignment, is read and standardized preprocessing is performed to ensure that the input data fully matches the input dimension requirements of the dual-branch lightweight LSTM model. The preprocessing process first performs min-max normalization on the temporal trajectory data, uniformly mapping continuous values ​​such as view rotation angular velocity and linear velocity to a fixed interval of 0 to 1, eliminating the dimensional differences between different dimensions of data. The temporal window is fixed at 10 consecutive frames of historical data, corresponding to a 333-millisecond view operation time sequence at a rendering frame rate of 30 frames per second. In the XR near-eye rendering scenario, the temporal window is synchronously adjusted to 15 consecutive frames to match the high frame rate rendering requirement of 90 frames per second. The coordinate system transformation of the spatial coordinates of the core elements of the scene is completed simultaneously, transforming the world space coordinates of all core elements to the camera coordinate system of the user's view in the current frame, so that the model's spatial position is directly related to the view prediction result without the need for additional coordinate transformation calculations. For the visual focus area data, a normalized mask corresponding to the field of view is generated. For eye-tracking terminals, the mask range is a 20-degree field of view around the user's gaze point; for terminals without eye-tracking, the mask range is the field of view corresponding to 30% of the screen's center area. Values ​​within the mask are marked as 1, and values ​​outside the mask are marked as 0. The preprocessing process simultaneously addresses minor gaps in the temporal data, using linear interpolation to fill in numerical gaps in single frames, ensuring the continuity of the temporal data. All preprocessed data is integrated into a fixed-dimensional feature tensor and written to a high-speed cache directly accessible by the neural network inference unit.

[0040] The preprocessed feature tensors are synchronously input into the pre-trained dual-branch lightweight LSTM model. Both branches share the underlying temporal feature extraction layer, simultaneously completing feature extraction and inference computation. The viewpoint prediction branch extracts temporal features of viewpoint movement through a two-layer LSTM structure, fusing spatial distribution features of core scene elements and visual focus area features to output the user's viewpoint switching direction for the next three frames. This corresponds to a 100-millisecond time window at a rendering frame rate of 30 frames per second, with the window length matching the average storage and retrieval time on mobile devices, ensuring that preloading can be completed before the corresponding frame is rendered. In an open-world game city scene, as the user continuously slides the joystick to the right to rotate the viewpoint, the viewpoint prediction branch can accurately predict the viewpoint orientation for the next three frames based on the sliding trajectory of 10 consecutive frames. The angle error between the prediction result and the user's actual operation is consistently within 8 degrees, completely covering the effective visual field of the frustum. In XR near-eye rendering scenes, the viewpoint prediction branch simultaneously fuses high-frequency acquisition data of the head's six degrees of freedom posture to predict the direction and amplitude of the user's head rotation, generating dual-prediction frustums corresponding to both eyes. Based on the output viewpoint switching direction, the corresponding predicted view frustum space range is generated synchronously. The near clipping plane and far clipping plane parameters of the view frustum are consistent with the view frustum rendered in the current frame, thereby locking the effective view boundary of the preloading action. Models that are completely outside the predicted view frustum range do not enter the preloading candidate list, thus avoiding the loading operation of invalid resources from the source.

[0041] Using a locked, predicted field of view as a spatial constraint, all 3D models within this range are selected as preloading candidates. The resource priority prediction branch combines the visual focus area mask, real-time performance data of the terminal device, and parameter matching rules of the linkage coupling mapping model to prioritize all candidate models and simultaneously output the loading priority of the LOD and ray tracing rendering resources corresponding to each model. The core criteria for priority ranking include the distance between the model and the center of the predicted viewpoint, the visual importance of the model in the scene, and the LOD level of the linkage coupling mapping model at the predicted viewpoint distance. The closer the model is to the center of the viewpoint and the higher its visual importance, the higher its loading priority. The priority results are divided into three levels: high-priority models are core models within the predicted visual focus area, corresponding to the loading of the highest matching LOD level geometry, ray tracing accelerated structures, and high-definition texture resources; medium-priority models are secondary models at the edge of the predicted field of view, corresponding to the loading of basic LOD level geometry and low-precision ray tracing accelerated structures; and low-priority models are distant, non-core models, which are not preloaded. In the digital twin park scenario, the user's current view is facing the main entrance of the park. The predicted view direction is the high-voltage power distribution room area to the left. High-priority loading is performed on the main model of the power distribution room and the internal equipment models, including Level 2 LOD resources and ray-tracing accelerated structures. Medium-priority loading is performed on the pipes and fence models surrounding the power distribution room, including Level 3 LOD resources. Low-priority models of distant mountains and vegetation are not preloaded. A threshold check is performed on the output loading priority, eliminating non-core models with priorities below a preset minimum threshold to avoid unnecessary memory usage and bandwidth loss. Preloading tasks are generated according to the checked priorities and sent to the terminal's resource scheduling module. The resource scheduling module performs the preloading operation during idle intervals of the main rendering thread, writing the corresponding rendering resources to the terminal's dedicated frame rendering buffer.

[0042] In one embodiment, step S4, which involves calling corresponding rendering resources to perform ray tracing rendering on the 3D scene based on the target LOD level and target ray tracing sampling rate of the verified 3D model, and outputting the current rendering frame, includes: S41, target rendering parameters and corresponding resource retrieval; S42, ray tracing rendering calculations with linked parameters; S43, Rendering screen verification and final frame output.

[0043] In the specific implementation process, the target LOD level, target ray tracing sampling rate, and view frustum parameters of the current frame's user perspective are read from a dedicated area of ​​the GPU's on-chip L2 cache after secondary verification, completing parameter synchronization and binding before rendering. The parameter binding process is completed before the vertex shader stage of the rendering pipeline, and all parameters are directly written to the GPU's constant registers, ensuring zero-latency reading during the rendering calculation stage. Rendering resource retrieval follows the principle of prioritizing preloaded resources, prioritizing the use of preloaded corresponding 3D model resources in the dedicated frame rendering cache, including geometric data matching the target LOD level, ray tracing acceleration structures, texture maps, and material properties. The retrieval of preloaded resources is short and will not block the execution of the main rendering thread. For the very few model resources that have not been preloaded, they are synchronously retrieved from the resource package stored locally on the terminal. The retrieval process uses an asynchronous I / O channel, completing the reading during the rasterization interval of the rendering pipeline to avoid main thread lag. In open-world game city scenes, when the user's perspective is directed towards the core area of ​​the commercial street, the pre-loaded core model resources such as shop facades, pedestrian NPCs, and road props are directly retrieved from the high-speed cache. Only the unexpected vehicle models that suddenly appear in the distance need to be read asynchronously from local storage. The core resource retrieval accounts for more than 95%.

[0044] Based on the target LOD level corresponding to each visible 3D model, the matching model geometry is loaded, vertex data is assembled and primitives are generated, and a ray tracing acceleration structure adapted to the target LOD level is loaded simultaneously. High LOD level models correspond to high-precision hierarchical bounding box structures, while low LOD level models correspond to simplified axial bounding box structures, reducing the computational cost of ray intersection calculations. Ray tracing rendering calculations are performed at the matched target ray tracing sampling rate. The calculation process is executed in parallel on the dedicated ray tracing core of the mobile GPU. Models within the visual focus area are calculated at the full sampling rate, while models in non-focus areas are calculated at an adapted downsampling rate. Simultaneously, a temporal and spatial joint denoising processor is used to complete image noise reduction processing. In the digital twin park scenario, when users closely observe the internal structure of a high-voltage power distribution cabinet, the cabinet model uses level 0 LOD geometry with a ray tracing sampling rate of 32 times per pixel. This accurately reproduces the reflection effect of the cabinet's metal material and the light and shadow details of the internal components. The surrounding pipe fence model uses level 3 LOD geometry with a ray tracing sampling rate of 4 times per pixel, significantly reducing computational power consumption while maintaining the overall visual quality. In XR near-eye rendering scenarios, synchronous rendering calculations are performed for the left and right eye's dual-view frustums, ensuring consistency between the rendering parameters and resources for both eyes. Furthermore, considering the foveal vision characteristic of the human eye, a full sampling rate is used for the central area of ​​the image, while a downsampling rate is used for the edge areas, further optimizing rendering power consumption.

[0045] After completing ray tracing rendering calculations and image compositing, the original rendered image of the current frame is obtained. Image quality compliance checks are then performed on the original rendered image. This check is executed in parallel on the GPU's compute shader units. The checks include calculating the structural similarity and peak signal-to-noise ratio (PSNR) of the entire image. Using the highest-precision rendered image in the same scene as a benchmark, the total visual perception loss of the entire image is confirmed to not exceed a preset global constraint threshold, corresponding to a structural similarity of not less than 0.99 and a PSNR of not less than 45 dB, ensuring that there is no visually perceptible image quality loss. If the loss value of a local area exceeds the threshold during the check, ray tracing resampling calculations are performed on the corresponding area until the image meets compliance requirements. The rendered image that has completed image quality verification is written to the front buffer of the terminal display system. Combined with the vertical synchronization signal, the image is output to the screen. In handheld terminal scenarios, a single complete frame is output; in XR near-eye scenarios, dual frames for both eyes are output simultaneously to ensure synchronization between the image and head posture. After the entire process is completed, the cycle for the next rendering frame begins, completing the continuous operation of the entire rendering method.

[0046] Reference Figure 2 Here is a structural block diagram of an adaptive energy-saving rendering device based on dynamic LOD and ray tracing in one embodiment of the present invention, comprising: The model pre-building module is used to build a unified visual perception loss function and linkage coupling mapping model, as well as a pre-trained dual-branch lightweight LSTM model; The data acquisition module is used to collect the rendering parameters and view prediction feature data of the current frame in parallel frame by frame during real-time rendering, and to complete data verification and caching. The rendering parameter linkage determination module is used to call the linkage coupling mapping model to determine and verify the target LOD level and target ray tracing sampling rate of the 3D model. The viewpoint prediction and resource preloading module is used to output the viewpoint switching direction and rendering resource loading priority through a dual-branch lightweight LSTM model to complete the corresponding resource preloading. The rendering execution and output module is used to call the corresponding resources based on the determined rendering parameters, perform ray tracing rendering, and output the rendering frame.

[0047] For the specific implementation of each module in the above device example, please refer to the above method embodiments, which will not be repeated here.

[0048] In summary, the device described in this embodiment is adapted to the entire workflow requirements of real-time 3D rendering, providing low-power, high-fidelity rendering support for scenarios such as mobile XR interaction, digital twins, and open-world games. The various functional modules within the device operate collaboratively, fully covering the entire pipeline from rendering pre-construction, data acquisition, parameter decision-making, resource scheduling, to image output. The model pre-construction module completes the construction of a unified visual perception loss function and a linkage coupling mapping model, simultaneously completing the pre-training and optimization of a dual-branch lightweight LSTM model, providing a unified image quality constraint benchmark and prediction tool for the entire rendering process. The data acquisition module collects rendering parameters and viewpoint prediction feature data of the current frame in parallel during the vertical synchronization interval of each frame, completing data verification and hierarchical caching, providing a stable and compliant input data source. The rendering parameter linkage determination module, relying on the linkage coupling mapping model, completes the matching and compliance verification of 3D model rendering parameters, achieving power optimization without perceptible visual loss. The viewpoint prediction and resource preloading module completes viewpoint prediction and resource preloading through a dual-branch lightweight LSTM model, reducing the resource retrieval latency of the main rendering thread. The rendering execution and output module completes ray tracing rendering with linked parameters and outputs rendered frames that meet the image quality requirements.

[0049] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0050] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the present invention and embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0051] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0052] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An adaptive energy-saving rendering method based on dynamic LOD and ray tracing, applied to real-time rendering of 3D scenes, characterized in that, Includes the following steps: During the real-time rendering of a 3D scene, the rendering parameters and view prediction feature data of the current frame are collected frame by frame. Using the relative distances of the various object models in the 3D scene as input, a pre-built linkage coupling mapping model is called, and the relative distances of the various object models are input into the linkage coupling mapping model. The model processing method of the linkage coupling mapping model is as follows: check the first LOD level corresponding to each object model and the target ray tracing sampling rate, and render each object model in conjunction with the current frame rendering parameters, the target LOD level and the target ray tracing sampling rate, and suppress the total visual perception loss value after rendering to be less than or equal to the constraint threshold. The view prediction feature data is input into a pre-trained dual-branch lightweight LSTM model. The model processing method of the dual-branch lightweight LSTM model is as follows: output the user's view vector in the three-dimensional scene through the view prediction feature data, simultaneously output the second LOD level and ray tracing rendering data within the predicted field of view corresponding to the view vector, load the priority of the LOD level and ray tracing rendering data, and preload the rendering resources of the corresponding three-dimensional model into the terminal memory according to the priority. Based on the target LOD level and target ray tracing sampling rate of the verified 3D model, the corresponding rendering resources are called to perform ray tracing rendering on the 3D scene and output the current rendering frame.

2. The adaptive energy-saving rendering method based on dynamic LOD and ray tracing according to claim 1, characterized in that, The steps for collecting two types of core data frame by frame during the real-time rendering of a 3D scene—the current frame rendering parameters and the viewpoint prediction feature data—include: Data acquisition is performed during the real-time rendering of the 3D scene, and data is acquired from the current frame rendering parameter acquisition channel and the view prediction feature data acquisition channel after initialization. The core parameters required to adapt to the linkage coupling mapping model are collected through the current frame rendering parameter acquisition channel. After the acquisition is completed, the parameters are cached in the current frame rendering parameter dedicated cache area. The multi-dimensional feature data required to adapt the dual-branch lightweight LSTM model is collected through the perspective prediction feature data acquisition channel. After the acquisition is completed, the time series data is timestamped and cached in the dedicated perspective prediction cache area.

3. The adaptive energy-saving rendering method based on dynamic LOD and ray tracing according to claim 1, characterized in that, The steps of verifying the first LOD level and target ray tracing sampling rate of each object model, and rendering each object model in conjunction with the current frame rendering parameters, target LOD level, and target ray tracing sampling rate, include: Read the spatial position, orientation, and view frustum range parameters of the user's viewpoint in the current frame from the dedicated buffer of the current frame rendering parameters, and perform visibility pre-filtering on all object models in the 3D scene; The relative distance between the current frame rendering parameter and the user's view camera is read from the dedicated buffer. The linkage coupling mapping model is called to calibrate the preliminary parameter combination of the target LOD level and the target ray tracing sampling rate of the only visible model among all object models. By combining the real-time computing power load and battery power status parameters of the terminal device read from the dedicated cache of the current frame rendering parameters, dynamic fine-tuning is performed on the initial parameter combination to obtain the implementation parameter combination; The unified visual perception loss function is invoked to perform a fast secondary verification on the implementation parameter combination to verify that the total visual perception loss after rendering the only visible model does not exceed the constraint threshold. Each of the aforementioned visible-only models is rendered according to the combination of implementation parameters.

4. The adaptive energy-saving rendering method based on dynamic LOD and ray tracing according to claim 3, characterized in that, The step of dynamically fine-tuning the initial parameter combination by combining the terminal device's real-time computing load and battery power basic status parameters read from the dedicated cache of the current frame rendering parameters to obtain the implementation parameter combination includes: A hard constraint is set that the total visual perception loss after rendering does not exceed the global constraint threshold, thus defining an unadjustable boundary. When the real-time computing load of the terminal device exceeds the preset load threshold, or the battery power is lower than the preset low power threshold, the low power adaptation mode is triggered, and the LOD level and ray tracing sampling rate are synchronously reduced for visible models outside the visual focus area. During the synchronous downscaling process, the LOD level of a single model is reduced by no more than 1 level in a single instance, and the reduction in the ray tracing sampling rate does not exceed 50% of the current value; After adjusting and integrating the parameter combinations of all models, the final implementation parameter combination is obtained.

5. The adaptive energy-saving rendering method based on dynamic LOD and ray tracing according to claim 1, characterized in that, The step of outputting the user's view vector in the 3D scene through the view prediction feature data, and simultaneously outputting the second LOD level and ray tracing rendering data corresponding to the view vector within the predicted field of view, includes: The view prediction feature data, which has been validated and aligned with timestamps, is read from the view prediction dedicated cache, and after standardization preprocessing, it is input into the pre-trained dual-branch lightweight LSTM model. The model predicts the user's viewpoint switching direction within a preset time period by outputting the viewpoint prediction branch, generating the corresponding predicted view frustum range, and locking the preloaded effective field of view boundary. Using the effective field of view boundary as a spatial constraint, the resource priority prediction branch of the model is combined with the parameter matching rules of the linkage coupling mapping model to synchronously output the LOD and ray tracing rendering resource loading priority of each 3D model within the range; A threshold check is performed on the loading priority, and non-core models below the preset minimum threshold are eliminated. The corresponding rendering resources are preloaded into the frame rendering dedicated cache area in the terminal memory according to the checked priority.

6. The adaptive energy-saving rendering method based on dynamic LOD and ray tracing according to claim 5, characterized in that, The steps of using the effective view boundary as a spatial constraint, predicting resource priorities in the model, and combining the parameter matching rules of the linkage coupling mapping model to synchronously output the LOD and ray tracing rendering resource loading priorities of each 3D model within the range, include: Filter the 3D models within the effective field of view boundary and remove irrelevant models that exceed the spatial constraints; By combining the relative distance between each 3D model and the predicted viewpoint, the corresponding LOD level and ray tracing sampling rate are matched through the linkage coupling mapping model. Then, the real-time computing power load of the terminal device, the basic state parameters of the battery power, and the visual focus area features of the screen are integrated to sort the resource priority of the 3D models after filtering. The loading weights are divided into multiple levels according to the sorting results, and the rendering resource loading standards corresponding to the core visual area, secondary auxiliary area, and distant weakened area are distinguished.

7. The adaptive energy-saving rendering method based on dynamic LOD and ray tracing according to claim 1, characterized in that, The steps of calling corresponding rendering resources, performing ray tracing rendering on the 3D scene, and outputting the current rendering frame based on the target LOD level and target ray tracing sampling rate of the verified 3D model include: Read the target LOD level, target ray tracing sampling rate, and user view frustum parameters of each visible 3D model after secondary verification from the dedicated buffer of rendering parameters for the current frame; Based on the read rendering parameters, the corresponding 3D model rendering resources that have been preloaded in the dedicated frame rendering cache are called first, and the resources that have not been preloaded are retrieved synchronously from the terminal's local storage. Load the 3D model geometry based on the target LOD level, perform ray tracing rendering calculations at the matching target ray tracing sampling rate, and generate the original rendered image of the current frame. Perform image quality compliance checks on the original rendered image. After verifying that the total visual perception loss of the entire image does not exceed the global constraint threshold, output the final current rendered frame.

8. An adaptive energy-saving rendering device based on dynamic LOD and ray tracing, characterized in that, The adaptive energy-saving rendering method based on dynamic LOD and ray tracing as described in any one of claims 1-7, wherein the adaptive energy-saving rendering apparatus based on dynamic LOD and ray tracing comprises: The model pre-building module is used to build a unified visual perception loss function and linkage coupling mapping model, as well as a pre-trained dual-branch lightweight LSTM model; The data acquisition module is used to collect the rendering parameters and view prediction feature data of the current frame in parallel frame by frame during real-time rendering, and to complete data verification and caching. The rendering parameter linkage determination module is used to call the linkage coupling mapping model to determine and verify the target LOD level and target ray tracing sampling rate of the 3D model. The viewpoint prediction and resource preloading module is used to output the viewpoint switching direction and rendering resource loading priority through a dual-branch lightweight LSTM model to complete the corresponding resource preloading. The rendering execution and output module is used to call the corresponding resources based on the determined rendering parameters, perform ray tracing rendering, and output the rendering frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the adaptive energy-saving rendering method based on dynamic LOD and ray tracing as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the adaptive energy-saving rendering method based on dynamic LOD and ray tracing as described in any one of claims 1 to 7.

Images