Mobile phone end simulation scene interactive display method based on cloud rendering

Abstract

The invention relates to the technical field of mobile phone end cloud rendering display, and discloses a mobile phone end simulation scene interactive display method based on cloud rendering. The method comprises the following steps: collecting user interaction data, equipment state information and network environment parameters of a mobile phone end, and generating an interaction state monitoring value through validity verification, noise filtering and multi-source integration; extracting a rendering engine configuration parameter based on the monitoring value, and selecting a resolution and detail level balance point through an optimization algorithm to obtain a rendering parameter optimization value; in combination with real-time network delay and equipment electric quantity, a rendering strategy is adjusted according to the optimization value, and an environment response adjustment value is generated; setting a frame rate and an image quality threshold and dynamically applying the frame rate and the image quality threshold to obtain a display threshold set value; capturing a user behavior mode and scene complexity change, analyzing a prediction demand through a time sequence, and generating a scene demand prediction value; and comparing the actual rendering output with the predicted value, calculating an error, feeding back and controlling a rendering process, and outputting an optimized interactive display result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile phone end cloud rendering display, in particular to a mobile phone end simulation scene interactive display method based on cloud rendering. BACKGROUND

[0002] With the rapid development of mobile Internet and virtual reality technology, mobile phone end simulation scene interactive display is increasingly widely used in fields such as game entertainment, education and training, and industrial simulation. Users have higher requirements for the picture quality performance, interactive fluency and device adaptability of mobile phone end simulation scenes, and cloud rendering technology can transfer complex scene rendering tasks to cloud servers due to its powerful computing power, and then transmit the rendering results to the mobile phone end display, which becomes an important way to solve the lack of local computing power of mobile phones and realize high-quality simulation scene display.

[0003] However, the current mobile phone end cloud rendering simulation scene interactive display technology still has many deficiencies. In the data collection link, most schemes only focus on user interaction data such as touch operations and gesture instructions, ignoring the collection of mobile device state information (such as CPU occupancy, memory usage, screen brightness) and network environment parameters (such as network bandwidth, data packet loss rate). This single data collection method leads to a lack of comprehensive and accurate basic data support for subsequent rendering decisions, and is prone to problems such as mismatch between rendering parameters and actual device and network conditions. At the same time, some schemes do not verify the effectiveness of the collected data and filter noise, and there may be invalid instructions, transient network fluctuation interference and other situations in the data, which directly affects the reliability of the subsequent data integration results, and then misleads the rendering strategy adjustment.

[0004] In terms of rendering parameter configuration, existing technologies often use fixed resolution and detail level combinations, without considering the performance differences of different mobile devices, network environment fluctuations and changes in user interaction needs. For example, when the network bandwidth is sufficient but the mobile phone battery is low, if high-resolution rendering is still maintained, it will increase device power consumption and accelerate battery consumption; while when the network delay is high but the device performance is good, if the low-detail-level rendering is maintained, it will waste device computing power and reduce scene picture quality performance. This lack of dynamic balance in parameter configuration makes it difficult to find the optimal solution between picture quality and performance, power consumption, affecting user experience.

[0005] The current rendering strategy adjustment depends on preset rules and cannot respond to the dynamic changes of network delay and device power in real time. When the network environment suddenly deteriorates, the preset rules may not be able to reduce the rendering load in time, resulting in frame freezing and increased interaction delay. When the device power rises, the rendering quality cannot be quickly improved, causing resource waste. At the same time, the frame rate and image quality thresholds are mostly static values and do not change with the scene complexity (such as the number of models in the scene and the complexity of light and shadow effects) and user behavior patterns (such as fast operation and slow browsing). In complex scenes, the frame rate may suddenly drop, and in simple scenes, there may be image quality redundancy.

[0006] The existing solutions lack the ability to predict user needs and can only adjust the rendering process according to the current state, which cannot respond to changes in user behavior patterns and scene complexity fluctuations in advance. For example, when the user is about to enter a high-complexity scene, if the rendering resource allocation cannot be adjusted in advance, the scene loading will be slow and the interaction response will be delayed. When the user behavior pattern changes from fast operation to static observation, if the image quality is not improved in time, it will affect the user's perception of scene details. These problems collectively result in the current mobile phone cloud rendering simulation scene interactive display being unable to meet the user's diverse needs in terms of adaptability, smoothness, and image quality performance, limiting its application in more fields. SUMMARY

[0007] The present application aims to provide a mobile phone simulation scene interactive display method based on cloud rendering to solve the problems raised in the background art.

[0008] To achieve the above-mentioned purpose, the present application provides a mobile phone simulation scene interactive display method based on cloud rendering, which comprises: Collecting user interaction data, device state information, and network environment parameters of the mobile phone, verifying the effectiveness of the data and filtering noise, integrating multi-source data, and generating an interaction state monitoring value; Based on the interaction state monitoring value, extracting the configuration parameters of the rendering engine, selecting the balance point of resolution and detail level through an optimization algorithm, and generating a rendering parameter optimization value; According to the rendering parameter optimization value, combining the real-time network delay and device power, adjusting the rendering strategy to match the current environmental conditions, and obtaining an environmental response adjustment value; Based on the environmental response adjustment value, setting the frame rate threshold and image quality threshold, dynamically applying the thresholds to scene rendering, and generating a display threshold setting value; Using the display threshold setting value, capturing changes in user behavior patterns and scene complexity, using time series analysis for demand prediction, and obtaining a scene demand prediction value; According to the scene demand prediction value, by comparing the actual rendering output with the prediction value, the error is calculated and the rendering process is feedback controlled to generate an optimized interactive display output.

[0009] Preferably, the step of generating the interaction state monitoring value comprises: Through the built-in sensors of the mobile phone, the user touch trajectory and gesture intensity data are collected, the device processor load and memory usage are monitored, and the network bandwidth fluctuation information is obtained; The collected raw data is processed in real time, the sliding window mechanism is used to eliminate transient noise, and the data normalization method is applied to scale the multi-source data to a unified dimension; The processed data is integrated, aligned and fused based on the time stamp, the user interaction parameter set, the device state parameter set and the network environment parameter set are generated, and together constitute the interaction state monitoring value.

[0010] Preferably, the step of generating the rendering parameter optimization value by selecting the balance point of resolution and detail level through the optimization algorithm comprises: From the interaction state monitoring value, the scene complexity index and the user preference mode are analyzed, and the configuration options supported by the rendering engine including the resolution ladder and the detail level range are extracted; A multi-objective optimization algorithm is used to balance the rendering overhead and the visual quality, the utility score of different parameter combinations is calculated, and the combination with the highest score is selected as the candidate; The stability of the candidate combination is verified by iteration, the parameter boundary is adjusted to avoid overfitting, and finally the rendering parameter optimization value including the resolution parameter set and the detail level parameter set is generated.

[0011] Preferably, the step of obtaining the environment response adjustment value comprises: Real-time monitoring of network delay fluctuation and device battery remaining capacity, collecting temperature sensor data as auxiliary environmental factors; Based on the monitoring data, a rendering strategy decision tree is dynamically constructed, the texture quality is automatically reduced under high delay conditions, and the geometric details are reduced in low power scenarios; The strategy decision tree output and the rendering parameter optimization value are cross-verified to ensure parameter compatibility, and the environment response adjustment value including the network delay parameter set and the device power parameter set is generated.

[0012] Preferably, the step of generating the display threshold setting value comprises: According to the performance history data in the environment response adjustment value, the frame rate smoothness index and the picture clarity threshold are calculated; A dynamic threshold adjustment mechanism is used to update the upper and lower limits of the threshold according to the real-time rendering load cycle, and a hysteresis effect is applied to avoid frequent fluctuations; The threshold value is embedded into the rendering pipeline control logic to generate display threshold setting values including a frame rate threshold parameter set and a picture quality threshold parameter set.

[0013] Preferably, the step of obtaining the scene demand prediction value comprises: Continuously capturing the user navigation path sequence and the scene object dynamic change frequency, and recording the time distribution of the interaction events; Applying a time series prediction model to analyze the historical data trend to infer the resource demand peak and valley in the future period; Classifying the prediction results into different priority levels to generate the scene demand prediction value including a user behavior parameter set and a scene complexity parameter set.

[0014] Preferably, the step of generating the optimized interactive display output comprises: Real-time acquisition of actual rendering frame rate and delay data, comparison with scene demand prediction value, calculation of absolute error and relative error indicators; Adjusting rendering parameters based on error indicators using proportional-integral-derivative control principle to gradually correct detail level and resolution settings; Applying the adjusted parameters to the cloud rendering server to generate and stream rendering frames to the mobile end, forming the interactive display output.

[0015] Preferably, the step of applying a time series prediction model to analyze the historical data trend comprises: Sliding average processing of user behavior data to eliminate random fluctuations and extract periodic patterns; Using an autoregressive integrated moving average algorithm to train the prediction model, optimizing model parameters to minimize prediction error; After verifying the accuracy of the model, deploying it to the real-time system for online prediction to update the scene demand prediction value.

[0016] Preferably, the step of calculating absolute error and relative error indicators comprises: Acquiring actual value and prediction value sequences within a continuous time window, calculating the deviation of each frame; Using a weighted average method to aggregate multi-frame errors, giving higher weight to recent errors; Normalizing error values to a standard range to generate error indicators for parameter adjustment.

[0017] Preferably, the step of adjusting rendering parameters based on error indicators using proportional-integral-derivative control principle comprises: According to the error indicators, calculating the control output of the proportional term, integral term and derivative term; Limiting the change rate of control output to prevent visual card from stuttering caused by parameter mutation; The control output is mapped to a rendering parameter adjustment amount to achieve a smooth transition interactive display.

[0018] Compared with the prior art, the present application has the beneficial effects that: The method not only collects user interaction data, but also includes mobile device state information and network environment parameters, covering key factors affecting cloud rendering display effects, avoiding the partiality of decisions caused by single data collection. At the same time, invalid data is removed through effectiveness verification, interference information is eliminated by noise filtering, and then the interactive state monitoring value is generated through multi-source data integration, ensuring that the data can truly reflect the current user operation, device operation and network connection, providing accurate basis for subsequent rendering parameter optimization and strategy adjustment, making the rendering decision more suitable for actual application scenarios.

[0019] In the rendering parameter optimization link, the method extracts rendering engine configuration parameters based on the interactive state monitoring value, and finds the balance point of resolution and detail level through optimization algorithm, realizing the dynamic adaptation of rendering parameters. Unlike fixed parameter configuration method, this method can flexibly adjust the combination of resolution and detail level according to device performance (such as CPU occupancy, memory remaining), network conditions (such as bandwidth size, delay level) and user potential demand (such as attention to picture quality, preference for smoothness). For example, when the device performance is strong and the network bandwidth is sufficient, the resolution and detail level can be appropriately increased to present higher picture quality; when the device performance is limited or the network delay is high, the resolution and detail level can be reduced to ensure the smoothness of interaction, avoiding resource waste or poor experience caused by fixed parameters, and finding the optimal balance between picture quality and running smoothness.

[0020] The method also adjusts the rendering strategy in combination with real-time network delay and device power, further improving the environmental adaptability of cloud rendering display. Network delay directly affects the transmission efficiency of rendering results, and device power is related to the battery life of mobile phone, both of which are key variables in mobile phone use scenarios. By including real-time network delay and device power in the rendering strategy adjustment basis, the rendering load and resource allocation can be flexibly adjusted according to the actual environmental changes. When the network delay is high, the rendering data volume can be reduced and the data transmission format can be optimized to reduce the impact of delay on display effect; when the device power is low, the rendering power consumption can be reduced and unnecessary rendering calculation can be reduced to prolong the device usage time. This dynamic adjustment method makes the rendering strategy quickly respond to environmental changes, avoids display lag, short battery life and other problems caused by environmental variables, and improves the user experience in different environments.

[0021] In terms of threshold setting and demand prediction, the method dynamically sets frame rate threshold and image quality threshold based on environmental response adjustment value, and applies it to scene rendering, realizing the matching of threshold and actual situation of scene. The frame rate and image quality threshold are not fixed, but are adjusted with the change of device state, network environment and scene complexity. When the scene complexity is high, the frame rate threshold is appropriately relaxed to ensure basic fluency, and when the scene complexity is low, the image quality threshold is improved to improve the display effect, avoiding the waste of resources caused by too low threshold. At the same time, by capturing user behavior patterns (such as operation frequency, stay time) and scene complexity changes, time series analysis is used for demand prediction to predict user's subsequent operation intention and scene change trend, so that the rendering process can adjust resource allocation and parameter configuration in advance. For example, when it is predicted that the user will enter a high complexity scene, the rendering resources can be prepared in advance and the parameters can be adjusted to cope with the scene change; when it is predicted that the user's operation frequency is reduced, the image quality can be improved in advance to meet the user's demand for detail observation, avoiding the experience fault caused by the delay of temporary adjustment.

[0022] The method compares the actual rendering output with the scene demand prediction value, calculates the error and feeds back the control of the rendering process, forming a closed-loop optimization mechanism. By continuously correcting the rendering parameters, strategies and threshold settings through error feedback, the gap between actual output and user demand is gradually reduced, and the interactive display effect is gradually optimized. For example, if the actual frame rate is lower than the predicted demand frame rate, the detail level can be reduced and the rendering algorithm can be optimized to improve the frame rate through feedback adjustment; if the actual image quality does not meet the predicted demand image quality, the resolution and detail level can be improved under the condition that the device and network conditions allow. This closed-loop mechanism ensures that the rendering process can continuously adapt to user demand and environmental changes, so that the interactive display of mobile phone end cloud rendering simulation scene can maintain good performance in different users, different devices and different network environments, enhance the application adaptability of the technology in the fields of game, education, industry, etc., and improve the user's satisfaction with the use of mobile phone end simulation scene interactive display. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The working principle diagram of the mobile phone end simulation scene interactive display method based on cloud rendering is described. Figure 2 The flowchart for generating interactive state monitoring value is described. Figure 3 The flowchart for obtaining environmental response adjustment value is described. DETAILED DESCRIPTION

[0024] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0025] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application. Figure 1 The present application provides a mobile phone terminal simulation scene interactive display method based on cloud rendering, which comprises the following steps: collecting user interaction data, device state information and network environment parameters through a mobile phone terminal, performing validity verification and noise filtering processing on the collected original data, and integrating the multi-source data to generate a comprehensive interaction state monitoring value. Based on the interaction state monitoring value, the configuration parameters of the rendering engine are extracted, an optimal balance point is selected between resolution and detail level by using an optimization algorithm, and a set of optimized rendering parameter optimization values are generated. The rendering parameter optimization values are combined with real-time network delay and device power information, the rendering strategy is dynamically adjusted to match the current environmental conditions, and an environmental response adjustment value is generated. The environmental response adjustment value is used to set dynamic frame rate threshold and image quality threshold, and the threshold is applied to the scene rendering process to form a display threshold setting value. The system continuously captures the changes of user behavior patterns and scene complexity, predicts future resource requirements by using time series analysis technology, and obtains a scene demand prediction value. The scene demand prediction value is compared with the actual rendering output, the calculated error is used for feedback control of the rendering process, and finally an optimized and smooth interactive display output is generated and presented on the mobile phone terminal.

[0026] Embodiment 1: With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application. Figure 2 The various sensors built-in in the mobile phone terminal constitute the basic layer of data acquisition, the touch screen sensor captures the contact coordinate sequence of the user's finger at a high sampling rate to form touch trajectory data, and the motion sensor such as a three-axis accelerometer and a gyroscope monitors the attitude change and rotation angular velocity of the mobile phone. These data are converted into gesture intensity data representing the operation intention and intensity after preliminary filtering. The monitoring of the device running state is continuously performed in the background of the system, the processor load data are obtained by reading the performance counters exposed by the operating system kernel, the real-time CPU and GPU computing resource occupation percentages are reflected, the memory usage rate is obtained by monitoring the allocation of available physical memory and virtual memory, the network interface controller continuously measures the data transmission rate and stability between the cloud rendering server and the mobile phone terminal, and the network bandwidth fluctuation information is provided. These original data streams collected from different sources are not completely synchronized in time, and there are significant differences in data format and dimension. The touch trajectory is a two-dimensional coordinate sequence, the processor load is a percentage value, and the network bandwidth is a bit rate value. Direct comparison or fusion lacks feasibility.

[0027] The real-time processing of raw data stream is carried out in a dedicated data preprocessing module, which processes the incoming data packets one by one using a stream processing architecture. The sliding window mechanism is the core technology to eliminate transient noise. The system maintains a fixed-size time window, and the data points within the window participate in the calculation. New data points are added while the oldest data points are removed. For touch coordinate data, the sharp jumps of coordinate points in the sliding window are regarded as noise points and smoothed out by the median filtering algorithm. For network bandwidth data, the transient high or low rate fluctuations are averaged within the window, resulting in a more stable bandwidth estimate. The data normalization method scales the processed multi-source data to a unified numerical range. The min-max normalization method linearly transforms each data item to between zero and one. For example, the processor load is mapped from the original range of zero to one hundred percent to the interval of zero to one, and the network bandwidth is also mapped from zero to the theoretical maximum value to the interval of zero to one. After normalization, different sensors and different physical data are converted into dimensionless scalars, making it possible to perform joint analysis. The integrated processed data requires precise time synchronization. Each batch of collected data is stamped with a high-precision timestamp. The timestamp alignment algorithm matches the data streams from user interaction, device status, and network environment according to the timestamps, and classifies the data points generated at the same time or within a very short time interval into the same time slot. The data fusion operation combines the normalized data in the same time slot into a multi-dimensional feature vector. This feature vector contains the user interaction parameter set, the device status parameter set, and the network environment parameter set. The user interaction parameter set may include the normalized touch operation frequency and gesture amplitude average value. The device status parameter set includes the normalized processor load and memory usage pressure. The network environment parameter set includes the normalized current bandwidth and delay estimate. This complete multi-dimensional feature vector constitutes the interaction state monitoring value, which is a quantitative snapshot of the current comprehensive state of the mobile phone.

[0028] The interaction state monitoring values are passed to the parameter extraction and optimization module, the scene complexity indicator is resolved from the interaction state monitoring values, the resolution process is based on the analysis of the current simulation scene's description file and the statistics of the user's historical interaction behavior, the number of rendering objects, the total number of polygon faces, and the number of dynamic light sources in the scene are summarized into a complexity score. The user preference pattern is inferred by analyzing the user's operation history in a time period, if the user frequently performs fast view switching or panning operations, the system tends to infer that the user pays more attention to the smoothness of interaction, if the user often performs static observation or fine operations, the system infers that the user pays more attention to the detail quality of the picture. The rendering engine reports its supported configuration options to the system when initializing, these options constitute the set of adjustable parameters, the resolution ladder refers to the optional width and height pixel combinations of the rendering output image, usually divided into multiple ladders from low resolution such as 640x480 to high resolution such as 2560x1440, the detail level range refers to different levels of detail versions available for three-dimensional models according to the viewing distance or importance, from high polygon models to low polygon models or even simplified icon representations.

[0029] A multi-objective optimization algorithm is applied to find the best balance point of resolution and detail level, the algorithm considers rendering overhead and visual quality as two target functions that need to be considered simultaneously, rendering overhead is quantified by estimated rendering time per frame, power consumption, visual quality is quantified by theoretical image clarity, texture detail richness. The utility function is designed to combine the two objectives into a single score, different parameter combinations are substituted into the utility function for calculation, one combination may emphasize low overhead by choosing low resolution and low detail level, another combination may emphasize high quality by choosing high resolution and high detail level. The algorithm traverses or intelligently searches the possible parameter combination space, calculates the utility score of each combination, the parameter combination with the highest score is selected as the candidate solution. The stability of the candidate combination needs to be verified by actual testing, the verification process is carried out in a sandbox environment on the cloud rendering side, the candidate parameter combination is applied to render a representative scene sequence, the system observes whether the frame generation time is stable under this parameter, whether the picture quality meets the expectation. Parameter boundary conditions are checked to prevent overfitting, for example, in the extreme case of a sharp decline in network bandwidth, blindly choosing the lowest resolution may not be the best strategy, the system needs to set a resolution lower limit to ensure basic recognizability. After iterative verification and boundary adjustment, the finally determined parameter combination is formally generated as the rendering parameter optimization value, the rendering parameter optimization value explicitly includes the specific resolution parameter set to be adopted, such as 1920x1080, and the detail level parameter set to be adopted for each model object in the scene.

[0030] Embodiment 2: see Figure 3, Real-time monitoring of network latency fluctuation data relies on the network status interface provided by the mobile operating system and the timestamp information fed back by the cloud rendering server. By calculating the round-trip time of the data packet, the current network link data transmission delay in milliseconds can be accurately obtained. The percentage of device battery remaining capacity information is directly read from the mobile power management chip. The power value is sampled at a fixed frequency and the trend is recorded. The chip junction temperature data provided by the internal temperature sensor of the device is an important environmental factor that affects the continuous performance of the device and is collected synchronously. These real-time monitoring data constitute the basis of dynamic decision-making. A rendering strategy decision tree data structure is built and maintained in the system memory. The root node of the decision tree represents the current system state, and the branch conditions are composed of network delay threshold, device battery remaining capacity threshold and device temperature threshold. The construction rule of the decision tree is based on the preset strategy mapping relationship. High latency conditions are defined as scenarios where network delay in milliseconds exceeds a certain threshold. In this scenario, the decision tree triggers the strategy branch of reducing texture quality, and selects the texture map format with higher compression ratio and smaller data size. Low power scenarios are defined as scenarios where the percentage of device battery remaining capacity is below a certain threshold. The decision tree triggers the strategy branch of reducing geometric details in this scenario, instructing the rendering engine to use a simplified model with fewer polygons instead of a high-precision model.

[0031] The output of the rendering policy decision tree is a specific set of policy instructions, which needs to be cross-verified with the rendering parameter optimization values obtained from the previous stage to ensure the internal consistency of system parameters. The cross-verification process checks whether there is a logical conflict between the policy instructions and the rendering parameter optimization values, for example, in the case where the rendering parameter optimization values have set a higher target resolution, the policy instructions require a significant reduction in texture quality, which may cause visual defects in which the picture clarity does not match the texture details. The verification logic evaluates the compatibility of such a parameter combination, and when a potential conflict is detected, a parameter negotiation process is started, which may appropriately lower the target resolution to adapt to the policy of reducing texture quality, or fine-tune the texture compression rate to better match the resolution without touching the core policy target. The network delay parameter set is determined after cross-verification, and the parameter set includes specific delay threshold definitions and texture quality level mapping tables for different delay levels. The device power parameter set is also clearly defined, and the parameter set includes power threshold values that trigger power saving strategies, the range of model detail levels allowed at different power levels, and the minimum picture quality standard reserved for basic operation. The generation of the environment response adjustment value marks that the system has integrated real-time environmental constraints and basic rendering preferences. The performance history data contained in the environment response adjustment value is derived from the continuous recording of the rendering log module. The calculation of the frame rate smoothness index relies on the sequence of generation time intervals of consecutive rendering frames in the past period of time. By calculating the standard deviation or coefficient of variation of these time intervals, the degree of frame rate fluctuation is quantified. A lower coefficient of variation represents a more stable frame rate output. The determination of the picture clarity threshold value refers to the subjective quality evaluation database of historical rendering pictures at various compression levels and resolutions. The system maps the corresponding minimum acceptable clarity parameters from the database according to the target picture quality level set in the environment response adjustment value. The dynamic threshold adjustment mechanism starts to work, and the core of this mechanism is a feedback loop. Real-time rendering load data is used as the input of the loop, and the load data refers to the ratio of the theoretical calculation resources required by the current rendering frame to the actual consumption time.

[0032] The dynamic threshold adjustment mechanism updates the frame rate threshold upper and lower limits and the image quality threshold upper and lower limits according to the real-time rendering load cycle. When it is detected that the rendering load is continuously increasing and approaching the system processing capability limit, the mechanism will moderately lower the frame rate threshold upper limit to prevent system overload and screen freezing caused by excessive pursuit of high frame rate. The image quality threshold lower limit will also be appropriately raised when the system load is extremely high, allowing the rendering pipeline to temporarily use simpler rendering effects to ensure basic smoothness. Hysteresis is integrated into the adjustment logic, which means that the threshold adjustment action requires a clear trigger condition and needs to be maintained for a certain stable time. Only when the change amplitude of the rendering load exceeds the preset hysteresis interval and remains for a certain period of time, the threshold will be recalculated and updated. This design effectively avoids the frequent oscillation of the threshold caused by small normal fluctuations in the load. The final determined frame rate threshold parameter set and image quality threshold parameter set are directly embedded into the core control logic of the rendering pipeline. The frame rate threshold parameter set defines the frame rate target range that the system strives to achieve under the current environment and the frame rate warning line that must be taken. The image quality threshold parameter set clearly defines the conditions for each image quality effect switch, the selection criteria for texture filtering mode, and the adjustment boundaries for shadow quality level. These parameter sets together constitute the display threshold setting value, which serves as a specific guidance criterion for rendering each frame, directly participating in the generation process of rendering instructions, controlling the execution details of GPU shaders and texture sampling operations, and finally achieving sensitive response to environmental changes and fine control of resource allocation.

[0033] In embodiment 3, the capture of the user navigation path sequence relies on the fusion calculation of touch input and inertial measurement unit data on the mobile phone. The system records the continuous coordinate points of the user's movement in the virtual simulation scene, and these coordinate points form a motion trajectory with a time stamp. The monitoring of the dynamic change frequency of scene objects is achieved by tracking the number and state update rate of active entities in the scene graph, such as the number of moving vehicles in the environment and the switching animation frequency of interactive devices. The changes are recorded as event logs. The time distribution information of interactive events is recorded in detail, and the exact time and duration of each touch start, touch end, scaling, and rotation operation are stored in a time series database, forming a historical archive of user behavior. These massive time series data contain the user's usage habits and the inherent laws of the scene, which are too complex and lagging to be directly used for real-time control. A mechanism is needed to extract features and predict future trends from them.

[0034] Time series forecasting models aim to infer future resource demands from historical data. User behavior data needs preprocessing before being input into the model to eliminate the influence of random fluctuations. Moving average is a basic data smoothing technique. Moving average operates on user behavior data sequences by calculating the arithmetic mean of data points within a fixed window width. This average is used to represent the data level at the center of the window, and the window slides forward as new data arrives. For a data point sequence representing the intensity of user actions... Its simple moving average The calculation formula is: in: Indicates at a point in time Recorded raw values ​​of user action intensity This represents the smoothed value obtained after the moving average process. It is a positive integer that defines the half-width of the sliding window, so the total number of data points contained in the entire window is . This formula effectively filters out short-term random fluctuations by calculating the arithmetic mean of all data points within the window, highlighting long-term trends and periodic patterns in the sequence. After the data sequence is processed by moving average, short-term noise is suppressed, and periodic patterns that users may frequently operate in specific time periods (such as the first ten seconds of each minute), or trend patterns with gradually increasing operation intensity, become clearer.

[0035] The Autoregressive Ensemble Moving Average (AMA) algorithm was chosen to train the core prediction model. This algorithm combines the characteristics of autoregressive models, differencing methods, and moving average models, enabling it to model time series with certain trends and seasonality. The model requires defining three key parameters: the autoregressive order *p*, the differencing order *d*, and the moving average order *q*. The autoregressive order *p* represents the dependence of the predicted value on the observations over the past *p* time points; the differencing order *d* represents the number of differencing operations performed on the original series to make it more stationary; and the moving average order *q* represents the prediction error depending on the predicted value over the past *q* time points. An optimization algorithm was used to find the optimal combination of *p*, *d*, and *q* parameters. The algorithm performs multiple fitting attempts on historical datasets, using the minimization of the sum of squared errors between the predicted and actual observations as the objective function. After training, the model's accuracy is evaluated on an independent validation dataset. Evaluation metrics include mean absolute error and root mean square error. Only models that meet the preset accuracy standards are deployed.

[0036] The qualified ARIMA model is integrated into the real-time prediction system, which receives the latest user behavior parameter set and scene complexity parameter set in a streaming manner. The user behavior parameter set contains smoothed operation frequency, navigation path inflection point, etc. The scene complexity parameter set contains dynamic object count, texture memory occupancy estimation, etc. The model uses the latest time series data window to make forward multi-step prediction, inferring the time points and intensity of the peak and valley of the demand for cloud rendering server computing resources and network bandwidth in the future short period (e.g. 5-10 seconds). The prediction result is a set of resource demand estimation values at consecutive time points. The system divides the predicted demand into priority levels according to the rules defined by the application logic. For example, if the system predicts that the user will quickly turn to an area containing a large number of complex models, the demand for GPU computing resources at that moment is marked as high priority. If the system predicts that the user may enter a relatively static observation phase, the demand for that period is marked as low priority. The final generated scene demand prediction value is a structured data object, which contains the predicted sequence of user behavior parameters, such as predicted touch event distribution, and the predicted sequence of scene complexity parameters, such as predicted polygon number variation.

[0037] In embodiment 4, the cloud rendering server packages the actual rendering output performance data after completing the rendering of each frame of picture and sends it to the mobile client along with the video stream. These data include the actual rendering frame rate within a time sequence and the end-to-end delay data from the start of rendering to the arrival of frame data at the mobile end. The monitoring process on the mobile client receives and analyzes these data, extracts the actual rendering frame rate sequence and the actual delay sequence within the continuous time window, and provides the predicted frame rate sequence and the predicted delay sequence within the corresponding time window. The actual value sequence and the predicted value sequence are strictly aligned on the time axis. The system compares the actual rendering frame rate and the predicted rendering frame rate at the same frame number one by one, calculates the difference between them as the absolute error of that frame. The relative error is also calculated in frames. The absolute error value is divided by the actual rendering frame rate value of that frame to obtain a percentage value representing the degree of deviation.

[0038] The error data of consecutive multiple frames are collected to generate more representative error indicators. The weighted average method is applied to aggregate these errors. The errors of recent frames are given higher weight coefficients, so that the error indicators can more sensitively reflect the current performance deviation of the system. Referring to Table 1, an error calculation table containing five frames of data is shown, which shows the weight allocation and aggregation process.

[0039] Table 1: Frame rendering error calculation table In the frame rendering error calculation shown in Table 1, the frame sequence number identifies the order of rendering frames, the predicted frame rate comes from the scene demand prediction value, and the actual frame rate comes from the feedback data of the cloud rendering server. The absolute error is the absolute value of the difference between the predicted frame rate and the actual frame rate, and the relative error is the percentage of the absolute error to the actual frame rate. The assigned weight increases linearly with the increasing frame sequence number, the weight of the newer frame is higher, and the sum of the weights of all frames is 1. The weighted absolute error is the product of the absolute error and the assigned weight, and their sum of 3.45fps is the aggregated absolute error indicator in this time window. The relative error indicator is also calculated using the same weighted average method. The aggregated absolute error indicator and the relative error indicator need to be normalized to fall within a unified, unitless standard range, such as between zero and one, and the normalization process uses a preset maximum allowed error value as a reference. The normalized error indicator is used as the input signal of the control system, and the proportional-integral-derivative control principle is used to calculate how to adjust the rendering parameters to reduce the error. The proportional term calculates the adjustment amount proportional to the current error indicator, the larger the error, the larger the adjustment amplitude. The integral term calculates the adjustment amount proportional to the cumulative sum of the error indicators in the past period of time, which is used to eliminate the persistent static error of the system. The derivative term calculates the adjustment amount proportional to the rate of change of the current error indicator, which is used to predict the error trend and apply advance correction. The outputs of the proportional term, the integral term and the derivative term are weighted and summed to produce a total control output signal, which represents the desired direction and intensity of rendering parameter adjustment.

[0040] The control output signal needs to be processed by a rate limiting module before it acts on the rendering parameters. The rate limiting module sets a maximum allowed change magnitude of the control signal within two consecutive adjustment periods, which prevents abrupt changes in resolution or detail level settings caused by drastic fluctuations in the error indicator. Smooth parameter changes are crucial for maintaining the coherence of the user's visual experience, avoiding frequent jumps or perceptible stutters in picture quality within a short period of time. The stabilized control output signal after amplitude limiting is mapped to specific rendering parameter adjustment amounts, with the mapping relationship determined by a predefined parameter adjustment lookup table or a linear transformation function. The positive and negative signs and magnitudes of the control output signal are converted into step sizes for increasing or decreasing the detail level, as well as fine-tuning amounts for the resolution parameter. The adjusted rendering parameters are encapsulated into control instructions, which are sent to the cloud rendering server over the network. The cloud rendering server receives and parses these instructions and dynamically adjusts the configuration of its rendering pipeline. The cloud rendering server uses the new parameter settings to render subsequent frames, which are encoded and compressed before being sent to the mobile phone through a streaming protocol. The mobile phone receives the video stream and decodes and displays it. This process constitutes a complete closed-loop feedback control system, in which the actual rendering output collected in real time is compared with the predicted scene demand value, and the resulting error drives the control algorithm to adjust the rendering parameters. The cloud rendering server generates pictures based on the new parameters, and the final optimized interactive display output is formed on the mobile phone. This cycle continues to run continuously, allowing the system to adapt to dynamic changes in network conditions, scene complexity, and user operations, and maintain a rendering performance level as close as possible to the predicted target.

[0041] In the embodiment 5, the system collects the actual rendering frame rate sequence in the continuous time window and the predicted frame rate sequence in the scene demand prediction value, taking a sequence containing five continuous rendering frames as an example, the frame sequence number is from 1001 to 1005, and the corresponding predicted frame rate is 60 frames per second, 60 frames per second, 59 frames per second, 59 frames per second and 58 frames per second respectively. The actual rendering frame rate fed back by the cloud rendering server is 58 frames per second, 57 frames per second, 56 frames per second, 55 frames per second and 54 frames per second respectively. For each frame in the sequence, the deviation between the actual value and the predicted value, i.e. the absolute error, is calculated. For frame 1001, the absolute error is the absolute value of 60 minus 58, which is 2 frames per second. For frame 1002, the absolute error is the absolute value of 60 minus 57, which is 3 frames per second. The absolute error of frame 1003 is 3 frames per second, the absolute error of frame 1004 is 4 frames per second, and the absolute error of frame 1005 is 4 frames per second. The relative error is calculated in percentage form. The relative error of frame 1001 is 2 divided by 58 and multiplied by 100%, which is about 3.45%. The relative error of frame 1002 is 3 divided by 57, which is about 5.26%. The relative error of frame 1003 is about 5.36%. The relative error of frame 1004 is about 7.27%. The relative error of frame 1005 is about 7.41%.

[0042] To get a single indicator that can represent the overall performance fluctuation in recent time, the weighted average method is used to aggregate the errors of multiple consecutive frames. The core idea of the weighted average method is that the data points closer to the current time can better reflect the instantaneous state of the system. In the weighted average, the error data closer to the current frame in time is given a higher weight coefficient. A specific weight distribution scheme can be to assign an increasing set of weights to the five consecutive frames, for example, frame 1001 with a weight of 0.1, frame 1002 with a weight of 0.15, frame 1003 with a weight of 0.2, frame 1004 with a weight of 0.25, and frame 1005 with a weight of 0.3, and the sum of all weight coefficients is 1.0. The calculation of the weighted absolute error indicator is the absolute error of each frame multiplied by its corresponding weight. The contribution of frame 1001 is 2 multiplied by 0.1, which equals 0.2, the contribution of frame 1002 is 3 multiplied by 0.15, which equals 0.45, the contribution of frame 1003 is 0.6, the contribution of frame 1004 is 1.0, and the contribution of frame 1005 is 1.2. Summing these weighted values gives the aggregated absolute error indicator of 3.45 frames per second. The weighted relative error indicator is aggregated using the same weights. Three percent of 3.45 multiplied by 0.1 plus five percent of 5.26 multiplied by 0.15 plus five percent of 5.36 multiplied by 0.2 plus seven percent of 7.27 multiplied by 0.25 plus seven percent of 7.41 multiplied by 0.3 finally gives an aggregated relative error indicator of approximately six percent. The aggregated absolute error indicator and the relative error indicator are values with original dimensions. The absolute error indicator is in units of frames per second, and the relative error indicator is in units of percentage. To facilitate the proportional-integral-derivative controller and to unify the order of magnitude, normalization is required. The normalization function maps the actual error value to a standard range of zero to one. The mapping process is based on the pre-set maximum allowed error value of the system. For absolute error, the maximum allowed error can be set to 10 frames per second, so the absolute error indicator of 3.45 frames per second is normalized to 0.345. For relative error, the maximum allowed error can be set to 20 percent, so the relative error indicator of 6.01 percent is normalized to 0.3005. After normalization, the two error indicators are converted into dimensionless scalars, and the numerical value directly reflects the severity of the current performance deviation.

[0043] The normalized error indicator is input to the calculation unit of the proportional-integral-derivative controller. The output of the proportional term is proportional to the current error value. The current error is 0.345, and the proportional coefficient is set to 0.5. The proportional term output is 0.1725. The output of the integral term is proportional to the cumulative sum of all error values in the past period. The integrator records the error values of the past four control periods, which are assumed to be 0.2, 0.25, 0.28, and 0.31, respectively. The integral coefficient is set to 0.1, and the integral term output is the sum of these values multiplied by the coefficient, i.e. .

[0044] The output of the derivative term is proportional to the difference between the current error and the error of the last control cycle, which is 0.31, and the current cycle error is 0.345, the difference is 0.035, and the derivative coefficient is set to 0.2, so the output of the derivative term is 0.007. The total output value of the proportional-integral-derivative control is obtained by adding the proportional term output, the integral term output, and the derivative term output, which is 0.1725 plus 0.104 plus 0.007, equal to 0.2835, which represents the theoretically required control strength. The total output value of the proportional-integral-derivative control must be processed by the rate limiting module before being sent to the rendering parameter adjustment module, which sets the maximum allowed absolute change in the control signal between adjacent two adjustment cycles, for example, 0.1. The output value of the last control cycle is 0.2, and the theoretically calculated output value of the current cycle is 0.2835, and the theoretical change is 0.0835, which is less than the maximum allowed change of 0.1, so the rate limiting module allows the change to pass completely, and the final control output value of the current cycle is 0.2835. Suppose another situation, the theoretical change is calculated to be 0.15, which exceeds the maximum allowed change of 0.1, then the rate limiting module will limit the change to 0.1, and the final control output value of the current cycle is the output value of the last cycle 0.2 plus 0.1, equal to 0.3. This limitation effectively prevents the rendering parameters from jumping sharply due to sudden changes in error measurement or model transient fluctuations, and avoids frequent oscillation between high and low quality modes.

[0045] The final control output value after limiting is mapped to a specific rendering parameter adjustment, and the mapping relationship is realized through a predefined linear transformation function. The control output value 0.2835 is in the range of zero to one, while the detail level adjustment step is defined between negative 5 and positive 5. The mapping function maps the control output value zero to the adjustment step negative 5, and the control output value one to the adjustment step positive 5. The adjustment step corresponding to the control output value 0.2835 is calculated by linear interpolation as negative 5 plus (0.2835 times 10) equal to negative 2.165, rounded to negative 2. This negative 2 adjustment step means that the system needs to adjust the global detail level parameter downward by two levels, for example, from high detail level to medium detail level, to reduce the rendering load and make the actual frame rate closer to the predicted frame rate. The adjustment of the resolution parameter also follows similar mapping logic, and the control output value is mapped to the resolution scaling coefficient to realize the smooth increase and decrease of the resolution. The introduction of the proportional-integral-derivative control principle makes the adjustment of the rendering parameters a smooth and gradual process, and the parameter change is related to the sustained error trend, rather than an overreaction to transient fluctuations. This control method provides a basis for smooth transition in interactive display, and the user perceives a continuous rather than sudden change in picture quality.

[0046] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and illustrative figures, it should be apparent that the scope of the present application is not limited to these specific embodiments.

[0047] While the embodiments of the application have been shown and described herein, it will be understood by those skilled in the art that many changes, modifications, substitutions and alterations to these embodiments can be made without departing from the principles and spirits of the application, and it is intended that the scope of the application be limited solely by the scope of the appended claims and the equivalents thereof.

Claims

1. A mobile-based interactive display method for simulated scenes based on cloud rendering, characterized in that, The method includes: Collect user interaction data, device status information, and network environment parameters from mobile devices, verify the validity of the data and filter noise, integrate multi-source data, and generate interaction status monitoring values; Based on the interaction status monitoring values, the configuration parameters of the rendering engine are extracted, and the balance point between resolution and level of detail is selected through optimization algorithms to generate optimized rendering parameter values. Based on the optimized rendering parameter values, combined with real-time network latency and device power consumption, the rendering strategy is adjusted to match the current environmental conditions, resulting in an environmental response adjustment value. Based on the environmental response adjustment value, a frame rate threshold and an image quality threshold are set, and the thresholds are dynamically applied to scene rendering to generate a display threshold setting value. By using the aforementioned display threshold setting, user behavior patterns and changes in scene complexity are captured, and time series analysis is used to predict demand, thereby obtaining scene demand prediction values. Based on the predicted values ​​of the scenario requirements, the error is calculated by comparing the actual rendering output with the predicted values, and the error is fed back to control the rendering process to generate an optimized interactive display output.

2. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 1, characterized in that, The steps for generating interaction status monitoring values ​​include: The system collects user touch trajectory and gesture intensity data through built-in sensors on the mobile phone, while monitoring the device's processor load and memory usage, and obtaining network bandwidth fluctuation information. The collected raw data is processed in real time, a sliding window mechanism is used to eliminate instantaneous noise, and a data normalization method is applied to scale the multi-source data to a uniform dimension. The integrated data is aligned and merged based on timestamps to generate user interaction parameter sets, device status parameter sets, and network environment parameter sets, which together constitute the interaction status monitoring values.

3. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 1, characterized in that, The step of selecting a balance point between resolution and level of detail using an optimization algorithm to generate optimized rendering parameter values ​​includes: The scene complexity index and user preference patterns are analyzed from the interaction status monitoring values, and the configuration options supported by the rendering engine, including resolution steps and level of detail range, are extracted. A multi-objective optimization algorithm is adopted to balance rendering overhead and visual quality, calculate the utility score of different parameter combinations, and select the combination with the highest score as the candidate. By iteratively verifying the stability of candidate combinations and adjusting parameter boundaries to avoid overfitting, optimized rendering parameter values ​​containing resolution parameter sets and detail level parameter sets are finally generated.

4. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 1, characterized in that, The steps for obtaining the environmental response adjustment value include: Real-time monitoring of network latency fluctuations and remaining device battery power, and collection of temperature sensor data as an auxiliary environmental factor; A rendering strategy decision tree is dynamically constructed based on monitoring data, which automatically reduces texture quality under high latency conditions and reduces geometric details in low power scenarios. The strategy decision tree output is cross-validated with the optimized rendering parameters to ensure parameter compatibility, and environmental response adjustment values ​​containing network latency parameter sets and device power parameter sets are generated.

5. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 1, characterized in that, The step of generating the display threshold setting value includes: Based on historical performance data in the environmental response adjustment values, calculate the frame rate smoothness index and image quality clarity threshold; A dynamic threshold adjustment mechanism is adopted, which updates the upper and lower limits of the threshold cyclically according to the real-time rendering load, and applies the hysteresis effect to avoid frequent fluctuations. The threshold is embedded in the rendering pipeline control logic to generate display threshold settings that include a set of frame rate threshold parameters and a set of image quality threshold parameters.

6. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 1, characterized in that, The steps for obtaining the predicted value of scenario demand include: Continuously capture user navigation path sequences and the frequency of dynamic changes in scene objects, and record the temporal distribution of interaction events; Apply time series forecasting models to analyze historical data trends and infer the peak and trough values ​​of resource demand in future periods; The prediction results are categorized into different priority levels to generate scenario demand prediction values ​​that include user behavior parameter sets and scenario complexity parameter sets.

7. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 1, characterized in that, The step of generating the optimized interactive display output includes: Real-time collection of actual rendering frame rate and latency data, comparison with scene requirement prediction values, and calculation of absolute error and relative error indices; Based on the error index, the rendering parameters are adjusted using the proportional-integral-derivative control principle to gradually correct the level of detail and resolution settings. The adjusted parameters are applied to the cloud rendering server to generate and stream rendering frames to the mobile device, forming an interactive display output.

8. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 6, characterized in that, The steps for using the time series forecasting model to analyze historical data trends include: User behavior data is processed using a moving average to eliminate random fluctuations and extract periodic patterns; The prediction model was trained using an autoregressive ensemble moving average algorithm, and the model parameters were optimized to minimize the prediction error. After verifying the model's accuracy, it is deployed to a real-time system for online prediction, updating the predicted values ​​for scenario requirements.

9. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 7, characterized in that, The steps for calculating the absolute error and relative error indices include: Collect the sequence of actual and predicted values ​​within a continuous time window, and calculate the deviation for each frame; A weighted average method is used to aggregate errors from multiple frames, giving higher weight to recent errors. The error value is normalized to the standard range, and an error index is generated for parameter adjustment.

10. The interactive display method for mobile simulation scenes based on cloud rendering according to claim 7, characterized in that, The steps for adjusting rendering parameters based on error indicators using the proportional-integral-derivative control principle include: Calculate the control output of the proportional, integral, and derivative terms based on the error index; Limit the rate of change of the control output to prevent sudden parameter changes from causing visual stuttering; By mapping control outputs to rendering parameter adjustments, a smooth, interactive display can be achieved.

Images