A method for real-time regulation of a rendering frame rate of a web page based on an agent

Abstract

The application relates to the technical field of Web page rendering, and in particular to a method for real-time regulation of a Web page rendering frame rate based on an intelligent agent, which comprises the following steps: monitoring the current interaction intensity of a user on a Web page; when the interaction intensity reaches a preset threshold, capturing a user interaction event stream and page rendering pipeline state data, and generating a low-dimensional continuous state vector; inputting the low-dimensional continuous state vector into a policy network of a reinforcement learning intelligent agent, outputting original candidate actions of multiple preset discrete frame rate gears, calculating a softening coefficient based on the interaction intensity and the gear priority, performing compliance mapping on the original candidate actions to obtain a compliance action, and analyzing the rendering scheduling parameters corresponding to the compliance action to regulate the Web page. The application can realize smooth transition of the frame rate gears, and improve the page rendering fluency and the interaction response performance.

Description

Technical Field

[0001] This application relates to the field of web page rendering technology, and in particular to a method for real-time control of the rendering frame rate of a web page based on an intelligent agent. Background Technology

[0002] With the continuous evolution of internet applications, web pages have gradually developed from traditional information display interfaces into complex interactive platforms integrating dynamic graphics rendering, real-time data interaction, online editing, visual operation, and lightweight application support. In scenarios such as online office work, web-based design tools, data dashboards, and web games, users typically perform frequent interactive operations such as clicking, dragging, scrolling, zooming, and inputting, thus placing high demands on page rendering smoothness and interactive response speed. To ensure visual continuity and responsiveness, relevant web pages usually need to maintain a high rendering frame rate.

[0003] In existing technologies, the balance between page rendering performance and terminal resource consumption is typically addressed through methods such as fixed frame rate operation, static frequency reduction based on device performance status, frame rate switching based on scene recognition, or the introduction of reinforcement learning models to make dynamic decisions based on terminal computing power and resource usage. While these solutions can alleviate the problems of increased power consumption, aggravated heat generation, and reduced battery life caused by high frame rate rendering to some extent, they still have shortcomings when dealing with high-frequency interactive web pages.

[0004] Specifically, existing solutions often focus more on factors such as device computing power, temperature, resource consumption, or page content type, while insufficiently considering the instantaneous changes in user interaction intensity and their impact on frame rate control strategies. Especially when introducing reinforcement learning into web page rendering frame rate control, to avoid incorrectly selecting excessively low frame rate levels during high-interaction phases, it is usually necessary to set corresponding interaction priority limiting mechanisms to constrain the agent's output actions. However, when the user interaction intensity is near the threshold boundary, this limiting mechanism tends to frequently switch between being active and deactivated, causing the original actions generated by the agent during the exploration phase to be repeatedly overwritten. This results in a large amount of non-autonomous decision data being mixed into the training samples, leading to instability in the policy learning process.

[0005] For the reasons mentioned above, existing technologies are prone to the following problems: on the one hand, the page frame rate frequently jumps between different levels, resulting in uneven page rendering and increased perceptible stuttering and latency for users; on the other hand, reinforcement learning models have difficulty forming stable and effective strategies in the critical region of interaction intensity, which is not conducive to ensuring user interaction experience, nor to reducing rendering load and energy consumption. Summary of the Invention

[0006] To at least partially overcome the problem in related technologies of difficulty in simultaneously ensuring smooth interaction, smooth frame rate transition, and stable agent training in high-frequency interactive web pages, this application provides a method for real-time control of the rendering frame rate of web pages based on agents.

[0007] The proposed solution is as follows:

[0008] A method for real-time control of the rendering frame rate of a web page based on an intelligent agent includes: Monitor the current interaction intensity of users on web pages; When the current interaction intensity value reaches a preset threshold, the user interaction event stream and page rendering pipeline state data are captured as raw data. The raw data is then processed by feature extraction and normalization through a sliding time window to generate a low-dimensional continuous state vector. The low-dimensional continuous state vector includes at least the following features: interaction intensity features, frame time fluctuation features, device performance margin features, and mask edge distance features. The low-dimensional continuous state vector is input into the policy network of the reinforcement learning agent, and the original Q-value set corresponding to multiple preset discrete frame rate levels is calculated and output; the reinforcement learning agent is deployed in the main thread of the web page or in the web worker in the browser environment; the discrete frame rate levels include at least the full frame rate level, the balanced frame rate level, and the energy-saving frame rate level; Determine whether the reinforcement learning agent is currently in the exploration or utilization phase, and determine the original candidate actions from the original Q-value set based on the current phase of the reinforcement learning agent and the preset exploration strategy. Calculate the softening coefficient for each discrete frame rate level based on the current interaction intensity value, perform compliance mapping on the original candidate actions according to the softening coefficient, and output the compliance actions. The rendering scheduling parameters corresponding to the compliance action are analyzed, and the Web page is adjusted according to the rendering scheduling parameters. The actual rendering frame rate and user interaction response time of the adjusted page are then output.

[0009] Preferably, the method further includes: When the compliance action is inconsistent with the original candidate action, a constraint tag is generated and recorded in the exploration intent cache. The constraint markers, along with the adjusted actual page rendering frame rate and user interaction response time, are input into the reward function to calculate the instant reward value. The low-dimensional continuous state vector, the original Q-value set, the compliance action, the constraint label, and the immediate reward value are combined into an experience sample and stored in the experience replay pool; for experience samples with a constraint label that is true, a decay weight is assigned during sampling and replay. Experience samples are sampled from the experience replay pool and input into the reinforcement learning agent. The loss function is calculated and the network parameters of the policy network are updated by combining the difference information between the original candidate actions and the compliance actions recorded in the exploration intent cache.

[0010] Preferably, the reward function includes a frame rate maintenance reward, an energy consumption penalty, an interaction latency penalty, and a constraint cost term; The frame rate maintenance bonus is calculated from the frame rate value of the target frame rate level corresponding to the compliance action, the actual rendering frame rate of the page, and the frame rate bonus coefficient. The energy consumption penalty is calculated from the frame rate value of the target frame rate tier corresponding to the compliance action, the frame rate value of the full frame rate tier, the normalized value of the device's current power consumption level, and the energy consumption penalty coefficient. The interaction delay penalty is calculated based on the user interaction response time, a preset delay tolerance threshold, and a delay penalty coefficient. The constraint cost term is calculated from the indicator function of the constraint label and the constraint cost coefficient; wherein, the indicator function of the constraint label takes a value of 1 when the constraint label is true, and 0 otherwise; The method further includes: The constraint cost term is dynamically adjusted based on the degree of difference between the original candidate action and the compliance action, including: Obtain the original gear number corresponding to the original candidate action and the compliance gear number corresponding to the compliance action, and calculate the gear difference value based on the gear number; The constraint cost term is dynamically adjusted based on the gear difference value, so that the larger the gear difference value, the larger the absolute value of the negative reward of the constraint cost term.

[0011] Preferably, the method further includes: A weighting factor is maintained for each experience sample through the experience replay pool; For empirical samples where the constraints are marked as true, the current weight factor is calculated using the empirical replay pool based on the current training step count, total training steps, initial weights, and final weights; where the initial weight is less than the final weight. For empirical samples whose constraints are marked as false, their weight factor is set to 1; When sampling experience samples from the experience replay pool, priority sampling is performed according to a weight factor, and the probability of each experience sample being sampled is proportional to the weight factor.

[0012] Preferably, the method further includes: Obtain the interaction intensity value, the preset low-bit mask trigger threshold, and the high-bit mask full activation threshold within the current sliding time window, and calculate the mask edge distance feature based on these values. The mask edge distance feature is used to characterize the relative position of the current interaction intensity value within the mask transition interval; a positive value of the mask edge distance feature indicates that the mask transition interval has been entered, while a negative value indicates that the mask transition interval has not yet been entered; the absolute value of the mask edge distance feature represents the distance from the boundary of the transition interval.

[0013] Preferably, the rendering scheduling parameters corresponding to the compliance action are parsed, the Web page is adjusted according to the rendering scheduling parameters, and the actual rendering frame rate and user interaction response time of the adjusted page are output, including: The compliance action is parsed, and the target frame interval corresponding to the compliance action is obtained as a rendering scheduling parameter; Get the timestamp of the current request animation frame callback function and the timestamp of the last callback trigger, and calculate the time difference based on the timestamps; When the target frame interval is greater than the time difference, the browser's timer interface is invoked to set a delay, and the current rendering task is executed after the delay ends; when the target frame interval is less than or equal to the time difference, the current rendering task is executed immediately. Call the browser's compositor thread priority setting interface to adjust the compositor thread's wake-up priority to a value inversely proportional to the target frame interval, thereby adjusting the rendering task submission density per unit time. The output shows the actual rendering frame rate and user interaction response time of the page after adjustment, based on the results of the rendering task.

[0014] Preferably, the strategy network includes an online network and a target network; The online network is used to receive the low-dimensional continuous state vector and output the original Q-value set; The target network is used to calculate the target Q value during the policy network parameter update process; The online network and the target network have the same network structure, both including multiple fully connected hidden layers, and the number of neurons in the output layer is equal to the number of preset discrete frame rate levels; The parameters of the target network are synchronized with the parameters of the online network using a soft update method.

[0015] Preferably, the loss function is calculated and the network parameters of the policy network are updated by combining the difference information between the original candidate actions and the compliance actions recorded in the exploration intent cache, including: Read multiple storage units in batches from the exploration intent cache; For each of the aforementioned storage units, the current state vector stored therein is input into the policy network of the reinforcement learning agent, the difference between the predicted Q value and the target Q value corresponding to the original candidate action is calculated, and an auxiliary loss term is generated. The weighted auxiliary loss term is added to the main loss term corresponding to the experience sample obtained from the experience replay pool to form the total loss function; The network parameters of the policy network are updated based on the total loss function.

[0016] Preferably, the exploration intent cache is stored in a circular queue structure; each storage unit in the exploration intent cache includes a current state vector field, an original candidate action field, a compliance action field, and a timestamp field. The method further includes: When an inconsistency is detected between the original candidate action and the compliant action, the current low-dimensional continuous state vector, the original candidate action, the compliant action, and the current timestamp are written to the tail of the exploration intent buffer. When the exploration intent cache is full, overwrite the storage unit at the head of the queue.

[0017] Preferably, a softening coefficient is calculated for each discrete frame rate tier based on the current interaction intensity value. A compliance mapping is then performed on the original candidate actions according to the softening coefficient, and a compliance action is output, including: When the interaction intensity value is lower than the low-bit mask trigger threshold, the softening coefficient of each discrete frame rate level is set to the initial coefficient that allows the original candidate action to pass directly. When the interaction intensity value is between the low-bit mask trigger threshold and the high-bit mask fully effective threshold, the continuously changing softening coefficient corresponding to each discrete frame rate level is calculated according to the relative position of the interaction intensity value in the preset transition interval and the level priority of each discrete frame rate level, so that the probability of the low-priority discrete frame rate level being selected gradually decreases as the interaction intensity increases. When the interaction strength value is higher than the threshold for the high-bit mask to be fully effective, a restriction is imposed on discrete frame rate levels that are lower than the preset level threshold. Based on the softening coefficient of the discrete frame rate level corresponding to the original candidate action, a retention determination or replacement mapping is performed on the original candidate action, and a compliant action is output.

[0018] The technical solution provided in this application may include the following beneficial effects: This technical solution monitors the current interaction intensity of users on a web page. When the interaction intensity reaches a preset threshold, it generates a low-dimensional continuous state vector by combining user interaction event streams and page rendering pipeline state data. This low-dimensional continuous state vector is then input into the policy network of the reinforcement learning agent to achieve adaptive decision-making for multiple discrete frame rate levels. Simultaneously, based on the continuous value of the interaction intensity and the priority of each discrete frame rate level, a softening coefficient is calculated. After performing a retention or substitution mapping on the original candidate actions, a compliant action is output. This allows the frame rate adjustment process to move away from a rigid switching method under a fixed threshold and achieve a gradual and smooth transition of frame rate levels under different interaction intensities. Furthermore, by parsing the rendering scheduling parameters corresponding to the compliant actions and adjusting the rendering process of the web page accordingly, it is possible to reduce unnecessary rendering load and resource consumption while ensuring user interaction response speed and page smoothness. This, in turn, improves the rendering stability, adaptive adjustment capability, and overall performance of the web page in high-frequency interaction scenarios.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a flowchart illustrating a method for real-time frame rate control of web page rendering based on an intelligent agent, according to an embodiment of this application. Figure 2 This is a schematic diagram of a process for updating a reinforcement learning agent according to an embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] Example 1 Figure 1 This is a flowchart illustrating a method for real-time frame rate control of web page rendering based on an intelligent agent, according to an embodiment of this application. (Refer to...) Figure 1 A method for real-time control of the rendering frame rate of a web page based on an intelligent agent, comprising: S11. Monitor the current interaction intensity value of the user on the web page; S12. When the current interaction intensity value reaches a preset threshold, capture the user interaction event stream and page rendering pipeline state data as raw data, and perform feature extraction and normalization on the raw data through a sliding time window to generate a low-dimensional continuous state vector; the low-dimensional continuous state vector includes at least: interaction intensity features, frame time fluctuation features, device performance margin features, and mask edge distance features. S13. Input the low-dimensional continuous state vector into the policy network of the reinforcement learning agent, calculate and output the original Q-value set corresponding to multiple preset discrete frame rate levels; the reinforcement learning agent is deployed in the main thread of the web page or in the web worker in the browser environment; the discrete frame rate levels include at least the full frame rate level, the balanced frame rate level, and the energy-saving frame rate level; S14. Determine whether the reinforcement learning agent is currently in the exploration or utilization phase, and determine the original candidate actions from the original Q-value set based on the current phase of the reinforcement learning agent and the preset exploration strategy. S15. Calculate the softening coefficient for each discrete frame rate level based on the current interaction intensity value, perform compliance mapping on the original candidate actions according to the softening coefficient, and output the compliance actions. S16. Analyze the rendering scheduling parameters corresponding to the compliance actions, adjust the Web page according to the rendering scheduling parameters, and output the actual rendering frame rate and user interaction response time of the adjusted page.

[0024] For ease of understanding, the following explains some key terms in this embodiment: Interaction intensity value: This indicates the level of user activity on a web page, such as the frequency and density of actions like clicking, dragging, scrolling, zooming, and typing. This value is used to quantify users' needs for page responsiveness and smoothness.

[0025] Sliding time window: A data processing mechanism that defines a fixed-length time interval on a continuous data stream and slides the interval forward over time to process and analyze the data within the window in real time.

[0026] Low-dimensional continuous state vector: A vector composed of multiple feature values ​​of the web page's running state and user interaction behavior, used to represent the overall running state of the current web page. This vector has low dimensionality, making it easier for reinforcement learning agents to process. Specifically, the interaction intensity feature reflects the activity level of user interaction; the frame time fluctuation feature reflects the stable state of page rendering; the device performance margin feature reflects the currently available computing resources on the device; and the mask edge distance feature reflects the relative relationship between the interaction intensity and a preset threshold.

[0027] Reinforcement learning agent: A decision-making entity based on reinforcement learning algorithms that learns optimal strategies through interaction with the environment. In this method, the agent is deployed in the main thread of a web page or in a Web Worker within a browser environment, and is responsible for outputting frame rate adjustment actions based on the current page state.

[0028] Policy network: A neural network model in a reinforcement learning agent that takes the current state vector as input and outputs the probability distribution or Q-value of each possible action to be performed in the current state.

[0029] The original Q-value set: calculated by the policy network based on the current low-dimensional continuous state vector, corresponding to the expected reward value for each preset discrete frame rate level. This set reflects the agent's tendency to choose different frame rate levels in the current state.

[0030] Discrete frame rate tiers: A limited number of pre-defined frame rate levels, such as full frame rate tiers (providing high rendering smoothness), balanced frame rate tiers (balancing smoothness and power consumption), and energy-saving frame rate tiers (prioritizing power consumption reduction).

[0031] Exploration Phase and Exploitation Phase: These are two modes that exist in the learning process of reinforcement learning agents. The exploration phase refers to the agent trying new actions to discover potential better strategies; the exploitation phase refers to the agent selecting actions based on the currently learned optimal strategies to maximize rewards.

[0032] Softening coefficient: A parameter used to adjust the original candidate actions, its value is calculated based on the current interaction intensity. This coefficient is used to provide flexible constraints on the agent's decision-making within the critical region of user interaction intensity, avoiding frequent frame rate jumps.

[0033] Compliance mapping: The process of adjusting the original candidate actions output by a reinforcement learning agent based on a softening coefficient. This process aims to ensure that the final adjusted actions meet the overall requirements of user interaction experience and system performance.

[0034] Rendering scheduling parameters: These are specific parameters used to control the rendering process of a web page, such as the target frame interval and the compositor thread wake-up priority. By adjusting these parameters, the actual rendering frame rate of the page can be controlled.

[0035] This embodiment provides a method for real-time control of the rendering frame rate of a web page based on an intelligent agent.

[0036] First, monitor the current interaction intensity of the user on the web page. This monitoring can be achieved in several ways. For example, the frequency of user input events (such as mouse clicks, keyboard input, touch events, etc.) can be periodically sampled and accumulated or averaged to obtain an instantaneous interaction intensity value.

[0037] Furthermore, when the detected current interaction intensity value reaches a preset threshold, the system captures user interaction event streams and page rendering pipeline state data as raw data. This raw data is then processed through a sliding time window to achieve feature extraction and normalization. For example, a fixed-length sliding time window can be set to collect all relevant interaction events and rendering metrics within the window, and then calculate the statistical characteristics of these data, such as mean, variance, maximum, and minimum values. After normalization, these features generate a low-dimensional continuous state vector. This low-dimensional continuous state vector includes at least interaction intensity features, frame latency fluctuation features, device performance margin features, and mask edge distance features. The interaction intensity feature can simply be the total number or average frequency of interaction events within the window; the frame latency fluctuation feature can be the standard deviation of the rendering time for each frame within the window; the device performance margin feature can be the current CPU or GPU idle rate; and the mask edge distance feature can be the difference between the current interaction intensity value and a preset mask threshold.

[0038] Subsequently, the generated low-dimensional continuous state vector is input into the policy network of the reinforcement learning agent. After receiving the state vector, the policy network calculates and outputs a set of raw Q-values ​​corresponding to multiple preset discrete frame rate levels. This reinforcement learning agent can be deployed in the main thread of a web page in a browser environment to directly interact with the page rendering logic; alternatively, to avoid blocking the main thread, the agent can be deployed in a Web Worker to communicate with the main thread via message passing. The preset discrete frame rate levels can include full frame rate (e.g., 60 FPS), balanced frame rate (e.g., 30 FPS), and energy-saving frame rate (e.g., 15 FPS).

[0039] Based on this, the system determines whether the reinforcement learning agent is currently in the exploration or exploitation phase. In the exploration phase, the agent tends to try different actions to gather more environmental information; for example, it can randomly select an action from the original Q-value set or select a suboptimal action with a certain probability. In the exploitation phase, the agent tends to select the action currently considered the best. Based on the agent's current phase and the preset exploration strategy, initial candidate actions are determined from the original Q-value set.

[0040] Furthermore, a softening coefficient is calculated for each discrete frame rate tier based on the current interaction intensity value. This softening coefficient can be calculated using linear interpolation. For example, when the interaction intensity value is below a certain low-threshold, the softening coefficient is set to an initial value that allows all original candidate actions to pass; when the interaction intensity value is above a certain high-threshold, the softening coefficient is set to a value that imposes strict restrictions on low frame rate tiers; when the interaction intensity value is between the two thresholds, the softening coefficient changes linearly with the interaction intensity value. Based on this softening coefficient, compliance mapping is performed on the original candidate actions, and the compliance actions are output. For example, if the frame rate tier corresponding to an original candidate action is restricted by the softening coefficient under the current interaction intensity, it is mapped to an allowed high frame rate tier.

[0041] Finally, the rendering scheduling parameters corresponding to the compliance action are analyzed. This parameter can be a target frame interval; for example, if the compliance action is set to 30 FPS, the target frame interval is 33.3 ms. The web page is then adjusted based on this parameter. For example, the page rendering frame rate can be influenced by controlling the timing of rendering task submissions. After adjustment, the actual page rendering frame rate and user interaction response time are output. For example, the time from user input to screen update completion can be recorded as the user interaction response time.

[0042] This technical solution monitors the current interaction intensity of users on a web page. When the interaction intensity reaches a preset threshold, it generates a low-dimensional continuous state vector by combining user interaction event streams and page rendering pipeline state data. This low-dimensional continuous state vector is then input into the policy network of the reinforcement learning agent to achieve adaptive decision-making for multiple discrete frame rate levels. Simultaneously, based on the continuous value of the interaction intensity and the priority of each discrete frame rate level, a softening coefficient is calculated. After performing a retention or substitution mapping on the original candidate actions, a compliant action is output. This allows the frame rate adjustment process to move away from a rigid switching method under a fixed threshold and achieve a gradual and smooth transition of frame rate levels under different interaction intensities. Furthermore, by parsing the rendering scheduling parameters corresponding to the compliant actions and adjusting the rendering process of the web page accordingly, it is possible to reduce unnecessary rendering load and resource consumption while ensuring user interaction response speed and page smoothness. This, in turn, improves the rendering stability, adaptive adjustment capability, and overall performance of the web page in high-frequency interaction scenarios.

[0043] It should be noted that the calculation process for the mask edge distance feature includes: Obtain the interaction intensity value, preset low-bit mask trigger threshold, and high-bit mask full activation threshold within the current sliding time window, and calculate the mask edge distance feature based on these values. The mask edge distance feature is used to characterize the relative position of the current interaction intensity value within the mask transition zone; a positive value of the mask edge distance feature indicates that the mask transition zone has been entered, while a negative value indicates that the mask transition zone has not yet been entered; the absolute value of the mask edge distance feature represents the distance from the boundary of the transition zone.

[0044] The low-bit mask trigger threshold is a preset interaction intensity value that defines the starting point at which the system needs to begin paying attention to and potentially adjusting the frame rate when the user's interaction intensity starts to increase from a low level. This threshold is typically determined based on user experience research or experimental data to ensure that the system can respond promptly when the user begins to perceive changes in interaction activity.

[0045] The high-level masking full effectiveness threshold is another preset interaction intensity value. It defines the endpoint at which the masking mechanism should fully function when the user interaction intensity reaches a high level. When the interaction intensity exceeds this threshold, it usually means that the user is engaging in highly active interaction, and the system should prioritize ensuring a smooth user experience.

[0046] The calculation of the mask edge distance feature is based on the interaction intensity value, low-bit mask trigger threshold, and high-bit mask full effectiveness threshold within the current sliding time window.

[0047] The specific calculation formula is as follows: ; in, Indicates the distance feature of the mask edges; This represents the interaction intensity value within the current sliding time window; Indicates the low-order mask trigger threshold; This indicates the threshold at which the high-order bit mask is fully effective; .

[0048] The mask edge distance feature quantifies the relative position and distance between the current interaction intensity value and these two key thresholds. The mask transition range refers to the range of interaction intensity defined by the low-bit mask trigger threshold and the high-bit mask full-effect threshold. Within this range, the system's frame rate control strategy may need to smoothly transition to balance performance and user experience. The mask edge distance feature, through its numerical magnitude and sign, precisely indicates the specific position of the current interaction intensity value within this range, such as whether it has just entered the transition zone, is in the middle of the transition zone, or is near the end of the transition zone.

[0049] When the mask edge distance feature is negative, it indicates that the current interaction strength value is below the low-bit mask trigger threshold, meaning the user interaction activity is low and has not yet reached a level requiring special system attention; therefore, it is considered not yet in the mask transition zone. In this case, the larger the absolute value, the farther the interaction strength is from the low-bit mask trigger threshold, and the system can be more inclined towards energy-saving mode. When the mask edge distance feature is positive, it indicates that the current interaction strength value has reached or exceeded the low-bit mask trigger threshold, meaning the user interaction activity has entered or exceeded the mask transition zone. In this case, its absolute value represents the distance between the interaction strength value and the nearest transition zone boundary (i.e., the low-bit mask trigger threshold or the high-bit mask full-effectiveness threshold).

[0050] Preferably, in addition to using the linear normalization method, the calculation method of the mask edge distance feature can also be replaced by higher-order statistical features based on the rate of change of interaction intensity or interaction intention prediction features based on the hidden Markov model, so that the agent can predict the upcoming changes in the mask state.

[0051] It should be noted that the parsing process analyzes the rendering scheduling parameters corresponding to compliance actions, adjusts the web page based on these parameters, and outputs the actual rendering frame rate and user interaction response time of the adjusted page, including: Analyze compliance actions and obtain the target frame interval corresponding to the compliance actions as rendering scheduling parameters; Get the timestamp of the current request animation frame callback function and the timestamp of the last callback trigger, and calculate the time difference based on the timestamps; When the target frame interval is greater than the time difference, the browser's timer interface is called to set a delay, and the current rendering task is executed after the delay ends; when the target frame interval is less than or equal to the time difference, the current rendering task is executed immediately. Call the browser's compositor thread priority setting interface to adjust the compositor thread's wake-up priority to a value inversely proportional to the target frame interval, thereby adjusting the rendering task submission density per unit time. The output shows the actual rendering frame rate and user interaction response time of the page after adjustment, based on the results of the rendering task.

[0052] Specifically, upon receiving a compliance action from the reinforcement learning agent, the system first parses it to extract the desired rendering target represented by the action. The rendering scheduling parameter here is specifically defined as the target frame interval, i.e., the expected time required to render each frame. For example, if the compliance action indicates a target frame rate of 60 frames per second, then the target frame interval is approximately 16.67 milliseconds. This parameter forms the basis for all subsequent rendering control operations.

[0053] To achieve precise rendering scheduling, the system obtains the timestamp of the current requested animation frame callback function and the timestamp of the last callback trigger. Requesting an animation frame is a mechanism provided by the browser to optimize animation rendering; it executes a specified callback function before the browser's next repaint. By recording the timestamp of each callback, the actual time difference between two renderings can be accurately calculated.

[0054] After obtaining the target frame interval and the actual time difference, the system compares and judges them. If the target frame interval is greater than the currently calculated time difference, it means that the browser's current rendering speed is faster than the expected target frame rate. In order to achieve the target frame rate and save system resources, the system calls the browser's timer interface to set an appropriate delay. This delay is usually the difference between the target frame interval and the time difference, ensuring that the timing of the current rendering task execution meets the target frame interval requirement exactly after the delay ends. Conversely, if the target frame interval is less than or equal to the current time difference, it indicates that the browser's current rendering speed has reached or fallen below expectations. In this case, the current rendering task should be executed immediately to avoid further delays and ensure page responsiveness.

[0055] Furthermore, to further optimize rendering performance, this application also involves calling the browser's compositor thread priority setting interface. The compositor thread is a crucial component of the browser's rendering pipeline, responsible for compositing the various layers of the page into the final image and submitting it to the GPU. By adjusting the compositor thread's wake-up priority, the density of rendering tasks processed and submitted by the browser per unit time can be affected. Specifically, the compositor thread's wake-up priority is adjusted to a value inversely proportional to the target frame interval. This means that when the target frame interval is short (i.e., the expected frame rate is high), the compositor thread's priority is increased to ensure that rendering tasks can be processed and submitted more promptly; conversely, when the target frame interval is long (i.e., the expected frame rate is low), the priority is correspondingly decreased, thereby reducing unnecessary resource consumption.

[0056] Finally, after the rendering task is completed, the system will output the adjusted page rendering frame rate and user interaction response time based on the actual execution of the rendering task. This output data not only directly reflects the effect of this adjustment, but also provides important feedback information for the subsequent learning and decision-making of the reinforcement learning agent.

[0057] Preferably, in addition to using timer delay compensation, the throttling mechanism for requesting animation frame callbacks can also be replaced by using the browser's requestIdleCallback interface to reduce the rendering frequency during idle periods, or by combining the CSS animation's will-change property with the compositor thread's task queue for active clearing to adjust the frame rate.

[0058] Example 2 In some of the above implementations, although the frame rate of web page rendering is controlled by a reinforcement learning agent combined with a softening coefficient, the agent may generate some initial candidate actions that do not fully conform to the current user experience or device state during the exploration process. After these initial candidate actions are mapped to compliance, the actual actions executed may differ from the agent's initial intent. If this difference is not effectively handled, the agent may receive ambiguous or inaccurate reward signals during the learning process, thereby affecting the convergence speed and final performance of its policy network, making it difficult to efficiently learn the optimal frame rate control strategy.

[0059] To address the aforementioned issues, this application further proposes a technical solution for updating reinforcement learning agents, referring to... Figure 2 The methods also include: S21. When the compliance action is inconsistent with the original candidate action, a constraint mark is generated and recorded in the exploration intent cache. S22. Input the constraint markers, the actual rendering frame rate of the page after adjustment, and the user interaction response time into the reward function to calculate the instant reward value; S23. Combine the low-dimensional continuous state vector, the original Q-value set, the compliance action, the constraint label, and the immediate reward value into an experience sample and store it in the experience replay pool; for experience samples with a constraint label that is true, assign a decay weight during sampling and replay. S24. Sample experience samples from the experience replay pool, input the experience samples into the reinforcement learning agent, combine the difference information between the original candidate actions and the compliant actions recorded in the exploration intent buffer, calculate the loss function and update the network parameters of the policy network.

[0060] When the compliant action differs from the original candidate action, a constraint tag is generated and recorded in the exploration intent cache. Specifically, after the reinforcement learning agent determines the original candidate action, it calculates the softening coefficient for each discrete frame rate level based on the current interaction intensity value, and performs a compliance mapping on the original candidate action according to the softening coefficient, thereby outputting the compliant action. At this time, the system compares whether the compliant action is consistent with the original candidate action. If there is a difference, it indicates that the agent's original intent has been corrected by the compliance mechanism, and a constraint tag is generated. This constraint tag is usually a Boolean value (e.g., true or false) to indicate whether compliance has occurred. At the same time, this constraint tag, along with other relevant information (e.g., the current low-dimensional continuous state vector, the original candidate action, the compliant action, etc.), is recorded and stored in the exploration intent cache. The exploration intent cache aims to record detailed information about the deviation between the agent's "intent" and "actual execution" during the exploration process, providing key data for subsequent policy optimization.

[0061] Subsequently, the constraint flag, along with the adjusted actual page rendering frame rate and user interaction response time, are input into the reward function to calculate the immediate reward value. The reward function is a core component in reinforcement learning used to evaluate the quality of an agent's behavior. By using the constraint flag as one of the inputs to the reward function, the agent, upon receiving the reward signal, can not only evaluate performance metrics such as page rendering frame rate and user interaction response time resulting from the actual compliant actions, but also perceive the "cost" or "impact" of its original intent being corrected. For example, when the constraint flag is true, the reward function can apply an additional negative reward (penalty) to encourage the agent to choose actions that satisfy the constraints without significant compliance in future decisions, thereby guiding the agent to learn more "compliant" strategies.

[0062] Next, the low-dimensional continuous state vector, the original Q-value set, the compliance action, the constraint label, and the immediate reward value are combined into experience samples and stored in the experience replay pool. The experience replay pool is a data structure used in reinforcement learning to store the agent's experience of interacting with the environment. Each experience sample is a tuple containing all relevant information about the agent taking an action in a certain state and obtaining a reward. In this application, the experience samples specifically include the original Q-value set (the Q-values ​​output by the agent's policy network for all discrete frame rate levels in the current state), the actually executed compliance action, and the constraint label. This rich contextual information helps the agent to more comprehensively understand its decision-making process and its consequences. For experience samples with a constraint label that is true, a decay weight is assigned during sampling replay. This means that when randomly sampling samples from the experience replay pool for training the policy network, the probability of sampling or the weight in the loss calculation of experience samples whose original intention was modified due to compliance will be reduced. This move aims to reduce the interference of these "suboptimal" or "corrected" experiences on the agent's learning process, prevent the agent from overlearning behaviors forced by compliance mechanisms, and thus enable the agent to focus more on learning actions that are truly chosen by its strategy and bring good results.

[0063] Finally, experience samples are sampled from the experience replay pool and input into the reinforcement learning agent. Combined with the difference information between the original candidate actions and compliance actions recorded in the exploration intent cache, a loss function is calculated and the network parameters of the policy network are updated. During the training phase of the policy network, experience samples are sampled in batches from the experience replay pool. These samples are used to calculate the main loss function, for example, based on temporal difference (TD) error. Simultaneously, to further utilize the difference information between the original candidate actions and compliance actions recorded in the exploration intent cache, an auxiliary loss term can be introduced. This auxiliary loss term measures the deviation between the agent's original intent and the actual executed action, and its weighted sum is added to the main loss function to form the total loss function. By minimizing this total loss function, the parameters of the policy network are updated. This combination allows the agent to learn not only from the actually executed compliance actions but also from the difference between its original intent and the actual execution, thereby gaining a more comprehensive understanding of the impact of the compliance mechanism and adjusting its policy to generate original candidate actions that better conform to actual constraints and obtain higher rewards.

[0064] It should be noted that the reward function includes frame rate maintenance reward, energy consumption penalty, interaction latency penalty, and constraint cost term; The frame rate maintenance bonus is calculated from the frame rate value of the target frame rate level corresponding to the compliance action, the actual rendering frame rate of the page, and the frame rate bonus coefficient. The energy consumption penalty is calculated from the frame rate value of the target frame rate tier corresponding to the compliance action, the frame rate value of the full frame rate tier, the normalized value of the device's current power consumption level, and the energy consumption penalty coefficient. The interaction delay penalty is calculated based on the user interaction response time, the preset delay tolerance threshold, and the delay penalty coefficient; The constraint cost term is calculated from the indicator function of the constraint label and the constraint cost coefficient; wherein, the indicator function of the constraint label takes a value of 1 when the constraint label is true, and 0 otherwise; The method also includes: The constraint cost terms are dynamically adjusted based on the degree of difference between the original candidate actions and the compliance actions, including: Obtain the original gear number corresponding to the original candidate action and the compliance gear number corresponding to the compliance action, and calculate the gear difference value based on the gear number; The constraint cost term is dynamically adjusted based on the gear difference value, so that the larger the gear difference value, the larger the absolute value of the negative reward of the constraint cost term.

[0065] Specifically, the reward function is as follows: ; in, This indicates a frame rate maintenance bonus. Indicates energy consumption penalty. Indicates interaction delay penalty, Represents the constraint cost term; The formula for calculating the frame rate maintenance bonus is as follows: ; in, This indicates the frame rate value corresponding to the target frame rate tier for the compliance action; Indicates the actual frame rate of page rendering; This represents the frame rate bonus coefficient; The formula for calculating energy consumption penalty is as follows: ; in, This indicates the frame rate value for the full frame rate tier. This represents the normalized value of the device's current power consumption level. This represents the energy consumption penalty coefficient; The formula for calculating the interaction delay penalty is as follows: ; in, Indicates the user interaction response time; This indicates the preset latency tolerance threshold; Indicates the delay penalty coefficient; The formula for calculating the constraint cost term is as follows: ; in, Indicator functions representing constraint tags; This represents the constraint cost coefficient.

[0066] The subsequent step is to obtain the original gear number corresponding to the original candidate action. Compliance level number corresponding to compliance actions Calculate the gear difference value based on the gear number. Dynamically adjust constraint cost items based on gear difference values. The greater the difference in gear level, the greater the absolute value of the negative reward of the constraint cost term.

[0067] Preferably, in addition to adopting a linear penalty method that is proportional to the difference in gear level, the constraint cost term can also be replaced with an exponential growth penalty or a step-by-step penalty, so that the agent bears a greater cost gradient when it deviates more from the compliant action.

[0068] In this embodiment, the method further includes: A weighting factor is maintained for each experience sample through the experience replay pool; For empirical samples where the constraints are marked as true, the current weight factor is calculated using the empirical replay pool based on the current training step count, total training steps, initial weights, and final weights; where the initial weight is less than the final weight. For empirical samples whose constraints are marked as false, their weight factor is set to 1; When sampling experience samples from the experience replay pool, priority sampling is performed according to a weight factor, and the probability of each experience sample being sampled is proportional to the weight factor.

[0069] The experience replay pool is a memory structure used in reinforcement learning to store experience samples generated by the agent's interactions with the environment. Maintaining a weight factor for each experience sample means that, in addition to storing basic information such as state, action, and reward, the experience replay pool also stores a numerical value representing the importance or priority of that experience sample in subsequent training. This mechanism allows the system to adjust the influence of experience samples during training based on their specific attributes, thereby optimizing the learning process.

[0070] When the constraint label of an experience sample is true, it indicates that the agent's initial action has been corrected by the system to comply with regulations. This usually means that the agent made a suboptimal or non-compliant decision. To enable the agent to learn more effectively from these "errors," these samples need to be given higher learning priority. When calculating the current weight factor, the training progress (current training steps vs. total training steps) is considered, combined with preset initial and final weights. The initial weight is less than the final weight, meaning that in the early stages of training, even constrained samples may have relatively low weights to allow the agent some degree of exploration. As training progresses, the weights of these constrained samples gradually increase, prompting the agent to pay more attention to these key experiences, thereby accelerating learning to avoid non-compliant behavior. This dynamic adjustment mechanism helps balance exploration and exploitation and gradually guides the agent to learn safer strategies.

[0071] When the constraint of an empirical sample is marked as false, it means that the original candidate action chosen by the agent is consistent with the compliance action executed by the system, i.e., the agent's decision is compliant. For such samples, their weight factor is set to 1, which means that they have a baseline sampling probability during experience replay. This ensures that the agent can learn from all compliant interactions, maintain its understanding of normal and effective behavior, and highlight the importance of constraint samples by comparing them with other weight factors.

[0072] Priority sampling is an experience replay strategy that determines the probability of sampling an experience sample based on its weight factor. When drawing a batch of experience samples from the experience replay pool for training, samples with higher weight factors have a greater chance of being selected. This mechanism ensures that experiences deemed more important and informative (e.g., samples with constraint labels) participate more frequently in the policy network updates, thereby accelerating the agent's learning of key behavioral patterns and improving learning efficiency and policy convergence speed. By making the sampling probability proportional to the weight factor, the system can effectively guide the agent to focus on experiences that have the greatest impact on its learning performance.

[0073] By maintaining a weight factor for each experience sample in the experience replay pool and dynamically adjusting these weights based on the constraint label state, this application significantly optimizes the training process of reinforcement learning agents. Specifically, for experience samples whose original decisions are corrected by system compliance (i.e., the constraint label is true), since they contain the "error" information that the agent needs to focus on learning, their weight factors are dynamically increased based on the number of training steps, making these key samples more likely to be sampled during experience replay. This prompts the agent to encounter and learn how to avoid non-compliant actions more frequently, thereby accelerating the convergence of its policy network towards a safer direction that better conforms to system constraints. Meanwhile, for compliant experience samples, their weight factors are set to a baseline value of 1, ensuring the agent's continuous learning of normal behavior. This priority sampling mechanism effectively solves the problem of low learning efficiency caused by treating all samples equally in traditional experience replay, enabling the agent to learn strategies that optimize performance and meet compliance requirements in web page rendering frame rate control more quickly, ultimately improving the stability of control and user experience.

[0074] The specific formula for calculating the current weight factor is as follows: ; in, Indicates the current weighting factor; Indicates the initial weights; represents the final weight; t represents the current training step number; T represents the total training steps.

[0075] Preferably, in addition to adopting a linear growth method with the number of training steps, the decay weight can also be replaced by an adaptive decay mechanism based on the cumulative frequency of the constrained samples, or a dynamic weight adjustment method based on the output entropy of the policy network.

[0076] It should be noted that the policy network includes both the online network and the target network; Online networks are used to receive low-dimensional continuous state vectors and output the raw set of Q-values; The target network is used to calculate the target Q value during the policy network parameter update process; The online network and the target network have the same network structure, both including multiple fully connected hidden layers, and the number of neurons in the output layer is equal to the number of preset discrete frame rate levels; The parameters of the target network are synchronized with the parameters of the online network using a soft update method.

[0077] In practice, both the line network and the target network include two fully connected hidden layers. The first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons.

[0078] The default soft update factor is The target network is updated according to the following formula: ; in, These represent the parameters of the target network. Parameters representing online networks.

[0079] By introducing an online network and a target network, and employing a soft update mechanism, this application effectively solves the problem of unstable target Q-values ​​during reinforcement learning training. The online network is responsible for real-time decision-making and Q-value prediction, while the target network provides a relatively stable learning objective, thus decoupling the Q-value prediction and target calculation processes. This separation mechanism significantly reduces training oscillations, making parameter updates of the policy network more stable and efficient. Simultaneously, the parameters of the target network gradually converge to the parameters of the online network through soft updates, ensuring that the target Q-value continuously tracks the learning progress of the online network without introducing drastic changes. Therefore, the reinforcement learning agent can learn the optimal web page rendering frame rate control strategy more stably and effectively, improving the agent's decision-making accuracy and robustness in complex dynamic environments, ultimately resulting in more precise web page rendering frame rate control and a smoother user experience.

[0080] Preferably, the deep Q-network architecture in this technical solution can also be replaced with a dual deep Q-network architecture to alleviate the overestimation problem, or it can be replaced with a superior actor-critic architecture, which models the decision-making process of discrete frame rate levels as a boundary mapping of continuous action space and optimizes it through policy gradient.

[0081] It should be noted that, by combining the difference information between the original candidate actions and the compliance actions recorded in the exploration intent cache, the loss function is calculated and the network parameters of the policy network are updated, including: Batch read multiple storage units from the exploration intent cache; For each storage unit, the current state vector stored therein is input into the policy network of the reinforcement learning agent, the difference between the predicted Q value and the target Q value corresponding to the original candidate action is calculated, and an auxiliary loss term is generated. The auxiliary loss term is weighted and added to the main loss term corresponding to the empirical sample obtained from the empirical replay pool to form the total loss function; The network parameters of the policy network are updated based on the total loss function.

[0082] Through the above technical solutions, reinforcement learning agents can more comprehensively understand the actual impact of their decisions, thereby avoiding policy learning bias caused by compliance mapping. Specifically, the auxiliary loss term enables the policy network to predict whether original candidate actions will conflict with compliance rules when generating them, and adjusts its Q-value prediction in advance, allowing the agent to more accurately evaluate the value of its intended actions during the exploration phase. This not only improves the training efficiency and convergence speed of the policy network, but also enables the agent to learn optimization strategies that better conform to actual operational constraints, thereby achieving more precise rendering frame rate control while ensuring the user experience and system stability of web pages. This method effectively solves the problem of low learning efficiency of agents when original candidate actions are inconsistent with compliance actions, enabling agents to better adapt to complex web page rendering environments.

[0083] Preferably, the exploration intent cache is stored in a circular queue structure; each storage unit in the exploration intent cache contains a current state vector field, an original candidate action field, a compliance action field, and a timestamp field; Based on this, the method also includes: When an inconsistency is detected between the original candidate action and the compliant action, the current low-dimensional continuous state vector, the original candidate action, the compliant action, and the current timestamp are written to the tail of the exploration intent buffer. When the exploration intent cache is full, overwrite the storage unit at the head of the queue.

[0084] Specifically, the exploration intent cache is designed as a circular queue structure. A circular queue is an efficient data structure that utilizes a fixed-size storage space and achieves first-in, first-out (FIFO) data management through the cyclical movement of pointers at the head and tail of the queue. It also allows new data to overwrite the oldest data when the queue is full. This structure ensures that the exploration intent cache will not grow indefinitely and consume excessive system resources when storing a large amount of exploration intent information, making it particularly suitable for resource-constrained scenarios in browser environments.

[0085] Each storage unit in the exploration intent cache is precisely defined to include a current state vector field, an original candidate action field, a compliance action field, and a timestamp field. The current state vector field records the environmental state of the reinforcement learning agent when making a decision, which is crucial for the subsequent learning of the policy network as it provides contextual information for the decision. The original candidate action field records the initial action selected by the reinforcement learning agent based on its policy network and exploration policy, representing the agent's "intent." The compliance action field records the action actually executed after compliance mapping; it may differ from the original candidate action, and this difference is the focus of this application. The timestamp field records the specific time the exploration intent was stored, which helps in analyzing the timeliness of the data or performing time-based data filtering when needed.

[0086] When the system detects a discrepancy between the original candidate action generated by the reinforcement learning agent and the compliant action output after compliance mapping, it means that the agent's exploration intent has been corrected by external constraints or rules. At this point, the system combines the current low-dimensional continuous state vector, the original candidate action, the compliant action, and the current timestamp into a storage unit and writes it to the tail of the exploration intent cache. This conditional recording mechanism ensures that relevant information is only stored when the agent's exploration behavior is "corrected," making the data in the cache more targeted and valuable, and avoiding the storage of large amounts of redundant information.

[0087] Furthermore, to maintain a fixed size and data freshness in the exploration intent buffer, when the buffer is full, a new storage unit overwrites the oldest storage unit at the head of the queue. This overwriting mechanism is an inherent characteristic of circular queues, ensuring that the buffer always contains the most recent event information where the agent's exploration intent has been corrected. In this way, policy network updates can be based on the most recent and relevant exploration intent differences, enabling the agent to adapt to the constraints of the current environment more quickly and optimize its policy.

[0088] Preferably, in addition to using a circular queue structure, the recording mechanism of the exploration intent buffer can also be replaced by a priority-based experience replay structure, which uses the degree of difference in the original actions of the constrained samples as the priority basis and directly integrates them into the main experience replay pool for unified management, rather than using an independent buffer structure.

[0089] Example 3 This embodiment provides a progressive softening mask mechanism. Specifically, it calculates a softening coefficient for each discrete frame rate level based on the current interaction intensity value, performs compliance mapping on the original candidate actions according to the softening coefficient, and outputs compliant actions, including: When the interaction strength value is lower than the low-bit mask trigger threshold, the softening coefficient of each discrete frame rate level is set to the initial coefficient that allows the original candidate action to pass directly. When the interaction strength value is between the low-bit mask trigger threshold and the high-bit mask fully effective threshold, the continuously changing softening coefficient corresponding to each discrete frame rate level is calculated according to the relative position of the interaction strength value in the preset transition interval and the level priority of each discrete frame rate level, so that the probability of the low-priority discrete frame rate level being selected gradually decreases as the interaction strength increases. When the interaction strength value is higher than the threshold for the high-bit mask to be fully effective, restrictions are imposed on discrete frame rate levels that are lower than the preset level threshold. Based on the softening coefficient of the discrete frame rate level corresponding to the original candidate action, the original candidate action is subject to retention judgment or replacement mapping, and the compliant action is output.

[0090] When the interaction intensity value is lower than the low-bit mask trigger threshold, the softening coefficient of each discrete frame rate level is set to an initial coefficient that allows the original candidate action to pass directly. This means that when the user's interaction intensity on the web page is low and has not yet reached the preset low-bit mask trigger threshold, it indicates that the user is currently in a light browsing or inactive state. In this case, the system tends to allow the reinforcement learning agent to freely choose the frame rate level according to its learned strategy to maximize energy saving or maintain smoothness. Therefore, the softening coefficient of all discrete frame rate levels is set to an initial value, which is usually represented as "allow direct passage", that is, no substantial intervention or restriction is made on the original candidate action output by the reinforcement learning agent, so that it can be directly used as a compliant action. This can be achieved by setting the softening coefficient to 1 or an equivalent unrestricted value, ensuring that the intent of the original candidate action is preserved.

[0091] When the interaction intensity value is between the low-bit mask trigger threshold and the high-bit mask full-effect threshold, the system calculates a continuously changing softening coefficient for each discrete frame rate level based on the relative position of the interaction intensity value within the preset transition interval and the priority of each discrete frame rate level. This ensures that the probability of selecting a low-priority discrete frame rate level gradually decreases as the interaction intensity increases. Specifically, when the user interaction intensity value enters the transition interval between the low-bit mask trigger threshold and the high-bit mask full-effect threshold, the system needs to perform more refined adjustments. At this stage, the interaction intensity gradually increases but has not yet reached a level requiring a full response. Therefore, the system dynamically calculates the corresponding softening coefficient based on the relative position of the current interaction intensity value within the transition interval, for example, through linear interpolation or nonlinear function mapping, and the preset priority of each discrete frame rate level, such as the power-saving level and the balanced level. For lower-priority frame rate levels, the softening coefficient gradually decreases as the interaction intensity increases, meaning that the probability of these low frame rate levels being selected gradually decreases. Conversely, the softening factor for high-priority frame rate tiers will increase or remain high to ensure that as interaction intensity increases, the system can gradually tend to select higher frame rate tiers, thereby smoothly improving the user experience.

[0092] When the interaction intensity value exceeds the threshold for full high-level mask activation, restrictions are imposed on discrete frame rate tiers below a preset threshold. This means that when the user interaction intensity value exceeds the threshold for full high-level mask activation, it indicates that the user is engaging in highly active interactions, such as rapid scrolling, frequent clicking, or complex animation operations. In this high-intensity interaction scenario, the system should prioritize ensuring the smoothness and responsiveness of the user experience. Therefore, for discrete frame rate tiers below the preset threshold, such as below the full frame rate tier or a certain minimum frame rate tier that guarantees smoothness, the system will impose strict restrictions. This restriction can be manifested by setting the softening coefficient of these low frame rate tiers to an extremely low value or even zero, thereby effectively preventing the reinforcement learning agent from selecting these low frame rate tiers that may cause stuttering or latency, ensuring that page rendering can meet the needs of high interaction intensity.

[0093] Finally, based on the softening coefficient of the discrete frame rate level corresponding to the original candidate action, a retention decision or substitution mapping is performed on the original candidate action, and a compliant action is output. After calculating the softening coefficient of each discrete frame rate level, the system queries the corresponding softening coefficient of the original candidate action output by the reinforcement learning agent based on the frame rate level. If the softening coefficient allows the original candidate action to pass directly, for example, if the softening coefficient is 1 or higher than a certain threshold, the original candidate action will be retained and directly output as a compliant action. If the softening coefficient indicates that the original candidate action is restricted, for example, if the softening coefficient is low or zero, the system will perform a substitution mapping. The implementation of the substitution mapping may include: forcibly mapping the original candidate action to a frame rate level with higher priority and a larger softening coefficient, or selecting an optimal substitution level from multiple allowed levels, to ensure that the final output compliant action can both respond to the decision intent of the reinforcement learning agent and meet the user experience and performance requirements under the current interaction intensity.

[0094] Softening coefficient The specific calculation formula is as follows: when hour, ; when hour, ; in, Indicates the highest priority sequence number; when When a priority level is lower than a preset threshold, a softening coefficient is set. For gears with priority numbers not lower than the preset gear threshold, set a softening coefficient. Then, the softening coefficient of the gear corresponding to the original candidate action is compared with a preset random number to determine whether to perform the mapping operation and output the compliance action.

[0095] Through the above technical solution, this application provides a refined frame rate control mechanism, effectively solving the problem that the original frame rate decision output by the reinforcement learning agent may lead to inconsistent experience or wasted resources when the intensity of user interaction changes. Specifically, under low interaction intensity, the system allows the agent to freely explore the optimal frame rate to achieve energy saving or maintain smoothness; when the interaction intensity enters the transition range, the system can smoothly guide the frame rate to a higher level through continuously changing softening coefficients, avoiding sudden changes in frame rate, thereby gradually improving responsiveness while ensuring user experience. Under high interaction intensity, the system forcibly restricts the selection of low frame rate levels, ensuring that page rendering always maintains high smoothness and meets the user's need for instant response. This mechanism of dynamically adjusting the softening coefficient based on interaction intensity and performing compliant mapping allows the reinforcement learning agent's decision-making to better adapt to actual user scenarios, avoiding the instability that may be caused by purely relying on model output, thereby achieving a dynamic balance between user experience and resource consumption in web page rendering, significantly improving the user's perceived smoothness and the system's energy efficiency.

[0096] Preferably, in addition to using a linear softening coefficient calculation method based on interaction strength value and gear priority number, the progressive softening masking mechanism can also be replaced by a nonlinear softening function based on S-curve, or by using a continuous probability distribution mapping method based on Gaussian kernel function, which directly samples and fuses the Q value of the original candidate action with the probability distribution modulated by the interaction strength value, thereby achieving a progressive transition of the masking effect rather than a binary hard switch.

[0097] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0098] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0099] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0100] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0103] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0104] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0105] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for real-time control of the rendering frame rate of a web page based on an intelligent agent, characterized in that, include: Monitor the current interaction intensity of users on web pages; When the current interaction intensity value reaches a preset threshold, the user interaction event stream and page rendering pipeline state data are captured as raw data. The raw data is then processed by feature extraction and normalization through a sliding time window to generate a low-dimensional continuous state vector. The low-dimensional continuous state vector includes at least the following features: interaction intensity features, frame time fluctuation features, device performance margin features, and mask edge distance features. The low-dimensional continuous state vector is input into the policy network of the reinforcement learning agent, and the original Q-value set corresponding to multiple preset discrete frame rate levels is calculated and output; the reinforcement learning agent is deployed in the main thread of the web page or in the web worker in the browser environment; the discrete frame rate levels include at least the full frame rate level, the balanced frame rate level, and the energy-saving frame rate level; Determine whether the reinforcement learning agent is currently in the exploration or utilization phase, and determine the original candidate actions from the original Q-value set based on the current phase of the reinforcement learning agent and the preset exploration strategy. Calculate the softening coefficient for each discrete frame rate level based on the current interaction intensity value, perform compliance mapping on the original candidate actions according to the softening coefficient, and output the compliance actions. The rendering scheduling parameters corresponding to the compliance action are analyzed, and the Web page is adjusted according to the rendering scheduling parameters. The actual rendering frame rate and user interaction response time of the adjusted page are then output.

2. The method according to claim 1, characterized in that, The method further includes: When the compliance action is inconsistent with the original candidate action, a constraint tag is generated and recorded in the exploration intent cache. The constraint markers, along with the adjusted actual page rendering frame rate and user interaction response time, are input into the reward function to calculate the instant reward value. The low-dimensional continuous state vector, the original Q-value set, the compliance action, the constraint label, and the immediate reward value are combined into an experience sample and stored in the experience replay pool; for experience samples with a constraint label that is true, a decay weight is assigned during sampling and replay. Experience samples are sampled from the experience replay pool and input into the reinforcement learning agent. The loss function is calculated and the network parameters of the policy network are updated by combining the difference information between the original candidate actions and the compliance actions recorded in the exploration intent cache.

3. The method according to claim 2, characterized in that, The reward function includes frame rate maintenance reward, energy consumption penalty, interaction latency penalty, and constraint cost term; The frame rate maintenance bonus is calculated from the frame rate value of the target frame rate level corresponding to the compliance action, the actual rendering frame rate of the page, and the frame rate bonus coefficient. The energy consumption penalty is calculated from the frame rate value of the target frame rate tier corresponding to the compliance action, the frame rate value of the full frame rate tier, the normalized value of the device's current power consumption level, and the energy consumption penalty coefficient. The interaction delay penalty is calculated based on the user interaction response time, a preset delay tolerance threshold, and a delay penalty coefficient. The constraint cost term is calculated from the indicator function of the constraint label and the constraint cost coefficient; wherein, the indicator function of the constraint label takes a value of 1 when the constraint label is true, and 0 otherwise; The method further includes: The constraint cost term is dynamically adjusted based on the degree of difference between the original candidate action and the compliance action, including: Obtain the original gear number corresponding to the original candidate action and the compliance gear number corresponding to the compliance action, and calculate the gear difference value based on the gear number; The constraint cost term is dynamically adjusted based on the gear difference value, so that the larger the gear difference value, the larger the absolute value of the negative reward of the constraint cost term.

4. The method according to claim 2, characterized in that, The method further includes: A weighting factor is maintained for each experience sample through the experience replay pool; For empirical samples where the constraints are marked as true, the current weight factor is calculated using the empirical replay pool based on the current training step count, total training steps, initial weights, and final weights; where the initial weight is less than the final weight. For empirical samples whose constraints are marked as false, their weight factor is set to 1; When sampling experience samples from the experience replay pool, priority sampling is performed according to a weight factor, and the probability of each experience sample being sampled is proportional to the weight factor.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the interaction intensity value, the preset low-bit mask trigger threshold, and the high-bit mask full activation threshold within the current sliding time window, and calculate the mask edge distance feature based on these values. The mask edge distance feature is used to characterize the relative position of the current interaction intensity value within the mask transition interval; a positive value of the mask edge distance feature indicates that the mask transition interval has been entered, while a negative value indicates that the mask transition interval has not yet been entered; the absolute value of the mask edge distance feature represents the distance from the boundary of the transition interval.

6. The method according to claim 1, characterized in that, The rendering scheduling parameters corresponding to the compliance action are analyzed, and the web page is adjusted according to the rendering scheduling parameters. The actual rendering frame rate and user interaction response time of the adjusted page are output, including: The compliance action is parsed, and the target frame interval corresponding to the compliance action is obtained as a rendering scheduling parameter; Get the timestamp of the current request animation frame callback function and the timestamp of the last callback trigger, and calculate the time difference based on the timestamps; When the target frame interval is greater than the time difference, the browser's timer interface is invoked to set a delay, and the current rendering task is executed after the delay ends; when the target frame interval is less than or equal to the time difference, the current rendering task is executed immediately. Call the browser's compositor thread priority setting interface to adjust the compositor thread's wake-up priority to a value inversely proportional to the target frame interval, thereby adjusting the rendering task submission density per unit time. The output shows the adjusted actual rendering frame rate and user interaction response time of the page based on the rendering task execution results.

7. The method according to claim 2, characterized in that, The strategy network includes an online network and a target network; The online network is used to receive the low-dimensional continuous state vector and output the original Q-value set; The target network is used to calculate the target Q value during the policy network parameter update process; The online network and the target network have the same network structure, both including multiple fully connected hidden layers, and the number of neurons in the output layer is equal to the number of preset discrete frame rate levels; The parameters of the target network are synchronized with the parameters of the online network using a soft update method.

8. The method according to claim 7, characterized in that, Combining the difference information between the original candidate actions and the compliance actions recorded in the exploration intent cache, the loss function is calculated and the network parameters of the policy network are updated, including: Read multiple storage units in batches from the exploration intent cache; For each of the aforementioned storage units, the current state vector stored therein is input into the policy network of the reinforcement learning agent, the difference between the predicted Q value and the target Q value corresponding to the original candidate action is calculated, and an auxiliary loss term is generated. The weighted auxiliary loss term is added to the main loss term corresponding to the experience sample obtained from the experience replay pool to form the total loss function; The network parameters of the policy network are updated based on the total loss function.

9. The method according to claim 2, characterized in that, The exploration intent cache is stored in a circular queue structure; each storage unit in the exploration intent cache contains a current state vector field, an original candidate action field, a compliance action field, and a timestamp field. The method further includes: When an inconsistency is detected between the original candidate action and the compliant action, the current low-dimensional continuous state vector, the original candidate action, the compliant action, and the current timestamp are written to the tail of the exploration intent buffer. When the exploration intent cache is full, overwrite the storage unit at the head of the queue.

10. The method according to claim 5, characterized in that, Calculate the softening coefficient for each discrete frame rate tier based on the current interaction intensity value, perform compliance mapping on the original candidate actions according to the softening coefficient, and output the compliance actions, including: When the interaction intensity value is lower than the low-bit mask trigger threshold, the softening coefficient of each discrete frame rate level is set to the initial coefficient that allows the original candidate action to pass directly. When the interaction intensity value is between the low-bit mask trigger threshold and the high-bit mask fully effective threshold, the continuously changing softening coefficient corresponding to each discrete frame rate level is calculated according to the relative position of the interaction intensity value in the preset transition interval and the level priority of each discrete frame rate level, so that the probability of the low-priority discrete frame rate level being selected gradually decreases as the interaction intensity increases. When the interaction strength value is higher than the threshold for the high-bit mask to be fully effective, a restriction is imposed on discrete frame rate levels that are lower than the preset level threshold. Based on the softening coefficient of the discrete frame rate level corresponding to the original candidate action, a retention determination or replacement mapping is performed on the original candidate action, and a compliant action is output.

Images