A GPU resource dynamic allocation method and device, electronic equipment and medium

Abstract

The application provides a GPU resource dynamic allocation method and device, electronic equipment and medium. The method collects real-time running data of multiple cloud games of a target tenant in a cloud game platform and inputs the data into a pre-trained load prediction model to determine a total GPU resource demand prediction amount of the multiple cloud games in a future target time window and a GPU resource demand prediction value of each cloud game. The total resource amount of a GPU resource pool is adjusted based on the total GPU resource demand prediction amount. Based on the GPU resource demand prediction value of each cloud game, a preset frame rate and a preset delay target value, a target GPU resource adjustment amount of the cloud game is determined, and a game type-resource demand portrait is determined to determine a target GPU instance of the cloud game. Thus, the GPU cluster resource utilization is improved while ensuring user experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, electronic device, and medium for dynamic allocation of GPU resources. Background Technology

[0002] Existing GPU resource allocation schemes primarily employ static allocation or dynamic allocation strategies based on simple thresholds. Static allocation strategies typically allocate a fixed number of GPU resources to different applications or users during the initial system deployment phase, based on preliminary estimates of business load, and maintain this allocation throughout the entire runtime. While simple to implement, this strategy cannot adapt to fluctuations in business load. It is highly susceptible to resource shortages, request queuing, and degraded user experience during peak access times, while conversely, it leads to significant resource idleness, low resource utilization, and high operating costs during off-peak hours.

[0003] To overcome the shortcomings of static allocation, the industry has proposed a simple dynamic allocation scheme based on real-time monitoring. This scheme typically makes minor adjustments to the resource pool by setting resource utilization thresholds (e.g., triggering expansion when GPU utilization is consistently above 80% and triggering reduction when it is below 20%). However, such schemes are essentially a lagging, passive response mechanism with many inherent flaws. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, device, electronic device and medium for dynamic allocation of GPU resources, which can perform dynamic resource configuration and allocation for cloud gaming in a more precise and advanced manner.

[0005] This application provides a method for dynamic allocation of GPU resources, the method comprising:

[0006] The system collects real-time running data of multiple cloud games belonging to a target tenant on a cloud gaming platform. This real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate, and end-to-end latency. The system inputs this real-time running data of the multiple cloud games into a pre-trained load prediction model. The load prediction model processes this data to determine the total predicted GPU resource demand for the multiple cloud games within a future target time window and the predicted GPU resource demand for each cloud game.

[0007] Adjust the total amount of GPU resource in the target tenant's GPU resource pool based on the total predicted GPU resource demand of multiple cloud games;

[0008] In the adjusted GPU resource pool, the target GPU resource adjustment amount for each cloud game is determined based on the predicted GPU resource requirements, preset frame rate, and preset latency target value. Each cloud game corresponds to a game type-resource requirement profile, which represents the attribute requirements of the GPU instance matched to the cloud game.

[0009] Based on the target GPU resource adjustment amount and game type-resource requirement profile of the cloud game, the target GPU instance of the cloud game is determined in the resource pool after the total resource amount is adjusted; the target GPU instance matches the target GPU resource adjustment amount and attribute requirements of the cloud game.

[0010] In some embodiments, the dynamic allocation method for GPU resources, wherein adjusting the total amount of resources in the target tenant's GPU resource pool based on the predicted total GPU resource demand of multiple cloud games includes:

[0011] The total predicted GPU resource demand of multiple cloud games is compared with the current total resource quantity in the GPU resource pool to determine the total resource quantity comparison result.

[0012] If the total resource comparison result indicates insufficient resources, a resource expansion instruction is generated; the resource expansion instruction includes the number of first GPU instances;

[0013] If the total resource comparison result is that there is a resource surplus, a resource reduction instruction is generated; the resource reduction instruction includes the number of second GPU instances;

[0014] Execute the resource expansion or resource reduction instruction to adjust the total amount of resources in the GPU resource pool.

[0015] In some embodiments, in the GPU resource dynamic allocation method, determining the target GPU resource adjustment amount for each cloud game based on the predicted GPU resource demand, preset frame rate, and preset latency target value in the adjusted GPU resource pool includes: determining the basic GPU resource adjustment amount for each cloud game based on the predicted GPU resource demand.

[0016] Based on the aforementioned basic GPU resource adjustment amount, GPU instances are scheduled for the cloud game, and the real-time average frame rate and real-time latency of the cloud game after instance scheduling are monitored.

[0017] Based on the comparison results of the real-time average frame rate and the preset frame rate, the real-time latency and the preset latency, the target GPU resource adjustment amount for the cloud game is determined.

[0018] In some embodiments, the dynamic GPU resource allocation method determines the target GPU resource adjustment amount for the cloud game based on the comparison between the real-time average frame rate and the preset frame rate, the real-time latency, and the preset latency, including:

[0019] When expanding capacity, if the real-time average frame rate is lower than the first preset frame rate threshold, and / or the real-time latency is higher than the first preset latency threshold, then the resource adjustment amount is increased; the first preset frame rate threshold is determined by the ratio of a preset frame rate and a first preset frame rate tolerance; the first preset latency threshold is determined by the ratio of a preset latency and a first preset latency tolerance.

[0020] When scaling down, if the real-time average frame rate is lower than the second preset frame rate threshold, and / or the real-time latency is higher than the second preset latency threshold, then the amount of resources to be released is reduced; the second preset frame rate threshold is determined by the ratio of the preset frame rate and the second preset frame rate tolerance; the second preset latency threshold is determined by the ratio of the preset latency and the first preset latency tolerance.

[0021] In some embodiments, the dynamic GPU resource allocation method involves determining the target GPU instance for the cloud game within the resource pool after the total resource allocation has been adjusted, based on the target GPU resource adjustment amount and the game type-resource requirement profile. This includes:

[0022] Based on the GPU instance attribute requirements defined in the game type-resource requirement profile, candidate GPU instances that meet the GPU instance attribute requirements are selected from the resource pool; according to the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, a target GPU instance that meets the target GPU resource adjustment amount is selected from the candidate GPU instances.

[0023] In some embodiments, the dynamic GPU resource allocation method involves selecting a target GPU instance from the candidate GPU instances that meets the target GPU resource adjustment amount, based on a preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game.

[0024] Based on the real-time average frame rate and real-time latency of the cloud game, the system dynamically determines whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling.

[0025] If the cost-optimal strategy is enabled, candidate GPU instances are sorted from low to high based on their unit time cost.

[0026] If the performance optimization strategy is enabled, candidate GPU instances are sorted from highest to lowest according to their predefined performance scores.

[0027] The target GPU instance is determined according to the sorting order and the target GPU resource adjustment amount.

[0028] In some embodiments, the dynamic GPU resource allocation method dynamically determines whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling based on the real-time average frame rate and real-time latency of the cloud game, including:

[0029] Determine whether the real-time average frame rate is lower than the third preset frame rate threshold, and / or whether the real-time latency is higher than the third preset latency threshold; the third preset frame rate threshold is determined by a preset frame rate and a third preset frame rate tolerance ratio; the third preset latency threshold is determined by a preset latency and a third preset latency tolerance ratio.

[0030] If any judgment result is yes, then the optimal performance strategy will be used for this scheduling.

[0031] If all the judgment results are negative, then the cost-optimal strategy will be used for this scheduling.

[0032] In some embodiments, a GPU resource dynamic allocation apparatus is also provided, the apparatus comprising:

[0033] The acquisition module is used to acquire real-time running data of multiple cloud games of a target tenant in the cloud gaming platform; the real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate, and end-to-end latency; the first determination module is used to input the real-time running data of the multiple cloud games into a pre-trained load prediction model, process the real-time running data of the multiple cloud games through the load prediction model, and determine the total predicted GPU resource demand of the multiple cloud games and the predicted GPU resource demand value of each cloud game within a future target time window;

[0034] The adjustment module is used to adjust the total amount of resources in the target tenant's GPU resource pool based on the predicted total amount of GPU resource demand for multiple cloud games.

[0035] The second determining module is used to determine the target GPU resource adjustment amount for each cloud game in the adjusted GPU resource pool based on the predicted GPU resource requirements, preset frame rate, and preset latency target value for each cloud game; wherein, each cloud game corresponds to a game type-resource requirement profile, which represents the attribute requirements of the GPU instance matched by the cloud game;

[0036] The third determining module is used to determine the target GPU instance of the cloud game in the resource pool after the total resource volume is adjusted, based on the target GPU resource adjustment amount and game type-resource requirement profile of the cloud game; the target GPU instance matches the target GPU resource adjustment amount and attribute requirements of the cloud game.

[0037] In some embodiments, an electronic device is also provided, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the GPU resource dynamic allocation method are performed.

[0038] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, performs the steps of the GPU resource dynamic allocation method.

[0039] This application provides a method, apparatus, electronic device, and medium for dynamic allocation of GPU resources. The method collects real-time running data of multiple cloud games belonging to a target tenant in a cloud gaming platform. The real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate, and end-to-end latency. The real-time running data of the multiple cloud games is input into a pre-trained load prediction model. The load prediction model processes the real-time running data of the multiple cloud games to determine the total predicted GPU resource demand of the multiple cloud games within a future target time window and the predicted GPU resource demand value for each cloud game. Based on the total predicted GPU resource demand of the multiple cloud games, the total resource volume of the target tenant's GPU resource pool is adjusted. In the adjusted GPU resource pool, based on the predicted GPU resource demand value for each cloud game, a preset frame rate, and a preset latency target value... The target GPU resource adjustment amount for the cloud game is determined. Each cloud game corresponds to a game type-resource requirement profile, representing the attribute requirements of the GPU instance matched to the cloud game. Based on the target GPU resource adjustment amount and the game type-resource requirement profile, the target GPU instance for that type of cloud game is determined from the resource pool after the total resource volume is adjusted. The target GPU instance matches the target GPU resource adjustment amount and attribute requirements of that type of cloud game. In this way, by accurately predicting the load through multi-dimensional real-time data and making collaborative decisions based on the prediction results and user experience metrics (frame rate / latency), intelligent scaling of the total GPU resource pool and fine-grained scheduling of individual game sessions are achieved. This effectively solves the challenges brought by the suddenness and heterogeneity of resource demand in cloud gaming scenarios, and can significantly improve the utilization rate of GPU cluster resources and reduce overall operating costs while ensuring user experience (stable high frame rate, low latency). Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart of the GPU resource dynamic allocation method according to an embodiment of this application is shown;

[0042] Figure 2 A flowchart illustrating the method for adjusting the total resource volume of the target tenant's GPU resource pool according to an embodiment of this application is shown.

[0043] Figure 3 A flowchart illustrating the method for determining the target GPU instance for cloud gaming as described in an embodiment of this application is shown.

[0044] Figure 4 A flowchart illustrating the method for selecting a target GPU instance that meets the target GPU resource adjustment amount from the candidate GPU instances according to an embodiment of this application is shown.

[0045] Figure 5 A schematic diagram of the structure of the GPU resource dynamic allocation device according to an embodiment of this application is shown;

[0046] Figure 6 A schematic diagram of the structure of the electronic device described in an embodiment of this application is shown. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0048] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0049] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0050] Existing GPU resource allocation schemes primarily employ static allocation or dynamic allocation strategies based on simple thresholds. Static allocation strategies typically allocate a fixed number of GPU resources to different applications or users during the initial system deployment phase, based on preliminary estimates of business load, and maintain this allocation throughout the entire runtime. While simple to implement, this strategy cannot adapt to fluctuations in business load. It is highly susceptible to resource shortages, request queuing, and degraded user experience during peak access times, while conversely, it leads to significant resource idleness, low resource utilization, and high operating costs during off-peak hours.

[0051] To overcome the shortcomings of static allocation, the industry has proposed a simple dynamic allocation scheme based on real-time monitoring. This scheme typically makes minor adjustments to the resource pool by setting resource utilization thresholds (e.g., triggering expansion when GPU utilization is consistently above 80% and triggering reduction when it is below 20%). However, such schemes are essentially a lagging, passive response mechanism with many inherent flaws.

[0052] Specifically, the existing allocation mechanism is lagging: resource adjustments always lag behind load changes. There is a significant delay between monitoring, threshold exceeding, and resource allocation completion. For applications with extremely high real-time requirements, such as cloud gaming and VR / AR, this delay can lead to serious consequences for the user experience, such as frame rate drops and latency spikes.

[0053] The existing allocation mechanism is rather crude: its decisions rely solely on limited real-time metrics (such as GPU utilization), lacking insight into future load trends and unable to cope with sudden traffic spikes or complex periodic changes. Furthermore, it fails to consider the differentiated requirements of different task types (such as different games or rendering tasks) for GPU resources (such as video memory and core computing power), thus failing to achieve fine-grained resource scheduling.

[0054] In a multi-tenant shared public cloud environment, simple threshold policies are difficult to intelligently coordinate resource allocation among the services of multiple tenants, which can easily lead to resource competition. To avoid this situation, operators often tend to over-allocate resources, resulting in high costs.

[0055] Based on this, this application provides a method, apparatus, electronic device, and medium for dynamic allocation of GPU resources. The method collects real-time running data of multiple cloud games belonging to a target tenant in a cloud gaming platform. The real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate, and end-to-end latency. The real-time running data of the multiple cloud games is input into a pre-trained load prediction model. The load prediction model processes the real-time running data of the multiple cloud games to determine the total predicted GPU resource demand of the multiple cloud games within a future target time window and the predicted GPU resource demand value for each cloud game. Based on the total predicted GPU resource demand of the multiple cloud games, the total resource volume of the target tenant's GPU resource pool is adjusted. In the adjusted GPU resource pool, based on the predicted GPU resource demand value for each cloud game, a preset frame rate, and a preset latency target... The system determines the target GPU resource adjustment amount for the cloud game. Each cloud game corresponds to a game type-resource requirement profile, representing the attribute requirements of the GPU instance matched to the cloud game. Based on the target GPU resource adjustment amount and the game type-resource requirement profile, the target GPU instance for that type of cloud game is determined in the resource pool after the total resource volume is adjusted. The target GPU instance matches the target GPU resource adjustment amount and attribute requirements of that type of cloud game. In this way, by accurately predicting the load through multi-dimensional real-time data, and making collaborative decisions based on the prediction results and user experience indicators (frame rate / latency), the system realizes intelligent scaling of the total GPU resource pool and fine-grained scheduling of individual game sessions. This effectively solves the challenges brought by the suddenness and heterogeneity of resource demand in cloud gaming scenarios, and can significantly improve the utilization rate of GPU cluster resources and reduce the overall operating cost while ensuring user experience (stable high frame rate, low latency).

[0056] Please refer to Figure 1 , Figure 1 A flowchart of the GPU resource dynamic allocation method described in an embodiment of this application is shown; as follows: Figure 1 As shown, the method includes the following steps S101-S105:

[0057] S101. Collect real-time running data of multiple cloud games of the target tenant in the cloud gaming platform; the real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate and end-to-end latency; S102. Input the real-time running data of the multiple cloud games into a pre-trained load prediction model, process the real-time running data of the multiple cloud games through the load prediction model, and determine the total predicted GPU resource demand of the multiple cloud games and the predicted GPU resource demand value of each cloud game within the future target time window;

[0058] S103. Adjust the total amount of GPU resource pool for the target tenant based on the total predicted GPU resource demand of multiple cloud games;

[0059] S104. In the adjusted GPU resource pool, based on the predicted GPU resource requirements, preset frame rate, and preset latency target value of each cloud game, determine the target GPU resource adjustment amount for that cloud game; wherein, each cloud game corresponds to a game type-resource requirement profile, which represents the attribute requirements of the GPU instance matched to the cloud game.

[0060] S105. Based on the target GPU resource adjustment amount and game type-resource requirement profile of the cloud game, determine the target GPU instance of the cloud game in the resource pool after the total resource amount is adjusted; the target GPU instance matches the target GPU resource adjustment amount and attribute requirements of the cloud game.

[0061] In step S101, real-time running data of multiple cloud games of the target tenant in the cloud gaming platform is collected; the real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate and end-to-end latency.

[0062] The game type refers to the task type that the cloud game runs (such as VR game, 2D game, 4K rendering, etc.). Different game types require different GPU resources, so the weighted average of GPU resource requirements is estimated by the task complexity during cloud game operation.

[0063] The number of user sessions refers to the number of currently established and active game connections. For example, there are currently 15,342 users playing games on the platform. The number of user sessions is the most direct load indicator, reflecting the current pressure on the platform. It is one of the core input features of the load prediction model, used to predict future user growth trends and total resource requirements.

[0064] The event type refers to pre-planned external events that may affect the load. Examples include new game launches, large-scale promotional events, e-sports live broadcasts, and holidays, used to cope with non-periodic sudden load surges. Traditional load forecasting models (such as LSTM) can capture periodic patterns, but cannot predict these sudden event-driven traffic surges. By using this indicator as a feature input to the model, the accuracy of predicting sudden traffic surges can be greatly improved.

[0065] GPU utilization is the percentage of time a GPU computing unit (such as a CUDA core) is busy. For example, a server's GPU utilization is 85%. GPU utilization reflects the stress level of GPU computing power, is a direct manifestation of high load, and is one of the core threshold indicators that trigger scaling up (e.g., >85%) or scaling down (e.g., <20%).

[0066] Video memory usage rate refers to the percentage of dedicated video memory used by the GPU. For example, a game session might use 12GB / 16GB of video memory, used to assess memory resource bottlenecks. Even if GPU utilization is low, a new game session cannot be launched if video memory is fully occupied.

[0067] Video encoding frame rate is the speed at which the cloud GPU encodes rendered footage into a video stream, measured in FPS (frames per second), for example, 60 FPS. Frame rate directly determines the smoothness of game visuals and is the ultimate goal of ensuring Quality of Service (SLA).

[0068] End-to-end latency, measured in milliseconds (ms), is the total time from player input to the screen displaying the response; for example, 80ms. Latency determines how responsive the action is. It is another ultimate goal in ensuring SLA (Service Level Agreement). High latency severely impacts the user experience.

[0069] The real-time operational data is rich in dimensions, which together constitute a multi-layered and comprehensive monitoring system.

[0070] In some embodiments, the acquired real-time running data is stored to obtain historical running data.

[0071] The real-time data is stored in a time-series database (such as Prometheus or InfluxDB) with a time granularity of 1 minute or 5 minutes to ensure low-latency access.

[0072] The historical operational data: retain at least 90 days of data, stored in a distributed database (such as Apache Cassandra, Amazon S3), for model training and long-term trend analysis.

[0073] The collected real-time operational data needs to be preprocessed to ensure the data quality input into the load prediction model.

[0074] Preprocessing includes data cleaning, data normalization, feature engineering, and data windowing.

[0075] Data cleaning: Remove outliers, such as sudden changes in GPU utilization (>100% or <0%) or abnormal fluctuations in the number of users (more than 50% increase / decrease in a single minute); handle missing values ​​by using linear interpolation or mean to fill in missing values ​​and preserve data integrity.

[0076] Data normalization: Min-Max normalization or Z-Score standardization is performed on data of different dimensions (such as number of user sessions, GPU utilization, and video memory usage) to unify them to [0,1] or the standard normal distribution range.

[0077] Feature engineering: Extract key features, including user growth rate, task queue length change rate, GPU utilization trend, time periodic features (such as hour, week), event markers, etc.

[0078] Construct combined features, such as the "product of the number of users and the task complexity", to reflect comprehensive resource requirements.

[0079] Data Windowing: Aggregate data by time window (5 minutes, 15 minutes, 30 minutes) to generate a structured dataset suitable for time series forecasting.

[0080] In step S102, the real-time running data of multiple cloud games is input into a pre-trained load prediction model. The load prediction model processes the real-time running data of the multiple cloud games to determine the total predicted GPU resource demand of the multiple cloud games and the predicted GPU resource demand value of each cloud game within a future target time window.

[0081] The load prediction model can employ time series models, such as LSTM (Long Short-Term Memory Network) or Prophet, to capture the periodic patterns of indicators such as user numbers and GPU utilization (e.g., daily peaks, weekly patterns); or it can employ machine learning regression models, such as XGBoost or LightGBM, to handle nonlinear relationships and complex feature interactions (e.g., the coupling effect between "user growth rate" and "event type").

[0082] The load prediction model is also configured with a rule supplementation mechanism, such as setting threshold trigger rules (e.g., "user growth rate > 15% and GPU utilization > 85%)" to deal with sudden load scenarios and ensure prediction robustness.

[0083] For example, an LSTM neural network model that has been trained with historical data from the past 3 months takes a 7-day data window as input and outputs a predicted value for the next day.

[0084] The training and optimization process of the load prediction model is as follows: Training cycle: The model is updated daily or weekly based on the latest data to adapt to changes in user behavior.

[0085] Validation method: 80% of the data is used for training and 20% for validation. Evaluation metrics include MAE (mean absolute error) and RMSE (root mean square error).

[0086] Hyperparameter tuning: Adjust model parameters (such as the number of hidden layer units in LSTM and the depth of XGBoost tree) through grid search or Bayesian optimization.

[0087] Real-time correction: Dynamically correct prediction bias by combining real-time monitoring data (such as current GPU utilization).

[0088] The future target time window refers to a future time interval predicted by the load forecasting model; the starting point of this time window is the moment when the forecast is executed, and the ending point is a specific moment in the future.

[0089] In some embodiments, the length of the future target time window is 10-30 minutes.

[0090] The multiple cloud games refer to the multiple cloud games operated by the target tenant.

[0091] The total predicted GPU resource demand refers to the sum of GPU resources required by multiple cloud games within the target time window, which is the total GPU resources that need to be allocated to the target tenant.

[0092] The predicted GPU resource requirement for each cloud game refers to the amount of GPU resources required for each specific cloud game at the end of the target time window.

[0093] In other words, the load forecasting model performs two calculations simultaneously: macro forecasting: treating all data as a whole to predict the overall resource demand trend of the platform (total predicted GPU resource demand); micro forecasting: identifying and analyzing the data of each game to predict the resource demand of each individual game (predicted GPU resource demand value for each cloud game).

[0094] In some embodiments, in addition to GPU resource demand prediction, the load prediction model also outputs parameters such as the expected number of user sessions.

[0095] In some embodiments, the predicted total GPU resource demand and the predicted GPU resource demand for each cloud game are stored in an in-memory database (such as Redis) for use by the resource allocation module.

[0096] In some embodiments, a prediction is run every 5 minutes to generate predictions for the next 10, 15, or 30 minutes.

[0097] Based on the predicted total GPU resource demand of multiple cloud games and the predicted GPU resource demand of each cloud game, an efficient dynamic allocation strategy for GPU resources is formulated.

[0098] In step S103, the total amount of resources in the target tenant's GPU resource pool is adjusted based on the predicted total amount of GPU resource demand for multiple cloud games.

[0099] Please refer to Figure 2 The method of adjusting the total GPU resource pool of the target tenant based on the predicted total GPU resource demand of multiple cloud games includes the following steps S201-S204:

[0100] S201. Compare the predicted total GPU resource demand of multiple cloud games with the current total resource in the GPU resource pool to determine the total resource comparison result.

[0101] S202. If the total resource comparison result is insufficient, a resource expansion instruction is generated; the resource expansion instruction includes the number of first GPU instances;

[0102] S203. If the total resource comparison result is that resources are excessive, a resource reduction instruction is generated; the resource reduction instruction includes the number of second GPU instances.

[0103] S204. Execute the resource expansion instruction or resource reduction instruction to adjust the total amount of resources in the GPU resource pool.

[0104] The resource total comparison result indicates insufficient resources, meaning the ratio of the current total resources to the predicted total GPU resource demand is greater than a preset expansion threshold; for example, the preset expansion threshold is 85%.

[0105] In other words, when the real-time GPU utilization exceeds 85% of the total predicted GPU resource demand, it indicates insufficient resources.

[0106] The resource total comparison result is the resource process, that is, the ratio of the current total resource and the predicted total GPU resource demand is greater than the preset shrinkage threshold; for example, the preset expansion threshold is 20%.

[0107] In other words, when the real-time GPU utilization is less than 20% of the total predicted GPU resource demand, it indicates that resources are in excess.

[0108] In this embodiment, the total resource volume of the GPU resource pool is adjusted based on prediction, moving from "passive response" to "active planning". This improves response speed and user experience, reduces performance degradation and service interruption time caused by insufficient resources, and ensures a smooth and stable user experience.

[0109] Here, the total amount of resources in the GPU resource pool refers to the total amount of resources for multiple cloud games of the target tenant, rather than the cloud architecture mode of "tenant-isolated resource pool" or "dedicated cluster" for the entire platform.

[0110] Here, the resource expansion or reduction instruction is executed to adjust the total amount of resources in the GPU resource pool, taking into account the target tenant's resource limit, priority, and cost.

[0111] Suppose a cloud platform provides cloud gaming services to both "Company A" and "Company B". Company A holds a popular promotional event, resulting in a surge in users. If no resource limit is set for tenants, the sudden traffic from Company A may consume GPU resources that should belong to Company B, causing lag for Company B's users. Therefore, a logically isolated resource pool is allocated to each tenant, and the total resource limit for that tenant is adjusted in advance.

[0112] Alternatively, Company A is a large game developer that demands the highest service quality and performance and is willing to pay higher fees; Company B is a startup that is extremely cost-sensitive and can accept occasional performance fluctuations. In this case, it is also necessary to allocate a logically isolated resource pool to each tenant to provide differentiated services.

[0113] In other words, the number of first GPU instances and the number of second GPU instances in the resource expansion instruction are determined based on the total resource limit, priority, and cost of the tenants operating the multiple cloud games.

[0114] In step S104, the target GPU resource adjustment amount for each cloud game is determined based on the predicted GPU resource requirements, preset frame rate, and preset latency target value of each cloud game in the adjusted GPU resource pool.

[0115] Specifically, in the adjusted GPU resource pool, the target GPU resource adjustment amount for each cloud game is determined based on the predicted GPU resource demand, preset frame rate, and preset latency target value. This includes: determining the basic GPU resource adjustment amount for each cloud game based on the predicted GPU resource demand.

[0116] Based on the aforementioned basic GPU resource adjustment amount, GPU instances are scheduled for the cloud game, and the real-time average frame rate and real-time latency of the cloud game after instance scheduling are monitored.

[0117] Based on the comparison results of the real-time average frame rate and the preset frame rate, the real-time latency and the preset latency, the target GPU resource adjustment amount for the cloud game is determined.

[0118] In other words, determining the target GPU resource adjustment amount for the cloud game is done in two stages.

[0119] Based on the comparison results of the real-time average frame rate, the preset frame rate, the real-time latency, and the preset latency, the target GPU resource adjustment amount for the cloud game is determined, including:

[0120] When expanding capacity, if the real-time average frame rate is lower than the first preset frame rate threshold, and / or the real-time latency is higher than the first preset latency threshold, then the resource adjustment amount is increased; the first preset frame rate threshold is determined by the ratio of a preset frame rate and a first preset frame rate tolerance; the first preset latency threshold is determined by the ratio of a preset latency and a first preset latency tolerance.

[0121] When scaling down, if the real-time average frame rate is lower than the second preset frame rate threshold, and / or the real-time latency is higher than the second preset latency threshold, then the amount of resources to be released is reduced; the second preset frame rate threshold is determined by the ratio of the preset frame rate and the second preset frame rate tolerance; the second preset latency threshold is determined by the ratio of the preset latency and the first preset latency tolerance.

[0122] Based on the predicted GPU resource requirements of each cloud game, the basic GPU resource adjustment amount for that cloud game is determined. Specifically, the formula for calculating the basic GPU resource adjustment amount (ΔB) is: ΔB = P – C; where P represents the predicted GPU resource requirements of the cloud game within the future target time window given by the load forecasting model; and C represents the amount of GPU resources currently allocated to the cloud game.

[0123] If ΔB > 0, the cloud game needs to be expanded; if ΔB < 0, the cloud game can be considered for downsizing.

[0124] The system performs initial GPU instance scheduling for the cloud game based on the calculated base GPU resource adjustment (ΔB). For example, if ΔB = +2, the scheduler is invoked to attempt to allocate two new GPU instances for the game from the resource pool.

[0125] At the same time, the system initiated high-frequency real-time monitoring, continuously collecting the cloud game's real-time average frame rate (Current_FPS) and real-time latency (Current_Latency) after this initial scheduling. The monitoring data was collected at a granular level of seconds or minutes using monitoring tools such as Prometheus.

[0126] After adjusting based on the base GPU resource adjustment amount (ΔB), the system compares the real-time monitored user experience metrics (real-time average frame rate, real-time latency) with the preset target, and dynamically corrects the base adjustment amount (ΔB) based on the comparison results to obtain the final, more accurate target GPU resource adjustment amount (ΔF).

[0127] The first preset frame rate threshold (FPS_Threshold) is determined by the preset frame rate (Target_FPS) and the first preset frame rate tolerance ratio (α). Its calculation formula is as follows:

[0128] FPS_Threshold = Target_FPS * α (where α < 1, for example, α = 0.95);

[0129] For example, if Target_FPS is 60 FPS and α is 0.95, then FPS_Threshold is 57 FPS.

[0130] The first preset latency threshold (Latency_Threshold) is determined by the preset latency (Target_Latency) and the first preset latency tolerance ratio (β). Its calculation formula is as follows:

[0131] Latency_Threshold = Target_Latency * β (where β > 1, for example, β = 1.1)

[0132] For example, if Target_Latency is 100ms and β is 1.1, then Latency_Threshold is 110ms.

[0133] The setting of the second preset frame rate threshold and the second preset latency threshold is based on the same principle as the first preset frame rate threshold and the first preset latency threshold, and will not be repeated here.

[0134] In a capacity expansion scenario, if the real-time average frame rate is lower than the first preset frame rate threshold, and / or the real-time latency is higher than the first preset latency threshold, it indicates that the initial scheduled basic resource amount (ΔB) is still insufficient, failing to restore the user experience to a safe range. In this case, the system needs to increase the resource adjustment amount.

[0135] In a scaling-down scenario, if the real-time average frame rate is lower than the first preset frame rate threshold, and / or the real-time latency is higher than the first preset latency threshold, it indicates that even without scaling down, the current user experience is already on the verge of danger. Scaling down at this point carries extremely high risk. Therefore, the system should reduce the amount of resources to be released, or even pause scaling down altogether.

[0136] In this embodiment, after determining the overall resource pool of a tenant, a two-stage adjustment mechanism for individual cloud games—initially adjusting GPU resources based on prediction and verifying user experience—brings significant technological advancements. This effectively overcomes potential errors in load prediction models and ensures that resource allocation always prioritizes a genuine user experience through real-time feedback. Furthermore, it provides a safety buffer, preventing the erroneous release of resources when system performance is already strained, thus ensuring service stability. In sudden load scenarios, it can respond quickly by adding resources to ensure that the user experience recovers to the target level as quickly as possible.

[0137] In step S105, based on the target GPU resource adjustment amount and game type-resource requirement profile of the cloud game, the target GPU instance of the cloud game is determined in the resource pool after the total resource amount is adjusted; the target GPU instance matches the target GPU resource adjustment amount and attribute requirements of the cloud game.

[0138] Please refer to Figure 3 Based on the target GPU resource adjustment amount and game type-resource requirement profile of the cloud game, the target GPU instance for this type of cloud game is determined in the resource pool after the total resource amount is adjusted; including the following steps S301-S302:

[0139] S301. Based on the GPU instance attribute requirements defined in the game type-resource requirement profile, select candidate GPU instances that meet the GPU instance attribute requirements from the resource pool; S302. According to the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, select the target GPU instance that meets the target GPU resource adjustment amount from the candidate GPU instances.

[0140] The game type-resource requirement profile is a predefined data structure that explicitly specifies the technical attribute requirements of the GPU instances needed to run such games. For example, for "AAA" games, the profile might be defined as: {GPU memory: >= 16GB, GPU model: ABC; Computing power level: High}.

[0141] Each GPU instance in the resource pool has been tagged with a technical label describing its specifications. The attributes (i.e., technical labels) of each instance are matched with the resource requirement profile, and all instances that meet the requirements are added to the candidate GPU instance list.

[0142] In this embodiment, instead of randomly selecting instances from the candidate list, the most suitable instance is intelligently selected based on a dynamic decision-making strategy.

[0143] It should be noted that the core basis for dynamic strategy determination is the current real user experience of the cloud game, namely the real-time average frame rate (Current_FPS) and real-time average latency (Current_Latency) monitored in real time.

[0144] In some embodiments, please refer to Figure 4 Based on the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, a target GPU instance that meets the target GPU resource adjustment amount is selected from the candidate GPU instances, including the following steps S401-S404:

[0145] S401. Based on the real-time average frame rate and real-time latency of the cloud game monitored in real time, dynamically determine whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling.

[0146] S402. If the cost-optimal strategy is enabled, the candidate GPU instances are sorted from low to high based on their unit time cost.

[0147] S403. If the performance optimization strategy is enabled, the candidate GPU instances are sorted from highest to lowest according to their predefined performance scores.

[0148] S404. Determine the target GPU instance according to the sorting order and the target GPU resource adjustment amount.

[0149] If the cost-optimal strategy is enabled, query the unit time cost of each candidate instance (e.g., 3 yuan / hour for on-demand instances and 1 yuan / hour for preemptive instances); then sort them from low to high unit time cost to form a cost-optimal candidate queue; the instance with the lowest cost is at the top of the queue.

[0150] If the optimal performance strategy is enabled, the system calculates a score for each candidate instance based on a predefined performance score model. This model comprehensively considers hardware metrics such as the instance's GPU model, video memory size, number of vCPUs, and network bandwidth (for example, an A100 instance has a performance score of 100, a V100 instance has a score of 70, and a T4 instance has a score of 40). The instances are then sorted from highest to lowest performance score to form an optimal performance candidate queue, with the most powerful instances at the top.

[0151] It should be noted that the predefined performance score for each instance can be predefined, thereby improving real-time performance.

[0152] Assume the target GPU resource adjustment requires 3 GPUs.

[0153] Under the cost-optimal strategy, the first, second, and third instances with the lowest costs are selected sequentially from the cost-optimal queue.

[0154] Under the optimal performance strategy, the 1st, 2nd, and 3rd highest performing instances are selected sequentially from the optimal performance queue.

[0155] In some embodiments, based on the real-time average frame rate and real-time latency of the cloud game as monitored in real time, the system dynamically determines whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling, including:

[0156] Determine whether the real-time average frame rate is lower than the third preset frame rate threshold, and / or whether the real-time latency is higher than the third preset latency threshold; the third preset frame rate threshold is determined by a preset frame rate and a third preset frame rate tolerance ratio; the third preset latency threshold is determined by a preset latency and a third preset latency tolerance ratio.

[0157] If any judgment result is yes, then the optimal performance strategy will be used for this scheduling.

[0158] If all the judgment results are negative, then the cost-optimal strategy will be used for this scheduling.

[0159] The third preset frame rate tolerance ratio, for example, is 0.8; the third preset latency tolerance ratio, for example, is 1.2.

[0160] If any judgment result is yes, it means that the current user experience is already at or about to fall below the acceptable lower limit. At this time, ensuring service performance and quality is the top priority, and the optimal performance strategy will be used for this scheduling.

[0161] If both judgments result in no, it indicates that the current user experience is good and there are sufficient resource reserves. At this point, optimizing operating costs becomes possible while ensuring a good user experience, therefore the system determines that the cost-optimal strategy will be used for this scheduling.

[0162] It should be noted that this embodiment involves two layers of resource scheduling. The first layer is the adjustment of the total amount of resources at the tenant level, and the second layer is the fine-grained scheduling of games within the tenant. The platform's global scheduler is only responsible for managing the expansion and contraction of the resource pool at the tenant level, and the scheduling target is relatively small. Each tenant's dedicated scheduler (or scheduling strategy) is only responsible for managing the allocation of dozens or hundreds of instances within its own resource pool, and the scheduling target is also relatively small. Without the first layer of scheduling, only the second layer of scheduling would not only result in a large scheduling volume, but also make it difficult and unfair for games like "Cloud Game A" belonging to Company A and "Cloud Game B" belonging to Company B to compete for instances on the platform, regardless of whether the cost-optimal strategy or the performance-optimal strategy is implemented.

[0163] The second level is fine-grained scheduling at the game level within the tenant. Essentially, it involves resource adaptation for each game within the tenant. Within the dedicated resource pool of a fixed size allocated in the first step, the goal is to optimally allocate the GPU instances in the pool to different games within the tenant to improve the cloud gaming experience.

[0164] In some embodiments, the technical architecture and tools for executing the GPU resource dynamic allocation method are as follows.

[0165] Monitoring system: Prometheus + Grafana: Real-time collection and visualization of metrics, supporting dynamic alerts; NVIDIA DCGM: Collection of specific GPU metrics (such as memory, core utilization, and power consumption).

[0166] Resource Management: Kubernetes + NVIDIA GPU Operator: Implements containerized GPU task scheduling and supports dynamic allocation.

[0167] AWS Auto Scaling / Azure VM Scale Sets: Automates the management of the number of GPU instances.

[0168] Load forecasting model deployment: Deploy the forecasting model using ML frameworks (such as TensorFlow, PyTorch) or time series tools (such as Prophet); integrate the forecast results into the scheduling system via RESTful APIs (such as Flask, FastAPI).

[0169] Load balancing: Use AWS Elastic Load Balancer or NGINX to ensure that user requests are efficiently distributed to available GPU instances.

[0170] Based on the same inventive concept, this application also provides a GPU resource dynamic allocation device corresponding to the GPU resource dynamic allocation method. Since the principle of the device in this application is similar to the GPU resource dynamic allocation method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0171] Please refer to Figure 5 , Figure 5 A schematic diagram of the structure of the GPU resource dynamic allocation device according to an embodiment of this application is shown; the device includes:

[0172] The acquisition module 501 is used to acquire real-time running data of multiple cloud games of a target tenant in the cloud gaming platform; the real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate and end-to-end latency; the first determination module 502 is used to input the real-time running data of the multiple cloud games into a pre-trained load prediction model, process the real-time running data of the multiple cloud games through the load prediction model, and determine the total predicted GPU resource demand of the multiple cloud games and the predicted GPU resource demand value of each cloud game within a future target time window;

[0173] Adjustment module 503 is used to adjust the total amount of resources in the target tenant's GPU resource pool based on the total predicted GPU resource demand of multiple cloud games;

[0174] The second determining module 504 is used to determine the target GPU resource adjustment amount for each cloud game in the adjusted GPU resource pool based on the predicted GPU resource requirement value, preset frame rate and preset latency target value of each cloud game; wherein, each cloud game corresponds to a game type-resource requirement profile, which represents the attribute requirements of the GPU instance matched by the cloud game.

[0175] The third determining module 505 determines the target GPU instance of the cloud game based on the target GPU resource adjustment amount and the game type-resource requirement profile. The target GPU instance matches the target GPU resource adjustment amount and attribute requirements of the cloud game.

[0176] In some embodiments, in the GPU resource dynamic allocation device, when the adjustment module adjusts the total resource volume of the target tenant's GPU resource pool based on the predicted total GPU resource demand of multiple cloud games, it is specifically used for:

[0177] The total predicted GPU resource demand of multiple cloud games is compared with the current total resource quantity in the GPU resource pool to determine the total resource quantity comparison result.

[0178] If the total resource comparison result indicates insufficient resources, a resource expansion instruction is generated; the resource expansion instruction includes the number of first GPU instances;

[0179] If the total resource comparison result is that there is a resource surplus, a resource reduction instruction is generated; the resource reduction instruction includes the number of second GPU instances;

[0180] Execute the resource expansion or resource reduction instruction to adjust the total amount of resources in the GPU resource pool.

[0181] In some embodiments, in the GPU resource dynamic allocation device, when the second determining module determines the target GPU resource adjustment amount for each cloud game based on the predicted GPU resource demand value, preset frame rate, and preset latency target value in the adjusted GPU resource pool, it is specifically used to: determine the basic GPU resource adjustment amount for each cloud game based on the predicted GPU resource demand value for each cloud game.

[0182] Based on the aforementioned basic GPU resource adjustment amount, GPU instances are scheduled for the cloud game, and the real-time average frame rate and real-time latency of the cloud game after instance scheduling are monitored.

[0183] Based on the comparison results of the real-time average frame rate and the preset frame rate, the real-time latency and the preset latency, the target GPU resource adjustment amount for the cloud game is determined.

[0184] In some embodiments, in the GPU resource dynamic allocation device, the second determining module, when determining the target GPU resource adjustment amount for the cloud game based on the comparison result of the real-time average frame rate and the preset frame rate, the real-time latency and the preset latency, is specifically used for:

[0185] When expanding capacity, if the real-time average frame rate is lower than the first preset frame rate threshold, and / or the real-time latency is higher than the first preset latency threshold, then the resource adjustment amount is increased; the first preset frame rate threshold is determined by the ratio of a preset frame rate and a first preset frame rate tolerance; the first preset latency threshold is determined by the ratio of a preset latency and a first preset latency tolerance.

[0186] When scaling down, if the real-time average frame rate is lower than the second preset frame rate threshold, and / or the real-time latency is higher than the second preset latency threshold, then the amount of resources to be released is reduced; the second preset frame rate threshold is determined by the ratio of the preset frame rate and the second preset frame rate tolerance; the second preset latency threshold is determined by the ratio of the preset latency and the first preset latency tolerance.

[0187] In some embodiments, in the GPU resource dynamic allocation device, when the third determining module determines the target GPU instance of the cloud game from the resource pool after the total resource volume adjustment based on the target GPU resource adjustment amount and the game type-resource requirement profile, it is specifically used for:

[0188] Based on the GPU instance attribute requirements defined in the game type-resource requirement profile, candidate GPU instances that meet the GPU instance attribute requirements are selected from the resource pool; according to the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, a target GPU instance that meets the target GPU resource adjustment amount is selected from the candidate GPU instances.

[0189] In some embodiments, in the GPU resource dynamic allocation device, the third determining module, when selecting a target GPU instance that meets the target GPU resource adjustment amount from the candidate GPU instances according to the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, is specifically used for:

[0190] Based on the real-time average frame rate and real-time latency of the cloud game, the system dynamically determines whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling.

[0191] If the cost-optimal strategy is enabled, candidate GPU instances are sorted from low to high based on their unit time cost.

[0192] If the performance optimization strategy is enabled, candidate GPU instances are sorted from highest to lowest according to their predefined performance scores.

[0193] The target GPU instance is determined according to the sorting order and the target GPU resource adjustment amount.

[0194] In some embodiments, in the GPU resource dynamic allocation device, when the third determining module dynamically determines whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling based on the real-time average frame rate and real-time latency of the cloud game, it is specifically used for:

[0195] Determine whether the real-time average frame rate is lower than the first preset frame rate threshold, and / or whether the real-time latency is higher than the first preset latency threshold; the second preset frame rate threshold is determined by the ratio of a preset frame rate to a second preset frame rate tolerance; the first preset latency threshold is determined by the ratio of a preset latency to a second preset latency tolerance.

[0196] If any judgment result is yes, then the optimal performance strategy will be used for this scheduling.

[0197] If all the judgment results are negative, then the cost-optimal strategy will be used for this scheduling.

[0198] Based on the same inventive concept, this application also provides an electronic device corresponding to the GPU resource dynamic allocation method. Since the principle of solving the problem by the electronic device in this application is similar to the GPU resource dynamic allocation method described above in this application, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be described again.

[0199] Please refer to Figure 6 , Figure 6A schematic diagram of the structure of the electronic device according to an embodiment of this application is shown; the electronic device 600 includes: a processor 602, a memory 601 and a bus. The memory 601 stores machine-readable instructions that can be executed by the processor 602. When the electronic device 600 is running, the processor 602 communicates with the memory 601 through the bus. When the machine-readable instructions are executed by the processor 602, the steps of the GPU resource dynamic allocation method are performed.

[0200] Based on the same inventive concept, this application also provides a computer-readable storage medium corresponding to the GPU resource dynamic allocation method. Since the principle of the computer-readable storage medium in this application is similar to the GPU resource dynamic allocation method described above in this application, the implementation of the computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be described again.

[0201] A computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the GPU resource dynamic allocation method described above.

[0202] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0203] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0205] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0206] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for dynamically allocating GPU resources, characterized in that, The method includes: The system collects real-time running data of multiple cloud games belonging to a target tenant on a cloud gaming platform. This real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate, and end-to-end latency. The system inputs this real-time running data of the multiple cloud games into a pre-trained load prediction model. The load prediction model processes this data to determine the total predicted GPU resource demand for the multiple cloud games within a future target time window and the predicted GPU resource demand for each cloud game. The total resource volume of the target tenant's GPU resource pool is adjusted based on the predicted total GPU resource demand of multiple cloud games. Specifically, the predicted total GPU resource demand of multiple cloud games is compared with the current total resource volume in the GPU resource pool to determine the resource volume comparison result. If the resource volume comparison result indicates insufficient resources, a resource expansion instruction is generated, including the first number of GPU instances. If the resource volume comparison result indicates excess resources, a resource reduction instruction is generated, including the second number of GPU instances. The resource expansion or reduction instruction is executed to adjust the total resource volume of the GPU resource pool. In the adjusted GPU resource pool, the target GPU resource adjustment amount for each cloud game is determined based on its predicted GPU resource requirements, preset frame rate, and preset latency target value. Each cloud game corresponds to a game type-resource requirement profile, representing the attribute requirements of the GPU instances matched to the cloud game. A basic GPU resource adjustment amount is determined based on the predicted GPU resource requirements for each cloud game. GPU instances are scheduled for the cloud game based on this basic GPU resource adjustment amount, and the real-time average frame rate and real-time latency of the cloud game after instance scheduling are monitored. The target GPU resource adjustment amount for the cloud game is determined based on the comparison results of the real-time average frame rate, preset frame rate, real-time latency, and preset latency. Based on the target GPU resource adjustment amount and game type-resource requirement profile of the cloud game, the target GPU instance for this type of cloud game is determined in the resource pool after the total resource amount is adjusted. The target GPU instance is matched with the target GPU resource adjustment amount and attribute requirements of the cloud game. Specifically, based on the GPU instance attribute requirements defined by the game type-resource requirement profile, candidate GPU instances that meet the GPU instance attribute requirements are selected from the resource pool. According to the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, the target GPU instance that meets the target GPU resource adjustment amount is selected from the candidate GPU instances.

2. The GPU resource dynamic allocation method according to claim 1, characterized in that, Based on the comparison results of the real-time average frame rate, the preset frame rate, the real-time latency, and the preset latency, the target GPU resource adjustment amount for the cloud game is determined, including: When expanding capacity, if the real-time average frame rate is lower than the first preset frame rate threshold, and / or the real-time latency is higher than the first preset latency threshold, then the resource adjustment amount is increased; the first preset frame rate threshold is determined by the ratio of a preset frame rate and a first preset frame rate tolerance; the first preset latency threshold is determined by the ratio of a preset latency and a first preset latency tolerance. When scaling down, if the real-time average frame rate is lower than the second preset frame rate threshold, and / or the real-time latency is higher than the second preset latency threshold, then the amount of resources to be released is reduced; the second preset frame rate threshold is determined by the ratio of the preset frame rate and the second preset frame rate tolerance; the second preset latency threshold is determined by the ratio of the preset latency and the first preset latency tolerance.

3. The GPU resource dynamic allocation method according to claim 2, characterized in that, Based on the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud gaming, a target GPU instance that meets the target GPU resource adjustment amount is selected from the candidate GPU instances, including: Based on the real-time average frame rate and real-time latency of the cloud game, the system dynamically determines whether to use the cost-optimal strategy or the performance-optimal strategy for this scheduling. If the cost-optimal strategy is enabled, candidate GPU instances are sorted from low to high based on their unit time cost. If the performance optimization strategy is enabled, candidate GPU instances are sorted from highest to lowest according to their predefined performance scores. The target GPU instance is determined according to the sorting order and the target GPU resource adjustment amount.

4. The GPU resource dynamic allocation method according to claim 3, characterized in that, Based on the real-time average frame rate and real-time latency of the cloud game, the system dynamically determines whether to use the most cost-effective or most performance-effective strategy for this scheduling, including: Determine whether the real-time average frame rate is lower than a third preset frame rate threshold, and / or whether the real-time latency is higher than the third preset latency threshold; the third preset frame rate threshold is determined by a preset frame rate and a third preset frame rate tolerance ratio; the third preset latency threshold is determined by a preset latency and a third preset latency tolerance ratio. If any judgment result is yes, then the optimal performance strategy will be used for this scheduling. If all the judgment results are negative, then the cost-optimal strategy will be used for this scheduling.

5. A GPU resource dynamic allocation device, characterized in that, The device includes: The acquisition module is used to acquire real-time running data of multiple cloud games of a target tenant in the cloud gaming platform; the real-time running data includes game type, number of user sessions, event type, GPU utilization, video memory usage, video encoding frame rate, and end-to-end latency; the first determination module is used to input the real-time running data of the multiple cloud games into a pre-trained load prediction model, process the real-time running data of the multiple cloud games through the load prediction model, and determine the total predicted GPU resource demand of the multiple cloud games and the predicted GPU resource demand value of each cloud game within a future target time window; An adjustment module is used to adjust the total resource volume of the target tenant's GPU resource pool based on the predicted total GPU resource demand of multiple cloud games. Specifically, the predicted total GPU resource demand of multiple cloud games is compared with the current total resource volume in the GPU resource pool to determine the resource volume comparison result. If the resource volume comparison result indicates insufficient resources, a resource expansion instruction is generated, including a first number of GPU instances. If the resource volume comparison result indicates excess resources, a resource reduction instruction is generated, including a second number of GPU instances. The resource expansion or reduction instruction is executed to adjust the total resource volume of the GPU resource pool. The second determining module is used to determine the target GPU resource adjustment amount for each cloud game within the adjusted GPU resource pool, based on the predicted GPU resource requirements, preset frame rate, and preset latency target value. Each cloud game corresponds to a game type-resource requirement profile, representing the attribute requirements of the GPU instances matched to the cloud game. Specifically, based on the predicted GPU resource requirements for each cloud game, a basic GPU resource adjustment amount is determined; GPU instances are scheduled for the cloud game based on the basic GPU resource adjustment amount, and the real-time average frame rate and real-time latency of the cloud game after instance scheduling are monitored; based on the comparison results of the real-time average frame rate, preset frame rate, and real-time latency with the preset latency, the target GPU resource adjustment amount for the cloud game is determined. The third determining module is used to determine the target GPU instance of the cloud game from the resource pool after the total resource volume is adjusted, based on the target GPU resource adjustment amount and the game type-resource requirement profile of the cloud game; the target GPU instance is matched with the target GPU resource adjustment amount and attribute requirements of the cloud game; wherein, based on the GPU instance attribute requirements defined by the game type-resource requirement profile, candidate GPU instances that meet the GPU instance attribute requirements are selected from the resource pool; according to the preset cost-optimal strategy and / or performance-optimal strategy corresponding to the cloud game, the target GPU instance that meets the target GPU resource adjustment amount is selected from the candidate GPU instances.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the GPU resource dynamic allocation method as described in any one of claims 1 to 4 are performed.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the GPU resource dynamic allocation method as described in any one of claims 1 to 4.

Images