Game rendering and llm inference cooperative scheduling method in single-gpu environment, medium and equipment

Abstract

The application provides a game rendering and LLM inference cooperative scheduling method, medium and equipment in a single GPU environment. The method comprises: an offline preparation stage: obtaining a plurality of available time windows of large language model inference and corresponding required layer skipping ranges, and setting corresponding neural network adapters according to the layer skipping ranges; an online runtime scheduling stage: predicting the available time of the next inference window, and determining the layer skipping range and the corresponding neural network adapter according to the predicted available time; decomposing the large language model inference process into a plurality of fine-grained rendering subtask sequences and coarse-grained rendering subtask sequences, and scheduling and distributing the fine-grained rendering subtasks and the coarse-grained rendering subtasks according to whether there is a GPU gap in the current frame rendering task operation. The application uses a resource-oriented layer skipping adapter, maximizes the use of the resource scheduling gap of the LLM, and effectively solves the resource conflict problem of game rendering and LLM inference in a single GPU environment.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to the field of large language model control technology. Background Technology

[0002] With the development of artificial intelligence technology, Large Language Models (LLMs) are increasingly being applied to modern video games to control the combat behavior of non-player characters (NPCs) or generate dynamic dialogues, thereby enhancing game immersion and interactivity. However, most users' personal computers are typically equipped with only one GPU, requiring the simultaneous handling of high-load game rendering tasks and computationally intensive LLM inference tasks on the same hardware. This task co-location faces significant challenges: on the one hand, game rendering has extremely high real-time requirements, typically needing to be stable at 60 FPS, meaning a processing time of no more than 16.6 milliseconds per frame; on the other hand, LLM inference needs to simulate the action frequency (APM) of players at different skill levels, for example, requiring a response time of 200 milliseconds in the scenario of a professional player.

[0003] Existing solutions have significant drawbacks. Using cloud-based LLM services typically results in network latency ranging from 20 to 110 milliseconds or even higher, making it difficult to meet the real-time response requirements of high-APM game scenarios, and incurring high long-term operating costs. Using local co-location, due to limited and dynamically changing computing resources, direct execution leads to severe resource contention, causing game stuttering or inference timeouts. Furthermore, existing LLM acceleration technologies primarily employ confidence-based dynamic layer skipping methods (such as LITE or CALM), which determine whether to exit early based on the confidence level of the token. This mechanism cannot guarantee inference time will conform to a strict rendering idle window. If layers are skipped early to force compliance with time constraints, layers deemed important by the model are often skipped, leading to a significant drop in inference accuracy. Therefore, there is currently a lack of a scheduling scheme that can adapt to the game's dynamic headroom, ensuring both dual-task latency targets and maintaining high accuracy. Summary of the Invention

[0004] This application provides a method, medium, and device for collaborative scheduling of game rendering and LLM inference in a single GPU environment, which is used to solve the resource conflict problem between game rendering and large language model inference in a single GPU environment.

[0005] In a first aspect, embodiments of this application provide a method for collaborative scheduling of game rendering and LLM inference in a single GPU environment, comprising: an offline preparation stage: based on the target game, obtaining multiple available time windows for large language model inference and corresponding required layer skip ranges, and setting corresponding neural network adapters according to each layer skip range; an online runtime scheduling stage: predicting the available time of the next inference window, and determining the layer skip ranges and corresponding neural network adapters according to the predicted available time; decomposing the large language model inference process into multiple fine-grained rendering subtask sequences and coarse-grained rendering subtask sequences, and scheduling and distributing fine-grained rendering subtasks and coarse-grained rendering subtasks according to whether there is a GPU gap in the current frame rendering task.

[0006] In one implementation of the first aspect, obtaining multiple available time windows and corresponding required layer skip ranges for large language model inference includes: obtaining the execution time of the rendering task running the game; obtaining the minimum and maximum available time windows for large language model inference under different game loads and operations per minute; obtaining the execution time of each layer of the large language model; and obtaining the layer skip range required to satisfy each minimum and maximum available time window based on the execution time of each layer.

[0007] In one implementation of the first aspect, setting the corresponding neural network adapter according to the skip range of each layer includes: constructing a similarity heatmap of the output tensors of each layer of the large language model; obtaining the continuous layer with the highest similarity to the skip range of the layer from the similarity heatmap as the skip object; obtaining a pre-trained neural network adapter that matches the skip object, and simulating the function of the skipped layer through the neural network adapter.

[0008] In one implementation of the first aspect, the method for obtaining the pre-trained neural network adapter that matches the skipped object is as follows: obtaining the minimum mean square error between the output of each neural network adapter and the output of the skipped layer in the large language model; and using the neural network adapter corresponding to the minimum value as the neural network adapter that matches the skipped object.

[0009] In one implementation of the first aspect, predicting the available time of the next inference window includes: collecting the total rendering idle time within multiple historical inference windows of the large language model; inputting the total rendering idle time into a pre-trained linear regression model; and predicting the available time of the next inference window through the output of the linear regression model.

[0010] In one implementation of the first aspect, the step of scheduling and distributing fine-grained rendering subtasks and coarse-grained rendering subtasks based on whether there is a GPU gap in the current rendering subtask includes: when it is detected that the current frame rendering task is running and there is a GPU gap, scheduling and distributing fine-grained rendering subtasks with task times shorter than the minimum inter-frame gap; when it is detected that the current frame rendering task has completely ended, scheduling and distributing coarse-grained rendering subtasks containing multiple layers to utilize the idle time between frames until the next frame rendering task begins.

[0011] In one implementation of the first aspect, the thread state of the game engine's rendering tasks is monitored through hooking or integration.

[0012] Secondly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the game rendering and LLM inference co-scheduling method described above in a single GPU environment.

[0013] Thirdly, embodiments of this application provide a computer program product, which includes computer program code. When the computer program code is run on a computer, the computer implements the game rendering and LLM inference co-scheduling method described above in a single GPU environment.

[0014] Fourthly, embodiments of this application provide an electronic terminal, including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the game rendering and LLM inference co-scheduling method in a single GPU environment as described above.

[0015] The game rendering and LLM inference collaborative scheduling method, medium, and device provided in this application embodiment have the following beneficial effects:

[0016] This application effectively solves the resource conflict problem between game rendering and LLM inference in a single-GPU environment by skipping the adapter through a resource-oriented layer and maximizing the use of LLM resource scheduling gaps. Attached Figure Description

[0017] Figure 1 The diagram shown is an overall flowchart of a game rendering and LLM inference co-scheduling method in a single GPU environment according to an embodiment of this application.

[0018] Figure 2 The flowchart shown is a process for obtaining the layer skip range in a game rendering and LLM inference co-scheduling method under a single GPU environment according to an embodiment of this application.

[0019] Figure 3The flowchart shown is a process for setting corresponding neural network adapters for the skip range of each layer in a game rendering and LLM inference co-scheduling method under a single GPU environment according to an embodiment of this application.

[0020] Figure 4 The flowchart shown is a detailed flowchart of the neural network adapter corresponding to the skip range matching of each layer in the game rendering and LLM inference co-scheduling method in a single GPU environment according to an embodiment of this application.

[0021] Figure 5 The flowchart shown is a process for predicting the available time of the next inference window in a game rendering and LLM inference co-scheduling method under a single GPU environment, according to an embodiment of this application.

[0022] Figure 6 The diagram illustrates the principle of scheduling and distributing fine-grained and coarse-grained rendering subtasks in a single-GPU environment game rendering and LLM inference collaborative scheduling method according to an embodiment of this application.

[0023] Figure 7 The diagram shown is a schematic diagram of the principle structure of an electronic terminal in one embodiment of this application. Detailed Implementation

[0024] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0025] This embodiment provides a method, medium, and device for collaborative scheduling of game rendering and LLM inference in a single GPU environment. It forms an efficient scheduling scheme for collaborative resource management of tasks by involving a resource-oriented layer skipping adapter and an LLM (Large Language Model) scheduler that maximizes the utilization of the headroom (inference time window). This embodiment abandons the traditional confidence-based layer skipping strategy and instead adopts a resource-based layer skipping strategy, coupled with a fine-grained scheduler that maximizes the utilization of fragmented time.

[0026] The following will refer to the appendices in the embodiments of this application. Figure 1 To be continued Figure 6This application provides a detailed description of the technical solution for the game rendering and LLM inference co-scheduling method in a single-GPU environment, as described in the embodiments of this application. This allows those skilled in the art to understand and implement the game rendering and LLM inference co-scheduling method in a single-GPU environment without inventive effort.

[0027] This embodiment provides a method for collaborative scheduling of game rendering and LLM inference in a single-GPU environment. Figure 1 The flowchart shown is a presentation of the game rendering and LLM inference co-scheduling method in a single-GPU environment, as described in this application embodiment. Figure 1 As shown in the embodiments of this application, the game rendering and LLM inference collaborative scheduling method in a single GPU environment includes the following steps S100 to S300.

[0028] Offline preparation phase:

[0029] Step S100: Based on the target game, obtain multiple available time windows for large language model inference and corresponding required layer skip ranges, and set the corresponding neural network adapter according to each layer skip range.

[0030] Online runtime scheduling phase:

[0031] Step S200: Predict the available time of the next inference window, and determine the layer skip range and the corresponding neural network adapter based on the predicted available time;

[0032] Step S300: Decompose the large language model inference process into multiple fine-grained rendering subtask sequences and coarse-grained rendering subtask sequences, and schedule and distribute fine-grained rendering subtasks and coarse-grained rendering subtasks according to whether there is a GPU gap in the current frame rendering task.

[0033] The following combination Figures 2 to 6 The above steps S100 to S300 of the game rendering and LLM inference co-scheduling method in a single GPU environment in this embodiment will be described in detail.

[0034] Offline preparation phase:

[0035] Step S100: Based on the target game, obtain multiple available time windows for large language model inference and corresponding required layer skip ranges, and set the corresponding neural network adapter according to each layer skip range.

[0036] This embodiment addresses the trade-off between accuracy and speed in large language model inference by constructing a resource-oriented layer-skipping adapter. First, it analyzes the output similarity of each Transformer layer in the LLM offline to identify the range of low-importance layers that can be skipped under specific computational resources. Unlike existing technologies that directly discard these layers, this embodiment pre-trains a lightweight feedforward neural network (Adaptor) to learn and approximate the knowledge representation of the skipped layers. At runtime, instead of judging based on the confidence level of the token content, it directly selects the corresponding skipping strategy and connects to the appropriate adapter based solely on the available GPU computing time resources. This ensures that inference time is strictly controllable while minimizing accuracy loss.

[0037] Figure 2 The flowchart shown is a process for obtaining the layer skip range in a game rendering and LLM inference co-scheduling method under a single GPU environment, according to an embodiment of this application. Figure 2 As shown, in one implementation of this embodiment, obtaining multiple available time windows for large language model inference and the corresponding required layer skipping range includes:

[0038] Step S110: Obtain the execution time of the rendering task running the game, and obtain the minimum and maximum available time windows for large language model inference under different game loads and operations per minute.

[0039] Step S120: Obtain the execution time of each layer of the large language model, and obtain the layer skip range required to satisfy each minimum available time window and the maximum available time window based on the execution time of each layer.

[0040] The available time window is the idle time within a single frame cycle, that is, the remaining time after deducting rendering time, input response time, and system background overhead from the single frame budget time. The available time window is calculated as follows:

[0041] Available time window = single frame budget time - rendering task execution time - APM-related response time - system background overhead.

[0042] The minimum available time window is calculated by taking the peak rendering time and the maximum APM response time, and then substituting them into the above formula to obtain the minimum idle time; the maximum available time window is calculated by taking the valley rendering time and the minimum APM response time, and then substituting them into the above formula to obtain the maximum idle time.

[0043] The goal of layer skipping is to keep the total inference time of the LLM between the minimum and maximum available time windows.

[0044] In this embodiment, the execution time of each layer of the large language model is obtained as follows: for example, the execution time of each layer is obtained through offline benchmark measurement and online deployment measurement. In the offline scenario, the profile function of transformers or torch.profiler is used to fix the input token length and disable layer skipping, and the average time of the embedding layer, each Transformer layer and the output layer is recorded. In the online scenario, combined with the game running environment, tools such as NVIDIA Nsight Systems are used to monitor the time fluctuation of each layer under different game loads and APM conditions, and the influencing factors such as input sequence length and hardware resource status are labeled.

[0045] In this embodiment, the method for determining the layer skipping range is as follows:

[0046] First, calculate the baseline total inference time of the LLM without layer skipping. Then, based on the minimum / maximum available time window provided by the game, deduce the time range to be reduced by skipping layers. Sort the Transformer layers by time from high to low, prioritize skipping the layers with longer time consumption, and accumulate the time consumption of the sorted layers to determine the minimum and maximum number of skipped layers that meet the time window constraints. Define the final layer skipping range. In actual deployment, the number of skipped layers can be dynamically adjusted according to the real-time game load, while taking into account model accuracy and controlling the maximum number of skipped layers to not exceed a reasonable proportion of the total number of layers.

[0047] Figure 3 This is a flowchart illustrating the process of setting corresponding neural network adapters for the skip range of each layer in a single-GPU environment game rendering and LLM inference co-scheduling method according to an embodiment of this application. Figure 3 As shown, in one implementation of this embodiment, setting the corresponding neural network adapter according to the skip range of each layer includes:

[0048] Step S130: Construct a similarity heatmap of the output tensors of each layer of the large language model.

[0049] In this embodiment, the process of constructing a similarity heatmap of the output tensors of each layer of the large language model, exemplarily, includes:

[0050] 1) Capture Tensors: Obtain the final output tensors of each Transformer layer (and embedding / output layer) during LLM inference through the frame hook function to ensure that the data is the effective feature output of the layer;

[0051] 2) Tensor preprocessing: Dimensionality reduction (e.g., taking the mean of sequence dimensions) and normalization are performed on high-dimensional tensors to transform them into feature representative vectors for each layer, eliminating the interference of numerical scale and sequence length on similarity;

[0052] 3) Calculate the similarity matrix: Using cosine similarity as the core indicator, calculate the similarity values ​​between all layers to generate a layer × layer similarity matrix;

[0053] 4) Visualization: The similarity matrix is ​​plotted as a heatmap, and the degree of similarity between layers is intuitively displayed by the intensity of the color (e.g., red for high and blue for low). The layer number and similarity value are labeled to assist in the interpretation.

[0054] Step S140: Obtain the continuous layer with the highest similarity to the layer skip range from the similarity heatmap as the skip target.

[0055] First, the number of anchor layer skips is constrained: First, clarify the previously calculated range of the number of layers to skip [K_min, K_max] (minimum / maximum number of layers to skip), and determine that the number of consecutive layers to be selected must fall within this range. For example, if 2-7 layers need to be skipped, then only select consecutive layer combinations of 2-7 layers.

[0056] Then, extract the inter-layer similarity data from the heatmap: derive the pairwise similarity values ​​of all layers from the similarity heatmap, and focus on the similarity features of continuous layers: calculate the average similarity within each combination of continuous layers, such as the average similarity of continuous layers i~j = the mean of the pairwise similarities of all layers within the combination. The higher the average similarity, the more redundant the feature output of the group of layers is, and the less impact skipping will have on the model accuracy.

[0057] Next, select high similarity continuous layer combinations: sort all continuous layer combinations that meet the layer number range from high to low according to average similarity, and select the top-ranked combination as the core skip target; if there are multiple combinations with similar average similarity (difference < 0.05), prioritize the higher-level continuous layer (LLM has higher feature redundancy, and skipping it has less impact on basic semantic understanding).

[0058] Finally, validation and fine-tuning can be performed: select the continuous layers as the skip targets, test whether the LLM inference time falls within the minimum / maximum available time window, and at the same time verify the semantic accuracy of the text generated by the model (such as BLEU value, human semantic score); if the accuracy loss exceeds the acceptable range, replace it with a combination of continuous layers with the second highest similarity.

[0059] Step S150: Obtain a pre-trained neural network adapter that matches the skipped object, and simulate the function of the skipped layer through the neural network adapter.

[0060] When matching a pre-trained neural network adapter to a selected continuous layer to be skipped, it is necessary to first clarify the core parameters such as the output dimension and number of attention heads of the skipped layer to ensure that the adapter's input and output are aligned with the feature dimensions of the layers before and after the skipped layer. Lightweight pre-trained adapters such as LoRA should be preferred, or the output of the original LLM without layer skipping can be used as a supervision signal to customize the training adapter to learn the feature transformation logic of the skipped layer. Then, the adapter is embedded into the LLM inference process, and its output tensor is used to replace the calculation results of the skipped layer to complete the functional simulation. Finally, it is verified whether the total inference time of the LLM after integrating the adapter meets the time window constraint. At the same time, the accuracy of the functional simulation is verified by indicators such as semantic similarity and generated text score. If necessary, the adapter parameters are fine-tuned to achieve a balance between time consumption control and accuracy loss.

[0061] Figure 4 The diagram shown is a flowchart illustrating the specific process of the neural network adapters corresponding to the skip range matching of each layer in the game rendering and LLM inference co-scheduling method under a single GPU environment, as described in an embodiment of this application. Figure 4 As shown, in one implementation of this embodiment, the method for obtaining a pre-trained neural network adapter that matches the skipped object is as follows:

[0062] Step S151: Obtain the minimum mean square error between the output of each neural network adapter and the output of the skipped layer in the large language model.

[0063] In this embodiment, the original output tensor of the skipped layer when skipping layers in LLM is obtained as the benchmark; then each adapter to be matched is embedded into the inference process, and the corresponding simulation tensor is output. The element-wise mean square error (MSE=1 / n×Σ(simulated value-benchmark value)²) is calculated between the output of each adapter and the benchmark tensor; after traversing all adapters, the minimum MSE value is extracted (this value corresponds to the adapter with the best simulation accuracy). During the calculation, it is necessary to ensure that the tensor dimensions are completely aligned, and the mean value is taken from multiple tests to eliminate random errors. At the same time, it is verified that the integrated adapter still meets the LLM inference time window constraints.

[0064] Step S152: Use the neural network adapter corresponding to the minimum value as the neural network adapter that matches the skipped object.

[0065] In this embodiment, during the offline preparation phase, performance analysis is first performed on the target game and the LLM. The game is run and the execution time of rendering tasks is recorded. The minimum and maximum available time windows (headroom) for LLM inference are calculated under different game loads and APM (Actions Per Minute) settings. Next, the execution time of each layer of the LLM is measured, and the range of layers that need to be skipped to meet these time windows is calculated, for example, from skipping M layers to skipping N layers.

[0066] Subsequently, a similarity heatmap of the output tensors of each LLM layer is constructed. For each possible number of skipped layers (e.g., skipping 4 layers, 8 layers, etc.), the consecutive layers with the highest similarity in the heatmap are selected as the skipped layers. For each given skip configuration, a corresponding feedforward neural network adapter is trained. By minimizing the mean squared error (MSE) between the adapter output and the output of the skipped layer in the original model, the adapter can simulate the function of the skipped layer. These adapters have a small number of parameters, are highly efficient to train, and are packaged with the game after preparation.

[0067] Online runtime scheduling phase:

[0068] Step S200: Predict the available time of the next inference window, and determine the layer skip range and the corresponding neural network adapter based on the predicted available time.

[0069] Idle time in game rendering tasks exists not only between frames (inter-rendering headroom) but also in tiny gaps within the rendering task itself (intra-rendering headroom), such as gaps caused by engine optimization or CPU-GPU synchronization. To utilize these fragmented resources, the scheduler first uses a linear regression model to accurately predict the total available idle time for the next window based on historical data from the past few execution windows.

[0070] Figure 5 This is a flowchart illustrating the prediction of the available time of the next inference window in a single-GPU environment game rendering and LLM inference co-scheduling method according to an embodiment of this application. Figure 5 As shown, in one implementation of this embodiment, the available time for predicting the next inference window includes:

[0071] Step S210: Collect the total rendering idle time within multiple historical inference windows of the large language model;

[0072] Step S220: Input the total rendering idle time into the pre-trained linear regression model, and predict the available time of the next inference window through the output of the linear regression model.

[0073] Specifically, in this embodiment, the historical reasoning window is divided according to the game operation scenario (such as single battle or casual exploration). Each window contains N consecutive game frames (N is recommended to be ≥60 to cover the complete load fluctuation cycle). Each window is marked with a unique identifier (such as timestamp or load level), and the average APM value and game load type (high / medium / low) within the window are recorded synchronously.

[0074] Next, core time-consuming data within a single window is collected: using game engine performance tools (such as UnityProfiler, Unreal Insights) or third-party monitoring tools (such as NVIDIA Nsight Systems), the single-frame budget time, rendering task time, APM-related response time, and system background overhead are extracted for each frame within each window; the idle time of each frame is calculated using the formula: Single-frame rendering idle time = Single-frame budget time - Rendering time - APM response time - System background overhead.

[0075] Then, the total rendering idle time of a single window is summarized: the rendering idle time of all frames within a single inference window is summed to obtain the total rendering idle time of that window, with the formula: Total rendering idle time of window = Σ (rendering idle time of a single frame); the result is then associated with the window identifier and load / APM information and stored to form a historical data list.

[0076] It can remove abnormal frame data from the historical window (such as negative idle time caused by game lag, or extreme time-consuming frames caused by background processes competing for resources); it can also average the data from multiple historical windows with the same load and APM level to improve data reliability.

[0077] Using historical total rendering idle time (and load / APM features) as input features, and the total rendering idle time of the next window as labels, a linear regression model is trained. The linear relationship between the features and labels is fitted using the least squares method to verify the model's goodness of fit (e.g., R² value), ensuring that the prediction error is within an acceptable range. The total rendering idle time (and current game load, APM) of several recently collected historical inference windows is input into the pre-trained model, and the model output is the predicted available time for the next inference window. The average of multiple prediction results can be taken to reduce random errors.

[0078] In this embodiment, during the online runtime scheduling phase, idle time prediction is performed first. For example, the scheduler collects the total rendering idle time over the past three inference windows (each window contains dozens of game frames) and inputs it into a linear regression (LR) model to predict the available time budget for the next inference window. This prediction model has extremely low computational overhead (approximately 0.9ms), making it very suitable for real-time operation. Based on the predicted available time, the scheduler selects the most suitable layer skipping strategy from a pre-prepared configuration, such as skipping 8 layers and loading the corresponding adapters.

[0079] Step S300: Decompose the large language model inference process into multiple fine-grained rendering subtask sequences and coarse-grained rendering subtask sequences, and schedule and distribute fine-grained rendering subtasks and coarse-grained rendering subtasks according to whether there is a GPU gap in the current frame rendering task.

[0080] After determining the layer skipping strategy based on the prediction results, the scheduler divides the LLM inference task into extremely fine-grained subtasks (such as a single Transformer layer or even smaller operators). During runtime, the scheduler monitors the rendering engine's status in real time. When it detects that a rendering task is in progress but the GPU is temporarily idle, it inserts fine-grained subtasks to fill intra-frame gaps; when a rendering task completes and enters an inter-frame gap, it switches to coarse-grained subtasks to utilize larger blocks of time. This mechanism ensures that the LLM can run opportunistically, without blocking game rendering, while fully utilizing all available computing power.

[0081] Figure 6 This diagram illustrates the principle of scheduling and distributing fine-grained and coarse-grained rendering subtasks in a single-GPU environment co-scheduling method for game rendering and LLM inference, according to an embodiment of this application. Figure 6 As shown, in one implementation of this embodiment, the step of scheduling and distributing fine-grained and coarse-grained rendering subtasks based on whether there is GPU downtime during the current rendering subtask execution includes:

[0082] 1) When it is detected that the current frame rendering task is running and there is a GPU gap, schedule and distribute fine-grained rendering subtasks whose task time is less than the minimum inter-frame gap.

[0083] 2) When it is detected that the current frame rendering task has completely ended, a coarse-grained rendering subtask containing multiple layers is scheduled and distributed to take advantage of the idle time between frames until the next frame rendering task begins.

[0084] In this embodiment, the scheduler decomposes the LLM inference process into multiple sub-task sequences. The rendering thread status of the game engine is monitored through hooking or integration.

[0085] When a rendering subtask is detected running but there is a GPU gap (Intra-gap), the scheduler distributes fine-grained subtasks with extremely short execution times (such as a single decoding layer or attention layer), strictly constraining the subtask time to be less than the minimum inter-frame gap to prevent rendering latency. When the current frame rendering is completely finished (Inter-gap), the scheduler distributes coarse-grained subtasks containing multiple layers to utilize the larger idle time between frames until the next frame rendering begins. If a sudden rendering load spike is detected during execution, the scheduler dynamically adjusts the skipping strategy when generating the next token to ensure that real-time requirements are continuously met.

[0086] The implementation process of the game rendering and LLM inference collaborative scheduling method in a single GPU environment described in this application embodiment is as follows:

[0087] 1. User (game developer) interaction with the offline adapter:

[0088] a) Users first perform performance analysis on the rendering time of the game in typical scenarios and calculate the minimum and maximum rendering idle time under different APM settings;

[0089] b) Based on the measured execution time of each layer of the LLM, the user determines the range of layers that need to be skipped (e.g., skipping layers M to N), and for each type of skipped layer configuration, trains the corresponding neural network adapter using the dataset to distill the knowledge of the skipped layers;

[0090] c) Users package the trained adapter files along with the game and LLM model for deployment on the terminal.

[0091] 2. Interaction between the user (end-player) and the system initialization module:

[0092] a) This process is transparent to the end player. When the player starts the game, the game engine, LLM model weights, and pre-trained adapter parameters are automatically loaded into the video memory, without the player needing to manually configure hardware parameters or participate in the compilation process.

[0093] 3. Interaction between the user (end player / game engine) and the runtime scheduling module:

[0094] a) Player actions in the game (such as attacking or talking) trigger LLM inference requests. Based on historical data from the past three execution windows, a linear regression model is used to predict the available rendering idle time for the next window.

[0095] b) Based on the prediction results, automatically select the most suitable layer skipping strategy (e.g., skip 8 layers) and divide the inference task;

[0096] c) Monitor the rendering status in real time, dynamically submit LLM subtasks during rendering task intervals (Inter-gap) or intra-gap, and return the final generated Action or dialogue result to the game engine for execution.

[0097] As shown above, this embodiment provides a resource-oriented large language model (LLM) and game rendering task collaborative scheduling method for single-GPU environments. Its primary technical key point is the proposal of a layer skipping mechanism based on available computing resources rather than token confidence. This mechanism pre-analyzes the characteristics of the large language model's Transformer layers to determine the specific range of layers to skip under different resource budgets, and designs a dedicated neural network adapter to replace the skipped layers. By training a lightweight feedforward network (FFN) to fit the functionality of the skipped layers, this adapter construction method significantly reduces the inference accuracy loss caused by forced layer skipping, ensuring that inference is completed within a strict time window.

[0098] This embodiment also provides a fine-grained scheduling strategy to maximize the utilization of fragmented GPU resources, particularly for the exploitation and utilization of intra-rendering headroom. This strategy protects a dynamic task splitting and distribution mechanism: LLM inference tasks are split into extremely fine-grained subtasks (such as a single Transformer layer or Attention / FFN sub-layers). During runtime, the scheduler monitors the state of the rendering thread in real time. When it detects that a rendering task is in progress but the GPU is briefly idle, it inserts a fine-grained subtask; when it detects a large gap between frames, it switches to a coarse-grained subtask. Furthermore, the scheduler includes a linear regression prediction model based on the execution window to smooth game frame rate fluctuations and accurately predict the available computing resources for the next time window.

[0099] The game rendering and LLM inference co-scheduling method described in this embodiment enables efficient co-location of AAA games (such as *Black Myth: Wukong* and *Red Dead Redemption 2*) and high-performance LLMs (such as Llama3-8B) on a single consumer-grade GPU (such as the Nvidia RTX 4090). Experiments demonstrate that this method simultaneously meets the latency target (60 FPS) for game rendering and the frequency target (up to 300 APM) for LLM inference in all test scenarios. Compared to existing simple co-location or confidence-based layer skipping methods, this method improves the utilization of rendering headroom resources by up to 28.6%. More importantly, thanks to the resource-oriented adapter distillation technology, under the same stringent latency constraints, the game rendering and LLM inference co-scheduling method described in this embodiment reduces the accuracy loss of LLM inference by up to 86.3% compared to the existing forced truncation layer skipping method, thereby ensuring the accuracy and consistency of agent behavior in the game.

[0100] The scope of protection for the game rendering and LLM inference co-scheduling method in a single GPU environment described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0101] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the game rendering and LLM inference co-scheduling method provided in any embodiment of this application under a single GPU environment.

[0102] In this embodiment, any combination of one or more storage media can be used. The storage medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0103] This application also provides an electronic terminal. Figure 7 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 7 As shown, the electronic terminal in this embodiment includes at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. The various components in the device are coupled together via a bus system 404. It is understood that the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as bus systems.

[0104] The user interface 405 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0105] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0106] In this embodiment of the invention, the memory 402 is used to store various types of data to support the operation of the electronic terminal 400. Examples of this data include: any executable program for operation on the electronic terminal 400, such as the operating system 4021 and application programs 4022; the operating system 4021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 4022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The game rendering and LLM inference co-scheduling method provided in this embodiment of the invention can be included in the application program 4022.

[0107] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in software form. The processor 401 described above is a general-purpose GPU capable of rendering tasks and large model language tasks. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 401 can be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0108] In an exemplary embodiment, the electronic terminal 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0109] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figures 1 to 7 The method of any of the embodiments shown.

[0110] In summary, this application effectively solves the resource conflict problem between game rendering and LLM inference in a single-GPU environment by skipping the adapter through a resource-oriented layer and maximizing the utilization of LLM resource scheduling gaps. Therefore, this application effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0111] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for collaborative scheduling of game rendering and LLM inference in a single GPU environment, characterized in that, include: Offline preparation phase: Based on the target game, obtain multiple available time windows for large language model inference and the corresponding required layer skip ranges, and set the corresponding neural network adapters according to each layer skip range; Online runtime scheduling phase: Predict the available time of the next inference window, and determine the layer skip range and the corresponding neural network adapter based on the predicted available time; The large language model inference process is decomposed into multiple fine-grained rendering subtask sequences and coarse-grained rendering subtask sequences. Based on whether there is GPU gap in the current frame rendering task, the fine-grained rendering subtasks and coarse-grained rendering subtasks are scheduled and distributed.

2. The game rendering and LLM inference collaborative scheduling method in a single GPU environment according to claim 1, characterized in that, The process of obtaining multiple available time windows for large language model inference and the corresponding required layer skip range includes: Get the execution time of the rendering task running the game, and get the minimum and maximum available time windows for large language model inference under different game loads and operations per minute; The execution time of each layer of the large language model is obtained, and the layer skipping range required to satisfy the minimum available time window and the maximum available time window is obtained based on the execution time of each layer.

3. The game rendering and LLM inference collaborative scheduling method in a single GPU environment according to claim 1 or 2, characterized in that, The step of setting the corresponding neural network adapter according to the skip range of each layer includes: Construct similarity heatmaps of the output tensors of each layer of the large language model; The consecutive layers with the highest similarity to the layer skip range are selected from the similarity heatmap and used as skip targets. Obtain a pre-trained neural network adapter that matches the skipped object, and simulate the function of the skipped layer through the neural network adapter.

4. The game rendering and LLM inference collaborative scheduling method in a single GPU environment according to claim 3, characterized in that, The method for obtaining a pre-trained neural network adapter that matches the skipped object is as follows: Obtain the minimum mean square error between the output of each neural network adapter and the output of the skipped layer in the large language model; The neural network adapter corresponding to the minimum value is used as the neural network adapter that matches the skipped object.

5. The game rendering and LLM inference collaborative scheduling method in a single GPU environment according to claim 1, characterized in that, The available time for predicting the next inference window includes: Collect the total rendering idle time across multiple historical inference windows of the large language model; The total rendering idle time is input into a pre-trained linear regression model, and the output of the linear regression model is used to predict the available time of the next inference window.

6. The game rendering and LLM inference collaborative scheduling method in a single GPU environment according to claim 1, characterized in that, The step of scheduling and distributing fine-grained and coarse-grained rendering subtasks based on whether there is GPU downtime during the current rendering subtask execution includes: When it is detected that the current frame rendering task is running and there is a GPU gap, fine-grained rendering subtasks with task time less than the minimum inter-frame gap are scheduled and distributed. When it is detected that the current frame rendering task has completely ended, a coarse-grained rendering subtask containing multiple layers is dispatched to take advantage of the idle time between frames until the next frame rendering task begins.

7. The game rendering and LLM inference collaborative scheduling method in a single GPU environment according to claim 6, characterized in that, Monitor the thread status of the game engine's rendering tasks through hooking or integration.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the game rendering and LLM inference co-scheduling method in a single GPU environment as described in any one of claims 1 to 7.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, enables the computer to implement the game rendering and LLM inference co-scheduling method in a single GPU environment as described in any one of claims 1 to 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the game rendering and LLM inference co-scheduling method in a single GPU environment as described in any one of claims 1 to 7.

Images