Distributed cloud rendering task dynamic allocation method based on reinforcement learning

Abstract

This invention provides a method for dynamic allocation of distributed cloud rendering tasks based on reinforcement learning. The method includes: acquiring cloud rendering requests and generating request parameter records, scheduling context records, and scheduling observation packages; dividing the rendering tasks to be executed into reversible rendering precursor tasks and irreversible final rendering tasks based on the request parameter records, scheduling context records, and scheduling observation packages, calculating branch survival probabilities, and generating state inheritance identifiers and task inheritance mapping sets; inputting the scheduling observation packages, branch survival probabilities, and task inheritance mapping sets into a reinforcement learning policy model, whereby node agents and scheduling coordination agents generate global scheduling action values, task allocation result sets, and scheduling execution schemes; executing reversible rendering precursor tasks according to the scheduling execution scheme, generating continueable intermediate rendering states and writing them into a state inheritance pool, and when conditions are met, selecting a target node to extract the corresponding continueable intermediate rendering states.

Description

Technical Field

[0001] This invention relates to the field of cloud rendering task scheduling technology, and in particular to a method for dynamic allocation of distributed cloud rendering tasks based on reinforcement learning. Background Technology

[0002] With the development of applications such as cloud gaming, digital human interaction, and real-time 3D display, cloud rendering tasks have gradually shifted from traditional offline processing to online dynamic processing oriented towards interactive processes. Existing distributed cloud rendering platforms typically distribute rendering tasks to different cluster nodes based on their computational load, memory usage, network bandwidth, and queuing status. Some solutions introduce intelligent scheduling mechanisms such as reinforcement learning to improve node utilization, shorten waiting time, and alleviate node congestion. Therefore, dynamic task allocation based on node status has become a common technical approach for current cloud rendering scheduling.

[0003] However, existing technologies typically still treat rendering tasks as complete tasks that need to be finished in one go. Scheduling focuses primarily on timely task allocation and efficient node utilization, with less emphasis on considering the continuous changes in scene and interaction states to differentiate between the reusability and validity of rendering tasks. In real-time interactive scenarios, camera state, viewpoint state, lighting state, and interaction branch state are constantly changing. Even if a rendering task dispatched at a certain moment completes its calculations, the output result may no longer correspond to the current scene, forcing it to be discarded or recalculated, thus wasting computing power. Furthermore, existing technologies typically lack mechanisms for inheriting and continuing intermediate rendering results. When scene states change or tasks need to be migrated, the system often has to re-initiate a complete rendering task instead of continuing execution based on existing intermediate results, leading to increased redundant calculations, accumulated delivery delays, and decreased real-time performance.

[0004] Therefore, this invention proposes a method for dynamic allocation of distributed cloud rendering tasks based on reinforcement learning. The information disclosed in the background section is only for enhancing understanding of the background of this disclosure and may therefore contain prior art information that is not common knowledge to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a dynamic allocation method for distributed cloud rendering tasks based on reinforcement learning, thereby solving the technical problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The reinforcement learning-based method for dynamic allocation of distributed cloud rendering tasks includes the following steps: S1. Obtain the cloud rendering request, perform request integrity verification, and generate request parameter records, scheduling context records, and scheduling observation packages; S2. Based on the request parameter record, scheduling context record and scheduling observation package, divide the rendering task to be executed into reversible rendering precursor task and irreversible final rendering task, calculate the branch survival probability, and generate state inheritance identifier and task inheritance mapping set. S3. Input the scheduling observation package, branch survival probability and task inheritance mapping set into the reinforcement learning policy model, and generate global scheduling action values, task allocation result set and scheduling execution scheme by the node agent and the scheduling coordination agent. S4. Execute the reversible rendering precursor task according to the scheduling execution plan, generate a continueable intermediate rendering state and write it into the state inheritance pool. When the branch survival probability and state integrity rate meet the conditions, select the target node to extract the corresponding continueable intermediate rendering state and start the irreversible final rendering task. S5. Monitor scene state changes, perform continuation assignment for unfinished rendering tasks, output target rendering results, verify the validity of execution results delivery, generate scheduling feedback records and reward values, and update the reinforcement learning strategy model based on the reward values.

[0007] S1 specifically includes: acquiring cloud rendering requests; performing request integrity verification on lens identifiers, viewpoint parameters, image quality levels, latency constraints, output specifications, request reception time, and request source nodes, and generating request parameter records; reading the scene state sequence, interaction state sequence, and cluster node state set corresponding to the cloud rendering request based on the request parameter records, performing unified timeline alignment, missing segment removal, and abnormal state marking, and generating scheduling context records; mapping each cluster node to a node agent in a multi-agent system, setting up a scheduling coordination agent, constructing a node agent state vector set and a global scheduling observation set based on the request parameter records and scheduling context records, and generating a scheduling observation package.

[0008] S2 specifically includes: performing task-level processing on the rendering task to be executed based on request parameter records, scheduling context records, and scheduling observation packages; distinguishing between reversible rendering precursor tasks and irreversible final rendering tasks; generating task-level records; calculating branch survival probabilities for each reversible rendering precursor task based on the camera change trends, viewpoint movement trends, and lighting change trends in the scene state sequence, as well as the user operation trends and object response trends in the interaction state sequence; and establishing state inheritance relationships between each reversible rendering precursor task and its corresponding irreversible final rendering task, generating state inheritance identifiers and task inheritance mapping sets.

[0009] S3 specifically includes: inputting the scheduling observation package, branch survival probability table, and task inheritance mapping set into the reinforcement learning policy model; in the multi-agent system, each node agent outputs a local execution score, which is then aggregated by the scheduling and coordinating agent to generate a global scheduling action value; based on the global scheduling action value, combined with computational load, memory usage, cache heat, network bandwidth, and estimated available time, determining the distributed pre-allocation results of the reversible rendering precursor task, the candidate execution node set and state inheritance matching degree of the irreversible final rendering task, and generating a task allocation result set; and based on the task allocation result set, resolving resource conflicts, sorting by time period, and reserving continuation resources to generate a scheduling execution plan.

[0010] S4 specifically includes: according to the scheduling execution plan, distributing each reversible rendering precursor task to the corresponding cluster node for execution, and writing scene state fragments, interaction state fragments, and state inheritance identifiers into the execution context to generate precursor execution records; outputting continuable intermediate rendering states based on the precursor execution records, writing them into the state inheritance pool according to the state inheritance identifiers to generate state inheritance pool records; when the branch survival probability reaches the trigger threshold and the state integrity rate meets the conditions, selecting the target node from the candidate execution node set, extracting the corresponding continuable intermediate rendering states, starting the irreversible final rendering task, and generating final execution records.

[0011] S5 specifically includes: continuously monitoring the scene state sequence and interaction state sequence in the scheduling context record during the rendering process; when a scene state change is identified and the corresponding irreversible final rendering task has not yet been completed, performing continuation allocation for the incomplete rendering task based on the scheduling execution plan, final execution record, state inheritance pool record, task inheritance mapping set, and continuation resource reservation record, and generating a task continuation record; outputting the target rendering result based on the request parameter record, scheduling context record, final execution record, and task continuation record; validating the execution result delivery and generating a scheduling feedback record; calculating the reward value based on the scheduling feedback record, updating the reinforcement learning strategy model based on the reward value, and generating a historical scheduling feedback set.

[0012] The beneficial effects of this invention are as follows: This invention divides the rendering task to be executed into a reversible rendering pre-processing task and an irreversible final rendering task, so that reusable intermediate calculation content is separated from the directly output final calculation content, avoiding the continuous investment of all computing resources in the easily invalidated complete rendering task, and reducing the proportion of invalid calculations from the source.

[0013] This invention establishes a state inheritance identifier, a task inheritance mapping set, and a state inheritance pool, enabling the continuous intermediate rendering states to be directly invoked during subsequent finalization and task continuation processes. This reduces the overhead of repeated calculations after scene state changes and improves the inheritance utilization rate of existing intermediate results.

[0014] This invention introduces node agents and scheduling and coordination agents into a multi-agent system, and combines branch survival probability, state inheritance matching degree, computational load, video memory usage, cache heat, network bandwidth and expected available time to generate a scheduling execution scheme, so that the task allocation process takes into account both delivery effectiveness and node carrying capacity, thereby improving the dynamic allocation accuracy of distributed cloud rendering tasks.

[0015] After identifying changes in scene state, this invention can perform continuation or migration continuation of unfinished rendering tasks by the original target node based on the final execution record, state inheritance pool record, task inheritance mapping set, and continuation resource reservation record. This shortens the recalculation path after scene changes and improves the real-time delivery stability of target rendering results.

[0016] This invention verifies the validity of the delivery of the target rendering result and writes the continuation time, node resource consumption, and delivery result into the scheduling feedback record. Then, it updates the reinforcement learning policy model based on the reward value, so that subsequent scheduling can continuously improve the policy by using both valid delivery samples and failed delivery samples. This improves the scheduling adaptability and resource utilization efficiency in long-term operation.

[0017] This invention constructs a complete closed loop around request parameter recording, scheduling context recording, scheduling observation package, task layering recording, branch survival probability table, task inheritance mapping set, scheduling execution scheme, state inheritance pool recording, and scheduling feedback recording. This makes the input-output relationship of each stage clear and the processing link stable. It can support the continuous scheduling, continuation execution, and strategy iteration of distributed cloud rendering tasks while ensuring sufficient disclosure. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the distributed cloud rendering task dynamic allocation method based on reinforcement learning according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: As Figure 1 As shown, this embodiment provides a method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning, including the following steps: S1. Obtain the cloud rendering request, perform request integrity verification, and generate request parameter records, scheduling context records, and scheduling observation packages; S2. Based on the request parameter record, scheduling context record and scheduling observation package, divide the rendering task to be executed into reversible rendering precursor task and irreversible final rendering task, calculate the branch survival probability, and generate state inheritance identifier and task inheritance mapping set. S3. Input the scheduling observation package, branch survival probability and task inheritance mapping set into the reinforcement learning policy model, and generate global scheduling action values, task allocation result set and scheduling execution scheme by the node agent and the scheduling coordination agent. S4. Execute the reversible rendering precursor task according to the scheduling execution plan, generate a continueable intermediate rendering state and write it into the state inheritance pool. When the branch survival probability and state integrity rate meet the conditions, select the target node to extract the corresponding continueable intermediate rendering state and start the irreversible final rendering task. S5. Monitor scene state changes, perform continuation assignment for unfinished rendering tasks, output target rendering results, verify the validity of execution results delivery, generate scheduling feedback records and reward values, and update the reinforcement learning strategy model based on the reward values.

[0021] S1 specifically includes the following sub-steps: S110. Obtain the cloud rendering request, perform a request integrity check on the cloud rendering request, and generate a request parameter record.

[0022] Specifically, after the cloud rendering request is received by the rendering access service, it is first unpacked according to the preset field template, and the lens identifier, viewpoint parameters, image quality level, latency constraint and output specifications are read. Among them, the lens identifier is used to uniquely indicate the lens to which the current request belongs, the viewpoint parameters are used to characterize the current observation direction, the frustum range and viewpoint change information, the image quality level is used to limit the sampling depth, resolution level and the allowed rendering operator level, the latency constraint is used to limit the latest delivery time of the target rendering result, and the output specifications are used to limit the output resolution, color format and target encoding format.

[0023] The request integrity check includes at least the checks of required fields, field value ranges, and field conflicts. When the lens identifier, viewpoint parameters, image quality level, latency constraints, or output specifications are missing, the cloud rendering request is recorded as an incomplete request and written to the abnormal request flag field. When the combination of image quality level and latency constraints exceeds the preset executable range, the lens identifier, viewpoint parameters, and output specifications are retained, the image quality level is downgraded by 1 level, and a downgrade processing flag is written to the request parameter record. When all fields meet the check conditions, the read fields, along with the request reception time, request source node, and abnormal request flag field, are written to the request parameter record.

[0024] To ensure traceability of subsequent processing, the request parameter record adopts a fixed record structure, which includes at least the lens identifier field, the viewpoint parameter field, the image quality level field, the latency constraint field, the output specification field, the request reception time field, the request source node field, and the abnormal request flag field.

[0025] For example, when a cloud rendering request has a quality level of 3 and a latency constraint of 40 milliseconds, but the preset rule requires that a quality level of 3 corresponds to a latency constraint of at least 60 milliseconds, the quality level is adjusted to level 2, and "quality downgraded by 1 level" is written in the downgrade processing flag, while the other fields remain unchanged. The generated request parameter record in this embodiment continues to be used by S120 for state alignment, by S210 for task splitting, and by S310 for policy decision input.

[0026] S120. Based on the request parameters, read the scene state sequence, interaction state sequence, and cluster node state set corresponding to the cloud rendering request. Perform unified timeline alignment, missing segment removal, and abnormal state marking on the scene state sequence, interaction state sequence, and cluster node state set, and generate scheduling context records.

[0027] Specifically, firstly, using the request reception time field in the request parameter record as a reference timestamp, time deviations are calculated for each scene state segment in the scene state sequence, each interaction state segment in the interaction state sequence, and each node state segment in the cluster node state set. The calculation formula is as follows: In the formula, For the first Time alignment deviation of state segments For the first The original timestamp of the state fragment. For reference timestamps.

[0028] During alignment, if the time alignment deviation is not greater than the preset alignment threshold, the original state segment is retained; if the time alignment deviation is greater than the preset alignment threshold but not greater than the compensation threshold, linear interpolation or nearest neighbor padding is performed on adjacent valid segments, and a compensation mark is written in the corresponding segment; if the time alignment deviation is greater than the compensation threshold, the state segment is recorded as an abnormal state segment.

[0029] To prevent abnormal data from entering subsequent scheduling, state segments with consecutive missing lengths exceeding a preset threshold are directly deleted, and the deletion reason is written into the scheduling context record. For state segments that do not meet the deletion conditions but exhibit jitter, disconnection, jump, or delayed return, the original segment is retained and an abnormal state flag field is written. The abnormal state flags for scene state sequences include at least lens change flags and lighting change flags, the abnormal state flags for interaction state sequences include at least input interruption flags and branch switching flags, and the abnormal state flags for cluster node state sets include at least node offline flags, insufficient video memory flags, and network congestion flags.

[0030] For example: when the time alignment deviation of a node state segment relative to the reference timestamp is 8 milliseconds, the preset alignment threshold is 5 milliseconds and the compensation threshold is 10 milliseconds, interpolation compensation is performed on the node state segment and a compensation flag is written; when the time alignment deviation of another node state segment is 15 milliseconds, the node state segment is recorded as an abnormal state segment.

[0031] After completing the unified timeline alignment, missing segment removal, and abnormal state marking, the processed scene state sequence, interaction state sequence, cluster node state set, and their corresponding marking information are encapsulated into a scheduling context record.

[0032] The generated scheduling context record in this embodiment continues to be used by S130 to construct the observation input of the multi-agent system, by S220 to calculate the branch survival probability, and by S510 to identify scene state changes.

[0033] S130. Map each cluster node in the cluster node state set to a node agent in a multi-agent system, and set up a scheduling and coordination agent to aggregate the decision results of each node agent; based on the request parameter record and the scheduling context record, construct a set of node agent state vectors and a global scheduling observation set, and generate a scheduling observation package.

[0034] Specifically, each cluster node corresponds to a node agent, which is used to characterize the cluster node's capacity to carry reversible rendering precursor tasks within the current request cycle; the scheduling and coordination agent is used to receive the state expression results of each node agent and form a unified observation input for the reinforcement learning policy model.

[0035] To ensure computability for subsequent scheduling, the node agent state vector is composed of fixed fields, including at least the computational load field, memory usage field, cache heat field, network bandwidth field, estimated available duration field, and abnormal state flag field. Among them, the computational load field represents the current node's processing queue and computational usage; the memory usage field represents the available memory for subsequent tasks; the cache heat field represents the current node's hit rate on the current shot's related assets, geometric data, and texture data; the network bandwidth field represents the available transmission capacity between the node and the rendering access service, as well as between nodes; the estimated available duration field represents the estimated duration for the node to maintain stable service within the future scheduling window; and the abnormal state flag field directly inherits the node abnormal state flag from the scheduling context record.

[0036] The scheduling and coordination agent groups the state vectors of each node agent with the scene state sequence and interaction state sequence in the scheduling context record according to the camera identifier and time delay constraint in the request parameter record, forming a global scheduling observation set. When there is an abnormal request mark or an abnormal node state mark, the scheduling and coordination agent does not delete the corresponding observation item, but retains the abnormal mark together, so that the reinforcement learning policy model can use normal samples and abnormal samples together in the training and inference phases.

[0037] Subsequently, the set of node agent state vectors and the global scheduling observation set are encapsulated into a scheduling observation package. The scheduling observation package includes at least camera identifiers, time delay constraints, scene state sequence summaries, interaction state sequence summaries, set of node agent state vectors, and anomaly marker summaries.

[0038] In this embodiment, the generated scheduling observation package is used by S210 to identify reusable rendering content and by S310 to input reinforcement learning policy model, thereby forming a continuous input link from request parameter recording to scheduling context recording and then to scheduling observation package.

[0039] S2 specifically includes the following sub-steps: S210. Based on the request parameter record, scheduling context record, and scheduling observation package, perform task layering processing on the rendering task to be executed, distinguish between reversible rendering precursor tasks and irreversible final rendering tasks, and generate task layering records.

[0040] Specifically, the image quality level field, latency constraint field, and output specification field in the request parameter record are read first. Then, the scene state sequence, interaction state sequence, and cluster node state set in the scheduling context record are read. Finally, the scene state sequence summary, interaction state sequence summary, and node agent state vector set in the scheduling observation package are read to determine the computation target, delivery time limit, and node carrying boundary corresponding to the current cloud rendering request.

[0041] In this embodiment, the reversible rendering pre-processing task is defined as an intermediate computation task that can still be called by the subsequent rendering process after the scene state changes. Its output is not directly delivered as the target rendering result, but serves as the state basis for subsequent final rendering calculations. The irreversible final rendering task is defined as a final state computation task that is directly oriented towards the target rendering result output. Its output enters the final image generation chain and is no longer used as a general intermediate state for subsequent tasks to continue calling.

[0042] Based on the above definition, the rendering tasks to be performed are divided according to their output purpose and reuse conditions: the calculation part that only forms visibility relationships, coarse-grained lighting estimation, reusable sampling buffer, low-resolution occlusion relationships and intermediate shading base map is called the reversible rendering precursor task; the calculation part that forms high-resolution shading results, final pixel fusion results, final denoising results and target encoding output results is called the irreversible final rendering task.

[0043] To avoid instability in task type division, the output is checked simultaneously during the division process to ensure that it meets two conditions: first, whether the output can retain its calling value while detached from the current instantaneous perspective; and second, whether the output can still support the execution of the final draft within the time delay constraints limited by the request parameter record. Only when both conditions are met will this part of the task be written into the reversible rendering precursor task.

[0044] After the division is completed, a task layer record is generated. The task layer record includes at least the task identifier field, task type field, belonging shot identifier field, input status source field, expected output type field, reusable tag field, and subsequent call relationship field.

[0045] For example, when the output of a certain computational part is a low-resolution occlusion relationship graph that can be called by multiple subsequent view branches, it is recorded as a reversible rendering precursor task; when the output of another computational part is the final encoded image data, it is recorded as an irreversible final rendering task.

[0046] In this embodiment, the task hierarchical record continues to be used by S220 to calculate the branch survival probability and by S310 to generate distributed pre-allocation results.

[0047] S220. Based on the camera change trend, viewpoint movement trend, and lighting change trend in the scene state sequence, as well as the user operation trend and object response trend in the interaction state sequence, calculate the branch survival probability for each reversible rendering pre-task and generate a branch survival probability table.

[0048] Specifically, firstly, probability calculation items are established for each reversible rendering precursor task in the task layering record. Then, continuous scene segments and continuous interaction segments corresponding to the task are extracted from the scheduling context record, and camera continuity index, viewpoint stability index, lighting stability index, user operation stability index, and object response stability index are calculated. All of the above indices are normalized before entering the unified calculation to ensure their values ​​are within a preset probability range. The branch survival probability is calculated using the following formula: In the formula, For the first The branch survival probability value corresponding to each reversible rendering precursor task; For the first A shot continuity index corresponding to a reversible rendering precursor task. For the first A viewpoint stability index corresponding to a reversible rendering precursor task. For the first Lighting stability metrics for a reversible rendering precursor task. For the first User operation stability metrics for each reversible rendering precursor task. For the first The object response stability index corresponding to a reversible rendering precursor task. to Here are the weighting coefficients for the corresponding indicators, and .

[0049] In this embodiment, the shot continuity index is used to characterize the consistency of shot identification, shot movement direction and shot switching frequency in adjacent scene segments; the viewpoint stability index is used to characterize the variation range of viewpoint position and view frustum; the illumination stability index is used to characterize the continuity of light source position, illumination intensity and shadow distribution; the user operation stability index is used to characterize the triggering frequency and change range of interactive events per unit time; and the object response stability index is used to characterize the response delay and response consistency of scene object state to interactive events.

[0050] When a scene change marker, branch switching marker, or node offline marker exists in the scheduling context record, the branch survival probability value of the corresponding task is adjusted downward; when the abnormal marker continuously crosses the preset scheduling window, the task is recorded as a low survival probability task.

[0051] After the calculation is completed, a branch survival probability table is generated. The branch survival probability table includes at least the task identifier field, the branch survival probability value field, the probability update time field, the anomaly correction flag field, and the reserved state inheritance flag field.

[0052] For example, when the view stability index and lighting stability index of a certain reversible rendering precursor task are both higher than the preset stability threshold, while the user operation stability index is lower, the influence ratio of the view stability index and lighting stability index can be increased in the weight configuration so that the task still retains a high branch survival probability value.

[0053] In this embodiment, the branch survival probability table continues to be used by S310 to screen candidate execution nodes, by S430 to trigger irreversible final rendering tasks, and by S510 to execute the continuation allocation of unfinished rendering tasks.

[0054] S230. Establish a state inheritance relationship between each reversible rendering precursor task and its corresponding irreversible final rendering task. Generate a unique state inheritance identifier for each group of tasks with inheritance relationship. Associate the task layering record and branch survival probability table according to the state inheritance identifier to generate a task inheritance mapping set.

[0055] Specifically, the expected output type field of each reversible rendering precursor task and the input state source field of each irreversible final rendering task are first read from the task layering record. When the two satisfy the conditions of consistent output type, consistent lens identifier, consistent output specification constraints and valid reusable marker field, it is determined that there is a state inheritance relationship between the two.

[0056] To ensure the uniqueness of subsequent retrieval and invocation, the state inheritance identifier is generated by combining the belonging shot identifier field, the task layer sequence number field, and the time window sequence number field. The belonging shot identifier field is used to represent the shot where the inheritance relationship is located, the task layer sequence number field is used to represent the task position of the inheritance relationship within the current request, and the time window sequence number field is used to represent the scheduling window to which the inheritance relationship belongs.

[0057] After generating the state inheritance identifier, it is written into the reserved field of the state inheritance identifier in the branch survival probability table, and simultaneously written into the subsequent call relationship field in the task hierarchy record, forming a one-to-one inheritance chain. Subsequently, according to the state inheritance identifier, the reversible rendering precursor task, the irreversible final rendering task, and the corresponding branch survival probability value are grouped to generate a task inheritance mapping set. The task inheritance mapping set includes at least the state inheritance identifier field, the precursor task identifier field, the final task identifier field, the inheritance validity field, and the inheritance update time field.

[0058] To ensure the maintainability of inheritance relationships during operation, this embodiment further sets inheritance failure rules and inheritance replacement rules: when the branch survival probability value is lower than the preset inheritance retention threshold, or the corresponding reversible rendering precursor task does not generate a continueable intermediate rendering state in subsequent execution, or a shot switching mark appears in the scheduling context record, the inheritance relationship corresponding to the original state inheritance identifier is deleted, and an failure mark is written in the inheritance validity field; when a new reversible rendering precursor task is regenerated under the same shot identifier, and its expected output type field matches the input state source field of the original irreversible final rendering task again, the original state inheritance identifier is replaced with the new state inheritance identifier, and the task inheritance mapping set is updated.

[0059] In this embodiment, the task inheritance mapping set continues to provide S310 with a set of candidate execution nodes, S420 with a set of intermediate rendering states that can be continued, and S430 and S510 with a set of state inheritance matching.

[0060] Furthermore, the reinforcement learning policy model adopts a multi-agent proximal policy optimization (MAPPO) network architecture with centralized training and distributed execution (CTDE), including local Actor networks deployed on each cluster node and a global Critic network deployed on the scheduling and coordinating agent.

[0061] Before step S310 is executed, the scheduling observation packet must first be converted into a multidimensional tensor input: For scene state sequence summaries and interaction state sequence summaries, a Long Short-Term Memory (LSTM) network is used to perform temporal feature extraction, outputting a hidden state vector with a fixed-dimensional time step. ; For a set of node agent state vectors containing numerical values ​​(such as computational load and memory usage) and discrete classification labels (such as abnormal state labels), the numerical fields are normalized, and the discrete classification labels are one-hot encoded. After concatenation, features are extracted using a multilayer perceptron (MLP) to generate the node state vectors. .

[0062] Will The tensor corresponding to the branch survival probability table and the tensor corresponding to the task inheritance mapping set are concatenated to form the local observation feature tensor of the node agent. The scheduling and coordination agent collects data from all nodes. Concatenate to generate a global environment feature tensor .

[0063] S3 specifically includes the following sub-steps: S310. Input the scheduling observation package, branch survival probability table and task inheritance mapping set into the reinforcement learning policy model. In the multi-agent system, each node agent outputs its local execution score for each reversible rendering predecessor task. The scheduling and coordination agent then aggregates the local execution scores to generate a global scheduling action value.

[0064] Specifically, the node agent state vector set in the scheduling observation package is first read in the order of node identifiers. Then, the branch survival probability value in the branch survival probability table is read in the order of task identifiers. The predecessor task identifier and finalized task identifier in the task inheritance mapping set are grouped according to the state inheritance identifier to form the task-node corresponding input matrix for the current scheduling cycle. When there is an anomaly correction flag field in the branch survival probability table, or a record with an invalid inheritance validity field in the task inheritance mapping set, an invalid masking flag is added to the relevant input item in the corresponding task-node corresponding input matrix, so that the reinforcement learning policy model will no longer regard it as a priority scheduling object in this round of inference.

[0065] Subsequently, each node agent uses its own local Actor network to process the local observation feature tensor. The local Actor network consists of two fully connected hidden layers and a softmax output layer. Its forward propagation process is as follows: After inputting a fully connected hidden layer, the output node shows its execution preference features for each of the current reversible rendering precursor tasks. These features are then mapped to probability distributions corresponding to each task via the Softmax function.

[0066] The output value of this probability distribution is the local execution score. The formula for its calculation is: In the formula, This represents the preference feature vector output by the hidden layer of the local Actor network; and These are the network weights and bias parameters corresponding to the j-th task in the output layer, respectively; and These are the network weights and bias parameters corresponding to the k-th task, respectively, where k is the summation index for all J tasks. This represents the total number of reversible rendering precursor tasks to be assigned within the current scheduling cycle.

[0067] Perform scoring in the output section. Subsequently, if the abnormal state markers in the node's state vector indicate that the node is offline, has insufficient video memory, or is experiencing network congestion, an action masking mechanism is applied to force the corresponding task to be masked. Set it to infinitesimal so that its probability after Softmax output approaches 0, and directly record it as an unselectable level.

[0068] After receiving the local execution scores output by each node agent, the scheduling and coordination agent generates the global scheduling action value according to the following formula: In the formula, For the first The global scheduling action value corresponding to each task to be assigned; For the first The node agent on the first The partial execution score output by each pending task; The number of node agents participating in the current scheduling cycle; For the first The aggregation weights corresponding to each node agent.

[0069] The aggregation weight is determined based on the expected available time field and the abnormal status flag field. The longer the expected available time and the more the abnormal status flag field does not record the abnormal status, the higher the aggregation weight of the node agent. The aggregation weight of the node agent with the abnormal status flag is reduced according to the preset decay rule. The aggregation weight of the node agent with the node offline flag is directly set to zero.

[0070] To ensure that the scheduling results in this round reflect historical execution performance, the reinforcement learning strategy model also calls upon the delivery effectiveness, continuation time, and node resource consumption from the historical scheduling feedback set to correct historical biases in local execution scores. After completing the calculation, it outputs the global scheduling action value corresponding to each reversible rendering precursor task, serving as a unified decision-making basis for subsequent task allocation.

[0071] S320. Based on the global scheduling action value, combined with the computational load, memory usage, cache heat, network bandwidth and expected available time in the state vector of each node agent, determine the distributed pre-allocation result for each reversible rendering precursor task, determine the candidate execution node set for each irreversible final rendering task, and calculate the state inheritance matching degree of each candidate execution node according to the state inheritance identifier to generate the task allocation result set.

[0072] Specifically, for each reversible rendering precursor task, cluster nodes that are marked as offline, have insufficient video memory, or are in an unexecutable state due to network congestion are first removed. Then, a preset number of nodes with the highest local execution scores corresponding to the global scheduling action values ​​are selected from the remaining cluster nodes as the pre-allocated node group for the reversible rendering precursor task. When multiple reversible rendering precursor tasks under the same state inheritance identifier compete for the same cluster node, the task with the higher branch survival probability value and tighter latency constraints is retained first, and the remaining tasks are deferred to the cluster node with the second highest score.

[0073] For each irreversible final rendering task, only cluster nodes that simultaneously meet the following conditions are written into the candidate execution node set: there exists a state inheritance identifier corresponding to the irreversible final rendering task, there is an access path that can read the intermediate rendering state that can be continued under the corresponding state inheritance identifier, the current node has spare continuation resources or has the conditions for reserving continuation resources, and the abnormal state flag field is not in an unexecutable state.

[0074] To ensure that the final execution directly inherits the results of the predecessor, this embodiment further calculates the state inheritance matching degree for each cluster node in the candidate execution node set. The calculation formula is as follows: In the formula, For the first The candidate execution node for the first The state inheritance matching degree of each state inheritance identifier; For the first The candidate execution node for the first Each state inheritance identifier corresponds to the reachability of inheritance between cached and intermediate states; For the first Available network bandwidth metrics for each candidate execution node; For the first The estimated availability time of each candidate execution node; , , To match the weight coefficients.

[0075] Inheritance reachability is used to characterize the reachability of candidate execution nodes to read cached data, intermediate state data, and state inheritance pool records under the corresponding state inheritance identifier. The higher the inheritance reachability, the higher the available network bandwidth, and the longer the expected available time, the higher the state inheritance matching degree. When a candidate execution node does not have an access path under the corresponding state inheritance identifier, its state inheritance matching degree is directly recorded as unselectable level.

[0076] After completing the above processing, a task allocation result set is generated. The task allocation result set includes at least the preceding task identifier field, the pre-allocated node group field, the finalized task identifier field, the candidate execution node set field, the status inheritance matching degree field, and the allocation update time field.

[0077] In this embodiment, the task allocation result set is used by S330 to perform resource conflict resolution and time period sorting, by S410 to issue reversible rendering precursor tasks, and by S430 to select target nodes.

[0078] S330. Based on the task allocation result set, perform resource conflict resolution, time period sorting, and continuation resource reservation for reversible rendering precursor tasks and irreversible final rendering tasks that involve the same state inheritance identifier, and generate a scheduling execution plan.

[0079] Specifically, the pre-allocated node group field and candidate execution node set field belonging to the same state inheritance identifier in the task allocation result set are traversed first to identify whether there is competition for the same computing time period, the same video memory interval, or the same cache access channel for the same cluster node. When there is competition, resources are preferentially allocated to tasks with higher branch survival probability values, higher state inheritance matching degree, and tighter latency constraints. For tasks that are not preferentially allocated, they are migrated to the second-best node in the candidate execution node set according to the scoring order recorded in the task allocation result set, or postponed to the next executable time period.

[0080] After resolving resource conflicts, tasks on the same cluster node are sorted by execution order and time slots. Reversible rendering precursor tasks are given priority to use the current idle time slot to generate a continueable intermediate rendering state. Irreversible final rendering tasks are inserted into reserved time slots for execution after their corresponding branch survival probability value reaches the trigger threshold, ensuring that the final rendering task can directly call the precursor result under the corresponding state inheritance identifier when it starts. Subsequently, a continuation resource reservation record is established for each irreversible final rendering task. This record includes at least a node identifier field, a reserved time slot field, a reserved video memory field, a corresponding state inheritance identifier field, and a reserved validity period field.

[0081] Among them, the node identifier field is used to identify the cluster node to which the reserved resource belongs, the reserved time period field is used to limit the time interval during which the irreversible final rendering task can be started directly, the reserved video memory field is used to limit the video memory resources that can be directly called when continuing the task, and the reserved validity period field is used to limit the reserved resource to be valid only within the time range during which the corresponding state inheritance identifier has not expired.

[0082] When the inheritance validity field in the task inheritance mapping set is updated to invalid, the corresponding continuation resource reservation record is automatically released. Finally, the results after resolving resource conflicts, sorting by time period, and reserving continuation resources are written into the scheduling execution plan. The scheduling execution plan includes at least the distributed pre-allocation results, the candidate execution node set, the node execution order, the continuation resource reservation record, and the scheduling effective time.

[0083] In this embodiment, the scheduling and execution scheme continues to provide S410 with task issuance, S430 with irreversible final rendering tasks, and S510 with continuation migration after scene state changes, thus forming a complete closed loop from strategy decision-making to execution scheduling.

[0084] S4 specifically includes the following sub-steps: S410. According to the scheduling and execution plan, each reversible rendering precursor task is sent to the cluster node where the corresponding node agent is located for execution, and the scene state fragment, interaction state fragment and state inheritance identifier corresponding to the reversible rendering precursor task are written into the execution context to generate the precursor execution record.

[0085] Specifically, the distributed pre-allocation results, node execution order, and scheduling effective time in the scheduling execution scheme are read first. Then, based on the input state source field in the task layering record, the scene state fragment and interaction state fragment corresponding to each reversible rendering precursor task are selected. The scene state fragment is used to limit the camera state, view state, and lighting state on which the current precursor calculation depends, and the interaction state fragment is used to limit the user operation state and object response state on which the current precursor calculation depends.

[0086] Subsequently, an execution context is constructed on a task-by-task basis. The execution context includes at least the following fields: predecessor task identifier field, node identifier field, scene state fragment identifier field, interaction state fragment identifier field, state inheritance identifier field, scheduled start time field, expected output type field, and execution timeout threshold field.

[0087] After receiving the execution context, the cluster node first verifies whether the state inheritance identifier field is consistent with the task inheritance mapping set, whether the expected output type field is consistent with the expected output type field corresponding to the reversible rendering predecessor task in the task layering record, and whether the scheduled start time field falls within the executable range defined by the node execution order of the current node. When all three verifications pass, the corresponding scene state fragment and interaction state fragment are loaded, and the task reception status is recorded as successful reception and the resource loading status is recorded as loading completed. When any verification fails, the task reception status is recorded as execution rejection, and the rejection reason is written in the execution exception flag field.

[0088] After distribution is complete, a precursor execution record is generated. This record includes at least the task reception status field, resource loading status field, startup time field, corresponding state inheritance identifier field, and execution exception flag field. For example, if the scheduled startup period for a reversible rendering precursor task is within the current node's idle period of 20 milliseconds, and its state inheritance identifier matches the task inheritance mapping set, the cluster node loads the corresponding low-resolution occlusion relationship input data and interaction state fragments, and writes the task reception status as successfully received.

[0089] In this embodiment, the preceding execution record continues to provide S420 with a continueable intermediate rendering state, which is used by S510 to determine the continuation entry for incomplete rendering tasks.

[0090] S420: Controls the cluster nodes that execute the reversible rendering precursor task, outputs the continueable intermediate rendering state according to the precursor execution record, and writes each continueable intermediate rendering state into the state inheritance pool according to the state inheritance identifier, generating the state inheritance pool record.

[0091] Specifically, for the predecessor execution record where the task reception status field is "successfully received" and the resource loading status field is "loaded successfully", the predecessor calculation is initiated, and the corresponding intermediate status data is generated according to the expected output type field. In this embodiment, the continueable intermediate rendering status is limited to the intermediate results that can be directly inherited and called by the subsequent irreversible final rendering task, including at least one or more of the following: intermediate visibility relationship data, coarse-grained lighting estimation data, reusable sampling cache data, low-resolution occlusion relationship data, and intermediate shading base map data.

[0092] To prevent incomplete intermediate results from entering the final draft process, a state integrity rate calculation is performed on intermediate state data generated under the same state inheritance identifier. The calculation formula is as follows: In the formula, For the first The state integrity rate corresponding to each state inheritance identifier; For the first The number of intermediate state data items that have been written and verified under each state inheritance identifier; For the first The total number of intermediate state data items that should be generated under each state inheritance identifier.

[0093] Here, "passing the verification" means that the intermediate state data has the same state inheritance identifier field, the same expected output type field, and the generation time is within the range of the execution timeout threshold field. When the state integrity rate is not lower than the preset integrity rate threshold, and the execution exception flag field in the predecessor execution record does not record any exception, the intermediate result is identified as a continueable intermediate rendering state, and is written into the state inheritance pool along with the corresponding expected output type field and output specification constraint field; when the state integrity rate is lower than the preset integrity rate threshold, it is only retained as an incomplete intermediate state and is not written into the state inheritance pool.

[0094] The state inheritance pool record includes at least the state inheritance identifier field, intermediate state storage location field, expected output type field, output specification constraint field, state integrity rate field, generation time field, validity period field, and state validity field. The validity period field is used to limit the duration for which the reproducible intermediate rendering state can be called later, and the state validity field is used to characterize whether the current record is allowed to be extracted by the target node.

[0095] When a new, reproducible intermediate rendering state is generated under the same state inheritance identifier, the older record is overwritten with the newer record and a version update flag is written; when the corresponding inheritance validity field becomes invalid, or the validity period expires and is not called, the state inheritance pool record is deleted.

[0096] In this embodiment, the state inheritance pool records the intermediate rendering states that S430 can select from those that match the irreversible final rendering task, and S510 can perform the continuation allocation after the scene state changes.

[0097] S430. When the branch survival probability corresponding to the branch survival probability table reaches the trigger threshold, select the cluster node with the highest state inheritance matching degree and the reserved record of continuation resources from the candidate execution node set as the target node, extract the continuable intermediate rendering state corresponding to the target node from the state inheritance pool record, start the irreversible final rendering task, and generate the final execution record.

[0098] Specifically, the branch survival probability value in the branch survival probability table is read first, and then combined with the state integrity rate field in the state inheritance pool record and the reservation validity period field in the continuation resource reservation record for joint judgment. The irreversible final rendering task is only allowed to start when the branch survival probability value is not lower than the trigger threshold, the state integrity rate is not lower than the preset integrity rate threshold, and the continuation resource reservation record is within the validity period.

[0099] Subsequently, among the corresponding candidate execution node set, the cluster node with the highest state inheritance matching degree is selected first; when the state inheritance matching degree of multiple candidate execution nodes is at the same matching level, the reserved time period field, reserved video memory field and network bandwidth index are further compared, and the cluster node with an earlier reserved time period, more reserved video memory and higher network bandwidth index is selected as the target node.

[0100] Before extracting a resumable intermediate rendering state, the target node first performs a consistency check on the state inheritance pool records. The consistency check includes at least the consistency of the state inheritance identifier field, the consistency of the expected output type field, the consistency of the output specification constraints, and the validity of the state validity field. Only when all four conditions are met will the corresponding resumable intermediate rendering state be read from the storage location pointed to by the intermediate state storage location field and the irreversible final rendering task be started. If any condition is not met, the current state inheritance pool record is abandoned and the extraction is re-initiated from the second-best cluster node in the candidate execution node set.

[0101] After the irreversible final rendering task is started, a final execution record is generated. The final execution record includes at least the target node identifier field, the state inheritance identifier field of the call, the final task start time field, the final task execution status field, and the execution exception flag field.

[0102] In this embodiment, the finalized execution record continues to be used by S510 to determine whether to continue the process, and by S520 to extract the target rendering result.

[0103] S5 specifically includes the following sub-steps: S510. During the rendering process, continuously monitor the scene state sequence and interaction state sequence in the scheduling context record. When a scene state change is identified and the corresponding irreversible final rendering task has not been completed, according to the scheduling execution plan, final execution record, state inheritance pool record, task inheritance mapping set and continuation resource reservation record, perform continuation allocation for the incomplete rendering task, call the continuable intermediate rendering state under the corresponding state inheritance identifier to continue execution on the original target node or the migrated candidate execution node, and generate a task continuation record.

[0104] Specifically, based on the state inheritance identifier field of the finalization execution record, the camera state, view state, lighting state, and interaction branch state corresponding to the current irreversible finalization rendering task are extracted from the scheduling context record. Then, they are compared item by item with the scene state fragments and interaction state fragments corresponding to the start of the irreversible finalization rendering task. When the camera identifier changes, the change in view parameters exceeds the preset view deviation threshold, the change in lighting state exceeds the preset lighting deviation threshold, or the interaction branch state is switched, it is determined that a scene state change triggering the continuation allocation is triggered. When none of the above changes exceed the corresponding threshold, the original target node continues to execute, and the migration continuation is not initiated.

[0105] For incomplete rendering tasks that have already triggered continuation allocation, first verify whether the execution status field of the finalized task corresponding to the original target node is "can continue to be executed", whether the reservation validity period field in the continuation resource reservation record is still valid, and whether there is a continuationable intermediate rendering state in the state inheritance pool record that is consistent with the current state inheritance identifier and has a valid state validity field. When all three conditions are met, continuation allocation is performed on the original target node. When any condition is not met, the candidate execution nodes after migration are selected in turn according to the state inheritance matching degree in the candidate execution node set.

[0106] When performing a continuation assignment, the continuable intermediate rendering state pointed to by the intermediate state storage location field in the state inheritance pool record is called, and the lens state increment, view state increment, lighting state increment and interaction branch correction amount corresponding to the current scene state change are written into the continuation execution context, so that the irreversible final rendering task after continuation can complete the calculation of the difference part based on the inheritance of the existing intermediate results.

[0107] To ensure consistency in subsequent statistical methods, task continuation records are generated. Each task continuation record includes at least the following fields: continuation task identifier field, continuation node identifier field, call status inheritance identifier field, continuation start time field, continuation source type field, and continuation exception flag field. The continuation source type field is used to distinguish between continuation of the original target node and continuation of the migrated candidate execution node.

[0108] For example, when the change in the viewpoint parameter is 1.2 times the preset viewpoint deviation threshold and the original target node's reserved validity period has expired, the unfinished rendering task is migrated to the candidate execution node ranked first in state inheritance matching degree, and the migration of the candidate execution node is written into the task continuation record.

[0109] In this embodiment, the task continuation record continues to provide the target rendering result for S520 to output, and for S530 to calculate the reward value.

[0110] S520: Output the target rendering result based on the request parameter record, scheduling context record, final execution record, and task continuation record. Perform a valid verification of the target rendering result delivery. The valid verification of the result delivery includes at least whether the output time meets the latency constraint, whether the output content is consistent with the current scene state and the current interaction branch, and whether the output quality meets the request parameter record. At the same time, calculate the continuation time and node resource consumption, and generate a scheduling feedback record.

[0111] Specifically, the original target node or the migrated candidate execution node first outputs the target rendering result after completing the irreversible final rendering task, and then the execution results are delivered for validity verification in a fixed order.

[0112] First, read the latency constraint field in the request parameter record and compare the time difference between the output time of the target rendering result and the current request reception time; when the time difference falls within the range of the latency constraint field, it is recorded as passing the latency constraint verification; otherwise, it is recorded as invalid delivery.

[0113] Second, read the current camera state and interaction branch state from the scheduling context record, and compare the camera identifier and interaction branch corresponding to the target rendering result; when they match, it is recorded as passing the scene consistency check, otherwise it is recorded as delivering the wrong branch.

[0114] Third, read the output specification field from the request parameter record and compare the resolution, color format, and target encoding format of the target rendering result. If all three meet the requirements of the output specification field, it is recorded as passing the output specification check; otherwise, it is recorded as a specification mismatch delivery. Only when the target rendering result passes the latency constraint check, scene consistency check, and output specification check simultaneously is the result delivery validity field recorded as a valid delivery.

[0115] Subsequently, the continuation time and node resource consumption were calculated. The continuation time was determined by the time difference between the continuation start time field in the task continuation record and the target rendering result output time. The node resource consumption included at least the computation time, video memory usage, and network transmission volume during the continuation period.

[0116] To ensure that subsequent reward value calculations have directly callable inputs, a scheduling feedback record is generated. The scheduling feedback record includes at least the following fields: result delivery validity field, delivery result category field, continuation time field, calculation duration field, video memory usage field, network transmission volume field, and feedback generation time field. Among them, the delivery result category field is used to distinguish between valid delivery, invalid delivery, wrong branch delivery, and specification mismatch delivery.

[0117] For example: when the target rendering result is output 38 milliseconds after the request is received, the latency constraint field limits the upper limit to 40 milliseconds, and the lens identifier, interaction branch, and output specifications are all consistent, the result delivery validity field is recorded as valid delivery, and the occupied time, video memory usage, and network transmission volume are calculated according to the actual continuation period.

[0118] In this embodiment, the scheduling feedback record continues to be used by S530 to calculate the reward value and update the reinforcement learning policy model.

[0119] S530. Calculate the reward value based on the result delivery validity, continuation time and node resource consumption in the scheduling feedback record, write the reward value back to the reinforcement learning policy model, update the local policy parameters of each node agent and the global policy parameters of the scheduling and coordination agent, and generate a historical scheduling feedback set.

[0120] Specifically, the result delivery validity field in the scheduling feedback record is first converted into a result delivery validity quantification value, the continuation time field is converted into a continuation time quantification value, and the computation time, memory usage, and network transmission volume fields are weighted and synthesized into a node resource consumption quantification value. Among these, valid delivery corresponds to a higher result delivery validity quantification value, while incorrect branch delivery, specification mismatch delivery, and invalid delivery correspond to progressively lower result delivery validity quantification values. The reward value is calculated using the following formula: In the formula, This is the reward value; Quantify the effectiveness of deliverable results; To quantify the time consumed during the continuation process; This represents a quantified value of node resource consumption. , , This is the reward weighting coefficient.

[0121] Here, the higher the quantitative value of the result delivery validity, the higher the reward value; the higher the quantitative value of the continuation time or the higher the quantitative value of the node resource consumption, the lower the reward value.

[0122] Completion Reward Value The calculation will include local observations, global state, action probabilities, and reward values. The trajectory data are stored together in the experience replay pool. When the amount of data in the experience replay pool reaches the preset batch size, the parameter update of the reinforcement learning policy model is initiated. First, the global Critic network is based on the input of the new round of global environment feature tensor and the actual reward value obtained. The time difference error (TD-Error) is calculated, and the mean squared error (MSE) is used as the Critic loss function. Perform gradient descent to update the Critic network parameters. : In the formula, For sampling batch size, As a discount factor, The state value estimate output by the Critic network. For the current global environment feature tensor, This is the global environment feature tensor for the next round after the scheduling action is performed.

[0123] Secondly, each node agent utilizes the advantage function returned by the Critic network. (Based on TD-Error calculation), the parameters of the local Actor network are updated by maximizing the pruned agent objective function. Its Actor loss function for: In the formula, t is the time step index of the trajectory data in the experience replay pool; This is the estimated value of the dominance function at the t-th time step; This is the ratio of the probability of the action output under the old and new strategies; Hyperparameters are pruned to limit the step size of policy updates and ensure training stability. The parameters of both the Actor and Critic networks undergo backpropagation iterations using the Adam optimizer.

[0124] To ensure that the next round of scheduling utilizes both positive and negative samples, the newest valid delivery records and the newest failed delivery records are written together into the historical scheduling feedback set. The historical scheduling feedback set includes at least the feedback generation time field, the successor node identifier field, the state inheritance identifier field, the reward value field, and the delivery result category field. When the capacity of the historical scheduling feedback set reaches the preset limit, the earliest generated scheduling feedback record that has participated in at least one policy update is deleted.

[0125] Finally, the historical scheduling feedback set is used as the historical scheduling feedback input for S310 in the next round of scheduling, thus forming a complete closed loop from scene state change identification, continuation allocation, target rendering result output, result delivery validity verification, reward value calculation to policy update.

[0126] All the above formulas are performed using dimensionless numerical calculations; the relevant formulas are based on empirical models that approximate the real situation, obtained through extensive data collection and software simulation fitting. The preset parameters and thresholds involved in the formulas can be conventionally set and adjusted by those skilled in the art according to the physical constraints of the actual application scenario.

[0127] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0129] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning, characterized in that, Includes the following steps: S1. Obtain the cloud rendering request, perform request integrity verification, and generate request parameter records, scheduling context records, and scheduling observation packages; S2. Based on the request parameter record, scheduling context record and scheduling observation package, divide the rendering task to be executed into reversible rendering precursor task and irreversible final rendering task, calculate the branch survival probability, and generate state inheritance identifier and task inheritance mapping set. S3. Input the scheduling observation package, branch survival probability and task inheritance mapping set into the reinforcement learning policy model, and generate global scheduling action values, task allocation result set and scheduling execution scheme by the node agent and the scheduling coordination agent. S4. Execute the reversible rendering precursor task according to the scheduling execution plan, generate a continueable intermediate rendering state and write it into the state inheritance pool. When the branch survival probability and state integrity rate meet the conditions, select the target node to extract the corresponding continueable intermediate rendering state and start the irreversible final rendering task.

2. The method for dynamic allocation of distributed cloud rendering tasks based on reinforcement learning according to claim 1, characterized in that, Also includes: S5. Monitor scene state changes, perform continuation assignment for unfinished rendering tasks, output target rendering results, verify the validity of execution results delivery, generate scheduling feedback records and reward values, and update the reinforcement learning strategy model based on the reward values.

3. The method for dynamic allocation of distributed cloud rendering tasks based on reinforcement learning according to claim 1, characterized in that, S1 specifically includes: Obtain cloud rendering requests, perform request integrity checks on lens identifiers, viewpoint parameters, image quality levels, latency constraints, output specifications, request reception time, and request source node, and generate request parameter records; Based on the request parameters, the system reads the scene state sequence, interaction state sequence, and cluster node state set corresponding to the cloud rendering request, performs unified timeline alignment, removes missing segments, and marks abnormal states, and generates a scheduling context record.

4. The method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning according to claim 3, characterized in that, Also includes: Each cluster node is mapped to a node agent in a multi-agent system, and a scheduling and coordination agent is set up. Based on the request parameter record and scheduling context record, a set of node agent state vectors and a global scheduling observation set are constructed to generate a scheduling observation package.

5. The method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning according to claim 1, characterized in that, S2 specifically includes: Based on the request parameter record, scheduling context record and scheduling observation package, the rendering task to be executed is processed in a hierarchical manner, distinguishing between reversible rendering pre-processing tasks and irreversible final rendering tasks, and generating a task hierarchical record. Based on the camera change trend, viewpoint movement trend, and lighting change trend in the scene state sequence, as well as the user operation trend and object response trend in the interaction state sequence, calculate the branch survival probability for each reversible rendering pre-task and generate a branch survival probability table. Establish a state inheritance relationship between each reversible rendering precursor task and its corresponding irreversible final rendering task, and generate state inheritance identifiers and task inheritance mapping sets.

6. The method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning according to claim 1, characterized in that, S3 specifically includes: The scheduling observation package, branch survival probability table and task inheritance mapping set are input into the reinforcement learning policy model. In the multi-agent system, each node agent outputs a local execution score, which is then aggregated by the scheduling and coordinating agent to generate a global scheduling action value. Based on the global scheduling action value, combined with computing load, video memory usage, cache heat, network bandwidth and expected available time, the distributed pre-allocation results of reversible rendering precursor tasks, the candidate execution node set and state inheritance matching degree of irreversible final rendering tasks are determined, and a task allocation result set is generated.

7. The method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning according to claim 6, characterized in that, Also includes: Based on the task allocation result set, resource conflict resolution, time period sorting, and resource reservation for continuation are performed to generate a scheduling execution plan.

8. The method for dynamic allocation of distributed cloud rendering tasks based on reinforcement learning according to claim 1, characterized in that, S4 specifically includes: According to the scheduling and execution plan, each reversible rendering precursor task is sent to the corresponding cluster node for execution, and the scene state fragment, interaction state fragment and state inheritance identifier are written into the execution context to generate the precursor execution record; Based on the output of the previous execution record, the intermediate rendering state can be continued. It is written into the state inheritance pool according to the state inheritance identifier, and a state inheritance pool record is generated. When the branch survival probability reaches the trigger threshold and the state integrity rate meets the conditions, the target node is selected from the candidate execution node set, the corresponding continueable intermediate rendering state is extracted, the irreversible final rendering task is started, and the final execution record is generated.

9. The method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning according to claim 1, characterized in that, S5 specifically includes: During the rendering process, the scene state sequence and interaction state sequence in the scheduling context record are continuously monitored. When a scene state change is identified and the corresponding irreversible final rendering task has not been completed, the unfinished rendering task is assigned a continuation based on the scheduling execution plan, final execution record, state inheritance pool record, task inheritance mapping set and continuation resource reservation record, and a task continuation record is generated. Based on the request parameter record, scheduling context record, finalized execution record, and task continuation record, the target rendering result is output, the validity of the execution result delivery is verified, and a scheduling feedback record is generated.

10. The method for dynamically allocating distributed cloud rendering tasks based on reinforcement learning according to claim 9, characterized in that, Also includes: The reward value is calculated based on the scheduling feedback record, and the reinforcement learning policy model is updated based on the reward value to generate a historical scheduling feedback set.

Images