Three-dimensional scene rendering method, device, system and equipment, medium and program product

Abstract

The invention provides a three-dimensional scene rendering method, device, system and equipment, a medium and a program product, and relates to the technical field of IT support. The method comprises the following steps: acquiring three-dimensional data of a three-dimensional scene to be rendered; for each available computing power node, determining state information of the computing power node; wherein the state information comprises one or more of task scheduling time delay, node residual computing power, a resource utilization rate and a strategy intervention value; the state information of each computing power node is input into a scheduling strategy generation model, a scheduling strategy is obtained, and the scheduling strategy comprises a target computing power node used for three-dimensional scene rendering; and rendering the three-dimensional data through the target computing power node to obtain a three-dimensional rendering result. According to the method, multiple scheduling influence factors are used as model input, a more balanced and more efficient scheduling strategy can be obtained, and then the three-dimensional scene is rendered based on the target computing power node determined by the scheduling strategy, so that the scheduling efficiency and the resource utilization rate can be improved.

Description

Technical Field

[0001] This invention relates to the field of IT support technology, and in particular to a three-dimensional scene rendering method, apparatus, system, device, medium and program product. Background Technology

[0002] With the development of digital twin rendering needs and technologies, traditional single-machine rendering methods are no longer sufficient to meet business development needs. Cloud-edge collaborative rendering technology has emerged, in which the cloud master node is responsible for resource scheduling and large-scale data processing, while the edge node achieves high-reliability and low-latency scene rendering. This type of rendering solution needs to consider the scheduling of rendering resources between nodes and the overall rendering strategy.

[0003] Existing technical solutions for distributed node rendering resource scheduling include:

[0004] Option 1 involves scheduling cloud and LAN resources based on information such as cluster node status, computing power, and network quality. It utilizes cloud rendering machine clusters in the public network and edge rendering machine clusters in the LAN, determining the target rendering machine cluster based on computing power usage and network quality information. However, while this scheduling strategy combining cloud and LAN resources considers computing power usage and network quality information for resource coordination and scheduling, it is difficult to integrate the LAN and cloud resource environments in practical applications.

[0005] Option 2 involves using primitive splitting and merging for node-based rendering and scheduling. This involves splitting the 3D scene model into N image units, determining the computational load required for each unit, and determining the computational power required for each rendering node. Based on the computational load and power, the N image units are allocated to M rendering nodes for rendering, resulting in M ​​sets of rendering data. These M sets of rendering data are then globally merged to obtain the rendered image. However, this image splitting and merging scheduling method, due to the large amount of cross-node transmission and computation, struggles to meet the high frame rate requirements of large scenes.

[0006] In other words, the existing scheduling of rendering resources among nodes suffers from low scheduling efficiency and low resource utilization. Summary of the Invention

[0007] The purpose of this invention is to provide a three-dimensional scene rendering method, apparatus, system, device, medium, and program product to solve the problems of low scheduling efficiency and low resource utilization in the existing rendering resource scheduling between node resources.

[0008] To achieve the above objectives, in a first aspect, embodiments of the present invention provide a three-dimensional scene rendering method, including:

[0009] Obtain the 3D data of the 3D scene to be rendered;

[0010] For each available computing power node, determine the status information of the computing power node; wherein, the status information includes one or more of the following: task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value;

[0011] The state information of each computing node is input into the scheduling strategy generation model to obtain a scheduling strategy, which includes target computing nodes for 3D scene rendering.

[0012] The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

[0013] In some embodiments, determining the state information of the computing node includes:

[0014] Collect information on the processing capacity, resource status, hardware load, and policy intervention values ​​of the computing nodes in specific scenarios;

[0015] The task scheduling delay is calculated based on the processing capacity information.

[0016] Based on the resource status information, the remaining computing power of the node is calculated;

[0017] The resource utilization rate is calculated based on the hardware load information.

[0018] In some embodiments, the processing capability information includes at least one of the following: the amount of data transmitted by the computing node, the transmission bandwidth, the number of currently queued tasks, and the average queuing latency of the tasks;

[0019] The resource status information includes the total computing power and the computing power used by the computing power node;

[0020] The hardware load information includes at least one of CPU utilization, GPU utilization, memory usage, and video memory usage.

[0021] In some embodiments, the step of inputting the state information of each computing node into the scheduling policy generation model to obtain a scheduling policy includes:

[0022] For each computing node, the task scheduling latency, remaining computing power, resource utilization, and policy intervention value of the computing node are input into the scheduling policy generation model, and the following processing is performed:

[0023] Based on the weight coefficients corresponding to the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value, a linear weighted calculation is performed on the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value to obtain the scheduling score;

[0024] Based on the scheduling score, a scheduling strategy is obtained. The scheduling strategy includes a target computing power node, which is the computing power node with the highest scheduling score among all the computing power nodes.

[0025] In some embodiments, the scheduling policy generation model includes a scheduling policy generation sub-model, a scheduling policy evaluation sub-model, and a reinforcement learning model; the method further includes:

[0026] For each model training, the historical task scheduling latency, historical node remaining computing power, historical resource utilization rate and policy intervention value of each computing node are respectively input into the scheduling policy to generate a sub-model and obtain the historical scheduling policy.

[0027] The end-to-end latency, end-to-end frame rate, and corresponding policy intervention value output by the historical scheduling strategy are input into the scheduling strategy evaluation sub-model to obtain the historical scheduling evaluation score.

[0028] The historical scheduling score and the historical scheduling evaluation score corresponding to the historical scheduling strategy are used as input states and fed into the reinforcement learning model. Guided by the reward function, action k is output. Action k includes the network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model. The network parameters of the scheduling strategy generation sub-model include the weight coefficients corresponding to the historical task scheduling latency of the computing node, the remaining computing power of the historical node, the historical resource utilization rate, and the strategy intervention value. The network parameters of the scheduling strategy evaluation sub-model include the end-to-end latency and end-to-end frame rate output using the historical scheduling strategy, as well as the weight coefficients corresponding to the respective strategy intervention values.

[0029] The network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model are adjusted and updated using the objective loss function until the training termination condition is met, thus obtaining the scheduling strategy generation model.

[0030] In some embodiments, rendering the three-dimensional data through the target computing node to obtain a three-dimensional rendering result includes:

[0031] The three-dimensional data is divided according to the scene range to obtain grid data corresponding to multiple spatial grid regions;

[0032] The grid data of each spatial grid region is hierarchically divided to obtain the hierarchical grid data of the spatial grid region;

[0033] At least a portion of the hierarchical mesh data from all the hierarchical mesh data is loaded into the target computing node, and the target computing node performs rendering based on the at least a portion of the hierarchical mesh data to obtain a 3D rendering result.

[0034] In some embodiments, loading at least a portion of the hierarchical grid data from all the hierarchical grid data into the target computing node includes:

[0035] Load all the hierarchical grid data that meet the target conditions into the memory of the target computing node;

[0036] Based on the camera's viewing distance range and the camera's viewport vector direction, the hierarchical grid data in the memory is loaded into the video memory of the target computing node.

[0037] In some embodiments, the target condition includes at least one of the following conditions:

[0038] The most recent access time difference of the hierarchical grid data is less than a first threshold, and the most recent access time difference of the hierarchical grid data is the difference between the current time of the hierarchical grid data and the time when the hierarchical grid data was last accessed;

[0039] The hit rate of the hierarchical grid data is greater than the second threshold, and the hit rate of the hierarchical grid data is the proportion of the number of accesses to the hierarchical grid data in the total number of accesses to all the hierarchical grid data;

[0040] The preloading priority value of the hierarchical grid data is greater than the third threshold. The preloading priority value of the hierarchical grid data is obtained by linearly weighting the recent access time difference and the hit rate of the hierarchical grid data based on the weight coefficients corresponding to the recent access time difference and the hit rate of the hierarchical grid data.

[0041] The hierarchical grid data refers to the hierarchical grid data outside the camera's field of view.

[0042] The hierarchical grid data refers to the hierarchical grid data adjacent to the hierarchical grid data outside the camera's field of view.

[0043] In some embodiments, loading the hierarchical grid data in memory into the video memory of the target computing node according to the camera's viewing distance range and camera viewport vector direction includes:

[0044] The hierarchical grid data within the first hemispherical arc region formed by expanding the camera's viewing distance range and the positive and negative directions of the camera's viewport vector direction outward by a preset angle in the memory is loaded into the video memory.

[0045] The hierarchical grid data within the second hemispherical arc region, formed by translating the first hemispherical arc region a preset distance along the camera's movement direction, is loaded into the video memory.

[0046] In some embodiments, the step of performing rendering by the target computing node based on the at least partial hierarchical mesh data to obtain a 3D rendering result includes:

[0047] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0048] For each spatial grid region, areas within the camera's field of view and whose position is less than the view distance threshold are rendered using their corresponding high-level grid data, and / or areas within the camera's field of view and whose position is greater than or equal to the view distance threshold are rendered using their corresponding low-level grid data, to obtain a 3D rendering result; or...

[0049] If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the central area of ​​the screen is rendered using its corresponding high-level mesh data; otherwise, the central area of ​​the screen is rendered using its corresponding low-level mesh data. If the difference between the pixels of the current frame's rendered image and the pixels of the previous frame's rendered image, calculated using a global feature vector or hash comparison, is less than a fourth threshold, the peripheral area of ​​the current frame's image is rendered using the peripheral area image of the previous frame. Alternatively, if the difference is greater than or equal to the fourth threshold, the peripheral area of ​​the screen is rendered using the attribute information of the mesh annotations corresponding to the peripheral area of ​​the screen and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0050] In some embodiments, the step of performing rendering by the target computing node based on the at least partial hierarchical mesh data to obtain a 3D rendering result includes:

[0051] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0052] For each spatial grid region, the high-level grid data corresponding to the region within the camera's field of view where the camera position is less than the viewing distance threshold, and / or the low-level grid data corresponding to the region within the camera's field of view where the camera position is greater than or equal to the viewing distance threshold, are determined as the level grid data to be rendered.

[0053] If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the target level mesh data corresponding to the screen center region in the level mesh data to be rendered is determined, and the screen center region is rendered using the target level mesh data; otherwise, the screen center region is rendered using the attribute information of the mesh annotations corresponding to the screen center region and the corresponding sampled texture or color texture; and if the difference between the pixels of the rendered image in this frame and the pixels of the rendered image in the previous frame calculated by global feature vector or hash comparison is less than a fifth threshold, the screen periphery region of the current frame is rendered using the screen periphery region image of the previous frame; or if the difference is greater than or equal to the fifth threshold, the screen periphery region is rendered using the attribute information of the mesh annotations corresponding to the screen periphery region and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0054] Secondly, embodiments of the present invention also provide a three-dimensional scene rendering apparatus, comprising:

[0055] The acquisition module is used to acquire the 3D data of the 3D scene to be rendered;

[0056] The first processing module is used to determine the status information of each available computing power node; wherein the status information includes one or more of the following: task scheduling delay, remaining computing power of the node, resource utilization rate, and policy intervention value.

[0057] The second processing module is used to input the state information of each computing node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes target computing nodes for 3D scene rendering.

[0058] The rendering module is used to render the three-dimensional data through the target computing node to obtain a three-dimensional rendering result.

[0059] Thirdly, embodiments of the present invention also provide a three-dimensional scene rendering system, including: a scheduling strategy generation device and multiple computing power nodes, wherein each computing power node is communicatively connected to the scheduling strategy generation device;

[0060] The scheduling strategy generation device is used to acquire 3D data of the 3D scene to be rendered; for each available computing power node, determine the status information of the computing power node; wherein the status information includes one or more of the following: task scheduling delay, node remaining computing power, resource utilization rate, and strategy intervention value; and input the status information of each computing power node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes the target computing power node for 3D scene rendering.

[0061] The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

[0062] Fourthly, embodiments of the present invention also provide a three-dimensional scene rendering device, including a processor and a transceiver, wherein the transceiver receives and transmits data under the control of the processor, and the processor is used to perform the following operations:

[0063] Obtain the 3D data of the 3D scene to be rendered;

[0064] For each available computing power node, determine the status information of the computing power node; wherein, the status information includes one or more of the following: task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value;

[0065] The state information of each computing node is input into the scheduling strategy generation model to obtain a scheduling strategy, which includes target computing nodes for 3D scene rendering.

[0066] The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

[0067] Fifthly, embodiments of the present invention also provide a three-dimensional scene rendering device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, it implements the three-dimensional scene rendering method as described in the first aspect.

[0068] In a sixth aspect, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the three-dimensional scene rendering method as described in the first aspect.

[0069] In a seventh aspect, embodiments of the present invention also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the three-dimensional scene rendering method as described in the first aspect.

[0070] The above-described technical solution of the present invention has at least the following beneficial effects:

[0071] In this embodiment of the invention, firstly, 3D data of the 3D scene to be rendered is acquired; then, for each available computing power node, the state information of the computing power node is determined; wherein, the state information includes one or more of the following: task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value; the state information of each computing power node is input into the scheduling policy generation model to obtain a scheduling policy, wherein the scheduling policy includes a target computing power node for 3D scene rendering; finally, the 3D data is rendered through the target computing power node to obtain a 3D rendering result. In this way, by using multiple scheduling influencing factors, namely task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value, as model inputs, a more balanced and efficient scheduling policy can be obtained, and then the 3D scene is rendered based on the target computing power node determined by the scheduling policy. The method of the present invention can improve scheduling efficiency and resource utilization. Attached Figure Description

[0072] Figure 1 A schematic diagram illustrating the processing flow of the three-dimensional scene rendering system according to an embodiment of the present invention;

[0073] Figure 2 A flowchart illustrating a three-dimensional scene rendering method according to an embodiment of the present invention;

[0074] Figure 3 A schematic diagram of the modules of a three-dimensional scene rendering device according to an embodiment of the present invention;

[0075] Figure 4 This is a schematic diagram illustrating the hardware structure of a three-dimensional scene rendering device according to an embodiment of the present invention. Detailed Implementation

[0076] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0077] To address the issues of low scheduling efficiency and resource utilization in existing rendering resource scheduling among nodes, this invention provides a 3D scene rendering method, apparatus, system, device, medium, and program product. The method and apparatus are based on the same patent concept. Since the methods and apparatus solve problems based on similar principles, their implementations can be mutually referenced, and repeated details will not be elaborated further.

[0078] Before detailing the method of this invention, for ease of understanding, the rendering process of the cloud-edge coordinated rendering system involved in this invention will be briefly described below. First, rendering resource scheduling (i.e., the scheduling strategy described below) is performed, that is, determining which rendering nodes (hereinafter referred to as computing nodes) will be used in the network. Second, based on the determined rendering nodes, the 3D scene is rendered using the rendering strategy to obtain the rendering result. Finally, the rendering result is returned to the application side. The overall processing flow of the system is as follows: Figure 1 As shown:

[0079] 1. Initiate a rendering request for rendering the 3D digital twin scene and start the system rendering process.

[0080] 2. Perform 3D data preprocessing before rendering.

[0081] 3. Rendering resources are allocated and scheduled through scheduling strategies.

[0082] Specifically, the scheduling strategy can be as follows: for 3D scenes with large data volume, high complexity, and low real-time requirements, the data is sent to the cloud master node for processing; for 3D scenes with small data volume, low complexity, and high real-time requirements, the data is sent to the edge node for processing.

[0083] 4. The cloud master node or edge node renders the 3D scene based on the corresponding rendering strategy and returns the rendering results.

[0084] 5. Return the rendered results to the terminal for display, either in combination or in batches.

[0085] like Figure 2 As shown in the figure, an embodiment of the present invention provides a three-dimensional scene rendering method, wherein the method may include:

[0086] Step 201: Obtain the 3D data of the 3D scene to be rendered.

[0087] This step may specifically include: receiving a rendering request, which includes 3D data of the 3D scene to be rendered. Here, the 3D scene may be a 3D digital twin scene.

[0088] Step 202: For each available computing power node, determine the status information of the computing power node; wherein the status information includes one or more of the following: task scheduling delay, remaining computing power of the node, resource utilization rate, and policy intervention value.

[0089] It should be understood that the policy intervention value is used to intervene in the scheduling score of a specific node in a specific scenario (such as rendering a high-precision model). In this invention, by increasing the policy intervention value of the cloud master node, the scheduling score of the cloud master node is improved. This ensures that high-load tasks are scheduled to the cloud master node, that is, during scheduling, high-level data will be preferentially or even forcibly allocated to the cloud master node with powerful computing power for rendering.

[0090] Step 203: Input the state information of each computing node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes target computing nodes for 3D scene rendering;

[0091] The target computing node can be a cloud master node or an edge node. It should be understood that the scheduling strategy, including the target computing node, refers to assigning the rendering tasks of the 3D scene to that target computing node.

[0092] This invention generates a scheduling strategy input by using a pre-trained scheduling strategy generation model. The model input consists of the state information of each computing node, and the output is the scheduling strategy. This invention comprehensively considers multiple scheduling influencing factors, namely task scheduling latency, remaining node computing power, resource utilization, and strategy intervention values, as model inputs. This results in a more balanced and efficient scheduling strategy output by the model.

[0093] Step 204: Render the three-dimensional data using the target computing node to obtain a three-dimensional rendering result.

[0094] The 3D scene rendering method of the present invention first acquires the 3D data of the 3D scene to be rendered; then, for each available computing power node, the state information of the computing power node is determined; wherein, the state information includes one or more of the following: task scheduling delay, node remaining computing power, resource utilization rate, and policy intervention value; the state information of each computing power node is input into a scheduling policy generation model to obtain a scheduling policy, wherein the scheduling policy includes a target computing power node for 3D scene rendering; finally, the 3D data is rendered through the target computing power node to obtain a 3D rendering result. In this way, by using multiple scheduling influencing factors, namely task scheduling delay, node remaining computing power, resource utilization rate, and policy intervention value, as model inputs, a more balanced and efficient scheduling policy can be obtained, and then the 3D scene is rendered based on the target computing power node determined by the scheduling policy. The method of the present invention can improve scheduling efficiency and resource utilization.

[0095] In some embodiments, step 202 above, determining the state information of the computing node, includes:

[0096] Collect information on the processing capacity, resource status, hardware load, and policy intervention values ​​of the computing nodes in specific scenarios;

[0097] Optionally, the processing capacity information includes at least one of the following: the amount of data transmitted by the computing power node, the transmission bandwidth, the number of currently queued tasks, and the average queuing latency of the tasks; the resource status information includes the total computing power and the computing power used by the computing power node; and the hardware load information includes at least one of the following: CPU utilization, GPU utilization, memory occupancy, and video memory occupancy.

[0098] It should be noted that the strategy intervention value V n,s This represents a custom value for the nth computing node in a specific scenario, used for appropriate intervention in data scheduling for special scenarios. The default value is 0.

[0099] The task scheduling delay is calculated based on the processing capacity information.

[0100] Among them, the task scheduling latency T of the computing node delay Transmission delay can be calculated by dividing the amount of data transmitted by the computing node by the transmission bandwidth, queuing delay can be calculated by multiplying the current number of queued tasks by the average queuing delay of the tasks, and scheduling delay can be calculated by summing transmission delay and queuing delay.

[0101] The remaining computing power of the node is calculated based on the resource status information.

[0102] Among them, the remaining computing power C of the node compute =Total computing power - computing power used.

[0103] The resource utilization rate is calculated based on the hardware load information.

[0104] Among them, the resource utilization rate U of the computing power node rate It can be obtained by linearly weighting CPU utilization, GPU utilization, memory usage, and video memory usage.

[0105] In some embodiments, step 203 above, which involves inputting the state information of each computing node into the scheduling policy generation model to obtain a scheduling policy, includes:

[0106] For each computing node, the task scheduling latency, remaining computing power, resource utilization, and policy intervention value of the computing node are input into the scheduling policy generation model, and the following processing is performed:

[0107] Based on the weight coefficients corresponding to the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value, a linear weighted calculation is performed on the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value to obtain the scheduling score;

[0108] Among them, the scheduling score of computing nodes T delay Indicates task scheduling latency, C compute U represents the remaining computing power of a node. rate V represents resource utilization rate. n,s Let represent the policy intervention value, i represent the i-th scheduling, and α, β, δ, λ are dynamic weight coefficients that satisfy α+β+δ+λ=1.0. Specifically, it represents the weighted score of computing node n in the i-th scheduling. The higher the score, the higher the priority of the computing node being scheduled.

[0109] Based on the scheduling score, a scheduling strategy is obtained. The scheduling strategy includes a target computing power node, which is the computing power node with the highest scheduling score among all the computing power nodes.

[0110] Specifically, all computing power nodes are sorted from highest to lowest according to their scheduling scores, and the computing power node with the highest score is selected as the target computing power node.

[0111] The scheduling strategy includes assigning the 3D scene rendering task to the target computing power node. It's important to note that the scheduling strategy also includes making decisions regarding task migration between nodes. For example, for decomposable tasks supporting parallel rendering across multiple neighboring nodes (multiple neighboring nodes refer to nodes accessible to the terminal, including edge nodes and cloud master nodes), when the available computing power of edge nodes is insufficient, node scheduling latency is too high, or the rendering frame rate is too low, the idle GPU cluster of the cloud master node is invoked for rendering, and multiple master nodes can be used as backups for each other.

[0112] It should be noted that the scheduling policy generation model is pre-trained. This model includes a scheduling policy generation sub-model, a scheduling policy evaluation sub-model, and a reinforcement learning model. Before describing the training process of the scheduling policy generation model, the design concept of the model will be explained.

[0113] Among them, the scheduling strategy generation sub-model serves as the main model. Its model inputs include the task scheduling latency of the computing power node, the remaining computing power of the node, the resource utilization rate, and the strategy intervention value. The model output is the scheduling strategy, which includes the computing power node with the highest scheduling score.

[0114] The scheduling strategy evaluation sub-model is used to assess the effectiveness of scheduling strategies. Its design principle is as follows:

[0115] Evaluation metrics include end-to-end latency, end-to-end frame rate, and policy intervention value. By linearly weighting these metrics, the effectiveness of the scheduling policy can be more accurately evaluated. The specific scheduling policy evaluation method is as follows: .

[0116] in, It represents end-to-end latency, reflecting the speed of interactive response; It represents the end-to-end frame rate, reflecting the smoothness of the video. This represents the policy intervention value, which defaults to 0; φ+μ+γ=1.0. It is an evaluation metric for the effectiveness of scheduling strategies. The larger the value, the better the scheduling effect, that is, the lower the latency generated by scheduling and the higher the rendering frame rate.

[0117] It's worth noting that on the application side (i.e., the terminal), frame rate and latency measurement tools such as webrtc-internals and JavaScript can be used to collect and calculate end-to-end latency (Time) and end-to-end frame rate (Frame). Webrtc-internals is a built-in Web Real-Time Communication (WebRTC) debugging tool in the Chrome browser. JavaScript is a lightweight scripting language primarily used for web development to add interactive functionality to websites.

[0118] This invention introduces a scheduling strategy evaluation mechanism, which comprehensively weights multiple factors such as end-to-end latency, end-to-end frame rate, and strategy intervention value to achieve an accurate evaluation of the scheduling strategy's effectiveness.

[0119] Finally, the scheduling strategy is continuously optimized using scheduling scores and scheduling evaluation scores through a deep reinforcement learning model.

[0120] The training process of the scheduling policy generation model is described below. As an optional implementation, the method of this invention further includes:

[0121] For each model training iteration, the historical task scheduling latency, historical node remaining computing power, historical resource utilization rate, and policy intervention value of each computing node are input into the scheduling policy to generate a sub-model, thereby obtaining the historical scheduling policy.

[0122] The end-to-end latency, end-to-end frame rate, and corresponding policy intervention value output by the historical scheduling strategy are input into the scheduling strategy evaluation sub-model to obtain the historical scheduling evaluation score.

[0123] The historical scheduling score and the historical scheduling evaluation score corresponding to the historical scheduling strategy are used as input states and fed into the reinforcement learning model. Guided by the reward function, action k is output. Action k includes the network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model. The network parameters of the scheduling strategy generation sub-model include the weight coefficients corresponding to the historical task scheduling latency of the computing node, the remaining computing power of the historical node, the historical resource utilization rate, and the strategy intervention value. The network parameters of the scheduling strategy evaluation sub-model include the end-to-end latency and end-to-end frame rate output using the historical scheduling strategy, as well as the weight coefficients corresponding to the respective strategy intervention values.

[0124] The network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model are adjusted and updated using the objective loss function until the training termination condition is met, thus obtaining the scheduling strategy generation model.

[0125] In this implementation, a scheduling policy evaluation mechanism is used, with the scheduling policy evaluated based on a score. i and Quota i Historical and real-time data are used to train scheduling strategies using reinforcement learning models (optionally, including deep Q-networks). The scheduling strategy parameters can be dynamically adjusted to achieve optimal resource allocation. Based on the evaluation results, task allocation and migration decisions are adjusted to optimize the overall scheduling performance.

[0126] The specific training methods, procedures, and steps are explained below:

[0127] Input to a reinforcement learning model: state s t =[historical score] i Distribution, Historical Quota i distributed].

[0128] Training a reinforcement learning model: Action a t =[{α,β,δ,λ},{φ,μ,γ}].

[0129] Where α, β, δ, and λ are the network parameters of the scheduling strategy generation sub-model, and φ, μ, and γ are the network parameters of the scheduling strategy evaluation sub-model.

[0130] Output of the reinforcement learning model: reward r t =Quota i Target maximization Quota i .

[0131] Training process:

[0132] ① Initialize the experience replay cache;

[0133] Create a playback pool to store quadruples (s t ,a t ,r t ,s t+1 );

[0134] ②Utilizing exploration-equilibrium Strategy, based on probability Randomly select an action (exploration) with probability. Choose maxQ(s) t ,a;θ), The score decreases linearly from an initial value of 0.99 to a final value of 0.01. A represents the action behavior in the reinforcement learning model (such as the DQN neural network), used to dynamically adjust the score. i and Quota i For the action of the weight parameters, refer to the representation method of the DQN neural network model.

[0135] ③ Update parameters based on the loss function;

[0136] Objective function: ω represents the contribution of the discount factor to weighted future returns.

[0137] Here, the objective function is defined as the optimal goal or minimum loss sought by the algorithm model, focusing on long-term cumulative returns.

[0138] Target loss function: Loss is calculated using variance, parameters To prevent overfitting, here, Represents the parameters of the objective function. These are the parameters of the evaluation network (here referring to the scheduling policy generation sub-model and the scheduling policy evaluation sub-model). In the DQN neural network model, both are neural network parameters. Usually through To retrieve values, such as by directly copying... Assignment can also be a proportional system and For multiplication assignment, the default is to assign the value directly, but the value can also be assigned after adjusting the ratio coefficient in the configuration algorithm.

[0139] Gradient update: , The learning rate is 0.001, which controls the update step size. The parameters are... It is the set of weights of the neural network (referring here to the scheduling policy generation sub-model and the scheduling policy evaluation sub-model).

[0140] ④ Target network synchronization, updating target network parameters in each step: θ − ←θ. Here, the target network refers to the scheduling policy generation sub-model and the scheduling policy evaluation sub-model.

[0141] This invention introduces improved strategies, including assigning sample weights based on the error of the target loss function to accelerate convergence, adding dense rewards to guide learning, and discretizing the action space.

[0142] Training is terminated when the average reward of the sliding window changes by less than 1% over 100 consecutive iterations, or when the maximum number of training rounds (10,000) is reached.

[0143] By combining the above training with simulation data, a better scheduling strategy can be obtained. For example, the reinforcement learning model has learned that latency-sensitive rendering tasks should be preferentially allocated to high-bandwidth, low-latency nodes.

[0144] This invention comprehensively considers multiple factors, including task scheduling latency, remaining node computing power, node resource utilization, and custom values ​​(policy intervention values) for specific scenarios. The method integrates and analyzes necessary influencing factors to achieve more efficient and balanced resource scheduling among multiple nodes in the system, thereby improving overall rendering efficiency and resource utilization. Furthermore, by utilizing a deep reinforcement learning model for pre-training and dynamic adjustment of the scheduling strategy, it can dynamically optimize the scheduling strategy based on real-time data feedback, adapting to changing needs in different scenarios.

[0145] After determining the scheduling strategy based on the above method, the rendering task of the 3D data is assigned to the target computing power node, which then performs the rendering work using the rendering strategy. Regarding the system rendering strategy, a typical existing technical solution is as follows: Solution 1 uses a hierarchical and batch rendering strategy. It determines the distance between the 3D model and the current position of the view frustum based on information such as the current position of the view frustum, the forward direction, and the 3D model hierarchy, and uses different levels of detail for rendering. It calculates the priority score of the target 3D model by combining parameters such as depth layer distance index, accuracy index, and occlusion index. The priority score is compared with a preset threshold, and then rendered and optimized in batches. Dynamic loading of hierarchical data is achieved through the division of the visual center and sub-levels. The model with high detail is rendered in the visual center region, and the corresponding central sub-level is rendered with the highest quality. Other sub-levels are loaded into memory and rendered with the lowest level of mesh loading. However, this approach still presents several challenges: Predicting and preloading data by combining the view frustum direction and historical trajectory data is problematic because the camera's position and rotation angle in real-time cloud rendering scenarios are highly random, making it difficult to effectively predict and preload data for the next moment. Rendering in batches after calculating priority scores using the model's depth layer distance index, accuracy index, and occlusion index requires pre-rendering and priority calculation of the target 3D model in the scene. In large-scale 3D scenes, this results in a significant additional computational load for frame rendering, impacting the overall rendering efficiency of the system. Dynamically loading hierarchical data through visual center and sub-level division lacks integration with the actual scene depth information of the model, potentially leading to distortion in close-up scenes. Furthermore, the dynamic switching between multiple sub-levels places high demands on system resources, making smooth switching between multiple sub-levels difficult.

[0146] Option 2 uses scene camera-related data for preloading and pre-rendering. It combines camera displacement, velocity, and angle data to construct a displacement prediction function to predict trajectory coordinates and perform scene data preloading and pre-rendering. Although the preloading and pre-rendering strategy based on scene camera-related data combines trajectory coordinates, device velocity, and angle for scene pre-rendering, it does not fully combine historical data and model hierarchical information, which can easily cause rendering stuttering in large 3D scenes.

[0147] Option 3 uses scene-related performance parameters for dynamic rendering. It sets screen-space error (SSE) and frame rate (FPS) thresholds. When the scene camera changes, it dynamically updates the true 3D model tile data and the virtual 3D geographic scene in real time based on geographic features, visibility, FPS, and Level of Detail (LOD). When the camera exceeds a certain preset threshold, geographic feature tiles are culled. While this strategy of dynamic rendering based on scene performance parameters optimizes loading through SSE, FPS, and geographic feature classification, introducing distance thresholds for model culling and forcibly lowering the SSE threshold to load low-precision 3D models may result in the loss of accuracy for some key models.

[0148] To address the aforementioned issues, this invention proposes a novel rendering mechanism that improves data loading strategies and rendering efficiency, effectively controls system resource consumption, and ensures stable and efficient system operation in large-scene rendering. Specifically, in some embodiments, step 104 above, which renders the 3D data through the target computing node to obtain a 3D rendering result, includes:

[0149] The three-dimensional data is divided according to the scene range to obtain grid data corresponding to multiple spatial grid regions;

[0150] It should be noted that, before dividing the three-dimensional data, the method of this invention further includes:

[0151] The 3D data is preprocessed. Specifically, preprocessing includes data lightweighting (removing data that does not need to be rendered), layer management (facilitating dynamic rendering and loading, such as rendering data closer to the camera based on a higher-level model), data annotation (i.e., data attribute and feature annotations, which facilitate subsequent processing and judgment), and format conversion (converting formats that are not suitable for rendering to those that are suitable for rendering).

[0152] The grid data of each spatial grid region is hierarchically divided to obtain the hierarchical grid data of the spatial grid region;

[0153] This invention further divides the aforementioned spatial grid region into different levels of hierarchical grids through hierarchical mesh partitioning. Different levels of mesh data correspond to different rendering precision and detail. The hierarchical mesh data is a subset of the mesh data within the spatial grid region.

[0154] This invention also requires the annotation of hierarchical grid attributes to optimize rendering data preloading and rendering efficiency. Specifically, each grid is labeled with its number, location, level, model type (classified by combining Geographic Information System (GIS) data for roads, green spaces, buildings, water features, green belts, roadbed equipment, etc.) and related attributes. The texture color of the grid model (i.e., the 3D data corresponding to the grid) is sampled and the default texture color is set periodically. The recent access time and rendering hit rate of each grid's rendering data are recorded. At the same time, the hierarchical grid rendering hit rate Hrate(x,y,z,h) is calculated, where x, y, and z represent the grid spatial location code, and h represents the grid level.

[0155] At least a portion of the hierarchical mesh data from all the hierarchical mesh data is loaded into the target computing node, and the target computing node performs rendering based on the at least a portion of the hierarchical mesh data to obtain a 3D rendering result.

[0156] This invention redesigns a layered, node-based rendering mechanism. Large-scale scene models (3D data) are divided into layers based on accuracy requirements and application scenarios, primarily into a three-tiered system: low-level models (resolution ≥ 50cm), suitable for large-scale terrain overviews and regional planning; mid-level models (resolution 5-50cm), suitable for urban area-level 3D modeling and display; and high-level models (resolution 1-5cm), suitable for detailed 3D model display for cultural heritage, engineering monitoring, etc. After dividing the large-scale scene model into multiple layers, each layer corresponds to different rendering accuracies and levels of detail.

[0157] This invention enhances the score of high-level data on the cloud master node through the aforementioned strategy intervention value. It enables low- and mid-level data to be rendered on edge computing nodes determined by the aforementioned scheduling strategy, while high-level data is rendered on a few cloud master nodes. This allows data of different levels to be loaded onto different nodes in a sequential manner from low to high levels. Furthermore, combined with the rendering mechanism of this invention, it is possible to dynamically render data by selecting cloud master nodes or edge nodes based on information such as the position and angle of the rendering camera. Through the collaborative work between nodes, efficient data loading and rendering can be achieved.

[0158] Based on this, the above steps, loading at least a portion of the hierarchical grid data from all the hierarchical grid data into the target computing node, include:

[0159] Load all the hierarchical grid data that meet the target conditions into the memory of the target computing node;

[0160] Based on the camera's viewing distance range and the camera's viewport vector direction, the hierarchical grid data in the memory is loaded into the video memory of the target computing node.

[0161] Specifically, the target condition includes at least one of the following conditions:

[0162] (1) The most recent access time difference of the hierarchical grid data is less than the first threshold, and the most recent access time difference of the hierarchical grid data is the difference between the current time of the hierarchical grid data and the time when the hierarchical grid data was last accessed;

[0163] Here, we first calculate the difference between the current time of the hierarchical grid data and the time when the hierarchical grid data was last accessed, i.e., the most recent access time difference ΔT of the hierarchical grid data. Then, we filter out data where ΔT is less than the first threshold T. max The hierarchical grid data is then processed, and the selected hierarchical grid data is finally loaded into the memory of the target computing node.

[0164] (2) The hit rate of the hierarchical grid data is greater than the second threshold, and the hit rate of the hierarchical grid data is the proportion of the number of accesses to the hierarchical grid data in the total number of accesses to all the hierarchical grid data;

[0165] Here, we first calculate the percentage of hierarchical grid data accesses in the total number of accesses to all hierarchical grid data, i.e., the hit rate of hierarchical grid data Hrate = Hcount / SUM(Hcount), where Hcount represents the number of accesses to hierarchical grid data and SUM(Hcount) represents the total number of accesses to all hierarchical grid data; then, we filter out data with a hit rate greater than the second threshold C. min The hierarchical grid data is loaded into the memory of the target computing node.

[0166] (3) The preloading priority value of the hierarchical grid data is greater than the third threshold. The preloading priority value of the hierarchical grid data is obtained by linearly weighting the recent access time difference and the hit rate of the hierarchical grid data based on the weight coefficients corresponding to the recent access time difference and the hit rate of the hierarchical grid data.

[0167] Here, based on the weight coefficients corresponding to the recent access time difference and hit rate of the hierarchical grid data, a linear weighted calculation is first performed on the recent access time difference and hit rate of the hierarchical grid data to obtain the preloading priority value of the hierarchical grid data, i.e., the preloading priority value of the hierarchical grid data. Then, if the Pri value is greater than the third threshold P, min The hierarchical grid data is loaded into the memory of the target computing node, where α and β are set empirical values.

[0168] (4) The hierarchical grid data is the hierarchical grid data outside the camera's field of view;

[0169] (5) The hierarchical grid data is the hierarchical grid data that is adjacent to the hierarchical grid data outside the camera's field of view.

[0170] For the layered grid data outside the camera's field of view, obtain its display grid layer h, and preload its h+1 and h-1 layer data (if they exist) into memory. Data already in memory is directly cached.

[0171] This invention utilizes the camera's viewing distance and viewport vector direction to preload video memory data. Normally, rendering data flows from the hard drive to RAM and then to video memory. Therefore, in this invention, the pre-rendered data is first loaded into RAM according to the above process, and then video memory data is loaded based on the camera's viewing distance and viewport vector direction.

[0172] Specifically, in some embodiments, loading the hierarchical grid data in memory into the video memory of the target computing node according to the camera's viewing distance range and camera viewport vector direction includes:

[0173] The hierarchical grid data within the first hemispherical arc region formed by expanding the camera's viewing distance range and the positive and negative directions of the camera's viewport vector direction outward by a preset angle in the memory is loaded into the video memory.

[0174] Here, for example, hierarchical grid data is formed within a 180-degree hemispherical arc region by expanding the camera's viewing distance range and the positive and negative directions of the camera's viewport vector by 90 degrees each, and then loaded into the video memory.

[0175] The hierarchical grid data within the second hemispherical arc region, formed by translating the first hemispherical arc region a preset distance along the camera's movement direction, is loaded into the video memory.

[0176] Here, the first hemispherical arc region is translated along the camera movement direction by a preset distance (a settable threshold d) to form a new hemispherical arc region, namely the second hemispherical arc region. The hierarchical grid data in the second hemispherical arc region is loaded into the video memory, and the data already in the video memory is directly cached.

[0177] This invention addresses a data loading method for large-scale scene models. First, it combines hierarchical grids and GIS data to label model data attributes within the grid, recording information such as the most recent access time and rendering hit rate of the grid rendering data. Second, it performs weight calculation and analysis using a weighted function, and preloads the grid data into memory when the weight value reaches a certain threshold. Finally, it predicts and loads video memory data based on the viewing distance, camera viewport forward vector, and camera running direction parameters, thereby improving the efficiency of data flow, reducing rendering resource requirements, increasing data loading speed, and solving the stuttering problem in large-scale scene data rendering.

[0178] As an optional implementation, the above-mentioned rendering by the target computing node based on the at least partial layer grid data to obtain the 3D rendering result includes:

[0179] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0180] For each spatial grid region, the region within the camera's field of view and whose position is less than the viewing distance threshold is rendered using its corresponding high-level grid data, and / or the region within the camera's field of view and whose position is greater than or equal to the viewing distance threshold is rendered using its corresponding low-level grid data, to obtain the 3D rendering result;

[0181] Here, screen error adjustments can be made separately for each spatial grid region, and the objects within that grid region can be analyzed. Specifically, within the camera's field of view and where the camera position is less than the viewing distance threshold D... max (This threshold is the distance at which the camera needs to render a high-precision model, not the furthest distance the camera can observe in the scene.) For the region where this threshold is located, high-level mesh data (i.e., a high-precision model) is used for rendering. In other words, the mesh data in this region has a low SSE (Screen Error Correction) value, thus maintaining a high-precision model within the near-field-of-view range. Meanwhile, within the camera's field of view, the distance from the camera position to the view distance threshold D is greater than or equal to this threshold. max The region is rendered using the corresponding low-level mesh data (i.e., a low-precision model). In other words, the mesh data in this region has a high SSE screen error value, displaying a low-precision model.

[0182] Among them, the viewing distance threshold D max =k*Ln(Height)+B, where Height is the camera's height above the ground (initial value is 1), k is an adjustment coefficient, and B is a custom base view distance threshold. As the camera height increases, the view distance threshold will gradually increase. Appropriate pixel color interpolation is performed at the seams of models with different precision. Compared with the traditional static screen error calculation scheme, this can further improve the loading efficiency of large-scale data in large scenes and the smooth transition of rendering effects.

[0183] Alternatively, if the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the central area of ​​the screen is rendered using its corresponding high-level mesh data; otherwise, the central area of ​​the screen is rendered using its corresponding low-level mesh data. If the difference between the pixels of the current frame's rendered image and the pixels of the previous frame's rendered image, calculated using a global feature vector or hash comparison, is less than a fourth threshold, the peripheral area of ​​the current frame's image is rendered using the peripheral area image of the previous frame. Alternatively, if the difference is greater than or equal to the fourth threshold, the peripheral area of ​​the screen is rendered using the attribute information of the mesh annotations corresponding to the peripheral area of ​​the screen and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0184] Here, the screen center area is selected by obtaining the screen resolution and aspect ratio, taking the geometric center of the monitor as the screen center, and reducing each aspect ratio by a preset percentage (such as 0.5). The area outside the screen center area is the screen perimeter area.

[0185] It should be noted that this invention improves rendering efficiency by first rendering the hierarchical mesh model shape and texture of the central area of ​​the screen that the user is focusing on, and then mapping the screen center point to the view frustum scene center point. The specific rendering strategy is as follows: First, basic rendering optimization is performed. Real-time rendering involves camera viewport and dynamic data detection. When the camera position, rotation angle, and scene element data remain unchanged, and the scene model and materials have no temporal correlation characteristics, the real-time scene rendering frame rate is automatically reduced to save rendering resources. Static and dynamic pixel regions in the scene are identified through image detection, and only dynamic pixel regions are rendered in real-time, thereby improving rendering efficiency. That is, when the camera's movement speed is less than or equal to a speed threshold S... max And / or the rotation speed is less than or equal to the rotation threshold R max In the case of a certain condition, the high-level mesh data corresponding to the center area of ​​the screen is rendered (that is, the model shape and real texture in the mesh corresponding to the level of the screen center area that the user is focusing on, i.e., the mesh model and texture near the center point of the view frustum scene). Otherwise, the low-level mesh data corresponding to the center area of ​​the screen is rendered to reduce rendering complexity and avoid scene stuttering and unnecessary resource waste caused by excessive camera shaking.

[0186] After rendering the central area of ​​the screen, the surrounding area of ​​the screen is rendered. The specific rendering strategy is as follows:

[0187] First, the pixels of the rendered image in the current frame are compared with the pixels of the rendered image in the previous frame using a global feature vector or hash comparison algorithm to quickly calculate the difference. If the difference is less than the fourth threshold P, then... maxIn the case of [a specific condition], the screen perimeter region of the current frame is rendered using the image of the screen perimeter region of the previous frame; it should be noted that the obtained difference is less than the fourth threshold P. max This indicates that the changes between frames are relatively small. Therefore, when the rendering of the screen perimeter area of ​​the current frame is not yet complete, the screen perimeter area image of the previous frame can be used as a temporary substitute. A certain linear difference and blending overlap transition is performed between the pixel colors of the center and perimeter areas of the screen to avoid image seam issues. If the obtained difference is greater than or equal to the fourth threshold P... max In this case, the attribute information of the grid annotations corresponding to the screen perimeter area and the corresponding sampled texture or color texture are used to render the screen perimeter area; it should be noted that the obtained difference is greater than or equal to the fourth threshold P. max This indicates that there are significant changes between frames. At this point, we analyze the surrounding area's buildings, green spaces, water systems, roads, green belts, and other elements, remove the complex material textures of these models, and use their grid annotation attribute information and corresponding sampled textures or color textures for fast rendering. This allows us to replace distant models in the surrounding area with default textures or sampled textures, reducing the computational load of the system rendering. After calculating that the camera's movement position and rotation angle are relatively stable, we then gradually render the models in the surrounding area and distant scenes.

[0188] This invention prioritizes rendering the central area of ​​the screen because users naturally focus on this area. Furthermore, many digital twin visualization applications place opaque or semi-transparent UI elements on either side of the screen, overlaying the rendered scene image; therefore, these areas are not the primary focus for users. Additionally, the spatial distance between the model and the camera, as well as the scene depth, within the camera's field of view are detected. When these distances and depths are below set thresholds, the same priority rendering strategy as for the central area is applied to these models, preventing distortion of nearby models in the peripheral areas due to prioritizing the central area.

[0189] As another optional implementation, the above-mentioned rendering by the target computing node based on the at least partial hierarchical mesh data to obtain the 3D rendering result includes:

[0190] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0191] For each spatial grid region, the high-level grid data corresponding to the region within the camera's field of view where the camera position is less than the viewing distance threshold, and / or the low-level grid data corresponding to the region within the camera's field of view where the camera position is greater than or equal to the viewing distance threshold, are determined as the level grid data to be rendered.

[0192] Similar to the previous implementation, screen error adjustments can be made separately for each spatial grid region. The difference is that within the camera's field of view and when the camera position is less than the viewing distance threshold D, adjustments are made separately. max At that time, the high-level mesh data corresponding to that area is determined as the layer mesh data to be rendered (whether to use this data for rendering depends on the subsequent rendering strategy); within the camera's field of view and at a distance greater than or equal to the camera position, the following layers are considered: max The region is determined by the low-level mesh data corresponding to that region as the level mesh data to be rendered (whether to use this data for rendering depends on the subsequent rendering strategy).

[0193] Next, when the camera's moving speed is less than or equal to a speed threshold and / or its rotation speed is less than or equal to a rotation threshold, the target layer mesh data corresponding to the screen center region in the layer mesh data to be rendered is determined, and the screen center region is rendered using the target layer mesh data.

[0194] Here, when the camera's moving speed is less than or equal to the speed threshold S max And / or the rotation speed is less than or equal to the rotation threshold R max In this case, target layer mesh data corresponding to the center area of ​​the screen is selected from the layer mesh data to be rendered, and the center area of ​​the screen is rendered using the target layer mesh data.

[0195] Otherwise, the central area of ​​the screen is rendered using the attribute information of the grid annotations corresponding to the central area of ​​the screen and the corresponding sampled texture or color texture; this reduces rendering complexity and avoids scene stuttering and unnecessary resource waste caused by excessive camera shake.

[0196] For the screen perimeter area: if the difference between the pixels of the rendered image in the current frame and the pixels of the rendered image in the previous frame, calculated by comparing the global feature vector or hash, is less than the fifth threshold, the screen perimeter area of ​​the current frame is rendered using the screen perimeter area image of the previous frame; or if the difference is greater than or equal to the fifth threshold, the screen perimeter area is rendered using the attribute information of the grid annotation corresponding to the screen perimeter area and the corresponding sampled texture or color texture to obtain a three-dimensional rendering result.

[0197] The rendering strategy for the area around the screen is described in the above implementation and will not be repeated here.

[0198] This invention redesigns the large-scene model rendering mechanism, which has lower computational complexity than existing solutions and is more suitable for real-time cloud rendering scenarios. It also combines data preloading methods and screen regional rendering strategies to further improve overall rendering efficiency. In addition, it optimizes the screen error mechanism and introduces a frame-reduction rendering mechanism for static scenes.

[0199] like Figure 3 As shown, an embodiment of the present invention provides a three-dimensional scene rendering device, including:

[0200] The acquisition module 301 is used to acquire the 3D data of the 3D scene to be rendered;

[0201] The first processing module 302 is used to determine the status information of each available computing power node; wherein the status information includes one or more of the following: task scheduling delay, remaining computing power of the node, resource utilization rate, and policy intervention value.

[0202] The second processing module 303 is used to input the state information of each computing node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes target computing nodes for 3D scene rendering.

[0203] The rendering module 304 is used to render the three-dimensional data through the target computing node to obtain a three-dimensional rendering result.

[0204] In some embodiments, the first processing module 302 includes:

[0205] The acquisition unit is used to acquire information on the processing power of the computing nodes, resource status, hardware load, and the policy intervention value of the nodes in a specific scenario.

[0206] The first computing unit is used to calculate the task scheduling delay based on the processing capacity information.

[0207] The second computing unit is used to calculate the remaining computing power of the node based on the resource status information.

[0208] The third calculation unit is used to calculate the resource utilization rate based on the hardware load information.

[0209] Optionally, the processing capability information includes at least one of the following: the amount of data transmitted by the computing node, the transmission bandwidth, the number of currently queued tasks, and the average queuing latency of the tasks.

[0210] The resource status information includes the total computing power and the computing power used by the computing power node;

[0211] The hardware load information includes at least one of CPU utilization, GPU utilization, memory usage, and video memory usage.

[0212] In some embodiments, the second processing module 303 includes:

[0213] The first processing unit is configured to input the task scheduling latency, remaining computing power, resource utilization, and policy intervention value of each computing power node into the scheduling policy generation model, and perform the following processing:

[0214] Based on the weight coefficients corresponding to the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value, a linear weighted calculation is performed on the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value to obtain the scheduling score;

[0215] Based on the scheduling score, a scheduling strategy is obtained. The scheduling strategy includes a target computing power node, which is the computing power node with the highest scheduling score among all the computing power nodes.

[0216] In some embodiments, the scheduling policy generation model includes a scheduling policy generation sub-model, a scheduling policy evaluation sub-model, and a reinforcement learning model; the apparatus of the present invention further includes:

[0217] The third processing module is used to input the historical task scheduling latency, historical node remaining computing power, historical resource utilization rate and policy intervention value of each computing node into the scheduling policy to generate a sub-model for each model training, and to obtain the historical scheduling policy.

[0218] The fourth processing module is used to input the end-to-end latency, end-to-end frame rate and corresponding policy intervention value output by the historical scheduling strategy into the scheduling strategy evaluation sub-model to obtain the historical scheduling evaluation score.

[0219] The fifth processing module is used to input the historical scheduling score and the historical scheduling evaluation score corresponding to the historical scheduling strategy into the reinforcement learning model, and then output action k guided by the reward function. Action k includes the network parameters of the scheduling strategy generation sub-model and the network parameters of the scheduling strategy evaluation sub-model. The network parameters of the scheduling strategy generation sub-model include the weight coefficients corresponding to the historical task scheduling latency of the computing node, the remaining computing power of the historical node, the historical resource utilization rate, and the strategy intervention value. The network parameters of the scheduling strategy evaluation sub-model include the end-to-end latency and end-to-end frame rate output by the historical scheduling strategy, as well as the weight coefficients corresponding to the corresponding strategy intervention values.

[0220] The sixth processing module is used to adjust and update the network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model using the objective loss function until the training termination condition is met, thereby obtaining the scheduling strategy generation model.

[0221] In some embodiments, the rendering module 304 includes:

[0222] The second processing unit is used to divide the three-dimensional data according to the scene range to obtain grid data corresponding to multiple spatial grid regions;

[0223] The third processing unit is used to hierarchically divide the grid data of each spatial grid region to obtain the hierarchical grid data of the spatial grid region.

[0224] The fourth processing unit is used to load at least a portion of the hierarchical mesh data from all the hierarchical mesh data into the target computing node, and the target computing node performs rendering based on the at least a portion of the hierarchical mesh data to obtain a three-dimensional rendering result.

[0225] In some embodiments, the fourth processing unit is specifically used for:

[0226] Load all the hierarchical grid data that meet the target conditions into the memory of the target computing node;

[0227] Based on the camera's viewing distance range and the camera's viewport vector direction, the hierarchical grid data in the memory is loaded into the video memory of the target computing node.

[0228] Optionally, the target condition includes at least one of the following conditions:

[0229] The most recent access time difference of the hierarchical grid data is less than a first threshold, and the most recent access time difference of the hierarchical grid data is the difference between the current time of the hierarchical grid data and the time when the hierarchical grid data was last accessed;

[0230] The hit rate of the hierarchical grid data is greater than the second threshold, and the hit rate of the hierarchical grid data is the proportion of the number of accesses to the hierarchical grid data in the total number of accesses to all the hierarchical grid data;

[0231] The preloading priority value of the hierarchical grid data is greater than the third threshold. The preloading priority value of the hierarchical grid data is obtained by linearly weighting the recent access time difference and the hit rate of the hierarchical grid data based on the weight coefficients corresponding to the recent access time difference and the hit rate of the hierarchical grid data.

[0232] The hierarchical grid data refers to the hierarchical grid data outside the camera's field of view.

[0233] The hierarchical grid data refers to the hierarchical grid data adjacent to the hierarchical grid data outside the camera's field of view.

[0234] In some embodiments, the fourth processing unit is specifically used for:

[0235] The hierarchical grid data within the first hemispherical arc region formed by expanding the camera's viewing distance range and the positive and negative directions of the camera's viewport vector direction outward by a preset angle in the memory is loaded into the video memory.

[0236] The hierarchical grid data within the second hemispherical arc region, formed by translating the first hemispherical arc region a preset distance along the camera's movement direction, is loaded into the video memory.

[0237] In some embodiments, the fourth processing unit is specifically used for:

[0238] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0239] For each spatial grid region, areas within the camera's field of view and whose position is less than the view distance threshold are rendered using their corresponding high-level grid data, and / or areas within the camera's field of view and whose position is greater than or equal to the view distance threshold are rendered using their corresponding low-level grid data, to obtain a 3D rendering result; or...

[0240] If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the central area of ​​the screen is rendered using its corresponding high-level mesh data; otherwise, the central area of ​​the screen is rendered using its corresponding low-level mesh data. If the difference between the pixels of the current frame's rendered image and the pixels of the previous frame's rendered image, calculated using a global feature vector or hash comparison, is less than a fourth threshold, the peripheral area of ​​the current frame's image is rendered using the peripheral area image of the previous frame. Alternatively, if the difference is greater than or equal to the fourth threshold, the peripheral area of ​​the screen is rendered using the attribute information of the mesh annotations corresponding to the peripheral area of ​​the screen and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0241] In some embodiments, the fourth processing unit is specifically used for:

[0242] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0243] For each spatial grid region, the high-level grid data corresponding to the region within the camera's field of view where the camera position is less than the viewing distance threshold, and / or the low-level grid data corresponding to the region within the camera's field of view where the camera position is greater than or equal to the viewing distance threshold, are determined as the level grid data to be rendered.

[0244] If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the target level mesh data corresponding to the screen center region in the level mesh data to be rendered is determined, and the screen center region is rendered using the target level mesh data; otherwise, the screen center region is rendered using the attribute information of the mesh annotations corresponding to the screen center region and the corresponding sampled texture or color texture; and if the difference between the pixels of the rendered image in this frame and the pixels of the rendered image in the previous frame calculated by global feature vector or hash comparison is less than a fifth threshold, the screen periphery region of the current frame is rendered using the screen periphery region image of the previous frame; or if the difference is greater than or equal to the fifth threshold, the screen periphery region is rendered using the attribute information of the mesh annotations corresponding to the screen periphery region and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0245] The 3D scene rendering apparatus of this invention first acquires 3D data of the 3D scene to be rendered; then, for each available computing power node, it determines the state information of the computing power node; wherein, the state information includes one or more of the following: task scheduling delay, remaining computing power of the node, resource utilization rate, and policy intervention value; the state information of each computing power node is input into a scheduling policy generation model to obtain a scheduling policy, the scheduling policy including a target computing power node for 3D scene rendering; finally, the 3D data is rendered through the target computing power node to obtain a 3D rendering result. In this way, by using multiple scheduling influencing factors, namely task scheduling delay, remaining computing power of the node, resource utilization rate, and policy intervention value, as model inputs, a more balanced and efficient scheduling policy can be obtained, and then the 3D scene is rendered based on the target computing power node determined by the scheduling policy. The method of this invention can improve scheduling efficiency and resource utilization.

[0246] To achieve the above objectives, the present invention also provides a three-dimensional scene rendering system, comprising: a scheduling strategy generation device and multiple computing power nodes, wherein each computing power node is communicatively connected to the scheduling strategy generation device;

[0247] The scheduling strategy generation device is used to acquire 3D data of the 3D scene to be rendered; for each available computing power node, determine the status information of the computing power node; wherein the status information includes one or more of the following: task scheduling delay, node remaining computing power, resource utilization rate, and strategy intervention value; and input the status information of each computing power node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes the target computing power node for 3D scene rendering.

[0248] The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

[0249] To better achieve the above objectives, such as Figure 4 As shown, this embodiment of the invention also provides a three-dimensional scene rendering device, including a processor 400 and a transceiver 410. The transceiver 410 receives and transmits data under the control of the processor 400, and the processor 400 is used to execute the following process:

[0250] Obtain the 3D data of the 3D scene to be rendered;

[0251] For each available computing power node, determine the status information of the computing power node; wherein, the status information includes one or more of the following: task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value;

[0252] The state information of each computing node is input into the scheduling strategy generation model to obtain a scheduling strategy, which includes target computing nodes for 3D scene rendering.

[0253] The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

[0254] In some embodiments, the processor 400 is further configured to:

[0255] Collect information on the processing capacity, resource status, hardware load, and policy intervention values ​​of the computing nodes in specific scenarios;

[0256] The task scheduling delay is calculated based on the processing capacity information.

[0257] Based on the resource status information, the remaining computing power of the node is calculated;

[0258] The resource utilization rate is calculated based on the hardware load information.

[0259] Optionally, the processing capability information includes at least one of the following: the amount of data transmitted by the computing node, the transmission bandwidth, the number of currently queued tasks, and the average queuing latency of the tasks.

[0260] The resource status information includes the total computing power and the computing power used by the computing power node;

[0261] The hardware load information includes at least one of CPU utilization, GPU utilization, memory usage, and video memory usage.

[0262] In some embodiments, the processor 400 is further configured to:

[0263] For each computing node, the task scheduling latency, remaining computing power, resource utilization, and policy intervention value of the computing node are input into the scheduling policy generation model, and the following processing is performed:

[0264] Based on the weight coefficients corresponding to the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value, a linear weighted calculation is performed on the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value to obtain the scheduling score;

[0265] Based on the scheduling score, a scheduling strategy is obtained. The scheduling strategy includes a target computing power node, which is the computing power node with the highest scheduling score among all the computing power nodes.

[0266] In some embodiments, the scheduling policy generation model includes a scheduling policy generation sub-model, a scheduling policy evaluation sub-model, and a reinforcement learning model; correspondingly, the processor 400 is further configured to:

[0267] For each model training, the historical task scheduling latency, historical node remaining computing power, historical resource utilization rate and policy intervention value of each computing node are respectively input into the scheduling policy to generate a sub-model and obtain the historical scheduling policy.

[0268] The end-to-end latency, end-to-end frame rate, and corresponding policy intervention value output by the historical scheduling strategy are input into the scheduling strategy evaluation sub-model to obtain the historical scheduling evaluation score.

[0269] The historical scheduling score and the historical scheduling evaluation score corresponding to the historical scheduling strategy are used as input states and fed into the reinforcement learning model. Guided by the reward function, action k is output. Action k includes the network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model. The network parameters of the scheduling strategy generation sub-model include the weight coefficients corresponding to the historical task scheduling latency of the computing node, the remaining computing power of the historical node, the historical resource utilization rate, and the strategy intervention value. The network parameters of the scheduling strategy evaluation sub-model include the end-to-end latency and end-to-end frame rate output using the historical scheduling strategy, as well as the weight coefficients corresponding to the respective strategy intervention values.

[0270] The network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model are adjusted and updated using the objective loss function until the training termination condition is met, thus obtaining the scheduling strategy generation model.

[0271] In some embodiments, the processor 400 is further configured to:

[0272] The three-dimensional data is divided according to the scene range to obtain grid data corresponding to multiple spatial grid regions;

[0273] The grid data of each spatial grid region is hierarchically divided to obtain the hierarchical grid data of the spatial grid region;

[0274] At least a portion of the hierarchical mesh data from all the hierarchical mesh data is loaded into the target computing node, and the target computing node performs rendering based on the at least a portion of the hierarchical mesh data to obtain a 3D rendering result.

[0275] In some embodiments, the processor 400 is further configured to:

[0276] Load all the hierarchical grid data that meet the target conditions into the memory of the target computing node;

[0277] Based on the camera's viewing distance range and the camera's viewport vector direction, the hierarchical grid data in the memory is loaded into the video memory of the target computing node.

[0278] In some embodiments, the target condition includes at least one of the following conditions:

[0279] The most recent access time difference of the hierarchical grid data is less than a first threshold, and the most recent access time difference of the hierarchical grid data is the difference between the current time of the hierarchical grid data and the time when the hierarchical grid data was last accessed;

[0280] The hit rate of the hierarchical grid data is greater than the second threshold, and the hit rate of the hierarchical grid data is the proportion of the number of accesses to the hierarchical grid data in the total number of accesses to all the hierarchical grid data;

[0281] The preloading priority value of the hierarchical grid data is greater than the third threshold. The preloading priority value of the hierarchical grid data is obtained by linearly weighting the recent access time difference and the hit rate of the hierarchical grid data based on the weight coefficients corresponding to the recent access time difference and the hit rate of the hierarchical grid data.

[0282] The hierarchical grid data refers to the hierarchical grid data outside the camera's field of view.

[0283] The hierarchical grid data refers to the hierarchical grid data adjacent to the hierarchical grid data outside the camera's field of view.

[0284] In some embodiments, the processor 400 is further configured to:

[0285] The hierarchical grid data within the first hemispherical arc region formed by expanding the camera's viewing distance range and the positive and negative directions of the camera's viewport vector direction outward by a preset angle in the memory is loaded into the video memory.

[0286] The hierarchical grid data within the second hemispherical arc region, formed by translating the first hemispherical arc region a preset distance along the camera's movement direction, is loaded into the video memory.

[0287] In some embodiments, the processor 400 is further configured to:

[0288] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0289] For each spatial grid region, areas within the camera's field of view and whose position is less than the view distance threshold are rendered using their corresponding high-level grid data, and / or areas within the camera's field of view and whose position is greater than or equal to the view distance threshold are rendered using their corresponding low-level grid data, to obtain a 3D rendering result; or...

[0290] If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the central area of ​​the screen is rendered using its corresponding high-level mesh data; otherwise, the central area of ​​the screen is rendered using its corresponding low-level mesh data. If the difference between the pixels of the current frame's rendered image and the pixels of the previous frame's rendered image, calculated using a global feature vector or hash comparison, is less than a fourth threshold, the peripheral area of ​​the current frame's image is rendered using the peripheral area image of the previous frame. Alternatively, if the difference is greater than or equal to the fourth threshold, the peripheral area of ​​the screen is rendered using the attribute information of the mesh annotations corresponding to the peripheral area of ​​the screen and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0291] In some embodiments, the processor 400 is further configured to:

[0292] The target computing node performs the following processing based on at least a portion of the hierarchical grid data:

[0293] For each spatial grid region, the high-level grid data corresponding to the region within the camera's field of view where the camera position is less than the viewing distance threshold, and / or the low-level grid data corresponding to the region within the camera's field of view where the camera position is greater than or equal to the viewing distance threshold, are determined as the level grid data to be rendered.

[0294] If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the target level mesh data corresponding to the screen center region in the level mesh data to be rendered is determined, and the screen center region is rendered using the target level mesh data; otherwise, the screen center region is rendered using the attribute information of the mesh annotations corresponding to the screen center region and the corresponding sampled texture or color texture; and if the difference between the pixels of the rendered image in this frame and the pixels of the rendered image in the previous frame calculated by global feature vector or hash comparison is less than a fifth threshold, the screen periphery region of the current frame is rendered using the screen periphery region image of the previous frame; or if the difference is greater than or equal to the fifth threshold, the screen periphery region is rendered using the attribute information of the mesh annotations corresponding to the screen periphery region and the corresponding sampled texture or color texture to obtain a 3D rendering result.

[0295] The 3D scene rendering device of this invention first acquires 3D data of the 3D scene to be rendered; then, for each available computing power node, it determines the status information of the computing power node; wherein, the status information includes one or more of the following: task scheduling latency, node remaining computing power, resource utilization rate, and policy intervention value; the status information of each computing power node is input into a scheduling policy generation model to obtain a scheduling policy, the scheduling policy including a target computing power node for 3D scene rendering; finally, the 3D data is rendered through the target computing power node to obtain a 3D rendering result. In this way, by using multiple scheduling influencing factors, namely task scheduling latency, node remaining computing power, resource utilization rate, and policy intervention value, as model inputs, a more balanced and efficient scheduling policy can be obtained, and then the 3D scene is rendered based on the target computing power node determined by the scheduling policy. The method of this invention can improve scheduling efficiency and resource utilization.

[0296] This invention also provides a three-dimensional scene rendering device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the various processes in the three-dimensional scene rendering method embodiments described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0297] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the various processes described above in the three-dimensional scene rendering method embodiments, achieving the same technical effects. To avoid repetition, further details are omitted here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0298] This invention also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes in the three-dimensional scene rendering method embodiments described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0299] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0300] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 A device for one or more processes and / or the functions specified in one or more boxes.

[0301] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce a paper article including an instruction means, the instruction means being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0302] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment, causing the computer or other programmable equipment to perform a series of operational steps to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0303] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A three-dimensional scene rendering method, characterized in that, include: Obtain the 3D data of the 3D scene to be rendered; For each available computing power node, determine the status information of the computing power node; wherein, the status information includes one or more of the following: task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value; The state information of each computing node is input into the scheduling strategy generation model to obtain a scheduling strategy, which includes target computing nodes for 3D scene rendering. The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

2. The method according to claim 1, characterized in that, Determining the status information of the computing node includes: Collect information on the processing capacity, resource status, hardware load, and policy intervention values ​​of the computing nodes in specific scenarios; The task scheduling delay is calculated based on the processing capacity information. Based on the resource status information, the remaining computing power of the node is calculated; The resource utilization rate is calculated based on the hardware load information.

3. The method according to claim 2, characterized in that, The processing capacity information includes at least one of the following: the amount of data transmitted by the computing node, the transmission bandwidth, the number of currently queued tasks, and the average queuing latency of the tasks. The resource status information includes the total computing power and the computing power used by the computing power node; The hardware load information includes at least one of CPU utilization, GPU utilization, memory usage, and video memory usage.

4. The method according to claim 1, characterized in that, The step of inputting the state information of each computing node into the scheduling strategy generation model to obtain the scheduling strategy includes: For each computing node, the task scheduling latency, remaining computing power, resource utilization, and policy intervention value of the computing node are input into the scheduling policy generation model, and the following processing is performed: Based on the weight coefficients corresponding to the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value, a linear weighted calculation is performed on the task scheduling delay, the remaining computing power of the node, the resource utilization rate, and the policy intervention value to obtain the scheduling score; Based on the scheduling score, a scheduling strategy is obtained. The scheduling strategy includes a target computing power node, which is the computing power node with the highest scheduling score among all the computing power nodes.

5. The method according to claim 1, characterized in that, The scheduling policy generation model includes a scheduling policy generation sub-model, a scheduling policy evaluation sub-model, and a reinforcement learning model; the method further includes: For each model training, the historical task scheduling latency, historical node remaining computing power, historical resource utilization rate and policy intervention value of each computing node are respectively input into the scheduling policy to generate a sub-model and obtain the historical scheduling policy. The end-to-end latency, end-to-end frame rate, and corresponding policy intervention value output by the historical scheduling strategy are input into the scheduling strategy evaluation sub-model to obtain the historical scheduling evaluation score. The historical scheduling score and the historical scheduling evaluation score corresponding to the historical scheduling strategy are used as input states and fed into the reinforcement learning model. Guided by the reward function, action k is output. Action k includes the network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model. The network parameters of the scheduling strategy generation sub-model include the weight coefficients corresponding to the historical task scheduling latency of the computing node, the remaining computing power of the historical node, the historical resource utilization rate, and the strategy intervention value. The network parameters of the scheduling strategy evaluation sub-model include the end-to-end latency and end-to-end frame rate output using the historical scheduling strategy, as well as the weight coefficients corresponding to the respective strategy intervention values. The network parameters of the scheduling strategy generation sub-model and the scheduling strategy evaluation sub-model are adjusted and updated using the objective loss function until the training termination condition is met, thus obtaining the scheduling strategy generation model.

6. The method according to claim 1, characterized in that, The process of rendering the 3D data using the target computing node to obtain a 3D rendering result includes: The three-dimensional data is divided according to the scene range to obtain grid data corresponding to multiple spatial grid regions; The grid data of each spatial grid region is hierarchically divided to obtain the hierarchical grid data of the spatial grid region; At least a portion of the hierarchical mesh data from all the hierarchical mesh data is loaded into the target computing node, and the target computing node performs rendering based on the at least a portion of the hierarchical mesh data to obtain a 3D rendering result.

7. The method according to claim 6, characterized in that, The step of loading at least a portion of the hierarchical grid data from all the hierarchical grid data into the target computing node includes: Load all the hierarchical grid data that meet the target conditions into the memory of the target computing node; Based on the camera's viewing distance range and the camera's viewport vector direction, the hierarchical grid data in the memory is loaded into the video memory of the target computing node.

8. The method according to claim 7, characterized in that, The target condition includes at least one of the following conditions: The most recent access time difference of the hierarchical grid data is less than a first threshold, and the most recent access time difference of the hierarchical grid data is the difference between the current time of the hierarchical grid data and the time when the hierarchical grid data was last accessed; The hit rate of the hierarchical grid data is greater than the second threshold, and the hit rate of the hierarchical grid data is the proportion of the number of accesses to the hierarchical grid data in the total number of accesses to all the hierarchical grid data; The preloading priority value of the hierarchical grid data is greater than the third threshold. The preloading priority value of the hierarchical grid data is obtained by linearly weighting the recent access time difference and the hit rate of the hierarchical grid data based on the weight coefficients corresponding to the recent access time difference and the hit rate of the hierarchical grid data. The hierarchical grid data refers to the hierarchical grid data outside the camera's field of view. The hierarchical grid data refers to the hierarchical grid data adjacent to the hierarchical grid data outside the camera's field of view.

9. The method according to claim 7, characterized in that, The step of loading the hierarchical grid data in memory into the video memory of the target computing node according to the camera's viewing distance range and camera viewport vector direction includes: The hierarchical grid data within the first hemispherical arc region formed by expanding the camera's viewing distance range and the positive and negative directions of the camera's viewport vector direction outward by a preset angle in the memory is loaded into the video memory. The hierarchical grid data within the second hemispherical arc region, formed by translating the first hemispherical arc region a preset distance along the camera's movement direction, is loaded into the video memory.

10. The method according to claim 6, characterized in that, The process of rendering by the target computing node based on at least a portion of the hierarchical mesh data to obtain a 3D rendering result includes: The target computing node performs the following processing based on at least a portion of the hierarchical grid data: For each spatial grid region, areas within the camera's field of view and whose position is less than the view distance threshold are rendered using their corresponding high-level grid data, and / or areas within the camera's field of view and whose position is greater than or equal to the view distance threshold are rendered using their corresponding low-level grid data, to obtain a 3D rendering result; or... If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the central area of ​​the screen is rendered using its corresponding high-level mesh data; otherwise, the central area of ​​the screen is rendered using its corresponding low-level mesh data. If the difference between the pixels of the current frame's rendered image and the pixels of the previous frame's rendered image, calculated using a global feature vector or hash comparison, is less than a fourth threshold, the peripheral area of ​​the current frame's image is rendered using the peripheral area image of the previous frame. Alternatively, if the difference is greater than or equal to the fourth threshold, the peripheral area of ​​the screen is rendered using the attribute information of the mesh annotations corresponding to the peripheral area of ​​the screen and the corresponding sampled texture or color texture to obtain a 3D rendering result.

11. The method according to claim 6, characterized in that, The process of rendering by the target computing node based on at least a portion of the hierarchical mesh data to obtain a 3D rendering result includes: The target computing node performs the following processing based on at least a portion of the hierarchical grid data: For each spatial grid region, the high-level grid data corresponding to the region within the camera's field of view where the camera position is less than the viewing distance threshold, and / or the low-level grid data corresponding to the region within the camera's field of view where the camera position is greater than or equal to the viewing distance threshold, are determined as the level grid data to be rendered. If the camera's moving speed is less than or equal to a speed threshold, and / or its rotation speed is less than or equal to a rotation threshold, the target level mesh data corresponding to the screen center region in the level mesh data to be rendered is determined, and the screen center region is rendered using the target level mesh data; otherwise, the screen center region is rendered using the attribute information of the mesh annotations corresponding to the screen center region and the corresponding sampled texture or color texture; and if the difference between the pixels of the rendered image in this frame and the pixels of the rendered image in the previous frame calculated by global feature vector or hash comparison is less than a fifth threshold, the screen periphery region of the current frame is rendered using the screen periphery region image of the previous frame; or if the difference is greater than or equal to the fifth threshold, the screen periphery region is rendered using the attribute information of the mesh annotations corresponding to the screen periphery region and the corresponding sampled texture or color texture to obtain a 3D rendering result.

12. A three-dimensional scene rendering device, characterized in that, include: The acquisition module is used to acquire the 3D data of the 3D scene to be rendered; The first processing module is used to determine the status information of each available computing power node; wherein the status information includes one or more of the following: task scheduling delay, remaining computing power of the node, resource utilization rate, and policy intervention value. The second processing module is used to input the state information of each computing node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes target computing nodes for 3D scene rendering. The rendering module is used to render the three-dimensional data through the target computing node to obtain a three-dimensional rendering result.

13. A three-dimensional scene rendering system, characterized in that, include: The system includes a scheduling strategy generation device and multiple computing power nodes, with each computing power node communicating with the scheduling strategy generation device. The scheduling strategy generation device is used to acquire 3D data of the 3D scene to be rendered; for each available computing power node, determine the status information of the computing power node; wherein the status information includes one or more of the following: task scheduling delay, node remaining computing power, resource utilization rate, and strategy intervention value; and input the status information of each computing power node into the scheduling strategy generation model to obtain a scheduling strategy, wherein the scheduling strategy includes the target computing power node for 3D scene rendering. The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

14. A three-dimensional scene rendering device, characterized in that, It includes a processor and a transceiver, the transceiver receiving and transmitting data under the control of the processor, characterized in that the processor is configured to perform the following operations: Obtain the 3D data of the 3D scene to be rendered; For each available computing power node, determine the status information of the computing power node; wherein, the status information includes one or more of the following: task scheduling latency, remaining computing power of the node, resource utilization rate, and policy intervention value; The state information of each computing node is input into the scheduling strategy generation model to obtain a scheduling strategy, which includes target computing nodes for 3D scene rendering. The target computing node is used to render the three-dimensional data to obtain a three-dimensional rendering result.

15. A three-dimensional scene rendering device, comprising a memory, a processor, and a computer program stored in the memory and running thereon, characterized in that, When the processor executes the program, it implements the three-dimensional scene rendering method as described in any one of claims 1 to 11.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the three-dimensional scene rendering method as described in any one of claims 1 to 11.

17. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps in the three-dimensional scene rendering method as described in any one of claims 1 to 11.

Images