A collaborative processing system and method for cross-platform 3D rendering task allocation

Abstract

The present invention discloses a collaborative processing system and method for cross-platform 3D rendering task allocation. In the system, a cloud rendering task scheduling module constructs a rendering time prediction model for rendering tasks based on triangle throughput calibration and video memory bandwidth calibration, predicts the rendering time during local rendering, and schedules cloud rendering tasks based on a frame-level preemption formula. The present invention adopts a dynamic unloading strategy, which can achieve load balancing between the rendering tasks of the mobile device itself and the rendering tasks of the edge node, which not only reduces the load pressure of the local rendering tasks, but also saves the computing power of the edge node to a certain extent; the present invention adopts a three-order pipeline + frame-level preemptive scheduling method to effectively reduce the delay between the end and the end; and in the case of network jitter, it realizes dynamic screening of cloud rendering tasks, mainly focusing on local rendering, and reducing the impact of delay fluctuations.

Description

Technical Field

[0001] The present invention relates to the technical field of rendering task allocation and collaborative processing, and in particular to a system and method for cross-platform three-dimensional rendering task allocation and collaborative processing. Background Art

[0002] 3D rendering technology has been widely used in games, virtual reality (VR), augmented reality (AR), film and television special effects, architectural design visualization, industrial simulation, digital twins, and other fields. These applications place increasingly stringent demands on rendering real-time performance, image quality, and complexity. Achieving high-quality, high-frame-rate 3D rendering, especially when complex lighting (such as global illumination and ray tracing), detailed materials, and large scenes are involved, requires enormous computing resources (primarily GPU resources). However, the computing power, memory capacity, and bandwidth of a single GPU device are limited. When the rendering task exceeds the processing capacity of a single device, it results in a drop in frame rate, increased latency, or reduced image quality, severely impacting the user experience.

[0003] Current cross-platform 3D rendering systems are primarily impacted by energy consumption and latency. Mobile devices are limited by GPU computing power and battery capacity, making high-load rendering prone to overheating and frequency throttling. Traditional cloud rendering solutions, however, struggle to meet real-time interactive demands (e.g., VR / AR requirements of ≤20ms) due to network latency. Existing technologies employ static task allocation that cannot adapt to dynamic network conditions and lack quantitative awareness of device energy consumption (e.g., battery level and temperature), resulting in poor energy efficiency. Furthermore, task scheduling and data synchronization management during multi-device collaborative rendering can further exacerbate latency and waste resources. Consequently, existing technologies suffer from significant drawbacks. Summary of the Invention

[0004] The object of the present invention is to provide a system and method for cross-platform 3D rendering task allocation and collaborative processing to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solution: a method for cross-platform 3D rendering task allocation and collaborative processing, the method comprising:

[0006] S1. Collect the current mobile terminal GPU utilization and the amount of data to be transmitted, and calculate the energy consumption coefficient of the mobile device at the current time;

[0007] S2. Obtain the task type corresponding to the current pending rendering task and the current network delay time, and construct a rendering task offloading trigger condition based on the current energy consumption coefficient of the mobile device. Then, lock the rendering task to be offloaded to the edge node; the edge node performs the rendering operation on the corresponding offloaded pending rendering task.

[0008] S3: Based on triangle throughput calibration and memory bandwidth calibration, a rendering time prediction model is built to predict local rendering time and schedule cloud rendering tasks based on a frame-level preemption formula.

[0009] S4. Perform cross-platform collaborative rendering according to a preset three-stage pipeline, and feed back the cross-platform collaborative rendering results to the mobile terminal. The preset three-stage pipeline includes local preprocessing, edge parallel rendering, cloud-based accelerated rendering, and local synthetic display.

[0010] The present invention performs cross-platform collaborative rendering by constructing an energy consumption coefficient model and a three-order pipeline, jointly models energy consumption and network latency, and achieves the best matching of cross-platform resources.

[0011] Furthermore, S1 records the utilization rate of the mobile terminal GPU at the current time as Ug, where Ug is the ratio of the number of active GPU clocks to the total number of GPU clocks in the mobile terminal GPU cycle sampled by the mobile terminal system API;

[0012] In the process of collecting the amount of data to be transmitted by the mobile GPU at the current time, the most recent N frames are obtained, and the amount of texture data corresponding to each frame is counted through graphics API calls. A function of the change of the amount of texture data in the frame as the frame sequence is fitted through a linear regression equation, which is recorded as a texture quantity fitting function. The independent variable value mapped by the Nth frame in the texture quantity fitting function during the fitting process is recorded as N; N+1 is substituted into the texture data amount as a new independent variable to obtain the amount of texture data in the current frame; the average value of the difference between the amount of texture data in each frame in the most recent N frames and the corresponding function value of the texture quantity fitting function of the corresponding frame is obtained, which is recorded as the texture data amount prediction deviation; the sum of the texture data amount prediction deviation and the texture data amount in the current frame is used as the amount of data to be transmitted by the mobile GPU at the current time;

[0013] The calculation formula for calculating the energy consumption coefficient of the mobile device at the current time is as follows: ,

[0014] Wherein, Ec represents the energy consumption coefficient of the mobile device at the current time; Dn represents the amount of data to be transmitted by the mobile GPU at the current time; r1 and r2 represent the preset first weight coefficient and second weight coefficient respectively.

[0015] Furthermore, the rendering task offloading triggering condition constructed in S2 includes a first offloading triggering condition, a second offloading triggering condition and a third offloading triggering condition.

[0016] The first unloading triggering condition is that the energy consumption coefficient of the mobile device at the current time is greater than a dynamic threshold value R of the energy consumption coefficient, and the dynamic threshold value R of the energy consumption coefficient changes dynamically with the operating temperature of the mobile terminal GPU. When the operating temperature of the mobile terminal GPU is greater than or equal to a preset value, the dynamic threshold value R of the energy consumption coefficient is determined to be equal to the first threshold; otherwise, the dynamic threshold value R of the energy consumption coefficient is determined to be equal to the second threshold;

[0017] The second uninstall triggering condition is that the current network delay time is less than the average network delay time of the corresponding mobile terminal monitored in historical data;

[0018] The third unloading triggering condition is that the task type corresponding to the to-be-rendered task belongs to a pre-installed unloadable form in the database, and the pre-installed unloadable form includes shadow generation, particle simulation and post-processing;

[0019] All tasks to be rendered that simultaneously meet the first uninstall trigger condition, the second uninstall trigger condition, and the third uninstall trigger condition are regarded as locked tasks to be rendered.

[0020] In the process of constructing the trigger condition for rendering task unloading, the present invention introduces the concept of "dynamic threshold R of energy consumption coefficient" to realize dynamic regulation of the unloading trigger condition. Under different working temperatures, different dynamic thresholds of energy consumption coefficient are called to realize the judgment of the task unloading trigger behavior corresponding to the energy consumption coefficient of the mobile device at the current time; the second threshold in the present invention is smaller than the first threshold, and its purpose is to automatically tighten the rendering task unloading condition in a high temperature environment, reduce the workload of the mobile device, and reduce the working temperature of the mobile device.

[0021] Furthermore, the task object for executing local rendering in S3 is a collection of all tasks to be rendered that do not simultaneously meet the first unloading trigger condition, the second unloading trigger condition, and the third unloading trigger condition;

[0022] The method for constructing a rendering time prediction model for a rendering task in S3 includes the following steps:

[0023] S31. Obtain benchmark test results when the device is first run, and obtain linear fitting functions for frame times corresponding to rendering models of different complexity. Each point in the obtained linear fitting function corresponds to a frame model of a certain complexity. Frame models of different complexity include different numbers of triangles. The first element of the midpoint of the linear fitting function is the number of triangles included in the frame model of the corresponding complexity. The second element of the midpoint of the linear fitting function is the measured frame time corresponding to the frame model of the corresponding complexity. The slope of the obtained linear fitting function is recorded as β.

[0024] S32. Obtain theoretical bandwidth from a graphics API, measure loading times for textures of different resolutions, and calculate the average of the quotients between the measured loading times and the theoretical loading times for textures of different resolutions as the effective bandwidth utilization coefficient δ. The theoretical loading times for textures of different resolutions are the quotients of the amount of texture data corresponding to the textures of the corresponding resolutions and the theoretical bandwidth.

[0025] S33: Construct a rendering time prediction model for the rendering task. The rendering time prediction model for the rendering task is as follows: ,

[0026] Among them, t （ren，i） Indicates the predicted rendering time corresponding to the i-th task to be rendered in the task object performing local rendering; B i C represents the quotient of the number of triangles in the i-th task to be rendered in the task object performing local rendering and the triangle throughput rate of the mobile GPU; i Indicates the quotient of the texture resolution and video memory bandwidth corresponding to the i-th task to be rendered in the task object for local rendering performed by the mobile GPU;

[0027] S34, based on the constructed rendering time prediction model, predict the sum of the rendering times corresponding to the tasks to be rendered in the task object for executing local rendering, which is recorded as t ren .

[0028] Furthermore, the frame-level preemption formula in S3 is as follows:

[0029] When t ren >t max -t rt When -g is used, it is determined that the cloud acceleration task should be submitted k frames in advance, and the ray tracing request for the corresponding frame is submitted to the cloud. The local post-processing of the current frame and the rasterization of the next frame of the current frame are processed in parallel;

[0030] Among them, t rt Indicates the current network delay time; t max Indicates the maximum network delay time of the corresponding mobile terminal monitored in historical data; g indicates the preset safety margin time; Indicates t ren / t max Round up;

[0031] The latest start time corresponding to the cloud acceleration task is recorded as Ts. The calculation formula involved is as follows: ,

[0032] Among them, T diaplay Indicates the display deadline of the corresponding frame; tc ren Indicates the estimated rendering time of the corresponding frame of the corresponding rendering task in the cloud;

[0033] The method for obtaining the estimated rendering time of the frame corresponding to the corresponding task to be rendered on the cloud in the present invention is the same as the method for constructing a rendering time prediction model for the rendering task in step S3 for prediction; that is: obtaining the slope β of the linear fitting function of the frame time corresponding to the cloud device rendering models of different complexity, obtaining the theoretical bandwidth and effective bandwidth utilization coefficient δ corresponding to the graphics API in the cloud device, and constructing a rendering time prediction model for the rendering task based on the cloud device, thereby realizing the estimated rendering time prediction of the frame corresponding to the corresponding task to be rendered on the cloud.

[0034] When t ren ≤t max -t rt When -g is used, acceleration tasks will not be submitted to the cloud, and corresponding rendering tasks will be scheduled normally.

[0035] The present invention calculates the latest start time of the cloud acceleration task in order to ensure that long and time-consuming tasks such as ray tracing must be completed before the display deadline, thereby reducing the risk of frame discarding.

[0036] Furthermore, in the process of performing cross-platform collaborative rendering according to the preset three-stage pipeline in S4, the execution results of the rendering tasks belonging to the same frame in the corresponding edge node, local and cloud of the mobile GPU are summarized, and the final frame is synthesized and fed back to the mobile terminal for display.

[0037] A collaborative processing system for allocating three-dimensional rendering tasks across platforms, the system comprising:

[0038] An energy consumption coefficient analysis module, which collects the current mobile terminal GPU utilization rate and the amount of data to be transmitted, and calculates the energy consumption coefficient of the mobile device at the current time;

[0039] A task offloading analysis module, which obtains the task type corresponding to the current rendering task and the current network delay time, and combines the energy consumption coefficient of the mobile device at the current time to construct a rendering task offloading trigger condition, and locks the rendering task to be offloaded to the edge node;

[0040] A cloud rendering task scheduling module, which builds a rendering time prediction model for rendering tasks based on triangle throughput calibration and video memory bandwidth calibration, predicts the rendering time during local rendering, and schedules cloud rendering tasks based on a frame-level preemption formula;

[0041] A rendering task collaborative management module performs cross-platform collaborative rendering according to a preset three-stage pipeline, and feeds back the cross-platform collaborative rendering results to the mobile terminal. The preset three-stage pipeline includes local preprocessing, edge parallel rendering, cloud-based accelerated rendering, and local synthetic display.

[0042] Furthermore, the task offloading analysis module includes a task offloading trigger condition construction unit and an offloading task locking unit.

[0043] The task offloading trigger condition construction unit obtains the task type corresponding to the task to be rendered at the current time and the network delay time at the current time, and constructs the rendering task offloading trigger condition in combination with the energy consumption coefficient of the mobile device at the current time;

[0044] The offload task locking unit locks the rendering task to be offloaded to the edge node based on the constructed rendering task offload triggering condition.

[0045] Furthermore, the cloud rendering task scheduling module includes a rendering time prediction model building unit and a cloud rendering task scheduling unit.

[0046] The rendering time prediction model construction unit constructs a rendering time prediction model for the rendering task based on triangle throughput calibration and video memory bandwidth calibration, and predicts the rendering time during local rendering;

[0047] The cloud rendering task scheduling unit schedules the cloud rendering task based on the acquisition result of the rendering time prediction model building unit and the frame-level preemption formula.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) The present invention adopts a dynamic offloading strategy to achieve load balancing between the rendering tasks of the mobile device itself and the rendering tasks of the edge node, which not only reduces the load pressure of the local rendering tasks, but also saves the computing power of the edge node to a certain extent;

[0050] (2) The present invention adopts a three-stage pipeline + frame-level preemptive scheduling method to effectively reduce the delay between ends; and in the case of network jitter, it realizes dynamic screening of cloud rendering tasks, mainly focusing on local rendering, and reduces the impact of delay fluctuations. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 This is a schematic structural diagram of a cross-platform collaborative processing system for allocating 3D rendering tasks according to the present invention;

[0053] Figure 2 It is a flow chart of a collaborative processing method for allocating three-dimensional rendering tasks across platforms according to the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] See also Figure 1-Figure 2 , the present invention provides a technical solution: Figure 1 As shown, this embodiment provides a cross-platform 3D rendering task allocation and collaborative processing system, the system comprising:

[0056] An energy consumption coefficient analysis module, which collects the current mobile terminal GPU utilization rate and the amount of data to be transmitted, and calculates the energy consumption coefficient of the mobile device at the current time;

[0057] A task offloading analysis module, the task offloading analysis module includes a task offloading trigger condition construction unit and an offloading task locking unit,

[0058] The task offloading trigger condition construction unit obtains the task type corresponding to the task to be rendered at the current time and the network delay time at the current time, and constructs the rendering task offloading trigger condition in combination with the energy consumption coefficient of the mobile device at the current time;

[0059] The offloading task locking unit locks the rendering task to be offloaded to the edge node based on the constructed rendering task offloading trigger condition. The task offloading analysis module obtains the task type corresponding to the rendering task to be rendered at the current time and the network delay time at the current time, and constructs the rendering task offloading trigger condition in combination with the energy consumption coefficient of the mobile device at the current time, and locks the rendering task to be offloaded to the edge node;

[0060] A cloud rendering task scheduling module includes a rendering time prediction model building unit and a cloud rendering task scheduling unit.

[0061] The rendering time prediction model construction unit constructs a rendering time prediction model for the rendering task based on triangle throughput calibration and video memory bandwidth calibration, and predicts the rendering time during local rendering;

[0062] The cloud rendering task scheduling unit schedules the cloud rendering task based on the acquisition result of the rendering time prediction model building unit and the frame-level preemption formula;

[0063] A rendering task collaborative management module performs cross-platform collaborative rendering according to a preset three-stage pipeline, and feeds back the cross-platform collaborative rendering results to the mobile terminal. The preset three-stage pipeline includes local preprocessing, edge parallel rendering, cloud-based accelerated rendering, and local synthetic display.

[0064] like Figure 2 As shown, this embodiment provides a method for cross-platform 3D rendering task allocation and collaborative processing, the method comprising:

[0065] S1. Collect the current mobile terminal GPU utilization and the amount of data to be transmitted, and calculate the energy consumption coefficient of the mobile device at the current time;

[0066] S1 records the utilization rate of the mobile terminal GPU at the current time as Ug, where Ug is the ratio of the number of active GPU clocks to the total number of GPU clocks in the mobile terminal GPU cycle sampled through the mobile terminal system API (such as VK_EXT_device_fault / MTLDevice.activity in Android / iOS);

[0067] In the process of collecting the amount of data to be transmitted by the mobile GPU at the current time, the latest N frames are obtained, and the amount of texture data corresponding to each frame is counted through graphics API calls (such as the number of bytes of vkCmdCopyBuffer or glTexImage2D). The function of the change of the texture data amount in the frame as the frame sequence is fitted by the linear regression equation, which is recorded as the texture quantity fitting function. The independent variable value mapped by the Nth frame in the texture quantity fitting function during the fitting process is recorded as N; N+1 is substituted as the new independent variable into the texture data amount to obtain the texture data amount in the current frame; the texture data amount in the latest N frames is obtained. The average value of the difference between the amount of texture data in each frame and the corresponding function value of the texture quantity fitting function of the corresponding frame is recorded as the texture data amount prediction deviation; the sum of the texture data amount prediction deviation and the amount of texture data in the current frame is used as the amount of data to be transmitted by the mobile GPU at the current time; each frame corresponds to multiple tasks to be rendered; a single frame can be regarded as a main task container, but it contains multiple parallel subtasks, including various types of subtasks, such as geometric preprocessing (view frustum culling), block rasterization (dividing the screen into multiple area blocks that can be assigned to different edge nodes) and global illumination calculations (such as ray tracing shadows / reflections);

[0068] The calculation formula for calculating the energy consumption coefficient of the mobile device at the current time is as follows: ,

[0069] Wherein, Ec represents the energy consumption coefficient of the mobile device at the current time; Dn represents the amount of data to be transmitted by the mobile GPU at the current time; r1 and r2 represent the preset first weight coefficient and second weight coefficient respectively.

[0070] S2. Obtain the task type corresponding to the current pending rendering task and the current network delay time, and construct a rendering task offloading trigger condition based on the current energy consumption coefficient of the mobile device. Then, lock the rendering task to be offloaded to the edge node; the edge node performs the rendering operation on the corresponding offloaded pending rendering task.

[0071] The rendering task offloading triggering condition constructed in S2 includes a first offloading triggering condition, a second offloading triggering condition and a third offloading triggering condition.

[0072] The first unloading triggering condition is that the energy consumption coefficient of the mobile device at the current time is greater than the energy consumption coefficient dynamic threshold R, and the energy consumption coefficient dynamic threshold R changes dynamically with the operating temperature of the mobile terminal GPU. When the operating temperature of the mobile terminal GPU is greater than or equal to the preset value, the energy consumption coefficient dynamic threshold R is determined to be equal to the first threshold; otherwise, the energy consumption coefficient dynamic threshold R is determined to be equal to the second threshold. In this embodiment, the first unloading triggering condition is that when the operating temperature of the mobile terminal GPU is greater than or equal to the preset value of the mobile terminal GPU operating temperature of 40°, the energy consumption coefficient dynamic threshold R is determined to be equal to the first threshold 0.91; otherwise, the energy consumption coefficient dynamic threshold R is determined to be equal to the second threshold 0.75.

[0073] The second uninstall triggering condition is that the current network delay time is less than the average network delay time of the corresponding mobile terminal monitored in historical data;

[0074] The third unloading triggering condition is that the task type corresponding to the to-be-rendered task belongs to a pre-installed unloadable form in the database, and the pre-installed unloadable form includes shadow generation, particle simulation and post-processing;

[0075] All tasks to be rendered that simultaneously meet the first uninstall trigger condition, the second uninstall trigger condition, and the third uninstall trigger condition are regarded as locked tasks to be rendered.

[0076] S3: Based on triangle throughput calibration and memory bandwidth calibration, a rendering time prediction model is built to predict local rendering time and schedule cloud rendering tasks based on a frame-level preemption formula.

[0077] The task object for executing local rendering in S3 is a collection of all tasks to be rendered that do not simultaneously meet the first uninstall trigger condition, the second uninstall trigger condition, and the third uninstall trigger condition;

[0078] The method for constructing a rendering time prediction model for a rendering task in S3 includes the following steps:

[0079] S31. Obtain benchmark test results when the device is first run, and obtain linear fitting functions for frame times corresponding to rendering models of different complexity. Each point in the obtained linear fitting function corresponds to a frame model of a certain complexity. Frame models of different complexity include different numbers of triangles. The first element of the midpoint of the linear fitting function is the number of triangles included in the frame model of the corresponding complexity. The second element of the midpoint of the linear fitting function is the measured frame time corresponding to the frame model of the corresponding complexity. The slope of the obtained linear fitting function is recorded as β.

[0080] S32. Obtain theoretical bandwidth from a graphics API, measure loading times for textures of different resolutions, and calculate the average of the quotients between the measured loading times and the theoretical loading times for textures of different resolutions as the effective bandwidth utilization coefficient δ. The theoretical loading times for textures of different resolutions are the quotients of the amount of texture data corresponding to the textures of the corresponding resolutions and the theoretical bandwidth.

[0081] S33: Construct a rendering time prediction model for the rendering task. The rendering time prediction model for the rendering task is as follows: ,

[0082] Among them, t （ren，i） Indicates the predicted rendering time corresponding to the i-th task to be rendered in the task object performing local rendering; B i C represents the quotient of the number of triangles in the i-th task to be rendered in the task object performing local rendering and the triangle throughput rate of the mobile GPU; i Indicates the quotient of the texture resolution and video memory bandwidth corresponding to the i-th task to be rendered in the task object for local rendering performed by the mobile GPU;

[0083] S34, based on the constructed rendering time prediction model, predict the sum of the rendering times corresponding to the tasks to be rendered in the task object for executing local rendering, which is recorded as t ren .

[0084] The frame-level preemption formula in S3 is as follows:

[0085] When t ren >t max -t rt When -g is used, it is determined that the cloud acceleration task should be submitted k frames in advance, and the ray tracing request for the corresponding frame is submitted to the cloud. The local post-processing of the current frame and the rasterization of the next frame of the current frame are processed in parallel;

[0086] Among them, t rt Indicates the current network delay time; t max Indicates the maximum network delay time of the corresponding mobile terminal monitored in historical data; g indicates the preset safety margin time; Indicates tren / t max Round up;

[0087] The latest start time corresponding to the cloud acceleration task is recorded as Ts. The calculation formula involved is as follows: ,

[0088] Among them, T diaplay Indicates the display deadline of the corresponding frame; tc ren Indicates the estimated rendering time of the frame corresponding to the corresponding task to be rendered by the cloud; in this embodiment, the method for obtaining the estimated rendering time of the frame corresponding to the corresponding task to be rendered by the cloud is the same as the method for constructing the rendering time prediction model for the rendering task in step S3 for prediction;

[0089] When t ren ≤t max -t rt When -g is used, acceleration tasks will not be submitted to the cloud, and corresponding rendering tasks will be scheduled normally.

[0090] S4. Perform cross-platform collaborative rendering according to a preset three-stage pipeline, and feed back the cross-platform collaborative rendering results to the mobile terminal. The preset three-stage pipeline includes local preprocessing, edge parallel rendering, cloud-based accelerated rendering, and local synthetic display.

[0091] In the process of performing cross-platform collaborative rendering according to the preset three-stage pipeline in S4, the execution results of the rendering tasks belonging to the same frame in the corresponding edge node, local and cloud of the mobile GPU are summarized, and the final frame is synthesized and fed back to the mobile terminal for display.

[0092] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0093] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A collaborative processing method for allocating 3D rendering tasks across platforms, characterized in that: The method comprises: S1. Collect the current mobile terminal GPU utilization and the amount of data to be transmitted, and calculate the energy consumption coefficient of the mobile device at the current time; S2. Obtain the task type corresponding to the current rendering task and the current network delay time, and build a rendering task offloading trigger condition based on the current energy consumption coefficient of the mobile device, and lock the rendering task to be offloaded to the edge node; S3: Based on triangle throughput calibration and memory bandwidth calibration, a rendering time prediction model is built to predict local rendering time and schedule cloud rendering tasks based on a frame-level preemption formula. The method for constructing a rendering time prediction model for a rendering task in S3 includes the following steps: S31. Obtain benchmark test results when the device is first run, and obtain linear fitting functions for frame times corresponding to rendering models of different complexity. Each point in the obtained linear fitting function corresponds to a frame model of a certain complexity. Frame models of different complexity include different numbers of triangles. The first element of the midpoint of the linear fitting function is the number of triangles included in the frame model of the corresponding complexity. The second element of the midpoint of the linear fitting function is the measured frame time corresponding to the frame model of the corresponding complexity. The slope of the obtained linear fitting function is recorded as β. S32. Obtain theoretical bandwidth from a graphics API, measure loading times for textures of different resolutions, and calculate the average of the quotients between the measured loading times and the theoretical loading times for textures of different resolutions as the effective bandwidth utilization coefficient δ. The theoretical loading times for textures of different resolutions are the quotients of the amount of texture data corresponding to the textures of the corresponding resolutions and the theoretical bandwidth. S33: Construct a rendering time prediction model for the rendering task. The rendering time prediction model for the rendering task is as follows: , Among them, t （ren，i） Indicates the predicted rendering time corresponding to the i-th task to be rendered in the task object performing local rendering; B i C represents the quotient of the number of triangles in the i-th task to be rendered in the task object performing local rendering and the triangle throughput rate of the mobile GPU; i Indicates the quotient of the texture resolution and video memory bandwidth corresponding to the i-th task to be rendered in the task object for local rendering performed by the mobile GPU; S34, based on the constructed rendering time prediction model, predict the sum of the rendering times corresponding to the tasks to be rendered in the task object for executing local rendering, which is recorded as t ren ; The frame-level preemption formula in S3 is as follows: When t ren >t max -t rt When -g is used, the cloud acceleration task is submitted k frames in advance, and the ray tracing request for the corresponding frame is submitted to the cloud. The local post-processing of the current frame and the rasterization of the next frame of the current frame are processed in parallel. Among them, t rt Indicates the current network delay time; t max Indicates the maximum network delay time of the corresponding mobile terminal monitored in historical data; g indicates the preset safety margin time; Indicates t ren / t max Round up; The latest start time corresponding to the cloud acceleration task is recorded as Ts. The calculation formula involved is as follows: , Among them, T diaplay Indicates the display deadline of the corresponding frame; tc ren Indicates the estimated rendering time of the corresponding frame of the corresponding rendering task in the cloud; When t ren ≤t max -t rt When -g is used, the acceleration task will not be submitted to the cloud, and the corresponding rendering task will be scheduled normally; S4. Perform cross-platform collaborative rendering according to a preset three-stage pipeline, and feed back the cross-platform collaborative rendering results to the mobile terminal. The preset three-stage pipeline includes local preprocessing, edge parallel rendering, cloud-based accelerated rendering, and local synthetic display.

2. The method for cross-platform 3D rendering task allocation and collaborative processing according to claim 1, characterized in that: S1 records the utilization rate of the mobile terminal GPU at the current time as Ug, where Ug is the ratio of the number of active GPU clocks to the total number of GPU clocks in the mobile terminal GPU cycle sampled by the mobile terminal system API; In the process of collecting the amount of data to be transmitted by the mobile terminal GPU at the current time, the latest N frames are obtained, and the amount of texture data corresponding to each frame is counted through graphics API calls. The function of the amount of texture data in the frame that changes with the frame sequence is fitted through a linear regression equation, which is recorded as a texture quantity fitting function. The independent variable value mapped by the Nth frame in the texture quantity fitting function during the fitting process is recorded as N; N+1 is substituted into the texture data amount as a new independent variable to obtain the amount of texture data in the current frame; the average value of the difference between the amount of texture data in each frame in the latest N frames and the corresponding function value of the texture quantity fitting function of the corresponding frame is obtained, which is recorded as the texture data amount prediction deviation; the sum of the texture data amount prediction deviation and the texture data amount in the current frame is taken as the amount of data to be transmitted by the mobile terminal GPU at the current time; The calculation formula for calculating the energy consumption coefficient of the mobile device at the current time is as follows: , Wherein, Ec represents the energy consumption coefficient of the mobile device at the current time; Dn represents the amount of data to be transmitted by the mobile GPU at the current time; r1 and r2 represent the preset first weight coefficient and second weight coefficient respectively.

3. The method for cross-platform 3D rendering task allocation and collaborative processing according to claim 1, characterized in that: The rendering task offloading triggering condition constructed in S2 includes a first offloading triggering condition, a second offloading triggering condition and a third offloading triggering condition. The first unloading triggering condition is that the energy consumption coefficient of the mobile device at the current time is greater than a dynamic threshold value R of the energy consumption coefficient, and the dynamic threshold value R of the energy consumption coefficient changes dynamically with the operating temperature of the mobile terminal GPU. When the operating temperature of the mobile terminal GPU is greater than or equal to a preset value, the dynamic threshold value R of the energy consumption coefficient is determined to be equal to the first threshold; otherwise, the dynamic threshold value R of the energy consumption coefficient is determined to be equal to the second threshold; The second uninstall triggering condition is that the current network delay time is less than the average network delay time of the corresponding mobile terminal monitored in historical data; The third unloading triggering condition is that the task type corresponding to the to-be-rendered task belongs to a pre-installed unloadable form in the database, and the pre-installed unloadable form includes shadow generation, particle simulation and post-processing; All tasks to be rendered that simultaneously meet the first uninstall trigger condition, the second uninstall trigger condition, and the third uninstall trigger condition are regarded as locked tasks to be rendered.

4. The method for cross-platform 3D rendering task allocation and collaborative processing according to claim 3, characterized in that: The task object for executing local rendering in S3 is a collection of all tasks to be rendered that do not simultaneously meet the first uninstall trigger condition, the second uninstall trigger condition, and the third uninstall trigger condition.

5. The method for cross-platform 3D rendering task allocation and collaborative processing according to claim 1, characterized in that: In the process of performing cross-platform collaborative rendering according to the preset three-stage pipeline in S4, the execution results of the rendering tasks belonging to the same frame in the corresponding edge node, local and cloud of the mobile GPU are summarized, and the final frame is synthesized and fed back to the mobile terminal for display.

6. A system for cross-platform 3D rendering task allocation and collaborative processing, applying the method for cross-platform 3D rendering task allocation and collaborative processing according to any one of claims 1 to 5, characterized in that: The system comprises: An energy consumption coefficient analysis module, which collects the current mobile terminal GPU utilization rate and the amount of data to be transmitted, and calculates the energy consumption coefficient of the mobile device at the current time; A task offloading analysis module, which obtains the task type corresponding to the current rendering task and the current network delay time, and combines the energy consumption coefficient of the mobile device at the current time to construct a rendering task offloading trigger condition, and locks the rendering task to be offloaded to the edge node; A cloud rendering task scheduling module, which builds a rendering time prediction model for rendering tasks based on triangle throughput calibration and video memory bandwidth calibration, predicts the rendering time during local rendering, and schedules cloud rendering tasks based on a frame-level preemption formula; A rendering task collaborative management module performs cross-platform collaborative rendering according to a preset three-stage pipeline, and feeds back the cross-platform collaborative rendering results to the mobile terminal. The preset three-stage pipeline includes local preprocessing, edge parallel rendering, cloud-based accelerated rendering, and local synthetic display.

7. The cross-platform 3D rendering task allocation and collaborative processing system according to claim 6, characterized in that: The task offloading analysis module includes a task offloading trigger condition construction unit and an offloading task locking unit. The task offloading trigger condition construction unit obtains the task type corresponding to the task to be rendered at the current time and the network delay time at the current time, and constructs the rendering task offloading trigger condition in combination with the energy consumption coefficient of the mobile device at the current time; The offload task locking unit locks the rendering task to be offloaded to the edge node based on the constructed rendering task offload triggering condition.

8. The cross-platform 3D rendering task allocation and collaborative processing system according to claim 6, characterized in that: The cloud rendering task scheduling module includes a rendering time prediction model building unit and a cloud rendering task scheduling unit. The rendering time prediction model construction unit constructs a rendering time prediction model for the rendering task based on triangle throughput calibration and video memory bandwidth calibration, and predicts the rendering time during local rendering; The cloud rendering task scheduling unit schedules the cloud rendering task based on the acquisition result of the rendering time prediction model building unit and the frame-level preemption formula.

Images