A distributed real-time rendering method, system, and medium based on edge computing

Abstract

This application provides a distributed real-time rendering method, system, and apparatus based on edge computing. The method includes: assessing the scene complexity of the task to be rendered and matching it with the preset rendering capabilities of edge nodes; if the rendering capabilities of the edge nodes are insufficient, sending the task to be rendered to the cloud for processing; if the rendering capabilities of the edge nodes are sufficient, splitting the task to be rendered based on the rendering scene category feature data to obtain sub-tasks to be rendered, and allocating them to edge nodes for real-time rendering to obtain the sub-task rendering screen; simultaneously, assessing the rendering consistency of each node, performing compositing processing, obtaining the real-time rendered screen, and transmitting it to the user terminal for display; this application achieves real-time rendering based on edge computing by intelligently matching and verifying the rendering capabilities of edge nodes, splitting and allocating the task to be rendered based on the rendering scene, rendering difficulty, and edge node load status, and performing real-time node rendering consistency assessment.

Description

Technical Field

[0001] This application relates to the fields of computer graphics and edge computing technology, and more specifically, to a distributed real-time rendering method, system, and apparatus based on edge computing. Background Technology

[0002] Traditional rendering techniques relying on dedicated local graphics processors or cloud computing centers are limited by hardware performance and transmission latency, easily leading to issues such as stuttering and latency in rendering, failing to meet the demands of applications with high real-time requirements. While research has explored applying edge computing to real-time rendering, the rationality and adaptability of rendering task splitting and allocation remain insufficient, resulting in uneven load distribution across edge computing nodes, inconsistent real-time rendering quality, and a lack of intelligence in real-time rendering evaluation and compositing, making it difficult to guarantee the quality of composite images. Therefore, a highly adaptable distributed real-time rendering method is urgently needed.

[0003] Effective technical solutions are urgently needed to address the above problems. Summary of the Invention

[0004] The purpose of this application is to provide a distributed real-time rendering method, system, and device based on edge computing. It can intelligently match and verify the rendering capabilities of edge nodes, split and allocate rendering tasks according to the rendering scene, rendering difficulty, and edge node load status, and perform real-time node rendering consistency evaluation, thereby realizing real-time rendering based on edge computing.

[0005] This application also provides a distributed real-time rendering method based on edge computing, including the following steps:

[0006] The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted.

[0007] The scene complexity of the task to be rendered is obtained by processing the rendering complexity evaluation data.

[0008] The scene complexity is matched with the preset rendering capabilities of the edge nodes. If the rendering capabilities of the edge nodes are insufficient, the rendering task is sent to the cloud for processing.

[0009] If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data.

[0010] The subtask to be rendered is assigned to the edge node using a preset method to obtain the node rendering subtask, and then rendered in real time to obtain the subtask rendering screen.

[0011] The subtask rendering screen is composited using a preset compositing method to obtain a real-time rendering screen, which is then transmitted to the user terminal for display.

[0012] Optionally, in the edge computing-based distributed real-time rendering method described in this application, the step of obtaining rendering requirement information according to rendering instructions and extracting rendering scene category feature data and rendering complexity evaluation data includes:

[0013] The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted.

[0014] The rendered scene category feature data includes static scene feature data, dynamic scene feature data, or multi-dimensional independent scene feature data;

[0015] The rendering complexity evaluation data includes dynamic object magnitude, geometric feature data, texture feature data, and light source feature data.

[0016] Optionally, in the edge computing-based distributed real-time rendering method described in this application, the step of processing the rendering complexity evaluation data to obtain the scene complexity of the task to be rendered includes:

[0017] Based on the dynamic object magnitude, a preset dynamic object magnitude and weight value mapping table is queried to obtain geometric feature weight values, texture feature weight values, and light source feature weight values.

[0018] The geometric feature data, texture feature data, and light source feature data are combined with the geometric feature weight value, texture feature weight value, and light source feature weight value, and then weighted and summed to obtain the scene complexity of the task to be rendered.

[0019] Optionally, in the distributed real-time rendering method based on edge computing described in this application, if the rendering capability of the edge nodes is sufficient, the task to be rendered is split into sub-tasks based on the rendering scene category feature data, including:

[0020] The task to be rendered is split into sub-tasks to be rendered, including sub-tasks to be rendered block, sub-tasks to be rendered frame, and sub-tasks to be rendered scene.

[0021] If the scene feature data is static, the task to be rendered will be split into spatial tasks to obtain sub-tasks to be rendered.

[0022] If the data is dynamic scene feature data, the task to be rendered will be split into time tasks to obtain the sub-tasks of the frames to be rendered.

[0023] If the data consists of multidimensional independent scene features, the task to be rendered will be split into scene tasks to obtain sub-tasks of the scene to be rendered.

[0024] Optionally, the distributed real-time rendering method based on edge computing described in this application further includes:

[0025] Obtain the corresponding rendering requirement information based on the block subtask to be rendered, the frame subtask to be rendered, or the scene subtask to be rendered.

[0026] The rendering requirements information includes rendering accuracy, lighting requirements data, and screen display requirements data;

[0027] The rendering accuracy, lighting requirements data and screen display requirements data are input into a preset rendering difficulty prediction model for processing to obtain rendering difficulty category labels, including high difficulty or low difficulty.

[0028] The sub-tasks to be rendered (blocks, frames, or scenes) are classified according to their difficulty level (high or low) to obtain sets of sub-tasks to be rendered (blocks, frames, or scenes).

[0029] Optionally, the distributed real-time rendering method based on edge computing described in this application further includes:

[0030] Obtain data on CPU utilization, GPU utilization, memory usage, and network bandwidth usage of edge nodes;

[0031] The CPU utilization, GPU utilization, memory usage, and network bandwidth usage data are weighted and summed to obtain the real-time load data of the edge nodes.

[0032] The real-time load data of the edge nodes is compared with a preset load warning threshold to obtain the real-time load status of the edge nodes;

[0033] If the load is less than or equal to the preset load warning threshold, the real-time load status is determined to be low load.

[0034] If the load exceeds the preset load warning threshold, the real-time load status is determined to be high load.

[0035] Optionally, in the distributed real-time rendering method based on edge computing described in this application, the step of allocating the subtask to be rendered to edge nodes through a preset method, obtaining node rendering subtasks, and performing real-time rendering to obtain the subtask rendering screen includes:

[0036] The task allocation is performed by combining the set of subtasks to be rendered (blocks), the set of subtasks to be rendered (frames), or the set of subtasks to be rendered (scenes) with the real-time load status to obtain node rendering subtasks.

[0037] Based on the node rendering subtasks, a test set is extracted to obtain the rendering test subtasks;

[0038] The edge nodes render the rendering test subtask and obtain the rendering test time data;

[0039] The rendering test time data is processed to obtain node rendering time prediction data.

[0040] The node rendering time prediction data is compared with the preset rendering time limit threshold.

[0041] If the timeout is less than the preset rendering time limit threshold, real-time rendering will be performed to obtain the subtask rendering screen.

[0042] If the value is greater than or equal to the preset rendering time limit threshold, an early warning response will be output.

[0043] Optionally, the distributed real-time rendering method based on edge computing described in this application further includes:

[0044] The rendering test subtask is simultaneously assigned to each edge node for real-time rendering to obtain the rendering test screen corresponding to the edge node.

[0045] The consistency of the rendered test images will be evaluated.

[0046] If the consistency assessment passes, then real-time rendering will proceed.

[0047] If the consistency assessment fails, the corresponding edge node will be sent to the user terminal for display.

[0048] Secondly, this application provides a distributed real-time rendering system based on edge computing. The system includes a memory and a processor. The memory includes a program for a distributed real-time rendering method based on edge computing. When the program for the distributed real-time rendering method based on edge computing is executed by the processor, it implements the following steps:

[0049] The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted.

[0050] The scene complexity of the task to be rendered is obtained by processing the rendering complexity evaluation data.

[0051] The scene complexity is matched with the preset rendering capabilities of the edge nodes. If the rendering capabilities of the edge nodes are insufficient, the rendering task is sent to the cloud for processing.

[0052] If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data.

[0053] The subtask to be rendered is assigned to the edge node using a preset method to obtain the node rendering subtask, and then rendered in real time to obtain the subtask rendering screen.

[0054] The subtask rendering screen is composited using a preset compositing method to obtain a real-time rendering screen, which is then transmitted to the user terminal for display.

[0055] Thirdly, this application also provides a distributed real-time rendering device based on edge computing, comprising:

[0056] The edge node and cloud collaboration module includes edge computing node units and cloud server units, which are used to build a rendering edge computing node and cloud collaborative rendering system;

[0057] The rendering acquisition module is used to collect data and information from rendering tasks in real time.

[0058] The rendering evaluation module is used to evaluate rendering tasks and edge computing nodes;

[0059] The rendering task processing module is used to split and allocate tasks to be rendered;

[0060] The rendering task compositing module is used to compose the rendering images of subtasks.

[0061] As can be seen from the above, the distributed real-time rendering method, system and device based on edge computing provided in this application realizes real-time rendering based on edge computing by intelligently matching and verifying the rendering capabilities of edge nodes, splitting and allocating rendering tasks according to the rendering scene, rendering difficulty and edge node load status, and performing real-time node rendering consistency evaluation.

[0062] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A flowchart illustrating a distributed real-time rendering method based on edge computing, provided as an embodiment of this application;

[0065] Figure 2 A flowchart illustrating the process of obtaining the scene complexity of a task to be rendered using a distributed real-time rendering method based on edge computing, as provided in an embodiment of this application.

[0066] Figure 3 A flowchart illustrating the process of obtaining a subtask to be rendered using a distributed real-time rendering method based on edge computing, as provided in an embodiment of this application.

[0067] Figure 4 A flowchart illustrating the process of obtaining a subtask rendering screen using a distributed real-time rendering method based on edge computing, as provided in an embodiment of this application.

[0068] Figure 5 This is a device diagram of a distributed real-time rendering device based on edge computing, provided for an embodiment of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0070] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0071] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a distributed real-time rendering method based on edge computing, as described in some embodiments of this application. This distributed real-time rendering method based on edge computing is used in terminal devices, such as computers and mobile terminals. The distributed real-time rendering method based on edge computing includes the following steps:

[0072] S11. Obtain rendering requirement information based on rendering instructions, and extract rendering scene category feature data and rendering complexity evaluation data;

[0073] S12. Process the rendering complexity evaluation data to obtain the scene complexity of the task to be rendered;

[0074] S13. Match the scene complexity with the preset rendering capabilities of the edge nodes. If the rendering capabilities of the edge nodes are insufficient, send the rendering task to the cloud for processing.

[0075] S14. If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data to obtain subtasks to be rendered.

[0076] S15. The subtask to be rendered is assigned to the edge node through a preset method to obtain the node rendering subtask, and is rendered in real time to obtain the subtask rendering screen.

[0077] S16. The subtask rendering screen is composited using a preset compositing method to obtain a real-time rendering screen, which is then transmitted to the user terminal for display.

[0078] It should be noted that upon receiving the rendering command, to improve rendering real-time performance, edge computing technology is employed. To verify the rendering capabilities of the edge nodes, the scene complexity of the rendering task is first assessed to determine if the rendering capabilities of the edge nodes are sufficient. If insufficient, the rendering task is sent to the cloud computing center for rendering processing, achieving collaborative processing between the edge nodes and the cloud. If sufficient, the task is split according to the rendering scene category feature data representing the scene type to obtain sub-tasks to be rendered. Then, the rendering difficulty of the obtained sub-tasks to be rendered and the real-time load of the edge nodes are assessed. Based on the assessment, the sub-tasks to be rendered are allocated to each edge node for real-time rendering. At the same time, the consistency and rendering quality of each node are monitored in real time. Finally, the rendered images of the sub-tasks completed by each edge node are composited using a preset compositing method to obtain real-time rendered images, which are then transmitted to the user terminal for display. This achieves the rationality and adaptability of rendering task splitting and allocation.

[0079] According to an embodiment of the present invention, the step of obtaining rendering requirement information according to rendering instructions and extracting rendering scene category feature data and rendering complexity evaluation data includes:

[0080] The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted.

[0081] The rendered scene category feature data includes static scene feature data, dynamic scene feature data, or multi-dimensional independent scene feature data;

[0082] The rendering complexity evaluation data includes dynamic object magnitude, geometric feature data, texture feature data, and light source feature data.

[0083] It should be noted that, in order to improve the rationality of task splitting and allocation and to improve rendering quality, the corresponding rendering complexity is evaluated according to different rendering scenarios. This is used to assess whether the rendering capabilities of edge nodes are matched. Static scene feature data includes factory layout diagrams, dynamic scene feature data includes animation videos, and multi-dimensional independent scene feature data includes multiple player characters in a 3D game. The dynamic object quantity level is defined by those skilled in the art based on the number of dynamic objects in the scene and can be dynamically adjusted. There are 10 levels from 0 to 10, with level 10 having the largest number of dynamic objects. This level is used to query the scene complexity evaluation weight value.

[0084] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the process of obtaining the scene complexity of a task to be rendered using a distributed real-time rendering method based on edge computing, as described in some embodiments of this application. According to embodiments of the present invention, the step of processing the rendering complexity evaluation data to obtain the scene complexity of the task to be rendered includes:

[0085] S21. Based on the dynamic object magnitude, query the preset dynamic object magnitude and weight value mapping table to obtain the geometric feature weight value, texture feature weight value and light source feature weight value.

[0086] S22. The geometric feature data, texture feature data, and light source feature data are combined with the geometric feature weight value, texture feature weight value, and light source feature weight value for weighted summation to obtain the scene complexity of the task to be rendered.

[0087] It should be noted that the weight values ​​corresponding to each evaluation data are obtained by querying the preset dynamic object magnitude and weight value mapping table from the evaluated dynamic object magnitude, and then weighted summation is performed to obtain the scene complexity corresponding to the rendering task. This is used to match the rendering capabilities of edge nodes in the future. Among them, geometric feature data refers to the sum of the number of polygons and the number of surfaces, texture feature data is obtained by those skilled in the art based on different texture types and the corresponding number and resolution of textures, and light source feature data is obtained by those skilled in the art based on the number of different types of light sources.

[0088] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating the process of obtaining subtasks to be rendered in a distributed real-time rendering method based on edge computing, as described in some embodiments of this application. According to an embodiment of the present invention, if the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data, including:

[0089] S31. The task to be rendered is split into sub-tasks to be rendered, including sub-tasks to be rendered block, sub-tasks to be rendered frame, and sub-tasks to be rendered scene.

[0090] S32. If the data is static scene feature data, the task to be rendered will be split into spatial tasks to obtain the sub-tasks to be rendered.

[0091] S33. If it is dynamic scene feature data, then the task to be rendered is split into time tasks to obtain the sub-tasks of the frames to be rendered.

[0092] S34. If the data is multi-dimensional independent scene features, the task to be rendered will be split into scene tasks to obtain sub-tasks of the scene to be rendered.

[0093] It should be noted that, in order to improve the adaptability of task splitting and allocation, static scenes are split into spatial tasks, with each block assigned to an edge node; dynamic scenes are split into temporal tasks, with consecutive video frames assigned to an edge node; and multi-dimensional independent scenes are split into scene tasks, with each independent scene assigned to an edge node. In addition, the subtask of the block to be rendered can be further divided into multiple small blocks, the subtask of the frame to be rendered can be further divided into multiple video frames, and the subtask of the scene to be rendered can be further divided into multiple small scenes, which are used for subsequent rendering time evaluation and rendering consistency evaluation of each node.

[0094] According to an embodiment of the present invention, it further includes:

[0095] Obtain the corresponding rendering requirement information based on the block subtask to be rendered, the frame subtask to be rendered, or the scene subtask to be rendered.

[0096] The rendering requirements information includes rendering accuracy, lighting requirements data, and screen display requirements data;

[0097] The rendering accuracy, lighting requirements data and screen display requirements data are input into a preset rendering difficulty prediction model for processing to obtain rendering difficulty category labels, including high difficulty or low difficulty.

[0098] The sub-tasks to be rendered (blocks, frames, or scenes) are classified according to their difficulty level (high or low) to obtain sets of sub-tasks to be rendered (blocks, frames, or scenes).

[0099] It should be noted that, due to the varying rendering difficulty of the sub-tasks to be rendered, in order to classify and allocate tasks, a preset rendering difficulty prediction model is used to process the corresponding rendering precision, lighting requirement data, and display requirement data to obtain rendering difficulty category labels. This preset rendering difficulty prediction model is trained by acquiring a large amount of historical rendering task rendering precision, lighting requirement data, and display requirement data, along with the corresponding rendering difficulty category labels. Rendering precision is obtained by weighted summation of geometric precision, texture precision, and color precision; lighting requirement data is obtained by weighted summation of static lighting, dynamic lighting, and lighting animation; and display requirement data is obtained by weighted summation of screen resolution and frame rate. Specific weight values ​​are preset by those skilled in the art based on the specific rendering task and can be dynamically adjusted. Then, the sub-tasks to be rendered (blocks, frames, or scenes) are classified according to high or low difficulty, resulting in sets of sub-tasks to be rendered (blocks, frames, and scenes) including high and / or low difficulty.

[0100] According to an embodiment of the present invention, it further includes:

[0101] Obtain data on CPU utilization, GPU utilization, memory usage, and network bandwidth usage of edge nodes;

[0102] The CPU utilization, GPU utilization, memory usage, and network bandwidth usage data are weighted and summed to obtain the real-time load data of the edge nodes.

[0103] The real-time load data of the edge nodes is compared with a preset load warning threshold to obtain the real-time load status of the edge nodes;

[0104] If the load is less than or equal to the preset load warning threshold, the real-time load status is determined to be low load.

[0105] If the load exceeds the preset load warning threshold, the real-time load status is determined to be high load.

[0106] It should be noted that, in order to achieve the adaptation between the subtask to be rendered and the edge node, the real-time load of the edge node is evaluated, the real-time load data of the edge node is obtained, and a threshold comparison is performed to divide the real-time load status into low load or high load. Low load is responsible for rendering high-difficulty tasks, and high load is responsible for rendering low-difficulty tasks.

[0107] Please refer to Figure 4 , Figure 4This is a flowchart illustrating the process of obtaining a subtask rendering screen using a distributed real-time rendering method based on edge computing, as described in some embodiments of this application. According to an embodiment of the present invention, the step of allocating the subtask to be rendered to edge nodes using a preset method, obtaining node-rendered subtasks, and performing real-time rendering to obtain the subtask rendering screen includes:

[0108] S41. Combine the set of subtasks to be rendered, the set of subtasks to be rendered, or the set of subtasks to be rendered with the real-time load status to allocate tasks and obtain node rendering subtasks.

[0109] S42. Extract the test set based on the node rendering subtask to obtain the rendering test subtask;

[0110] S43. The edge node renders the rendering test subtask and obtains the rendering test time data.

[0111] S44. Process the rendering test time data to obtain node rendering time prediction data.

[0112] S45. Compare the node rendering time prediction data with the preset rendering time limit threshold.

[0113] S46. If the time limit is less than the preset rendering time limit threshold, then perform real-time rendering to obtain the subtask rendering screen.

[0114] S47. If the value is greater than or equal to the preset rendering time limit threshold, an early warning response will be output.

[0115] It should be noted that after assigning high-difficulty subtask sets to low-load edge nodes and low-difficulty subtask sets to high-load edge nodes, before real-time rendering, in order to determine whether the rendering time of each node can meet the rendering timeliness, firstly, each edge node extracts rendering test subtasks for rendering processing and counts the rendering test time data. Then, edge node time prediction is performed to obtain node rendering time prediction data. Finally, threshold comparison is used to determine whether the time limit requirement is met.

[0116] According to an embodiment of the present invention, it further includes:

[0117] The rendering test subtask is simultaneously assigned to each edge node for real-time rendering to obtain the rendering test screen corresponding to the edge node.

[0118] The consistency of the rendered test images will be evaluated.

[0119] If the consistency assessment passes, then real-time rendering will proceed.

[0120] If the consistency assessment fails, the corresponding edge node will be sent to the user terminal for display.

[0121] It should be noted that, in order to evaluate the consistency of task rendering across edge nodes and improve the image composition effect, each edge node performs real-time rendering of the same test subtask to obtain the corresponding rendering test image of each edge node. The consistency of the image is then evaluated, and real-time rendering is only allowed if the evaluation is passed, so as to avoid wasting computing resources.

[0122] It is worth mentioning that, according to embodiments of the present invention, it further includes:

[0123] Obtain the synthetic test subtask of the first edge node;

[0124] The synthetic test subtask is assigned to the first edge node and the adjacent second edge node for real-time rendering to obtain the corresponding synthetic test screen;

[0125] The quality of the synthesized test footage will be evaluated.

[0126] If the quality assessment is passed, the subtask rendering images of the first edge node and the adjacent second edge node are composited.

[0127] If the quality assessment fails, the corresponding edge node will be sent to the user's terminal for display.

[0128] It should be noted that since the sub-task rendering images of each edge node need to be composited, in order to avoid unqualified image quality after composited images, after the real-time rendering consistency evaluation of each edge node passes, the rendering quality of adjacent edge nodes that need to be composited is further evaluated before composited images. First, the composited test sub-task of the first edge node is extracted. Then, it is simultaneously assigned to the two adjacent edge nodes that need to be composited for real-time rendering, and the composited test images corresponding to the first edge node and the adjacent second edge node are obtained. The quality is evaluated separately. If both adjacent edge nodes pass, composited images are allowed. Otherwise, the edge nodes that fail the quality evaluation are sent to the user for display.

[0129] It is worth mentioning that, according to an embodiment of the present invention, the consistency evaluation of the rendered test screen includes:

[0130] The rendered test screen is evaluated for spatial consistency, temporal consistency, visual consistency, and interactive consistency to obtain spatial consistency evaluation results, temporal consistency evaluation results, visual consistency evaluation results, and interactive consistency evaluation results.

[0131] Perform an AND operation on the spatial consistency assessment results, temporal consistency assessment results, visual consistency assessment results, and interaction consistency assessment results;

[0132] If the assessment is passed, the consistency assessment is deemed to have passed.

[0133] It should be noted that the evaluation of rendering consistency of each edge node includes spatial consistency, temporal consistency, visual consistency, and interactive consistency evaluation. Spatial consistency is evaluated by the deviation of the geometric position, size, and orientation of an object or scene from the corresponding preset geometric position, preset size, and preset orientation, respectively. Temporal consistency is evaluated by the timing of dynamic scenes to determine whether there are frame skips or excessive latency. Visual consistency is evaluated by the deviation of brightness, contrast, and color from the corresponding preset brightness, preset contrast, and preset color, respectively. Interactive consistency is evaluated by the consistency of the feedback of different edge nodes to state changes. The evaluation results for spatial consistency, temporal consistency, visual consistency, and interactive consistency are respectively categorized as passed or failed.

[0134] This invention also discloses a distributed real-time rendering system based on edge computing, including a memory and a processor. The memory includes a distributed real-time rendering method program based on edge computing. When the processor executes the distributed real-time rendering method program based on edge computing, it performs the following steps:

[0135] The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted.

[0136] The scene complexity of the task to be rendered is obtained by processing the rendering complexity evaluation data.

[0137] The scene complexity is matched with the preset rendering capabilities of the edge nodes. If the rendering capabilities of the edge nodes are insufficient, the rendering task is sent to the cloud for processing.

[0138] If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data.

[0139] The subtask to be rendered is assigned to the edge node using a preset method to obtain the node rendering subtask, and then rendered in real time to obtain the subtask rendering screen.

[0140] The subtask rendering screen is composited using a preset compositing method to obtain a real-time rendering screen, which is then transmitted to the user terminal for display.

[0141] It should be noted that upon receiving the rendering command, to improve rendering real-time performance, edge computing technology is employed. To verify the rendering capabilities of the edge nodes, the scene complexity of the rendering task is first assessed to determine if the rendering capabilities of the edge nodes are sufficient. If insufficient, the rendering task is sent to the cloud computing center for rendering processing, achieving collaborative processing between the edge nodes and the cloud. If sufficient, the task is split according to the rendering scene category feature data representing the scene type to obtain sub-tasks to be rendered. Then, the rendering difficulty of the obtained sub-tasks to be rendered and the real-time load of the edge nodes are assessed. Based on the assessment, the sub-tasks to be rendered are allocated to each edge node for real-time rendering. At the same time, the consistency and rendering quality of each node are monitored in real time. Finally, the rendered images of the sub-tasks completed by each edge node are composited using a preset compositing method to obtain real-time rendered images, which are then transmitted to the user terminal for display. This achieves the rationality and adaptability of rendering task splitting and allocation.

[0142] According to an embodiment of the present invention, the step of obtaining rendering requirement information according to rendering instructions and extracting rendering scene category feature data and rendering complexity evaluation data includes:

[0143] The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted.

[0144] The rendered scene category feature data includes static scene feature data, dynamic scene feature data, or multi-dimensional independent scene feature data;

[0145] The rendering complexity evaluation data includes dynamic object magnitude, geometric feature data, texture feature data, and light source feature data.

[0146] It should be noted that, in order to improve the rationality of task splitting and allocation and to improve rendering quality, the corresponding rendering complexity is evaluated according to different rendering scenarios. This is used to assess whether the rendering capabilities of edge nodes are matched. Static scene feature data includes factory layout diagrams, dynamic scene feature data includes animation videos, and multi-dimensional independent scene feature data includes multiple player characters in a 3D game. The dynamic object quantity level is defined by those skilled in the art based on the number of dynamic objects in the scene and can be dynamically adjusted. There are 10 levels from 0 to 10, with level 10 having the largest number of dynamic objects. This level is used to query the scene complexity evaluation weight value.

[0147] According to an embodiment of the present invention, the step of processing the rendering complexity evaluation data to obtain the scene complexity of the task to be rendered includes:

[0148] Based on the dynamic object magnitude, a preset dynamic object magnitude and weight value mapping table is queried to obtain geometric feature weight values, texture feature weight values, and light source feature weight values.

[0149] The geometric feature data, texture feature data, and light source feature data are combined with the geometric feature weight value, texture feature weight value, and light source feature weight value, and then weighted and summed to obtain the scene complexity of the task to be rendered.

[0150] It should be noted that the weight values ​​corresponding to each evaluation data are obtained by querying the preset dynamic object magnitude and weight value mapping table from the evaluated dynamic object magnitude, and then weighted summation is performed to obtain the scene complexity corresponding to the rendering task. This is used to match the rendering capabilities of edge nodes in the future. Among them, geometric feature data refers to the sum of the number of polygons and the number of surfaces, texture feature data is obtained by those skilled in the art based on different texture types and the corresponding number and resolution of textures, and light source feature data is obtained by those skilled in the art based on the number of different types of light sources.

[0151] According to an embodiment of the present invention, if the rendering capability of the edge nodes is sufficient, the task to be rendered is split into sub-tasks based on the rendering scene category feature data to obtain sub-tasks to be rendered, including:

[0152] The task to be rendered is split into sub-tasks to be rendered, including sub-tasks to be rendered block, sub-tasks to be rendered frame, and sub-tasks to be rendered scene.

[0153] If the scene feature data is static, the task to be rendered will be split into spatial tasks to obtain sub-tasks to be rendered.

[0154] If the data is dynamic scene feature data, the task to be rendered will be split into time tasks to obtain the sub-tasks of the frames to be rendered.

[0155] If the data consists of multidimensional independent scene features, the task to be rendered will be split into scene tasks to obtain sub-tasks of the scene to be rendered.

[0156] It should be noted that, in order to improve the adaptability of task splitting and allocation, static scenes are split into spatial tasks, with each block assigned to an edge node; dynamic scenes are split into temporal tasks, with consecutive video frames assigned to an edge node; and multi-dimensional independent scenes are split into scene tasks, with each independent scene assigned to an edge node. In addition, the subtask of the block to be rendered can be further divided into multiple small blocks, the subtask of the frame to be rendered can be further divided into multiple video frames, and the subtask of the scene to be rendered can be further divided into multiple small scenes, which are used for subsequent rendering time evaluation and rendering consistency evaluation of each node.

[0157] According to an embodiment of the present invention, it further includes:

[0158] Obtain the corresponding rendering requirement information based on the block subtask to be rendered, the frame subtask to be rendered, or the scene subtask to be rendered.

[0159] The rendering requirements information includes rendering accuracy, lighting requirements data, and screen display requirements data;

[0160] The rendering accuracy, lighting requirements data and screen display requirements data are input into a preset rendering difficulty prediction model for processing to obtain rendering difficulty category labels, including high difficulty or low difficulty.

[0161] The sub-tasks to be rendered (blocks, frames, or scenes) are classified according to their difficulty level (high or low) to obtain sets of sub-tasks to be rendered (blocks, frames, or scenes).

[0162] It should be noted that, since the rendering difficulty of the sub-tasks to be rendered varies, in order to classify and allocate tasks, the rendering difficulty is predicted by a preset rendering difficulty prediction model based on the corresponding rendering precision, lighting requirements and display requirements to obtain rendering difficulty category labels.

[0163] The preset rendering difficulty prediction model is trained by acquiring a large amount of historical rendering task rendering accuracy, lighting requirement data, and image display requirement data, as well as corresponding rendering difficulty category labels. The rendering accuracy is obtained by weighted summation of geometric accuracy, texture accuracy, and color accuracy. The lighting requirement data is obtained by weighted summation of static lighting, dynamic lighting, and lighting animation. The image display requirement data is obtained by weighted summation of image resolution and frame rate. The specific weight values ​​are preset by those skilled in the art based on the specific rendering task and can be dynamically adjusted. Then, the sub-tasks to be rendered, the sub-tasks to be rendered (blocks), the sub-tasks to be rendered (frames), or the sub-tasks to be rendered (scenes) are classified according to high or low difficulty to obtain a set of sub-tasks to be rendered (blocks), a set of sub-tasks to be rendered (frames), and a set of sub-tasks to be rendered (scenes).

[0164] According to an embodiment of the present invention, it further includes:

[0165] Obtain data on CPU utilization, GPU utilization, memory usage, and network bandwidth usage of edge nodes;

[0166] The CPU utilization, GPU utilization, memory usage, and network bandwidth usage data are weighted and summed to obtain the real-time load data of the edge nodes.

[0167] The real-time load data of the edge nodes is compared with a preset load warning threshold to obtain the real-time load status of the edge nodes;

[0168] If the load is less than or equal to the preset load warning threshold, the real-time load status is determined to be low load.

[0169] If the load exceeds the preset load warning threshold, the real-time load status is determined to be high load.

[0170] It should be noted that, in order to achieve the adaptation between the subtask to be rendered and the edge node, the real-time load of the edge node is evaluated, the real-time load data of the edge node is obtained, and a threshold comparison is performed to divide the real-time load status into low load or high load. Low load is responsible for rendering high-difficulty tasks, and high load is responsible for rendering low-difficulty tasks.

[0171] According to an embodiment of the present invention, the step of allocating the subtask to be rendered to edge nodes through a preset method to obtain node rendering subtasks and performing real-time rendering to obtain the subtask rendering screen includes:

[0172] The task allocation is performed by combining the set of subtasks to be rendered (blocks), the set of subtasks to be rendered (frames), or the set of subtasks to be rendered (scenes) with the real-time load status to obtain node rendering subtasks.

[0173] Based on the node rendering subtasks, a test set is extracted to obtain the rendering test subtasks;

[0174] The edge nodes render the rendering test subtask and obtain the rendering test time data;

[0175] The rendering test time data is processed to obtain node rendering time prediction data.

[0176] The node rendering time prediction data is compared with the preset rendering time limit threshold.

[0177] If the timeout is less than the preset rendering time limit threshold, real-time rendering will be performed to obtain the subtask rendering screen.

[0178] If the value is greater than or equal to the preset rendering time limit threshold, an early warning response will be output.

[0179] It should be noted that after assigning high-difficulty subtask sets to low-load edge nodes and low-difficulty subtask sets to high-load edge nodes, before real-time rendering, in order to determine whether the rendering time of each node can meet the rendering timeliness, firstly, each edge node extracts rendering test subtasks for rendering processing and counts the rendering test time data. Then, edge node time prediction is performed to obtain node rendering time prediction data. Finally, threshold comparison is used to determine whether the time limit requirement is met.

[0180] According to an embodiment of the present invention, it further includes:

[0181] The rendering test subtask is simultaneously assigned to each edge node for real-time rendering to obtain the rendering test screen corresponding to the edge node.

[0182] The consistency of the rendered test images will be evaluated.

[0183] If the consistency assessment passes, then real-time rendering will proceed.

[0184] If the consistency assessment fails, the corresponding edge node will be sent to the user terminal for display.

[0185] It should be noted that, in order to evaluate the consistency of task rendering across edge nodes and improve the image composition effect, each edge node performs real-time rendering of the same test subtask to obtain the corresponding rendering test image of each edge node. The consistency of the image is then evaluated, and real-time rendering is only allowed if the evaluation is passed, so as to avoid wasting computing resources.

[0186] It is worth mentioning that, according to embodiments of the present invention, it further includes:

[0187] Obtain the synthetic test subtask of the first edge node;

[0188] The synthetic test subtask is assigned to the first edge node and the adjacent second edge node for real-time rendering to obtain the corresponding synthetic test screen;

[0189] The quality of the synthesized test footage will be evaluated.

[0190] If the quality assessment is passed, the subtask rendering images of the first edge node and the adjacent second edge node are composited.

[0191] If the quality assessment fails, the corresponding edge node will be sent to the user's terminal for display.

[0192] It should be noted that since the sub-task rendering images of each edge node need to be composited, in order to avoid unqualified image quality after composited images, after the real-time rendering consistency evaluation of each edge node passes, the rendering quality of adjacent edge nodes that need to be composited is further evaluated before composited images. First, the composited test sub-task of the first edge node is extracted. Then, it is simultaneously assigned to the two adjacent edge nodes that need to be composited for real-time rendering, and the composited test images corresponding to the first edge node and the adjacent second edge node are obtained. The quality is evaluated separately. If both adjacent edge nodes pass, composited images are allowed. Otherwise, the edge nodes that fail the quality evaluation are sent to the user for display.

[0193] It is worth mentioning that, according to an embodiment of the present invention, the consistency evaluation of the rendered test screen includes:

[0194] The rendered test screen is evaluated for spatial consistency, temporal consistency, visual consistency, and interactive consistency to obtain spatial consistency evaluation results, temporal consistency evaluation results, visual consistency evaluation results, and interactive consistency evaluation results.

[0195] Perform an AND operation on the spatial consistency assessment results, temporal consistency assessment results, visual consistency assessment results, and interaction consistency assessment results;

[0196] If the assessment is passed, the consistency assessment is deemed to have passed.

[0197] It should be noted that the evaluation of rendering consistency of each edge node includes spatial consistency, temporal consistency, visual consistency, and interactive consistency evaluation. Spatial consistency is evaluated by the deviation of the geometric position, size, and orientation of an object or scene from the corresponding preset geometric position, preset size, and preset orientation, respectively. Temporal consistency is evaluated by the timing of dynamic scenes to determine whether there are frame skips or excessive latency. Visual consistency is evaluated by the deviation of brightness, contrast, and color from the corresponding preset brightness, preset contrast, and preset color, respectively. Interactive consistency is evaluated by the consistency of the feedback of different edge nodes to state changes. The evaluation results for spatial consistency, temporal consistency, visual consistency, and interactive consistency are respectively categorized as passed or failed.

[0198] Please refer to Figure 5 , Figure 5 This is a device diagram of a distributed real-time rendering apparatus based on edge computing, as shown in some embodiments of this application.

[0199] A third aspect of the present invention provides a distributed real-time rendering device 5 based on edge computing, comprising:

[0200] The edge node and cloud collaboration module 51 includes an edge computing node unit 511 and a cloud server unit 512, which are used to build a rendering edge computing node and cloud collaborative rendering system.

[0201] The rendering acquisition module 52 is used to collect data and information from rendering tasks in real time.

[0202] The rendering evaluation module 53 is used to evaluate rendering tasks and edge computing nodes;

[0203] The rendering task processing module 54 is used to split and allocate tasks to be rendered;

[0204] The rendering task compositing module 55 is used to compose the rendering screen of subtasks.

[0205] This invention discloses a distributed real-time rendering method, system, and device based on edge computing. It realizes real-time rendering based on edge computing by intelligently matching and verifying the rendering capabilities of edge nodes, splitting and allocating rendering tasks according to the rendering scene, rendering difficulty, and edge node load status, and performing real-time node rendering consistency evaluation.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0207] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0208] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0209] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0210] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A distributed real-time rendering method based on edge computing, characterized in that, Includes the following steps: The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted. The scene complexity of the task to be rendered is obtained by processing the rendering complexity evaluation data. The scene complexity is matched with the preset rendering capabilities of the edge nodes. If the rendering capabilities of the edge nodes are insufficient, the rendering task is sent to the cloud for processing. If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data. The subtask to be rendered is assigned to the edge node using a preset method to obtain the node rendering subtask, and then rendered in real time to obtain the subtask rendering screen. The subtask rendering screen is composited using a preset compositing method to obtain a real-time rendering screen, which is then transmitted to the user terminal for display. The task to be rendered is split into sub-tasks to be rendered, including sub-tasks to be rendered block, sub-tasks to be rendered frame, and sub-tasks to be rendered scene. Obtain the corresponding rendering requirement information based on the block subtask to be rendered, the frame subtask to be rendered, or the scene subtask to be rendered. The rendering requirements information includes rendering accuracy, lighting requirements data, and screen display requirements data; The rendering accuracy, lighting requirements data and screen display requirements data are input into a preset rendering difficulty prediction model for processing to obtain rendering difficulty category labels, including high difficulty or low difficulty. The sub-tasks to be rendered (blocks, frames, or scenes) are classified according to their difficulty level (high or low) to obtain sets of sub-tasks to be rendered (blocks, frames, or scenes). Obtain data on CPU utilization, GPU utilization, memory usage, and network bandwidth usage of edge nodes; The CPU utilization, GPU utilization, memory usage, and network bandwidth usage data are weighted and summed to obtain the real-time load data of the edge nodes. The real-time load data of the edge nodes is compared with a preset load warning threshold to obtain the real-time load status of the edge nodes; The task allocation is performed by combining the set of subtasks to be rendered (blocks), the set of subtasks to be rendered (frames), or the set of subtasks to be rendered (scenes) with the real-time load status to obtain node rendering subtasks. Based on the node rendering subtasks, a test set is extracted to obtain the rendering test subtasks; The edge nodes render the rendering test subtask and obtain the rendering test time data; The rendering test time data is processed to obtain node rendering time prediction data. The rendering test subtask is simultaneously assigned to each edge node for real-time rendering to obtain the rendering test screen corresponding to the edge node. Obtain the synthetic test subtask of the first edge node; The synthetic test subtask is assigned to the first edge node and the adjacent second edge node for real-time rendering to obtain the corresponding synthetic test screen; The quality of the synthesized test footage will be evaluated. If the quality assessment is passed, the subtask rendering images of the first edge node and the adjacent second edge node are composited. If the quality assessment fails, the corresponding edge node will be sent to the user's terminal for display.

2. The distributed real-time rendering method based on edge computing according to claim 1, characterized in that, The step of obtaining rendering requirement information based on rendering instructions and extracting rendering scene category feature data and rendering complexity evaluation data includes: The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted. The rendered scene category feature data includes static scene feature data, dynamic scene feature data, or multi-dimensional independent scene feature data; The rendering complexity evaluation data includes dynamic object magnitude, geometric feature data, texture feature data, and light source feature data.

3. The distributed real-time rendering method based on edge computing according to claim 2, characterized in that, The process of processing the rendering complexity evaluation data to obtain the scene complexity of the task to be rendered includes: Based on the dynamic object magnitude, a preset dynamic object magnitude and weight value mapping table is queried to obtain geometric feature weight values, texture feature weight values, and light source feature weight values. The geometric feature data, texture feature data, and light source feature data are combined with the geometric feature weight value, texture feature weight value, and light source feature weight value, and then weighted and summed to obtain the scene complexity of the task to be rendered.

4. The distributed real-time rendering method based on edge computing according to claim 3, characterized in that, If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data, including: If the data is static scene feature data, the task to be rendered will be split into spatial tasks to obtain sub-tasks to be rendered. If the data is dynamic scene feature data, the task to be rendered is split into time tasks to obtain the sub-tasks of the frames to be rendered. If the data consists of multidimensional independent scene features, the task to be rendered will be split into scene tasks to obtain sub-tasks of the scene to be rendered.

5. The distributed real-time rendering method based on edge computing according to claim 4, characterized in that, Also includes: If the load is less than or equal to the preset load warning threshold, the real-time load status is determined to be low load. If the load exceeds the preset load warning threshold, the real-time load status is determined to be high load.

6. The distributed real-time rendering method based on edge computing according to claim 5, characterized in that, The step of allocating the subtask to be rendered to edge nodes using a preset method, obtaining node rendering subtasks, and performing real-time rendering to obtain the subtask rendering screen includes: The node rendering time prediction data is compared with the preset rendering time limit threshold. If the timeout is less than the preset rendering time limit threshold, real-time rendering will be performed to obtain the subtask rendering screen. If the value is greater than or equal to the preset rendering time limit threshold, an early warning response will be output.

7. A distributed real-time rendering system based on edge computing, characterized in that, Includes a memory and a processor. The memory contains a program for a distributed real-time rendering method based on edge computing. When the program for the distributed real-time rendering method based on edge computing is executed by the processor, it performs the following steps: The rendering requirements information is obtained based on the rendering instructions, and the rendering scene category feature data and rendering complexity evaluation data are extracted. The scene complexity of the task to be rendered is obtained by processing the rendering complexity evaluation data. The scene complexity is matched with the preset rendering capabilities of the edge nodes. If the rendering capabilities of the edge nodes are insufficient, the rendering task is sent to the cloud for processing. If the rendering capability of the edge nodes is sufficient, the task to be rendered is split into subtasks based on the rendering scene category feature data. The subtask to be rendered is assigned to the edge node using a preset method to obtain the node rendering subtask, and then rendered in real time to obtain the subtask rendering screen. The subtask rendering screen is composited using a preset compositing method to obtain a real-time rendering screen, which is then transmitted to the user terminal for display. The task to be rendered is split into sub-tasks to be rendered, including sub-tasks to be rendered block, sub-tasks to be rendered frame, and sub-tasks to be rendered scene. Obtain the corresponding rendering requirement information based on the block subtask to be rendered, the frame subtask to be rendered, or the scene subtask to be rendered. The rendering requirements information includes rendering accuracy, lighting requirements data, and screen display requirements data; The rendering accuracy, lighting requirements data and screen display requirements data are input into a preset rendering difficulty prediction model for processing to obtain rendering difficulty category labels, including high difficulty or low difficulty. The sub-tasks to be rendered (blocks, frames, or scenes) are classified according to their difficulty level (high or low) to obtain sets of sub-tasks to be rendered (blocks, frames, or scenes). Obtain data on CPU utilization, GPU utilization, memory usage, and network bandwidth usage of edge nodes; The CPU utilization, GPU utilization, memory usage, and network bandwidth usage data are weighted and summed to obtain the real-time load data of the edge nodes. The real-time load data of the edge nodes is compared with a preset load warning threshold to obtain the real-time load status of the edge nodes; The task allocation is performed by combining the set of subtasks to be rendered (blocks), the set of subtasks to be rendered (frames), or the set of subtasks to be rendered (scenes) with the real-time load status to obtain node rendering subtasks. Based on the node rendering subtasks, a test set is extracted to obtain the rendering test subtasks; The edge nodes render the rendering test subtask and obtain the rendering test time data; The rendering test time data is processed to obtain node rendering time prediction data. The rendering test subtask is simultaneously assigned to each edge node for real-time rendering to obtain the rendering test screen corresponding to the edge node. Obtain the synthetic test subtask of the first edge node; The synthetic test subtask is assigned to the first edge node and the adjacent second edge node for real-time rendering to obtain the corresponding synthetic test screen; The quality of the synthesized test footage will be evaluated. If the quality assessment is passed, the subtask rendering images of the first edge node and the adjacent second edge node are composited. If the quality assessment fails, the corresponding edge node will be sent to the user's terminal for display.

Images