A main terminal game distributed rendering system calling multiple weak terminal computing powers

Abstract

The application relates to the technical field of distributed rendering, in particular to a main terminal game distributed rendering system for calling multiple weak terminal computing powers. The system comprises a rendering complexity evaluation unit, which determines the rendering complexity evaluation degree of an initial rendering block according to the vertex density of the initial rendering block; a rendering time delay estimation unit, which determines the rendering capability value of each weak terminal device based on the rendering complexity evaluation degree and the rendering processing time delay of a historical rendering task, and analyzes the rendering time delay estimation value of each weak terminal device for the initial rendering block; a running state detection unit, which detects whether each weak terminal device is overloaded through running state data; and a block adjustment and rendering unit, which adjusts the rendering block based on the rendering time delay estimation value and the rendering overload weight when there is an overloaded node or the rendering time delay estimation value exceeds a range, and performs rendering; the system effectively prevents the weak terminal device from crashing due to task overload, ensures the stable operation of the distributed rendering system, and improves the smoothness of game operation.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of distributed rendering, in particular to a main terminal game distributed rendering system for calling multiple weak terminal computing powers. BACKGROUND

[0002] With the development of game image quality in the digital entertainment industry towards 4K / 8K, high frame rate, ray tracing and other directions, high-quality games have increasingly stringent requirements for hardware performance. Not only is it required to process massive geometric data, complex physical simulation and high-resolution texture rendering in real time, but also a powerful graphics processing unit (GPU) is needed to provide continuous computing power support. The existing technology usually adopts cloud gaming solutions and rendering farms to overcome the computing power bottleneck of a single device. Cloud gaming relies on remote server rendering, which is highly sensitive to network delay, and the user experience is very poor under weak network conditions. The rendering farm needs to build a high-performance computing cluster, which is generally used in scenarios such as film production and game development, and has a high cost, which is not suitable for home game scenarios.

[0003] In the home scenario, PC, smart TV, tablet, mobile phone and other weak computing power terminal devices are in an idle or light load state, the resource utilization rate is low, and the computing power aggregation potential is great, which can meet the rendering requirements of high-quality games. The GPU computing power of different home terminal devices differs greatly, and the collaboration of multiple home terminal devices for distributed rendering is weak, which is prone to problems such as uneven relative load distribution, flickering game image and forced frame dropping. SUMMARY

[0004] In order to solve the above technical problems, the purpose of the present application is to provide a main terminal game distributed rendering system for calling multiple weak terminal computing powers, and the technical solution adopted is as follows:

[0005] The application provides a main terminal game distributed rendering system for calling multiple weak terminal computing powers, which comprises:

[0006] A rendering complexity evaluation unit: the rendering complexity evaluation unit obtains the vertex density in an initial rendering block of each weak terminal device in a three-dimensional scene to be rendered in each frame of game running, and determines the rendering complexity evaluation degree of the initial rendering block of each weak terminal device;

[0007] A rendering time delay estimation unit: the rendering time delay estimation unit determines the rendering capability value of each weak terminal device based on the rendering complexity evaluation degree and the rendering processing time delay of the historical rendering block of each weak terminal device, and obtains the rendering time delay estimation value required by each weak terminal device in combination with the rendering complexity evaluation degree of the initial rendering block in all frames of three-dimensional scene corresponding to each weak terminal device;

[0008] A running state detection unit: the running state detection unit obtains the running state data of each weak terminal device, calculates the rendering overload weight of each weak terminal device, and detects whether each weak terminal device is an overload node;

[0009] The block adjustment and rendering unit determines whether to adjust the initial rendering block of all weak terminal devices based on the number of overloaded nodes in all weak terminal devices and the numerical distribution range of the rendering time delay estimation value. When adjustment is needed, the initial rendering block of each weak terminal device in each frame of three-dimensional scene is adjusted based on the rendering time delay estimation value and the rendering overload weight to obtain the final rendering block of each weak terminal device. Each weak terminal device renders the corresponding rendering block in each frame of three-dimensional scene.

[0010] In one embodiment, the rendering complexity evaluation degree acquisition process is as follows:

[0011] The variation coefficients of the coordinate values of all vertices in each initial rendering block in the x-axis direction, the y-axis direction and the z-axis direction are calculated.

[0012] The rendering complexity evaluation degree of each initial rendering block is positively correlated with the number of vertices in each initial rendering block, and is negatively correlated with the variation coefficients of the coordinate values of all vertices in each initial rendering block in each axis direction.

[0013] In one embodiment, the rendering capability value acquisition process is as follows:

[0014] The rendering subtask refers to the rendering block to which the weak terminal device is divided. The ratio of the rendering complexity evaluation degree to the rendering processing time delay of each historical rendering subtask of each weak terminal device is calculated, and the average of the ratios of all historical rendering subtasks of each weak terminal device is taken as the rendering capability value of each weak terminal device.

[0015] In one embodiment, the rendering time delay estimation value acquisition process is as follows:

[0016] The rendering complexity evaluation degrees of the corresponding initial rendering blocks of each weak terminal device in all frames of three-dimensional scene are fused, and the ratio of the fusion result to the rendering capability value of each weak terminal device is taken as the rendering time delay estimation value of each weak terminal device.

[0017] In one embodiment, the rendering overload weight acquisition process is as follows:

[0018] The running state data includes the CPU usage, memory occupancy and GPU usage of the weak terminal device. The overload amount of the current weak terminal device is determined based on the difference between each type of running state data of the current weak terminal device and the preset resource usage threshold.

[0019] If the overload amount is greater than or equal to 0, the overload amount is taken as the rendering overload weight of the current weak terminal device; otherwise, the rendering overload weight of the current weak terminal device is set to 0.

[0020] In one embodiment, the overload amount is the sum of the difference between all the type running state data of the current weak terminal device and the preset resource usage threshold.

[0021] In one embodiment, the overload node is a weak terminal device whose rendering overload weight is greater than 0.

[0022] In one embodiment, the determination of whether the initial rendering block of all the weak terminal devices needs to be adjusted is as follows:

[0023] The rendering time delay estimation value of each weak terminal device is multiplied by the inverse function of the preset game time delay threshold to obtain a time delay adjustment coefficient of each weak terminal device. If there is no overload node in all the weak terminal devices and the time delay adjustment coefficients of all the weak terminal devices belong to a preset numerical range, the initial rendering block is not adjusted. Otherwise, the initial rendering block needs to be adjusted.

[0024] In one embodiment, the process of obtaining the final rendering block of each weak terminal device is as follows:

[0025] The height update value of the rendering block of the weak terminal device i is calculated as follows: The expression is as follows: , wherein, is a floor function; is the height of the initial rendering block in any frame of the three-dimensional scene of the weak terminal device i; is the time delay adjustment coefficient of the weak terminal device i; is the rendering overload weight of the weak terminal device i;

[0026] Based on the rendering processing time delay of all the weak terminal devices in rendering the same rendering task, the rendering capacity of each weak terminal device is determined, and the final rendering block of each weak terminal device is determined in sequence in combination with the height update value.

[0027] In one embodiment, the process of determining the final rendering block of each weak terminal device in sequence is as follows:

[0028] The rendering block of the weak terminal device with weaker rendering capacity is closer to the upper and lower edges of the current frame of the three-dimensional scene, and the rendering block of the weak terminal device with stronger rendering capacity is closer to the middle height region of the current frame of the three-dimensional scene. The height of the rendering block of each weak terminal device is the height update value thereof.

[0029] The present application has the following advantages:

[0030] ​This application incorporates various weak terminal devices within a home network into a distributed rendering system, enabling idle weak terminal devices to participate in high-quality game rendering, releasing their potential computing power, and meeting users' demands for high-quality games. Through the division of labor and collaboration between the main terminal and weak terminal devices, this application efficiently processes game rendering tasks. The main terminal, with its stronger computing power, is responsible for distributed rendering scheduling and early-stage rendering, while the weaker terminal devices, with their lower computing power, are responsible for later-stage rendering. This significantly shortens the total processing time for game rendering tasks, supports rendering requirements for higher resolutions and more complex scenes, and improves game smoothness and frame rate. When allocating rendering sub-tasks, this application assesses the complexity of the initially allocated rendering tasks and the rendering performance of the corresponding weak terminal devices, dynamically monitors the load of each weak terminal device, adaptively adjusts the task allocation strategy, and effectively prevents weak terminal devices from crashing due to task overload. This ensures the stable operation of the distributed rendering system, effectively coordinates load distribution, improves game smoothness, and enhances the reliability and comfort of the user experience. Attached Figure Description

[0031] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This application provides a block diagram of a distributed rendering system for a main terminal game that utilizes the computing power of multiple weak terminals, as one embodiment of the present application.

[0033] Figure 2 This is a schematic diagram of the rendering block division;

[0034] Figure 3 A block diagram showing the breakdown of the distribution module for rendering tasks. Detailed Implementation

[0035] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a distributed rendering system for a main terminal game that utilizes the computing power of multiple weak terminals, as proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0037] The application provides a specific scheme of a main terminal game distributed rendering system calling multiple weak terminal computing powers.

[0038] Please refer to Figure 1 which shows a block diagram of a main terminal game distributed rendering system calling multiple weak terminal computing powers provided by an embodiment of the application, and the system comprises:

[0039] The rendering editor module 101: in the game distributed rendering system, all home terminal devices (the home terminal devices need to be equipped with GPUs (Graphics Processing Unit)) are connected through a home network, the home terminal device running the game is taken as a main terminal device, other GPU home terminal devices are taken as weak terminal devices, a distributed rendering client is configured on the main terminal device to perform distributed rendering of the game. The home terminal devices specifically include smart TVs, PCs, smart phones, smart tablets and the like. A user can operate through the main terminal device using an input device, and the game distributed 3D rendering needs to render a 3D model under a specific view angle to generate a game 2D image and finally display on a main terminal output device. Therefore, the rendering editor needs to synchronously perceive the operation instructions of the main terminal input device, and the main terminal input device specifically includes a keyboard, a mouse, a joystick and the like. The rendering editor automatically calls the game settings of the main terminal device through an API interface to obtain the rendering parameters of the main terminal game, and the rendering parameters specifically include an image resolution, a frame rate, a shadow, illumination, a texture and the like.

[0040] The rendering server management module 102: the rendering server management module mainly realizes scanning, monitoring and fault-tolerant management of weak terminal rendering nodes, including a rendering node scanning management unit, a rendering node monitoring management unit and a fault-tolerant management unit of the rendering node.

[0041] The rendering node scanning management unit: after the distributed rendering client of the main terminal device is started, a comprehensive scanning mode based on a network protocol, such as ARP scanning, ICMP scanning and the like, is used to traverse and scan all home terminal devices in the home network, and a connection is established, and the weak terminal devices are taken as rendering nodes of the main terminal distributed rendering.

[0042] Rendering Node Monitoring and Management Unit: The main terminal device sends information query requests to connected weak terminals, requesting to obtain the hardware configuration information of the weak terminals. After receiving the request, the weak terminals call their own hardware information acquisition interface to collect data such as video memory capacity, video memory bandwidth, CUDA core count, and graphics card frequency, and then feed this information back to the rendering server management module of the main terminal device through the home network. The main terminal device also sends running status query requests to connected weak terminals. The weak terminals monitor their own running status data in real time, including CPU utilization, memory usage, and GPU utilization, and feed this data back to the rendering server management module of the main terminal device through the home network.

[0043] Fault-tolerant management unit for rendering nodes: When a rendering node fails or goes offline, if the rendering tasks of the failed or offline weak terminal are assigned to other weak terminal rendering nodes, the main terminal device needs to reissue the rendering tasks, increasing transmission latency and easily causing the rendering nodes to be overloaded. Therefore, this application uniformly assigns the rendering tasks of failed or offline weak terminals to the main terminal device for rendering processing to ensure the continuity of rendering tasks and the availability of the distributed rendering system.

[0044] Rendering Task Management Module 103: The rendering task management module is a key component of the distributed rendering client. It utilizes rendering task management methods to achieve state awareness of rendering tasks, pre-rendering, task breakdown and distribution, and compositing, including:

[0045] Rendering task status awareness module:

[0046] The rendering task management module monitors the rendering subtask information of M weak terminal rendering nodes in real time. The rendering subtask information specifically refers to the most recently processed tasks by each weak terminal rendering node. The rendering complexity assessment and rendering processing latency of each historical rendering subtask are evaluated. The rendering processing latency is specifically the time interval from the start of rendering on a weak terminal device to the completion of rendering.

[0047] A rendering task F is pre-built. Whenever a new weak terminal device establishes a connection with the main terminal device, the rendering task management module needs to send a rendering task F to the new weak terminal device and record the rendering processing latency of rendering task F. According to the rendering processing latency obtained from rendering task F in descending order, the rendering subtask allocation sequence for the weak terminal devices is obtained. The weak terminal devices that are listed earlier have slower rendering speeds and lower rendering performance, and therefore need to be prioritized for rendering subtask allocation.

[0048] Main terminal pre-rendering module:

[0049] A graphics rendering pipeline can render a dynamic three-dimensional scene, which is mainly related to three-dimensional objects, virtual cameras and light sources, into a frame-by-frame 2D image and present it on a display. The complete rendering pipeline process is mainly divided into application, geometry, rasterization and pixel stages.

[0050] In order to fully call the computing power of the main terminal and the weak terminal in the home network, the pre-rendering work of the game is preliminarily completed on the main terminal device, and the subsequent rendering task is divided into blocks and dynamically allocated to each connected weak terminal device for distributed rendering, and finally rendered and merged by the main terminal device. The pre-rendering work of the main terminal device specifically includes the application and geometry stages.

[0051] The rendering task management module receives the game running N frames of three-dimensional scene information to be rendered by the main terminal device, and the three-dimensional scene information specifically includes vertex coordinates, vertex shading information and virtual camera perspective information.

[0052] The vertex shading information is the color information of the R, G and B channels of the three-dimensional object vertex, the virtual camera perspective information is the position and orientation of the game rendering virtual camera, and the vertex coordinates use the model space (ModelSpace) coordinate system under the game scene.

[0053] The picture displayed on the display screen of the game scene is the scene that can be seen from the virtual camera, and the content of the screen changes with the change of the position and orientation of the virtual camera.

[0054] The main terminal device takes the three-dimensional scene information of each frame as input, uses MVP (Model-View-Projection) transformation to obtain the coordinates of the three-dimensional object vertex in the projection space (Projection Space), and through view frustum clipping and screen mapping, the three-dimensional object vertex projection space coordinates are converted to the window coordinate system again. The lower left corner of the main terminal screen is the origin of the window coordinate system, the horizontal right is the +x axis, the vertical up is the +y axis, and the z axis is the depth value of the three-dimensional object vertex, which is the straight line distance from the three-dimensional object vertex to the virtual camera.

[0055] The rendering task disassembling and distributing module includes a rendering complexity evaluation unit, a rendering time delay estimation unit, a running state detection unit and a block adjustment and rendering unit.

[0056] The rendering complexity evaluation unit obtains the vertex density in the initial rendering block of each weak terminal device in the three-dimensional scene to be rendered by the game running of each frame, and determines the rendering complexity evaluation degree of the initial rendering block of each weak terminal device.

[0057] The three-dimensional scene to be rendered for each frame of the game is a rendering task to be disassembled. The rendering task disassembly and distribution needs to disassemble the rendering task of each frame into M rendering sub-tasks, adaptively perform rendering block division according to the performance and load characteristics of the weak terminal device, distribute the rendering block as the disassembled rendering sub-task to the corresponding weak terminal device for distributed rendering, so that the low-performance device can also run high-quality games, and the processing of the game rendering task is more efficient and controllable.

[0058] After the N frames of three-dimensional scenes to be rendered by the game are converted into window coordinate space, each frame of three-dimensional scene needs to be horizontally divided into M rendering blocks in the xoy plane, and M weak terminal devices connected by a home network are used for distributed rendering. During the game running, the three-dimensional scene to be rendered may change in real time, and the rendering block size of each weak terminal device needs to be dynamically updated. Among them, the resolution of the main terminal display device is width*height, and the initial rendering block size of each weak terminal device is . In order to adapt to the high-quality and high-frame-rate running conditions of the game, and to prevent frequent adjustment of the rendering block size, the rendering task management module in this application dynamically adjusts the rendering block size of each weak terminal device once every N frames of three-dimensional scene. Preferably, the value of N in the embodiment of the application is set to 10. As other embodiments of the application, the implementer can set the value of N according to the actual situation.

[0059] According to the rendering block of each weak terminal device obtained by adjusting the N frames of three-dimensional scene in the last round, the N frames of three-dimensional scene in this round are preliminarily horizontally divided in the xoy plane, so that each weak terminal device corresponds to an initial rendering block in each frame of three-dimensional scene. That is, the position of the rendering block of each weak terminal device in the three-dimensional scene after the last adjustment is the initial rendering block of each weak terminal device in the three-dimensional scene in this adjustment. For any initial rendering block in any frame of three-dimensional scene, denoted as the current initial rendering block, the variation coefficients of the coordinate values of all vertices in the current initial rendering block in the x-axis direction, y-axis direction and z-axis direction are calculated, that is, the variation coefficients of the x coordinate values, y coordinate values and z coordinate values of all vertices in the initial rendering block are calculated, respectively, and denoted as , and . The rendering complexity evaluation degree of any initial rendering block is calculated, and the expression is:

[0060]

[0061] In the formula, is the rendering complexity evaluation degree of the current initial rendering block; is the total number of all vertices in the current initial rendering block; is a preset non-zero factor, which is used to prevent the denominator from being zero, and the value range is [0.01, 0.1], and in this embodiment is set to 0.01, is a logarithmic function with the natural constant e as the true number. Since , adding the natural number 1 in the formula is to ensure that the numerator is positive.

[0062] is used to reflect the rendering workload of the weak terminal device when rendering the initial rendering block, and the more the number of vertices that need to be processed during rendering, the greater the rendering workload, and at the same time reflects the spatial dispersion of the vertices during the initial rendering block rendering, the smaller, the higher the spatial concentration of the vertices of the three-dimensional object to be rendered, the more detailed the depth level, and the more complex the surface of the three-dimensional object to be rendered, increasing the rendering difficulty, and the rendering complexity evaluation is greater.

[0063] The rendering delay estimation unit determines the rendering capability value of each weak terminal device based on the rendering complexity evaluation and the rendering processing delay of each historical rendering block of each weak terminal device, and obtains the required rendering delay estimation value of each weak terminal device in combination with the rendering complexity evaluation of the initial rendering block corresponding to each weak terminal device in all frames of the three-dimensional scene.

[0064] The GPU computing power of different home terminal devices differs greatly, and the synchronization of the rendering processing delay of the weak terminal devices needs to be considered when performing distributed rendering of multiple home terminals, and the rendering subtask with high rendering complexity evaluation is allocated to the weak terminal device with high performance as much as possible to ensure that the game running scene can be smoothly and quickly rendered and displayed on the output display screen of the main terminal.

[0065] For any weak terminal device, the rendering complexity evaluation and the rendering processing delay of historical rendering subtasks can be obtained, and the rendering performance ratio of the weak terminal device i can be obtained by the following formula , and the expression is:

[0066]

[0067] In the formula, is the rendering capability value of the weak terminal device i, is the number of historical rendering subtasks of the weak terminal device i, is the rendering complexity evaluation of the nth historical rendering subtask of the weak terminal device i, is the rendering processing delay of the nth historical rendering subtask of the weak terminal device i. to reflect the rendering complexity evaluation degree of the rendering subtask that the weak terminal device i can process in a unit of time, The greater the value is, the stronger the rendering performance of the weak terminal device i is, and the more complex rendering subtasks the weak terminal device i can render and process in a short time. The historical rendering subtasks of the weak terminal device correspond to the rendering blocks allocated to the weak terminal device after the last round of adjustment. Since the size and position of the rendering blocks corresponding to the weak terminal device are adjusted every N frames in the embodiment of the present application, the present application obtains historical rendering subtasks.

[0068] For the N-frame three-dimensional scene to be rendered, there are M initial rendering blocks per frame of the three-dimensional scene in the present application, M is the number of weak terminal devices, and each weak terminal device corresponds to N initial rendering blocks according to the frame number. The cumulative value of the rendering complexity evaluation degrees of the N initial rendering blocks of the weak terminal device i is taken as the rendering complexity estimation value of the weak terminal device i for continuing to use the last rendering block for rendering .

[0069] Further, the rendering time delay estimation value of the weak terminal device i for rendering the N initial rendering blocks is obtained , and the expression is: , which reflects the estimation value of the time consumed by the weak terminal device for rendering a similar rendering complexity. to quantify the rendering processing performance of the weak terminal device. If the rendering complexity increases, that is, the greater the value is, the greater the difficulty of rendering the N-frame three-dimensional scene is, and if the size of the rendering block of the weak terminal device is not changed, the rendering time delay of the weak terminal device will increase.

[0070] The running state detection unit obtains the running state data of each weak terminal device, calculates the rendering overload weight of each weak terminal device, and detects whether each weak terminal device is an overload node.

[0071] The main terminal game distributed rendering method adopted in the present application needs the weak terminal to provide most of the rendering computing power, but the weak terminal device often has limited GPU, CPU and memory capacity, and limited rendering processing performance. The user can use the weak terminal device at any time, and it is necessary to prevent the weak terminal device from being allocated a rendering subtask with high complexity and high rendering delay, which leads to excessive load and increases the risk of the weak terminal device crashing, affecting the user experience.

[0072] Therefore, the rendering overload weight of the weak terminal device i is calculated , and the expression is:

[0073]

[0074]

[0075] In the formula, is the rendering overload weight of the weak terminal device i; is the overload amount of the weak terminal device i; C is the number of categories of the running state data of the weak terminal device i, which is 3 in this embodiment, and the running state data of the weak terminal device i in this embodiment includes the CPU usage rate, the memory occupancy rate and the GPU usage rate of the weak terminal device i; is the cth running state data of the weak terminal device i; is a preset resource usage rate threshold, which is used to prevent excessive load and reduce the risk of death of the weak terminal device, and the value range is [80%, 95%], and the value is 85% in this embodiment.

[0076] The CPU usage rate, the memory occupancy rate and the GPU usage rate directly reflect the running state of the weak terminal device, The greater the value is, the higher the overload risk of the weak terminal device is, and if , the rendering task management module needs to intervene in the rendering block size of the weak terminal device to prevent the rendering block from being too large, so that the computing power of the weak terminal device is called in large quantity and the load is excessive, thereby affecting the user experience.

[0077] The block adjustment and rendering unit determines whether the initial rendering block of all weak terminal devices needs to be adjusted based on the number of overload nodes in all weak terminal devices and the numerical distribution range of the rendering time delay estimation value, wherein when adjustment is needed, the initial rendering block of each weak terminal device in each frame of three-dimensional scene is adjusted based on the rendering time delay estimation value and the rendering overload weight, to obtain the final rendering block of each weak terminal device; each weak terminal device renders the corresponding rendering block in each frame of three-dimensional scene.

[0078] All weak terminal devices with a rendering overload weight greater than 0 are regarded as overload nodes, and the rendering block size of the overload nodes needs to be prevented from being overloaded. The game time delay threshold Th is set, the greater the game time delay threshold Th is, the higher the time delay requirement for the weak terminal device to be distributed for rendering, and the value is 10 ms in this embodiment. The difference between the rendering time delay estimation value and the game time delay threshold is taken as the numerator, the game time delay threshold is taken as the denominator, and the ratio result is taken as the time delay adjustment coefficient of the weak terminal device i .

[0079] Further, the adaptive update of the rendering block size of the M weak terminal devices is realized by the following method:

[0080] (1) If there is no overload node in all weak terminal devices, and the delay adjustment coefficient of all weak terminal devices belongs to [0.9, 1.1]. Under this condition, it is indicated that the weak terminal device has extremely low overload risk in the distributed rendering process, and the rendering delay is not much different from the historical rendering delay. In order to avoid the waste of main terminal computing resources caused by the frequent adjustment of the rendering block of the weak terminal device, and to improve the accuracy of the delay estimation, the weak terminal device under this condition uses the rendering block obtained last time, that is, the initial rendering block does not need to be adjusted.

[0081] (2) If there is an overload node in the weak terminal device, or the delay adjustment coefficient of the weak terminal device does not belong to [0.9, 1.1], the dynamic update of the rendering block size of all weak terminal devices is needed under this condition.

[0082] The height update value of the rendering block of the weak terminal device i is calculated , and the expression is: , wherein, is a floor function; is the height of the initial rendering block in any frame three-dimensional scene of the weak terminal device i; is the delay adjustment coefficient of the weak terminal device i; is the rendering overload weight of the weak terminal device i.

[0083] And the smaller, that is, The larger, it is indicated that the rendering estimated delay is lower than the game delay threshold, which meets the delay requirement of game rendering, and a larger weak terminal device rendering block height should be used, The larger, which is used to improve the resource utilization rate of the weak terminal device; The smaller, it is indicated that the possibility of overload of the weak terminal device is larger, and a smaller weak terminal device rendering block height should be used, The smaller, which is used to prevent the weak terminal device from crashing and enhance the user experience.

[0084] By arranging all the weak terminal devices in the order of the rendering processing delay obtained by rendering the same rendering task F from large to small, the rendering sub-task allocation sequence of the weak terminal device is obtained. The rendering processing delay reflects the rendering capability of the weak terminal device, and the larger the rendering processing delay, the smaller the rendering capability. The principle of "alternating first and last rows" is used to update the rendering block of the weak terminal device in turn to obtain the final rendering block of each weak terminal device. For example, the height range of each frame three-dimensional scene is Taking any frame three-dimensional scene as an example, the area with the height range between in the three-dimensional scene is taken as the rendering block of the first weak terminal device in the rendering sub-task allocation sequence, and the area with the height range between the area with height range between 0 and 0.5 as the rendering block of the second weak terminal device, the area with height range between 0.5 and 1 as the rendering block of the third weak terminal device, the area with height range between 1 and 1.5 as the rendering block of the fourth weak terminal device, and so on, as shown in FIG. 4. the area with height range between 0 and 0.5 as the rendering block of the second weak terminal device, the area with height range between 0.5 and 1 as the rendering block of the third weak terminal device, the area with height range between 1 and 1.5 as the rendering block of the fourth weak terminal device, and so on, as shown in FIG. 4. the area with height range between 0 and 0.5 as the rendering block of the second weak terminal device, the area with height range between 0.5 and 1 as the rendering block of the third weak terminal device, the area with height range between 1 and 1.5 as the rendering block of the fourth weak terminal device, and so on, as shown in FIG. 4. Figure 2 the area with height range between 0 and 0.5 as the rendering block of the second weak terminal device, the area with height range between 0.5 and 1 as the rendering block of the third weak terminal device, the area with height range between 1 and 1.5 as the rendering block of the fourth weak terminal device, and so on, as shown in FIG. 4.

[0085] It should be noted that: 1) if the accumulated value of the height update value of the rendering block of all weak terminal devices is less than 0, the rendering block composed of the remaining rows is rendered by the main terminal device; 2) if the accumulated value of the height update value of the rendering block of all weak terminal devices is greater than 1, the weak terminal device in the last column of the rendering sub-task assignment sequence cannot preempt the rendering block of the weak terminal device in the front row, for example, the rendering block of the weak terminal device t-2 is (35, 45], the rendering block of the weak terminal device t-1 is [60, 75], and the height update value of the rendering block of the weak terminal device t is greater than 1, so the rendering block of the weak terminal device t cannot preempt the rendering block of the weak terminal device in the front row, that is, the rendering block of the weak terminal device t-1 [60, 75], so the rendering block of the weak terminal device t is (45, 60).

[0086] The purpose of the "first and last row alternation" principle is that the weak terminal device with excellent rendering performance is more adaptable to the rendering sub-task difficulty, and the area of interest in the game rendering picture is often concentrated in the middle area of the screen, and the rendering complexity of the middle area is often higher. The "first and last row alternation" principle can avoid the problem of long rendering delay of the weak terminal device with low performance, causing flickering of the game picture, forced frame dropping and the like.

[0087] According to the above steps, the rendering block of N frames of three-dimensional objects in the window coordinate system is adaptively updated, each rendering block corresponds to a weak terminal device, and the rendering task management module distributes the rendering block of each frame of three-dimensional objects as a rendering sub-task to the corresponding weak terminal device.

[0088] After the weak terminal device receives the rendering sub-task, it performs the post-rendering operation, and the post-rendering operation specifically includes rasterization and pixel stage. The goal of the rasterization stage is to find all the pixels inside the primitive, and then convert the vertex to the pixel on the screen, each pixel is attached with depth and shading information, and is transmitted to the corresponding pixel. The pixel stage is executed on the GPU of the weak terminal device, and the GPU performs pixel operations on each fragment, such as color and depth calculation, texture sampling, blending, etc.

[0089] The rendering task synthesis module:

[0090] ​​​The rendering task management module receives the rendering results returned by the M weak terminal devices, and composes the rendering results according to the frame orders corresponding to the rendering sub-tasks to obtain a final rendering image. The rendering task management module outputs the final rendering image to the main terminal CPU memory for use in the output of the game running process of the main terminal device.

[0091] A block diagram of the rendering task disassembling and distributing module is shown in Figure 3

[0092] To sum up, the embodiments of the present application enable the idle home weak terminal devices to participate in high-quality game rendering by including the weak terminal devices in the home network in the distributed rendering system, release the potential computing power of the home weak terminal devices, and meet the demand of users for high-quality games. The embodiments of the present application efficiently process game rendering tasks through the division of labor and cooperation between the main terminal and the weak terminal devices. The main terminal with strong computing power is responsible for distributed rendering scheduling and preliminary rendering, and the weak terminal devices with poor computing power are responsible for later rendering, which greatly shortens the total processing time of game rendering tasks, supports rendering requirements of higher resolution and more complex scenes, and improves the smoothness and frame rate of games. When distributing rendering sub-tasks, the embodiments of the present application evaluate the complexity of the initially distributed rendering tasks and the rendering performance of the corresponding weak terminal devices, dynamically monitor the load conditions of each weak terminal device, adaptively adjust the task distribution strategy, and effectively prevent the weak terminal devices from crashing due to task overload, thereby ensuring the stable operation of the distributed rendering system, effectively coordinating the load distribution, improving the game running smoothness, and improving the reliability and comfort of user experience.

[0093] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or can be advantageous.

[0094] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment mainly describes the differences from other embodiments.

[0095] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; the technical solutions described in the above embodiments are modified, or some technical features are replaced, without changing the essence of the corresponding technical solutions out of the scope of the technical solutions of the embodiments of the present application, which should be included in the protection scope of the present application.​

Claims

1. A main terminal game distributed rendering system for calling multiple weak terminal computing power, characterized in that, The system comprises: a rendering complexity evaluation unit configured to obtain vertex density in each initial rendering block of a three-dimensional scene to be rendered in each frame of a game, and determine rendering complexity evaluation degrees of the initial rendering blocks of the weak terminal devices; a rendering time delay estimation unit configured to determine rendering capability values of the weak terminal devices based on the rendering complexity evaluation degrees and rendering processing time delays of historical rendering blocks of the weak terminal devices, and obtain rendering time delay estimation values of the weak terminal devices in combination with the rendering complexity evaluation degrees of the initial rendering blocks of the weak terminal devices in all frames of the three-dimensional scene; a running state detection unit configured to obtain running state data of the weak terminal devices, calculate rendering overload weights of the weak terminal devices, and detect whether the weak terminal devices are overload nodes; a block adjustment and rendering unit configured to determine whether the initial rendering blocks of the weak terminal devices need to be adjusted based on a number of overload nodes among the weak terminal devices and a numerical distribution range of the rendering time delay estimation values, and when adjustment is needed, adjust the initial rendering blocks of the weak terminal devices in each frame of the three-dimensional scene based on the rendering time delay estimation values and the rendering overload weights to obtain final rendering blocks of the weak terminal devices, and render the corresponding rendering blocks in each frame of the three-dimensional scene by the weak terminal devices. The final rendering blocks of the weak terminal devices are obtained in the following process: updating the height of the rendering block of the weak terminal device i , The expression is: , wherein, is a floor function; is the height of the initial rendering block in the three-dimensional scene of any frame of the weak terminal device i; is the time delay adjustment coefficient of the weak terminal device i; is the rendering overload weight of the weak terminal device i; wherein the operation result of the inverse proportional function of the game time delay threshold value and the rendering time delay estimation value of each weak terminal device is taken as the time delay adjustment coefficient of each weak terminal device. determine rendering capabilities of the weak terminal devices based on rendering processing time delays of the weak terminal devices in rendering the same rendering task, and determine the final rendering blocks of the weak terminal devices in turn in combination with the height update values; the rendering block of a weak terminal device with weaker rendering capability is closer to upper and lower edges in a current frame of the three-dimensional scene, and the rendering block of a weak terminal device with stronger rendering capability is closer to a middle height area in the current frame of the three-dimensional scene, and the height of the rendering block of each weak terminal device is the height update value thereof.

2. The main terminal game distributed rendering system of claim 1, wherein, The rendering complexity evaluation degrees are obtained in the following process: calculate variation coefficients of coordinate values of all vertices in each initial rendering block in x-axis direction, y-axis direction and z-axis direction; the rendering complexity evaluation degree of each initial rendering block is positively correlated with a number of vertices in the initial rendering block, and is negatively correlated with the variation coefficients of the coordinate values of all vertices in each initial rendering block in each axis direction.

3. The main terminal game distributed rendering system of claim 1, wherein, The rendering capability values are obtained in the following process: the rendering subtask refers to a rendering block to which a weak terminal device is divided; calculate a ratio of the rendering complexity evaluation degree to the rendering processing time delay of each historical rendering subtask of each weak terminal device, and take an average of the ratios of all historical rendering subtasks of each weak terminal device as the rendering capability value of the weak terminal device.

4. The main terminal game distributed rendering system of claim 1, wherein, The rendering time delay estimation values are obtained in the following process: fuse the rendering complexity evaluation degrees of the initial rendering blocks of each weak terminal device in all frames of the three-dimensional scene, and take a ratio of the fusion result to the rendering capability value of each weak terminal device as the rendering time delay estimation value of the weak terminal device.

5. The main terminal game distributed rendering system of claim 1, wherein, The rendering overload weights are obtained in the following process: The running state data includes CPU usage, memory occupancy and GPU usage of the weak terminal device; an overload of the current weak terminal device is determined based on a difference between each type of running state data of the current weak terminal device and a preset resource usage threshold; If the overload is greater than or equal to 0, the overload is taken as a rendering overload weight of the current weak terminal device; Otherwise, the rendering overload weight of the current weak terminal device is set to 0.

6. The main terminal game distributed rendering system of claim 5, wherein, The overload is a cumulative sum of differences between all types of running state data of the current weak terminal device and the preset resource usage threshold.

7. A distributed rendering system for a main terminal game that utilizes the computing power of multiple weak terminals as described in claim 1, characterized in that, The overload node is a weak terminal device with a rendering overload weight greater than 0.

8. The main terminal game distributed rendering system of claim 1, wherein, The determination of whether the initial rendering block of all weak terminal devices needs to be adjusted is specifically: If there is no overload node in all weak terminal devices, and the time delay adjustment coefficients of all weak terminal devices belong to a preset numerical range, the initial rendering block is not adjusted; Otherwise, the initial rendering block needs to be adjusted.

Images