Dynamic task configuration method for high-consistency three-dimensional rendering

Abstract

The application discloses a dynamic task configuration method for high-consistency three-dimensional rendering. The application takes the resolution and code rate of a user in a current time slot as a decision variable, constructs a rendering end-to-end total time delay and user experience function, further constructs a strong persistent consistency optimization target and determines a constraint condition, decomposes the strong persistent consistency optimization target into each time slot to obtain a single-time-slot optimization target and a single-time-slot constraint condition; a time delay consistency term in the single-time-slot optimization target is decoupled by introducing an intermediate variable mechanism, and each candidate resolution and code rate is combined to form a searchable joint configuration group set, the joint configuration result and the intermediate variable are alternately solved to obtain an optimal joint configuration result of the current time slot, and the optimal joint configuration result is taken as a rendering parameter configuration result of the current time slot. The application can significantly improve the end-to-end time delay consistency in a multi-user three-dimensional rendering process.

Description

Technical Field

[0001] This invention relates to the fields of 3D rendering, edge computing, and real-time interactive video streaming technology, specifically a dynamic task configuration method for highly consistent 3D rendering. Background Technology

[0002] With the development of cloud computing, edge computing, 3D graphics rendering, extended reality, and multi-party real-time interactive applications, 3D rendering tasks are gradually shifting from being executed on local terminals to being executed by edge servers or remote computing nodes. Terminal devices continuously receive remotely rendered 3D video frames via the network and decode and display them locally, thereby reducing the terminal's computing load and improving the real-time rendering capability of complex 3D scenes.

[0003] Existing rendering technologies primarily focus on improving viewing quality for individual users, reducing average latency, or optimizing bandwidth utilization. Examples include cloud-based or edge-based remote rendering solutions, adaptive transmission solutions that dynamically adjust video bitrate based on network bandwidth, adaptive streaming media solutions that adjust video resolution based on terminal performance or network conditions, and edge scheduling methods for multi-user real-time video streams. These typically reduce system latency or improve user experience by selecting service nodes, allocating computing resources, and adjusting bitrate or resolution. However, they pay insufficient attention to latency consistency among multiple users. In multi-user interactive 3D rendering scenarios, such as collaborative design, cloud gaming, virtual conferencing, remote simulation, digital twins, and immersive interactive applications, different users typically need to achieve the most synchronized, stable, and consistent visual presentation within the same virtual scene. This necessitates not only ensuring that the end-to-end latency for a single user meets real-time interaction requirements, but also addressing the differences in network round-trip latency, available edge bandwidth, terminal decoding capabilities, and fixed service entity rendering processing capabilities among different users. This leads to inconsistent total end-to-end latency, asynchronous visual presentation, and a degraded interactive experience for each user. Summary of the Invention

[0004] The purpose of this invention is to provide a dynamic task configuration method for highly consistent 3D rendering. By dynamically and jointly configuring resolution and bitrate, it can ensure end-to-end latency constraints while taking into account the quality of 3D rendered video, real-time performance, and multi-user latency consistency.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A dynamic task configuration method for high-consistency 3D rendering includes the following steps: Get the current time slot The network round-trip latency, available edge bandwidth limit, and terminal decoding latency for each user are recorded, and pre-fitted parameters corresponding to each fixed service entity are read, wherein the service entity is a pre-determined rendering execution node; Using the user's resolution and bitrate in the current time slot as decision variables, construct the end-to-end total rendering latency and user experience function; Based on the total end-to-end rendering latency and user experience function, a strong continuous consistency optimization objective is constructed and constraints are determined. The strong continuous consistency optimization objective is decomposed into each time slot to obtain the single time slot optimization objective and single time slot constraints. By introducing an intermediate variable mechanism, the latency consistency term in the single-slot optimization objective is decoupled. Furthermore, candidate resolutions and bitrates are combined to form a searchable set of joint configuration groups. The joint configuration results and intermediate variables are solved alternately until iterative convergence, yielding the current slot. Optimal joint configuration result The optimal joint configuration result This serves as the result of the rendering parameter configuration for the current time slot; Before the next time slot begins, each terminal will continuously obtain rendered video frames at the corresponding resolution and bitrate from the corresponding service entity.

[0006] Among them, the optimal joint configuration result The resolution and bitrate configuration corresponding to each user are used as the final rendering parameter configuration decision for the current time slot, and are used to guide each terminal to obtain the corresponding 3D rendering video frames at the corresponding resolution and bitrate from their respective fixed service entities before the start of the next time slot.

[0007] According to the above technical solution, the user experience function : ; In the formula, Indicates in Time slot terminal The resolution of the rendering task. Indicates in Time slot terminal The bitrate of the rendering task; The total end-to-end rendering latency : ; In the formula, and For fixed service entities The linear model parameters are obtained by pre-fitting the device performance. For example, for the rendering device Nvidia Jetson TX2, offline fitting can yield its... It is 0.000007. It is -0.78. The effective transmission coefficient corresponds to the rendering frame rate. This is a constant for the group length, typically 1KB; For users The corresponding network round-trip latency, This represents the maximum available edge bandwidth. This refers to the terminal decoding latency.

[0008] In 3D rendering scenarios, resolution primarily affects the rendering processing load, while bitrate primarily affects the network transmission load. Both together influence end-to-end latency and final viewing quality. Current technologies typically handle these two aspects separately, making it difficult to achieve overall optimization. The advantage of this approach is that by jointly adjusting resolution and bitrate, a fine-grained balance can be achieved between rendering latency, transmission latency, and user experience, thereby maximizing video quality while ensuring real-time performance.

[0009] Based on the above technical solution, the strong persistent consistency optimization objective is: ; In the formula, For any time slot of the application session duration, For the set of all terminals, Indicates the number of terminals; To build user experience functions, To render the total end-to-end latency, For time slots The average end-to-end total latency for rendering across all users within the system. This is the latency consistency weighting coefficient.

[0010] According to the above technical solution, the constraints of the strong persistent consistency optimization objective include: Duration of the application session Within any time slot t, the bitrate configured for each user must not exceed its available edge bandwidth limit: ; Duration of the application session For any time slot t within the timeframe, the total end-to-end latency configured for each user should meet the latency threshold constraint: ; application session duration For any time slot t within the range, the resolution and bitrate configured for each user satisfy a preset set: ; In the formula, Indicates user Configurable lower limit of bitrate, Indicates user Configurable upper limit of bitrate, This indicates the end-to-end latency threshold for the corresponding application. Indicates user Configurable lower limit of resolution. Indicates user Configurable resolution limit.

[0011] Based on the above technical solution, the single-slot optimization objective is: ; The single-slot constraint condition is as follows: ; In the formula, Indicates user In the time slot resolution, Indicates user In the time slot The bitrate configuration variable, where .

[0012] According to the above technical solution, in time slots Introducing intermediate variables And will optimize the delay consistency term in the single time slot optimization objective. Transform into Thus, the first optimization objective and the first constraint conditions are obtained; First optimization objective: ; First constraint: ; in, This represents an intermediate variable (i.e., the current time slot). (Latency consistency reference standard); Indicates user In the time slot A set of resolution and bitrate joint configurations; This indicates that all users are in the time slot. The joint configuration result, the joint configuration result It is obtained by grouping the candidate sets of resolution and bit rate using Cartesian product; This represents the set of candidate configuration groups.

[0013] For multiple users whose latency variables are coupled together, direct solutions are often computationally complex and difficult to meet online scheduling requirements. Therefore, an intermediate variable is introduced to transform the coupling consistency problem surrounding the average latency of all users into a deviation optimization problem for each user surrounding a unified intermediate variable. This significantly reduces the coupling between variables, allowing the complex problem to be broken down into more clearly structured subproblems and improving the model's solvability. It effectively reduces the solution complexity of the original optimization problem, improving online solution efficiency and convergence speed. By introducing an intermediate variable and an alternating solution strategy, this method is applicable to slot-level online scheduling scenarios, meeting the rapid decision-making requirements of real-time 3D rendering systems.

[0014] For each user, there are multiple candidate resolution and bitrate levels. Directly performing a joint search on all variables can easily lead to combinatorial explosion. This invention performs a Cartesian product on the candidate resolution set and the candidate bitrate set for each user, forming a structured joint configuration group. This facilitates unified organization and filtering of candidate configurations, provides a clear search unit for subsequent alternating solutions, reduces implementation complexity, and improves online scheduling efficiency.

[0015] Among them, the joint configuration result The resolution and bitrate candidate sets are grouped by Cartesian product, specifically as follows: For each user In the time slot resolution candidate set and bitrate candidate set Construct a set of candidate configuration groups ; in, Each element in is denoted as , indicating user In the time slot A set of resolution and bitrate joint configurations; All users in time slots The joint configuration result is denoted as .

[0016] Based on the above technical solution, the joint configuration result and the intermediate variables The steps for performing alternating solutions include: In the In this iteration, the intermediate variables are fixed. For each joint configuration result in the candidate joint configuration result set The process involves iterating through the configurations, calculating the joint configuration objective function value for each configuration result, and selecting the configuration result that satisfies the joint configuration constraints and maximizes the joint configuration objective function value as the optimal joint configuration result. ; Obtain the optimal joint configuration result Then, fix the optimal joint configuration result. Solve for the optimal value of the intermediate variable to obtain ; For the joint configuration results and intermediate variables Repeat the above alternating update process until the preset stopping condition is met, and then output the current time slot. Optimal joint configuration result .

[0017] Alternating the solution of joint configuration groups and intermediate variables can improve convergence speed and online solution efficiency while ensuring solution accuracy, making it more suitable for the real-time scheduling requirements of 3D rendering systems.

[0018] By fixing the service entity and dynamically configuring only the resolution and bitrate, this approach better aligns with the deployment constraints of real-world remote 3D rendering systems. It improves consistency and real-time performance without introducing additional service migration and state synchronization overhead, thus offering greater engineering applicability.

[0019] According to the above technical solution, the joint configuration objective function is: ; The joint configuration constraints are as follows: .

[0020] According to the above technical solution, when the optimal joint configuration result is fixed... At that time, the intermediate variable The update is based on the average total end-to-end latency of each user terminal, and the update method is as follows: ; In the formula, Indicates the updated user In the time slot A set of resolution and bitrate joint configurations.

[0021] According to the above technical solution, the preset stopping condition in the alternating solution process is at least one of the following: the change in the objective function between two adjacent iterations is less than a preset threshold, the change in the intermediate variable between two adjacent iterations is less than a preset threshold, the joint configuration result between two adjacent iterations no longer changes, and the number of iterations reaches a preset upper limit.

[0022] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: While balancing user experience and multi-user latency consistency, this invention decouples the joint configuration problem of resolution and bitrate by introducing an intermediate variable mechanism, and combines candidate resolutions and bitrates to form a searchable set of joint configurations. Finally, the system adopts an alternating solution method, traversing the set of joint configurations to obtain the optimal configuration when the intermediate variable is fixed, and directly calculating the optimal value of the intermediate variable after fixing the configuration. After iteration until convergence, the rendering parameter configuration result for the current time slot is output. This invention can significantly improve end-to-end latency consistency in multi-user 3D rendering. The image presentation time of different terminal devices in the same interactive scene is closer, thereby reducing phenomena such as image asynchrony and inconsistent interactive feedback. Comprehensive optimization of resolution and bitrate under latency threshold constraints can improve the overall user experience quality while ensuring real-time performance, balancing image quality and interactive performance. This invention can operate stably under actual deployment conditions with fixed service entities, and has strong engineering feasibility and application value. Consistency optimization can be completed without relying on service entity switching, making it more suitable for scenarios such as multi-user interactive 3D rendering, cloud gaming, virtual reality, digital twins, and remote collaborative design. It can also improve the overall stability of the system and its ability to ensure continuous consistency. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the dynamic task configuration method for high-consistency 3D rendering according to the present invention; Figure 2 It is a monitoring chart of terminal consistency levels over 500 time slots. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] In multi-user remote 3D rendering scenarios, the dynamic task configuration method for high-consistency 3D rendering of this invention is applied to dynamically and jointly configure resolution and bitrate. While ensuring end-to-end latency constraints, it also takes into account 3D rendering video quality, real-time performance, and multi-user latency consistency. Specific steps (…) Figure 1 )include: S1. At the beginning of the current time slot, each terminal device continuously obtains and presents 3D rendered video frames from its fixed corresponding service entity according to the rendering parameter configuration results determined in the previous time slot. The service entity is a pre-determined rendering execution node and is not used as a decision variable in the scheduling process.

[0026] Obtain the network round-trip latency, available edge bandwidth limit, and terminal decoding latency for each user in the current time slot, and read the pre-fitted parameters corresponding to each fixed service entity. The user set is denoted as... The user index is denoted as Each user The corresponding fixed service entity is denoted as .

[0027] S2. Based on the obtained network round-trip latency, available edge bandwidth limit, terminal decoding latency, and pre-fitted parameters corresponding to each fixed service entity, the user's resolution and bitrate in the current time slot are used as decision variables to construct the rendering end-to-end total latency and user experience function respectively.

[0028] User experience function : ; In the formula, Indicates in Time slot terminal The resolution of the rendering task. Indicates in Time slot terminal The bitrate of the rendering task.

[0029] Total end-to-end rendering latency : ; In the formula, and For fixed service entities The linear model parameters are obtained by pre-fitting the device performance. For example, for the rendering device Nvidia Jetson TX2, offline fitting can yield its... It is 0.000007. It is -0.78. The effective transmission coefficient corresponds to the rendering frame rate. This is a constant for the group length, typically 1KB; For users The corresponding network round-trip latency, This represents the maximum available edge bandwidth. This refers to the terminal decoding latency.

[0030] S3. Construct a strong, continuous consistency optimization objective and determine the constraints based on the total end-to-end rendering latency and user experience function.

[0031] Strong persistence consistency optimization objective: ; In the formula, For any time slot of the application session duration, For the set of all terminals, Indicates the number of terminals; To build user experience functions, To render the total end-to-end latency, For time slots The average of the total end-to-end latency for all users within the region. This is the latency consistency weighting coefficient.

[0032] The constraints of the strong continuous consistency optimization objective include: Duration of the application session Within any time slot t, the bitrate configured for each user must not exceed its available edge bandwidth limit: ; Duration of the application session For any time slot t within the timeframe, the total end-to-end latency configured for each user should meet the latency threshold constraint: ; application session duration For any time slot t within the range, the resolution and bitrate configured for each user satisfy a preset set: ; In the formula, Indicates user Configurable lower limit of bitrate, Indicates user Configurable upper limit of bitrate, This represents the end-to-end latency threshold for the corresponding application (which can also be understood as the latency threshold from the application end to the rendering end). Indicates user Configurable lower limit of resolution. Indicates user Configurable resolution limit.

[0033] The strong continuous consistency optimization objective is decomposed into each time slot to obtain the single time slot optimization objective and single time slot constraints.

[0034] Single time slot optimization objective: ; Single time slot constraint: ; In the formula, Indicates user In the time slot resolution, Indicates user In the time slot The bitrate configuration variable.

[0035] S4, in time slot Introducing intermediate variables And will optimize the delay consistency term in the single time slot optimization objective. Transform into Thus, the first optimization objective and the first constraint conditions are obtained; First optimization objective: ; First constraint: ; in, Indicates the current time slot Latency consistency reference standard; Indicates user In the time slot A set of resolution and bitrate joint configurations; This indicates that all users are in the time slot. The joint configuration result, the joint configuration result It is obtained by grouping the candidate sets of resolution and bit rate using Cartesian product; Show the set of candidate configuration groups.

[0036] For each user Time slot resolution candidate set and bitrate candidate set Construct a set of candidate configuration groups ; will allow all users to be in time slots The joint configuration result is denoted as In the In this iteration, the intermediate variables are fixed, and each joint configuration result in the candidate joint configuration result set is processed. The process involves iterating through the joint configurations, calculating the joint configuration objective function value for each joint configuration result, and selecting the joint configuration result that satisfies the joint configuration constraints and maximizes the joint configuration objective function value as the optimal joint configuration result. The joint configuration objective function is as follows: ; The joint configuration constraints are as follows: ; In the formula, Indicates user In the time slot The bitrate configuration variable.

[0037] Obtain the optimal joint configuration result Then, fix the optimal joint configuration result. Solve for the optimal value of the intermediate variable to obtain In the formula, Indicates the updated user In the time slot A set of resolution and bitrate joint configurations.

[0038] For the joint configuration results and intermediate variables Repeat the alternating update process described above until at least one of the following preset stopping conditions is met: the change in the objective function between two adjacent iterations is less than a preset threshold; the change in the intermediate variable between two adjacent iterations is less than a preset threshold; the joint configuration result between two adjacent iterations no longer changes; and the number of iterations reaches a preset upper limit. Output the current time slot. Optimal joint configuration result .

[0039] The above scheme is applied frame by frame to continuously adjust the resolution and bitrate dynamically, and the terminal consistency level is stably controlled within 140 over 500 time slots. Figure 2 This value can adequately meet the consistency level among users, ensuring a consistent experience for all users.

[0040] S5, Optimal Joint Configuration Result The resolution and bitrate configuration corresponding to each user are used as the final rendering parameter configuration decision for the current time slot, and are used to guide each terminal to obtain the corresponding 3D rendering video frames at the corresponding resolution and bitrate from their respective fixed service entities before the start of the next time slot.

[0041] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0042] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic task configuration method for high-consistency 3D rendering, characterized in that, The steps include: Get the current time slot The network round-trip latency, available edge bandwidth limit, and terminal decoding latency for each user are recorded, and pre-fitted parameters corresponding to each fixed service entity are read, wherein the service entity is a pre-determined rendering execution node; Using the user's resolution and bitrate in the current time slot as decision variables, construct rendering end-to-end total latency and user experience functions respectively; Based on the total end-to-end rendering latency and user experience function, construct a strong continuous consistency optimization objective and its constraints. Decompose the strong continuous consistency optimization objective into each time slot to obtain the single time slot optimization objective and single time slot constraints. By introducing an intermediate variable mechanism, the latency consistency term in the single-slot optimization objective is decoupled. Furthermore, candidate resolutions and bitrates are combined to form a searchable set of joint configuration groups. The joint configuration results and intermediate variables are solved alternately until iterative convergence, yielding the current slot. Optimal joint configuration result The optimal joint configuration result This serves as the result of the rendering parameter configuration for the current time slot; Before the next time slot begins, each terminal will continuously obtain rendered video frames at the corresponding resolution and bitrate from the corresponding service entity.

2. The dynamic task configuration method for high-consistency 3D rendering according to claim 1, characterized in that, The user experience function : ； In the formula, Indicates in Time slot terminal The resolution of the rendering task. Indicates in Time slot terminal The bitrate of the rendering task; The end-to-end total delay : ； In the formula, and Representing fixed service entities The corresponding prefit parameters, The group length is a constant. For effective transmission coefficient, For users The corresponding network round-trip latency, This represents the maximum available edge bandwidth. This refers to the terminal decoding latency.

3. The dynamic task configuration method for high-consistency 3D rendering according to claim 1, characterized in that, The objective of the strong persistent consistency optimization is: ； In the formula, For any time slot of the application session duration, For the set of all terminals, Indicates the number of terminals; To build user experience functions, To render the total end-to-end latency, For time slots The average end-to-end total latency for rendering across all users within the system. This is the latency consistency weighting coefficient.

4. The dynamic task configuration method for high-consistency 3D rendering according to claim 1, characterized in that, The constraints of the strong persistent consistency optimization objective include: Duration of the application session Within any time slot t, the bitrate configured for each user must not exceed its available edge bandwidth limit: ; Duration of the application session For any time slot t within the timeframe, the total end-to-end latency configured for each user should meet the latency threshold constraint: ; application session duration For any time slot t within the range, the resolution and bitrate configured for each user satisfy a preset set: ; In the formula, Indicates user Configurable lower limit of bitrate, Indicates user Configurable upper limit of bitrate, This indicates the end-to-end latency threshold for the corresponding application. Indicates user Configurable lower limit of resolution. Indicates user Configurable resolution limit.

5. The dynamic task configuration method for high-consistency 3D rendering according to claim 1, characterized in that, The single-slot optimization objective is: ； The single-slot constraint condition is as follows: ； In the formula, Indicates user In the time slot resolution, Indicates user In the time slot The bitrate configuration variable.

6. The dynamic task configuration method for high-consistency 3D rendering according to claim 1, characterized in that, In the time slot Introducing intermediate variables The delay consistency term in the single-slot optimization objective will be... Transform into Thus, the first optimization objective and the first constraint conditions are obtained; First optimization objective: ； First constraint: ; in, Indicates intermediate variables; Indicates user In the time slot A set of resolution and bitrate joint configurations; This indicates that all users are in the time slot. The joint configuration result, the joint configuration result It is obtained by grouping the candidate sets of resolution and bit rate using Cartesian product; This represents the set of candidate configuration groups.

7. The dynamic task configuration method for high-consistency 3D rendering according to claim 1, characterized in that, Regarding the joint configuration result and the intermediate variables The steps for performing alternating solutions include: In the In this iteration, the intermediate variables are fixed. For each joint configuration result in the candidate joint configuration result set The process involves iterating through the configurations, calculating the joint configuration objective function value for each configuration result, and selecting the configuration result that satisfies the joint configuration constraints and maximizes the joint configuration objective function value as the optimal joint configuration result. ; Obtain the optimal joint configuration result Then, fix the optimal joint configuration result. Solve for the optimal value of the intermediate variable to obtain ; For the joint configuration results Repeat the alternating update process described above with intermediate variables until the preset stopping condition is met, and then output the current time slot. Optimal joint configuration result .

8. The dynamic task configuration method for high-consistency 3D rendering according to claim 7, characterized in that, The joint configuration objective function: ； The joint configuration constraints are as follows: .

9. The dynamic task configuration method for high-consistency 3D rendering according to claim 7, characterized in that, When the optimal joint configuration result is fixed At that time, the intermediate variable The update is based on the average total end-to-end latency of each user terminal, and the update method is as follows: ； In the formula, Indicates the updated user In the time slot A set of resolution and bitrate joint configurations.

10. The dynamic task configuration method for high-consistency 3D rendering according to claim 7, characterized in that, The preset stopping conditions in the alternating solution process include at least one of the following: the change in the objective function between two adjacent iterations is less than a preset threshold, the change in the intermediate variable between two adjacent iterations is less than a preset threshold, the joint configuration result between two adjacent iterations no longer changes, and the number of iterations reaches a preset upper limit.

Images