Video imaging adaptive three-dimensional scene synchronous rendering method and system

Abstract

The invention discloses a video imaging adaptive three-dimensional scene synchronous rendering method and system, and relates to the technical field of 3D rendering, and the method comprises the steps: obtaining a rendering performance index of a video, and carrying out the preprocessing of the rendering performance index, and obtaining a rendering performance fluctuation index; dividing a current video scene into a plurality of resource regions, and determining the proportion of a high-density region and the proportion of a detail dense region; detail data are extracted from the synchronous rendering pipeline, and a material transition zone width interval and boundary adjustment parameters are determined; dynamically adjusting the computing resources by adopting a load balanced distribution method; and obtaining the original detail level and texture sampling density in the resource region, performing adaptive adjustment to obtain the optimized detail level and texture sampling density, and generating a balanced image output sequence. According to the method, the stability of the rendering frame rate in the complex dynamic virtual environment, the real consistency of picture details and the overall resource utilization efficiency are improved, and the rendering quality is improved.

Description

Technical Field

[0001] This invention relates to the field of 3D rendering technology, and in particular to a method and system for adaptive 3D scene synchronous rendering using video imaging. Background Technology

[0002] 3D rendering technology is generally achieved through the conversion from two-dimensional (2D) to three-dimensional (3D), and can typically include hardware-dependent solutions and purely software-based rendering solutions. Common hardware-dependent solutions include 3D raster screens, frequency-division screens, and 3D glasses, while purely software-based rendering solutions include depth maps and rangefinder projection.

[0003] In the fields of modern digital media and virtual reality, 3D rendering technology is the cornerstone of building immersive experiences, and its importance is self-evident. With the ever-increasing demands for video imaging, achieving high-quality visual presentation in dynamic environments has become a core driving force for industry development. This technology not only affects the user's intuitive experience but also directly relates to the rational allocation of computing resources and the stability of system operation. However, existing 3D rendering methods often reveal significant shortcomings when dealing with complex scenes. Many solutions struggle to find a balance between rendering quality and system performance, especially when facing large-scale data processing, easily leading to uneven resource allocation or decreased response speed. This limitation is not simply a technical shortcoming but stems from a lack of real-time perception and dynamic adjustment capabilities for data fluctuations during 3D rendering, preventing the system from flexibly responding to actual needs. A deeper technical challenge lies in the real-time monitoring of 3D rendering data volume and its matching with scene complexity. As a key factor affecting system load, fluctuations in 3D rendering data volume directly lead to the strain or waste of computing resources. If changes in data volume cannot be detected in a timely manner, it is impossible to rationally adjust the level of detail in the scene. For example, during video imaging, when a scene contains a large number of objects that need to be rendered simultaneously, the system may be unable to maintain smooth image output due to a sudden increase in data volume, or even experience stuttering or image distortion; this imbalance in resource allocation caused by fluctuations in data volume further exacerbates the contradiction between rendering quality and performance.

[0004] Therefore, how to balance image quality and system performance by monitoring the amount of rendering data in real time and dynamically adjusting the level of scene detail during video imaging has become a critical issue that urgently needs to be addressed. Solving this problem not only affects the improvement of user experience but also directly impacts the application prospects of 3D rendering technology in highly dynamic environments. Summary of the Invention

[0005] To address the problem of low rendering quality caused by the inability of existing technologies to adaptively adjust based on data volume, this invention proposes a video imaging adaptive 3D scene synchronous rendering method and system. By monitoring the amount of rendering data in real time and dynamically adjusting the level of scene detail, the method ensures a balance between image quality and system performance, thereby improving the quality of 3D rendering.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A video imaging adaptive 3D scene synchronous rendering method includes the following steps:

[0008] S1: Obtain the rendering performance metrics of the video and preprocess the rendering performance metrics to obtain the rendering performance fluctuation metrics; the rendering performance fluctuation metrics include the level of variation of rendering time variance and the level of variation of load peak.

[0009] S2: Use an adaptive threshold comparison method to judge whether the rendering performance fluctuation index is abnormal; if so, divide the current video scene into multiple resource regions, calculate the region density and high-frequency texture ratio of each resource region, and thus determine the proportion of high-density regions and the proportion of detailed regions; otherwise, return to S1.

[0010] S3: Based on the proportion of high-density areas and the proportion of detail-intensive areas, extract detail data of high-density areas and detail-intensive areas from the synchronous rendering pipeline, and perform cluster analysis on the detail data to determine the material classification, thereby determining the material transition band width range and boundary adjustment parameters;

[0011] S4: Based on the width range of the material transition zone and the boundary adjustment parameters, the load balancing allocation method is used to dynamically adjust the computing resources, and the information fusion of the resource occupancy ratio of each resource area is carried out in the dynamic adjustment to obtain the adjusted computing allocation strategy.

[0012] S5: Obtain the original detail level and texture sampling density in the resource area according to the adjusted calculation allocation strategy, and then adaptively adjust the original detail level and texture sampling density to obtain the optimized detail level and texture sampling density, generating a balanced image output sequence.

[0013] Preferably, it further includes:

[0014] S6: Perform continuous stability verification processing on the balanced image output sequence in the video imaging loop; if the cumulative deviation between the screen space projection distortion degree and the historical fluctuation trend exceeds the preset distortion range, re-acquire the updated indicators of the duration of the sudden load peak and the regional distribution imbalance, adjust the resource allocation strategy of the rendering task until the system returns to stability.

[0015] S7: Perform multiple rounds of iterative adjustments to the virtual environment rendering process based on the updated metrics, and merge all adjustment results to generate a stable image sequence.

[0016] Preferably, S1 includes:

[0017] S1-1: Obtain the rendering performance metrics of the current video in the virtual environment through the data acquisition module, including rendering time per frame, peak GPU utilization, and vertex shader execution cycles.

[0018] S1-2: Calculate the variance of the rendering time for each frame in the rendering performance metrics to obtain the rendering time variance sequence;

[0019] S1-3: Calculate the standard deviation of the rendering time variance sequence to determine the level of variation in rendering time variance;

[0020] S1-4: Obtain the GPU peak utilization sequence based on the rendering performance metrics, then perform weighted processing on the GPU peak utilization sequence to obtain a smooth peak trend sequence, and determine the load peak change level.

[0021] Preferably, S2 includes:

[0022] S2-1: Use an adaptive threshold comparison method to determine whether the rendering performance fluctuation index exceeds the preset threshold range. If not, return to S1; otherwise, proceed to S2-2.

[0023] S2-2: The perception module collects spatial geometric distribution information of the current video scene from the virtual environment, thereby dividing it into multiple resource regions, calculating the regional density and high-frequency texture ratio of each resource region, and thus determining the proportion of high-density regions and the proportion of detail-dense regions.

[0024] Preferably, S3 includes:

[0025] S3-1: Extract detail data of high-density and detail-intensive regions from the synchronous rendering pipeline using the rendering engine's frame buffer, and then extract detail feature points from the detail data; the detail data includes depth variation and normal variation rate;

[0026] S3-2: Classify detailed feature points to obtain different material regions and determine the width range of the material transition zone;

[0027] S3-3: Based on the edge detection algorithm, the boundaries of dynamic objects in high-density and detail-rich areas are adjusted. The boundary adjustment parameters include sharpness and error control rate.

[0028] Preferably, S3-2 includes:

[0029] S3-2-1: Use a convolutional neural network to classify the extracted detail feature points to obtain the material labels of the detail feature points;

[0030] S3-2-2: Spatial clustering of detailed feature points belonging to the same material label to form different material regions;

[0031] S3-2-3: By calculating the minimum distance between the boundaries of adjacent material regions, a line segment is sampled in the direction of the minimum distance. The probability change of detailed feature points is analyzed on the line segment, and the interval of probability change within the preset probability range is marked as the width interval of the material transition zone.

[0032] Preferably, S4 includes:

[0033] S4-1: Determine the computational resource W for each resource region based on the material transition zone width range and boundary adjustment parameters.

[0034] W = 0.6 × transition band width coefficient + 0.3 × boundary sharpness coefficient + 0.1 × motion speed factor; where the motion speed factor is linearly mapped per pixel / frame;

[0035] Based on the normal angle, the material transition band width is divided into three levels: when the normal angle is less than 15 degrees, it is set to the first level, with a transition band width coefficient of 0.8~1.2 pixels; when the normal angle is between 15 and 35 degrees, it is set to the second level, with a transition band width coefficient of 2.5~4.0 pixels; when the normal angle exceeds 35 degrees, it is set to the third level, with a transition band width coefficient of 0.4~0.8 pixels.

[0036] The boundary sharpness score S is calculated based on the boundary adjustment parameters. If S ≥ 0.85, the boundary sharpness coefficient is 1.0; if 0.65 ≤ S < 0.85, the boundary sharpness coefficient is 0.75; if S < 0.65, the boundary sharpness coefficient is 0.45.

[0037] S4-2: The load balancing method is used to dynamically adjust the computing resources to obtain the optimized resource utilization ratio;

[0038] S4-3: Based on the optimized resource utilization ratio, information fusion is performed to obtain the adjusted computational allocation strategy:

[0039]

[0040] in, This represents the adjusted computational allocation strategy at time t. Indicates the smoothing factor; This represents the adjusted computational allocation strategy at time t-1.

[0041] Preferably, S5 includes:

[0042] S5-1: Obtain the original detail level and texture sampling density according to the adjusted calculation allocation strategy;

[0043] S5-2: Adjust the detail level threshold of the original detail level to obtain the optimized detail level;

[0044] S5-3: Adaptively adjust the original texture sampling density to obtain the optimized texture sampling density;

[0045] S5-4: Generate preliminary image output sequence data based on the optimized detail level and optimized texture sampling density;

[0046] S5-5: Detect whether there is quality inhomogeneity in the initial image output sequence data. If so, perform secondary rendering on the quality inhomogeneity part and output a balanced image output sequence; otherwise, directly output a balanced image output sequence.

[0047] Preferably, S6 includes:

[0048] S6-1: Extract the projection coordinate information of each frame in the balanced image output sequence in screen space. By comparing the coordinate offset with the standard template, determine whether projection distortion exists. If yes, proceed to S6-2; otherwise, it indicates that the system is stable and subsequent processing stops.

[0049] S6-2: Calculate the cumulative deviation of historical fluctuation trends and compare it with the preset distortion range; when the cumulative deviation of historical fluctuation trends exceeds the preset distortion range, it is determined that it has deviated from the long-term stable state and proceeds to S6-3; when the cumulative deviation of historical fluctuation trends is within the preset distortion range, it is determined that it is in short-term distortion and continues the distortion judgment of the next window.

[0050] S6-3: Obtain updated metrics including the duration of the burst load peak and the regional distribution imbalance, and identify resource regions with a burst load peak duration greater than a preset duration and resource regions with a load imbalance less than a preset entropy value as priority resource regions.

[0051] S6-4: Adjust the computational resources of the rendering task for the priority resource area to obtain optimized rendering performance indicators;

[0052] S6-5: Regenerate the balanced image output sequence based on the optimized rendering performance metrics and perform stability verification until the system returns to a stable state.

[0053] The present invention also provides a video imaging adaptive 3D scene synchronous rendering system, comprising:

[0054] The data acquisition module is used to obtain the rendering performance metrics of the current video, including rendering time per frame, peak GPU utilization, and vertex shader execution cycles.

[0055] The preprocessing module is used to calculate the variance change and load peak change of rendering performance indicators to obtain rendering performance fluctuation indicators, including the level of variation of rendering time variance and the level of variation of load peak.

[0056] The perception module is used to collect spatial geometric distribution information of the current video scene based on rendering performance fluctuation indicators, and to determine the proportion of high-density areas and the proportion of detailed areas.

[0057] The detailed data analysis module is used to extract detailed data of the corresponding area based on the proportion of high-density areas and the proportion of detailed dense areas, and to determine the width range of the material transition zone and boundary adjustment parameters.

[0058] The resource allocation optimization module is used to dynamically adjust computing resources based on the width range of the material transition zone and boundary adjustment parameters to obtain the adjusted computing allocation strategy;

[0059] The adaptive optimization decision module is used to obtain the original detail level and texture sampling density in the resource area according to the adjusted calculation allocation strategy, and to adaptively adjust them to obtain the optimized detail level and texture sampling density.

[0060] The image output module is used to output a balanced image output sequence based on the optimized detail level and texture sampling density.

[0061] In summary, by adopting the above technical solution, the present invention has at least the following beneficial effects compared with the prior art:

[0062] This invention addresses the complex business issues in virtual scenes caused by dynamic changes in environmental complexity, including drastic fluctuations in rendering performance, local resource overload, and distortion and instability of image details. It collects rendering performance fluctuation indicators in real time and extracts load peaks and inter-frame variance changes. It intelligently senses and quantifies the geometric complexity block density and high-frequency detail ratio in resource-intensive areas, thereby generating an optimized environmental detail map. A load balancing strategy is employed to selectively reallocate computing resources and dynamically modify detail level thresholds and texture sampling density, ultimately outputting a balanced image sequence.

[0063] Meanwhile, through continuous stability verification and backtracking iteration mechanism, closed-loop correction of image distortion and historical fluctuation deviation is achieved, which significantly improves the stability of rendering frame rate, the real consistency of image details and the overall resource utilization efficiency in complex dynamic virtual environment, and effectively solves the problems of stuttering, blurring and quality jitter in high load scenarios. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of a video imaging adaptive three-dimensional scene synchronous rendering method according to an exemplary embodiment 1 of the present invention.

[0065] Figure 2 This is a schematic diagram of the preprocessing flow according to an exemplary embodiment 1 of the present invention.

[0066] Figure 3 A schematic diagram illustrating the process of determining the proportion of high-density regions and the proportion of detailed regions according to an exemplary embodiment 1 of the present invention.

[0067] Figure 4 This is a schematic diagram illustrating the process of determining the width range and boundary adjustment parameters of the material transition zone according to an exemplary embodiment 1 of the present invention.

[0068] Figure 5 This is a schematic diagram illustrating the adjustment of the computational allocation strategy according to an exemplary embodiment 1 of the present invention.

[0069] Figure 6 This is a schematic diagram of the balanced image output sequence generation process according to an exemplary embodiment 1 of the present invention.

[0070] Figure 7 This is a schematic diagram of the balanced image output sequence stability verification and adjustment process according to an exemplary embodiment 1 of the present invention.

[0071] Figure 8 This is a schematic diagram of a video imaging adaptive three-dimensional scene synchronous rendering system according to an exemplary embodiment 2 of the present invention. Detailed Implementation

[0072] The present invention will be further described in detail below with reference to embodiments and specific implementation methods. However, this should not be construed as limiting the scope of the above-described subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0073] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0074] Example 1

[0075] like Figure 1As shown, this invention provides a video imaging adaptive 3D scene synchronous rendering method, including the following steps:

[0076] S1: Obtain the rendering performance metrics of the video and preprocess the rendering performance metrics to obtain the rendering performance fluctuation metrics; the rendering performance metrics include frame rate, number of frames, rendering time, GPU utilization, number of vertex shader execution cycles, etc.; the rendering performance fluctuation metrics include the level of variation of rendering time variance and the level of variation of load peak.

[0077] Specifically, such as Figure 2 As shown, S1 includes the following steps:

[0078] S1-1: The data acquisition module obtains the current video rendering performance metrics, including rendering time per frame, peak GPU utilization, vertex shader execution cycles, etc., at a frequency of 60 frames per second in the virtual environment through graphics APIs (such as Vulkan's VK_EXT_performance_query extension or DirectX's GPU performance counters).

[0079] S1-2: Calculate the variance of the rendering time for each frame in the rendering performance metrics to obtain the rendering time variance sequence.

[0080] In this embodiment, a dynamic continuous sampling method (e.g., a sliding window method) can be used to obtain the rendering time within the i-th window (the most recent 120 frames, approximately 2 seconds), thus obtaining the rendering time sequence of the i-th window. , This represents the rendering time for the nth frame.

[0081] The variance of the rendering time sequence for the i-th window is:

[0082]

[0083] In formula (1), This represents the variance of the rendering time sequence for the i-th window; This represents the rendering time of the nth frame, in milliseconds. This represents the average rendering time of the i-th window; This indicates the total number of frames, for example, 120.

[0084] Next, the variance of the rendering time sequence for multiple windows is calculated to obtain the rendering time variance sequence. , This represents the variance of the rendering time sequence for the i-th window.

[0085] For example, rendering performance metrics can be collected for 10 windows (approximately 20 seconds), and then variance can be calculated to obtain the variance order. .

[0086] This embodiment also includes real-time updates to the variance of the rendering time sequence. When the window slides forward one frame, the rendering time of the (n+1)th frame will be introduced. Therefore, the variance of the rendering time sequence needs to be updated in real time to achieve efficient refresh:

[0087]

[0088] In formula (2), This represents the updated real-time variance; This represents the variance of the rendering time sequence for the i-th window; This represents the rendering time of the (n+1)th frame, in milliseconds. This represents the average rendering time of the i-th window, in milliseconds; This represents the rendering time of the nth frame, in milliseconds. This indicates the mean adjustment term.

[0089] S1-3: Calculate the standard deviation of the rendering time variance sequence to determine the level of variation in rendering time variance.

[0090] In this embodiment, the formula for calculating the standard deviation of the rendering time variance sequence is:

[0091]

[0092] In formula (3), This represents the standard deviation of the rendering time variance sequence; Indicates the number of capture windows; This represents the variance of the rendering time sequence for the i-th window; This represents the mean variance of the rendering time variance sequence.

[0093] In this embodiment, the standard deviation of the rendering time variance sequence is used as the first dimension of the rendering performance fluctuation index, i.e., the level of change in rendering time variance:

[0094] when The real-time variance is greater than a preset first threshold (e.g., 0.8). When the variance exceeds the preset second threshold (1.6 times the average variance of the first 30 seconds), the rendering time variance change level is determined to be the first fluctuation state (high fluctuation state); when Less than or equal to a preset first threshold (e.g., 0.8) or real-time variance When the value is less than or equal to the preset second threshold (1.6 times the average variance of the first 30 seconds), the level of change in rendering time variance is determined to be the second fluctuation state (low fluctuation state).

[0095] S1-4: Obtain the GPU peak utilization sequence based on the rendering performance metrics, then perform weighted processing on the GPU peak utilization sequence to obtain a smooth peak trend sequence, and determine the load peak change level.

[0096] In this embodiment, the peak GPU utilization rate for each frame is calculated within the same window (the i-th window, which is the same as the window that takes the rendering time). By statistically analyzing values ​​ranging from 0 to 100, a GPU peak utilization sequence can be obtained. Then, the exponentially weighted moving average (EWMA) method is used. =0.15, (Representing the smoothing factor) to obtain a smoothed peak trend sequence .

[0097] The calculation method for the load peak variation level is as follows:

[0098]

[0099] In formula (4), Indicates the level of load peak variation; Represents the floor function; This represents a sequence of GPU peak utilization rates; This indicates a smoothed peak trend sequence.

[0100] For example, the load peak change levels include -3, -2, -1, 0, +1, +2, +3, and +4, a total of 8 levels. Among them, +3 and above (including +3) indicates a sudden and severe peak surge, resulting in increased load; -2 and below (including -2) indicates fewer peaks, resulting in decreased load.

[0101] In this embodiment, the level of variation of rendering time variance and the level of variation of load peak are synchronized and aligned in time to form a complete rendering performance fluctuation index, which is then output to the subsequent adaptive optimization decision module in real time.

[0102] S2: Use an adaptive threshold comparison method to judge whether the rendering performance fluctuation index is abnormal; if so, divide the current video scene into multiple resource regions, calculate the region density and high-frequency texture ratio of each resource region, and thus determine the proportion of high-density regions and the proportion of detailed regions; otherwise, return to S1.

[0103] Specifically, such as Figure 3 As shown, step S2 includes the following steps:

[0104] S2-1: Use an adaptive threshold comparison method to determine whether the rendering performance fluctuation index exceeds the preset threshold range. If not, return to S1; otherwise, proceed to S2-2.

[0105] In this embodiment, the load peak change levels include -3, -2, -1, 0, +1, +2, +3, and +4, a total of 8 levels, and the threshold can be set to +2.5; the preset threshold for the rendering time variance change level can be set to 1.2 times the historical average (i.e., the historical average of the standard deviation of the rendering time variance sequence).

[0106] An abnormal state is identified when the load peak change level exceeds +2.5 or the rendering time variance change level exceeds 1.2 times the historical average (i.e., the historical average of the standard deviation of the rendering time variance series). For example, if the current load peak change level is +3 and the rendering time variance change level is 1.5 times the historical average, it is identified as an abnormal state, and the system will automatically trigger subsequent processing procedures.

[0107] S2-2: The perception module collects spatial geometric distribution information of the current video scene from the virtual environment, thereby dividing it into multiple resource regions, calculating the regional density and high-frequency texture ratio of each resource region, and thus determining the proportion of high-density regions and the proportion of detail-dense regions.

[0108] In this embodiment, the current video scene can be divided into multiple 512x512 pixel resource areas (i.e., gridded) based on the spatial geometric distribution information of the current video scene using a scene depth map analysis algorithm.

[0109] Then, the triangular facets of each resource region are counted to determine the region density of each resource region. When the region density of a resource region exceeds a preset third threshold, it is marked as a high-density region, and the proportion of high-density regions can be obtained, that is, the ratio of the number of high-density regions to the number of resource regions.

[0110] For example, if the area density of a resource region is 12,000 (i.e., it includes 12,000 triangular facets), which exceeds a preset third threshold (e.g., 8,000 triangular facets), then the resource region is marked as a high-density region.

[0111] In this embodiment, a texture frequency analysis algorithm can be used to extract high-frequency textures in each resource region (grid) to obtain the proportion of high-frequency textures. When the proportion of high-frequency textures exceeds a preset fourth threshold, the resource region is marked as a dense detail region, and the proportion of dense detail regions can be obtained, that is, the ratio of the number of dense detail regions to the number of resource regions.

[0112] For example, if the proportion of high-frequency textures in a resource area exceeds the preset fourth threshold (35%), it is marked as a densely detailed area. If the proportion of high-frequency textures in a resource area reaches 42%, it is recorded first.

[0113] S2-3: The comprehensive complexity index is obtained by weighting the proportion of high-density areas and the proportion of detailed dense areas. If the comprehensive complexity index exceeds the preset fifth threshold, subsequent optimization and adjustment will be carried out.

[0114] The overall complexity index = the proportion of high-density areas × weight 1 + the proportion of detailed dense areas × weight 2.

[0115] For example, if the proportion of high-density areas is 18%, the corresponding weight 1 is 0.6; if the proportion of detailed dense areas is 25%, the corresponding weight 2 is 0.4, then the calculated comprehensive complexity index is 20.2. When 20.2 exceeds the preset fifth threshold of 15.0, the system automatically transmits the comprehensive complexity index to the resource allocation optimization module, forming a complete logical chain from performance anomaly detection to video scene complexity analysis and then to resource adjustment suggestions, ensuring more accurate resource allocation in the virtual environment rendering process.

[0116] S3: Based on the proportion of high-density areas and the proportion of detail-intensive areas, extract detail data from the high-density and detail-intensive areas in the synchronous rendering pipeline, and perform cluster analysis on the detail data to determine material classification, thereby determining the material transition band width range and boundary adjustment parameters. Specifically, such as... Figure 4 As shown, step S3 includes the following steps:

[0117] S3-1: Use the frame buffer of the rendering engine to extract detail data of high-density regions and detail-intensive regions from the synchronous rendering pipeline, and then extract detail feature points from the detail data; the detail data includes depth variation and normal variation rate.

[0118] In this embodiment, depth information is defined as the depth value of each pixel (or cell) read from the frame buffer, representing the distance from the object to the camera. Normal data is defined as the normal vector of each pixel (or cell) read from the frame buffer, representing the orientation of the surface.

[0119] In this embodiment, detail data with depth changes exceeding a depth threshold (0.5 meters) or normal change rates exceeding a normal threshold (0.3) are marked as detail feature points. Each resource region has corresponding detail feature points, thus a set of detail feature points can be obtained.

[0120] S3-2: Classify the detailed feature points to obtain different material regions and determine the width range of the material transition zone.

[0121] S3-2-1: In this embodiment, a convolutional neural network can be used to classify the extracted detail feature points to obtain the material labels of the detail feature points.

[0122] For example, the kernel size can be set to 3x3 and the number of iterations to 100.

[0123] S3-2-2: Spatial clustering of detailed feature points belonging to the same material label to form different material regions.

[0124] In this embodiment, each detail feature point has location coordinates (x, y) and a material label output by the CNN (which may be a multi-class probability distribution). First, the detail feature points are divided according to the material label, and then spatial clustering is performed on each class of detail feature points (e.g., using DBSCAN or distance-based clustering) to obtain different material regions.

[0125] S3-2-3: Determine the width range of the material transition zone based on different material areas.

[0126] In this embodiment, firstly, the minimum distance between the boundaries of adjacent material regions can be calculated, and a line segment can be sampled in the direction of the minimum distance. The probability change of detail feature points (the distance from the detail feature point to the cluster center of each material region) can be analyzed on this line segment. The interval of probability change within the preset probability range is marked as the width interval of the material transition zone.

[0127] In this embodiment, the method for calculating the minimum boundary distance is as follows:

[0128] Two boundary point sets are formed by extracting the boundary pixels of two adjacent material regions respectively; then, the minimum boundary distance is obtained by calculating the shortest Euclidean distance between the two boundary point sets (the minimum distance from each point on the boundary of region A to the boundary of region B).

[0129] For example, from the interior of material A to the interior of material B, the probability of material A decreases from high to low. The width range of the material transition zone can be defined as the distance between where the probability decreases from 0.9 to 0.1 (or the threshold can be adjusted according to the actual situation). For example, the width range of the material transition zone can be set to 0.2 meters to 0.5 meters.

[0130] Then, multiple material regions are sampled to obtain multiple material transition band width ranges. The average or median of these widths is then taken as the final material transition band width range. Alternatively, statistical methods (such as mean ± standard deviation) can be used to determine the material transition band width range.

[0131] In this embodiment, S3-2-4 is also included:

[0132] If the width range of the material transition zone exceeds the preset width range, the detailed data of the high-density area and the detailed area will be collected again through the real-time sensing mechanism; then the updated material transition zone width range will be determined based on the detailed data collected in the second step.

[0133] S3-3: Based on the edge detection algorithm, the boundaries of dynamic objects in high-density and detail-rich areas are adjusted, and the adjustment parameters include sharpness, error control rate, etc.

[0134] In this embodiment, the sharpness enhancement factor is 1.2, and the boundary pixels are sharpened to ensure that the boundary error is controlled within 0.1 meters.

[0135] The above process is achieved through data-driven and algorithm-automated implementation. Logically, it forms a closed loop from the proportion of high-density areas and the quantification of detailed areas to the extraction of detailed data and feature optimization. At the same time, it is linked to the real-time feedback mechanism of the rendering engine to ensure that the mapping results adapt to dynamic scene changes and meet the needs of real-time rendering.

[0136] S4: The resource allocation optimization module dynamically adjusts computing resources using a load balancing method based on the width range of the material transition zone and boundary adjustment parameters. During the dynamic adjustment, information fusion is performed on the resource occupancy ratio of each resource area to obtain the adjusted computing allocation strategy.

[0137] Specifically, such as Figure 5 As shown, step S4 includes the following steps:

[0138] S4-1: Determine the computational resources for each resource region based on the material transition zone width range and boundary adjustment parameters.

[0139] In this embodiment, the width range of the material transition band is first dynamically divided into three levels based on the gradient change intensity of the transition band width range: when the included angle of the normal is less than 15 degrees, it is set as the first level, with a transition band width coefficient of 0.8~1.2 pixels; when the included angle of the normal is between 15 and 35 degrees, it is set as the second level, with a transition band width coefficient of 2.5~4.0 pixels; when the included angle of the normal exceeds 35 degrees, it is set as the third level, with a transition band width coefficient of 0.4~0.8 pixels.

[0140] Next, an edge sharpness evaluation model based on Sobel operator and temporal difference fusion is used to calculate the boundary sharpness score S. If S≥0.85, it is judged as a high-definition boundary with a boundary sharpness coefficient of 1.0; if 0.65≤S<0.85, it is medium with a boundary sharpness coefficient of 0.75; and if S<0.65, it is low-definition with a boundary sharpness coefficient of 0.45.

[0141] Finally, the computational resources, i.e., the resource requirement weights, are determined based on the transition band width coefficient and the boundary sharpness coefficient.

[0142] W = 0.6 × transition band width coefficient + 0.3 × boundary sharpness coefficient + 0.1 × motion speed factor; where the motion speed factor is linearly mapped by pixels / frame and ranges from 0 to 2.0.

[0143] S4-2: Use load balancing methods to dynamically adjust computing resources and obtain an optimized resource utilization ratio.

[0144] In this embodiment, the method for dynamic adjustment is as follows:

[0145] The occupancy rate is monitored in real time to determine if there is an uneven distribution of resources. If the computing resources of a certain resource area exceed the preset resource threshold, the computing resources of that area are prioritized, that is, the occupancy rate of computing resources is increased. If the computing resources of a certain resource area are less than or equal to the preset resource threshold, the priority is reduced, that is, the occupancy rate of computing resources is decreased.

[0146] For example, when the number of rendering threads in a certain resource area is 12 and the computing resource ratio is 23.7%, which exceeds the preset resource threshold of 20%, the computing resource utilization ratio needs to be increased, and the number of rendering threads should be adjusted from the original 12 to the total number of threads × 23.7% ≈ 17.

[0147] S4-3: Based on the optimized resource utilization ratio, information fusion is performed to obtain the adjusted calculation allocation strategy.

[0148] In this embodiment, the optimized resource occupancy ratio of three adjacent frames is fused using the exponentially weighted moving average (EWMA) method to obtain a smoothed adjusted resource allocation strategy (i.e., This avoids screen flickering caused by sudden changes in a single frame, achieving a dynamic optimal balance between computational resources and visual quality.

[0149]

[0150] In formula (5), This represents the adjusted computational allocation strategy at time t, i.e., the adjusted... ; This represents the smoothing factor, which is set to 0.35. This represents the adjusted computational allocation strategy at time t-1.

[0151] S5: Obtain the original detail level and texture sampling density in the resource area according to the adjusted calculation allocation strategy. Then, the adaptive optimization decision module adaptively adjusts the original detail level and texture sampling density to obtain the optimized detail level and texture sampling density. The image output module generates a balanced image output sequence.

[0152] Specifically, such as Figure 6 As shown, step S5 includes the following steps:

[0153] S5-1: Obtain the original detail level and texture sampling density according to the adjusted calculation allocation strategy.

[0154] In this embodiment, after the resource area is adjusted and the allocation strategy is optimized, the rendering quality parameters, including the original detail level and texture sampling density, can be obtained in real time.

[0155] S5-2: Adjust the detail level threshold of the original detail level to obtain the optimized detail level.

[0156] In this embodiment, when performing hierarchical analysis on the original detail levels, if the detail performance of a certain level is lower than the preset detail level threshold base_threshold, then the detail data of that level needs to be incrementally adjusted, that is, the preset detail level threshold base_threshold is reduced, thereby increasing the amount of detail data retained.

[0157] For example, in an actual rendering pipeline, the detail level threshold can be adjusted first for objects within the camera's range in the scene:

[0158]

[0159] In formula (6), This indicates the adjusted detail level threshold; This represents the original level of detail threshold; This represents the standardized value obtained by dividing the current distance between the object and the camera by 10.0 meters.

[0160] Experiments have shown that for objects within a 5.0-15.0 meter range from the camera in a scene, when the original detail level threshold is adjusted from 0.35 to 0.28, objects at medium distances can retain approximately 18% more geometric detail.

[0161] S5-3: Adaptively adjust the original texture sampling density to obtain the optimized texture sampling density.

[0162] In this embodiment, an adaptive anisotropic filtering algorithm is used to adaptively adjust the original texture sampling density: the texture sampling density is increased from isotropic 4x4 sampling to anisotropic 8x8 sampling. When the screen projection area is greater than 3200 pixels, 8x sampling is triggered; otherwise, it falls back to 4x sampling, so as to improve texture sharpness by about 27% while increasing performance by about 12%.

[0163] S5-4: Generate preliminary image output sequence data based on the optimized detail level and optimized texture sampling density.

[0164] In this embodiment, in the image output module, the rendering engine directly writes the optimized detail level and optimized texture sampling density into a unified material descriptor set, and performs a detail enhancement pass on the final rasterized segment through a compute shader. A 3×3 sharpening kernel (intensity coefficient 0.65) is used to perform targeted compensation on high-frequency areas, thereby generating preliminary image output sequence data (which is existing technology and will not be described in detail here).

[0165] S5-5: Detect whether there is quality inhomogeneity in the preliminary image output sequence data. If so, perform secondary rendering on the quality inhomogeneity part to determine the balanced image output sequence; otherwise, directly output the balanced image output sequence.

[0166] In this embodiment, the balanced image output sequence is formatted and stored, and it is determined whether there is data loss during the storage process. If there is data loss, an automatic completion mechanism is triggered to obtain a complete image output sequence file.

[0167] After the above continuous processing, the average frame rate of the resulting balanced image output sequence at 1080p resolution increased from 42fps to 51fps, while the PSNR value increased from 34.2dB to 36.8dB, achieving an effective trade-off between quality and performance.

[0168] S6: Perform continuous stability verification processing on the balanced image output sequence in the video imaging loop; if the cumulative deviation between the screen space projection distortion degree and the historical fluctuation trend exceeds the preset distortion range (indicating system instability), then re-acquire the updated indicators of the duration of the sudden load peak and the regional distribution imbalance, adjust the resource allocation strategy of the rendering task until the system returns to stability.

[0169] Specifically, such as Figure 7 As shown, step S6 includes the following steps:

[0170] S6-1: Extract the projection coordinate information of each frame in the balanced image output sequence in the screen space. By comparing the coordinate offset with the standard template, determine whether projection distortion exists. If yes, proceed to S6-2. Otherwise, it means the system is stable and subsequent processing stops.

[0171] In this embodiment, a root mean square error calculation method based on a sliding window can be used. The window size is the most recent 30 frames. The screen space projection distortion is calculated for each frame. The distortion value is obtained by the Euclidean distance between the projection coordinates and the ideal coordinates, denoted as d_i. When the average distortion value of 10 consecutive frames exceeds 0.038 pixels and the standard deviation is greater than 1.45 times the standard deviation of the historical 30 frames, it is judged as an abnormal fluctuation, that is, there is projection distortion.

[0172] S6-2: Calculate the cumulative deviation of historical fluctuation trends and compare it with the preset distortion range; when the cumulative deviation of historical fluctuation trends exceeds the preset distortion range, it is determined that it has deviated from the long-term stable state and proceeds to S6-3; when the cumulative deviation of historical fluctuation trends is within the preset distortion range, it is determined that it is in short-term distortion and continues the distortion judgment of the next window.

[0173] In this embodiment, the cumulative deviation Δ_trend of the historical fluctuation trend of the most recent 120 frames (i.e., 4 windows) can be calculated using an exponentially weighted moving average algorithm (α=0.12). When Δ_trend exceeds the preset distortion range (e.g., ±0.076), it is considered that the current distortion has deviated from the long-term stable state.

[0174] S6-3: Obtain updated metrics including the duration of sudden load peaks and the degree of regional distribution imbalance, and identify resource regions with a sudden load peak duration greater than a preset duration and resource regions with a load imbalance degree less than a preset entropy value as priority resource regions.

[0175] In this embodiment, the GPU samples the rendering timestamp every 2 milliseconds. When the peak value exceeds the normal average by 3.2 times and the duration is greater than 18 milliseconds, it is identified as a sudden load peak, and the corresponding duration of the sudden load peak is recorded.

[0176] In this embodiment, the load imbalance of each resource area is calculated, and the distribution uniformity is quantified using the Shannon entropy formula. When the entropy value is lower than 5.82 (the full entropy is 7.17), it is marked as a severely unbalanced regional distribution.

[0177] Therefore, resource areas with sudden load peaks and severely uneven regional distribution are identified as priority resource areas for processing.

[0178] In this embodiment, the updated index is smoothed by Kalman filtering (process noise Q=0.0045, measurement noise R=0.009) and then updated to the sequence buffer for the next round of stability verification, thereby forming a closed-loop adaptive adjustment mechanism to ensure that the video imaging process remains continuous and stable under dynamic load.

[0179] S6-4: Adjust the computational resources of the rendering task for the priority resource region to obtain optimized rendering performance indicators, including frame rate, frame count, rendering time, GPU utilization, vertex shader execution cycle count, etc.

[0180] In this embodiment, the computational resources of the priority resource region can be adjusted according to S4-2 and S4-3, for example, the number of rendering threads in the priority resource region can be increased.

[0181] S6-5: Regenerate the balanced image output sequence based on the optimized rendering performance metrics and perform stability verification until the system returns to a stable state.

[0182] S7: Perform multiple rounds of iterative adjustments to the virtual environment rendering process based on the updated metrics, and merge all the adjustment results to obtain a stable image sequence whose image stability consistently meets the requirements.

[0183] S7-1: Uses a load balancing algorithm to distribute the duration of burst load peaks to multiple rendering nodes, forming an optimized peak duration.

[0184] For example, by analyzing the duration of sudden load peaks, assuming that the peak load duration in a certain rendering task is 3.5 seconds, exceeding the normal load threshold by 1.2 seconds, the system uses a load balancing algorithm to distribute the peak load to multiple rendering nodes. Using a weighted average distribution method, the load weight of each rendering node is calculated to be 25% of the peak load, and the load of each node after distribution is monitored in real time to ensure that the load does not exceed the upper limit of 80%, ensuring that the rendering task is not interrupted due to single-point overload. The analysis results show that the peak duration is shortened to 1.8 seconds after distribution, which meets the expected stability requirements.

[0185] S7-2: Adjust the density of the regional distribution imbalance to obtain the optimized regional density.

[0186] In this embodiment, assuming that the pixel density deviation of a certain resource area in the rendered image exceeds 15%, the system divides the resource area into 4 sub-regions through a region division algorithm, calculates the average pixel density of each sub-region, and adjusts the rendering resource allocation ratio so that the resource ratio of high-density areas is increased to 40% and low-density areas are reduced to 10%. After adjustment, the deviation is reduced to 5%, and the overall balance of the image is significantly improved.

[0187] S7-3: Perform multiple iterations based on the optimized peak duration and optimized regional density. In each round, the adjustment parameters for the current round are generated based on the output of the previous round. If the change in the adjustment parameters of the current round is less than the preset threshold, the iterative adjustment process is stopped, and the final set of adjustment parameters, including load peak and distribution balance, is output.

[0188] In this embodiment, each iteration is spaced at 0.1-second intervals, recording the changes in load peak and distribution balance. The gradient descent algorithm is used to optimize the parameters. After 5 iterations, the duration of the load peak stabilizes at 1.5 seconds, and the distribution deviation stabilizes at 3%, achieving the optimization goal.

[0189] S7-4: Perform weighted fusion based on the final set of adjusted parameters to generate a stable image sequence.

[0190] In this embodiment, the weights are set to 60% for peak load and 40% for distribution balance, generating a final stable output sequence with image stability consistently above 95%, meeting business requirements. By linking with virtual environment rendering latency monitoring, the system further verifies the adaptability of the output sequence in different scenarios, ensuring latency does not exceed 50 milliseconds, forming a complete logical chain from load optimization to image stability.

[0191] Example 2

[0192] Based on the video imaging adaptive 3D scene synchronous rendering method provided in Embodiment 1, such as Figure 8 As shown, the present invention also provides a video imaging adaptive 3D scene synchronous rendering system, specifically including a data acquisition module, a preprocessing module, a perception module, a detail data analysis module, a resource allocation optimization module, an adaptive optimization decision module, and a screen output module; the output end of the data acquisition module is connected to the input end of the preprocessing module, the output end of the preprocessing module is connected to the input end of the perception module, the output end of the perception module is connected to the input end of the detail data analysis module, the output end of the detail data analysis module is connected to the input end of the resource allocation optimization module, the output end of the resource allocation optimization module is connected to the input end of the adaptive optimization decision module, and the output end of the adaptive optimization decision module is connected to the input end of the screen output module.

[0193] The data acquisition module is used to obtain the rendering performance metrics of the current video, including rendering time per frame, peak GPU utilization, and vertex shader execution cycles.

[0194] The preprocessing module is used to calculate the variance change and load peak change of rendering performance indicators to obtain rendering performance fluctuation indicators, including the level of variation of rendering time variance and the level of variation of load peak.

[0195] The perception module is used to collect spatial geometric distribution information of the current video scene based on rendering performance fluctuation indicators, and to determine the proportion of high-density areas and the proportion of detailed areas.

[0196] The detailed data analysis module is used to extract detailed data of the corresponding area based on the proportion of high-density areas and the proportion of detailed dense areas, and to determine the width range of the material transition zone and boundary adjustment parameters.

[0197] The resource allocation optimization module is used to dynamically adjust computing resources based on the width range of the material transition zone and boundary adjustment parameters to obtain the adjusted computing allocation strategy;

[0198] The adaptive optimization decision module is used to obtain the original detail level and texture sampling density in the resource area according to the adjusted calculation allocation strategy, and to adaptively adjust them to obtain the optimized detail level and texture sampling density.

[0199] The image output module is used to output a balanced image output sequence based on the optimized detail level and texture sampling density.

[0200] This embodiment also includes a stability adjustment module and an image adjustment module.

[0201] The stability adjustment module is used to verify and adjust the stability of the balanced image output sequence and obtain updated indicators including the duration of burst load peaks and the unevenness of regional distribution.

[0202] The image adjustment module is used to perform optimization iterations based on updated metrics, output the final set of adjustment parameters, including load peak and distribution balance, and generate a stable image sequence.

[0203] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A video imaging adaptive 3D scene synchronous rendering method, characterized in that, Includes the following steps: S1: Obtain the rendering performance metrics of the video and preprocess the rendering performance metrics to obtain the rendering performance fluctuation metrics; the rendering performance fluctuation metrics include the level of variation of rendering time variance and the level of variation of load peak. S2: Use an adaptive threshold comparison method to judge whether the rendering performance fluctuation index is abnormal; if so, divide the current video scene into multiple resource regions, calculate the region density and high-frequency texture ratio of each resource region, and thus determine the proportion of high-density regions and the proportion of detailed regions; otherwise, return to S1. S3: Based on the proportion of high-density areas and the proportion of detail-intensive areas, extract detail data of high-density areas and detail-intensive areas from the synchronous rendering pipeline, and perform cluster analysis on the detail data to determine the material classification, thereby determining the material transition band width range and boundary adjustment parameters; S4: Based on the width range of the material transition zone and the boundary adjustment parameters, the load balancing allocation method is used to dynamically adjust the computing resources, and the information fusion of the resource occupancy ratio of each resource area is carried out in the dynamic adjustment to obtain the adjusted computing allocation strategy. S5: Obtain the original detail level and texture sampling density in the resource area according to the adjusted calculation allocation strategy, and then adaptively adjust the original detail level and texture sampling density to obtain the optimized detail level and texture sampling density, generating a balanced image output sequence.

2. The video imaging adaptive 3D scene synchronous rendering method as described in claim 1, characterized in that, Also includes: S6: Perform continuous stability verification processing on the balanced image output sequence in the video imaging loop; If the cumulative deviation between the screen space projection distortion and the historical fluctuation trend exceeds the preset distortion range, then the updated indicators of the duration of the sudden load peak and the regional distribution imbalance are re-acquired, and the resource allocation strategy of the rendering task is adjusted until the system returns to stability. S7: Perform multiple rounds of iterative adjustments to the virtual environment rendering process based on the updated metrics, and merge all adjustment results to generate a stable image sequence.

3. The video imaging adaptive 3D scene synchronous rendering method as described in claim 1, characterized in that, S1 includes: S1-1: Obtain the rendering performance metrics of the current video in the virtual environment through the data acquisition module, including rendering time per frame, peak GPU utilization, and vertex shader execution cycles. S1-2: Calculate the variance of the rendering time for each frame in the rendering performance metrics to obtain the rendering time variance sequence; S1-3: Calculate the standard deviation of the rendering time variance sequence to determine the level of variation in rendering time variance; S1-4: Obtain the GPU peak utilization sequence based on the rendering performance metrics, then perform weighted processing on the GPU peak utilization sequence to obtain a smooth peak trend sequence, and determine the load peak change level.

4. The video imaging adaptive 3D scene synchronous rendering method as described in claim 1, characterized in that, S2 includes: S2-1: Use an adaptive threshold comparison method to determine whether the rendering performance fluctuation index exceeds the preset threshold range. If not, return to S1; otherwise, proceed to S2-2. S2-2: The perception module collects spatial geometric distribution information of the current video scene from the virtual environment, thereby dividing it into multiple resource regions, calculating the regional density and high-frequency texture ratio of each resource region, and thus determining the proportion of high-density regions and the proportion of detail-dense regions.

5. The video imaging adaptive 3D scene synchronous rendering method as described in claim 1, characterized in that, S3 includes: S3-1: Extract detail data of high-density and detail-intensive regions from the synchronous rendering pipeline using the rendering engine's frame buffer, and then extract detail feature points from the detail data; the detail data includes depth variation and normal variation rate; S3-2: Classify detailed feature points to obtain different material regions and determine the width range of the material transition zone; S3-3: Based on the edge detection algorithm, the boundaries of dynamic objects in high-density and detail-rich areas are adjusted. The boundary adjustment parameters include sharpness and error control rate.

6. The video imaging adaptive 3D scene synchronous rendering method as described in claim 5, characterized in that, S3-2 includes: S3-2-1: Use a convolutional neural network to classify the extracted detail feature points to obtain the material labels of the detail feature points; S3-2-2: Spatial clustering of detailed feature points belonging to the same material label to form different material regions; S3-2-3: By calculating the minimum distance between the boundaries of adjacent material regions, a line segment is sampled in the direction of the minimum distance. The probability change of detailed feature points is analyzed on the line segment, and the interval of probability change within the preset probability range is marked as the width interval of the material transition zone.

7. The video imaging adaptive 3D scene synchronous rendering method as described in claim 1, characterized in that, S4 includes: S4-1: Determine the computational resource W for each resource region based on the material transition zone width range and boundary adjustment parameters. W = 0.6 × transition band width coefficient + 0.3 × boundary sharpness coefficient + 0.1 × motion speed factor; where the motion speed factor is linearly mapped per pixel / frame; Based on the normal angle, the width range of the material transition zone is divided into three levels, with different levels corresponding to different transition zone width coefficients; The boundary sharpness score is calculated based on the boundary adjustment parameters, and different boundary sharpness scores correspond to boundary sharpness coefficients. S4-2: The load balancing method is used to dynamically adjust the computing resources to obtain the optimized resource utilization ratio; S4-3: Based on the optimized resource utilization ratio, information fusion is performed to obtain the adjusted computational allocation strategy: in, This represents the adjusted computational allocation strategy at time t; Indicates the smoothing factor; This represents the adjusted computational allocation strategy at time t-1.

8. The video imaging adaptive three-dimensional scene synchronous rendering method as described in claim 1, characterized in that, S5 includes: S5-1: Obtain the original detail level and texture sampling density according to the adjusted calculation allocation strategy; S5-2: Adjust the detail level threshold of the original detail level to obtain the optimized detail level; S5-3: Adaptively adjust the original texture sampling density to obtain the optimized texture sampling density; S5-4: Generate preliminary image output sequence data based on the optimized detail level and optimized texture sampling density; S5-5: Detect whether there is quality inhomogeneity in the initial image output sequence data. If so, perform secondary rendering on the quality inhomogeneity part and output a balanced image output sequence; otherwise, directly output a balanced image output sequence.

9. The video imaging adaptive three-dimensional scene synchronous rendering method as described in claim 1, characterized in that, S6 includes: S6-1: Extract the projection coordinate information of each frame in the balanced image output sequence in screen space. By comparing the coordinate offset with the standard template, determine whether projection distortion exists. If yes, proceed to S6-2; otherwise, it indicates that the system is stable and subsequent processing stops. S6-2: Calculate the cumulative deviation of historical fluctuation trends and compare it with the preset distortion range; when the cumulative deviation of historical fluctuation trends exceeds the preset distortion range, it is determined that it has deviated from the long-term stable state and proceeds to S6-3; when the cumulative deviation of historical fluctuation trends is within the preset distortion range, it is determined that it is in short-term distortion and continues the distortion judgment of the next window. S6-3: Obtain updated metrics including the duration of the burst load peak and the regional distribution imbalance, and identify resource regions with a burst load peak duration greater than a preset duration and resource regions with a load imbalance less than a preset entropy value as priority resource regions. S6-4: Adjust the computational resources of the rendering task for the priority resource area to obtain optimized rendering performance indicators; S6-5: Regenerate the balanced image output sequence based on the optimized rendering performance metrics and perform stability verification until the system returns to a stable state.

10. A video imaging adaptive three-dimensional scene synchronous rendering system based on the method of any one of claims 1-9, characterized in that, include: The data acquisition module is used to obtain the rendering performance metrics of the current video, including rendering time per frame, peak GPU utilization, and vertex shader execution cycles. The preprocessing module is used to calculate the variance change and load peak change of rendering performance indicators to obtain rendering performance fluctuation indicators, including the level of variation of rendering time variance and the level of variation of load peak. The perception module is used to collect spatial geometric distribution information of the current video scene based on rendering performance fluctuation indicators, and to determine the proportion of high-density areas and the proportion of detailed areas. The detailed data analysis module is used to extract detailed data of the corresponding area based on the proportion of high-density areas and the proportion of detailed dense areas, and to determine the width range of the material transition zone and boundary adjustment parameters. The resource allocation optimization module is used to dynamically adjust computing resources based on the width range of the material transition zone and boundary adjustment parameters to obtain the adjusted computing allocation strategy; The adaptive optimization decision module is used to obtain the original detail level and texture sampling density in the resource area according to the adjusted calculation allocation strategy, and to adaptively adjust them to obtain the optimized detail level and texture sampling density. The image output module is used to output a balanced image output sequence based on the optimized detail level and texture sampling density.

Images