Three-dimensional gaussian rendering optimization system and method based on cpu level splitting and gpu driving

Abstract

This application provides a 3D Gaussian rendering optimization system and method based on CPU hierarchical splitting and GPU driving, relating to the field of 3D graphics rendering technology. This application divides the 3D Gaussian scene into multiple bounding boxes on the CPU side, determines the view distance level, protection level, and reconstruction cost level of each bounding box, and uses the highest of these three levels as the lower limit of the target level. The scheduling module adjusts the current rendering level frame by frame, determining the summary state, partial unfolding mode, or unfolded state based on screen error and GPU budget. On the GPU side, representative Gaussian points and unfolded 3D Gaussian points are uniformly sorted and rendered, and the frame rate and the total number of activated 3D Gaussian points are fed back. This reduces the amount of data submitted, takes into account the protection of key areas, achieves smooth hierarchical transitions, and forms a rendering feedback loop, improving the frame rate stability and resource utilization of real-time rendering.

Description

Technical Field

[0001] This application relates to the field of 3D graphics rendering technology, and in particular to a 3D Gaussian rendering optimization system and method based on CPU hierarchical splitting and GPU driving. Background Technology

[0002] 3D Gaussian Splatting is an explicit representation and rendering method for 3D scenes using a large number of 3D Gaussian points with position, color, opacity, and covariance attributes. During rendering, the rendering pipeline projects each 3D Gaussian point in the scene onto the screen, sorts them by camera distance, and then performs opacity blending and rasterization to obtain the final image. Because it can achieve fast rendering speed while maintaining high image quality, this method is widely used in 3D reconstruction, digital twins, virtual reality, and real-time roaming of large-scale scenes.

[0003] As scene scale expands, the number of 3D Gaussian points in a single scene often reaches millions or even tens of millions. Projecting, sorting, blending, and rasterizing point by point places a heavy burden on the central processing unit (CPU) and graphics processing unit (GPU), making it difficult to maintain a stable real-time frame rate with limited computing resources. To alleviate the above problems, existing technologies usually introduce a Level of Detail (LOD) control mechanism, which selects the display details of 3D objects based on their distance from the camera: objects farther from the camera are drawn with fewer points or a coarser representation, while objects closer to the camera are drawn with more points or a finer representation.

[0004] The closest existing technology to this application is a rendering scheme that controls the Level of Detail (LOD) of 3D Gaussian points solely based on view distance. This scheme pre-constructs a multi-resolution hierarchy of 3D Gaussian points, and at runtime, selects the corresponding resolution representation for submission and rendering based on the distance of the object or region from the camera. According to the applicant's analysis, this scheme has the following main shortcomings: First, using only line-of-sight distance as the criterion for selection fails to reflect the varying degrees of importance of different areas within a scene. Areas that are close at hand but contain very little actual information will be allocated excessive rendering resources; while areas that are farther away but carry critical information such as alarms and monitoring footage may be oversimplified, making the critical information difficult to identify on the screen.

[0005] Secondly, it lacks consideration of the cost of layer switching. For complex areas with significant differences in color, opacity, and depth distribution, switching between different layers can easily cause obvious abrupt changes and flickering in the image. A solution based solely on viewing distance will frequently trigger layer switching in such areas during camera movement, disrupting the continuity of the image.

[0006] Third, layer switching is often completed directly within a single frame, jumping from one resolution to another without a smooth transition, easily resulting in visible jumps and flickering on the screen. Fourth, the layer selection and submission strategy on the CPU side is usually fixed, failing to utilize the actual rendering feedback information on the GPU side for dynamic constraints. When the scene complexity suddenly increases, it is easy to cause the frame rate to drop sharply due to excessive data submission, or to excessively reduce image quality in order to maintain the frame rate.

[0007] Therefore, how to balance rendering efficiency, image quality protection in key areas, smoothness of layer switching, and stability of resource utilization in real-time rendering of large-scale 3D Gaussian scenes is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The purpose of this application is to provide a 3D Gaussian rendering optimization system and method based on CPU hierarchical splitting and GPU driving, so as to solve the problems of excessive number of 3D Gaussian points submitted, low efficiency of CPU and GPU collaborative scheduling, insufficient protection of key areas, and difficulty in balancing real-time rendering frame rate and image quality when rendering large-scale 3D Gaussian scenes in the prior art.

[0009] To address the aforementioned technical problems, in a first aspect, this application provides a 3D Gaussian rendering optimization system based on CPU hierarchical splitting and GPU driving, comprising: The scene splitting module is used to divide a 3D Gaussian scene into multiple bounding boxes on the CPU side and record the correspondence between each bounding box and the 3D Gaussian points within it. The level determination module is used to determine the view distance level, protection level and reconstruction cost level of each bounding box under the same discrete level scale, and take the highest level among the three as the target level lower limit. The higher the level, the higher the required rendering accuracy. The protection level is determined by the overlap relationship between the bounding box and the fidelity constraint region. A protection level of zero indicates that it is not constrained by the fidelity constraint region. The scheduling module is used to adjust the current rendering level of the bounding box to the lower limit of the target level frame by frame, and adjust it by a maximum of one level per frame, and determine the rendering state of the bounding box as a summary state, a partial expansion mode or an expanded state based on the screen error. The rendering module is used on the GPU side to uniformly sort and render the summary state, partial unfolding method and representative Gaussian points or 3D Gaussian points generated by the unfolding state according to the distance from the camera from far to near, and feeds back the actual frame rate of the current frame and the total number of activated 3D Gaussian points to the scheduling module. Here, the representative Gaussian point is obtained by summing multiple three-dimensional Gaussian points within a single bounding box; when the actual frame rate is lower than the preset frame rate, the scheduling module selects a degraded object from the bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowed level, and reduces its current rendering level within a range not lower than the minimum allowed level; for bounding boxes with a non-zero protection level, the minimum allowed level is not lower than its protection level.

[0010] Optionally, the fidelity constraint region includes the alarm POI influence range and the video texture projection region; the system also includes an alarm monitor, a video projector, and a protection level merger; The alarm monitor is used to generate alarm points of interest (POIs) and their impact ranges based on alarm events in a 3D Gaussian scene, and to determine the corresponding protection level based on the alarm event level and the overlap ratio between the bounding box and the impact range of the alarm POI. The video projector is used to project the dynamic texture of multiple video streams onto the three-dimensional Gaussian points after the corresponding bounding box is unfolded. The area where the bounding box to be projected is located is determined as the video texture projection area, and the corresponding protection level is determined according to the projection area ratio of the bounding box in the video texture projection area and the update frequency of the corresponding video texture. The protection level merger is used to take the highest of the multiple protection levels corresponding to a single bounding box that is affected by multiple alarm POIs or is located in the video texture projection area as the protection level of the bounding box.

[0011] Optionally, the level determination module determines the reconstruction cost level based on the color dispersion, opacity dispersion, and depth dispersion of multiple three-dimensional Gaussian points within a single bounding box, as well as the distribution density of the three-dimensional Gaussian points within the screen projection area of ​​the bounding box under the current field of view. The reconstruction cost level increases with increasing color dispersion, opacity dispersion, or depth dispersion, and increases with decreasing 3D Gaussian point distribution density.

[0012] Optionally, the scheduling module determines the rendering method of the bounding box based on screen error and GPU budget: If the screen error does not exceed the first tolerance, then the bounding box is set to the summary state; If the screen error exceeds the first tolerance but does not exceed the second tolerance, then the three-dimensional Gaussian points with the product of the screen projection area and opacity within the bounding box are expanded to the first preset ratio, and the remaining three-dimensional Gaussian points are summarized into a representative Gaussian point. If the screen error exceeds the second tolerance and the protection level of the bounding box is not zero, the bounding box is set to the unfolded state when the GPU budget can accommodate the unfolded load, and a partial unfolding method with a protection level no lower than that is adopted when the GPU budget is insufficient. If the bounding box is outside the view frustum, then the bounding box is removed from the sorting range of the rendering module; wherein the first tolerance is less than the second tolerance.

[0013] Optionally, when the actual frame rate is lower than the preset frame rate or the total number of activated 3D Gaussian points exceeds the GPU budget, the scheduling module preferentially selects downgrade objects from the bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowed level, and reduces their current rendering level within a range not lower than the minimum allowed level. If the GPU budget is still exceeded, the current rendering level of the bounding box that is within the video texture projection area, has a corresponding video texture update frequency lower than the preset frequency, and has a current rendering level higher than the corresponding minimum allowed level is reduced; for bounding boxes with a protection level that is not zero, the minimum allowed level is not lower than its protection level.

[0014] Optionally, the position of the representative Gaussian point is obtained by weighting the positions of multiple three-dimensional Gaussian points within the corresponding bounding box by opacity, and the color and opacity are obtained by weighting the multiple three-dimensional Gaussian points within the corresponding bounding box by opacity. The display range of the representative Gaussian point is determined based on the screen projection range of multiple three-dimensional Gaussian points within the bounding box under the current field of view. The screen error is determined by the screen projection area, depth dispersion, and color variance of the bounding box.

[0015] Optionally, both the representative Gaussian point and the unfolded 3D Gaussian point are provided with a sorting key. The sorting key includes a bounding box identifier, a depth value, and a type identifier. The type identifier is used to distinguish between the representative Gaussian point and the unfolded 3D Gaussian point. The rendering module first performs mixed rendering of representative Gaussian points and unfolded 3D Gaussian points according to depth value and then according to type identifier in the same queue, and performs stable sorting according to bounding box identifier when the depth values ​​are the same or close.

[0016] Optionally, the scheduling module is provided with a transition period; When a single bounding box transitions from a summed state to an expanded state, the scheduling module reduces the contribution coefficient of the representative Gaussian point frame by frame during the transition period, and increases the contribution coefficient of the expanded 3D Gaussian point set frame by frame. Each expanded 3D Gaussian point retains its original opacity and is multiplied by the corresponding contribution coefficient when the opacity is mixed, and the sum of the contribution coefficients of the representative Gaussian point and the expanded 3D Gaussian point set does not exceed 1. When a single bounding box transitions from an expanded state to a summarized state, the scheduling module adjusts the contribution coefficients of the representative Gaussian point and the expanded 3D Gaussian point set frame by frame in the opposite direction during the transition period.

[0017] Optionally, when the actual frame rate drops or the total number of activated 3D Gaussian points exceeds the GPU budget during the transition period, the scheduling module adds a scheduling penalty term or reduces the expansion priority correction term for the bounding box that undergoes a state switch, so as to improve its priority in being subsequently set to the aggregated state or partial expansion mode. When multiple bounding boxes compete for GPU budget simultaneously, the scheduling module sorts the multiple bounding boxes in order of protection level from high to low, screen projection area from large to small, reconstruction cost level from high to low, and the time of the most recent state switch from early to late, and prioritizes setting or keeping the bounding box with the highest sorting to an expanded state.

[0018] Secondly, this application provides a 3D Gaussian rendering optimization method based on CPU hierarchical splitting and GPU driver, including: On the CPU side, the 3D Gaussian scene is divided into multiple bounding boxes, and the correspondence between each bounding box and the 3D Gaussian points within it is recorded. Under the same discrete level scale, the view distance level, protection level and reconstruction cost level of each bounding box are determined, and the highest level among the three is taken as the lower limit of the target level. The higher the level, the higher the required rendering accuracy. A protection level of zero indicates that there is no region constrained by fidelity constraints. The current rendering level of the bounding box is adjusted to the lower limit of the target level frame by frame, with a maximum adjustment of one level per frame; the bounding box is determined to be in a summarized state, a partially expanded state, or an expanded state based on the screen error of the bounding box. On the GPU side, the aggregated state, partial unfolding method, and representative Gaussian points or 3D Gaussian points generated by the unfolded state are uniformly sorted and rendered according to the camera distance from farthest to nearest; the actual frame rate of the current frame and the total number of activated 3D Gaussian points are counted and fed back to the CPU side. When the actual frame rate is lower than the preset frame rate, a degraded object is selected from the bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowed level, and its current rendering level is reduced. For bounding boxes with a non-zero protection level, the minimum allowed level is not lower than its protection level.

[0019] The beneficial effects of this application are as follows: First, this application pre-decomposes the 3D Gaussian scene into layers on the CPU side and records the corresponding 3D Gaussian points and representative Gaussian points for the bounding boxes. This eliminates the need for the rendering system to judge and submit all 3D Gaussian points one by one during runtime. Instead, it can first determine the rendering level at the bounding box level and then select and submit representative Gaussian points or specific 3D Gaussian points according to the level. This reduces the amount of data that the CPU side needs to traverse, filter, and submit, and reduces the pressure on the GPU side for sorting, blending, and rasterization.

[0020] Second, this application unifies the viewing distance level, protection level, and reconstruction cost level into comparable discrete levels, and takes the highest one as the lower limit of the target level. This ensures that the lower limit of the target level simultaneously reflects viewing distance, the protection requirements of key areas, and the cost of level switching. Compared to LOD control based solely on viewing distance, this application avoids problems such as insufficient detail in close-range areas, oversimplification of key areas, and frequent switching of high reconstruction cost areas. This improves the stability and recognizability of key areas while maintaining rendering efficiency.

[0021] Third, this application adjusts the current level frame by frame based on the lower limit of the target level, rather than jumping directly to the target level within a single frame. This allows for a smooth transition between the aggregated state, the partially expanded state, and the expanded state of the 3D Gaussian points. This reduces flickering, abrupt changes, and jumps in local details during level switching, improving the continuity of the 3D Gaussian rendering, and is especially suitable for real-time display scenarios when the camera is moving rapidly or key areas are entering the field of view.

[0022] Fourth, this application determines the processing methods for summarizing, partially expanding, expanding, or removing 3D Gaussian points within the bounding box based on screen errors. This allows distant, low-contribution, or currently invisible areas to participate in rendering as representative Gaussian points or to remain outside the current sorting range, while nearby areas with larger screen errors or those under protection can be expanded into specific 3D Gaussian points for rendering. This allows the limited rendering budget to be concentrated on areas with a greater impact on the final image, reducing invalid point submissions and calculations for invisible areas, and improving the image quality per unit of computing resources.

[0023] Fifth, this application also utilizes the frame rate, number of visible points, or rendering load information fed back from the GPU side to constrain the hierarchical scheduling and submission quantity of subsequent frames on the CPU side, enabling the CPU-side scheduling results to be dynamically adjusted according to the actual rendering pressure of the GPU. This forms a closed loop between CPU hierarchical scheduling and GPU rendering feedback, avoiding the problem of excessive submission quantity or excessive image quality degradation in complex scenes due to fixed hierarchical strategies, and improving frame rate stability and resource utilization during real-time rendering. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the system architecture of the 3D Gaussian rendering optimization system based on CPU hierarchical splitting and GPU driving in this application.

[0026] Figure 2 This is a schematic diagram illustrating the CPU-side bounding box hierarchy splitting and target level lower limit determination in an embodiment of this application.

[0027] Figure 3 This is a schematic diagram illustrating the determination of the bounding box rendering method and the output of the status in an embodiment of this application.

[0028] Figure 4 This is a flowchart illustrating the 3D Gaussian rendering optimization method based on CPU hierarchical splitting and GPU driving proposed in this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following description is provided in conjunction with embodiments. These embodiments are used to explain the implementation of this application, and those skilled in the art can make equivalent substitutions or adjustments without departing from the scope of the claims.

[0030] The 3D Gaussian point involved in this application refers to a basic rendering unit with color, opacity, and covariance attributes, based on its center position in 3D space. A bounding box is used to spatially organize a set of 3D Gaussian points, enabling the scheduling process to be performed at the region level, rather than judging all 3D Gaussian points point by point in each frame. A representative Gaussian point is obtained by summing multiple 3D Gaussian points within a single bounding box, used to approximate the overall appearance of the region without needing to expand all details. Screen error represents the potential image deviation under the current field of view when approximating all 3D Gaussian points within the bounding box using a representative Gaussian point or a partial expansion method; runtime screen error uses the same surrogate index, weighted by normalized screen projection area, normalized depth dispersion, and normalized color variance. A fidelity-constrained region refers to an area where high image fidelity needs to be maintained, such as the area near the alarm location or an area projected with real-time monitoring video.

[0031] The overall processing of this application can be understood as follows: The CPU first organizes the 3D Gaussian scene at the bounding box level and determines the target level lower limit for each bounding box; the scheduling module gradually adjusts the current rendering level in subsequent frames and determines the rendering method based on screen error, protection level, GPU budget and view frustum; the GPU uniformly sorts and renders the representative Gaussian points and the expanded 3D Gaussian points that enter the current frame, and feeds back the actual frame rate and the total number of activated 3D Gaussian points to the scheduling module.

[0032] like Figure 1 As shown, the system structure of this application includes a scene splitting module, a level determination module, a scheduling module, and a rendering module.

[0033] The scene splitting module divides the 3D Gaussian scene into multiple bounding boxes on the CPU side and records the correspondence between each bounding box and the 3D Gaussian points within it. The level determination module determines the view distance level, protection level, and reconstruction cost level of each bounding box under the same discrete level scale, and takes the highest level among the three as the target level lower limit. The scheduling module adjusts the current rendering level of the bounding box to the target level lower limit frame by frame, and determines the overall state, partial unfolding method, or unfolded state of the bounding box based on screen error, protection level, GPU budget, and view frustum. The rendering module sorts and renders the representative Gaussian points and unfolded 3D Gaussian points entering the current frame on the GPU side, and feeds back the actual frame rate of the current frame and the total number of activated 3D Gaussian points to the scheduling module. Figure 1 The feedback arrows indicate that the GPU-side rendering feedback output by the rendering module is used by the scheduling module for subsequent frame scheduling. This feedback relationship still belongs to the data interaction between the scheduling module and the rendering module mentioned above.

[0034] The scene splitting module performs scene organization on the CPU side. In actual processing, the positions of each 3D Gaussian point in the 3D Gaussian scene can be read first to determine the overall spatial range of the scene. Then, multiple bounding boxes are formed using octrees, binary space partitioning, or other spatial partitioning methods. The termination condition for partitioning does not need to be fixed to a certain absolute number of points. It can be determined by combining the number of 3D Gaussian points in a single region, the spatial range, the subsequent scheduling granularity, and the screen projection size.

[0035] For example, in a campus-level digital twin scenario, if the screen projection area of ​​a certain region under a typical viewpoint is less than approximately 64×64 pixels, a relatively coarse-grained rendering can be maintained even if the region contains thousands of 3D Gaussian points. If the screen projection area of ​​a certain region under a commonly used viewpoint exceeds approximately 256×256 pixels, or if the camera frequently enters a position within 10 meters of that region, further subdivision can be performed to allow for more precise control of its rendering level in subsequent scheduling. After subdivision, each bounding box has a corresponding bounding box identifier, and a correspondence is established between this identifier and the list of 3D Gaussian point indices within it.

[0036] like Figure 2As shown, in one specific embodiment, a 3D Gaussian scene can be organized into a parent bounding box, a group of child bounding boxes, and a set of 3D Gaussian points corresponding to each bounding box. For any current bounding box, it corresponds to both the bounding box organization relationship in the scene hierarchy and the source of determination for the view distance level, protection level, and reconstruction cost level. Specifically, the view distance level is determined by the distance between the current bounding box and the camera; the protection level is determined by the overlap between the alarm POI influence range or the video texture projection area and the current bounding box; and the reconstruction cost level is determined by color dispersion, opacity dispersion, depth dispersion, and point distribution density. After these three levels are mapped to the same discrete level scale, the highest level is taken as the lower limit of the target level for the current bounding box. Figure 2 The lines in the diagram only indicate the correspondence and level relationship between objects, and do not indicate the order of method steps.

[0037] The level determination module uses the same discrete level scale to allow direct comparison of the view distance level, protection level, and reconstruction cost level. The number of discrete levels can be determined based on scene size and hardware performance. For example, six discrete levels can be set, from level 0 to level 5, where level 0 represents the minimum accuracy requirement and level 5 represents the highest accuracy requirement. Small-scale scenes can use four levels, from level 0 to level 3, to reduce scheduling complexity, while large-scale scenes can use eight levels, from level 0 to level 7, to obtain finer rendering control. Higher levels indicate higher required rendering accuracy. If a bounding box is close to the camera, falls within a fidelity constraint region, or easily produces significant image differences after aggregation, its corresponding level will be increased. Ultimately, the highest level among the three categories forms the lower limit of the target level.

[0038] The viewing distance level can be determined based on the distance from the current camera to the center of the bounding box or the nearest point of the bounding box. When setting distance intervals, you can first statistically analyze the distance distribution of visible bounding boxes in the current scene, and then divide the distance distribution into several intervals from near to far; the near distance intervals correspond to higher levels, and the far distance intervals correspond to lower levels. For example, under a discrete scale from level 0 to level 5, distances from 0m to 10m can be mapped to level 5, 10m to 25m to level 4, 25m to 50m to level 3, 50m to 100m to level 2, 100m to 200m to level 1, and anything beyond 200m to level 0.

[0039] Alternatively, screen projection height or screen projection area can be used instead of pure spatial distance. For example, areas with a bounding box screen projection area greater than approximately 20,000 pixels can be mapped to a higher level, while areas smaller than approximately 1,000 pixels can be mapped to a lower level. This allows the same bounding box to achieve a level that better reflects its contribution to the image, both in wide-angle and narrow-angle viewing. Finer distance intervals increase the sensitivity to level changes; wider distance intervals reduce the frequency of level changes.

[0040] The protection level reflects the degree to which the bounding box is affected by the fidelity constraint region. If the bounding box does not overlap with the fidelity constraint region, the protection level is 0. If it overlaps with the influence range of an alarm POI or a video texture projection area, a higher protection level can be determined based on the overlap ratio, alarm level, video texture update status, or business importance. For example, if the overlap ratio between the bounding box and a normal video texture projection area reaches 20%, it can be set to level 2; if the overlap ratio reaches 50%, it can be set to level 3; if the area is also within the influence range of a high-level alarm POI, it can be set to level 4 or 5. This approach avoids reducing the rendering accuracy of critical areas simply because of distance.

[0041] The reconstruction cost level is determined based on the color, opacity, and depth dispersion of multiple 3D Gaussian points within a single bounding box, as well as the distribution density of these 3D Gaussian points within the screen projection area of ​​the bounding box in the current field of view. The more dispersed the color, opacity, or depth, the more likely color drift, edge misalignment, or abrupt changes in transparency will occur when approximating with a representative Gaussian point; the lower the point distribution density, the higher the risk of local details being swallowed up by the representative Gaussian point. Therefore, the reconstruction cost level increases with increasing dispersion and decreases with decreasing point distribution density.

[0042] As an example, when the normalized color variance is less than 0.05, the depth dispersion is less than 0.03, and the point distribution density is high, the reconstruction cost level can be set to level 1 or 2; when the normalized color variance is greater than 0.20, the depth dispersion is greater than 0.15, or the point distribution within the bounding box is significantly sparse, the reconstruction cost level can be set to level 4 or 5. This level can be obtained from statistics of the current frame, or it can be obtained by smoothing statistical values ​​from the most recent 8 to 16 frames to reduce level jitter caused by single-frame noise.

[0043] Once the line-of-sight level, protection level, and reconstruction cost level are determined, the level determination module takes the highest level among the three as the lower limit of the target level. For example, if a bounding box has a line-of-sight level of 2, a protection level of 4, and a reconstruction cost level of 3, then the lower limit of the target level for that bounding box is level 4. Using the highest level instead of the average level ensures that the bounding box will not be oversimplified if high accuracy is required in any aspect of viewing distance, fidelity protection, or reconstruction cost.

[0044] The scheduling module maintains the current rendering level of the bounding box in each frame and gradually brings it closer to the target level's lower limit. If the current rendering level is below the target level's lower limit, it is increased by one level; if the current rendering level is above the target level's lower limit and resources are scarce or its contribution to the image is low, it is decreased by one level within a range not lower than the corresponding minimum allowable level. Adjusting the setting by a maximum of one level per frame allows the rendering state to change gradually across multiple frames, avoiding flickering caused by single-frame jumps. For example, if a bounding box's current rendering level is 1, and the target level's lower limit is 4, it can be adjusted to level 2, 3, and 4 sequentially over three consecutive frames, instead of jumping directly to level 4 within a single frame.

[0045] The scheduling module determines the rendering method of the bounding box based on screen error, protection level, GPU budget, and view frustum range: If the bounding box is outside the view frustum, it is removed from the rendering module's sorting range; if the bounding box is inside the view frustum and the screen error does not exceed the first tolerance, it is set to a summary state; if the screen error exceeds the first tolerance but does not exceed the second tolerance, the 3D Gaussian points within the bounding box, ranked by the product of the screen projection area and opacity, are expanded at a preset ratio, and the remaining 3D Gaussian points are summarized into a representative Gaussian point; if the screen error exceeds the second tolerance and the protection level of the bounding box is non-zero, the bounding box is set to an expanded state when the GPU budget can accommodate the expansion load, and a partial expansion method is adopted according to the lowest rendering level corresponding to the protection level when the GPU budget is insufficient; if the screen error exceeds the second tolerance and the protection level of the bounding box is zero, the bounding box is set to an expanded state or the partial expansion ratio is increased when the GPU budget is sufficient, and a partial expansion method or the summary state is adopted when the GPU budget is insufficient; wherein, the first tolerance is less than the second tolerance.

[0046] The first tolerance is used to distinguish between boundaries that can be aggregated and those that need to be partially expanded. When setting it, several typical viewpoints can be selected to statistically analyze the distribution of the normalized screen projection area, normalized depth dispersion, and normalized color variance of the bounding boxes. It can also be combined with offline calibration, historical caching, or fully expanded reference results from sampled viewpoints to determine the proxy error level that is difficult for the observer to perceive or has no impact on business judgment as a candidate value for the first tolerance. At runtime, it is not necessary to perform a full expansion comparison of all bounding boxes every frame.

[0047] If screen errors are expressed as normalized pixel deviation or bounding box projection width ratio, the first tolerance can be determined based on typical viewing angles, historical cache, or fully expanded sample results. The lower limit of the first tolerance should be higher than the natural fluctuations caused by rendering quantization errors, pose jitter, or inter-frame noise, to avoid frequent partial expansion triggered by minor errors; its upper limit should be lower than the error level at which discernible detail loss begins to appear. For quality-priority scenarios, the first tolerance can be reduced; for frame-rate-priority scenarios, it can be appropriately increased.

[0048] The second tolerance is used to distinguish partial expansion from full expansion. It uses the same normalized proxy error scale as the first tolerance and forms a stable interval higher than the first tolerance. When setting it, the upper limit is the maximum acceptable detail loss in the protected area. As an example, when the screen error is small, it enters the summary state. When the screen error is between the first tolerance and the second tolerance, the partial expansion method is adopted. When the screen error exceeds the second tolerance, it is combined with the protection level and GPU budget to determine full expansion or partial expansion.

[0049] As a non-limiting example, when using a normalized proxy error scale from 0 to 1, the first tolerance can be taken as 0.05 to 0.15, and the second tolerance can be taken as 0.20 to 0.35, and the first tolerance is less than the second tolerance; the specific values can be calibrated according to the target frame rate, display resolution, picture quality requirements, and GPU budget status. For scenarios where the camera moves relatively smoothly, such as indoor patrol inspection and digital twin monitoring, lower first and second tolerances can be adopted; for scenarios of large-scale roaming or long-distance viewing, the first and second tolerances can be appropriately increased to reduce the expansion frequency.

[0050] The preset ratio is used to control the number of three-dimensional Gaussian points expanded in the partial expansion mode. This ratio can be selected within the range of 0 < r < 1, where r represents the ratio of the number of three-dimensional Gaussian points participating in the expansion to the total number of three-dimensional Gaussian points in the current bounding box. The lower limit of r should ensure that at least several three-dimensional Gaussian points that contribute the most to the current picture are covered, such as points with a large screen projection area or high opacity; the upper limit of r should be less than 1 to retain the computational savings of partial expansion compared to the expanded state. When setting the ratio, the three-dimensional Gaussian points in the bounding box can be sorted first according to the product of the screen projection area and opacity, and then the picture quality and the number of activated points can be compared under different candidate ratios.

[0051] As an example, a lower expansion ratio can be taken for ordinary areas, and a higher expansion ratio can be taken for building facades with more edge details, video texture edges, or warning areas. If the GPU load is high, this ratio can be reduced; if edge blurring or color blockiness appears in the picture, this ratio can be increased. The above values are only used to illustrate the setting method and do not limit the specific values.

[0052] The GPU budget represents the total number of activated three-dimensional Gaussian points or equivalent rendering load that the GPU is suitable for processing per frame under the current hardware and target real-time requirements. This budget can be determined through trial rendering in the startup phase or offline phase: gradually increase the number of three-dimensional Gaussian points participating in sorting and rendering, record the load level when the actual frame rate starts to be lower than the target frame rate, and then leave a margin below this load level as the GPU budget.

[0053] For example, on a device targeting a frame rate of 60fps, the number of critical activation points at which the frame rate begins to drop can be determined through trial rendering, and then a safety margin can be reserved as the GPU budget. If the same device is simultaneously performing video decoding and texture updates, the budget should be further reduced to allow for computational margin. The lower limit of the budget should be able to accommodate the total number of representative Gaussian points, partially expanded points, or fully expanded points corresponding to the non-zero protection level region at the minimum allowed rendering level. If the hardware performance is strong or the target frame rate is low, the budget can be increased; if the same device also needs to perform video decoding, texture updates, or other graphics tasks, the budget should be reduced.

[0054] like Figure 3 As shown, after determining the current rendering level of a single bounding box, the scheduling module jointly determines the rendering method of that bounding box in the current frame based on screen error, protection level, GPU budget, and view frustum. Screen error, protection level, GPU budget, and view frustum serve as the criteria for judgment, corresponding to state outputs such as summary state, partial unfolding mode, unfolded state, and removal from the sorting range. When the screen error is small, the current bounding box can enter the summary state and participate in rendering as a representative Gaussian point; when the screen error is in the intermediate range, the current bounding box can adopt the partial unfolding mode; when the screen error is large and the bounding box is constrained by the protection level, the current bounding box can enter the unfolded state if the GPU budget allows, and maintain a partial unfolding mode that is not lower than the protection level requirement when the budget is insufficient; when the screen error is large but the protection level is zero, the current bounding box can choose between the unfolded state, partial unfolding mode, or summary state based on the remaining GPU budget; when the current bounding box is outside the view frustum, it can be removed from the sorting range of the current frame. Figure 3 The lines in the diagram represent the correspondence between the decision criteria and the status output, and do not represent independent method flows.

[0055] After determining the rendering method for each bounding box, the system generates representative Gaussian points and unfolded 3D Gaussian points for the current frame to participate in rendering. The position of the representative Gaussian point can be obtained by weighting the positions of multiple 3D Gaussian points within the corresponding bounding box by opacity; color and opacity can also be obtained by weighted aggregation by opacity. The covariance of the representative Gaussian point can be obtained by aggregating the covariances of multiple 3D Gaussian points within the bounding box by opacity or screen contribution weight. Alternatively, a screen ellipse display kernel can be generated based on the screen projection range of the 3D Gaussian points within the bounding box under the current view, so that the representative Gaussian point has a scale and orientation approximately similar to the original set of 3D Gaussian points. Screen error can be determined by comprehensively considering the normalized screen projection area, normalized depth dispersion, and normalized color variance. The larger the projection area, the more dispersed the depth, or the greater the color difference, the more obvious the deviation will be when using representative Gaussian points for approximation.

[0056] For example, the normalized screen projection area can be obtained by dividing the bounding box screen projection area by the current screen area; the normalized depth dispersion can be obtained by dividing the standard deviation of the 3D Gaussian point depth within the bounding box by the average depth of the bounding box or the scene depth range; and the normalized color variance can be obtained by normalizing the color channel value range within the bounding box. These three factors are then weighted to obtain the runtime screen error. During offline calibration or sampling evaluation, the average color difference, edge position deviation, and depth deviation within the bounding box projection area between the fully expanded result and the approximate result representing the Gaussian point can be used as calibration criteria to adjust the weights or tolerance ranges of the three proxy features mentioned above. However, this is not a mandatory screen error calculation step performed for each frame.

[0057] On the GPU side, the rendering module places the representative Gaussian point and the unfolded 3D Gaussian point entering the current frame into the same queue, sorting them uniformly from farthest to closest to the camera before rendering. To reduce mixing jitter when depths are similar, both the representative Gaussian point and the unfolded 3D Gaussian point can be assigned a sorting key. The sorting key includes a bounding box identifier, a depth value, and a type identifier. The type identifier is used to distinguish between the representative Gaussian point and the unfolded 3D Gaussian point, while the bounding box identifier ensures stable sorting when depth values ​​are the same or close. After rendering is complete, the rendering module calculates the actual frame rate and the total number of active 3D Gaussian points for the current frame and feeds this data back to the scheduling module.

[0058] Regarding the fidelity constraint region and its related components: As a further implementation of the system, the fidelity constraint region includes the alarm POI influence range and the video texture projection area; correspondingly, the system may also include an alarm monitor, a video projector, and a protection level merger. The aforementioned alarm monitor, video projector, and protection level merger are used to determine or merge protection levels, and are further limitations based on the basic system structure, not... Figure 1 The necessary modules in the basic system structure shown.

[0059] The alarm monitor generates alarm Points of Interest (POIs) and their affected areas based on alarm events in a 3D Gaussian scene. It then determines the corresponding protection level based on the alarm event level and the overlap ratio between the bounding box and the alarm POI's affected area. Higher alarm event levels, or greater overlap ratios between the bounding box and the alarm's affected area, result in higher protection levels. For example, an alarm level of 1 with an overlap ratio less than 20% can be set to protection level 2; a level 2 alarm with an overlap ratio between 20% and 50% can be set to protection level 3; and a level 3 alarm or an overlap ratio exceeding 50% can be set to protection level 4 or 5. This way, even if the area near the alarm is at a considerable line-of-sight, the lower limit of the target protection level can be raised through the protection level.

[0060] The video projector can project the dynamic textures of multiple video streams onto the corresponding bounding box-expanded 3D Gaussian points. The area where the bounding box is located is defined as the video texture projection area. The protection level is determined based on the proportion of the projected area of ​​the bounding box within the video texture projection area and the update frequency of the corresponding video texture. The video texture update frequency can be obtained from the actual frame statistics of the video stream, or it can be determined by the nominal frame rate of the video source and recent frame drops.

[0061] As an example, for a 30fps video source, if the effective frame rate has remained above 24fps for the last 2 seconds, and the overlap between the bounding box and the video texture projection area exceeds 30%, the protection level can be set to level 3 or 4. If the effective frame rate is below 5fps and the image changes little, the protection level can be reduced by 1 level. The higher the update frequency, the richer the dynamic information carried by the area, and usually a higher protection level is required. If a video stream updates at a low frequency or is close to static for a long time, the corresponding area can have its current rendering level reduced first when resources are scarce.

[0062] The protection level merger is used to determine the highest protection level among multiple protection levels for a single bounding box affected by multiple alarm POIs or located within a video texture projection area. For example, if a bounding box receives a protection level of 3 due to its video texture projection area and a protection level of 5 due to the influence range of an alarm POI, then the final protection level is 5. Using the highest level avoids dilution when multiple fidelity sources are superimposed; if any source requires high fidelity, the bounding box participates in subsequent scheduling according to the higher requirement.

[0063] The degradation strategy under resource constraints is triggered by the actual frame rate, the total number of activated 3D Gaussian points, and the GPU budget. When the actual frame rate is lower than the preset frame rate or the total number of activated 3D Gaussian points exceeds the GPU budget, the scheduling module prioritizes selecting degradation targets from bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowable level, and reduces their current rendering level within a range not lower than the minimum allowable level. When there are many candidate bounding boxes with a protection level of zero, degradation targets can be selected in the following order: screen projection area from small to large, viewing distance from far to near, screen error from low to high, the time of the most recent state switch from late to early, and reconstruction cost level from low to high. This allows bounding boxes with smaller image contribution and lower switching cost to enter the partial expansion mode or the aggregated state first. If the GPU budget is still exceeded, the current rendering level of bounding boxes within the video texture projection area and whose corresponding video texture update frequency is lower than the preset frequency is further reduced; for bounding boxes with a non-zero protection level, the minimum allowable level is not lower than their protection level.

[0064] The preset frame rate is used to determine whether the current rendering meets real-time requirements. Its setting can be based on the display device's refresh rate, business latency requirements, or user interaction requirements. For interactive roaming scenarios, the preset frame rate can be close to the display device's refresh rate setting; for non-interactive inspection or background preview scenarios, it can be set according to the acceptable frame rate for the business. When the actual frame rate is lower than the preset frame rate for multiple consecutive frames, a degradation strategy is triggered. The higher this trigger value, the earlier the system degrades to protect the frame rate; the lower the trigger value, the more inclined it is to preserve image quality.

[0065] The preset frequency is used to determine the dynamic range of the video texture projection area and can be determined based on the actual update frequency distribution of the corresponding video source. When setting this frequency, the arrival frequency of effective frames for each video stream can be statistically analyzed over a period of time, and frequencies significantly lower than the normal video frame rate and with minimal image changes can be used as candidate values. This value should be higher than short-term fluctuations caused by occasional frame drops or network jitter, but lower than the normal update frequency of the video source to avoid misclassifying normal video as low-frequency video. When GPU budgets are limited, video texture projection areas with frequencies lower than the preset frequency can have their current rendering level reduced.

[0066] During the layer switching process, the scheduling module can set a transition duration. When a single bounding box transitions from a summarized state to an expanded state, the scheduling module decreases the contribution coefficient of the representative Gaussian point frame by frame during the transition duration, and increases the contribution coefficient of the expanded 3D Gaussian point set frame by frame. Each expanded 3D Gaussian point retains its original opacity and is multiplied by the corresponding contribution coefficient when participating in opacity blending. When transitioning from an expanded state to a summarized state, the contribution coefficients of the representative Gaussian point and the expanded 3D Gaussian point set are adjusted frame by frame in the opposite direction.

[0067] Transition duration represents the number of frames or time span required to complete one state switch, and can be set according to the target frame rate, camera movement speed, and image continuity requirements. The lower limit should cover at least several consecutive frames to ensure that the transition between the representative Gaussian point and the unfolded point is not abrupt in a single frame; the upper limit should not be so long as to produce obvious motion blur or cause lag in detail recovery. When the camera moves quickly, the transition duration can be shortened; when observing statically or slowly panning, it can be appropriately lengthened to obtain a smoother change.

[0068] When the actual frame rate drops or the total number of activated 3D Gaussian points exceeds the GPU budget during the transition period, the scheduling module can add a scheduling penalty or reduce the expansion priority correction for bounding boxes that have undergone state switching, making them more inclined to aggregate state or partial expansion mode in the future. This scheduling penalty or expansion priority correction only participates in the sorting when resources are tight and does not change the reconstruction cost level determined by color dispersion, opacity dispersion, depth dispersion, and point distribution density. When multiple bounding boxes compete for the GPU budget at the same time, they can be sorted in order of protection level from high to low, screen projection area from large to small, reconstruction cost level from high to low, and the time of the most recent state switch from early to late, and the bounding boxes with higher sorting are prioritized to be set to or kept in the expanded state; for bounding boxes that have already added scheduling penalties, their expansion priority can be appropriately shifted backward based on the above sorting.

[0069] As an example, if the GPU budget is 2.6 million activation points and the total number of activation points to be rendered reaches 3.1 million, the scheduling module can first reduce the bounding boxes with a protection level of 0 and a current rendering level higher than the minimum allowable level by one level. If this still exceeds the budget, it can then reduce the bounding boxes with lower protection levels in low-frequency video texture areas that are still higher than the minimum allowable level. This order places key areas, areas that contribute significantly to the image quality, and areas with high switching costs at the forefront, helping to allocate the limited budget to the areas that have the greatest impact on image quality.

[0070] The complete process of this application's method is illustrated below with a typical example: A digital twin scene at the park level consists of approximately 8 million 3D Gaussian points and includes several building facade areas projected with real-time monitoring videos. During scene loading, the CPU divides the 3D Gaussian scene into multiple bounding boxes and records the correspondence between the bounding boxes and the 3D Gaussian points. For example, the number of 3D Gaussian points in a single leaf bounding box can be controlled between approximately 2,000 and 8,000, and further subdivisions can be made for areas such as building entrances, road intersections, and video walls that the camera frequently passes through. At the beginning of each frame, the level determination module determines the target level lower limit for each bounding box based on the camera distance, fidelity constraint area, and reconstruction cost statistics. For example, bounding boxes within 0m to 10m are set to a higher line-of-sight level, and bounding boxes within a radius of 5m to 20m of alarm POIs are set to a higher protection level.

[0071] The scheduling module then adjusts the current rendering level frame by frame and determines whether the bounding box enters the summary state, partially expands, expands, or is removed from the sorting range based on screen error, GPU budget, and view frustum. After the GPU renders the representative Gaussian points and the expanded 3D Gaussian points in a unified sorted manner, it feeds back the actual frame rate and the total number of active 3D Gaussian points to the scheduling module. If the camera moves forward rapidly, causing a large number of bounding boxes to expand simultaneously, the total number of active points increases and the actual frame rate decreases. The scheduling module prioritizes reducing the bounding boxes with a protection level of 0 that is higher than the minimum allowable level. If necessary, it further reduces the video texture projection area with an effective update frequency lower than the preset frequency, thereby restoring the frame rate while keeping the monitored video area clearly distinguishable.

[0072] like Figure 4 As shown, the method flow of this embodiment includes hierarchy determination, frame-by-frame scheduling, state branching, unified rendering, and feedback constraints. First, S101 to S103 are executed on the CPU side, that is, the 3D Gaussian scene is divided into multiple bounding boxes, the view distance level, protection level, and reconstruction cost level of each bounding box are determined, and the highest level among the three is taken as the target level lower limit; then S104 is executed, and the current rendering level of the bounding box is adjusted frame by frame according to the target level lower limit; in S105, the scheduling module determines the rendering method of the bounding box according to the screen error, protection level, GPU budget, and view frustum, and generates status outputs such as summary state, partial expansion method, expansion state, or removal from the sorting range. In this process, bounding boxes in the summary state, partial expansion mode, and expansion state enter the rendering point generation and unified queue, while bounding boxes removed from the sorting range do not participate in the current frame rendering. Then, S106 and S107 are executed to generate representative Gaussian points or expanded 3D Gaussian points, and they are uniformly sorted and rendered on the GPU side. Finally, S108 is executed to feed back the actual frame rate of the current frame and the total number of activated 3D Gaussian points to the scheduling module, which is used to constrain the current rendering level adjustment, rendering method determination, and downgrade processing when necessary for subsequent frames.

[0073] It should be noted that the numerical values ​​in the above embodiments are used to illustrate the source and adjustment method of each parameter setting, and do not limit this application to a certain fixed value. The spatial partitioning structure, number of discrete levels, error measurement method, point contribution ranking method, GPU budget, security factor, video update frequency judgment method, and transition curve form can all be implemented using equivalent methods depending on the scene scale, hardware performance, and business image quality requirements. Any substitutions or adjustments made to the above implementation methods under the concept of this application should be included within the protection scope of this application.

[0074] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A CPU level split and GPU driven based three-dimensional Gaussian rendering optimization system, characterized in that, include: The scene splitting module is used to divide a 3D Gaussian scene into multiple bounding boxes on the CPU side and record the correspondence between each bounding box and the 3D Gaussian points within it. The level determination module is used to determine the view distance level, protection level and reconstruction cost level of each bounding box under the same discrete level scale, and take the highest level among the three as the target level lower limit. The higher the level, the higher the required rendering accuracy. The protection level is determined by the overlap relationship between the bounding box and the fidelity constraint region. A protection level of zero indicates that it is not constrained by the fidelity constraint region. The scheduling module is used to adjust the current rendering level of the bounding box to the lower limit of the target level frame by frame, and adjust it by a maximum of one level per frame, and determine the rendering state of the bounding box as a summary state, a partial expansion mode or an expanded state based on the screen error. The rendering module is used to sort and render the representative Gaussian points of the summary state and the 3D Gaussian points of the unfolded state in a unified manner according to the distance from the camera from far to near on the GPU side, and to feed back the actual frame rate of the current frame and the total number of activated 3D Gaussian points to the scheduling module. Here, the representative Gaussian point is obtained by summing multiple three-dimensional Gaussian points within a single bounding box; when the actual frame rate is lower than the preset frame rate, the scheduling module selects a degraded object from the bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowed level, and reduces its current rendering level within a range not lower than the minimum allowed level; for bounding boxes with a non-zero protection level, the minimum allowed level is not lower than its protection level.

2. The system according to claim 1, characterized in that, The fidelity constraint region includes the alarm POI influence range and the video texture projection region; the system also includes an alarm monitor, a video projector, and a protection level merger; The alarm monitor is used to generate alarm points of interest (POIs) and their impact ranges based on alarm events in a 3D Gaussian scene, and to determine the corresponding protection level based on the alarm event level and the overlap ratio between the bounding box and the impact range of the alarm POI. The video projector is used to project the dynamic texture of multiple video streams onto the three-dimensional Gaussian points after the corresponding bounding box is unfolded. The area where the bounding box to be projected is located is determined as the video texture projection area, and the corresponding protection level is determined according to the projection area ratio of the bounding box in the video texture projection area and the update frequency of the corresponding video texture. The protection level merger is used to take the highest of the multiple protection levels corresponding to a single bounding box that is affected by multiple alarm POIs or is located in the video texture projection area as the protection level of the bounding box.

3. The system according to claim 1, characterized in that, The level determination module determines the reconstruction cost level based on the color dispersion, opacity dispersion, and depth dispersion of multiple three-dimensional Gaussian points within a single bounding box, as well as the distribution density of the three-dimensional Gaussian points within the screen projection area of ​​the bounding box in the current field of view. The reconstruction cost level increases with increasing color dispersion, opacity dispersion, or depth dispersion, and increases with decreasing 3D Gaussian point distribution density.

4. The system according to claim 1, characterized in that, The scheduling module determines the rendering method of the bounding box based on screen error and GPU budget: If the screen error does not exceed the first tolerance, then the bounding box is set to the summary state; If the screen error exceeds the first tolerance but does not exceed the second tolerance, then expand the three-dimensional Gaussian points with the product of the screen projection area and opacity within the bounding box sorted by a preset ratio, and summarize the remaining three-dimensional Gaussian points into a representative Gaussian point. If the screen error exceeds the second tolerance and the protection level of the bounding box is not zero, the bounding box is set to the unfolded state when the GPU budget can accommodate the unfolded load, and a partial unfolding method with a protection level no lower than that is adopted when the GPU budget is insufficient. If the bounding box is outside the view frustum, then the bounding box is removed from the sorting range of the rendering module; wherein the first tolerance is less than the second tolerance.

5. The system according to claim 4, characterized in that, When the actual frame rate is lower than the preset frame rate or the total number of activated 3D Gaussian points exceeds the GPU budget, the scheduling module prioritizes selecting downgrade objects from bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowed level, and reduces their current rendering level within a range not lower than the minimum allowed level. If the GPU budget is still exceeded, the current rendering level of the bounding box that is within the video texture projection area, has a corresponding video texture update frequency lower than the preset frequency, and has a current rendering level higher than the corresponding minimum allowed level is reduced; for bounding boxes with a protection level that is not zero, the minimum allowed level is not lower than its protection level.

6. The system according to claim 1, characterized in that, The position of the representative Gaussian point is obtained by weighting the positions of multiple three-dimensional Gaussian points within the corresponding bounding box according to the opacity. The color and opacity are obtained by weighting the aggregation of multiple three-dimensional Gaussian points within the corresponding bounding box according to the opacity. The display range of the representative Gaussian point is determined according to the screen projection range of multiple three-dimensional Gaussian points within the bounding box under the current field of view. The screen error is determined by the screen projection area, depth dispersion, and color variance of the bounding box.

7. The system according to claim 6, characterized in that, Both the representative Gaussian point and the unfolded 3D Gaussian point are provided with a sorting key. The sorting key includes a bounding box identifier, a depth value, and a type identifier. The type identifier is used to distinguish between the representative Gaussian point and the unfolded 3D Gaussian point. The rendering module first performs mixed rendering of representative Gaussian points and unfolded 3D Gaussian points according to depth value and then according to type identifier in the same queue, and performs stable sorting according to bounding box identifier when the depth values ​​are the same or close.

8. The system according to claim 1, characterized in that, The scheduling module is configured with a transition period; When a single bounding box transitions from a summed state to an expanded state, the scheduling module reduces the contribution coefficient of the representative Gaussian point frame by frame during the transition period, and increases the contribution coefficient of the expanded 3D Gaussian point set frame by frame. Each expanded 3D Gaussian point retains its original opacity and is multiplied by the corresponding contribution coefficient when the opacity is mixed, and the sum of the contribution coefficients of the representative Gaussian point and the expanded 3D Gaussian point set does not exceed 1. When a single bounding box transitions from an expanded state to a summarized state, the scheduling module adjusts the contribution coefficients of the representative Gaussian point and the expanded 3D Gaussian point set frame by frame in the opposite direction during the transition period.

9. The system according to claim 8, characterized in that, When the actual frame rate drops or the total number of activated 3D Gaussian points exceeds the GPU budget during the transition period, the scheduling module adds a scheduling penalty term or reduces the expansion priority correction term for the bounding box that has undergone state switching, so as to improve its priority to be subsequently set to the summary state or partial expansion mode. When multiple bounding boxes compete for GPU budget simultaneously, the scheduling module sorts the multiple bounding boxes in order of protection level from high to low, screen projection area from large to small, reconstruction cost level from high to low, and the time of the most recent state switch from early to late, and prioritizes setting or keeping the bounding box with the highest sorting to an expanded state.

10. A 3D Gaussian rendering optimization method based on CPU hierarchical splitting and GPU driving, characterized in that, include: On the CPU side, the 3D Gaussian scene is divided into multiple bounding boxes, and the correspondence between each bounding box and the 3D Gaussian points within it is recorded. Under the same discrete level scale, the view distance level, protection level and reconstruction cost level of each bounding box are determined, and the highest level among the three is taken as the lower limit of the target level. The higher the level, the higher the required rendering accuracy. A protection level of zero indicates that there is no region constrained by fidelity constraints. The current rendering level of the bounding box is adjusted to the lower limit of the target level frame by frame, with a maximum adjustment of one level per frame; the bounding box is determined to be in a summarized state, a partially expanded state, or an expanded state based on the screen error of the bounding box. On the GPU side, the aggregated state, partial unfolding method, and representative Gaussian points or 3D Gaussian points generated by the unfolded state are uniformly sorted and rendered according to the camera distance from farthest to nearest; the actual frame rate of the current frame and the total number of activated 3D Gaussian points are counted and fed back to the CPU side. When the actual frame rate is lower than the preset frame rate, a degraded object is selected from the bounding boxes with a protection level of zero and a current rendering level higher than the corresponding minimum allowed level, and its current rendering level is reduced within a range not lower than the minimum allowed level. For bounding boxes with a non-zero protection level, the minimum allowed level is not lower than its protection level.

Images