Image rendering optimization method and device, vehicle, storage medium and program product

Abstract

This invention relates to the field of image processing technology, and discloses an image rendering optimization method, apparatus, vehicle, storage medium, and program product. In this invention, a three-dimensional clustering mesh is constructed within the camera's view frustum of the previous rendering frame, avoiding computational waste due to calculations in invisible areas. The access frequency of each cluster unit in the three-dimensional clustering mesh is statistically analyzed. When the camera's view frustum of the current rendering frame has not changed significantly compared to the previous rendering frame, the cluster units are divided into active and inactive units based on the relationship between the access frequency of each cluster unit and a preset access frequency. The current rendering frame only updates the rendering resources of the active units and reuses the rendering resources of the inactive units from the previous frame, avoiding the need to reconstruct the three-dimensional clustering mesh in every frame, thereby reducing the rendering load and effectively utilizing computational resources.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to image rendering optimization methods, apparatus, vehicles, storage media, and program products. Background Technology

[0002] With the development of virtual reality, game engines, and visualization applications, the complexity and dynamism of rendered scenes are growing explosively, and the performance requirements for real-time rendering are constantly increasing.

[0003] In existing technologies, commonly used spatial partitioning methods mainly include fixed voxel partitioning, octree partitioning, and spatial hash partitioning. Fixed voxel partitioning divides the world space evenly into a fixed-size 3D grid; octree partitioning recursively divides the space into eight child nodes, forming a tree structure, which is used to manage sparse scenes; and spatial hash partitioning maps the spatial coordinates of objects to a fixed-size hash table through a hash function.

[0004] In dynamic scenes, existing technologies require the reconstruction or frequent updates of the entire spatial structure every frame during rendering, resulting in high update overhead and increased burden on the graphics processing unit (GPU) and central processing unit (CPU). Summary of the Invention

[0005] This invention provides an image rendering optimization method, apparatus, vehicle, storage medium, and program product to solve the problem that existing technologies require reconstruction of spatial structure every frame in dynamic scenes, resulting in high rendering load.

[0006] In a first aspect, the present invention provides an image rendering optimization method, the method comprising: Obtain the first camera frustum extent of the previous rendered frame, and obtain the 3D clustering mesh constructed within the first camera frustum extent; wherein, the 3D clustering mesh includes multiple clustering units; Determine the rendering resources associated with each cluster unit in the previous rendering frame; Obtain the second camera frustum range of the current rendering frame. When the angle of view change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle of view change, determine the access frequency of each clustering unit in the historical rendering frame. Based on the relationship between access frequency and preset access frequency, multiple clustering units are divided into active units and inactive units, and the rendering resources associated with active units are updated. Reuse the rendering resources associated with the inactive unit in the previous rendering frame, and use the updated rendering resources associated with the active unit to render the image for the current rendering frame.

[0007] In this invention, a 3D clustering mesh is constructed within the camera's view frustum of the previous rendering frame, avoiding the waste of computational resources by calculating invisible areas. The access frequency of each cluster unit in the 3D clustering mesh is statistically analyzed. When the camera's view frustum of the current rendering frame has not changed significantly compared to the previous rendering frame, the cluster units are divided into active and inactive units based on the relationship between the access frequency of each cluster unit and a preset access frequency. The current rendering frame only updates the rendering resources of the active units and reuses the rendering resources of the inactive units from the previous frame, avoiding the need to reconstruct the 3D clustering mesh in every frame, thereby reducing the rendering load and effectively utilizing computational resources.

[0008] In one alternative implementation, after obtaining the three-dimensional clustering mesh constructed within the first camera's view frustum, the method further includes: The rendering information of each cluster unit in the 3D clustering mesh is statistically analyzed; the rendering information includes the number of instances and the access frequency. For each cluster unit, based on the rendering information of the cluster unit, either skip the rendering task of the cluster unit, or split or merge the cluster units.

[0009] This invention uses rendering information such as the number of instances and access frequency of each cluster unit to skip, subdivide, or merge cluster units. It maintains high rendering accuracy for active areas such as the center of the field of view and the foreground region, while reducing the rendering accuracy of inactive areas by merging them, thereby eliminating redundancy and balancing the rendering load.

[0010] In one optional implementation, based on the rendering information of the clustering units, the rendering task of the clustering units is skipped, or the clustering units are split or merged, including: If the number of instances and the access frequency of a clustering unit are both detected to be 0, skip the rendering task of the clustering unit. If the number of instances of a cluster unit exceeds a first quantity threshold and the access frequency exceeds a first frequency threshold, the cluster unit is split. If the number of instances of multiple adjacent cluster units is not greater than a second quantity threshold and the access frequency is not greater than a second frequency threshold, then the multiple adjacent cluster units are merged; wherein, the second quantity threshold is less than the first quantity threshold and the second frequency threshold is less than the first frequency threshold.

[0011] This invention takes into account the often uneven distribution and occlusion relationships of objects in a scene. Based on the number of instances bound to each cluster unit and its access frequency, it skips rendering tasks for empty cluster units, improving rendering efficiency; it further subdivides cluster units with a high number of instances and high access frequency, increasing rendering resolution; and it merges multiple adjacent cluster units with a low number of instances and low access frequency, compressing invalid data. Thus, it optimizes the 3D clustering mesh to improve rendering efficiency and resolution while compressing invalid data.

[0012] In an alternative implementation, after determining the rendering resource associated with each clustering unit in the previous rendering frame, the method further includes: Determine the resource identifier of the rendering resource associated with each cluster unit, store the resource identifier of each cluster unit in the first cluster data buffer, and generate the cluster index of each cluster unit. The clustering index of each clustering unit is stored in the first clustering index buffer; Based on the first clustering data buffer and the first clustering index buffer, the image is rendered on the previous rendering frame.

[0013] This invention stores resource identifiers of rendering resources by constructing a clustering data buffer, generates a clustering index based on the storage location of the resource identifiers, and stores the clustering index by constructing a clustering index buffer. Thus, during image rendering, the corresponding rendering resources can be quickly accessed by reading the clustering data buffer and the clustering index buffer, thereby improving the access efficiency of the shader.

[0014] In one optional implementation, the rendering resources associated with inactive units in the previous rendering frame are reused, and the updated rendering resources associated with active units are used to render the image for the current rendering frame, including: The resource identifiers stored in the inactive cells in the first clustering data buffer are copied to the second clustering data buffer, and a new clustering index for the inactive cells is generated. Determine the new resource identifier of the rendered resource after the active unit is updated, store the new resource identifier of the active unit in the second clustering data buffer, and generate a new clustering index for the active unit; Store the new clustering indexes of active and inactive cells into the second clustering index buffer; Based on the second clustering data buffer and the second clustering index buffer, the current rendering frame is rendered.

[0015] This invention places the incremental update process of clustering units in a background buffer, enabling data updates without interfering with real-time rendering and avoiding rendering anomalies caused by simultaneous data reading and updating. When the camera is stationary or the scene is stable, by reusing the rendering data of inactive units, redundant storage, copying, and recalculation of unchanged data per frame are avoided, significantly saving memory and computing resources and improving applicability on devices with limited computing power.

[0016] In one alternative implementation, the process of constructing the three-dimensional clustering mesh includes: The projection area of ​​the first camera's view frustum in screen space is determined, and the projection area is uniformly divided into multiple two-dimensional grids; where each two-dimensional grid corresponds to a two-dimensional coordinate. The depth space from the camera to each two-dimensional grid is divided into depth layers to obtain a three-dimensional clustered grid; wherein the interlayer spacing of different depth layers gradually increases along the depth direction of the first camera's field of view cone.

[0017] This invention projects the camera's field of view onto a screen space and then divides the projected area into multiple two-dimensional grids at equal intervals. Furthermore, along the depth direction of the camera's field of view, a logarithmic spacing is used to create a layered structure, with each two-dimensional grid corresponding to a stack of clustering units. This constructs a three-dimensional clustering grid with denser clustering units at near distances and sparser units at far distances. This better aligns with the focusing and occlusion patterns of human vision, improving near-field accuracy and reducing the overhead on distant scenes, thus balancing the high-precision details of near-field scenes with the low-density processing of distant scenes.

[0018] In one optional implementation, the projection area is uniformly divided into multiple two-dimensional grids, including: Determine the current performance of the target device's graphics processor; Determine a cluster density that matches the current performance, and then uniformly divide the projected region into multiple two-dimensional grids based on the cluster density; Depth layering is performed on the depth space from the camera to each 2D grid, including: The depth space from the camera to each 2D grid is divided into multiple depth layers based on the clustering density.

[0019] This invention is designed for target devices with limited computing power, such as vehicle terminals and mobile devices. During runtime, it can dynamically adjust the clustering density of the 3D clustering mesh based on the current performance of the target device's graphics processor, thereby achieving a match between rendering accuracy and hardware performance and reducing the hardware load on the target device.

[0020] In one alternative implementation, the method further includes: When the angle of view change of the second camera's frustum range compared to the first camera's frustum range exceeds a preset angle of view change, a three-dimensional clustering mesh is reconstructed within the second camera's frustum range.

[0021] When the rendering scene changes, this invention addresses the changes in depth information and associated rendering resources caused by changes in the camera's view frustum. It reconstructs a 3D clustering mesh to update the associated rendering resources and render the changed scene view.

[0022] In a second aspect, the present invention provides an image rendering optimization apparatus, the apparatus comprising: The first processing module is used to obtain the range of the first camera frustum of the previous rendering frame and to obtain the three-dimensional clustering mesh constructed within the range of the first camera frustum; wherein, the three-dimensional clustering mesh includes multiple clustering units; The second processing module is used to determine the rendering resources associated with each clustering unit in the previous rendering frame; The third processing module is used to obtain the second camera frustum range of the current rendering frame. When it is detected that the angle change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle change, the access frequency of each clustering unit in the historical rendering frame is determined. The fourth processing module is used to divide multiple clustering units into active and inactive units based on the relationship between the access frequency and the preset access frequency, and to update the rendering resources associated with the active units. The fifth processing module is used to reuse the rendering resources associated with inactive units in the previous rendering frame and use the updated rendering resources associated with active units to perform image rendering on the current rendering frame.

[0023] Thirdly, the present invention provides a vehicle comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the image rendering optimization method described in the first aspect or any corresponding embodiment thereof.

[0024] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the image rendering optimization method described in the first aspect or any corresponding embodiment thereof.

[0025] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the image rendering optimization method described in the first aspect or any corresponding embodiment thereof.

[0026] The beneficial effects of this invention are as follows: In this invention, a 3D clustering mesh is constructed within the camera's view frustum of the previous rendering frame, avoiding the waste of computational resources by calculating invisible areas. The access frequency of each cluster unit in the 3D clustering mesh is statistically analyzed. When the camera's view frustum of the current rendering frame has not changed significantly compared to the previous rendering frame, the cluster units are divided into active and inactive units based on the relationship between the access frequency of each cluster unit and a preset access frequency. The current rendering frame only updates the rendering resources of the active units and reuses the rendering resources of the inactive units from the previous frame, avoiding the need to reconstruct the 3D clustering mesh in every frame, thereby reducing the rendering load and effectively utilizing computational resources. Attached Figure Description

[0027] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first process of the image rendering optimization method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the incremental update process according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a second process of the image rendering optimization method according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the construction of a three-dimensional clustering mesh according to an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the optimization process of a three-dimensional clustering mesh according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the third process of the image rendering optimization method according to an embodiment of the present invention; Figure 8 This is a structural block diagram of an image rendering optimization apparatus according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the hardware structure of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0031] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0032] As an optional application scenario of this invention, such as Figure 1 As shown, the image rendering optimization system may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0033] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0034] Spatial partitioning methods in related technologies, such as fixed voxels, octrees, or spatial hashes, often have the following shortcomings: 1) High update overhead: In dynamic scenes, the entire spatial structure needs to be reconstructed or frequently updated every frame, which increases the burden on the GPU and CPU; 2) Complex data structure: The indexing and access logic is lengthy, making it difficult for the shader to efficiently obtain the required rendering resource information; 3) Uneven distribution: Fixed voxel meshes or tree structures cannot take into account both near details and far sparseness, resulting in low mesh culling efficiency and memory waste.

[0035] This invention provides an image rendering optimization method. When the camera frustum range of the current rendering frame has not changed significantly compared to the previous rendering frame, clustering units are divided into active and inactive units. The current rendering frame only updates the rendering resources of the active units and reuses the rendering resources of the inactive units from the previous frame. This avoids the need to reconstruct the 3D clustering mesh in every frame, thereby reducing the rendering load and effectively utilizing computing resources.

[0036] According to an embodiment of the present invention, an embodiment of an image rendering optimization method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] This embodiment provides an image rendering optimization method, which can be used in the aforementioned terminal devices, such as vehicle-mounted terminals. Figure 2 This is a flowchart of an image rendering optimization method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the range of the first camera frustum of the previous rendering frame, and obtain the three-dimensional clustering mesh constructed within the range of the first camera frustum; wherein, the three-dimensional clustering mesh includes multiple clustering units.

[0038] Specifically, before each frame rendering begins, the projection area of ​​the camera's view frustum corresponding to that rendering frame is determined in screen space, as well as the depth space from the camera to that projection area, using the camera's view frustum range as a reference. The projection area and depth space are then meshed to obtain multiple cluster cells. A three-dimensional cluster mesh is constructed within the camera's view frustum range to avoid wasting computational resources by calculating invisible areas.

[0039] Step S202: Determine the rendering resources associated with each cluster unit in the previous rendering frame.

[0040] Specifically, for each rendering frame, the rendering resources required, such as object instance models, light sources, and rendering materials, are determined. The distribution range of these rendering resources in the projection area and their corresponding distribution interval in depth space are then determined. The rendering resource is then added to the association list of all cluster units covered by its distribution range and interval. By repeating the above process, the rendering resources associated with each cluster unit are obtained.

[0041] Step S203: Obtain the second camera frustum range of the current rendering frame. When it is detected that the angle of view change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle of view change, determine the access frequency of each clustering unit in the historical rendering frame.

[0042] Specifically, when the rendering frame transitions from the previous frame to the current frame, the second camera frustum range corresponding to the current rendering frame is obtained. The first depth direction (i.e., the camera center direction) of the first camera frustum range and the second depth direction of the second camera frustum range are determined, and the angle between the second depth direction and the first depth direction is calculated to obtain the degree of change in viewpoint from the previous frame to the current frame.

[0043] Furthermore, such as Figure 3 As shown, the system determines whether the change in viewing angle exceeds a preset limit. If it does not, it indicates that the camera viewpoint and the rendered scene have not changed significantly, and the rendering resources required for the current frame have not been updated substantially compared to the previous frame. To reduce the redundant computational overhead of the 3D clustering mesh structure in each frame, the access frequency of each clustering unit in historical rendering frames is recorded. This access frequency represents the frequency with which the clustering unit participates in intra-frame rendering or lighting calculations; the higher the access frequency, the higher the usage frequency of the clustering unit.

[0044] In some embodiments, a time-sliding window mechanism (e.g., including 5 rendering frames) is used to record the access frequency of each cluster unit in each past rendering frame, and the frequency is smoothed by an exponentially weighted average. The calculated average value is used as the access frequency in the historical rendering frames, thereby predicting the importance of each cluster unit in future rendering.

[0045] Step S204: Based on the relationship between the access frequency and the preset access frequency, the multiple clustering units are divided into active units and inactive units, and the rendering resources associated with the active units are updated.

[0046] Specifically, see again Figure 3 For each cluster unit, it is determined whether the access frequency of the cluster unit exceeds the preset access frequency. If it does, the cluster unit is classified as an active unit and its associated rendering resources are updated; if it does not exceed the preset access frequency, the cluster unit is classified as an inactive unit.

[0047] It should be noted that the preset access frequency can be determined by a preset percentage (which can be adjusted according to the actual scenario). For example, the top K% of clustered units with the highest access frequency can be classified as active units and used as the targets to be updated in the current rendering frame. Inactive units that do not trigger significant changes in viewpoint or occlusion updates can reuse the rendering resource content cached in the previous frame.

[0048] Step S205: Reuse the rendering resources associated with the inactive unit in the previous rendering frame, and use the updated rendering resources associated with the active unit to render the image for the current rendering frame.

[0049] Specifically, the old rendering resources associated with inactive units in the previous frame and the updated rendering resources of active units are extracted, and the image rendering task of the current rendering frame is executed.

[0050] The image rendering optimization method provided in this embodiment constructs a 3D clustering mesh within the camera's view frustum of the previous rendering frame, avoiding wasted computational resources by calculating invisible areas. The access frequency of each cluster unit in the 3D clustering mesh is statistically analyzed. When the camera's view frustum of the current rendering frame has not changed significantly compared to the previous rendering frame, the cluster units are divided into active and inactive units based on the relationship between the access frequency of each cluster unit and a preset access frequency. The current rendering frame only updates the rendering resources of the active units and reuses the rendering resources of the inactive units from the previous frame, avoiding the need to reconstruct the 3D clustering mesh in every frame, thereby reducing the rendering load and effectively utilizing computational resources.

[0051] This embodiment provides an image rendering optimization method, which can be used in the aforementioned terminal devices, such as vehicle-mounted terminals. Figure 4 This is a flowchart of an image rendering optimization method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S401: Obtain the range of the first camera frustum of the previous rendering frame, and obtain the three-dimensional clustering mesh constructed within the range of the first camera frustum; wherein, the three-dimensional clustering mesh includes multiple clustering units.

[0052] In some alternative implementations, the process of constructing a three-dimensional clustering mesh includes: Step a1: Determine the projection area of ​​the first camera's viewing cone in the screen space, and divide the projection area evenly into multiple two-dimensional grids; where each two-dimensional grid corresponds to a two-dimensional coordinate.

[0053] Specifically, the current performance of the target device's graphics processor is determined. Then, a clustering density matching the current performance is determined, and the projected region is uniformly divided into multiple two-dimensional grids based on the clustering density.

[0054] In this embodiment, the GPU performance of the target device, such as an in-vehicle terminal, is read in real time, and a matching clustering density dimX×dimY×dimZ is selected based on the GPU performance, where dimX is the number of columns of the two-dimensional grid, dimY is the number of rows of the two-dimensional grid, and dimZ is the number of deep hierarchical divisions.

[0055] For example, when the GPU performance is lower than a preset GPU performance threshold, the clustering density is 16×9×24 (16 columns, 9 rows, 24 depth layers), which uniformly divides the projection area into a 16×9 two-dimensional grid; when the GPU performance is not lower than the GPU performance threshold, the clustering density is 32×18×48 (32 columns, 18 rows, 48 ​​depth layers), which uniformly divides the projection area into a 32×18 two-dimensional grid.

[0056] Step a2: Perform depth layering on the depth space from the camera to each two-dimensional grid to obtain a three-dimensional clustered grid; wherein, the interlayer spacing of different depth layers gradually increases along the depth direction of the first camera's field of view cone.

[0057] Specifically, the depth values ​​corresponding to different depth layers are determined based on the cluster density, and the depth space from the camera to each two-dimensional grid is divided into multiple depth layers based on the depth values. The depth value represents the depth distance from the camera center to the farthest plane center of the corresponding depth layer.

[0058] In some embodiments, the depth values ​​corresponding to different depth layers can be calculated according to the following formula:

[0059] in, This represents the depth value of the (i+1)th depth layer. This represents the near-plane distance of the camera's field of view cone (i.e., the depth value of the first depth layer). The distance in the far plane representing the field of view of the camera cone (i.e., the distance in the far plane) (Depth value of each depth layer).

[0060] For example, when the GPU performance of the target device, such as an in-vehicle terminal, is lower than a preset GPU performance threshold, a clustering density of 16×9×24 (16 columns, 9 rows, 24 depth layers) can be selected to divide the depth space from the camera to each two-dimensional grid into 24 depth layers; when the GPU performance of the target device is not lower than the preset GPU performance threshold, a clustering density of 32×18×48 (32 columns, 18 rows, 48 ​​depth layers) can be selected to divide the depth space into 48 depth layers.

[0061] In this embodiment, to adapt to computing-constrained platforms such as vehicle terminals and mobile terminals, the clustering density of the 3D clustering mesh (such as 16×9×24, 32×18×48) can be dynamically adjusted at runtime according to the GPU performance of the target device, so as to achieve a match between rendering accuracy and hardware performance and reduce the hardware load of the target device.

[0062] In this embodiment, as Figure 5 As shown, after projecting the camera's field of view onto the screen space, the projected area is divided into a two-dimensional grid of size dimX × dimY at equal intervals along the X and Y directions. Furthermore, along the depth direction (Z-axis) of the camera's field of view, the grid is layered using a logarithmic spacing method, with each two-dimensional grid corresponding to a column. A stack of deeply layered clustering units is used to construct a three-dimensional clustering mesh with denser clustering units at close range and sparser clustering units at long range. This better conforms to the focusing and occlusion rules of human vision, improves near-field accuracy and reduces far-field load, thus balancing high-precision details in the near-field with low-density processing in the far-field.

[0063] In some embodiments, during system initialization, the rendering engine can pre-define and allocate a set of buffer resources on the GPU for clustering calculations and data access. These mainly include a clustering constant buffer (ClusterMetaBuffer), a clustering index buffer (ClusterIndexBuffer), a clustering data buffer (ClusterDataBuffer), and an access frequency tracking buffer (ClusterHeatBuffer). This embodiment establishes two sets of data structures (foreground and background) for all buffers, allowing for switching between the foreground and background to achieve parallel execution of data updates and rendering tasks.

[0064] Specifically, the clustering constant buffer ClusterMetaBuffer is used to store the basic parameters of the 3D clustering mesh. These basic parameters include: screen partition dimensions (the partition dimensions of the projected area, dimX and dimY), the number of deep layers, dimZ, and camera frustum range parameters (the near-plane distance of the camera frustum range). Distance in the far plane of the camera's field of view of the vertebral body (and depth values ​​for different depth layers). See the descriptions of the examples below for details on the uses and working principles of other buffers.

[0065] Step S402: Calculate the rendering information of each clustering unit in the 3D clustering mesh; for each clustering unit, based on the rendering information of the clustering unit, skip the rendering task of the clustering unit, or split or merge the clustering unit.

[0066] In some optional implementations, step S402 above includes: Step b1: Calculate the rendering information of each clustering unit in the 3D clustering mesh.

[0067] Specifically, the rendering information of the clustering unit mainly includes the number of instances and the access frequency. The number of instances refers to the number of rendering instances (such as the object instance model, light source, rendering material, etc.) bound to the clustering unit, and the access frequency refers to the frequency of use of the clustering unit in historical rendering frames.

[0068] In some embodiments, during the construction phase of the 3D clustering mesh, the number of instances contained in each cluster unit is recorded to analyze the distribution density of instances bound to the cluster unit. During the rendering process, the pixel depth and coverage relationship of the rendering frame are sampled to analyze the occlusion relationship between instances, pixels are mapped to corresponding cluster units, and the access frequency of the cluster units is statistically analyzed.

[0069] Specifically, the access frequency tracking buffer ClusterHeatBuffer is used to record the access frequency of each clustering unit across frames, providing a reference for the subsequent adaptive update process of the 3D clustering mesh.

[0070] Step b2: If the number of instances and access frequency of a clustering unit are both detected to be 0, skip the rendering task of the clustering unit.

[0071] For example, such as Figure 6 As shown, if a cluster unit has no instance binding and an access frequency of 0 in several rendering frames, i.e., the cluster unit is empty, then the update and access of the cluster unit will be skipped directly in subsequent rendering frames, thereby skipping the rendering task of the cluster unit and improving rendering efficiency.

[0072] Step b3: If the number of instances of a cluster unit exceeds the first quantity threshold and the access frequency exceeds the first frequency threshold, the cluster unit is split.

[0073] For example, see again Figure 6 If the number of instances bound to a cluster unit exceeds the set first quantity threshold, and its access frequency exceeds the first frequency threshold, then it will be split, and the cluster unit with excessively high instance distribution density will be subdivided towards a higher resolution, thereby improving rendering accuracy when there are many instances.

[0074] Step b4: Determine multiple adjacent clustering units of a clustering unit. If the number of instances of multiple adjacent clustering units does not exceed the second quantity threshold and the access frequency does not exceed the second frequency threshold, merge the multiple adjacent clustering units.

[0075] For example, see again Figure 6If the number of instances of multiple adjacent clustering units does not exceed the second quantity threshold and the access frequency does not exceed the second frequency threshold, that is, if the clustering unit binding instances in a certain region are sparse and the frequency of use is low, then the above multiple adjacent clustering units can be merged into a larger clustering unit.

[0076] It should be noted that the first quantity threshold, the second quantity threshold, the first frequency threshold, and the second frequency threshold can be set according to actual needs, as long as the second quantity threshold is less than the first quantity threshold and the second frequency threshold is less than the first frequency threshold.

[0077] This embodiment takes into account the often uneven distribution and occlusion relationships of scene objects. Based on the number of instances bound to each cluster unit and its access frequency, it skips the rendering task of empty cluster units, improving rendering efficiency; it further subdivides cluster units with a high number of instances and high access frequency to improve rendering resolution; and it merges multiple adjacent cluster units with a low number of instances and low access frequency to compress invalid data. This optimizes the 3D clustering mesh to improve rendering efficiency and resolution while compressing invalid data.

[0078] The above embodiments use rendering information such as the number of instances and access frequency of each cluster unit to skip, subdivide or merge the cluster units. High rendering accuracy is maintained for active areas (areas with high access frequency) such as the center of the field of view and the foreground area, while the rendering accuracy of inactive areas is reduced by merging, thereby eliminating redundancy and balancing the rendering load.

[0079] Step S403: Determine the rendering resources associated with each cluster unit in the previous rendering frame.

[0080] Specifically, after executing step S403, the following steps are performed: Step c1: Determine the resource identifier of the rendering resource associated with each cluster unit, store the resource identifier of each cluster unit in the first cluster data buffer, and generate the cluster index of each cluster unit.

[0081] Specifically, the rendering resources of each cluster unit, such as instance models, light sources, and rendering materials, are determined. The resources of each cluster unit are recorded as a data access structure ClusterEntry. This data access structure ClusterEntry contains resource identifiers of the associated rendering resources, such as the bound object instance number instanceID, the local light source identifier lightID, and the rendering material identifier materialID.

[0082] In this embodiment, the ClusterDataBuffer is used to store the associated resource identifiers of each cluster unit, including object instance model number (instanceID), local light source identifier (lightID), rendering material identifier (materialID), etc., supporting parallel reading at the shader end. Based on the ClusterDataBuffer, a first ClusterDataBuffer (which can be located in the foreground for real-time data reading by the rendering task) and a second ClusterDataBuffer (located in the background for background data updates) with the same data structure are established for switching between the foreground and background.

[0083] Furthermore, at the beginning of rendering and before any rendering resource updates occur, the resource identifiers associated with each cluster unit are stored in the first cluster data buffer, arranged consecutively according to the unit identifier order. For each cluster unit, a cluster index is generated based on the starting offset stored in that cluster unit and the number of storage blocks it occupies (count).

[0084] Step c2: Store the clustering index of each clustering unit into the first clustering index buffer.

[0085] Specifically, the ClusterIndexBuffer is used to assign a cluster index to the cell identifier of each cluster unit and record the cluster index of the rendering resource bound to the cluster unit (including the starting offset and the number of storage blocks occupied by the cluster unit).

[0086] It should be noted that the cell identifier (X, Y, Z) of each cluster unit can be obtained based on its corresponding two-dimensional coordinates and depth layering. Morton encoding (Z-order Curve) is used to encode the cell identifier (X, Y, Z) into a one-dimensional identifier, compressing the index space and improving shader memory access continuity, achieving efficient cache coherency and fast GPU-side access. When initially constructing the 3D cluster mesh, all cluster units can be initialized in parallel within the GPU Compute Shader. Each thread is responsible for filling in the initialization data for one cluster unit; for example, setting the starting offset to -1 indicates an "empty cell".

[0087] This embodiment establishes a first clustering index buffer and a second clustering index buffer with the same data structure so that the rendering task can read the foreground buffer and the background buffer in real time to update the data.

[0088] In some embodiments, if multiple clustering units are bound to the same resource identifier, they can share data segments (pointing to the same storage data block via offset+count) to reduce redundant storage.

[0089] In some embodiments, after determining the rendering resource associated with each clustering unit, if the clustering units of the 3D clustering mesh have been merged or split, the clustering index information needs to be re-encoded and updated based on the merged or split clustering units, and simultaneously recorded in the clustering index buffer. Furthermore, the mapping relationship between the unit identifier and the clustering index is corrected in the clustering index buffer.

[0090] This embodiment records the mapping relationship between clustering units and clustering indices, enabling shaders to access rendering resources through indirect jumps, thus saving data structure construction and access overhead.

[0091] Step c3: Based on the first clustering data buffer and the first clustering index buffer, perform image rendering on the previous rendering frame.

[0092] Specifically, during image rendering, the shader can quickly determine the target cluster unit where the current fragment belongs by using the screen coordinates and depth value corresponding to the current fragment, and obtain the unit identifier of the target cluster unit. Then, based on the unit identifier, it reads the cluster index (including offset and count) in the first cluster index buffer, and then extracts the bound resource identifier data from the first cluster data buffer based on the cluster index, calling the corresponding rendering resources for image rendering. The entire data reading process uses simple integer calculations, eliminating the need for complex traversal or hash mapping, thus improving the shader's access efficiency.

[0093] This embodiment stores resource identifiers of rendering resources by constructing a clustering data buffer, generates a clustering index based on the storage location of the resource identifiers, and stores the clustering index by constructing a clustering index buffer. Thus, during image rendering, the corresponding rendering resources can be quickly accessed by reading the clustering data buffer and the clustering index buffer, thereby improving the access efficiency of the shader.

[0094] Step S404: Obtain the second camera frustum range of the current rendering frame. When it is detected that the angle of view change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle of view change, determine the access frequency of each clustering unit in the historical rendering frame.

[0095] In some optional implementations, when the angle of view change of the second camera's view frustum range compared to the first camera's view frustum range exceeds a preset angle of view change, a three-dimensional clustering mesh is reconstructed within the second camera's view frustum range.

[0096] For example, when the rendering scene changes, the camera's view frustum range will change significantly, causing changes in depth information and associated rendering resources. In this case, the 3D clustering mesh is reconstructed to update the rendering resources associated with the 3D clustering mesh and render the changed scene view.

[0097] Step S405: Based on the relationship between the access frequency and the preset access frequency, multiple clustering units are divided into active and inactive units, and the rendering resources associated with the active units are updated. For details, please refer to [link to details]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0098] Step S406: Reuse the rendering resources associated with the inactive unit in the previous rendering frame, and use the updated rendering resources associated with the active unit to render the image for the current rendering frame.

[0099] Specifically, step S406 includes: Step S4061: Copy the resource identifier stored in the first clustering data buffer of the inactive unit to the second clustering data buffer, and generate a new clustering index for the inactive unit.

[0100] Specifically, during cross-frame rendering, the rendering resources of inactive units can be directly reused. Since the first clustering data buffer in the foreground is used by the rendering task to read data in real time, to prevent rendering anomalies caused by reading data during data updates, the resource identifiers of inactive units in the first clustering data buffer are copied to the second clustering data buffer in the background. At the same time, a new clustering index for inactive units is generated based on the starting offset and quantity information stored in the second clustering data buffer.

[0101] Step S4062: Determine the new resource identifier of the rendered resource after the active unit is updated, store the new resource identifier of the active unit in the second clustering data buffer, and generate a new clustering index for the active unit.

[0102] Specifically, since the rendering resources associated with the active unit have been updated, the corresponding new resource identifier (including the new object instance model number instanceID, local light source identifier lightID, rendering material identifier materialID, etc.) is determined based on the updated rendering resources, and the new resource identifier is stored in the second clustering data buffer. At the same time, a new clustering index corresponding to the active unit is generated.

[0103] Step S4063: Store the new clustering indexes of active and inactive cells into the second clustering index buffer.

[0104] Specifically, the new clustering indices corresponding to active and inactive units are stored in the second clustering index buffer according to the unit arrangement order so that the shader can access them quickly.

[0105] Step S4064: Based on the second clustering data buffer and the second clustering index buffer, perform image rendering on the current rendering frame.

[0106] In this embodiment, the updating of clustering units adopts a double buffering mechanism. While one clustering data buffer and one clustering index buffer are updated in the background, the foreground continues to use the other clustering data buffer and one clustering index buffer for data rendering, thus avoiding data conflicts. See again Figure 3 Once the update is complete, the buffer pointer is switched using an atomic flag, thus enabling the switching between the foreground and background buffers without waiting for rendering to complete.

[0107] This embodiment places the incremental update process of clustering units in a background buffer, enabling data updates without interfering with real-time rendering and avoiding rendering anomalies caused by simultaneous data reading and updating. When the camera is stationary or the scene is stable, by reusing the rendering data of inactive units, redundant storage, copying, and recalculation of unchanged data for each frame are avoided, significantly saving memory and computing resources and improving applicability on devices with limited computing power.

[0108] The image rendering optimization scheme of the present invention will be described in detail below with reference to a specific application example.

[0109] like Figure 7 As shown, when constructing the 3D clustering mesh for a rendering frame, the camera's view frustum range is used as the reference range, and clustering units are divided only within the view-related space to reduce the computation of invalid regions. The X and Y axes of the screen space projection area are divided at equal intervals, while the Z-axis in the depth direction is layered according to a logarithmic distribution, resulting in finer clustering units near the edges and sparser ones further away. The 3D clustering mesh structure is constructed in parallel on the GPU, and only the affected clustering units are reconstructed based on camera transformations and scene geometry updates. Intra-frame culling and instance binding computations are also supported.

[0110] This avoids wasting computational resources by generating a 3D clustered mesh only within the camera's field of view frustum. Furthermore, by dividing the projected area into equal intervals along the X and Y axes and by logarithmic layering along the depth direction, it automatically balances high precision for near-field scenes with low density for distant scenes.

[0111] See you again Figure 7The algorithm counts the number of instances and access frequency of each cluster unit, skips empty clusters, adaptively subdivides overly dense clusters, and adaptively merges sparse adjacent clusters to eliminate computational redundancy. This improves the clustering accuracy of hot areas such as the view center and foreground regions, and merges or skips low-frequency regions. For clusters with the same associated resources, a shared index cache is used to compress and encode the optimized clusters, saving video memory bandwidth and storage overhead.

[0112] This invention adaptively merges, subdivides, or skips clusters by statistically analyzing the number of instances and access frequency of each cluster unit. It maintains high precision in active regions, such as the view center and foreground regions, while automatically reducing precision or merging inactive regions, eliminating redundant data and balancing the rendering load. Through this data-driven clustering self-optimization approach, it solves common problems such as hollow cluster structures, uneven density, and data redundancy, enhancing system efficiency and adaptability.

[0113] See you again Figure 7 A unified clustering index buffer and clustering data buffer are constructed. The shader quickly maps the current fragment or ray's screen coordinates to the corresponding clustering index and reads the clustering data buffer using the starting offset and quantity information. This allows for operations such as instance binding, lighting traversal, visibility determination, and dynamic LOD switching, ultimately performing image rendering. Furthermore, all data is stored in the buffer in compressed encoding, reducing random access overhead and improving parallel read efficiency.

[0114] This invention overcomes the problems of high coupling and lengthy shader access paths in traditional binding structures, achieving more efficient cross-channel data binding and decoding.

[0115] See you again Figure 7 The 3D clustering mesh is only fully reconstructed when there are significant changes in the camera or scene. When there are no significant changes in the camera or scene, a time-sliding window is used to count the access frequency of each cluster unit, quickly identifying active and inactive units. Active units are incrementally updated, while inactive units reuse cached rendering resources. A double-buffered clustering structure is adopted to ensure that the old structure can serve the current rendering in parallel during the update process, avoiding frame rate fluctuations.

[0116] This invention avoids the computational redundancy and structural overhead caused by traditional spatial partitioning methods in real-time rendering, improves the real-time performance and view relevance of cluster partitioning, solves the problem of excessive cost of cluster structure reconstruction in high dynamic scenes, and improves cross-frame resource reuse while maintaining real-time performance.

[0117] For low-power devices, different cluster densities (e.g., 16×9×24, 32×18×48, etc.) can be configured, and the appropriate cluster density is automatically selected based on the GPU performance of the target device. Morton encoding is used for cluster coordinate transformation and index compression to reduce computational load. In addition, a proxy table mechanism can be used for clusters with low access frequency, pointing to the actual rendering resource data through a small number of proxy indexes, reducing the GPU computation path.

[0118] This invention can ensure the real-time availability of clustering rendering on low-power platforms and has strong platform adaptability and portability.

[0119] This embodiment also provides an image rendering optimization apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] This embodiment provides an image rendering optimization device, such as... Figure 8 As shown, it includes: The first processing module 801 is used to obtain the range of the first camera frustum of the previous rendering frame and to obtain the three-dimensional clustering mesh constructed within the range of the first camera frustum; wherein the three-dimensional clustering mesh includes multiple clustering units; The second processing module 802 is used to determine the rendering resources associated with each clustering unit in the previous rendering frame; The third processing module 803 is used to obtain the second camera frustum range of the current rendering frame, and when it is detected that the angle change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle change, it determines the access frequency of each clustering unit in the historical rendering frame. The fourth processing module 804 is used to divide multiple clustering units into active units and inactive units based on the relationship between the access frequency and the preset access frequency, and to update the rendering resources associated with the active units. The fifth processing module 805 is used to reuse the rendering resources associated with the inactive unit in the previous rendering frame and use the updated rendering resources associated with the active unit to perform image rendering on the current rendering frame.

[0121] In some optional implementations, the construction process of the three-dimensional clustering mesh in the first processing module 801 includes: The projection area of ​​the first camera's view frustum in screen space is determined, and the projection area is uniformly divided into multiple two-dimensional grids; where each two-dimensional grid corresponds to a two-dimensional coordinate. The depth space from the camera to each two-dimensional grid is divided into depth layers to obtain a three-dimensional clustered grid; wherein the interlayer spacing of different depth layers gradually increases along the depth direction of the first camera's field of view cone.

[0122] In some optional implementations, the first processing module 801 is further configured to: Determine the current performance of the target device's graphics processor; Determine a cluster density that matches the current performance, and then uniformly divide the projected region into multiple two-dimensional grids based on the cluster density; The depth space from the camera to each 2D grid is divided into multiple depth layers based on the clustering density.

[0123] In some alternative implementations, after acquiring the 3D clustering mesh constructed within the first camera's view frustum, the first processing module 801 is further configured to: The rendering information of each cluster unit in the 3D clustering mesh is statistically analyzed; the rendering information includes the number of instances and the access frequency. For each cluster unit, based on the rendering information of the cluster unit, either skip the rendering task of the cluster unit, or split or merge the cluster units.

[0124] In some optional implementations, the first processing module 801 is further configured to: If the number of instances and the access frequency of a clustering unit are both detected to be 0, skip the rendering task of the clustering unit. If the number of instances of a cluster unit exceeds a first quantity threshold and the access frequency exceeds a first frequency threshold, the cluster unit is split. If the number of instances of multiple adjacent cluster units is not greater than a second quantity threshold and the access frequency is not greater than a second frequency threshold, then the multiple adjacent cluster units are merged; wherein, the second quantity threshold is less than the first quantity threshold and the second frequency threshold is less than the first frequency threshold.

[0125] In some alternative implementations, after determining the rendering resources associated with each clustering unit in the previous rendering frame, the second processing module 802 is further configured to: Determine the resource identifier of the rendering resource associated with each cluster unit, store the resource identifier of each cluster unit in the first cluster data buffer, and generate the cluster index of each cluster unit. The clustering index of each clustering unit is stored in the first clustering index buffer; Based on the first clustering data buffer and the first clustering index buffer, the image is rendered on the previous rendering frame.

[0126] In some optional implementations, the third processing module 803 is further configured to: When the angle of view change of the second camera's frustum range compared to the first camera's frustum range exceeds a preset angle of view change, a three-dimensional clustering mesh is reconstructed within the second camera's frustum range.

[0127] In some optional implementations, the fifth processing module 805 is further configured to: The resource identifiers stored in the inactive cells in the first clustering data buffer are copied to the second clustering data buffer, and a new clustering index for the inactive cells is generated. Determine the new resource identifier of the rendered resource after the active unit is updated, store the new resource identifier of the active unit in the second clustering data buffer, and generate a new clustering index for the active unit; Store the new clustering indexes of active and inactive cells into the second clustering index buffer; Based on the second clustering data buffer and the second clustering index buffer, the current rendering frame is rendered.

[0128] The image rendering optimization apparatus provided in this embodiment of the invention can execute the image rendering optimization method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0129] Figure 9 This is a structural schematic diagram of a vehicle provided in an embodiment of the present invention.

[0130] The following is a detailed reference. Figure 9 The diagram illustrates a structural schematic suitable for implementing a vehicle according to an embodiment of the present invention. The vehicle may include a processor (e.g., a central processing unit, graphics processor, etc.) 901, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 902 or a program loaded from memory 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for vehicle operation. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0131] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows the vehicle to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9Vehicles with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0132] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a memory 908, or installed from a ROM 902. When the computer program is executed by the processor 901, it performs the functions defined in the image rendering optimization method of the embodiments of the present invention.

[0133] Figure 9 The vehicle shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0134] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the image rendering optimization method shown in the above embodiments is implemented.

[0135] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0136] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An image rendering optimization method, characterized in that, The method includes: Obtain the first camera frustum range of the previous rendered frame, and obtain the three-dimensional clustering mesh constructed within the first camera frustum range; wherein, the three-dimensional clustering mesh includes multiple clustering units; Determine the rendering resources associated with each cluster unit in the previous rendering frame; Obtain the second camera frustum range of the current rendering frame. When it is detected that the angle change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle change, determine the access frequency of each clustering unit in the historical rendering frame. Based on the relationship between the access frequency and the preset access frequency, the multiple clustering units are divided into active units and inactive units, and the rendering resources associated with the active units are updated. The rendering resources associated with the inactive unit in the previous rendering frame are reused, and the updated rendering resources associated with the active unit are used to render the image for the current rendering frame.

2. The image rendering optimization method according to claim 1, characterized in that, After obtaining the 3D clustering mesh constructed within the field of view of the first camera, the method further includes: The rendering information of each clustering unit in the three-dimensional clustering grid is statistically analyzed; wherein, the rendering information includes the number of instances and the access frequency; For each clustering unit, based on the rendering information of the clustering unit, either skip the rendering task of the clustering unit, or split or merge the clustering unit.

3. The image rendering optimization method according to claim 2, characterized in that, The step of skipping the rendering task of a cluster unit based on the rendering information of the cluster unit, or splitting or merging the cluster units, includes: If the number of instances and the access frequency of the clustering unit are both detected to be 0, skip the rendering task of the clustering unit. If the number of instances of the cluster unit exceeds a first quantity threshold and the access frequency exceeds a first frequency threshold, the cluster unit is split. If the number of instances of the multiple adjacent clustering units of the clustering unit does not exceed a second quantity threshold and the access frequency does not exceed a second frequency threshold, the multiple adjacent clustering units are merged; wherein, the second quantity threshold is less than the first quantity threshold and the second frequency threshold is less than the first frequency threshold.

4. The image rendering optimization method according to claim 1, characterized in that, After determining the rendering resource associated with each cluster unit in the previous rendering frame, the method further includes: Determine the resource identifier of the rendering resource associated with each cluster unit, store the resource identifier of each cluster unit in the first cluster data buffer, and generate the cluster index of each cluster unit. The clustering index of each clustering unit is stored in the first clustering index buffer; Based on the first clustering data buffer and the first clustering index buffer, the image is rendered for the previous rendering frame.

5. The image rendering optimization method according to claim 4, characterized in that, The step of reusing the rendering resources associated with the inactive unit in the previous rendering frame and using the updated rendering resources associated with the active unit to perform image rendering on the current rendering frame includes: The resource identifiers of the inactive units stored in the first clustering data buffer are copied to the second clustering data buffer, and a new clustering index for the inactive units is generated. Determine the new resource identifier of the rendered resource after the active unit is updated, store the new resource identifier of the active unit in the second clustering data buffer, and generate a new clustering index for the active unit; Store the new clustering indexes of the active and inactive units into the second clustering index buffer; Based on the second clustering data buffer and the second clustering index buffer, the current rendering frame is rendered.

6. The image rendering optimization method according to any one of claims 1-5, characterized in that, The construction process of the three-dimensional clustering mesh includes: The projection area of ​​the first camera's field of view cone in screen space is determined, and the projection area is uniformly divided into multiple two-dimensional grids; wherein each two-dimensional grid corresponds to a two-dimensional coordinate. The depth space from the camera to each two-dimensional grid is divided into depth layers to obtain a three-dimensional clustered grid; wherein the interlayer spacing of different depth layers gradually increases along the depth direction of the first camera's field of view cone.

7. The image rendering optimization method according to claim 6, characterized in that, The step of uniformly dividing the projection area into multiple two-dimensional grids includes: Determine the current performance of the target device's graphics processor; Determine a cluster density that matches the current performance, and uniformly divide the projected region into multiple two-dimensional grids based on the cluster density; The process of depth layering from the camera to each 2D grid includes: The depth space from the camera to each 2D grid is divided into multiple depth layers based on the clustering density.

8. The image rendering optimization method according to claim 6, characterized in that, The method further includes: When the angle of view change of the second camera's view frustum range compared to the first camera's view frustum range exceeds a preset angle of view change, a three-dimensional clustering mesh is reconstructed within the second camera's view frustum range.

9. An image rendering optimization device, characterized in that, The device includes: The first processing module is used to obtain the range of the first camera frustum of the previous rendering frame and to obtain a three-dimensional clustering mesh constructed within the range of the first camera frustum; wherein, the three-dimensional clustering mesh includes multiple clustering units; The second processing module is used to determine the rendering resources associated with each clustering unit in the previous rendering frame; The third processing module is used to obtain the second camera frustum range of the current rendering frame, and when it is detected that the angle change of the second camera frustum range compared with the first camera frustum range does not exceed the preset angle change, it determines the access frequency of each clustering unit in the historical rendering frame. The fourth processing module is used to divide the multiple clustering units into active units and inactive units based on the relationship between the access frequency and the preset access frequency, and to update the rendering resources associated with the active units. The fifth processing module is used to reuse the rendering resources associated with the inactive unit in the previous rendering frame, and to use the updated rendering resources associated with the active unit to perform image rendering on the current rendering frame.

10. A vehicle, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the image rendering optimization method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the image rendering optimization method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the image rendering optimization method according to any one of claims 1 to 8.

Images