Method and device for displaying volume element raster data in three-dimensional space

Abstract

The invention discloses a method and device for displaying voxel raster data in a three-dimensional space, and the method comprises the steps: processing the voxel raster data in the three-dimensional space to obtain tile data, the tile data comprises a plurality of tile layers, each tile layer comprises at least one raster tile, and each raster tile comprises at least one raster tile; the pixels in each grid tile form a cube of which the pixel side length is a first number; the numbers of the grid tiles contained in the tile layers of different hierarchies are different; the data address ranges corresponding to the grid tiles contained in the tile layers of different levels are different; according to a data address range corresponding to the virtual camera angle, loading a third number of target grid tiles in a second number of target tile layers in the tile data to a video memory area; the second number is smaller than the total number of tile layers in the tile data; and at least providing the storage address of the video memory area to the shader, so that the graphics processor operates the shader, and the shader reads the volume element raster data according to the storage address and performs three-dimensional image rendering display.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional image technology, and in particular to a method and device for displaying voxel grid data in a three-dimensional space. Background Art

[0002] Voxel grid is a regular division of three-dimensional continuous space or phenomenon, which is composed of voxels (cubes / regular hexagonal prisms) arranged regularly in three-dimensional space. Voxel grid is usually used to express three-dimensional field data such as communication signals, temperature, wind field, pollution, and three-dimensional analysis results such as sunshine rate.

[0003] With the development of acquisition technology, the accuracy of voxel grid data is getting higher and higher, and the data acquisition range is getting larger and larger, resulting in an increasing amount of voxel grid data. If all the voxel grid data in the three-dimensional space is loaded into the video memory area for three-dimensional image rendering, the space requirement for the video memory area will be very high. If the space in the video memory area is limited, it will lead to low rendering performance.

[0004] Therefore, there is an urgent need for a technical solution that can render three-dimensional images based on a limited video memory area. Summary of the invention

[0005] In view of the above problems, the present application provides a method and device for displaying voxel grid data in a three-dimensional space, so as to realize the rendering and display of three-dimensional images based on a limited video memory area, and to achieve the purpose of improving the rendering performance of three-dimensional image rendering and display. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a method for displaying voxel grid data in a three-dimensional space, the method comprising:

[0007] Processing voxel grid data in a three-dimensional space to obtain tile data, wherein the tile data includes multiple tile layers, each tile layer includes at least one grid tile, and pixels in each grid tile form a cube with a first number of pixel side lengths; the number of grid tiles included in the tile layers at different levels is different; and the data address ranges corresponding to the grid tiles included in the tile layers at different levels are different;

[0008] According to the data address range corresponding to the virtual camera angle, a third number of target grid tiles in the second number of target tile layers in the tile data are loaded into a video memory area; the second number is less than the total number of tile layers in the tile data;

[0009] At least the storage address of the video memory area is provided to the shader, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address and performs three-dimensional image rendering and display.

[0010] In a possible implementation, processing voxel grid data in a three-dimensional space to obtain tile data includes:

[0011] Using the voxel grid data in the three-dimensional space as the current tile layer;

[0012] Dividing pixels in the voxel grid data in the current tile layer into tiles according to the first quantity to obtain grid tiles in the current tile layer;

[0013] Merging pixels of the voxel grid data in the current tile layer according to a neighboring merging rule to obtain a tile layer at a lower level;

[0014] Taking the tile layer of the lower level as the current tile layer, performing pixel merging on the voxel grid data in the current tile layer according to the neighboring merging rule to obtain the tile layer of the lower level, until there is only one grid tile in the tile layer of the lower level;

[0015] The neighboring merging rule includes: merging a target number of pixels that are adjacent on the horizontal axis, the vertical axis and the vertical axis respectively.

[0016] In a possible implementation, processing voxel grid data in a three-dimensional space to obtain tile data includes:

[0017] Using the voxel grid data in the three-dimensional space as the current tile layer;

[0018] Merging pixels of the voxel grid data in the current tile layer according to a neighboring merging rule to obtain a tile layer at a lower level;

[0019] Taking the tile layer of the lower level as the current tile layer, performing pixel merging on the voxel grid data in the current tile layer according to the neighboring merging rule to obtain a tile layer of the lower level, until the number of pixels of the voxel grid data in the tile layer of the lower level matches the first number;

[0020] Perform tile division on pixels in the voxel grid data in the tile layer of each level according to the first quantity, so as to obtain grid tiles in the tile layer of each level;

[0021] The neighboring merging rule includes: merging a target number of pixels that are adjacent on the horizontal axis, the vertical axis and the vertical axis respectively.

[0022] In a possible implementation, the video memory area includes a second number of sub-areas, each of which is uniquely characterized by an area identifier; each of the sub-areas includes a third number of tile areas, each of which is uniquely characterized by a tile identifier; each of the tile areas is used to store one of the grid tiles;

[0023] The step of loading a third number of target grid tiles in a second number of target tile layers in the tile data into a video memory area according to a data address range corresponding to the virtual camera angle includes:

[0024] If the video memory area has already loaded with historical grid tiles, an incremental address range is obtained according to the data address range corresponding to the virtual camera angle and the historical address range corresponding to the historical grid tiles; and according to the incremental address range, the incremental grid tiles corresponding to the incremental address range are loaded into the corresponding tile area in the corresponding sub-area of ​​the video memory area;

[0025] If no historical grid tiles are loaded in the video memory area, obtain the data address range corresponding to the second number of target tile layers according to the data address range corresponding to the virtual camera angle; and load the third number of target grid tiles in the second number of target tile layers into the corresponding tile areas in the corresponding sub-areas in the video memory area according to the data address range corresponding to the second number of target tile layers.

[0026] In a possible implementation, in the video memory area:

[0027] The region identifier of the sub-region into which the target tile layer is loaded in the video memory region is: the target level identifier of the target tile layer modulo the second number;

[0028] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis;

[0029] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0030] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0031] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the horizontal axis, modulo the number of grid tiles in the sub-region on the horizontal axis multiplied by the first number;

[0032] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-region on the vertical axis multiplied by the first number;

[0033] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number.

[0034] In a possible implementation, if the virtual camera angle is scaled, the incremental address range includes: a data address range corresponding to the third number of target grid tiles in a tile layer to which at least one first-level identifier belongs; the first-level identifier is obtained according to the spatial scale of the virtual camera angle scaling;

[0035] The first level identifier is continuous with the minimum target level identifier in the video memory area, or the first level identifier is continuous with the maximum target level identifier in the video memory area; and, in the video memory area, the area identifier of the sub-area to which the tile layer to which the first level identifier belongs is loaded in the video memory area is: the first level identifier modulo the second number;

[0036] If the virtual camera angle is translated, the incremental address range includes: a data address range corresponding to at least one incremental grid tile in each of the target tile layers; the incremental grid tile is obtained according to the address range of the virtual camera angle translation;

[0037] The tile identifier of the tile area to which the incremental grid tile is loaded on the horizontal axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis;

[0038] The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0039] The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0040] The storage address to which the voxel grid data in the incremental grid tile is loaded on the horizontal axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis, modulo the number of grid tiles in the sub-area on the horizontal axis multiplied by the first number;

[0041] The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number;

[0042] The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number.

[0043] In one possible implementation, the shader reads the voxel grid data in the following manner:

[0044] Normalizing the data address corresponding to the virtual camera viewing angle to obtain a texture address;

[0045] The highest level target tile layer in the video memory area is set as the current tile layer;

[0046] According to the normalized matrix corresponding to the current tile layer, determining whether the texture address matches the data address range corresponding to the current tile layer;

[0047] If the texture address matches the data address range corresponding to the current tile layer, read the corresponding voxel grid data from the sub-area corresponding to the current tile layer in the video memory area based on the storage address according to the product of the normalization matrix and the offset matrix corresponding to the current tile layer;

[0048] If the texture address does not match the data address range corresponding to the current tile layer, a tile layer one level lower than the current tile layer will be used as the current tile layer, and the normalization matrix corresponding to the current tile layer will be executed to determine whether the data address corresponding to the virtual camera perspective matches the data address range corresponding to the current tile layer, until the current tile layer is the lowest level target tile layer in the video memory area.

[0049] In a possible implementation, the normalized matrix corresponding to the current tile layer is a matrix of four rows and four columns, where:

[0050] The matrix element of the first row and the first column is: 2 raised to the power of a fourth number, and then divided by the number of grid tiles of the current tile layer on the horizontal axis; the fourth number is the target level identifier of the current tile layer;

[0051] The matrix element of the second row and the second column is: 2 raised to the power of the fourth number, and then divided by the number of grid tiles of the current tile layer on the vertical axis;

[0052] The matrix element in the third row and the third column is: 2 raised to the power of the fourth number, divided by the number of grid tiles of the current tile layer on the vertical axis;

[0053] The matrix element in the fourth row and fourth column is 1;

[0054] The matrix element in the fourth row and the first column is: the smallest tile identifier on the horizontal axis in the current tile layer is divided by the number of grid tiles on the horizontal axis of the current tile layer, and the result is a negative value;

[0055] The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis of the current tile layer, and the result is a negative value;

[0056] The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis of the current tile layer, and the result is a negative value;

[0057] The other matrix elements are 0.

[0058] In a possible implementation, the offset matrix corresponding to the current tile layer is a matrix of four rows and four columns, where:

[0059] The matrix element in the first row and first column is 1;

[0060] The matrix element in the second row and second column is 1;

[0061] The matrix element in the third row and third column is 1;

[0062] The matrix element in the fourth row and fourth column is 1;

[0063] The matrix element in the fourth row and the first column is: the smallest tile identifier on the horizontal axis in the current tile layer modulo the number of grid tiles on the horizontal axis of the current tile layer, and then divided by the number of grid tiles on the horizontal axis of the current tile layer;

[0064] The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, and then divided by the number of grid tiles on the vertical axis of the current tile layer;

[0065] The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, divided by the number of grid tiles on the vertical axis of the current tile layer;

[0066] The other matrix elements are 0.

[0067] A second aspect of the present application provides a device for displaying voxel grid data in a three-dimensional space, the device comprising:

[0068] A tile processing unit, used for processing voxel grid data in a three-dimensional space to obtain tile data, wherein the tile data includes multiple tile layers, each of which includes at least one grid tile, and pixels in each of the grid tiles form a cube with a first number of pixel side lengths; the number of grid tiles included in the tile layers at different levels is different, and the data address ranges corresponding to the grid tiles included in the tile layers at different levels are different;

[0069] A tile loading unit, configured to load a third number of target grid tiles in a second number of target tile layers in the tile data into a video memory area according to a data address range corresponding to a virtual camera angle; the second number is less than a total number of tile layers in the tile data;

[0070] The tile rendering unit is used to provide at least the storage address of the video memory area to the shader, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address and performs three-dimensional image rendering and display.

[0071] A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for displaying voxel grid data in a three-dimensional space of the above-mentioned first aspect or any implementation of the first aspect.

[0072] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0073] The memory is used to store computer programs;

[0074] The processor is used to execute the computer program so that the electronic device can implement the method for displaying voxel grid data in a three-dimensional space of the above-mentioned first aspect or any implementation manner of the first aspect.

[0075] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can display voxel grid data in a three-dimensional space according to the above-mentioned first aspect or any implementation method of the first aspect.

[0076] By means of the above technical solution, in a method and device for displaying voxel grid data in a three-dimensional space provided by the present application, by dividing the voxel grid data in the three-dimensional space into multiple layers of tile layers containing grid tiles, it is not necessary to load all the voxel grid data into the video memory area, and only the third number of target grid tiles in the second number of target tile layers need to be loaded into the video memory area, and the voxel grid data that exceeds the data address range corresponding to the virtual camera angle does not need to be loaded into the video memory area, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address of the video memory area and renders and displays the three-dimensional image. Therefore, in this embodiment, only the video memory area is required to be able to store the third number of target grid tiles in the second number of target tile layers, and the requirements for the video memory area are not very high, so that the voxel grid data with a large amount of data can be rendered based on the limited video memory area, thereby improving the rendering performance of the three-dimensional image rendering display. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0078] Figure 1 A flowchart of a method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application;

[0079] Figure 2 This is an example diagram of a grid tile in an embodiment of the present application;

[0080] Figure 3 This is an example diagram of a tile layer in an embodiment of the present application;

[0081] Figure 4 A partial flow chart of a method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application;

[0082] Figure 5 This is an example diagram of dividing grid tiles in each tile layer in an embodiment of the present application;

[0083] Figure 6Another partial flow chart of a method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application;

[0084] Figure 7 This is an example diagram of the video memory area in the embodiment of the present application;

[0085] Figure 8 Another partial flow chart of a method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application;

[0086] Fig. 9 This is an example diagram of replacing a tile layer of a new level in an embodiment of the present application;

[0087] Fig.10 Another partial flow chart of a method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application;

[0088] Fig.11 A schematic diagram of the structure of a device for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application;

[0089] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0090] Fig.13 An example diagram of creating 6 sub-areas in the video memory area in an embodiment of the present application;

[0091] Fig.14 This is an example diagram of replacing a tile layer of a new level in a video memory area in an embodiment of the present application;

[0092] Fig.15 A two-dimensional example diagram of storage offset of grid tiles in an embodiment of the present application;

[0093] Fig.16 This is an example diagram of levels 0 to 3 in the embodiments of the present application;

[0094] Fig.17 Another two-dimensional example diagram of the storage offset of the grid tile in the embodiment of the present application;

[0095] Fig.18 Another two-dimensional example diagram of the storage offset of the grid tile in the embodiment of the present application. DETAILED DESCRIPTION

[0096] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0097] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0098] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0099] The voxel grid data in this application can be used to express three-dimensional spatial data in three-dimensional scenes such as communication signals, temperature, wind field, pollution, and sunshine rate.

[0100] Currently, there are two main line-of-sight schemes for rendering voxel raster data.

[0101] Solution 1 is to load all voxel raster data into the video memory area in the form of textures and render it directly through volume rendering. The disadvantage is that the data volume is large and the loading is slow, especially in the browser-server structure where the data must be loaded through the network. At the same time, the large amount of data leads to high video memory usage and reduced rendering performance.

[0102] Solution 2 is to model the voxel grid data and implement the scheduling display of the voxel grid through tiling of the triangulated network. However, this method cannot show the continuous expression of the voxel grid. The internal parts are displayed separately, with obvious gaps at the boundaries, and the display effect feels discontinuous.

[0103] In order to solve the above problems, the embodiment of the present application provides a method for displaying voxel grid data in a three-dimensional space. The method for displaying voxel grid data in a three-dimensional space of the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0104] Reference Figure 1 , is a flowchart of a method for displaying voxel grid data in a three-dimensional space provided in an embodiment of the present application, which method can be applied to electronic devices with a graphics processor, such as mobile phones, tablet devices, notebooks, computers, servers, etc. The technical solution in this embodiment is mainly used to improve the rendering performance of three-dimensional image rendering display.

[0105] Specifically, the method in this embodiment may include the following steps:

[0106] Step 101: Process voxel grid data in three-dimensional space to obtain tile data.

[0107] The tile data includes multiple tile layers, each tile layer includes at least one grid tile, and the pixels in each grid tile form a cube with a pixel side length of a first number, such as Figure 2 As shown in . The first number can be set according to business needs. For example, the first number can be 64, and accordingly, the three-dimensional texture size of each grid tile is 64×64×64 pixels; or the first number can be 128, and accordingly, the three-dimensional texture size of each grid tile is 128×128×128 pixels.

[0108] It should be noted that the number of raster tiles contained in tile layers of different levels is different; the data address ranges corresponding to the raster tiles contained in tile layers of different levels are different. Specifically, the number of raster tiles contained in a tile layer of a higher level is a multiple of the number of raster tiles contained in a tile layer of a lower level, and the data address range corresponding to each raster tile in the raster tiles contained in the tile layer of a lower level is a multiple of the data address range corresponding to each raster tile in the raster tiles contained in the tile layer of a higher level.

[0109] For example, Figure 3 As shown in , the multiple tile layers in the tile data form a pyramid structure, the number of grid tiles contained in a higher-level tile layer is a multiple of the cube of the target number of grid tiles contained in a lower-level tile layer, and the data address range corresponding to each grid tile contained in the lower-level tile layer is a multiple of the cube of the target number of data address ranges corresponding to each grid tile contained in the higher-level tile layer.

[0110] Among them, the target number can be related to the adjacent merging rule used when obtaining the tile layer of a lower level from the tile layer of a higher level. For example, the adjacent merging rule can include: merging two adjacent pixels on the horizontal axis, the vertical axis and the vertical axis, based on which, the voxel grid data of the higher level is pixel-merged according to the adjacent merging rule to obtain the voxel grid data of the tile layer of a lower level. Thus, the number of grid tiles contained in the tile layer of a higher level is a multiple of 2 cubed of the number of grid tiles contained in the tile layer of a lower level, and the data address range corresponding to each grid tile contained in the tile layer of a lower level is a multiple of 2 cubed of the data address range corresponding to each grid tile contained in the tile layer of a higher level. It can be seen that the pixel resolution of the voxel grid data in the tile layer of a higher level is higher than the pixel resolution of the voxel grid data in the tile layer of a lower level.

[0111] Step 102: Load a third number of target grid tiles in a second number of target tile layers in the tile data into a video memory area according to a data address range corresponding to the virtual camera angle.

[0112] The virtual camera angle can be understood as the current viewing angle of the three-dimensional space, that is, the viewing angle of the rendered three-dimensional space. The second number is less than the total number of tile layers in the tile data. The third number is related to the area size of the video memory area and the number of grid tiles in the target tile layer.

[0113] For example, the second number can be 6 or 8, and the third number is a fixed value such as 64. Based on this, 64 target grid tiles corresponding to the virtual camera angles in 6 tile layers or 8 tile layers in the tile data are loaded into the video memory area. If there are no third number of grid tiles in the target tile layer, all grid tiles in the target tile layer are loaded into the video memory area as target grid tiles.

[0114] It should be noted that the third number of target grid tiles in each target tile layer matches the virtual camera angle. For example, the data address range corresponding to the target grid tiles loaded into the video memory area is consistent with the address range corresponding to the virtual camera angle.

[0115] Step 103: at least provide the storage address of the video memory area to the shader, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address and performs three-dimensional image rendering and display.

[0116] By means of the above technical solution, in a method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application, by dividing the voxel grid data in the three-dimensional space into multiple layers of tile layers containing grid tiles, it is not necessary to load all the voxel grid data into the video memory area, and only the third number of target grid tiles in the second number of target tile layers need to be loaded into the video memory area, and the voxel grid data that exceeds the data address range corresponding to the virtual camera angle does not need to be loaded into the video memory area, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address of the video memory area and renders and displays the three-dimensional image. Therefore, in this embodiment, only the video memory area is required to be able to store the third number of target grid tiles in the second number of target tile layers, and the requirements for the video memory area are not very high, so that the voxel grid data with a large amount of data can be rendered based on the limited video memory area, thereby improving the rendering performance of the three-dimensional image rendering and display.

[0117] In one implementation, step 101 may be implemented in the following manner: Figure 4 As shown in:

[0118] Step 401: Use the voxel grid data in the three-dimensional space as the current tile layer.

[0119] That is to say, the voxel grid data in the three-dimensional space is used as the highest-level tile layer, and the subsequent processes are executed starting from the highest-level tile layer.

[0120] Step 402: Divide pixels in the voxel grid data in the current tile layer into tiles according to a first quantity to obtain grid tiles in the current tile layer.

[0121] For example, the pixels in the voxel grid data in the current tile layer are tiled according to the number of 64 on the x-axis, y-axis and z-axis respectively, so that the grid tiles in the current tile layer can be obtained, and each grid tile has a pixel size of 64×64×64.

[0122] Step 403: Pixel merging is performed on the voxel grid data in the current tile layer according to the neighbor merging rule to obtain a tile layer of a lower level.

[0123] For example, Figure 5 As shown in , according to the neighbor merging rule, the voxel grid data in the tile layer of a higher level are pixel-merged to obtain the tile layer of a lower level.

[0124] The neighboring merging rule may include: merging pixels of a target number that are adjacent on the horizontal axis, the vertical axis, and the vertical axis, respectively.

[0125] For example, the pixel values ​​of the target number of pixels adjacent to each other on the horizontal axis, the vertical axis and the vertical axis are averaged to obtain the pixel value of a merged pixel. For example, taking the target number as 2 as an example, Figure 5 As shown in , the pixel values ​​of two adjacent pixels, i.e., a total of eight pixels, in the voxel grid data of the higher level in the three dimensions of the x-axis, y-axis, and z-axis are averaged, and the obtained average value is used as the pixel value of the merged pixel.

[0126] Among them, the execution order between step 402 and step 403 is not limited to the execution order shown in the accompanying drawings. In other embodiments, step 403 may be executed first, and then step 402, or step 402 and step 403 may be executed simultaneously. Different technical solutions formed by different execution orders are all within the protection scope of this application.

[0127] Step 404: Take the tile layer of the lower level as the current tile layer, and return to execute step 402 until there is only one grid tile in the tile layer of the lower level, that is, the number of pixels of the voxel grid data in the tile layer of the lower level matches the first number, and no longer return to execute step 402.

[0128] Here, the number of pixels of the voxel grid data in the tile layer of a lower level matches the first number, which means that the number of pixels of the voxel grid data in the tile layer of a lower level is less than or equal to the first number.

[0129] It can be seen that in this embodiment, starting from the highest-level tile layer, the voxel grid data in the highest-level tile layer is divided into tiles, and then pixels are merged to obtain a tile layer of a lower level. Then, the voxel grid data in the lower-level tile layer is divided into tiles, and then pixels are merged to obtain a tile layer of an even lower level, and so on, until only one grid tile can be divided in the lowest-level tile layer.

[0130] In another implementation, step 101 may be implemented in the following manner: Figure 6 As shown in:

[0131] Step 601: Use the voxel grid data in the three-dimensional space as the current tile layer.

[0132] That is to say, the voxel grid data in the three-dimensional space is used as the highest-level tile layer, and the subsequent processes are executed starting from the highest-level tile layer.

[0133] Step 602: Pixel merging is performed on the voxel grid data in the current tile layer according to the neighbor merging rule to obtain a tile layer of a lower level.

[0134] The specific implementation of step 602 may refer to the implementation of step 402 in the previous text, and will not be described in detail here.

[0135] Step 603: Use the tile layer at a lower level as the current tile layer, and return to execute step 602 until the pixel data of the voxel grid data in the tile layer at a lower level matches the first quantity.

[0136] Step 604: Perform tile division on pixels in the voxel grid data in the tile layer of each level according to a first quantity to obtain grid tiles in the tile layer of each level.

[0137] The specific implementation of tile division in step 604 may refer to the implementation of tile division in step 403 in the previous text, which will not be described in detail here.

[0138] It can be seen that in this embodiment, starting from the highest-level tile layer, the voxel grid data in the highest-level tile layer is obtained by pixel merging to obtain a tile layer of a lower level, and then the voxel grid data in the lower-level tile layer is obtained by pixel merging to obtain a tile layer of an even lower level, and so on, until the number of pixels of the voxel grid data in the tile layer of the lowest level matches the first number, and finally, the tile layer of each level is divided into tiles to obtain grid tiles in the tile layer of each level, and only one grid tile is divided out of the tile layer of the lowest level.

[0139] In one implementation, in this embodiment, a video memory region is created according to the second number, the video memory region includes a second number of sub-regions, each sub-region is uniquely characterized by a region identifier, and each sub-region stores a target tile layer. Each sub-region includes a third number of tile regions, each tile region is uniquely characterized by a tile identifier, and each tile region is used to store a grid tile.

[0140] For example, the second number is 6 and the third number is 64. Figure 7 As shown in , the video memory area contains 6 sub-areas, which can also be called storage blocks; each sub-area contains 64 tile areas, and each tile area is used to store a grid tile of 64×64×64 pixels. Therefore, the pixel size that can be stored in each sub-area is: 64×64×64×64.

[0141] Based on this, step 102 can be implemented in the following ways: Figure 8 As shown in:

[0142] Step 801: Determine whether historical raster tiles have been loaded in the video memory area. If historical raster tiles have been loaded in the video memory area, execute steps 802 to 803. If historical raster tiles have not been loaded in the video memory area, execute steps 804 to 805.

[0143] Step 802: Obtain an incremental address range according to the data address range corresponding to the virtual camera angle and the historical address range corresponding to the historical grid tile.

[0144] Among them, if historical raster tiles have been loaded in the video memory area, it indicates that the virtual camera angle has changed, such as zooming and translation. Based on this, if the virtual camera angle is zoomed, the incremental address range includes: a data address range corresponding to a third number of target raster tiles in the tile layer to which at least one first-level identifier belongs; if the virtual camera angle is translated, the incremental address range includes: a data address range corresponding to at least one incremental raster tile in each target tile layer, where the incremental raster tile is a raster tile in the target tile layer that has not been loaded into the video memory area, that is, the incremental raster tile is an increment relative to the target raster tile loaded in the video memory area.

[0145] Among them, the first level identifier is obtained according to the spatial scale of the virtual camera angle scaling. The first level identifier is continuous with the minimum target level identifier in the video memory area, or the first level identifier is continuous with the maximum target level identifier in the video memory area. For example, if the virtual camera angle is enlarged, the first level identifier includes: one or more level identifiers continuous with the maximum target level identifier in the video memory area, and the number of first level identifiers is related to the scale of the virtual camera angle enlargement, and the larger the enlargement scale, the more the number of first level identifiers. For another example, if the virtual camera angle is reduced, the first level identifier includes: one or more level identifiers continuous with the minimum target level identifier in the video memory area, and the number of first level identifiers is related to the scale of the virtual camera angle reduction, and the larger the reduction scale, the more the number of first level identifiers.

[0146] The incremental grid tiles are obtained according to the address range of the virtual camera angle translation. For example, for a certain target tile layer, if the virtual camera angle translates to the left, the grid tiles on the left side of the target grid tiles that have been loaded into the video memory area in the target tile layer are used as incremental grid tiles; if the virtual camera angle translates to the right, the grid tiles on the right side of the target grid tiles that have been loaded into the video memory area in the target tile layer are used as incremental grid tiles. The incremental grid tiles include: one or more grid tiles in the target tile layer that are in the translation direction of the target grid tiles that have been loaded into the video memory area, and the number of incremental grid tiles is related to the size of the address range of the virtual camera angle translation. The larger the translation address range, the greater the number of incremental grid tiles.

[0147] Step 803: according to the incremental address range, load the incremental grid tile corresponding to the incremental address range into the corresponding tile area in the corresponding sub-area of ​​the video memory area.

[0148] Among them, if the virtual camera angle is scaled, the incremental grid tiles corresponding to the incremental address range include: a third number of incremental grid tiles in the tile layer to which the first-level identifier belongs that match the virtual camera angle; if the virtual camera angle is translated, the incremental grid tiles corresponding to the incremental address range include: grid tiles in each target tile layer in the video memory area that are not loaded into the video memory area and match the incremental address range of the translation of the virtual camera angle.

[0149] Specifically, if the virtual camera angle is scaled, then after the grid tile loading in step 803, in the video memory area, the tile layer to which the first level identifier belongs is loaded into the sub-area in the video memory area with the region identifier of: the first level identifier modulo the second number. In other words, the tile layer to which the first level identifier belongs replaces (covers) the existing target tile layer in the sub-area corresponding to the region identifier of the first level identifier modulo the second number in the video memory area.

[0150] For example, Fig. 9 As shown in , there are 6 sub-areas in the video memory area, and the area identifiers are 0 to 5, respectively. The tile layers of the 8th to 13th levels have been loaded. If the virtual camera is zoomed in, the tile layer of the higher level, the 14th level, is loaded into the video memory area, and 14 modulo 6 is 2, so the third number of target grid tiles in the tile layer of the 14th level is loaded into the sub-area with area identifier 2.

[0151] If the virtual camera angle is translated, then after the grid tile loading in step 803, the incremental grid tile replaces the grid tile in the corresponding tile area in the video memory area in the following manner:

[0152] The tile identifier of the tile area to which the incremental grid tile is loaded on the horizontal axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis; for example, the data address x of the incremental grid tile on the horizontal axis modulo 4 is taken to obtain the tile identifier of the tile area on the horizontal axis in the sub-area where the target tile layer is located;

[0153] The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; for example, the data address y of the incremental grid tile on the vertical axis modulo 4 to obtain the tile identifier of the tile area on the vertical axis in the sub-area where the target tile layer is located;

[0154] The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; for example, the data address z of the incremental grid tile on the vertical axis modulo 4 is taken to obtain the tile identifier of the tile area on the vertical axis in the sub-area where the target tile layer is located;

[0155] Based on this, the tile area in the sub-area where the target tile layer is located for loading the incremental grid tile can be determined based on the tile identifier of the tile area on the horizontal axis, the tile identifier of the tile area on the vertical axis, and the tile identifier of the tile area on the vertical axis in the sub-area where the target tile layer is located, that is, one of the 64 tile areas in the sub-area where the target tile layer is located that is loaded into, and based on this, the incremental grid tile is loaded into the determined tile area.

[0156] In addition, the voxel grid data in the incremental grid tile replaces the voxel grid data in the corresponding tile area in the following manner:

[0157] The storage address to which the voxel grid data in the incremental grid tile is loaded on the horizontal axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis multiplied by the first number; for example, the data address x of the incremental grid tile on the horizontal axis modulo 4 is multiplied by 64 to obtain the storage address to which the voxel grid data in the incremental grid tile is loaded on the horizontal axis in the corresponding tile area;

[0158] The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number; for example, the data address y of the incremental grid tile on the vertical axis modulo 4 is multiplied by 64 to obtain the storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area;

[0159] The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis and multiplied by the first number; for example, the data address z of the incremental grid tile on the vertical axis is modulo 4 to obtain the storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area.

[0160] Based on this, the storage address for loading the voxel grid data in the incremental grid tile in the sub-region where the target tile layer is located can be determined based on the storage address loaded on the horizontal axis, the storage address loaded on the vertical axis, and the storage address loaded on the vertical axis in the sub-region where the target tile layer is located, and based on this, the voxel grid data in the incremental grid tile is loaded to the storage address in the determined tile region. Therefore, in this embodiment, it is not necessary to load all the voxel grid data, but only load the incremental voxel grid data, thereby reducing the amount of data loading and the amount of data reading during subsequent shader rendering and display.

[0161] Step 804: Obtain data address ranges corresponding to a second number of target tile layers according to the data address range corresponding to the virtual camera angle.

[0162] For example, according to the data address range corresponding to the virtual camera angle, the data address range corresponding to 6 target tile layers is obtained, such as the data address range corresponding to the target tile layers from the 8th level to the 13th level.

[0163] Step 805: According to the data address range corresponding to the second number of target tile layers, load the third number of target grid tiles in the second number of target tile layers into the corresponding tile areas in the corresponding sub-areas in the video memory area respectively.

[0164] Specifically, in the video memory area, the region identifier of the sub-region into which the target tile layer is loaded in the video memory area is: the target level identifier of the target tile layer modulo the second number. In other words, the target tile layer is loaded into the sub-region corresponding to the region identifier of the target level identifier modulo the second number in the video memory area.

[0165] For example, Fig. 9 As shown in , the area identifiers of the 6 sub-areas in the video memory area are 0 to 5 respectively, the target tile layer of the 8th level is loaded into the sub-area with an area identifier of 2 (8 modulo 6), the target tile layer of the 9th level is loaded into the sub-area with an area identifier of 3 (9 modulo 6), the target tile layer of the 10th level is loaded into the sub-area with an area identifier of 4 (10 modulo 6), the target tile layer of the 11th level is loaded into the sub-area with an area identifier of 5 (11 modulo 6), the target tile layer of the 12th level is loaded into the sub-area with an area identifier of 0 (12 modulo 6), and the target tile layer of the 13th level is loaded into the sub-area with an area identifier of 1 (13 modulo 6).

[0166] Further, after the grid tile loading in step 805, in the video memory area, the target grid tile in the target tile layer is loaded into the grid tile in the corresponding tile area in the following manner:

[0167] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis; for example, the data address x of the target grid tile in the target tile layer on the horizontal axis modulo 4 is taken to obtain the tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding sub-area;

[0168] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; for example, the data address y of the target grid tile in the target tile layer on the vertical axis modulo 4 is taken to obtain the tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area;

[0169] The tile identifier of the tile area into which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; for example, the data address z at the top of the target grid tile in the target tile layer is modulo 4 to obtain the tile identifier of the tile area into which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area.

[0170] Based on this, the tile area in the sub-area where the target tile layer is located for loading the corresponding target grid tile can be determined based on the tile identifier of the tile area on the horizontal axis, the tile identifier of the tile area on the vertical axis, and the tile identifier of the tile area on the vertical axis in the sub-area where the target tile layer is located, that is, one of the 64 tile areas loaded in the sub-area where the target tile layer is located. Based on this, each target grid tile in the target tile layer is loaded into the corresponding tile area determined respectively.

[0171] Further, after the grid tile loading in step 805, in the video memory area, the voxel grid data in the target grid tile in the target tile layer is loaded into the storage address in the corresponding tile area in the following manner:

[0172] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the horizontal axis modulo the number of grid tiles in the sub-region on the horizontal axis multiplied by the first number; for example, the data address x of the target grid tile on the horizontal axis modulo 4 is multiplied by 64 to obtain the storage address to which the voxel grid data in the target grid tile is loaded on the horizontal axis in the corresponding tile region;

[0173] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-region on the vertical axis multiplied by the first number; for example, the data address y of the target grid tile on the vertical axis modulo 4 multiplied by 64 to obtain the storage address to which the voxel grid data in the target grid tile is loaded on the vertical axis in the corresponding tile region;

[0174] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis and multiplied by the first number; for example, the data address y of the target grid tile on the vertical axis is modulo 4 and then multiplied by 64 to obtain the storage address to which the voxel grid data in the target grid tile is loaded on the vertical axis in the corresponding tile area.

[0175] Based on this, the storage address for loading the voxel grid data in the target grid tile in the sub-area where the target tile layer is located can be determined based on the storage address loaded on the horizontal axis, the storage address loaded on the vertical axis, and the storage address loaded on the vertical axis in the sub-area where the target tile layer is located, and based on this, the voxel grid data in the target grid tile is loaded to the storage address in the determined tile area.

[0176] In one implementation, the shader reads the voxel grid data in the following manner, such as Fig.10 As shown in:

[0177] Step 1001: Normalize the data address corresponding to the virtual camera viewing angle to obtain texture features.

[0178] For example, the data address corresponding to the virtual camera viewing angle is divided by the data address corresponding to the virtual camera viewing angle to obtain a texture feature between 0 and 1.

[0179] Step 1002: Set the highest-level target tile layer in the video memory area as the current tile layer.

[0180] Step 1003: According to the normalized matrix corresponding to the current tile layer, determine whether the texture address matches the data address range corresponding to the current tile layer. If yes, execute step 1004; if not, execute step 1005.

[0181] Whether the texture address matches the data address range corresponding to the current tile layer refers to whether the texture address is within the data address range corresponding to the current tile layer.

[0182] Specifically, in this embodiment, the texture address can be converted according to the normalized matrix corresponding to the current tile layer. If the component addresses of the converted texture address on the horizontal axis, vertical axis and vertical axis are greater than or equal to 0 and less than or equal to 1, then it can be determined that the texture address matches the data address range corresponding to the current tile layer. Otherwise, the texture address does not match the data address range corresponding to the current tile layer.

[0183] In one implementation, the normalized matrix corresponding to the current tile layer is a matrix with four rows and four columns, where:

[0184] The matrix element of the first row and first column is: 2 raised to the power of the fourth quantity, and then divided by the number of grid tiles of the current tile layer on the horizontal axis; the fourth quantity is the target level identifier of the current tile layer;

[0185] The matrix element of the second row and second column is: 2 raised to the power of the fourth number, divided by the number of grid tiles on the vertical axis of the current tile layer;

[0186] The matrix element in the third row and third column is: 2 raised to the power of the fourth number, divided by the number of grid tiles of the current tile layer on the vertical axis;

[0187] The matrix element in the fourth row and fourth column is 1;

[0188] The matrix element in the first column of the fourth row is: the smallest tile identifier on the horizontal axis in the current tile layer is divided by the number of grid tiles on the horizontal axis of the current tile layer, and the negative value is taken;

[0189] The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis in the current tile layer, and the negative value is taken;

[0190] The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis in the current tile layer, and the negative value is taken;

[0191] The other matrix elements are 0.

[0192] For example, taking the tile layer at level 0 as an example, its corresponding normalized matrix is ​​shown in formula (1):

[0193]

[0194] Among them, Level0 is the level identifier 0 of the tile layer of level 0; L0X1 is the largest tile identifier of level 0 on the horizontal axis; L0X0 is the smallest tile identifier of level 0 on the horizontal axis, and L0X1-L0X0+1 is the number of grid tiles of level 0 on the horizontal axis, such as 4; L0Y1 is the largest tile identifier of level 0 on the vertical axis; L0Y0 is the smallest tile identifier of level 0 on the vertical axis, and L0Y1-L0Y0+1 is the number of grid tiles of level 0 on the vertical axis, such as 4; L0Z1 is the largest tile identifier of level 0 on the vertical axis; L0Z0 is the smallest tile identifier of level 0 on the vertical axis, and L0Z1-L0Z0+1 is the number of grid tiles of level 0 on the vertical axis, such as 4.

[0195] For example, taking the 5th level tile layer as an example, its corresponding normalized matrix is ​​shown in formula (2):

[0196]

[0197] Among them, Level5 is the level identifier 0 of the tile layer of the 5th level; L5X1 is the largest tile identifier of the 5th level on the horizontal axis; L5X0 is the smallest tile identifier of the 0th level on the horizontal axis, and L5X1-L5X0+1 is the number of grid tiles of the 5th level on the horizontal axis, such as 4; L5Y1 is the largest tile identifier of the 5th level on the vertical axis; L5Y0 is the smallest tile identifier of the 5th level on the vertical axis, and L5Y1-L5Y0+1 is the number of grid tiles of the 5th level on the vertical axis, such as 4; L5Z1 is the largest tile identifier of the 5th level on the vertical axis; L5Z0 is the smallest tile identifier of the 5th level on the vertical axis, and L5Z1-L5Z0+1 is the number of grid tiles of the 5th level on the vertical axis, such as 4.

[0198] Step 1004: According to the product of the normalization matrix and the offset matrix corresponding to the current tile layer, based on the storage address, the corresponding voxel grid data is read from the sub-area corresponding to the current tile layer in the video memory area.

[0199] The offset matrix corresponds to the storage address of loading the voxel grid data in the current tile layer into the video memory area.

[0200] Specifically, in this embodiment, the storage address of the texture address in the tile area in the sub-area corresponding to the current tile layer can be determined in the storage address of the video memory area according to the product of the normalization matrix and the offset matrix, and then the corresponding voxel grid data, that is, the voxel grid data corresponding to the texture address, can be read according to the determined storage address. Based on this, the shader can render and display the read voxel grid data.

[0201] In one implementation, the offset matrix corresponding to the current tile layer is a matrix of four rows and four columns, where:

[0202] The matrix element in the first row and first column is 1;

[0203] The matrix element in the second row and second column is 1;

[0204] The matrix element in the third row and third column is 1;

[0205] The matrix element in the fourth row and fourth column is 1;

[0206] The matrix element in the first column of the fourth row is: the smallest tile identifier on the horizontal axis in the current tile layer modulo the number of grid tiles on the horizontal axis of the current tile layer, and then divided by the number of grid tiles on the horizontal axis of the current tile layer;

[0207] The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, and then divided by the number of grid tiles on the vertical axis of the current tile layer;

[0208] The matrix element in the fourth row and third column is: the modulo of the smallest tile identifier on the vertical axis in the current tile layer and the number of grid tiles on the vertical axis of the current tile layer, divided by the number of grid tiles on the vertical axis of the current tile layer;

[0209] The other matrix elements are 0.

[0210] For example, taking the tile layer at level 0 as an example, its corresponding offset matrix is ​​shown in formula (3):

[0211]

[0212] Among them, L0X1 is the largest tile identifier of the 0th level on the horizontal axis; L0X0 is the smallest tile identifier of the 0th level on the horizontal axis, L0X1-L0X0+1 is the number of grid tiles of the 0th level on the horizontal axis, such as 4; L0Y1 is the largest tile identifier of the 0th level on the vertical axis; L0Y0 is the smallest tile identifier of the 0th level on the vertical axis, L0Y1-L0Y0+1 is the number of grid tiles of the 0th level on the vertical axis, such as 4; L0Z1 is the largest tile identifier of the 0th level on the vertical axis; L0Z0 is the smallest tile identifier of the 0th level on the vertical axis, L0Z1-L0Z0+1 is the number of grid tiles of the 0th level on the vertical axis, such as 4.

[0213] For example, taking the 5th level tile layer as an example, its corresponding offset matrix is ​​shown in formula (4):

[0214]

[0215] Among them, L5X1 is the largest tile identifier of the 5th layer on the horizontal axis; L5X0 is the smallest tile identifier of the 0th layer on the horizontal axis, L5X1-L5X0+1 is the number of grid tiles of the 5th layer on the horizontal axis, such as 4; L5Y1 is the largest tile identifier of the 5th layer on the vertical axis; L5Y0 is the smallest tile identifier of the 5th layer on the vertical axis, L5Y1-L5Y0+1 is the number of grid tiles of the 5th layer on the vertical axis, such as 4; L5Z1 is the largest tile identifier of the 5th layer on the vertical axis; L5Z0 is the smallest tile identifier of the 5th layer on the vertical axis, L5Z1-L5Z0+1 is the number of grid tiles of the 5th layer on the vertical axis, such as 4.

[0216] Step 1005: Take a tile layer one level lower than the current tile layer as the current tile layer, and return to execute step 1003 until the current tile layer is the target tile layer of the lowest level in the video memory area.

[0217] Therefore, in this embodiment, after the data address corresponding to the virtual camera perspective is normalized into a texture address, the corresponding voxel raster data is read from the video memory area starting from the highest-level target tile layer, that is, the target tile layer with the highest resolution. If there is no voxel raster data corresponding to the texture address in the highest-level target tile layer in the video memory area, then the corresponding voxel raster data is read from the target tile layer of a lower level in the video memory area. If there is still no voxel raster data, then the corresponding voxel raster data is read from the target tile layer of a lower level in the video memory area, and so on until the corresponding voxel raster data is read. Therefore, in this embodiment, finer voxel raster data, that is, higher resolution voxel raster data is rendered preferentially, thereby performing high-resolution rendering and display on the nearby three-dimensional space, and low-resolution rendering and display on the distant three-dimensional space.

[0218] A method for displaying voxel grid data in a three-dimensional space provided by an embodiment of the present application is introduced above. A device for executing the method for displaying voxel grid data in a three-dimensional space will be introduced below.

[0219] refer to Fig.11 , is a schematic diagram of a display device for voxel grid data in a three-dimensional space provided in an embodiment of the present application, and the device can be configured in an electronic device with a graphics processor, such as a mobile phone, a tablet device, a notebook, a computer, a server, etc. The technical solution in this embodiment is mainly used to improve the rendering performance of three-dimensional image rendering display.

[0220] Specifically, the device in this embodiment may include the following units:

[0221] The tile processing unit 1101 is used to process the voxel grid data in the three-dimensional space to obtain tile data, wherein the tile data includes multiple tile layers, each of which includes at least one grid tile, and pixels in each of the grid tiles form a cube with a pixel side length of a first number; the number of grid tiles included in the tile layers of different levels is different, and the data address ranges corresponding to the grid tiles included in the tile layers of different levels are different;

[0222] A tile loading unit 1102 is used to load a third number of target grid tiles in a second number of target tile layers in the tile data into a video memory area according to a data address range corresponding to a virtual camera angle; the second number is less than a total number of tile layers in the tile data;

[0223] The tile rendering unit 1103 is used to provide at least the storage address of the video memory area to the shader, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address and performs three-dimensional image rendering and display.

[0224] By means of the above technical solution, in a display device for voxel grid data in a three-dimensional space provided by an embodiment of the present application, by dividing the voxel grid data in the three-dimensional space into multiple tile layers containing grid tiles, it is not necessary to load all the voxel grid data into the video memory area, and only the third number of target grid tiles in the second number of target tile layers need to be loaded into the video memory area, and the voxel grid data that exceeds the data address range corresponding to the virtual camera angle does not need to be loaded into the video memory area, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address of the video memory area and renders and displays the three-dimensional image. Therefore, in this embodiment, only the video memory area is required to be able to store the third number of target grid tiles in the second number of target tile layers, and the requirements for the video memory area are not very high, so that the voxel grid data with a large amount of data can be rendered based on the limited video memory area, thereby improving the rendering performance of the three-dimensional image rendering display.

[0225] In one implementation, the tile processing unit 1101 is specifically used to: take the voxel grid data in the three-dimensional space as the current tile layer; tile the pixels in the voxel grid data in the current tile layer according to the first number to obtain the grid tiles in the current tile layer; merge the pixels of the voxel grid data in the current tile layer according to the neighboring merging rule to obtain a tile layer of a lower level; take the tile layer of the lower level as the current tile layer, and execute the pixel merging of the voxel grid data in the current tile layer according to the neighboring merging rule to obtain a tile layer of a lower level, until there is only one grid tile in the tile layer of the lower level; wherein the neighboring merging rule includes: merging a target number of pixels that are adjacent on the horizontal axis, the vertical axis and the vertical axis respectively.

[0226] In one implementation, the tile processing unit 1101 is specifically used to: take the voxel grid data in the three-dimensional space as the current tile layer; perform pixel merging on the voxel grid data in the current tile layer according to a neighbor merging rule to obtain a tile layer of a lower level; take the tile layer of the lower level as the current tile layer, and perform pixel merging on the voxel grid data in the current tile layer according to a neighbor merging rule to obtain a tile layer of a lower level, until the number of pixels in the voxel grid data in the tile layer of the lower level matches the first number; tile the pixels in the voxel grid data in the tile layer of each level according to the first number to obtain grid tiles in the tile layer of each level; wherein the neighbor merging rule includes: merging a target number of pixels that are adjacent on the horizontal axis, vertical axis and vertical axis respectively.

[0227] In one implementation, the video memory area includes a second number of sub-areas, each of which is uniquely characterized by an area identifier; each of the sub-areas includes a third number of tile areas, each of which is uniquely characterized by a tile identifier; each of the tile areas is used to store one of the grid tiles;

[0228] The tile loading unit 1102 is specifically used for:

[0229] If the video memory area has already loaded with historical grid tiles, an incremental address range is obtained according to the data address range corresponding to the virtual camera angle and the historical address range corresponding to the historical grid tiles; and according to the incremental address range, the incremental grid tiles corresponding to the incremental address range are loaded into the corresponding tile area in the corresponding sub-area of ​​the video memory area;

[0230] If no historical grid tiles are loaded in the video memory area, obtain the data address range corresponding to the second number of target tile layers according to the data address range corresponding to the virtual camera angle; and load the third number of target grid tiles in the second number of target tile layers into the corresponding tile areas in the corresponding sub-areas in the video memory area according to the data address range corresponding to the second number of target tile layers.

[0231] In one implementation, in the video memory area:

[0232] The region identifier of the sub-region into which the target tile layer is loaded in the video memory region is: the target level identifier of the target tile layer modulo the second number;

[0233] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis;

[0234] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0235] The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0236] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the horizontal axis, modulo the number of grid tiles in the sub-region on the horizontal axis multiplied by the first number;

[0237] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-region on the vertical axis multiplied by the first number;

[0238] The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number.

[0239] In one implementation, if the virtual camera angle is scaled, the incremental address range includes: a data address range corresponding to the third number of target grid tiles in a tile layer to which at least one first-level identifier belongs; the first-level identifier is obtained according to the spatial scale of the virtual camera angle scaling;

[0240] The first level identifier is continuous with the minimum target level identifier in the video memory area, or the first level identifier is continuous with the maximum target level identifier in the video memory area; and, in the video memory area, the area identifier of the sub-area to which the tile layer to which the first level identifier belongs is loaded in the video memory area is: the first level identifier modulo the second number;

[0241] If the virtual camera angle is translated, the incremental address range includes: a data address range corresponding to at least one incremental grid tile in each of the target tile layers; the incremental grid tile is obtained according to the address range of the virtual camera angle translation;

[0242] The tile identifier of the tile area to which the incremental grid tile is loaded on the horizontal axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis;

[0243] The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0244] The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis;

[0245] The storage address to which the voxel grid data in the incremental grid tile is loaded on the horizontal axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis, modulo the number of grid tiles in the sub-area on the horizontal axis multiplied by the first number;

[0246] The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number;

[0247] The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number.

[0248] In one implementation, the shader reads the voxel grid data in the following manner:

[0249] Normalizing the data address corresponding to the virtual camera viewing angle to obtain a texture address;

[0250] The highest level target tile layer in the video memory area is set as the current tile layer;

[0251] According to the normalized matrix corresponding to the current tile layer, determining whether the texture address matches the data address range corresponding to the current tile layer;

[0252] If the texture address matches the data address range corresponding to the current tile layer, read the corresponding voxel grid data from the sub-area corresponding to the current tile layer in the video memory area based on the storage address according to the product of the normalization matrix and the offset matrix corresponding to the current tile layer;

[0253] If the texture address does not match the data address range corresponding to the current tile layer, a tile layer one level lower than the current tile layer will be used as the current tile layer, and the normalization matrix corresponding to the current tile layer will be executed to determine whether the data address corresponding to the virtual camera perspective matches the data address range corresponding to the current tile layer, until the current tile layer is the lowest level target tile layer in the video memory area.

[0254] In one implementation, the normalized matrix corresponding to the current tile layer is a matrix of four rows and four columns, where:

[0255] The matrix element of the first row and the first column is: 2 raised to the power of a fourth number, and then divided by the number of grid tiles of the current tile layer on the horizontal axis; the fourth number is the target level identifier of the current tile layer;

[0256] The matrix element of the second row and the second column is: 2 raised to the power of the fourth number, divided by the number of grid tiles of the current tile layer on the vertical axis;

[0257] The matrix element in the third row and the third column is: 2 raised to the power of the fourth number, divided by the number of grid tiles of the current tile layer on the vertical axis;

[0258] The matrix element in the fourth row and fourth column is 1;

[0259] The matrix element in the fourth row and the first column is: the smallest tile identifier on the horizontal axis in the current tile layer is divided by the number of grid tiles on the horizontal axis of the current tile layer, and the result is a negative value;

[0260] The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis of the current tile layer, and the result is a negative value;

[0261] The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis of the current tile layer, and the result is a negative value;

[0262] The other matrix elements are 0.

[0263] In one implementation, the offset matrix corresponding to the current tile layer is a matrix of four rows and four columns, where:

[0264] The matrix element in the first row and first column is 1;

[0265] The matrix element in the second row and second column is 1;

[0266] The matrix element in the third row and third column is 1;

[0267] The matrix element in the fourth row and fourth column is 1;

[0268] The matrix element in the fourth row and the first column is: the smallest tile identifier on the horizontal axis in the current tile layer modulo the number of grid tiles on the horizontal axis of the current tile layer, and then divided by the number of grid tiles on the horizontal axis of the current tile layer;

[0269] The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, and then divided by the number of grid tiles on the vertical axis of the current tile layer;

[0270] The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, divided by the number of grid tiles on the vertical axis of the current tile layer;

[0271] The other matrix elements are 0.

[0272] It should be noted that the specific implementation method of each unit in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0273] An embodiment of the present application further provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0274] The memory is used to store computer programs;

[0275] The processor is used to execute the computer program so that the electronic device can implement any method for displaying voxel grid data in a three-dimensional space provided in the embodiments of the present application.

[0276] refer to Fig.12 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include but is not limited to fixed terminals such as mobile phones, notebook computers, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.12 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0277] like Fig.12 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage device 1208 to a random access memory (RAM) 1203, for implementing any method for displaying voxel grid data in a three-dimensional space provided in an embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1203. The processing device 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0278] Typically, the following devices may be connected to the I / O interface 1205: an input device 1206 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1208 including, for example, a memory card, a hard disk, etc.; and a communication device 1209. The communication device 1209 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.12 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0279] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any method for displaying voxel grid data in a three-dimensional space provided in the embodiment of the present application.

[0280] A computer storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any method for displaying voxel grid data in a three-dimensional space provided in an embodiment of the present application.

[0281] The following is an example of the specific implementation of this application. The technical solution of this application is mainly divided into the following steps:

[0282] Step 1: Split the tiles:

[0283] The voxel grid data is divided into 128×128×128 or 64×64×64 grid tiles of multiple resolution levels. The division method is similar to the way raster and image data are divided into tiles. Because the voxel grid expresses a three-dimensional continuous space, the division of tiles here adds a dimension z direction. The tiles are indexed and named by x, y, z, and level.

[0284] Here, cutting tiles can be understood as cutting a cake, cutting them into cubes with a side length of 128 pixels or 64 pixels, and numbering them in the x, y, and z directions starting from the upper left corner.

[0285] Here we need to build a multi-layer pyramid. We average eight adjacent pixels in the x, y, and z dimensions into one pixel to generate a first-level pyramid. We repeat this operation to generate a multi-layer pyramid until the resolution of the entire data is less than 128×128×128 or 64×64×64. Each layer of the pyramid corresponds to a level.

[0286] Since the data of each level is obtained by merging adjacent pixels of the data of the higher level (thumbnail), the number of x, y, and z directions of the data of the lower level is half of the number of the higher level, and the geographical range of each block in the x, y, and z directions is twice that of the higher level.

[0287] Step 2: Load the display:

[0288] Here, the display of the voxel grid is likened to looking at a large picture. The resolution of the picture is much larger than the resolution of the display screen. When the full screen is displayed, the details are not clear and you can only see a thumbnail. When you zoom in to see the details, you can only see part of the picture. If you drag it to the left, you can't see the right side. If you drag it up, you can't see the bottom. You can only see the data information with the resolution of your display screen.

[0289] Similarly, no matter how you browse, you can select 6 levels from near to far, and the data of each level with a resolution of 512×512×512 can achieve a good display effect, so the video memory is of fixed size and created in advance. Each time, you only need to replace part of the video memory according to the changes in the camera browsed by the user. The data at the closest position of the camera has the highest resolution, and the resolution gradually decreases from near to far, so as to achieve a better display effect. The specific method (taking 64*64*64 grid tiles as an example). The specific process is as follows:

[0290] Step 2.1, create video memory:

[0291] Create 6 3D textures corresponding to 6 levels of voxel grid data, with texture indexes of 0, 1, 2, 3, 4, and 5, where layer 0 is the coarsest layer and layer 5 is the finest layer. Each level can accommodate 4×4×4=64 grid tiles. That is, the size of the 3D texture of each level is 64×64×64×64=16777216 pixels.

[0292] Step 2.2, load tiles:

[0293] The pixel resolution of the data projected onto the screen is calculated based on the data range of the voxel grid (i.e., the data address range corresponding to the virtual camera angle), and 6 consecutive levels of voxel grid data tiles are loaded. The corresponding Index (3D texture index) and the Offset (pixel offset) in the x, y, and z directions of the cache for each tile data loading are determined based on the x, y, z, and level of the tile. Specifically, as shown in formulas (5) to (8):

[0294] Index=level%6 (5)

[0295] OffsetX=(x%4)×64 (6)

[0296] OffsetY=(y%4)×64 (7)

[0297] OffsetZ=(z%4)×64 (8)

[0298] For example: the storage space corresponding to the currently loaded levels 0-5, such as Fig.13 When the camera zooms in to see a more detailed level of voxel grid data, the voxel grid data of level 6 is loaded and the voxel grid data of level 0 is deleted. At this time, the voxel grid data of level 0 can be replaced with the data of level 6, as shown in Fig.14 As shown in , the storage space of layers 1-5 does not need to be changed, thereby reducing the amount of data loading and data reading.

[0299] That is to say, the location where the voxel grid data of the 6th level is stored is the location where the voxel grid data of the original 0th level is stored, so the location where each layer of data is loaded, that is, the area identifier, is determined by formula (5).

[0300] Similarly, when a translation operation is performed, the storage space changes from displaying X0Y0-X3Y3 to displaying X1Y2-X4Y5 within a layer of data (the example ignores the z direction), such as Fig.15 As shown in . The storage address can be determined by formula (6) to formula (8).

[0301] Step 2.3, data display:

[0302] Data display mainly involves the calculation of the texture matrix and the selection of texture data of the corresponding resolution (level) for sampling.

[0303] First, the original texture coordinates 0-1 represent the starting and ending positions of the data of the entire voxel grid (obtained through normalization), and the data of each level is only a small part of the whole, so the first step is to multiply the texture coordinates by a translation and scaling matrix, that is, the offset matrix, to convert the texture coordinates to the texture coordinates corresponding to a certain resolution texture. In other words, the data of a resolution currently loaded is only a part of the whole, and it is necessary to map the part to the whole.

[0304] Secondly, the voxel grid data is stored in the texture according to the x, y, and z direction numbers divided by 4, so the data as a whole has an offset, and sampling through repeated addressing mode must be multiplied by an offset matrix.

[0305] For example, Fig.16 and Fig.17 As shown in Fig.16Several consecutive tiles are randomly marked in the 0-3 levels. Among them, the texture coordinates of the four grid tiles in the 3rd level corresponding to this data address range are from the upper left corner (0.375, 0.5) to the lower right corner (0.875, 1.0). The data in this range should correspond to Fig.17 The entire data in the range of (0.0,0.0) to (1.0,1.0) requires an offset and scaling to transform (0.375,0.5)-(0.875,1.0) to (0.0,0.0)-(1.0,1.0). The offset is (-0.375,-0.5) and the scaling is (2.0,2.0).

[0306] Among them, (-0.375, -0.5) is obtained by formula (9):

[0307] (-L0X0 / (2^Level0)-L0Y0 / (2^Level0))(9)

[0308] Among them, L0X0 in formula (9) is 3, Level0 is 3, and L0Y0 is 4.

[0309] Among them, (2.0, 2.0) is obtained by formula (10):

[0310]

[0311] In formula (10), L0X1 is 6, L0X0 is 3, Level0 is 3, L0Y1 is 7, and L0Y0 is 4.

[0312] Secondly, calculate an offset, such as Fig.18 As shown in , an offset and scaling is needed to transform (0.375,0.5)-(0.875,1.0) to Fig.18 The arrows in the middle mark the start and end positions of the data, which means that an offset of 0.75 is required in the X direction.

[0313] Among them, (0.75, 0.0) is obtained by formula (11):

[0314]

[0315] In formula (9), L0X0 is 3, Level0 is 3, L0Y0 is 4, L0X1 is 6, L0X0 is 3, and L0Y1 is 7.

[0316] Based on this, the two texture matrices are the normalization matrix and the offset matrix, respectively. For example, the texture matrix of the 0th level is recorded as L0M0 (normalization matrix) and L0M1 (offset matrix), and the texture matrix of the 5th level is recorded as L5M0 (normalization matrix) and L5M1 (offset matrix).

[0317] Correspondingly, the sampling process of the shader is as follows:

[0318] First, start from the fine layer. Multiply the original texture coordinates by L5M0. If the resulting coordinate satisfies 0.0 < x < 1.0 && 0.0 < y < 1.0 && 0.0 < z < 1.0, then multiply the resulting coordinate by L5M1 to obtain the final sampling coordinate and sample using the texture of this layer. Otherwise, multiply the original coordinate by L4M0. Similarly, if it satisfies 0.0 < x < 1.0 && 0.0 < y < 1.0 && 0.0 < z < 1.0, then multiply the resulting coordinate by L4M1 to obtain the final sampling coordinate and sample using the texture of this layer. Judge in turn until finally, if the original texture coordinates multiplied by L1M0 still do not satisfy 0.0 < x < 1.0 && 0.0 < y < 1.0 && 0.0 < z < 1.0, then multiply the original coordinate by L0M0 and then by L0M1 to obtain the final sampling coordinate and sample using the texture of Level0 (the coarsest layer).

[0319] The calculation method using the texture matrix is as follows:

[0320] The 6 loaded resolution levels are Level0 to Level5,

[0321] The range of the data loaded in Level0 in the x direction is from L0X0 to L0X1, the range in the y direction is from L0Y0 to L0Y1, and the range in the z direction is from L0Z0 to L0Z1. And so on, the range of the data loaded in Level5 in the x direction is from L5X0 to L5X1, the range in the y direction is from L5Y0 to L5Y1, and the range in the z direction is from L5Z0 to L5Z1.

[0322] The texture matrices from Level0 (the coarsest layer) to Level5 (the finest layer), i.e., the normalization matrices, can be expressed as the result of multiplying a scaling matrix and a translation matrix. The translation matrix is shown in formula (12), and the scaling matrix is shown in formula (13):

[0323]

[0324]

[0325] Based on this, the normalization matrix of the 0th level as shown in formula (1) is obtained. Similarly, the normalization matrix of the 5th level as shown in formula (2) can be obtained.

[0326] The texture matrix 2 can be expressed as a translation matrix, i.e., an offset matrix, which is the offset matrix of the 0th level as shown in formula (3). Similarly, the offset matrix of the 5th level as shown in formula (4) can be obtained.

[0327] It can be seen that the technical solution of this application has at least the following advantages:

[0328] First, by dynamically updating some 3D textures and dynamically scheduling voxel raster data, the problem of large data loading scheduling is solved.

[0329] Second, the display effect and rendering performance of large amounts of volumetric raster data are guaranteed through the display method of multi-resolution three-dimensional textures.

[0330] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0331] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0332] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0333] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for displaying voxel grid data in three-dimensional space, characterized in that: The method comprises: Processing voxel grid data in a three-dimensional space to obtain tile data, wherein the tile data includes multiple tile layers, each tile layer includes at least one grid tile, and pixels in each grid tile form a cube with a first number of pixel side lengths; the number of grid tiles included in the tile layers at different levels is different; and the data address ranges corresponding to the grid tiles included in the tile layers at different levels are different; According to the data address range corresponding to the virtual camera angle, a third number of target grid tiles in the second number of target tile layers in the tile data are loaded into a video memory area; the second number is less than the total number of tile layers in the tile data; At least the storage address of the video memory area is provided to the shader, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address and performs three-dimensional image rendering and display.

2. The method according to claim 1, characterized in that Process the voxel grid data in three-dimensional space to obtain tile data, including: Using the voxel grid data in the three-dimensional space as the current tile layer; Dividing pixels in the voxel grid data in the current tile layer into tiles according to the first quantity to obtain grid tiles in the current tile layer; Merging pixels of the voxel grid data in the current tile layer according to a neighboring merging rule to obtain a tile layer at a lower level; Taking the tile layer of the lower level as the current tile layer, performing pixel merging on the voxel grid data in the current tile layer according to the neighboring merging rule to obtain the tile layer of the lower level, until there is only one grid tile in the tile layer of the lower level; The neighboring merging rule includes: merging a target number of pixels that are adjacent on the horizontal axis, the vertical axis and the vertical axis respectively.

3. The method according to claim 1, characterized in that Process the voxel grid data in three-dimensional space to obtain tile data, including: Using the voxel grid data in the three-dimensional space as the current tile layer; Merging pixels of the voxel grid data in the current tile layer according to a neighboring merging rule to obtain a tile layer at a lower level; Taking the tile layer of the lower level as the current tile layer, performing pixel merging on the voxel grid data in the current tile layer according to the neighboring merging rule to obtain a tile layer of the lower level, until the number of pixels of the voxel grid data in the tile layer of the lower level matches the first number; Perform tile division on pixels in the voxel grid data in the tile layer of each level according to the first quantity, so as to obtain grid tiles in the tile layer of each level; The neighboring merging rule includes: merging a target number of pixels that are adjacent on the horizontal axis, the vertical axis and the vertical axis respectively.

4. The method according to claim 1, 2 or 3, characterized in that: The video memory area includes a second number of sub-areas, each of which is uniquely characterized by an area identifier; each of the sub-areas includes a third number of tile areas, each of which is uniquely characterized by a tile identifier; each of the tile areas is used to store one of the grid tiles; The step of loading a third number of target grid tiles in a second number of target tile layers in the tile data into a video memory area according to a data address range corresponding to the virtual camera angle includes: If the video memory area has already loaded with historical grid tiles, an incremental address range is obtained according to the data address range corresponding to the virtual camera angle and the historical address range corresponding to the historical grid tiles; and according to the incremental address range, the incremental grid tiles corresponding to the incremental address range are loaded into the corresponding tile area in the corresponding sub-area of ​​the video memory area; If no historical grid tiles are loaded in the video memory area, obtain the data address range corresponding to the second number of target tile layers according to the data address range corresponding to the virtual camera angle; and load the third number of target grid tiles in the second number of target tile layers into the corresponding tile areas in the corresponding sub-areas in the video memory area according to the data address range corresponding to the second number of target tile layers.

5. The method according to claim 4, characterized in that In the video memory area: The region identifier of the sub-region into which the target tile layer is loaded in the video memory region is: the target level identifier of the target tile layer modulo the second number; The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis; The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; The tile identifier of the tile area to which the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding sub-area is: the data address of the target voxel grid data in the target tile layer on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the horizontal axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the horizontal axis, modulo the number of grid tiles in the sub-region on the horizontal axis multiplied by the first number; The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile region is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-region on the vertical axis multiplied by the first number; The storage address to which the voxel grid data in the target grid tile in the target tile layer is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the target grid tile in the target tile layer on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number.

6. The method according to claim 4, characterized in that If the virtual camera angle is scaled, the incremental address range includes: a data address range corresponding to the third number of target grid tiles in a tile layer to which at least one first-level identifier belongs; the first-level identifier is obtained according to a spatial scale of the virtual camera angle scaling; The first level identifier is continuous with the minimum target level identifier in the video memory area, or the first level identifier is continuous with the maximum target level identifier in the video memory area; and, in the video memory area, the area identifier of the sub-area to which the tile layer to which the first level identifier belongs is loaded in the video memory area is: the first level identifier modulo the second number; If the virtual camera angle is translated, the incremental address range includes: a data address range corresponding to at least one incremental grid tile in each of the target tile layers; the incremental grid tile is obtained according to the address range of the virtual camera angle translation; The tile identifier of the tile area to which the incremental grid tile is loaded on the horizontal axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis modulo the number of grid tiles in the sub-area on the horizontal axis; The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; The tile identifier of the tile area to which the incremental grid tile is loaded on the vertical axis in the corresponding sub-area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis modulo the number of grid tiles in the sub-area on the vertical axis; The storage address to which the voxel grid data in the incremental grid tile is loaded on the horizontal axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the horizontal axis, modulo the number of grid tiles in the sub-area on the horizontal axis multiplied by the first number; The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number; The storage address to which the voxel grid data in the incremental grid tile is loaded on the vertical axis in the corresponding tile area is: the data address of the voxel grid data in the incremental grid tile on the vertical axis, modulo the number of grid tiles in the sub-area on the vertical axis multiplied by the first number.

7. The method according to claim 1, characterized in that The shader reads the voxel grid data in the following way: Normalizing the data address corresponding to the virtual camera viewing angle to obtain a texture address; The highest level target tile layer in the video memory area is set as the current tile layer; According to the normalized matrix corresponding to the current tile layer, determining whether the texture address matches the data address range corresponding to the current tile layer; If the texture address matches the data address range corresponding to the current tile layer, read the corresponding voxel grid data from the sub-area corresponding to the current tile layer in the video memory area based on the storage address according to the product of the normalization matrix and the offset matrix corresponding to the current tile layer; If the texture address does not match the data address range corresponding to the current tile layer, a tile layer one level lower than the current tile layer will be used as the current tile layer, and the normalization matrix corresponding to the current tile layer will be executed to determine whether the data address corresponding to the virtual camera perspective matches the data address range corresponding to the current tile layer, until the current tile layer is the lowest level target tile layer in the video memory area.

8. The method according to claim 7, characterized in that The normalized matrix corresponding to the current tile layer is a matrix with four rows and four columns, where: The matrix element of the first row and the first column is: 2 raised to the power of a fourth number, and then divided by the number of grid tiles of the current tile layer on the horizontal axis; the fourth number is the target level identifier of the current tile layer; The matrix element of the second row and the second column is: 2 raised to the power of the fourth number, and then divided by the number of grid tiles of the current tile layer on the vertical axis; The matrix element in the third row and the third column is: 2 raised to the power of the fourth number, divided by the number of grid tiles of the current tile layer on the vertical axis; The matrix element in the fourth row and fourth column is 1; The matrix element in the fourth row and the first column is: the smallest tile identifier on the horizontal axis in the current tile layer is divided by the number of grid tiles on the horizontal axis of the current tile layer, and the result is a negative value; The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis of the current tile layer, and the result is a negative value; The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer divided by the number of grid tiles on the vertical axis of the current tile layer, and the result is a negative value; The other matrix elements are 0.

9. The method according to claim 7, characterized in that: The offset matrix corresponding to the current tile layer is a matrix of four rows and four columns, where: The matrix element in the first row and first column is 1; The matrix element in the second row and second column is 1; The matrix element in the third row and third column is 1; The matrix element in the fourth row and fourth column is 1; The matrix element in the fourth row and the first column is: the smallest tile identifier on the horizontal axis in the current tile layer modulo the number of grid tiles on the horizontal axis of the current tile layer, and then divided by the number of grid tiles on the horizontal axis of the current tile layer; The matrix element in the fourth row and second column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, and then divided by the number of grid tiles on the vertical axis of the current tile layer; The matrix element in the fourth row and third column is: the smallest tile identifier on the vertical axis in the current tile layer modulo the number of grid tiles on the vertical axis of the current tile layer, divided by the number of grid tiles on the vertical axis of the current tile layer; The other matrix elements are 0.

10. A display device for voxel grid data in three-dimensional space, characterized in that: The device comprises: A tile processing unit, used for processing voxel grid data in a three-dimensional space to obtain tile data, wherein the tile data includes multiple tile layers, each of which includes at least one grid tile, and pixels in each of the grid tiles form a cube with a first number of pixel side lengths; the number of grid tiles included in the tile layers at different levels is different, and the data address ranges corresponding to the grid tiles included in the tile layers at different levels are different; A tile loading unit, configured to load a third number of target grid tiles in a second number of target tile layers in the tile data into a video memory area according to a data address range corresponding to a virtual camera angle; the second number is less than a total number of tile layers in the tile data; The tile rendering unit is used to provide at least the storage address of the video memory area to the shader, so that the graphics processor runs the shader, and the shader reads the voxel grid data according to the storage address and performs three-dimensional image rendering and display.

Images