Three-dimensional grid data interpolation rendering method, device, equipment, medium and product

Abstract

The application discloses a three-dimensional grid data interpolation rendering method, device, equipment, medium and product, and relates to the technical field of image processing. The method comprises the following steps: performing multi-level down-sampling aggregation on a plurality of three-dimensional grid tiles to generate a tile pyramid; determining target tile features of each three-dimensional grid tile in the tile pyramid; determining a model based on an interpolation strategy, determining a target interpolation strategy of the three-dimensional grid tile according to the target tile features; performing interpolation processing on the three-dimensional grid tile in the tile pyramid according to the target interpolation strategy of the three-dimensional grid tile in parallel, obtaining interpolation tile data, and storing the interpolation tile data in a GPU internal memory buffer; determining to-be-rendered grid data from the interpolation tile data according to a viewpoint parameter of a current viewpoint and the three-dimensional grid tile in the tile pyramid; and performing volume rendering on the to-be-rendered grid data to generate a rendered image. The above scheme can improve the interpolation and rendering efficiency and precision of the three-dimensional grid data.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to the field of computer graphics and spatial data processing technology, specifically to a three-dimensional raster data interpolation rendering method, apparatus, device, medium, and product. Background Technology

[0002] 3D raster data is a type of 3D volumetric data organized in a regular spatial grid format. It uses voxels as the basic unit to regularly divide a continuous 3D space, with each voxel storing the attribute value of that spatial location. With the rapid development of sensor technology and data acquisition methods, the accuracy of 3D raster data acquisition has improved from meters to centimeters, and its coverage has expanded from local areas to global scales, resulting in an exponential increase in data volume. For example, a data structure covering 100km... 2 Three-dimensional geological attribute data with a regional resolution of 1m can have a voxel count of up to 10¹¹, with a total data volume of over 400GB, which poses a serious challenge to subsequent data processing and visualization rendering.

[0003] In existing technologies, the interpolation and rendering processes of 3D raster data are typically handled separately. In the interpolation stage, mainstream algorithms include inverse distance weighted interpolation (IDW), kriging interpolation, and spline interpolation. Among these, while kriging interpolation provides the best linear unbiased estimate, its computational complexity is O(n^2). 3 (where n is the number of sample points), and processing time increases exponentially with the number of sample points. In geological data scenarios of 2048×2048×512 scale, traditional CPU-based Kriging interpolation takes approximately 45 minutes, which is completely unacceptable for real-time requirements. Inverse distance-weighted interpolation only considers planar distance as weight, and when the variable values ​​within the vicinity of the interpolation point exhibit strong variability, the interpolation error exceeds 15%. Furthermore, most existing interpolation methods rely on single-core CPU computation, and even with parallel optimization techniques such as Web Workers, efficiency improvements are very limited.

[0004] In recent years, some studies have attempted to migrate interpolation calculations to the GPU to improve efficiency. However, these solutions generally adopt a "one-size-fits-all" algorithm selection strategy, that is, using the same interpolation algorithm for all data regions, failing to adaptively balance accuracy and efficiency based on data characteristics. At the same time, existing GPU interpolation schemes still adopt the "pre-calculate all interpolation results → transmit to the GPU → render" model, performing a large amount of invalid calculations on invisible areas, and failing to solve the data transmission overhead problem caused by separating interpolation and rendering.

[0005] In the rendering process, the visualization of 3D raster data typically employs volumetric rendering techniques. However, loading massive amounts of volumetric raster data into video memory for rendering places extremely high demands on hardware resources. Existing solutions often employ Level of Detail (LOD) techniques or tile-based strategies for optimization, but these solutions select the LOD level solely based on viewpoint distance and screen projection area, lacking an awareness of data complexity. This results in insufficient rendering quality in areas with drastic data changes, while wasting computational resources in areas with relatively flat data.

[0006] Furthermore, existing technologies have significant shortcomings in data updates. When 3D raster data undergoes a local update, it is usually necessary to recalculate the interpolation of the entire dataset, resulting in extremely low update efficiency. At the same time, GPU memory cache management mechanisms are relatively simple, relying solely on access frequency for eviction and failing to consider the impact of the spatial distance between tiles and the viewpoint on rendering quality.

[0007] In summary, the existing technology has the following main shortcomings: First, the interpolation calculation is inefficient and cannot adaptively select the optimal interpolation strategy based on data characteristics, making it difficult to balance accuracy and performance; Second, the separation of interpolation and rendering leads to a large amount of unnecessary data transmission overhead and invalid calculations; Third, the rendering scheme lacks intelligent perception of data complexity, and the LOD scheduling strategy is not reasonable enough; Fourth, the data update efficiency is low, and GPU memory resources are not fully utilized. Summary of the Invention

[0008] This invention provides a method, apparatus, device, medium, and product for interpolating and rendering three-dimensional raster data, in order to solve the technical problems of low efficiency, difficulty in balancing accuracy and performance, lack of intelligent adaptive capabilities, and slow data updates in the interpolation calculation and rendering process of large-scale three-dimensional raster data.

[0009] According to one aspect of the present invention, a three-dimensional raster data interpolation rendering method is provided, the method comprising: Multi-level downsampling and aggregation of multiple 3D raster tiles are performed to generate a tile pyramid; Determine the target tile features of each three-dimensional grid tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; The model is determined based on the interpolation strategy, and the target interpolation strategy of the three-dimensional grid tile is determined according to the target tile features. In parallel, the three-dimensional grid tiles in the tile pyramid are interpolated according to the target interpolation strategy of the three-dimensional grid tiles to obtain interpolated tile data, and the interpolated tile data is stored in the GPU memory buffer. Based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid, determine the raster data to be rendered from the interpolated tile data; The raster data to be rendered is volumetric rendered to generate a rendered image.

[0010] According to another aspect of the present invention, a three-dimensional raster data interpolation rendering apparatus is provided, the apparatus comprising: The tile pyramid determination module is used to perform multi-level downsampling aggregation on multiple 3D grid tiles to generate a tile pyramid; The tile feature determination module is used to determine the target tile features of each three-dimensional grid tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; An interpolation strategy determination module is used to determine a model based on an interpolation strategy, and to determine the target interpolation strategy for the three-dimensional raster tile according to the target tile features. An interpolation processing module is used to perform interpolation processing on the three-dimensional grid tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional grid tiles, to obtain interpolated tile data, and to store the interpolated tile data in the GPU memory buffer. The module for determining data to be rendered is used to determine the raster data to be rendered from the interpolated tile data based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid. The rendering module is used to perform volume rendering on the raster data to be rendered and generate a rendered image.

[0011] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the three-dimensional raster data interpolation rendering method according to any embodiment of the present invention.

[0012] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the three-dimensional raster data interpolation rendering method according to any embodiment of the present invention.

[0013] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the three-dimensional raster data interpolation rendering method according to any embodiment of the present invention.

[0014] The technical solution of this invention generates a tile pyramid by performing multi-level downsampling aggregation on multiple three-dimensional raster tiles; determines the target tile features of each three-dimensional raster tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; determines a model based on an interpolation strategy, and determines the target interpolation strategy for the three-dimensional raster tiles according to the target tile features; performs interpolation processing on the three-dimensional raster tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional raster tiles to obtain interpolated tile data, and stores the interpolated tile data in the GPU memory buffer; determines the raster data to be rendered from the interpolated tile data according to the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid; and performs volume rendering on the raster data to be rendered to generate a rendered image. The above technical solution determines the model through interpolation strategy and outputs a three-in-one strategy parameter set that integrates interpolation algorithm selection, hyperparameter configuration, and rendering sampling accuracy. This avoids the one-size-fits-all algorithm selection strategy of traditional solutions. At the same time, it migrates the interpolation calculation to the GPU and stores the interpolation results directly in the GPU memory buffer for the rendering pipeline to call, realizing the full GPU-based interpolation-rendering process.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a three-dimensional raster data interpolation rendering method provided by an embodiment of the present invention; Figure 2 This is a flowchart of a three-dimensional raster data interpolation rendering method provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an interpolation strategy determination model provided by an embodiment of the present invention; Figure 4 This is a flowchart of a three-dimensional raster data interpolation rendering method provided by an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a three-dimensional raster data interpolation rendering device according to an embodiment of the present invention; Figure 6This is a schematic diagram of the structure of an electronic device that implements the three-dimensional raster data interpolation rendering method of this invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Furthermore, it should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of data related to three-dimensional grid tiles involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0021] Figure 1 This is a flowchart illustrating a three-dimensional raster data interpolation rendering method according to an embodiment of the present invention. This embodiment is applicable to the visualization of massive three-dimensional spatial field data in fields such as digital twins, geological exploration, meteorological simulation, and marine environmental monitoring. The method can be executed by a three-dimensional raster data interpolation rendering device, which can be implemented in hardware and / or software. This device can be configured in an electronic device carrying the three-dimensional raster data interpolation rendering function, such as a server. Figure 1 As shown, the method includes: S110. Perform multi-level downsampling aggregation on multiple three-dimensional grid tiles to generate a tile pyramid.

[0022] In this embodiment, a tile pyramid refers to a tile pyramid structure composed of three-dimensional grid tiles layer by layer; the construction of the pyramid follows a bottom-up, layer-by-layer aggregation strategy.

[0023] An optional approach involves performing multi-level downsampling aggregation on multiple 3D raster tiles to generate a tile pyramid, including: using the 3D raster tiles at the original resolution level as the bottom layer of the pyramid; aggregating adjacent tiles in the current level according to a preset spatial proximity merging rule, and merging the effective voxels of multiple 3D raster tiles using a weighted average method during aggregation to generate tile data at a low resolution level, until a preset minimum resolution threshold is met to obtain the tile pyramid.

[0024] Specifically, the original resolution 3D raster tiles are used as the 0th layer (bottom layer) of the pyramid. For the kth layer (k≥0), adjacent tiles are aggregated according to a preset spatial proximity merging rule, such as 2×2×2, i.e., two adjacent tiles are taken in each of the three directions of 3D space, and every 8 adjacent tiles are merged into one parent tile. During aggregation, the effective voxels of the 8 child tiles are weighted and averaged one voxel at a time, with the weights being the normalized values ​​of the effective voxel count of each child tile to maintain data consistency. The parent tile is... attribute values ​​of individual elements The calculation is as follows: ; in, Let j be the data density of the j-th sub-tile. The attribute value is the voxel corresponding to the j-th sub-tile.

[0025] The assembled tiles form the first layer of the pyramid. Layer. Repeat the above aggregation process until the aggregated tiles meet the preset minimum resolution threshold, for example, stop when the entire layer contains only one tile, and finally form an L-level tile pyramid structure from coarse to fine.

[0026] Understandably, by converting 3D raster tiles into a pyramid structure, with each layer of the pyramid corresponding to a different resolution, it is not necessary to load all 3D raster tiles during real-time rendering; only the 3D raster tiles corresponding to the corresponding layer are needed. This solves the problem of real-time roaming rendering of ultra-large 3D raster tiles.

[0027] S120. Determine the target tile features of each three-dimensional grid tile in the tile pyramid.

[0028] In this embodiment, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map. The sparsity feature map characterizes the distribution of valid and invalid data within the 3D raster tile and is a 3D feature map consistent with the spatial dimensions of the 3D raster tile. The spatial gradient feature map characterizes the degree of difference between voxels within the 3D raster tile and is a 3D feature map consistent with the spatial dimensions of the 3D raster tile. The spatial correlation feature map characterizes the attribute distribution differences between adjacent 3D raster tiles and is a 3D feature map consistent with the spatial dimensions of the 3D raster tile.

[0029] An alternative approach to determine the target tile features of each 3D grid tile in a tile pyramid includes: determining the tile density based on the quotient of the number of effective voxels in the 3D grid tile and the total number of voxels in the 3D grid tile; determining the data sparsity of the 3D grid tile based on the tile density, and determining a sparsity feature map based on the data sparsity; determining a gradient feature map based on the gradient magnitude of voxels in the 3D grid tile; and determining a spatial association feature map based on the attribute differences between the 3D grid tile and its neighboring tiles.

[0030] In this context, a valid voxel refers to a voxel that contains valid attribute values, rather than null or padded values. Tile density refers to the data density of a tile.

[0031] Specifically, the data sparsity value of each 3D raster tile is calculated, and the quotient of the number of effective voxels in the 3D raster tile and the total number of voxels in the 3D raster tile is taken as the tile density. : ;in, The number of effective voxels within the tile. Let be the total number of voxels within the tile. Then, based on the following formula, the data sparsity of the 3D raster tile is determined according to the binary information entropy of the tile density. : The value range of this indicator is: A larger value indicates a more complex distribution of valid and invalid data within the tile, and consequently, a higher difficulty in interpolation processing. Finally, the data sparsity of the 3D raster tile is extended to a 3D feature map consistent with the spatial dimensions of the 3D raster tile.

[0032] Next, by examining the tiles Internal elements in The gradient magnitudes are obtained by calculating the first-order differences in three directions, forming a gradient feature map. (Vollette) gradient magnitude at The calculation is as follows: ; The gradient magnitude is expanded into a three-dimensional feature map with the same spatial size as the three-dimensional raster tile, thus obtaining the gradient feature map.

[0033] Then, by quantifying the attribute value distribution differences between the current tile and its six adjacent tiles (using KL divergence metric), the attribute distribution differences are expanded into a 3D feature map consistent with the spatial dimensions of the 3D grid tiles, thus obtaining the associated control feature map. Among these features, the current tile... With adjacent tiles The KL divergence is calculated as follows: ; in, Number of bins for attribute values The current tile attribute value falls within the first... The probability of each bin. For adjacent tile attribute values ​​falling within the first The probability of each bin.

[0034] It is understandable that by determining the sparsity characteristics, spatial gradient characteristics, and spatial correlation characteristics of the tiles, the interpolation strategy for the tiles can be determined more accurately in the future.

[0035] S130. Determine the model based on the interpolation strategy. Based on the characteristics of the target tile, determine the target interpolation strategy for the three-dimensional raster tile.

[0036] In this embodiment, the interpolation strategy determination model is a lightweight convolutional neural network model used to predict the interpolation algorithm selection identifier, interpolation algorithm hyperparameter configuration, and rendering sampling accuracy level of the interpolation strategy parameter set. The target interpolation strategy includes the algorithm selection identifier, hyperparameter configuration, and rendering sampling accuracy level. The algorithm selection identifier is a three-element probability vector, corresponding to the selection probabilities of Kriging interpolation, inverse distance weighted interpolation, and fast linear interpolation, respectively. The algorithm with the highest probability is selected as the interpolation algorithm for the current tile. In the hyperparameter configuration, if Kriging interpolation is selected, it includes the selection identifier of the semivariogram function model type (spherical model, exponential model, or Gaussian model) and the predicted values ​​of the range and sill value; if inverse distance weighted interpolation is selected, it includes the predicted value of the distance weight exponent; if fast linear interpolation is selected, there are no additional hyperparameters. The rendering sampling accuracy level is an integer value ranging from 0 to 3, indicating the recommended sampling step size for rendering the tile; a larger value indicates a higher accuracy requirement.

[0037] Specifically, the target tile features are input into the interpolation strategy to determine the model. After model decision-making, the target interpolation strategy for the three-dimensional raster tiles is obtained.

[0038] S140. In parallel, based on the target interpolation strategy of the three-dimensional raster tiles, the three-dimensional raster tiles in the tile pyramid are interpolated to obtain interpolated tile data, and the interpolated tile data is stored in the GPU memory buffer.

[0039] In this embodiment, interpolated tile data refers to the three-dimensional raster tiles after interpolation.

[0040] Specifically, based on the target interpolation strategy of the 3D raster tiles, the corresponding GPU parallel interpolation kernel is dynamically selected, and each voxel to be interpolated in the 3D raster tiles is mapped to a single computing thread of the GPU. Each thread performs interpolation calculations according to the corresponding interpolation strategy parameters, and the interpolated tile data is directly stored in the GPU memory buffer.

[0041] S150. Determine the raster data to be rendered from the interpolated tile data based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid.

[0042] In this embodiment, the raster data to be rendered refers to the raster data that needs to be rendered.

[0043] Specifically, during the rendering phase, the spatial distance between the 3D raster tiles in the tile pyramid and the viewpoint, as well as the screen projection area, are calculated. The target resolution level of each tile to be rendered is determined by combining the rendering sampling accuracy level in the target interpolation strategy. Interpolated tile data that has been interpolated is preferentially obtained from the GPU memory buffer and used as raster data to be rendered. For tile data that has not yet been interpolated, the GPU parallel interpolation calculation of S140 is dynamically triggered, and the interpolated tile data is used as raster data to be rendered.

[0044] S160. Perform volume rendering on the raster data to be rendered to generate a rendered image.

[0045] Optionally, volume rendering is performed on the raster data to be rendered in the GPU rendering pipeline to generate a rendered image. First, a 3D texture object is constructed, packing the interpolated tile data of the tiles to be rendered (containing the attribute values ​​of each voxel) into a set of 3D textures. If the data volume exceeds the size limit of a single texture, a texture array is used. In the vertex shader stage, a proxy geometry for the rendering volume is constructed, typically a cube bounding box covering the 3D space of all the raster tiles to be rendered, and the position of each vertex in screen space is calculated. In the fragment shader stage, ray stepping volume rendering is performed for each pixel. The specific process is as follows: Starting from the viewpoint, the ray direction vector passing through the current pixel is calculated, determining the intersection point of the ray entering and leaving the rendering volume for ray initialization. Then, starting from the ray entry point, ray stepping is performed along the ray direction in steps of... Proceed with equal steps. Based on rendering sampling precision level Dynamic adjustment: hour , hour , hour , hour At each sampling position, the 3D texture is queried via texture coordinate mapping to obtain the attribute value at that location for ray stepping. The attribute value is then mapped to RGB color values ​​and an absorption coefficient (opacity) using a preset transfer function. The transfer function can be predefined according to application requirements; for example, geological data can map different attribute value ranges to different geological color scales. Color and opacity are accumulated using a front-to-back volumetric rendering integral method for color synthesis. ; ; in, and The first Cumulative color and cumulative opacity after step sampling and For the first The color and opacity of each sample point. Initial values ​​are... .

[0046] When cumulative opacity When the value exceeds 0.98, the light is essentially completely blocked by the medium, and the contribution of subsequent sampling points is negligible, so the light stepping is terminated early. The final accumulated color value is then calculated. Write to the corresponding pixel position in the frame buffer.

[0047] In addition, during the rendering process, the gradient of each sampling position is calculated based on the scene lighting conditions as the normal direction, and the Blinn-Phong lighting model is applied for shading enhancement to improve the three-dimensional sense and detail.

[0048] The technical solution of this invention generates a tile pyramid by performing multi-level downsampling aggregation on multiple three-dimensional raster tiles; determines the target tile features of each three-dimensional raster tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; determines a model based on an interpolation strategy, and determines the target interpolation strategy for the three-dimensional raster tiles according to the target tile features; performs interpolation processing on the three-dimensional raster tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional raster tiles to obtain interpolated tile data, and stores the interpolated tile data in the GPU memory buffer; determines the raster data to be rendered from the interpolated tile data according to the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid; and performs volume rendering on the raster data to be rendered to generate a rendered image. The above technical solution determines the model through interpolation strategy and outputs a three-in-one strategy parameter set that integrates interpolation algorithm selection, hyperparameter configuration, and rendering sampling accuracy. This avoids the one-size-fits-all algorithm selection strategy of traditional solutions. At the same time, it migrates the interpolation calculation to the GPU and stores the interpolation results directly in the GPU memory buffer for the rendering pipeline to call, realizing the full GPU-based interpolation-rendering process.

[0049] Based on the above embodiments, before generating the tile pyramid by multi-level downsampling aggregation of multiple three-dimensional raster tiles, the method further includes: spatial normalization processing of the original three-dimensional raster data to obtain multiple three-dimensional raster tiles, and establishing a spatial index for each three-dimensional raster tile.

[0050] Specifically, the original 3D raster data in the entire raster dataset is scanned, and the minimum and maximum coordinate values ​​in each direction of 3D space are recorded to determine the 3D spatial range of the raster data. The attribute values ​​of all valid voxels are statistically analyzed, and the minimum and maximum attribute values ​​are recorded to determine the attribute value range. Then, based on a preset 3D tile size (preferably 64×64×64 voxels), the original 3D raster data is divided into multiple 3D raster tiles. Each tile constitutes a regular cubic sub-region in 3D space. For tiles whose boundaries are smaller than the preset size, zero-padding is used to fill the gaps. A spatial index is established for each 3D raster tile; for example, an octree index structure can be used to map the spatial location of the tile (represented by tile coordinates) to its data storage address.

[0051] It is understandable that by normalizing the original 3D raster data, the coordinate reference and attribute dimensions of raster data from different sources and at different scales can be unified.

[0052] Figure 2 This is a flowchart of a three-dimensional raster data interpolation rendering method according to an embodiment of the present invention; this embodiment, based on the above embodiment, elaborates in detail the training process of the interpolation strategy determination model. Figure 2 As shown, the method includes: S210. Perform multi-level downsampling aggregation on multiple three-dimensional grid tiles to generate a tile pyramid.

[0053] S220. Determine the target tile features of each three-dimensional grid tile in the tile pyramid.

[0054] Among them, the target tile features include sparsity feature map, spatial gradient feature map and spatial correlation feature map; S230. Determine the model based on the interpolation strategy. Based on the characteristics of the target tile, determine the target interpolation strategy for the three-dimensional raster tile.

[0055] S240: In parallel, based on the target interpolation strategy of the three-dimensional raster tiles, the three-dimensional raster tiles in the tile pyramid are interpolated to obtain interpolated tile data, and the interpolated tile data is stored in the GPU memory buffer.

[0056] S250. Determine the raster data to be rendered from the interpolated tile data based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid.

[0057] S260. Perform volume rendering on the raster data to be rendered to generate a rendered image.

[0058] Optionally, the interpolation strategy determines that the model includes at least three consecutive 3D convolutional modules, a global average pooling layer, and at least two fully connected layers, such as... Figure 3 As shown, the three-dimensional convolutional module includes a three-dimensional convolutional layer, a batch normalization layer, and a ReLU activation function layer; the number of convolutional kernels in the three three-dimensional convolutional modules are 16, 32, and 32, respectively.

[0059] For example, the interpolation strategy determination model is trained as follows: The sample tile features of the sample 3D tiles are convolved using a 3D convolution module to obtain 3D features; the 3D features are reduced in dimensionality using a global average pooling layer to obtain dimensionality-reduced features; the dimensionality-reduced features are predicted using a fully connected layer to obtain the predicted interpolation strategy and the predicted interpolation error; the strategy loss is determined based on the predicted interpolation strategy and the interpolation strategy label; the error loss is determined based on the predicted interpolation error and the true interpolation error; the strategy loss and the error loss are weighted and summed to obtain the training loss, and the interpolation strategy determination model is iteratively trained using the training loss.

[0060] The sample tile features include sparse feature maps, spatial gradient feature maps, and spatial correlation feature maps. Three-dimensional features refer to the features after convolution processing. Dimensionally reduced features refer to the features after dimensionality reduction through pooling layers, resulting in one-dimensional feature vectors. The true interpolation error is the interpolation error obtained by performing interpolation on each sample three-dimensional tile using the three interpolation algorithms respectively. For example, the root mean square error (RMSE) can be used to measure the true interpolation error.

[0061] Specifically, the sample 3D tile features are input into a 3D convolution module for 3D convolution processing to obtain 3D features. These 3D features are then input into a global average pooling layer for dimensionality reduction, resulting in dimensionality-reduced features. These dimensionality-reduced features are then input into a fully connected layer for prediction, outputting an interpolation strategy prediction parameter set. This set includes the predicted interpolation strategy and the predicted interpolation error. Based on the cross-entropy loss function, the strategy loss is determined according to the predicted interpolation strategy and its label. As shown in the formula below: ;in, For the first The true label of the algorithm is the interpolation strategy label (1 for the optimal algorithm and 0 for the rest). The first output of the model The algorithm predicts the probability of the interpolation strategy. Then, based on the mean squared error loss function, the error loss is determined according to the predicted interpolation error and the actual interpolation error. As shown in the formula below: ;in, For batch size, For the first The true interpolation error for each sample This represents the interpolation error predicted by the model. Finally, the policy loss and error loss are weighted and summed to obtain the training loss. ,Right now The interpolation strategy is determined by using training loss to iteratively train the model until at least the training stopping condition is met, at which point training stops; wherein the training stopping condition is that the number of training iterations reaches a set number or the training loss reaches a set value; wherein the set number of iterations and the set value are set by those skilled in the art according to actual needs.

[0062] It should be noted that the Adam optimizer was used to train the model during training, with a batch size of 32, an initial learning rate of 0.001, and the learning rate was reduced to 0.5 every 10 rounds, for a total of 50 training rounds.

[0063] Understandably, by introducing a lightweight 3D convolutional neural network model and simultaneously outputting a three-in-one strategy parameter set encompassing interpolation algorithm selection, hyperparameter configuration, and rendering sampling accuracy, the traditional "one-size-fits-all" algorithm selection strategy is avoided, achieving an adaptive global optimal balance between interpolation accuracy and computational efficiency.

[0064] It should be noted that the sample 3D tiles are a collection of 100,000 sample tiles from different fields such as geology, meteorology, and oceanography, with different data characteristics.

[0065] The technical solution of this invention generates a tile pyramid by performing multi-level downsampling aggregation on multiple three-dimensional raster tiles; determines the target tile features of each three-dimensional raster tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; determines a model based on an interpolation strategy, and determines the target interpolation strategy for the three-dimensional raster tiles according to the target tile features; performs interpolation processing on the three-dimensional raster tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional raster tiles to obtain interpolated tile data, and stores the interpolated tile data in the GPU memory buffer; determines the raster data to be rendered from the interpolated tile data according to the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid; and performs volume rendering on the raster data to be rendered to generate a rendered image. The above technical solution determines the model through interpolation strategy and outputs a three-in-one strategy parameter set that integrates interpolation algorithm selection, hyperparameter configuration, and rendering sampling accuracy. This avoids the one-size-fits-all algorithm selection strategy of traditional solutions. At the same time, it migrates the interpolation calculation to the GPU and stores the interpolation results directly in the GPU memory buffer for the rendering pipeline to call, realizing the full GPU-based interpolation-rendering process.

[0066] Based on the above embodiments, as an optional approach of the present invention, the three-dimensional raster tiles in the tile pyramid are interpolated in parallel according to the target interpolation strategy of the three-dimensional raster tiles to obtain interpolated tile data. This includes: if the target interpolation strategy is Kriging interpolation, each thread block is responsible for a first number of voxel regions. Threads within the thread block collaboratively search for the second number of sample points closest to the voxel to be interpolated in the three-dimensional raster tile through shared memory, and solve the Kriging equation system using the LU decomposition method to obtain the interpolated three-dimensional raster tile; if the target interpolation strategy is inverse distance weighted interpolation, each thread independently searches for the third number of sample points closest to the voxel to be interpolated in the three-dimensional raster tile for calculation to obtain the interpolated three-dimensional raster tile; if the target interpolation strategy is fast linear interpolation, each thread directly uses the attribute values ​​of effective voxels in the neighborhood to perform an arithmetic average to obtain the interpolated tile data.

[0067] Specifically, based on the strategy parameter set in the target interpolation strategy, the corresponding GPU parallel interpolation kernel is dynamically selected, and interpolation calculations are performed in parallel on the GPU. The voxels to be interpolated in the 3D raster tiles are organized into a 3D threaded mesh, with each voxel corresponding to one computation thread of the GPU, and the thread block size is 8×8×8.

[0068] During the execution phase, the policy parameter set and neighborhood sample point dataset of the target interpolation strategy are first loaded into the GPU device via constant memory or texture memory. The sample point dataset is a set of valid voxels with known attribute values ​​within the raster tiles, organized using a fast neighborhood search algorithm based on spatial hashing. Then, the corresponding interpolation calculation kernel is started.

[0069] The execution flow of each computation thread is as follows: Determine the coordinates of the voxel to be interpolated corresponding to the current thread. Check if the voxel already contains a valid attribute value: if it does, skip it and keep the original value; if the voxel is empty or invalid and needs interpolation, select the corresponding interpolation calculation branch according to the algorithm selection flag in the interpolation strategy parameter set.

[0070] When Kriging interpolation is selected: each thread block is responsible for a first number of voxel regions, such as 8×8×8. Threads within a thread block collaborate through shared memory to complete the following operations: search for a second number of sample points, such as 64, that are closest to the current voxel to be interpolated in the neighborhood sample point dataset; use the spatial distance between these sample points as input, calculate the semivariance value according to the specified semivariance function model, and construct the Kriging equation system; solve the Kriging weight coefficients using the LU decomposition method under the collaboration of multiple threads within the thread block; finally, each thread uses the obtained weight coefficients and sample point attribute values ​​to calculate the interpolation result for its corresponding voxel to be interpolated.

[0071] The final result of Kriging interpolation is calculated using the following formula: ; in, The number of sample points involved in the interpolation (K=64). For the first Kriging weight coefficients for each sample point For the first The attribute values ​​of each sample point.

[0072] When inverse distance weighted interpolation is selected: each thread performs interpolation calculations independently. Each thread searches the neighborhood sample point dataset for the third nearest sample point (e.g., 32 points) to the current voxel to be interpolated. For each sample point... Calculate the three-dimensional spatial distance between it and the voxel to be interpolated: ; in, Represents three-dimensional spatial distance. For sample points The coordinates.

[0073] The interpolation result is calculated using the inverse distance weighted formula: ; in, For the 32 most recent sample points The set, The distance weighted exponent (provided by the hyperparameter configuration in step S130), For sample points The attribute value.

[0074] When choosing the fast linear interpolation method: each thread searches for valid voxels within a 2×2×2 neighborhood of the voxel to be interpolated, and performs a simple arithmetic average on the attribute values ​​of the found valid voxels to obtain the interpolation result in the fastest way: ; Where M is the number of effective voxels in a 2×2×2 neighborhood. For the first The attribute value of each effective voxel.

[0075] After all threads have completed the interpolation calculation, they write the interpolation results back to the result buffer in the GPU's global memory.

[0076] Understandably, migrating interpolation calculations to the GPU and storing the interpolation results directly in the GPU memory buffer for use by the rendering pipeline can improve overall processing efficiency.

[0077] As another optional aspect of the present invention, it further includes: if a local update of the three-dimensional grid tile is detected, the target interpolation strategy of the three-dimensional grid tile is re-determined based on the interpolation strategy determination model, and the three-dimensional grid tile is interpolated and updated according to the new target interpolation strategy.

[0078] Specifically, when the 3D raster data undergoes a local update (such as when a new batch of sensor data covers part of the spatial area, or when the attribute values ​​of a local area change after the data is recalibrated), an incremental update process is triggered. First, the 3D bounding box of the changed area is calculated. All tiles intersecting with this bounding box are queried using the spatial index, and all parent tiles containing these tiles are traced upwards in the tile pyramid, marking them all as "to be updated." For the marked tiles, the interpolation strategy decision in S230 and the GPU parallel interpolation calculation in S240 are re-executed. The remaining tiles unaffected by the changed area retain their original interpolation results.

[0079] Understandably, timely updates to the 3D raster tiles ensure the real-time accuracy and validity of the 3D raster data.

[0080] As another optional aspect of the present invention, it further includes: determining the priority weight corresponding to the interpolated tile data based on the spatial distance between the tile center point of the interpolated tile data and the current viewpoint and the base distance threshold; when the capacity of the GPU memory buffer reaches the upper limit, determining the elimination score of the interpolated tile data based on the access timestamp, access frequency calculator and priority weight corresponding to the interpolated tile data; and updating the interpolated tile data in the GPU memory cache based on the elimination score.

[0081] Specifically, a GPU memory cache management mechanism is constructed based on a least recently used strategy that combines access frequency and viewpoint distance. A GPU memory cache is maintained to store recently used interpolation result tile data. Each cached item is maintained with an access timestamp, access frequency counter, and priority weight.

[0082] First, based on the following formula, the priority weight of the interpolated tile data is determined according to the spatial distance between the tile center point of the interpolated tile data and the current viewpoint, and the base distance threshold: ;in, Indicates priority weight. The spatial distance between the center point of the tile and the current viewpoint. Based on the distance threshold. The value range is (0,1], and the closer the distance, the higher the priority.

[0083] When the GPU memory buffer reaches its capacity and needs to be evicted, the eviction score of the interpolated tile data is determined based on the access timestamp, access frequency calculator, and priority weight of the interpolated tile data. ; in, To eliminate points, This is the time since the last visit. This refers to the frequency of access. The higher the value, the lower the cache value of the tile, and the more it should be evicted. Tiles with the highest overall evicting score are prioritized for eviction.

[0084] Understandably, a GPU cache eviction strategy that combines access frequency and viewpoint distance improves cache hit rate while increasing GPU memory utilization, ensuring efficient use of limited GPU memory resources and prioritizing the caching of tile data that contributes the most to the current rendering quality.

[0085] Figure 4 This is a flowchart of a three-dimensional raster data interpolation rendering method according to an embodiment of the present invention. Based on the above embodiments, this embodiment further optimizes the step of "determining the raster data to be rendered from the interpolated tile data according to the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid," providing an optional solution. For example... Figure 4 As shown, the method includes: S310. Perform multi-level downsampling aggregation on multiple three-dimensional grid tiles to generate a tile pyramid.

[0086] S320. Determine the target tile features of each three-dimensional grid tile in the tile pyramid.

[0087] The target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map. S330. Determine the model based on the interpolation strategy. Based on the characteristics of the target tile, determine the target interpolation strategy for the three-dimensional raster tile.

[0088] S340: In parallel, based on the target interpolation strategy of the three-dimensional raster tiles, the three-dimensional raster tiles in the tile pyramid are interpolated to obtain interpolated tile data, and the interpolated tile data is stored in the GPU memory buffer.

[0089] S350. Determine the raster data to be rendered from the interpolated tile data based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid.

[0090] S360 performs volume rendering on the raster data to be rendered, generating a rendered image.

[0091] An alternative approach involves determining the raster data to be rendered from interpolated tile data based on the viewpoint parameters of the current viewpoint and the 3D raster tiles in the tile pyramid. This includes: obtaining the viewpoint parameters of the current viewpoint; wherein the viewpoint parameters include camera position, viewing direction vector, near and far clipping plane distance of the view frustum, and field of view angle; performing view frustum clipping on the 3D raster tiles in the tile pyramid to obtain tiles within the view frustum; calculating the spatial distance between the tiles within the view frustum and the current viewpoint, as well as the projected area of ​​the tiles within the view frustum on the current screen; determining the target tile level based on the spatial distance and projected area; and determining the raster data to be rendered based on the target tile level and the interpolated tile data.

[0092] Specifically, the camera position of the current viewpoint is first obtained. Line of sight vector The parameters include the near and far clipping plane distances of the view frustum, and the field of view (FOV). Then, each 3D grid tile in the tile pyramid is clipped using the view frustum, removing tiles completely outside the view frustum to obtain the tiles within the view frustum. For each tile within the view frustum, the Euclidean distance between the tile's center point and the current viewpoint is calculated. And the projected area of ​​the tile on the current screen. The projected area is calculated by first estimating the radius of the bounding sphere of the tile in three-dimensional space. Then, calculate the projected area based on perspective projection relationships. : ; in, This represents the total area of ​​the screen pixels.

[0093] Constructing a LOD selection function based on distance and projected area: ; in, This represents a limiting function that restricts the result to 0 and... between; This represents the floor function. The base distance threshold (preferably 100 meters) is used. The area sensitivity coefficient is preferably 0.5. The area threshold is (preferably 0.1% of the total screen area). The largest level of the pyramid, Adjustment value corresponding to the rendering sampling precision level ( ). The larger The smaller the value, the more detailed the tile data will be.

[0094] Based on the LOD selection function, the target tile level is determined; based on the target tile level, the corresponding tile data is found from the interpolated tile data to determine the raster data to be rendered.

[0095] Understandably, incorporating rendering sampling precision levels into the LOD selection function automatically increases resolution and sampling density for areas with high data complexity, thus allocating hardware resources to high-value data.

[0096] For example, determining the raster data to be rendered based on the target tile level and the interpolated tile data includes: firstly searching for the interpolated tile data corresponding to the target tile level in the GPU memory buffer; if found, using the interpolated tile data as the raster data to be rendered; if not found, interpolating the three-dimensional raster tiles corresponding to the target tile level, and using the interpolated three-dimensional raster tiles as the raster data to be rendered.

[0097] Understandably, retrieving raster data to be rendered from the GPU memory buffer can improve rendering speed.

[0098] The technical solution of this invention generates a tile pyramid by performing multi-level downsampling aggregation on multiple three-dimensional raster tiles; determines the target tile features of each three-dimensional raster tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; determines a model based on an interpolation strategy, and determines the target interpolation strategy for the three-dimensional raster tiles according to the target tile features; performs interpolation processing on the three-dimensional raster tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional raster tiles to obtain interpolated tile data, and stores the interpolated tile data in the GPU memory buffer; determines the raster data to be rendered from the interpolated tile data according to the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid; and performs volume rendering on the raster data to be rendered to generate a rendered image. The above technical solution determines the model through interpolation strategy and outputs a three-in-one strategy parameter set that integrates interpolation algorithm selection, hyperparameter configuration, and rendering sampling accuracy. This avoids the one-size-fits-all algorithm selection strategy of traditional solutions. At the same time, it migrates the interpolation calculation to the GPU and stores the interpolation results directly in the GPU memory buffer for the rendering pipeline to call, realizing the full GPU-based interpolation-rendering process.

[0099] Figure 5 This is a schematic diagram of a three-dimensional raster data interpolation and rendering device according to an embodiment of the present invention. This embodiment is applicable to the visualization of massive three-dimensional spatial field data in fields such as digital twins, geological exploration, meteorological simulation, and marine environmental monitoring. The three-dimensional raster data interpolation and rendering device can be implemented in hardware and / or software, and can be configured in an electronic device carrying the three-dimensional raster data interpolation and rendering function, such as a server. Figure 5 As shown, the device includes: The tile pyramid determination module 410 is used to perform multi-level downsampling aggregation on multiple three-dimensional grid tiles to generate a tile pyramid; The tile feature determination module 420 is used to determine the target tile features of each three-dimensional grid tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; The interpolation strategy determination module 430 is used to determine the model based on the interpolation strategy and to determine the target interpolation strategy of the three-dimensional raster tile according to the target tile features. The interpolation processing module 440 is used to perform interpolation processing on the three-dimensional grid tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional grid tiles, to obtain interpolated tile data, and to store the interpolated tile data in the GPU memory buffer. The module 450 for determining the data to be rendered is used to determine the raster data to be rendered from the interpolated tile data based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid. The rendering module 460 is used to perform volume rendering on the raster data to be rendered and generate a rendered image.

[0100] The technical solution of this invention generates a tile pyramid by performing multi-level downsampling aggregation on multiple three-dimensional raster tiles; determines the target tile features of each three-dimensional raster tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; determines a model based on an interpolation strategy, and determines the target interpolation strategy for the three-dimensional raster tiles according to the target tile features; performs interpolation processing on the three-dimensional raster tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional raster tiles to obtain interpolated tile data, and stores the interpolated tile data in the GPU memory buffer; determines the raster data to be rendered from the interpolated tile data according to the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid; and performs volume rendering on the raster data to be rendered to generate a rendered image. The above technical solution determines the model through interpolation strategy and outputs a three-in-one strategy parameter set that integrates interpolation algorithm selection, hyperparameter configuration, and rendering sampling accuracy. This avoids the one-size-fits-all algorithm selection strategy of traditional solutions. At the same time, it migrates the interpolation calculation to the GPU and stores the interpolation results directly in the GPU memory buffer for the rendering pipeline to call, realizing the full GPU-based interpolation-rendering process.

[0101] Optionally, the interpolation strategy determines the model by including at least three consecutive 3D convolutional modules, a global average pooling layer, and at least two fully connected layers; the 3D convolutional module includes a 3D convolutional layer, a batch normalization layer, and a ReLU activation function layer.

[0102] Optionally, the device also includes a model training module for: The sample tile features of the sample 3D tile are processed by convolution using a 3D convolution module to obtain 3D features; The dimensionality of the three-dimensional features is reduced by using a global average pooling layer to obtain the dimensionality-reduced features; The dimensionality reduction features are predicted using a fully connected layer, resulting in a prediction interpolation strategy and prediction interpolation error. Determine the policy loss based on the prediction interpolation policy and the interpolation policy label; The error loss is determined based on the predicted interpolation error and the actual interpolation error; The training loss is obtained by weighted summation of the policy loss and error loss, and the interpolation policy is determined by using the training loss for iterative training of the model.

[0103] Optionally, the tile feature determination module 420 is used for: The tile density is determined by the quotient of the number of effective voxels in the three-dimensional grid tile and the total number of voxels in the three-dimensional grid tile. The sparsity of the three-dimensional raster tiles is determined based on the tile density, and the sparsity feature map is determined based on the data sparsity. The gradient feature map is determined based on the gradient magnitude of voxels in the three-dimensional raster tile. Based on the attribute differences between 3D raster tiles and their neighboring tiles, a spatial association feature map is determined.

[0104] Optionally, the tile pyramid determining module 410 is used for: Use the original resolution-level 3D raster tiles as the bottom layer of the pyramid; Adjacent tiles in the current level are aggregated according to a preset spatial proximity merging rule. During aggregation, a weighted average method is used to merge the effective voxels of multiple 3D raster tiles to generate low-resolution level tile data until the preset minimum resolution threshold is met, thus obtaining the tile pyramid.

[0105] Optionally, the interpolation processing module 440 is used for: If the target interpolation strategy is Kriging interpolation, each thread block is responsible for the first number of voxel regions. The threads within the thread block cooperate to search for the second number of sample points closest to the voxel to be interpolated in the 3D raster tile through shared memory, and use the LU decomposition method to solve the Kriging equation system to obtain the interpolated 3D raster tile. If the target interpolation strategy is inverse distance weighted interpolation, each thread independently searches for the third number of sample points closest to the voxel to be interpolated in the 3D raster tile and performs calculations to obtain the interpolated 3D raster tile. If the target interpolation strategy is fast linear interpolation, each thread directly uses the attribute values ​​of effective voxels in the neighborhood to perform an arithmetic average to obtain the interpolated tile data.

[0106] Optionally, the data to be rendered determination module 450 is used for: Obtain the viewpoint parameters of the current viewpoint; among which, the viewpoint parameters include camera position, line of sight direction vector, near and far clipping plane distance of the view frustum, and field of view angle; By performing a view frustum clipping on the three-dimensional grid tiles in the tile pyramid, the tiles within the view frustum are obtained; Calculate the spatial distance between the tile within the view frustum and the current viewpoint, as well as the projected area of ​​the tile within the view frustum on the current screen; Determine the target tile level based on spatial distance and projected area; The raster data to be rendered is determined based on the target tile level and the interpolated tile data.

[0107] Optionally, the module 450 for determining the data to be rendered is specifically used for: First, retrieve the interpolated tile data corresponding to the target tile level from the GPU memory buffer; If found, the interpolated tile data will be used as the raster data to be rendered; If not found, interpolate the 3D raster tiles corresponding to the target tile level, and use the interpolated 3D raster tiles as the raster data to be rendered.

[0108] Optionally, the device also includes an interpolation update module for: If a local update of a 3D raster tile is detected, the target interpolation strategy for the 3D raster tile is redefined based on the interpolation strategy determination model, and the 3D raster tile is interpolated and updated according to the new target interpolation strategy.

[0109] Optionally, the device also includes an interpolation tile data update module for: The priority weights of the interpolated tile data are determined based on the spatial distance between the tile center point and the current viewpoint, and the base distance threshold. When the GPU memory buffer reaches its capacity limit, the elimination score of the interpolated tile data is determined based on the access timestamp, access frequency calculator, and priority weight corresponding to the interpolated tile data. The interpolated tile data in the GPU memory cache is updated based on the elimination score.

[0110] Optionally, the device also includes a three-dimensional grid tile determination module for: Before generating the tile pyramid, the original 3D raster data is spatially normalized to obtain multiple 3D raster tiles by performing multi-level downsampling aggregation on multiple 3D raster tiles, and a spatial index is established for each 3D raster tile.

[0111] The three-dimensional raster data interpolation and rendering apparatus provided in the embodiments of the present invention can execute the three-dimensional raster data interpolation and rendering method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0112] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0113] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the three-dimensional raster data interpolation rendering method of this invention. Figure 6A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0114] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0115] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0116] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as three-dimensional raster data interpolation rendering methods.

[0117] In some embodiments, the three-dimensional raster data interpolation rendering method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the three-dimensional raster data interpolation rendering method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the three-dimensional raster data interpolation rendering method by any other suitable means (e.g., by means of firmware).

[0118] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0120] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0122] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0123] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0124] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0125] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A three-dimensional raster data interpolation rendering method, characterized in that, include: Multi-level downsampling and aggregation of multiple 3D raster tiles are performed to generate a tile pyramid; Determine the target tile features of each three-dimensional grid tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; The model is determined based on the interpolation strategy, and the target interpolation strategy of the three-dimensional grid tile is determined according to the target tile features. In parallel, the three-dimensional grid tiles in the tile pyramid are interpolated according to the target interpolation strategy of the three-dimensional grid tiles to obtain interpolated tile data, and the interpolated tile data is stored in the GPU memory buffer. Based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid, determine the raster data to be rendered from the interpolated tile data; The raster data to be rendered is volumetric rendered to generate a rendered image.

2. The method according to claim 1, characterized in that, The interpolation strategy determines the model, which includes at least three consecutive 3D convolutional modules, a global average pooling layer, and at least two fully connected layers; the 3D convolutional module includes a 3D convolutional layer, a batch normalization layer, and a ReLU activation function layer.

3. The method according to claim 2, characterized in that, The interpolation strategy determination model is trained in the following manner: The 3D convolution module performs convolution processing on the sample tile features of the sample 3D tiles to obtain 3D features; The dimensionality of the three-dimensional features is reduced by the global average pooling layer to obtain dimensionality-reduced features; The dimensionality reduction features are predicted through the fully connected layer to obtain the prediction interpolation strategy and prediction interpolation error; Based on the prediction interpolation strategy and the interpolation strategy label, determine the strategy loss; The error loss is determined based on the predicted interpolation error and the actual interpolation error; The training loss is obtained by weighted summation of the policy loss and the error loss, and the interpolation policy determination model is iteratively trained using the training loss.

4. The method according to claim 1, characterized in that, Determining the target tile features of each three-dimensional grid tile in the tile pyramid includes: The tile density is determined by the quotient of the number of effective voxels in the three-dimensional grid tile and the total number of voxels in the three-dimensional grid tile. The data sparsity of the three-dimensional grid tiles is determined based on the tile density, and a sparsity feature map is determined based on the data sparsity. The gradient feature map is determined based on the gradient magnitude of the voxels in the three-dimensional raster tiles; Based on the attribute differences between the three-dimensional grid tile and its neighboring tiles, a spatial association feature map is determined.

5. The method according to claim 1, characterized in that, Multi-level downsampling and aggregation of multiple 3D raster tiles generates a tile pyramid, including: Use the original resolution-level 3D raster tiles as the bottom layer of the pyramid; Adjacent tiles in the current level are aggregated according to a preset spatial proximity merging rule. During aggregation, a weighted average method is used to merge the effective voxels of multiple 3D raster tiles to generate low-resolution level tile data until the preset minimum resolution threshold is met, thus obtaining the tile pyramid.

6. The method according to claim 1, characterized in that, In parallel, based on the target interpolation strategy of the three-dimensional raster tiles, the three-dimensional raster tiles in the tile pyramid are interpolated to obtain interpolated tile data, including: If the target interpolation strategy is Kriging interpolation, each thread block is responsible for the first number of voxel regions. The threads within the thread block cooperate to search for the second number of sample points closest to the voxel to be interpolated in the three-dimensional raster tile through shared memory, and use the LU decomposition method to solve the Kriging equation system to obtain the interpolated three-dimensional raster tile. If the target interpolation strategy is inverse distance weighted interpolation, each thread independently searches for the third number of sample points closest to the voxel to be interpolated in the 3D raster tile and performs calculations to obtain the interpolated 3D raster tile. If the target interpolation strategy is fast linear interpolation, each thread directly uses the attribute values ​​of effective voxels in the neighborhood to perform an arithmetic average to obtain the interpolated tile data.

7. The method according to claim 1, characterized in that, Based on the viewpoint parameters of the current viewpoint and the 3D raster tiles in the tile pyramid, the raster data to be rendered is determined from the interpolated tile data, including: Obtain the viewpoint parameters of the current viewpoint; wherein, the viewpoint parameters include camera position, line-of-sight vector, near and far clipping plane distance of the view frustum, and field of view angle; The three-dimensional grid tiles in the tile pyramid are subjected to view frustum clipping to obtain the tiles within the view frustum; Calculate the spatial distance between the tile within the view frustum and the current viewpoint, as well as the projected area of ​​the tile within the view frustum on the current screen; The target tile level is determined based on the spatial distance and the projected area; The raster data to be rendered is determined based on the target tile level and the interpolated tile data.

8. The method according to claim 7, characterized in that, Based on the target tile level and the interpolated tile data, determine the raster data to be rendered, including: First, retrieve the interpolated tile data corresponding to the target tile level from the GPU memory buffer; If found, the interpolated tile data will be used as the raster data to be rendered; If not found, the three-dimensional raster tiles corresponding to the target tile level are interpolated, and the interpolated three-dimensional raster tiles are used as raster data to be rendered.

9. The method according to claim 1, characterized in that, Also includes: If a local update of the 3D grid tile is detected, the target interpolation strategy of the 3D grid tile is re-determined based on the interpolation strategy determination model, and the 3D grid tile is interpolated and updated according to the new target interpolation strategy.

10. The method according to claim 1, characterized in that, Also includes: Based on the spatial distance between the tile center point of the interpolated tile data and the current viewpoint, and the base distance threshold, the priority weight corresponding to the interpolated tile data is determined; When the capacity of the GPU memory buffer reaches its limit, the elimination score of the interpolated tile data is determined based on the access timestamp, access frequency calculator, and priority weight corresponding to the interpolated tile data. The interpolated tile data in the GPU memory cache is updated based on the elimination score.

11. The method according to claim 1, characterized in that, Before generating the tile pyramid, the process includes multi-level downsampling aggregation of multiple 3D raster tiles, followed by: The original 3D raster data is spatially normalized to obtain multiple 3D raster tiles, and a spatial index is created for each 3D raster tile.

12. A three-dimensional raster data interpolation and rendering device, characterized in that, include: The tile pyramid determination module is used to perform multi-level downsampling aggregation on multiple 3D grid tiles to generate a tile pyramid; The tile feature determination module is used to determine the target tile features of each three-dimensional grid tile in the tile pyramid; wherein, the target tile features include a sparsity feature map, a spatial gradient feature map, and a spatial correlation feature map; An interpolation strategy determination module is used to determine a model based on an interpolation strategy, and to determine the target interpolation strategy for the three-dimensional raster tile according to the target tile features. An interpolation processing module is used to perform interpolation processing on the three-dimensional grid tiles in the tile pyramid in parallel according to the target interpolation strategy of the three-dimensional grid tiles, to obtain interpolated tile data, and to store the interpolated tile data in the GPU memory buffer. The module for determining the data to be rendered is used to determine the raster data to be rendered from the interpolated tile data based on the viewpoint parameters of the current viewpoint and the three-dimensional raster tiles in the tile pyramid. The rendering module is used to perform volume rendering on the raster data to be rendered and generate a rendered image.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the three-dimensional raster data interpolation rendering method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the three-dimensional raster data interpolation rendering method according to any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the three-dimensional raster data interpolation rendering method according to any one of claims 1-11.

Images