Large-scale uniform grid parallel reduction method based on error adaptive representation

Through the large-scale uniform grid parallel reduction method based on error adaptive representation, the octree algorithm is used to downsample large-scale data, which solves the problem of data reduction time and reconstruction in the existing technology, and realizes efficient data Reduction and visual feature preservation.

CN118941741APending Publication Date: 2024-11-12TIANJIN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411070745.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

When reducing large-scale data, the prior art requires training of neural networks, which takes a long time and needs to be decompressed and reconstructed during data visualization, affecting efficiency.

Method used

Using a large-scale uniform grid parallel reduction method based on error adaptive representation, the octree algorithm is used to uniformly divide and downsample the three-dimensional cube data, and judge whether the downsampling is qualified based on the error tolerance value until the minimum dimension requirement is met. Finally, data visualization is achieved without reconstructing the original data scale.

Benefits of technology

It realizes efficient reduction of large-scale data, shortens time, retains the visual characteristics of data, and improves efficiency in parallel processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941741A_ABST
    Figure CN118941741A_ABST
Patent Text Reader

Abstract

The invention discloses a large-scale uniform grid parallel reduction method based on error adaptive representation, which comprises the following steps of: S1, uniformly dividing original large-scale data of a three-dimensional cube into eight blocks by using an octree algorithm, S2, respectively carrying out downsampling on the eight data blocks until a downsampling ending condition is met, and S3, carrying out downsampling on the eight data blocks until the downsampling ending condition is met; s3, restoring the down-sampling data reserved by each data block to the position of the down-sampling data in the original data, so that the reduced data is visualized; according to the method, the octree algorithm is used for directly reducing large-scale data, down-sampling is carried out on an original data feature region for a few times according to error tolerance, down-sampling is carried out on a non-feature region for a plurality of times, training of a neural network and reconstruction of the scale of the original data are not needed, and the method is easy to implement. According to the method, important features of data visualization can be reserved while the purpose of data reduction is achieved, the data division process and the data reduction process are carried out in parallel, and the reduction time is further shortened.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of uniform grid reduction of large-scale data volumes, and in particular to a large-scale uniform grid parallel reduction method based on error adaptive representation. Background Art

[0002] As computer performance continues to improve, the scale of data is expanding. The huge scale of data has brought tremendous pressure to data storage, transmission and processing, and has also brought great challenges to data visualization. Therefore, how to effectively simplify large-scale data and retain the visualization characteristics of the data has become a research focus.

[0003] Among the existing technical solutions for data reduction, most studies use deep learning methods to achieve simplification of large-scale data. The neurcomp method proposed by Y. Lu et al. uses implicit neural representation to compress volume scalar fields; R. Yang proposed the TINC method, which specifically uses an implicit neural compression method with an octree structure to compress data; J. Han et al. proposed a data reduction method based on implicit neural representation (KD-INR) of knowledge distillation for compressing large-scale time-varying data.

[0004] In all the above methods, it is necessary to train the corresponding network model to achieve the purpose of data simplification. When training the network, it usually takes a lot of time. After using deep learning methods to simplify the data, it is generally necessary to decompress the network model to reconstruct large-scale data when visualizing the data, and the data still needs to be restored to the original scale when visualizing it. Summary of the invention

[0005] The purpose of the present invention is to provide a large-scale uniform grid parallel reduction method based on error adaptive representation to solve the above technical problems.

[0006] To this end, the technical solution of the present invention is as follows:

[0007] A large-scale uniform grid parallel reduction method based on error adaptive representation, the steps are:

[0008] S1. Use the octree algorithm to evenly divide the original large-scale data of the three-dimensional cube into eight blocks;

[0009] S2, downsampling the eight data blocks obtained in step S1 respectively until the downsampling end conditions are met, including the minimum dimension of the data block in the three spatial dimensions and the error tolerance value of the interpolated data after downsampling;

[0010] S3. The downsampled data retained in each data block is restored to its position in the original data, so that the simplified data can be visualized while retaining the complex features of the original data as much as possible.

[0011] Further, in step S1, the original large-scale data is data of all original voxels constituting the three-dimensional cube.

[0012] Further, in step S1, all original voxel data of the three-dimensional cube are evenly divided into eight blocks in a visual seamless manner, and the specific steps are as follows:

[0013] 1) Determine the number of data points N of the original voxel data of the three-dimensional cube in the three spatial dimensions of X, Y, and Z x , N y and N z ;

[0014] 2) According to the parity of the number of data points in each dimension, determine the ordinal number n of the boundary data points when dividing the data block x , n y and n z :in,

[0015] When the number of data points in the i-th dimension is an odd number, the ordinal number of the boundary data point is n i =(N i +1) / 2,

[0016] When the number of data points in the i-th dimension is even, the ordinal number of the boundary data point n i =N i / 2+1, in the above two formulas, i is x, y or z;

[0017] 3) According to the coordinate list of all original voxel data in each dimension, the coordinates of the boundary data points in three spatial dimensions are determined, and the starting point coordinates and the ending point coordinates of each data block are determined by repeatedly using the coordinates of the boundary data points.

[0018] Furthermore, the specific implementation steps of step S2 are:

[0019] S201, setting the minimum dimension of the downsampled data block in three spatial dimensions, and the error tolerance value of the interpolated data after downsampling;

[0020] S202, downsampling the data points of each spatial dimension of the data block at alternate points, and supplementing the missing boundary data in the downsampling by filling in data slices;

[0021] S203, using an interpolation algorithm to interpolate the downsampled data back to the point size of the data point before downsampling;

[0022] S204, calculating the maximum absolute value of the difference between the interpolated data and the corresponding original data before downsampling;

[0023] S205, comparing the maximum value of the absolute value of the difference with the error tolerance value: if the maximum value of the absolute value of the difference is less than the error tolerance value, it is determined that the data block downsampling is qualified; if the maximum value of the absolute value of the difference is greater than or equal to the error tolerance value, it is determined that the data block downsampling is unqualified;

[0024] S206, for the data block judged as qualified, retain its downsampled data and proceed to step S207; for the data block judged as unqualified, ignore the previously downsampled data, use the method of step S1 to continue to evenly divide the data block into eight blocks, and then repeat the above steps S202 to S205 for each newly divided data block, and repeat the cycle until any of the following three conditions is met: 1) the downsampled data block cannot be further downsampled, 2) the downsampled data block meets the minimum dimension setting requirement, 3) the data block has not been downsampled but meets the minimum dimension setting requirement after multiple rounds of division;

[0025] S207. After each downsampling, compare the minimum dimension of each dimension of the current data block with the set minimum dimension: if the minimum dimension of each dimension of the data block is larger than the set minimum dimension, continue to recursively downsample the data in the manner of step S202 until the minimum dimension setting requirement is met; if the minimum dimension of each dimension of the data block is less than or equal to the minimum dimension, the downsampling is completed and the process proceeds to step S3.

[0026] Furthermore, in step S202, the data slice filling operation method is:

[0027] Step 1: According to the number of data points in each spatial dimension, determine the spatial dimension with an even number of data points.

[0028] Step 2: Get the data slices that need to be filled according to the number of spatial dimensions with an even number of data points:

[0029] Case 1: When the number of data points is even and there is 1 spatial dimension, first determine the coordinate value of the last data point in the dimension from the original data of the spatial dimension, then obtain the data points corresponding to the other two spatial dimensions based on the coordinate value of the last data point in the dimension, and downsample the data of the other two spatial dimensions in an alternate sampling manner, thereby forming the data slice that needs to be filled in the spatial dimension;

[0030] Case 2: When the number of data points is even and the number of spatial dimensions is 2, the method in Case 1 is first used to obtain the data slices to be filled in the two spatial dimensions respectively, and then the data points in the third spatial dimension are obtained based on the coordinate values ​​of the last data points in the two spatial dimensions in the two dimensions, and downsampled in an alternate sampling manner to form the data slice to be filled between the two spatial dimensions;

[0031] Case 3: When the number of data points is an even number and the number of spatial dimensions is 3, the method in case 1 is first used to obtain the data slices that need to be filled in the three spatial dimensions, and then the method in case 2 is used to obtain the data slices that need to be filled between every two spatial dimensions;

[0032] Step 3: Supplement the data slice to be filled obtained in step 2 to the downsampled data obtained in step 1, so as to obtain the downsampled data after the boundary missing is supplemented.

[0033] Furthermore, in step S203, the interpolation algorithm adopts a linear interpolation method.

[0034] Furthermore, the specific implementation steps of step S3 are:

[0035] S301, taking each data block obtained by the division in step S2 as a leaf node, traversing all leaf nodes to create a RectilinearGrid in VTK for each leaf node according to the coordinate list of the three spatial dimensions; wherein, in this embodiment, the reason for selecting this type of grid is that it can store points with different intervals;

[0036] S302. Use MultiBlockDataSet in VTK to combine all RectilinearGrids in step S301 and generate a .vtm file.

[0037] S303, converting the .vtm file generated in step S302 into a .vtk file, and then opening the .vtk file in Paraview software to realize visualization of the simplified data.

[0038] Compared with the existing technology, this large-scale uniform grid parallel simplification method based on error adaptive representation only uses the octree algorithm to directly simplify large-scale data, downsamples the feature area of ​​the original data a few times according to the error tolerance, and downsamples the non-feature area multiple times. There is no need to train a neural network, and the final visualization is only to directly visualize the simplified data, without the need to reconstruct the simplified data back to the size of the original data. While achieving the purpose of data simplification, it also retains the important features of data visualization; and the data partitioning and data simplification processes in the algorithm are carried out in parallel, further shortening the simplification time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the large-scale uniform grid parallel reduction method based on error adaptive representation of the present invention;

[0040] Figure 2 This is a schematic diagram of data partitioning in step S103 of the large-scale uniform grid parallel reduction method based on error adaptive representation of the present invention;

[0041] Figure 3 It is a schematic diagram of downsampling for even-numbered points in step S202 of the large-scale uniform grid parallel reduction method based on error adaptive representation of the present invention;

[0042] Figure 4 A visual comparison diagram of the original data of data set I in an embodiment of the present invention and the data after data simplification using the method of the present invention;

[0043] Figure 5 A grid visualization comparison diagram of the original data of the data set I in an embodiment of the present invention and the data after data simplification using the method of the present invention;

[0044] Figure 6 A visual comparison diagram of the original data of data set II in an embodiment of the present invention and the data after data simplification using the method of the present invention;

[0045] Figure 7 A grid visualization comparison diagram of the original data of data set II in an embodiment of the present invention and the data after data simplification using the method of the present invention;

[0046] Figure 8 A visual comparison diagram of the original data of data set III in an embodiment of the present invention and the data after data simplification using the method of the present invention;

[0047] Fig. 9 This is a grid visualization comparison diagram of the original data of data set III in an embodiment of the present invention and the data after data simplification using the method of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention in any way.

[0049] See also Figure 1 The specific implementation steps of the large-scale uniform grid parallel reduction method based on error adaptive representation are as follows:

[0050] S1. Use the octree algorithm to evenly divide the original large-scale data of the three-dimensional cube into eight blocks;

[0051] The specific implementation steps of step S1 are:

[0052] S101, obtaining data of all original voxels constituting a three-dimensional cube, wherein each voxel data is composed of voxel coordinates and voxel values; wherein the voxel value corresponds to the voxel color after data visualization;

[0053] S102, setting important attributes of the octree node, including: downsampling level, starting coordinates of the node, flags of whether the data points in each dimension are even points, coordinate lists of each dimension, coordinate lists of the last piece of data in each dimension, and flags of whether it has been downsampled; specifically,

[0054] The downsampling level is used to record the number of downsampling times of the data block, so that the starting position of the block before downsampling and the position of each data point in the original data before downsampling can be obtained according to the downsampling level later; in this embodiment, the initial downsampling level of the data block is 0, and each time it undergoes downsampling, the value of the downsampling level is increased by 1;

[0055] The starting coordinates of the node are used to record the starting position of a single data block in the original data before downsampling, so as to prepare for visualization of the reduced data; in this embodiment, the starting coordinate node of each data block is set to the lower left corner vertex of the data block;

[0056] The flag of whether the data points in each dimension are even is used to mark whether the number of data points in each data block in the three spatial dimensions of X, Y and Z is odd or even. This is set because the two results correspond to different subsequent reduction processing steps. In this embodiment, if the number of data points in a certain dimension of the data block is odd, the flag is recorded as 0. Conversely, if the number of data points in a certain dimension of the data block is even, the flag is recorded as 1.

[0057] The coordinate list of each dimension is used to record the coordinates of all data points in the specified dimension starting from the starting node of each data block in the three spatial dimensions of X, Y and Z;

[0058] The coordinate list of the last piece of data in each dimension is used to supplement the sheet data of any dimension of the three spatial dimensions X, Y and Z during the simplification process to avoid the loss of boundary data during the simplification process; wherein, the sheet data acquisition method is: determine the coordinate value of the last point in any dimension, find the relevant boundary point, and form the sheet data;

[0059] The flag of whether the downsampling has been performed is used to record whether any data block has been downsampled. In this embodiment, the flag of whether the downsampling has been performed is 1, and the flag of whether the downsampling has been performed is 0.

[0060] S103, dividing all original voxel data of the three-dimensional cube into eight blocks evenly in a visualized seamless manner; wherein the visualized seamless manner is:

[0061] 1) Determine the number of data points N of the original voxel data of the three-dimensional cube in the three spatial dimensions of X, Y, and Z x , N y and N z ;

[0062] 2) According to the parity of the number of data points in each dimension, determine the ordinal number n of the boundary data points when dividing the data block x , n y and n z :in,

[0063] When the number of data points in the i-th dimension is an odd number, the ordinal number of the boundary data point n i =(N i +1) / 2,

[0064] When the number of data points in the i-th dimension is even, the ordinal number of the boundary data point n i =N i / 2+1, in the above two formulas, i is x, y or z;

[0065] 3) According to the coordinate list of all the original voxel data in each dimension, the coordinates of the boundary data points in the three spatial dimensions are determined, and the starting point coordinates and the ending point coordinates of each data block are determined by repeatedly using the boundary data point coordinates; that is, in any dimension, starting from the starting node, its boundary data point serves as both the ending point coordinate of the previous data block division and the starting point coordinate of the next data block division.

[0066] In order to facilitate the explanation of the above data division method, Figure 2 The simple one-dimensional data shown is used as an example to explain; Figure 2The left side shows the case where there are five data points in a single dimension, that is, the number of data points in any dimension is an odd number. If the first two data points are grouped together and the last three data points are grouped together, then the subsequent image will have obvious edge defects. Therefore, the correct division method is to group the first three points together and the last three points together, that is, the third point is reused. Figure 2 The right side shows the case where there are eight data points in a single dimension, that is, the number of data points in any dimension is an even number. If the first four data points are in one group and the last four data points are in one group, the subsequent visualized image will also have obvious edge defects. Therefore, the correct division method is to group the first five points into one group and the last four points into one group, that is, the fifth point is reused, or to group the first four points into one group and the last five points into one group, that is, the fourth point is reused. This embodiment adopts the former, that is, the principle of more in the front and less in the back when grouping; in addition, it should be noted that for the case of an even number of data points, the same division method should be maintained in the three spatial dimensions, that is, either the principle of more in the front and less in the back when grouping is adopted, or the principle of less in the front and more in the back when grouping is adopted, otherwise equal division cannot be achieved.

[0067] S2, downsampling the eight data blocks obtained in step S1 respectively until the downsampling end condition is met;

[0068] In step S2, the specific implementation steps of downsampling each data block are:

[0069] S201, setting the minimum dimension that the data block should meet in the three spatial dimensions of X, Y, and Z during downsampling, and the error tolerance value of the interpolated data after downsampling; wherein the error tolerance value is determined according to the target compression rate of the three-dimensional data volume;

[0070] S202, downsampling and data slice filling are performed on the data points of each spatial dimension of the data block; wherein,

[0071] The specific method of downsampling is: in each spatial dimension, all data points in the dimension are downsampled at intervals starting from the starting data point to obtain downsampled data; the above downsampling process reduces the data to half of the original along each spatial dimension;

[0072] The data slice filling operation is:

[0073] Step 1: According to the number of data points in each spatial dimension, determine the spatial dimension with an even number of data points.

[0074] Step 2: Get the data slices that need to be filled according to the number of spatial dimensions with an even number of data points:

[0075] Case 1: When the number of data points is even and there is 1 spatial dimension, first determine the coordinate value of the last data point in the dimension from the original data of the spatial dimension, then obtain the data points corresponding to the other two spatial dimensions based on the coordinate value of the last data point in the dimension, and downsample the data of the other two spatial dimensions in an alternate sampling manner, thereby forming the data slice that needs to be filled in the spatial dimension;

[0076] Case 2: When the number of data points is even and the number of spatial dimensions is 2, the method in Case 1 is first used to obtain the data slices to be filled in the two spatial dimensions respectively, and then the data points in the third spatial dimension are obtained based on the coordinate values ​​of the last data points in the two spatial dimensions in the two dimensions, and downsampled in an alternate sampling manner to form the data slice to be filled between the two spatial dimensions;

[0077] Case 3: When the number of data points is an even number and the number of spatial dimensions is 3, the method in case 1 is first used to obtain the data slices that need to be filled in the three spatial dimensions, and then the method in case 2 is used to obtain the data slices that need to be filled between every two spatial dimensions;

[0078] Step 3: Supplement the data slice to be filled obtained in step 2 to the downsampled data obtained in step 1, so as to obtain the downsampled data after the boundary missing is supplemented.

[0079] To illustrate the importance of data slice filling operations, see Figure 3 The one-dimensional data on the left side of the figure shows that when the number of data points is even, normal downsampling will lead to boundary loss, so it is necessary to change the downsampling method to supplement the boundary; the three-dimensional data downsampling of the present invention is more complicated; see Figure 3 In the figure on the right, the data after normal downsampling of a three-dimensional data is represented by a blue cube. If the data points in the x-axis and y-axis directions of the original data are both even numbers (i.e., case 2 above), then it is necessary to first supplement the boundary data in the x-axis direction, that is, take the last piece of data in the x-axis direction of the original data, and downsample the data of the y-axis and z-axis by one. The yellow quadrilateral in the figure represents the supplementary boundary data in the x-axis direction. This piece of data is spliced ​​with the data after normal downsampling (blue cube), and then the boundary data in the y-axis direction is supplemented. The last piece of data in the y-axis direction of the original data is taken, and the data of the x-axis and z-axis are downsampled by one. This piece of data corresponds to Figure 2 The green quadrilateral on the right side, it is worth noting that since the data in the x-axis direction is an even number of points and has been supplemented at the border in the x-axis direction, the last piece of data in the y-axis direction needs to be supplemented with a column of the original data's x-axis and the last column of the y-axis after the x-axis and z-axis data are downsampled normally, corresponding to Figure 3The red bar on the right; therefore, when the x-axis and y-axis data in the original three-dimensional data are even points, the final result of data simplification is as follows Figure 3 The right figure shows the result of splicing the blue cube after normal downsampling with the boundary supplement (yellow quadrilateral, green quadrilateral and red bar). Here is only one case as an example, and other cases are handled in this way. The process of simplifying each child node is also parallel, and the simplification of eight nodes is assigned to eight different threads respectively; based on this, the above data slice filling operation extracts the boundary points and splices them with the downsampled data, ensuring that the downsampled data still maintains the original boundary when it is restored to the position of the original data;

[0080] S203, using an interpolation algorithm to interpolate the downsampled data obtained in step S202 back to the point size of the data point before downsampling; in this embodiment, the interpolation algorithm uses a linear interpolation method;

[0081] S204, calculating the maximum absolute value of the difference between the interpolated data obtained in step S203 and the corresponding original data before downsampling;

[0082] S205, comparing the maximum absolute value of the difference in step S204 with the error tolerance value set in step S201:

[0083] If the maximum absolute value of the difference is less than the error tolerance value, the data block is judged to be qualified for downsampling;

[0084] If the maximum absolute value of the difference is greater than or equal to the error tolerance value, the data block is judged to be unqualified for downsampling;

[0085] S206, for the data block judged as qualified, retain its downsampled data and proceed to step S207; for the data block judged as unqualified, ignore the previously downsampled data, use the method of step S1 to continue to evenly divide the data block into eight blocks, and then repeat the above steps S202 to S205 for each newly divided data block, and repeat the cycle until any of the following three conditions is met: 1) the downsampled data block cannot be further downsampled, 2) the downsampled data block meets the minimum dimension setting requirement, 3) the data block has not been downsampled but meets the minimum dimension setting requirement after multiple rounds of division;

[0086] S207, after each downsampling, compare the minimum dimension of each dimension of the current data block with the size of the set minimum dimension:

[0087] If the minimum dimension of each dimension of the data block is greater than the set minimum dimension, the data is recursively downsampled in the manner of step S202, that is, based on the previously saved downsampled data, the data points are continuously downsampled in a manner of sampling every other point and the boundary data lost due to downsampling is supplemented to obtain new downsampled data, until the downsampled data of the data block meets the minimum dimension setting requirement in step S201;

[0088] If the minimum dimension of each dimension of the data block is less than or equal to the minimum dimension, the downsampling is completed and the process proceeds to step S3.

[0089] S3, restoring the downsampled data retained in each data block to its position in the original data, so that the visualization result of the simplified data retains the complex features of the original data as much as possible;

[0090] The specific implementation steps of step S3 are:

[0091] S301, taking each data block obtained by the division in step S2 as a leaf node, traversing all leaf nodes to create a RectilinearGrid in VTK for each leaf node according to the coordinate list of the three spatial dimensions; wherein, in this embodiment, the reason for selecting this type of grid is that it can store points with different intervals;

[0092] S302. Use MultiBlockDataSet in VTK to combine all RectilinearGrids in step S301 and generate a .vtm file.

[0093] S303, converting the .vtm file generated in step S302 into a .vtk file, and then opening the .vtk file in Paraview software to realize visualization of the simplified data.

[0094] In order to further prove that the method of the present application can maintain the excellent characteristic that the data visualization effect is consistent with the original data visualization effect after large-scale data is compressed by a large proportion, based on the method described in the above embodiment, data set I: Research Vessel Tangaroa (300×180×120) (see: https: / / cgl.ethz.ch / research / visualization / data.php), data set II: Half Cylinder Ensemble (640×240×80) (see: https: / / cgl.ethz.ch / research / visualization / data.php), and data set III: Rayleigh-Taylor Instability (the original data set is 1024×1024×1024, cropped to 800×800×800 (see: https: / / klacansky.com / open-scivi s-datasets / sorted-by-voxels.html) to perform compression and simplification at different ratios, and visualize the compressed and simplified data to compare the visualization effect with the original data. Among them, since both datasets I and II are time-varying data and both are vector field data, the data of the last time step are taken during the test and converted into scalar field data.

[0095] The simplification and visualization results of testing the method of the present application using the above three data sets are shown in Table 1 and Table 2 below.

[0096] Table 1: Compression of three datasets under different error tolerance values

[0097]

[0098] Table 1 above shows the compression conditions of three data sets under different error tolerance values. It can be seen from Table 1 that the method of the present invention achieves compression effects with different compression ratios by giving different error tolerance values ​​according to the simplification requirements.

[0099] Table 2: Changes in data reduction time before and after parallelization of the three data sets

[0100] Dataset Compression ratio Data reduction time before parallelization (s) Data reduction time after parallelization (s) 300×180×120 52.90:1 4.44 3.25 640×240×80 55.17:1 12.75 6.27 800×800×800 136.3:1 309.12 109.24

[0101] Table 2 above compares the changes in data reduction time before and after parallelization; it can be seen from Table 2 that when the method of the present invention is used, the larger the data size of the data set, the more significant the effect of parallelization in shortening the time.

[0102] like Figures 4 to 9 The figure shows the visual comparison results before and after the data of the above three data sets are simplified using the method of the present application.

[0103] like Figure 4 The figure shows the visualization results of the 300×180×120 dataset I before and after data simplification using the method of the present application. Figure 4 The left side shows the visualization result of the original data. Figure 4 The right side shows the visualization result of the simplified data with a compression ratio of 52.90:1; from the comparison of the two figures, it can be seen that the simplified data still retains the characteristics of the original data.

[0104] like Figure 5 The figure shows the grid visualization results of the 300×180×120 dataset I before and after data simplification using the method of the present application. Figure 5 The left side shows the visualization result of the original data. Figure 5 On the right is the visualization of the reduced data grid with a compression ratio of 52.90:1. From the comparison of the two figures, it can be seen that only a few downsamplings were performed in the feature areas of the original data during the simplification, among which, Figure 5 The grid of the simplified data on the right side is denser in the feature area of ​​the original data, while multiple downsampling is performed in the non-feature area of ​​the original data. Figure 5 The grid of the non-feature area of ​​the original data corresponding to the simplified data on the right is sparser. Therefore, the visualization result of the simplified data can still retain the characteristics of the original data.

[0105] like Figure 6 The diagram shows the visualization results of the 640×240×80 dataset II before and after data simplification using the method of this application. Figure 6 The left side shows the visualization result of the original data. Figure 6 The right side shows the visualization result of the simplified data with a compression ratio of 55.17:1; from the comparison of the two figures, it can be seen that the simplified data still retains the characteristics of the original data.

[0106] like Figure 7 The figure shows the grid visualization results of the 640×240×80 dataset II before and after data simplification using the method of this application. Figure 7 The left side shows the visualization result of the original data. Figure 7 On the right is the visualization of the reduced data grid with a compression ratio of 55.17:1. From the comparison of the two figures, it can be seen that only a few downsamplings were performed in the feature areas of the original data during the simplification. Figure 7The grid of the simplified data on the right side is denser in the feature area of ​​the original data, while multiple downsampling is performed in the non-feature area of ​​the original data. Figure 7 The grid of the non-feature area of ​​the original data corresponding to the simplified data on the right is sparser; therefore, the visualization result of the simplified data can still retain the characteristics of the original data.

[0107] like Figure 8 The figure shows the visualization result of the 800×800×800 dataset III before and after data simplification using the method of the present application. Figure 8 The left side shows the visualization result of the original data. Figure 8 The right side shows the visualization result of the simplified data with a compression ratio of 136.3:1; from the comparison of the two figures, it can be seen that the simplified data still retains the characteristics of the original data.

[0108] like Fig. 9 The figure shows the grid visualization result of the 800×800×800 dataset III before and after data simplification using the method of the present application. Fig. 9 The left side shows the visualization result of the original data. Fig. 9 On the right is the visualization of the reduced data grid with a compression ratio of 136.3:1. From the comparison of the two figures, it can be seen that only a few downsamplings were performed in the feature areas of the original data during the simplification, among which, Fig. 9 The grid of the simplified data on the right side is denser in the feature area of ​​the original data, while multiple downsampling is performed in the non-feature area of ​​the original data. Fig. 9 The grid of the non-feature area of ​​the original data corresponding to the simplified data on the right is sparser. Therefore, the visualization result of the simplified data can still retain the characteristics of the original data.

Claims

1. A large-scale uniform grid parallel reduction method based on error adaptive representation, characterized in that: The steps are: S1. Use the octree algorithm to evenly divide the original large-scale data of the three-dimensional cube into eight blocks; S2, downsampling the eight data blocks obtained in step S1 respectively until the downsampling end conditions are met, including the minimum dimension of the data block in the three spatial dimensions and the error tolerance value of the interpolated data after downsampling; S3. The downsampled data retained in each data block is restored to its position in the original data to visualize the simplified data.

2. The large-scale uniform grid parallel reduction method based on error adaptive representation according to claim 1 is characterized in that: In step S1 , the original large-scale data is the data of all original voxels constituting the three-dimensional cube.

3. The large-scale uniform grid parallel reduction method based on error adaptive representation according to claim 1 is characterized in that: In step S1, all original voxel data of the three-dimensional cube are evenly divided into eight blocks in a visual seamless manner, and the specific steps are as follows: 1) Determine the number of data points N of the original voxel data of the three-dimensional cube in the three spatial dimensions of X, Y, and Z x , N y and N z ; 2) According to the parity of the number of data points in each dimension, determine the ordinal number n of the boundary data points when dividing the data block x , n y and n z :in, When the number of data points in the i-th dimension is an odd number, the ordinal number of the boundary data point n i =(N i +1) / 2, When the number of data points in the i-th dimension is even, the ordinal number of the boundary data point n i =N i / 2+1, In the above two formulas, i is x, y or z; 3) According to the coordinate list of all original voxel data in each dimension, the coordinates of the boundary data points in three spatial dimensions are determined, and the starting point coordinates and the ending point coordinates of each data block are determined by repeatedly using the coordinates of the boundary data points.

4. The large-scale uniform grid parallel reduction method based on error adaptive representation according to claim 1, characterized in that: The specific implementation steps of step S2 are: S201, setting the minimum dimension of the downsampled data block in three spatial dimensions, and the error tolerance value of the interpolated data after downsampling; S202, downsampling the data points of each spatial dimension of the data block at alternate points, and supplementing the missing boundary data in the downsampling by filling in data slices; S203, using an interpolation algorithm to interpolate the downsampled data back to the point size of the data point before downsampling; S204, calculating the maximum absolute value of the difference between the interpolated data and the corresponding original data before downsampling; S205, comparing the maximum value of the absolute value of the difference with the error tolerance value: if the maximum value of the absolute value of the difference is less than the error tolerance value, it is determined that the data block downsampling is qualified; if the maximum value of the absolute value of the difference is greater than or equal to the error tolerance value, it is determined that the data block downsampling is unqualified; S206, for the data block judged as qualified, retain its downsampled data and proceed to step S207; for the data block judged as unqualified, ignore the previously downsampled data, use the method of step S1 to continue to evenly divide the data block into eight blocks, and then repeat the above steps S202 to S205 for each newly divided data block, and repeat the cycle until any of the following three conditions is met: 1) the downsampled data block cannot be further downsampled, 2) the downsampled data block meets the minimum dimension setting requirement, 3) the data block has not been downsampled but meets the minimum dimension setting requirement after multiple rounds of division; S207. After each downsampling, compare the minimum dimension of each dimension of the current data block with the set minimum dimension: if the minimum dimension of each dimension of the data block is larger than the set minimum dimension, continue to recursively downsample the data in the manner of step S202 until the minimum dimension setting requirement is met; if the minimum dimension of each dimension of the data block is less than or equal to the minimum dimension, the downsampling is completed and the process proceeds to step S3.

5. The large-scale uniform grid parallel reduction method based on error adaptive representation according to claim 4 is characterized in that: In step S202, the data slice filling operation method is: Step 1: According to the number of data points in each spatial dimension, determine the spatial dimension with an even number of data points. Step 2: Get the data slices that need to be filled according to the number of spatial dimensions with an even number of data points: Case 1: When the number of data points is even and there is 1 spatial dimension, first determine the coordinate value of the last data point in the dimension from the original data of the spatial dimension, then obtain the data points corresponding to the other two spatial dimensions based on the coordinate value of the last data point in the dimension, and downsample the data of the other two spatial dimensions in an alternate sampling manner, thereby forming the data slice that needs to be filled in the spatial dimension; Case 2: When the number of data points is an even number and the number of spatial dimensions is 2, the method in Case 1 is first used to obtain the data slices that need to be filled in the two spatial dimensions respectively, and then the data points in the third spatial dimension are obtained based on the coordinate values ​​of the last data points in the two spatial dimensions in the two spatial dimensions, and downsampled in an alternate sampling manner to form the data slice that needs to be filled between the two spatial dimensions; Case 3: When the number of data points is an even number and the number of spatial dimensions is 3, the method in case 1 is first used to obtain the data slices that need to be filled in the three spatial dimensions, and then the method in case 2 is used to obtain the data slices that need to be filled between every two spatial dimensions; Step 3: Supplement the data slice to be filled obtained in step 2 to the downsampled data obtained in step 1, so as to obtain the downsampled data after the boundary missing is supplemented.

6. The large-scale uniform grid parallel reduction method based on error adaptive representation according to claim 4, characterized in that: In step S203, the interpolation algorithm adopts a linear interpolation method.

7. The large-scale uniform grid parallel reduction method based on error adaptive representation according to claim 4 is characterized in that: The specific implementation steps of step S3 are: S301, taking each data block obtained by the division in step S2 as a leaf node, traversing all leaf nodes to create a RectilinearGrid in VTK for each leaf node according to the coordinate list of the three spatial dimensions; wherein, in this embodiment, the reason for selecting this type of grid is that it can store points with different intervals; S302. Use MultiBlockDataSet in VTK to combine all RectilinearGrids in step S301 and generate a .vtm file. S303, converting the .vtm file generated in step S302 into a .vtk file, and then opening the .vtk file in Paraview software to realize visualization of the simplified data.