Three-dimensional scene data processing method and device, and electronic device

Abstract

The application provides a three-dimensional scene data processing method, device and electronic equipment, and belongs to the technical field of three-dimensional space data processing. The method comprises the following steps: obtaining three-dimensional scene data corresponding to a three-dimensional scene, performing multi-dimensional feature extraction on the three-dimensional scene data to obtain comprehensive density field data; performing gradient solving and centroid calculation processing on the comprehensive density field data to obtain a density gradient vector and a density distribution centroid; determining a space offset parameter according to the density gradient vector and the density distribution centroid, wherein the space offset parameter is an offset amount used for adjusting the offset of a segmentation position from the space center; performing asymmetric segmentation processing on the space corresponding to the three-dimensional scene data based on the space offset parameter to obtain an asymmetric octree space index; and performing tile conversion processing on the three-dimensional scene data based on the asymmetric octree space index to obtain a three-dimensional tile data set. The above scheme can effectively improve the data rendering speed of the three-dimensional scene data.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional spatial data processing technology, specifically to three-dimensional scene data processing methods, devices, and electronic equipment. Background Technology

[0002] Current 3D scene data processing typically employs the traditional octree spatial partitioning technique to construct a spatial index. During processing, the traditional octree uses a uniform octet approach, dividing the 3D space corresponding to each parent node into eight identical sub-cube spaces without differentiation, and recursively partitioning downwards in this manner. After spatial partitioning, based on this uniformly distributed octree spatial index, the 3D scene data undergoes tile conversion processing to generate a 3D tile dataset for network transmission and rendering loading.

[0003] However, in practical applications, existing technologies fail to consider density differences within the space during partitioning. The uniform octet division results in individual child nodes in high-density regions having the same spatial volume as those in low-density regions. This leads to a dramatic increase in the data volume of a single tile in high-density regions, far exceeding that in low-density regions. This severe imbalance in data volume causes drastic fluctuations in network transmission load. Furthermore, the excessively long loading time of high-density tiles significantly slows down the overall progress, resulting in highly unstable rendering frame rates and low loading efficiency.

[0004] Therefore, there is an urgent need for an octree spatial segmentation method that can dynamically adjust the spatial segmentation position according to the density distribution inside the 3D scene, so that the spatial volume of the child node matches the density, in order to solve the technical problems of uneven tile data, drastic network transmission fluctuations, and low loading and rendering efficiency caused by the existing uniform octet method. Summary of the Invention

[0005] To address the aforementioned problems, this application provides a 3D scene data processing method to improve the data rendering speed of 3D scene data. The technical solution of this application is as follows: In a first aspect, the present invention provides a three-dimensional scene data processing method, including: Obtain the 3D scene data corresponding to the 3D scene, perform multi-dimensional feature extraction on the 3D scene data, and obtain comprehensive density field data; Gradient calculation and centroid calculation are performed on the comprehensive density field data to obtain the density gradient vector and the density distribution centroid. The spatial offset parameter is determined based on the density gradient vector and the centroid of the density distribution. The spatial offset parameter is used to adjust the offset of the segmentation position from the spatial center. Based on the spatial offset parameter, the space corresponding to the 3D scene data is asymmetrically segmented to obtain an asymmetric octree spatial index. The spatial volume of the child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node. A 3D scene data is transformed into a 3D tile dataset by using an asymmetric octree spatial index.

[0006] Combining the first aspect and the above implementation methods, in some possible implementation methods, multi-dimensional feature extraction is performed on the 3D scene data to obtain comprehensive density field data, including: The bounding box space corresponding to the 3D scene data is spatially divided to obtain a voxel mesh. Geometric density features, visual complexity density features, and semantic importance density features of 3D scene data are extracted based on voxel meshes; The geometric density features, visual complexity density features, and semantic importance density features are adaptively weighted and fused to obtain comprehensive density field data.

[0007] Combining the first aspect and the above implementation methods, some possible implementation methods extract geometric density features, visual complexity density features, and semantic importance density features of 3D scene data based on voxel meshes, including: Obtain the number of geometric features within the voxel mesh, and derive the geometric density feature based on the number of geometric features; Variance and edge detection processing is performed on the texture data within the voxel mesh to obtain visual complexity density features; Semantic annotation information of 3D scene data is obtained, and weights are assigned based on the semantic annotation information to obtain semantic importance density features.

[0008] Combining the first aspect and the above implementation methods, in some possible implementation methods, gradient solving and centroid calculation are performed on the comprehensive density field data to obtain the density gradient vector and the density distribution centroid, including: The comprehensive density field data is subjected to three-dimensional smoothing filtering to obtain density gradient components in three orthogonal directions; The density gradient components in the three orthogonal directions are integrated to obtain the density gradient vector; The centroid location is obtained by solving the density value distribution in the comprehensive density field data.

[0009] Combining the first aspect and the above implementation methods, in some possible implementations, the space corresponding to the 3D scene data is asymmetrically segmented based on the spatial offset parameter to obtain an asymmetric octree spatial index, including: Obtain the variance of the voxel density of the current node to be segmented in the space corresponding to the 3D scene data. The current node to be segmented is the node space in the space corresponding to the 3D scene data where the segmentation operation is being performed. If the variance meets the segmentation triggering condition, then the current node to be segmented is subjected to planar positioning and segmentation based on the spatial offset parameter to obtain asymmetric child nodes; The spatial volume of asymmetric child nodes is normalized and recursively partitioned to obtain the asymmetric octree spatial index.

[0010] Combining the first aspect and the above implementation methods, in some possible implementations, the space corresponding to the 3D scene data is asymmetrically segmented based on the spatial offset parameter to obtain an asymmetric octree spatial index, including: Based on the spatial offset parameter, the space corresponding to the 3D scene data is initially segmented to obtain the initial octree nodes; A sampling ray is emitted from the preset observation viewpoint to the space corresponding to the initial octree node, and an intersection test is performed on the sampling ray to obtain the visible leaf nodes; The visual reachability field is calculated based on the visible leaf nodes, and the initial octree nodes are differentiated and merged according to the visual reachability field to obtain the asymmetric octree spatial index.

[0011] Combining the first aspect and the above implementation methods, in some possible implementations, the initial octree nodes are differentiated and merged based on the line-of-sight reachability field to obtain an asymmetric octree spatial index, including: The visibility of leaf nodes is obtained by calculating the ratio of the number of visible leaf nodes. If the visibility of a leaf node meets the visibility condition and the average density of the corresponding region meets the density threshold condition, then the visible leaf nodes will be further subdivided. If the visibility of a leaf node satisfies the invisible condition and the depth of a visible leaf node satisfies the depth condition, then the visible leaf nodes are merged to obtain an asymmetric octree spatial index.

[0012] Combining the first aspect and the above implementation methods, in some possible implementations, after performing tile conversion processing on the 3D scene data based on the asymmetric octree spatial index to obtain the 3D tile dataset, the following is also included: Acquire the current viewpoint state data, and perform view frustum range prediction processing based on the current viewpoint state data to obtain the predicted view frustum; The target pre-fetched tiles are obtained by filtering and prioritizing the tile density data in the predicted view frustum and 3D tile dataset. The target prefetch tiles are dynamically adjusted and scheduled based on the current network bandwidth and rendering frame rate, so that the client can render 3D scene data based on the target prefetch tiles.

[0013] Secondly, the present invention also provides a three-dimensional scene data processing device, comprising: The data acquisition unit is used to acquire the three-dimensional scene data corresponding to the three-dimensional scene, and to extract multi-dimensional features from the three-dimensional scene data to obtain comprehensive density field data. The first data processing unit is used to perform gradient calculation and centroid calculation on the comprehensive density field data to obtain the density gradient vector and the density distribution centroid. The second data processing unit is used to determine the spatial offset parameter based on the density gradient vector and the centroid of the density distribution. The spatial offset parameter is used to adjust the offset of the segmentation position from the spatial center. The index building unit is used to perform asymmetric segmentation of the space corresponding to the 3D scene data based on the spatial offset parameter to obtain an asymmetric octree spatial index. The spatial volume of the child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node. The data generation unit is used to perform tile conversion processing on 3D scene data based on the asymmetric octree spatial index to obtain a 3D tile dataset.

[0014] Thirdly, the present invention also provides an electronic device, including a data acquisition unit, a memory, and a processor, wherein, A data acquisition device is used to collect 3D scene data corresponding to a 3D scene. Memory, used to store executable program code; The processor is used to extract multi-dimensional features from 3D scene data to obtain comprehensive density field data; perform gradient solving and centroid calculation on the comprehensive density field data to obtain density gradient vector and density distribution centroid; determine spatial offset parameters based on density gradient vector and density distribution centroid; perform asymmetric segmentation on the space corresponding to the 3D scene data based on the spatial offset parameters to obtain an asymmetric octree spatial index; and perform tile conversion on the 3D scene data based on the asymmetric octree spatial index to obtain a 3D tile dataset.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps of the three-dimensional scene data processing method in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: By acquiring the 3D scene data corresponding to the 3D scene, multi-dimensional feature extraction is performed on the 3D scene data to obtain comprehensive density field data, enabling subsequent segmentation to be based on actual density and overcoming the defect of ignoring density differences. Gradient solving and centroid calculation are performed on the comprehensive density field data to obtain the density gradient vector and density distribution centroid, providing a basis for adjusting the segmentation position. Spatial offset parameters are determined based on the density gradient vector and density distribution centroid. These spatial offset parameters are used to adjust the segmentation position from the spatial center, breaking the limitation of uniform segmentation. Based on the spatial offset parameters, asymmetric segmentation processing is performed on the space corresponding to the 3D scene data to obtain an asymmetric octree spatial index. The spatial volume of a child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node, allowing high-density regions to be allocated smaller spatial volumes and balancing the amount of tile data. Based on the asymmetric octree spatial index, tile conversion processing is performed on the 3D scene data to obtain a 3D tile dataset, solving the transmission fluctuation problem caused by uneven data volume and accelerating the rendering speed. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the three-dimensional scene data processing method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S101; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 4 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 6 For the present invention Figure 1 A schematic diagram of an embodiment of S105; Figure 7 This is a schematic diagram of the structure of the three-dimensional scene data processing device provided by the present invention; Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] For example, taking a 3D modeling scenario of a city as an example, the complete city area includes a bustling commercial district in the center, densely populated high-rise residential areas, open green spaces in the suburbs, and open mountainous areas in the far suburbs. When using a traditional uniform octree to construct a spatial index and generate 3D tiles, the algorithm will indiscriminately assign 3D cubes of the same size to the densely built commercial district plots and the almost empty suburban mountainous plots. A single tile in the commercial district contains a large amount of 3D geometric and texture data such as building facades, road components, and green features. The data capacity of a single tile can reach tens of megabytes or even hundreds of megabytes, while the same size tile in the blank suburban area is only a few megabytes in size. The difference in data volume between the two is huge.

[0024] When the front-end loads the scene, the browser or rendering engine requests tile resources sequentially. When downloading low-density suburban tiles, the network transmission time is short and rendering can be completed quickly. However, once the high-density tiles in the commercial area are requested, there will be a peak in instantaneous network bandwidth usage and a significant increase in transmission time. This continuously occupies the rendering thread while waiting for data parsing and loading, which not only causes drastic fluctuations in network load, but also causes problems such as screen stuttering, sudden drops in frame rate, and long-term blank loading of parts of the scene. Overall, the smoothness of 3D scene browsing and loading and the running efficiency are greatly reduced.

[0025] To address the aforementioned technical problems, this application provides a three-dimensional scene data processing method. The executing entity of this method is a three-dimensional scene data processing device or an electronic device equipped with a three-dimensional scene data processing device. A detailed description follows; it should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0026] Please see Figure 1 , Figure 1 This is a flowchart illustrating the three-dimensional scene data processing method provided in an embodiment of this application. Figure 1 As shown, the method in this application embodiment may include the following steps S101-S105: S101: Obtain the 3D scene data corresponding to the 3D scene, perform multi-dimensional feature extraction on the 3D scene data, and obtain comprehensive density field data.

[0027] In this embodiment, a 3D scene refers to a virtual or real environment space containing various 3D models and their spatial distribution. 3D scene data refers to a dataset used to describe the geometry, surface texture, and semantic categories of models within a 3D scene. Multi-dimensional feature extraction refers to the process of calculating and separating the density features of 3D scene data from multiple dimensions, including geometric, visual, and semantic dimensions. Comprehensive density field data refers to a 3D data field that characterizes the comprehensive density distribution of voxel positions in 3D space by fusing geometric density features, visual complexity density features, and semantic importance density features according to adaptive weights.

[0028] In one feasible implementation, the input 3D scene data is read, the corresponding Axis-Aligned Bounding Box (AABB) is calculated, and the bounding box space is divided into an initial voxel mesh. Then, a neighborhood window is extracted centered on each voxel, and the number of geometric elements per unit volume is calculated to obtain the geometric density feature. The weighted fusion value of texture color variance, texture edge density, and normal vector change rate is calculated to obtain the visual complexity density feature, and the area-weighted average value is calculated based on the importance weight of the semantic category to obtain the semantic importance density feature. Based on an adaptive weight fusion algorithm, the above three density features are fused into a comprehensive density value, thereby generating 3D spatial comprehensive density field data.

[0029] S102, perform gradient solving and centroid calculation on the comprehensive density field data to obtain the density gradient vector and density distribution centroid.

[0030] In this embodiment, gradient solving and centroid calculation refer to the process of solving the spatial rate of change after smoothing and filtering the comprehensive density field data, and then calculating the density-weighted spatial center position. The density gradient vector is the vector formed by the direction and magnitude of the fastest rate of change of the comprehensive density field data in three-dimensional space. The density distribution centroid is the spatial geometric center position calculated with the comprehensive density value as the weight within the current node space.

[0031] In one feasible implementation, a three-dimensional Gaussian smoothing filter is first applied to the composite density field data to eliminate noise interference, resulting in smoothed density field data. Subsequently, the density gradient components in the three orthogonal directions (X, Y, Z) of the smoothed density field data are calculated using a multi-scale Sobel difference operator. These three density gradient components are then integrated to generate a density gradient vector and its gradient magnitude and direction unit vector. Simultaneously, based on the density value distribution in the composite density field data, a weighted average is calculated on the spatial coordinates of the voxels using the voxel composite density value as weight, to solve for the three-dimensional coordinates of the density distribution centroid.

[0032] For example, the three-dimensional Gaussian smoothing filter formula in step S102 above is:

[0033] Among them, formula parameters This represents the smoothed density value. The representative standard deviation is The three-dimensional Gaussian kernel function, where * represents the three-dimensional convolution operation. This represents the overall density value before smoothing.

[0034] For example, the formula for generating the density gradient vector in the above steps is:

[0035] Among them, formula parameters Represents the density gradient vector. , , These represent the density gradient components in the X, Y, and Z directions, respectively.

[0036] For example, the formula for calculating the centroid of density distribution in step S102 above is:

[0037] Among them, formula parameters , , These represent the coordinates of the centroid of the density distribution in the X, Y, and Z directions, respectively. This represents the sum of the total density values ​​of all voxels within the current node. , Representing the The overall density value of individual elements, , , Representing the Spatial coordinates of an individual element.

[0038] S103, determine the spatial offset parameters based on the density gradient vector and the centroid of the density distribution.

[0039] In this embodiment, the spatial offset parameter refers to the offset ratio coefficient used to adjust the deviation of the segmentation plane from the spatial geometric center of the current node.

[0040] In one feasible implementation, the spatial extent of the current octree node is obtained, the coordinates of the geometric center of the node's space are calculated, and the normalized offset of the density distribution centroid relative to the geometric center is calculated by combining the gradient direction unit vector in the density gradient vector and the coordinates of the density distribution centroid. This normalized offset is multiplied by the corresponding component of the gradient direction unit vector, multiplied by the segmentation offset intensity coefficient, and then added to the standard center ratio to determine the spatial offset parameters in the three orthogonal directions.

[0041] For example, the formula for calculating the spatial offset parameter in step S103 above is:

[0042]

[0043]

[0044] Among them, formula parameters , , These represent the spatial offset parameters in the X, Y, and Z directions, respectively. This represents the segmentation offset intensity coefficient, and its value ranges from [0, 0.4]. , , This represents the components of the unit gradient vector in the X, Y, and Z directions. , , The coordinates representing the centroid of the density distribution. , , Represents the coordinates of the geometric center of the current node in space. , , Represents the maximum boundary coordinates of the current node space. , , Represents the minimum boundary coordinates of the current node space.

[0045] S104. Based on the spatial offset parameter, the space corresponding to the 3D scene data is asymmetrically segmented to obtain an asymmetric octree spatial index.

[0046] In this embodiment, asymmetric segmentation refers to locating the segmentation plane based on spatial offset parameters, dividing the parent node space into child nodes with unequal volumes, and performing differentiated optimization in conjunction with line-of-sight occlusion perception. An asymmetric octree spatial index refers to a tree-shaped spatial data structure generated by asymmetric segmentation, where the volume of child node space is inversely proportional to the overall density and has undergone occlusion optimization.

[0047] In one feasible implementation, the variance of the voxel composite density within the current node to be segmented is first obtained. If this variance satisfies the segmentation triggering condition, the positions of three segmentation planes are located based on spatial offset parameters, and the current node to be segmented is divided into eight asymmetric child nodes. The spatial volume of each child node is then determined by normalizing it according to the inverse relationship of the mean composite density of the corresponding region. Next, sampling line-of-sight rays are emitted from a preset observation viewpoint, and intersection tests are performed on the rays to establish a line-of-sight accessibility field, calculating the visibility of leaf nodes. Leaf nodes that are visible and whose mean composite density is greater than the visible density threshold are further subdivided, while leaf nodes that are invisible and whose depth is greater than the minimum merging depth are merged. Finally, the above operations are recursively performed to obtain the asymmetric octree spatial index.

[0048] For example, the segmentation plane positioning formula in step S104 above is:

[0049]

[0050]

[0051] Among them, formula parameters , , These represent the coordinate positions of the dividing plane in the X, Y, and Z directions, respectively.

[0052] For example, the normalization formula for the volume of asymmetric child nodes in step S104 above is:

[0053] Among them, formula parameters Representing the The spatial volume of each child node Represents the spatial volume of the parent node. Representing the The average comprehensive density within the region corresponding to each child node. This represents the regularization parameter to prevent division by zero. The index number representing the child node.

[0054] S105, based on the asymmetric octree spatial index, performs tile conversion processing on the 3D scene data to obtain a 3D tile dataset.

[0055] In this embodiment, tile conversion processing refers to the process of converting node data in an asymmetric octree spatial index into a three-dimensional tile format, adaptively determining geometric errors, and performing incremental updates. The three-dimensional tile dataset refers to the converted data set containing tile files at each level and corresponding geometric error parameters.

[0056] In one feasible implementation, the nodes of the asymmetric octree spatial index are traversed, and the geometric error of each tile is adaptively determined based on the density sensitivity coefficient and the average density of the nodes. The 3D scene data within the nodes is then converted into 3D tile format files to generate a 3D tile dataset. When the scene data is updated, the spatial range of the updated data is located, and the intersection relationship with the spatial range is recursively determined from the root node to track the affected nodes. Density recalculation and local segmentation readjustment are re-performed for the affected nodes, and only the tile files of the affected nodes are regenerated to achieve incremental updates.

[0057] For example, the above formula for adaptive determination of geometric error can be:

[0058] Among them, formula parameters Represents the geometric error of the current tile. Represents the fundamental geometric error. This represents the density sensitivity coefficient, and its value ranges from [0,1]. This represents the overall density value of the current node. This represents the average comprehensive density value of the current node's level.

[0059] In summary, this application obtains 3D scene data corresponding to a 3D scene, performs multi-dimensional feature extraction on the 3D scene data to obtain comprehensive density field data, enabling subsequent segmentation to be based on actual density and overcoming the defect of ignoring density differences. Gradient solving and centroid calculation are performed on the comprehensive density field data to obtain the density gradient vector and density distribution centroid, providing a basis for adjusting the segmentation position. Spatial offset parameters are determined based on the density gradient vector and density distribution centroid. These spatial offset parameters are used to adjust the segmentation position from the spatial center, breaking the limitation of uniform segmentation. Based on the spatial offset parameters, asymmetric segmentation processing is performed on the space corresponding to the 3D scene data to obtain an asymmetric octree spatial index. The spatial volume of child nodes in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node, allowing high-density regions to be allocated smaller spatial volumes and balancing the amount of tile data. Based on the asymmetric octree spatial index, tile conversion processing is performed on the 3D scene data to obtain a 3D tile dataset, solving the transmission fluctuation problem caused by data imbalance and accelerating rendering speed.

[0060] Please see Figure 2 , Figure 2 This is a flowchart illustrating the three-dimensional scene data processing method provided in an embodiment of this application. Figure 2 As shown, the method in this application embodiment may include the following steps S201-S202: S201, spatially divides the bounding box space corresponding to the 3D scene data to obtain a voxel mesh.

[0061] In this embodiment, the bounding box space refers to the smallest cuboid space containing all 3D scene data with edges parallel to the coordinate axes. Space partitioning refers to the process of discretizing the continuous bounding box space into a 3D mesh according to a preset voxel resolution. A voxel mesh refers to a 3D mesh structure composed of multiple cubic volume units of the same size.

[0062] In one feasible implementation, the input 3D model data is read, and the range and size of the axis-aligned bounding box space corresponding to the data in the X, Y, and Z directions are calculated. An initial voxel resolution is determined based on the scene scale and accuracy requirements. The bounding box space is then divided along the three coordinate axes according to the initial voxel resolution to generate a discretized voxel mesh, and the number of voxels in each direction is calculated.

[0063] For example, the formula for calculating the number of voxels in the above steps is:

[0064] Among them, formula parameters , , These represent the number of voxels in the X, Y, and Z directions, respectively. , , These represent the dimensions of the bounding box space in the X, Y, and Z directions, respectively. , , These represent the voxel resolution in the X, Y, and Z directions, respectively.

[0065] S202 extracts geometric density features, visual complexity density features, and semantic importance density features from 3D scene data based on voxel meshes.

[0066] In this embodiment, geometric density features refer to the statistical value of the number of geometric elements per unit volume. Visual complexity density features refer to a comprehensive visual index reflecting texture color changes, edge richness, and normal vector change rate. Semantic importance density features refer to area-weighted importance evaluation values ​​based on the semantic categories of the model.

[0067] In one feasible implementation, each voxel in the voxel grid is traversed, and a neighborhood window is taken with the current voxel as the center. The number of geometric elements in the window is counted and divided by the neighborhood volume to obtain the geometric density feature. The texture data in the window is extracted to calculate the mean of the standard deviations of the red, green and blue channels, the proportion of edge pixels after Sobel edge detection, and the average of the angle between the normal vectors of adjacent triangular facets. The three are weighted and fused to obtain the visual complexity density feature. At the same time, the semantic annotation information associated with the current voxel is obtained, and importance weights are assigned according to the semantic category. The area-weighted average is calculated to obtain the semantic importance density feature.

[0068] S203 uses adaptive weight fusion processing to combine geometric density features, visual complexity density features, and semantic importance density features to obtain comprehensive density field data.

[0069] In the embodiments of this application, adaptive weight fusion processing refers to the process of dynamically adjusting the weight ratio of geometric, visual and semantic density features and performing linear combination according to different application scenarios.

[0070] In one feasible implementation, the application scenario type corresponding to the current 3D scene is obtained. Based on a preset weight configuration table, the geometric density feature weight, visual complexity density feature weight, and semantic importance density feature weight corresponding to that application scenario type are queried. The three density features are multiplied by their corresponding weights and then summed to calculate the comprehensive density value for each voxel in the voxel mesh. The comprehensive density values ​​of all voxels constitute the comprehensive density field data in 3D space.

[0071] For example, the preset weight configuration table includes four application scenarios: urban digital twin, autonomous driving simulation, cultural heritage digitization, and BIM collaborative platform. The three weights correspond to the geometric density feature weight wg, the visual complexity density feature weight wv, and the semantic importance density feature weight ws, respectively. Specifically, the three weights for the urban digital twin scenario are 0.50, 0.20, and 0.30, respectively; the three weights for the autonomous driving simulation scenario are 0.60, 0.15, and 0.25, respectively; the three weights for the cultural heritage digitization scenario are 0.35, 0.35, and 0.30, respectively; and the three weights for the BIM collaborative platform scenario are 0.55, 0.25, and 0.20, respectively.

[0072] For example, the adaptive weight fusion formula in the above steps is:

[0073] Among them, formula parameters Represents the overall density value. Represents geometric density characteristics, Represents visual complexity density features. Represents semantic importance density features. Represents the geometric density feature weights. The weights of the density features represent the visual complexity. The weights of the semantic importance density features are represented and satisfy the following conditions:

[0074] In summary, this application discretizes the bounding box space corresponding to the 3D scene data into a voxel mesh and extracts density features from the geometric, visual, and semantic dimensions respectively for adaptive weight fusion, thereby achieving a multi-dimensional and accurate quantitative evaluation of the spatial density of the 3D scene and providing reliable data support for subsequent density-based asymmetric segmentation.

[0075] In one feasible implementation, when performing the step of extracting geometric density features, visual complexity density features, and semantic importance density features of the 3D scene data based on the voxel mesh, the following is also specifically performed: Obtain the number of geometric features within the voxel mesh, and then obtain the geometric density feature based on the number of geometric features.

[0076] In this embodiment, the number of geometric elements refers to the number of point cloud points, triangular mesh faces, or polygon vertices contained within the voxel neighborhood window.

[0077] In one feasible implementation, for each voxel unit, a neighborhood window is taken centered on the current voxel, and the number of geometric features of all 3D model data within the neighborhood window is counted. The total volume of the neighborhood window is obtained, and the number of geometric features is divided by the total neighborhood volume to calculate the number of geometric features per unit volume as the geometric density feature. The size of the neighborhood window is adaptively adjusted according to the current octree level; the deeper the level, the smaller the window.

[0078] For example, the formula for obtaining the geometric density feature is as follows:

[0079] Among them, formula parameters Represents geometric density characteristics, This represents the number of geometric features within the neighborhood window. Represents the total volume of the neighboring window.

[0080] Variance and edge detection processing is performed on the texture data within the voxel mesh to obtain visual complexity density features.

[0081] In this embodiment, texture data refers to the color pattern and normal angle information of the surface of the three-dimensional model. Variance and edge detection processing refers to the process of calculating the mean standard deviation of the red, green and blue color channels, extracting the proportion of edge pixels, and calculating the average angle between the normal vectors.

[0082] In one feasible implementation, texture data of the 3D model within the voxel neighborhood window is extracted, the standard deviations of the red, green, and blue color channels are calculated and their mean values ​​are obtained, and then normalized and mapped to the [0,1] interval to obtain the texture color variance index. The Sobel edge detection operator is used to process the texture image, and the proportion of edge pixels to the total number of pixels is calculated to obtain the texture edge density index. The average angle between the normal vectors of adjacent triangular facets is calculated to obtain the normal vector change rate index. These three indices are then weighted and fused according to preset weights to obtain the visual complexity density feature.

[0083] Semantic annotation information of 3D scene data is obtained, and weights are assigned based on the semantic annotation information to obtain semantic importance density features.

[0084] In this embodiment, semantic annotation information refers to data labels that identify the category attribute to which the 3D model belongs. Weight assignment processing refers to the process of assigning importance coefficients to models of different categories and calculating an area-weighted average value based on a preset semantic category importance weight mapping table.

[0085] For example, the semantic category importance weight mapping table includes seven semantic categories: buildings, bridges, roads, water bodies, terrain, vegetation, and pipelines, with corresponding importance weights of 0.90, 0.85, 0.70, 0.50, 0.20, 0.30, and 0.60, respectively.

[0086] In one feasible implementation, semantic annotation information corresponding to each model in the 3D scene data is acquired, and the corresponding semantic categories are identified. Semantic categories include buildings, bridges, roads, water bodies, terrain, vegetation, pipelines, etc. Based on a preset semantic category importance weight mapping table, each semantic category is assigned a corresponding importance weight. The coverage area of ​​each semantic category within the voxel neighborhood window is calculated. The importance weight of each semantic category is multiplied by its coverage area, summed, and then divided by the total coverage area to obtain the area-weighted average semantic importance density feature.

[0087] For example, the formula for obtaining the semantic importance density feature is as follows:

[0088] Among them, formula parameters Represents semantic importance density features. Representing the The coverage area of ​​each semantic category within the neighborhood window Representing the The importance weights corresponding to each semantic category.

[0089] In summary, this application achieves precise quantitative extraction of geometric, visual, and semantic features of 3D scene data by separately calculating the geometric element density, the weighted sum of visual complexity indicators, and the area-weighted average of semantic importance, ensuring the independence and accuracy of feature evaluation for each dimension.

[0090] Please see Figure 3 , Figure 3 This is a flowchart illustrating the three-dimensional scene data processing method provided in an embodiment of this application. Figure 3 As shown, the method in this application embodiment may include the following steps S301-S303: S301 performs three-dimensional smoothing filtering on the comprehensive density field data to obtain density gradient components in three orthogonal directions.

[0091] In this embodiment, the three-dimensional smoothing filtering process refers to the process of using a three-dimensional Gaussian kernel function to perform convolution operations on the comprehensive density field data to eliminate noise interference. The smoothed density field data refers to the comprehensive density field data after three-dimensional Gaussian smoothing filtering to remove local noise. The three orthogonal directions refer to the mutually perpendicular X, Y, and Z coordinate axes in three-dimensional space. The density gradient component refers to the rate of change of the smoothed density field data in the three orthogonal directions of X, Y, and Z.

[0092] In one feasible implementation, a three-dimensional Gaussian kernel function is constructed, and the standard deviation corresponding to the current octree level is determined. This three-dimensional Gaussian kernel function is then convolved with the comprehensive density field data in three dimensions to obtain smoothed density field data. Subsequently, a multi-scale Sobel difference operator is used to calculate the partial derivatives of the smoothed density field data in the three orthogonal directions (X, Y, and Z), obtaining the density gradient components in the X, Y, and Z directions.

[0093] For example, the specific form of the three-dimensional Gaussian smoothing filter formula in the above steps is as follows:

[0094] Among them, formula parameters Represents smooth density field data, The representative standard deviation is The three-dimensional Gaussian kernel function, where * represents the three-dimensional convolution operation. This represents the overall density field data before smoothing.

[0095] For example, the specific form of the three-dimensional Gaussian kernel function in step S301 above is:

[0096] Among them, formula parameters , , This represents the spatial coordinate offset relative to the center of the voxel. This represents the standard deviation of the Gaussian kernel function, and its value is adaptively adjusted according to the octree level. For example, it is 40m for level 0, 30m for level 1, 20m for level 2, 15m for level 3, 10m for level 4, and 5m for level 5 and above.

[0097] For example, the specific form of the formula for calculating the density gradient components in the above steps is as follows:

[0098] Among them, formula parameters , , These represent the density gradient components in the three orthogonal directions X, Y, and Z, respectively. , , These represent the multi-scale Sobel difference operators in the X, Y, and Z directions, respectively.

[0099] S302 integrates the density gradient components in the three orthogonal directions to obtain the density gradient vector.

[0100] In the embodiments of this application, the density gradient vector refers to a three-dimensional vector composed of density gradient components in three orthogonal directions, which characterizes the direction and magnitude of the fastest change in the integrated density field data in space.

[0101] In one feasible implementation, the density gradient components in the X, Y, and Z directions are vector-synthesized to generate a density gradient vector. The gradient magnitude of the density gradient vector is further calculated, and the density gradient vector is divided by the gradient magnitude to obtain a unit vector in the gradient direction. The gradient magnitude is compared with a preset gradient threshold; if the gradient magnitude is greater than the preset threshold, the corresponding region is identified as a region of abrupt density change.

[0102] For example, the specific form of the density gradient vector synthesis and magnitude calculation formula in the above steps is as follows:

[0103]

[0104] Among them, formula parameters Represents the density gradient vector. This represents the gradient magnitude of the density gradient vector.

[0105] For example, the specific form of the formula for calculating the unit vector of the gradient direction in the above steps is as follows:

[0106] Among them, formula parameters Represents the unit vector of the gradient direction. , , These represent the components of the gradient direction unit vector in the X, Y, and Z directions, respectively.

[0107] S303, based on the density value distribution in the comprehensive density field data, the centroid position is solved to obtain the density distribution centroid.

[0108] In this embodiment, density value distribution refers to the spatial arrangement of the comprehensive density field data at each voxel location. The centroid location determination process refers to the process of calculating the density-weighted spatial center location by weighting the voxel spatial coordinates using the comprehensive density value of the voxels as weights.

[0109] In one feasible implementation, the comprehensive density value of all voxels in the current node space and their corresponding spatial coordinates are obtained, and the sum of the comprehensive density values ​​of all voxels is calculated as a normalization factor. The comprehensive density value of each voxel is multiplied by its X, Y, and Z coordinates and then summed, and then divided by the normalization factor to calculate the coordinates of the centroid of the density distribution in the X, Y, and Z directions.

[0110] For example, the specific form of the formula for calculating the centroid of the density distribution in the above steps is as follows:

[0111] Among them, formula parameters , , These represent the coordinates of the centroid of the density distribution in the X, Y, and Z directions, respectively. Represents the sum of the total density values ​​of all voxels within the current node and , Representing the The overall density value of individual elements, , , Representing the Spatial coordinates of an individual element.

[0112] In summary, this application achieves accurate quantification of density abrupt changes and distribution centroids in three-dimensional space by performing three-dimensional Gaussian smoothing and multi-scale difference solution on the comprehensive density field, and solving the density gradient vector by density weighting. This provides a mathematical basis for the direction and position offset of adaptive asymmetric segmentation.

[0113] Please see Figure 4 , Figure 4 This is a flowchart illustrating the three-dimensional scene data processing method provided in an embodiment of this application. Figure 4 As shown, the method in this application embodiment may include the following steps S401-S403: S401, obtain the variance of the voxel comprehensive density within the current node to be segmented in the space corresponding to the 3D scene data.

[0114] In this embodiment, the current node to be segmented refers to the octree node space in the space corresponding to the 3D scene data where segmentation judgment and segmentation operations are being performed. The voxel composite density refers to the numerical value of the composite density field data at each voxel position within the current node to be segmented. Variance is a statistical measure used to measure the degree of dispersion between the composite density of all voxels within the current node to be segmented and its mean.

[0115] In one feasible implementation, the average value of the overall density of all voxels in the current node to be segmented is calculated, and each voxel in the node is traversed to calculate the square of the difference between its overall density and the average value. The sum of the squares of all differences is then divided by the total number of voxels to obtain the variance of the overall density of the voxels.

[0116] For example, the specific form of the above variance calculation formula is as follows:

[0117] Among them, formula parameters The variance representing the overall density of voxels. This represents the total number of voxels within the current node to be segmented. Representing the The overall density of individual elements, Represents the average value of the total density of all voxels within the current node to be segmented, and .

[0118] S402, if the variance meets the segmentation triggering condition, then the current node to be segmented is subjected to planar positioning and segmentation based on the spatial offset parameter to obtain asymmetric child nodes.

[0119] In this embodiment, the segmentation triggering condition refers to a logical combination of the following conditions: the variance of the voxel integrated density is greater than a preset density variance threshold, the depth of the current node to be segmented is less than the maximum segmentation depth, and the node data volume is greater than the minimum data volume threshold. Planar positioning and segmentation processing refers to the process of determining the coordinate positions of three segmentation planes based on spatial offset parameters, and then dividing the current node to be segmented into eight non-overlapping subspaces. Asymmetric child nodes refer to octree child nodes generated after planar positioning and segmentation processing, whose spatial volume is inversely proportional to the integrated density of the corresponding region.

[0120] In one feasible implementation, it is determined whether the variance is greater than a preset density variance threshold, whether the current depth is less than the maximum segmentation depth, and whether the node data volume is greater than a minimum data volume threshold. If all three conditions are met, node segmentation is triggered. Spatial offset parameters are calculated based on the density gradient vector and the centroid of the density distribution. The coordinate positions of three segmentation planes are then calculated based on the spatial offset parameters and the boundary range of the node to be segmented. The node to be segmented is cut along these three segmentation planes, dividing it into eight asymmetric child nodes.

[0121] Please refer to step S103 above for the specific form of the plane segmentation positioning formula, which will not be repeated here.

[0122] S403 normalizes and recursively partitions the spatial volume of asymmetric child nodes to obtain the asymmetric octree spatial index.

[0123] In the embodiments of this application, normalization and recursive segmentation processing refers to allocating the volume of the parent node according to the inverse relationship of the average comprehensive density of the child node region, and repeatedly performing segmentation judgment and segmentation operation on each child node until the termination condition is met.

[0124] In one feasible implementation, the mean comprehensive density of the region corresponding to each asymmetric child node is calculated. An inverse proportional weight is then calculated based on this mean comprehensive density, and the weight is normalized using a regularization parameter to prevent division by zero, thus determining the final spatial volume of each asymmetric child node. Variance calculation and segmentation trigger judgment are recursively performed on each generated asymmetric child node. If the segmentation trigger condition is met, planar positioning and segmentation continue until all leaf nodes no longer meet the segmentation trigger condition, ultimately constructing an asymmetric octree spatial index.

[0125] For example, the specific form of the asymmetric child node volume normalization formula in the above steps is as follows:

[0126] Among them, formula parameters Representing the The spatial volume of asymmetric child nodes This represents the spatial volume of the node to be segmented, which serves as the parent node. Representing the The average comprehensive density within the region corresponding to each asymmetric child node. This represents the regularization parameter to prevent division by zero (e.g., a value of 0.1). The index number representing the asymmetric child node.

[0127] In summary, this application calculates the variance of the density within the node to be segmented to trigger the segmentation condition, and combines spatial offset parameters to locate the segmentation plane and normalize the volume inversely proportional to the density, thereby realizing adaptive asymmetric octree segmentation based on density distribution characteristics, effectively balancing the amount of data in each node.

[0128] Please see Figure 5 , Figure 5 This is a flowchart illustrating the three-dimensional scene data processing method provided in an embodiment of this application. Figure 5 As shown, the method in this application embodiment may include the following steps S501-S503: S501 performs preliminary segmentation of the space corresponding to the 3D scene data based on the spatial offset parameter to obtain the initial octree nodes.

[0129] In this embodiment, the preliminary segmentation process refers to determining spatial offset parameters based on the density gradient vector and the centroid of the density distribution, and then performing asymmetric segmentation of the 3D scene space accordingly to construct a basic tree structure. The initial octree node refers to the set of octree nodes generated after the preliminary asymmetric segmentation process, which has not yet undergone line-of-sight occlusion perception optimization.

[0130] In one feasible implementation, the density variance calculation, segmentation trigger judgment, spatial offset parameter determination and asymmetric segmentation processing described in steps S401 to S403 above are performed on the space corresponding to the three-dimensional scene data. The three-dimensional scene space is recursively segmented into a series of subspaces, and these subspaces are used as initial octree nodes to construct the initial asymmetric octree structure.

[0131] S502, a sampling ray is emitted from the preset observation viewpoint to the space corresponding to the initial octree node, and the intersection test is performed on the sampling ray to obtain the visible leaf node.

[0132] In this embodiment, the observation viewpoint refers to the spatial location of the virtual camera during 3D scene rendering. A sampling ray is a parametric geometric ray emitted from the observation viewpoint along a specific direction to simulate human eye line of sight or ray tracing. The intersection test process refers to calculating whether the sampling ray intersects with the bounding box of the initial octree nodes and determining the first intersecting leaf node. A visible leaf node is the leaf node in the octree space that the sampling ray first hits and is not occluded by other nodes.

[0133] In one feasible implementation, a preset viewpoint coordinate is obtained, and a preset number of sampling rays are uniformly emitted into the three-dimensional scene space starting from this viewpoint. For each sampling ray, the initial octree nodes are traversed, and the intersection of the ray and the node bounding box is recursively determined. If the ray enters a leaf node and that leaf node is the first node reached by the ray along the propagation direction, then that leaf node is marked as a visible leaf node.

[0134] For example, the specific form of the parameterized equation for the sampled ray in the above steps is as follows:

[0135] Among them, formula parameters Representing the The parameters on the sampling ray are The coordinates of a point in space. Spatial coordinates representing the observation viewpoint The parameter representing the propagation distance of the ray. Representing the The unit vector of the direction of the sampling ray.

[0136] S503 calculates the visual reachability field based on the visible leaf nodes, and performs differential subdivision and merging of the initial octree nodes according to the visual reachability field to obtain the asymmetric octree spatial index.

[0137] In this embodiment, the line-of-sight accessibility field refers to the spatial distribution field characterizing the probability that each leaf node in the initial octree node is hit by the line of sight from the current viewing point. Differential subdivision and merging refers to the process of further subdividing visible and high-density leaf nodes, while merging and optimizing invisible and deep leaf nodes.

[0138] In one feasible implementation, the number of times each leaf node is marked as a visible leaf node is counted, and the ratio of this number to the total number of sampled rays is calculated to establish a line-of-sight accessibility field. The leaf nodes in the line-of-sight accessibility field are traversed. For leaf nodes whose visibility meets the visibility condition and whose overall density mean meets the density threshold condition, supplementary subdivision processing is performed; for leaf nodes whose visibility meets the invisibility condition and whose node depth meets the depth condition, merging processing is performed, ultimately forming an optimized asymmetric octree spatial index.

[0139] In summary, this application achieves an effective balance between rendering quality and performance by subdividing and merging nodes based on the line-of-sight accessibility field, providing finer segmentation in the visible area to improve rendering details, and merging in the invisible area to save storage and computing resources.

[0140] In one feasible implementation, when performing the step of differentially subdividing and merging the initial octree nodes based on the line-of-sight accessibility field to obtain the asymmetric octree spatial index, the following is also specifically executed: The visibility of leaf nodes is obtained by calculating the ratio of the number of visible leaf nodes.

[0141] In this embodiment, the ratio calculation process refers to the process of calculating the ratio of the number of rays that mark a single leaf node as a visible leaf node to the total number of rays emitted from the observation viewpoint. Leaf node visibility refers to the numerical result obtained from the above ratio calculation process, used to quantify the visibility probability of the leaf node at the current viewpoint.

[0142] In one feasible implementation, for each leaf node in the initial octree nodes, the number of visible line-of-sight rays hitting the leaf node is counted as the visible ray count of the leaf node, and the total number of sampled line-of-sight rays emitted from the observation viewpoint is obtained. The visible ray count is divided by the total number of rays to obtain the leaf node visibility.

[0143] For example, the specific form of the formula for calculating the visibility of leaf nodes is as follows:

[0144] Among them, formula parameters Represents the visibility of leaf nodes. This represents the number of visible line-of-sight rays that hit the current leaf node. This represents the total number of rays emitted from the observation viewpoint along the sampling line of sight.

[0145] If the visibility of a leaf node meets the visibility condition and the average density of the corresponding region meets the density threshold condition, then the visible leaf nodes will be further subdivided.

[0146] In this embodiment, the visibility condition refers to the leaf node's visibility being greater than or equal to a preset visibility threshold. The average comprehensive density refers to the arithmetic mean of the comprehensive density of all voxels within the spatial region corresponding to the leaf node. The density threshold condition refers to the average comprehensive density being greater than a preset visibility density threshold. The supplementary subdivision processing refers to the process of further performing asymmetric octree segmentation on the leaf nodes that meet the conditions to generate deeper child nodes.

[0147] In one feasible implementation, it is determined whether the visibility of the current leaf node is greater than or equal to a preset visibility threshold (e.g., 0.1). If so, the leaf node is determined to meet the visibility condition. The mean comprehensive density of the region corresponding to the leaf node is further calculated, and it is determined whether this mean comprehensive density is greater than a preset visibility density threshold (e.g., 2.5). If so, it is determined to meet the density threshold condition. For leaf nodes that simultaneously meet both the visibility condition and the density threshold condition, the spatial offset parameters are recalculated based on the density gradient vector and density distribution centroid within the leaf node, and asymmetric segmentation is performed on the leaf node to generate more refined child nodes.

[0148] If the visibility of a leaf node satisfies the invisible condition and the depth of a visible leaf node satisfies the depth condition, then the visible leaf nodes are merged to obtain an asymmetric octree spatial index.

[0149] In this embodiment, the invisible condition refers to the leaf node's visibility being less than a preset visibility threshold. The depth condition refers to the leaf node's level depth in the octree being greater than a preset minimum merge depth. The merge process refers to the inverse splitting operation of merging leaf nodes that meet the conditions with their sibling nodes into a single parent node.

[0150] In one feasible implementation, it is determined whether the visibility of the current leaf node is less than a preset visibility threshold (e.g., 0.1). If so, the leaf node is determined to meet the invisible condition. The layer depth of the leaf node is further obtained, and it is determined whether this depth is greater than a preset minimum merge depth (e.g., 2). If so, it is determined to meet the depth condition. For leaf nodes that simultaneously meet both the invisible and depth conditions, they are merged with their sibling nodes that share the same parent node. These child nodes are removed, and their parent node is restored to a leaf node state. Finally, after global traversal and adjustment, the asymmetric octree spatial index is obtained.

[0151] In summary, this application achieves adaptive optimization of the octree structure under line-of-sight occlusion perception by quantifying the visibility of leaf nodes, supplementing and subdividing visible high-density nodes, and merging invisible deep-level nodes. This significantly reduces resource consumption in invisible areas while ensuring rendering accuracy in visible areas.

[0152] Please see Figure 6 , Figure 6 This is a flowchart illustrating the three-dimensional scene data processing method provided in an embodiment of this application. Figure 6 As shown, the method in this application embodiment may include the following steps S601-S603: S601, acquire the current viewpoint state data, and perform view frustum range prediction processing based on the current viewpoint state data to obtain the predicted view frustum.

[0153] In this embodiment, the current viewpoint state data refers to the spatial position, movement speed, and line-of-sight vector of the current viewing point. The view frustum range prediction process refers to the process of calculating the future viewing point position based on the current viewpoint state data and constructing the future view frustum. The predicted view frustum refers to the visible spatial pyramidal region corresponding to the future viewpoint obtained from the view frustum range prediction process.

[0154] In one feasible implementation, the spatial position, movement speed, and line-of-sight vector of the current observation viewpoint are acquired in real time as the current viewpoint state data. Based on the movement speed and line-of-sight vector, the displacement in the future time increment is calculated. This displacement is added to the current viewpoint position to obtain the predicted viewpoint position in the future. Based on this predicted viewpoint position and the field of view angle parameters, a predicted view frustum is constructed.

[0155] S602, based on the predicted view frustum and the tile density data in the 3D tile dataset, performs filtering and priority calculation to obtain the target pre-fetched tile.

[0156] In this embodiment, tile density data refers to the comprehensive density mean information of each tile in the 3D tile dataset. The filtering and priority calculation process refers to the process of filtering tiles that intersect with the predicted view frustum and have high density, and calculating the loading priority based on density, depth, and visibility time. The target prefetch tiles refer to the set of tiles determined after filtering and priority calculation that need to be prefetched to the client with priority.

[0157] In one feasible implementation, the 3D tile dataset is traversed, and tiles whose bounding boxes intersect with the predicted view frustum are selected. Furthermore, tiles with a mean overall density greater than a preset prefetch density threshold are selected as high-density tiles. The mean overall density, node depth, and historical visibility time of these high-density tiles are obtained, and tile priorities are calculated using corresponding weighting coefficients. Tiles are then sorted from highest to lowest priority, and the tiles ranked highest are selected as target prefetch tiles.

[0158] For example, the specific form of the tile priority calculation formula in step S602 above is as follows:

[0159] Among them, formula parameters Represents tile priority. This represents the average overall density of the current tile. This represents the mean of the maximum integrated density in the 3D tile dataset. This represents the depth of the current tile in the asymmetric octree spatial index. Represents the maximum allowed depth. Represents the minimum depth. Represents the visible historical time of the current tile. Represents the longest historically visible period.

[0160] S603 dynamically adjusts and schedules the target prefetch tiles based on the current network bandwidth and rendering frame rate, so that the client can render 3D scene data based on the target prefetch tiles.

[0161] Current network bandwidth refers to the currently available data transfer rate between the client and the server. Rendering frame rate refers to the number of frames rendered and displayed per second by the client. Dynamic adjustment and scheduling refers to the process of calculating the upper limit of prefetched data based on bandwidth and dynamically increasing or decreasing the prefetch quantity and loading granularity in conjunction with the rendering frame rate.

[0162] In one feasible implementation, the current network bandwidth and a preset prefetch time window are obtained, and the upper limit of the current allowed prefetch data volume is calculated. Based on this upper limit, a corresponding number of tiles are extracted from the target prefetch tiles to generate a prefetch queue. During scheduling, the client's rendering frame rate is monitored in real time. If the rendering frame rate is lower than a preset frame rate threshold, the loading granularity is reduced; if the rendering frame rate is greater than or equal to the preset frame rate threshold and the current bandwidth utilization is less than a preset utilization threshold, the prefetch quantity is increased so that the client renders 3D scene data based on the adjusted target prefetch tiles.

[0163] For example, the specific form of the formula for calculating the upper limit of prefetched data in the above steps is as follows:

[0164] Among them, formula parameters This represents the maximum amount of data that can be prefetched at present (in megabytes). This represents the current network bandwidth (in megabits per second). This represents the preset prefetch time window (in seconds).

[0165] In summary, this application achieves on-demand loading and adaptive adjustment of 3D scene tiles by predicting the viewpoint frustum range and calculating the priority of intersecting tiles, and by dynamically prefetching and scheduling based on the current network bandwidth and rendering frame rate. This effectively shortens the scene loading time and ensures the stability of the rendering frame rate.

[0166] The following will combine Figure 7 This application provides a detailed description of the three-dimensional scene data processing apparatus provided in its embodiments. It should be noted that... Figure 7 The three-dimensional scene data processing device 700 in the present application is used to execute the data processing device 700 of the present application. Figures 1-6 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figures 1-6 In the embodiment shown, the 3D scene data processing device 700 may include a data acquisition unit 701, a first data processing unit 702, a second data processing unit, an index construction unit 704, and a data generation unit 705, as detailed below: The data acquisition unit 701 is used to acquire the three-dimensional scene data corresponding to the three-dimensional scene, perform multi-dimensional feature extraction on the three-dimensional scene data, and obtain comprehensive density field data. The first data processing unit 702 is used to perform gradient solution and centroid calculation on the comprehensive density field data to obtain the density gradient vector and the density distribution centroid. The second data processing unit 703 is used to determine the spatial offset parameter based on the density gradient vector and the centroid of the density distribution. The spatial offset parameter is used to adjust the offset of the segmentation position from the spatial center. The index building unit 704 is used to perform asymmetric segmentation processing on the space corresponding to the 3D scene data based on the spatial offset parameter to obtain an asymmetric octree spatial index. The spatial volume of the child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node. The data generation unit 705 is used to perform tile conversion processing on the 3D scene data based on the asymmetric octree spatial index to obtain a 3D tile dataset.

[0167] The three-dimensional scene data processing device 700 provided in the above embodiments can implement the technical solutions described in the above three-dimensional scene data processing method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above three-dimensional scene data processing method embodiments, and will not be repeated here.

[0168] like Figure 8 As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802, a display 803, and a data acquisition device 804. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0169] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the three-dimensional scene data processing method of the present invention.

[0170] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0171] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0172] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0173] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display information from electronic device 800 and to display a visual user interface. Data acquisition unit 804 is used to acquire 3D scene data corresponding to the 3D scene. Components 801-804 of electronic device 800 communicate with each other via a bus.

[0174] In one embodiment, when the processor 801 executes the three-dimensional scene data processing program in the memory 802, the following steps can be implemented: Obtain the 3D scene data corresponding to the 3D scene, perform multi-dimensional feature extraction on the 3D scene data, and obtain comprehensive density field data; Gradient calculation and centroid calculation are performed on the comprehensive density field data to obtain the density gradient vector and the density distribution centroid. The spatial offset parameter is determined based on the density gradient vector and the centroid of the density distribution. The spatial offset parameter is used to adjust the offset of the segmentation position from the spatial center. Based on the spatial offset parameter, the space corresponding to the 3D scene data is asymmetrically segmented to obtain an asymmetric octree spatial index. The spatial volume of the child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node. A 3D scene data is transformed into a 3D tile dataset by using an asymmetric octree spatial index.

[0175] It should be understood that when the processor 801 executes the three-dimensional scene data processing program in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0176] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0177] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions in the three-dimensional scene data processing methods provided in the above-described method embodiments.

[0178] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0179] The above provides a detailed description of the three-dimensional scene data processing method, apparatus, and electronic device provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for processing three-dimensional scene data, characterized in that, include: Obtain the three-dimensional scene data corresponding to the three-dimensional scene, and perform multi-dimensional feature extraction on the three-dimensional scene data to obtain comprehensive density field data; The integrated density field data is processed by gradient solving and centroid calculation to obtain the density gradient vector and density distribution centroid. The spatial offset parameter is determined based on the density gradient vector and the centroid of the density distribution. The spatial offset parameter is used to adjust the offset of the segmentation position from the spatial center. Based on the spatial offset parameter, the space corresponding to the three-dimensional scene data is asymmetrically segmented to obtain an asymmetric octree spatial index. The spatial volume of the child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node. The 3D scene data is processed by tile conversion based on the asymmetric octree spatial index to obtain a 3D tile dataset.

2. The method according to claim 1, characterized in that, The process of extracting multi-dimensional features from the three-dimensional scene data to obtain comprehensive density field data includes: The bounding box space corresponding to the three-dimensional scene data is spatially divided to obtain a voxel mesh. Based on the voxel mesh, the geometric density features, visual complexity density features, and semantic importance density features of the 3D scene data are extracted; The geometric density features, visual complexity density features, and semantic importance density features are adaptively weighted and fused to obtain comprehensive density field data.

3. The method according to claim 2, characterized in that, The extraction of geometric density features, visual complexity density features, and semantic importance density features from the 3D scene data based on the voxel mesh includes: The number of geometric elements within the voxel mesh is obtained, and the geometric density feature is obtained based on the number of geometric elements. Variance and edge detection processing is performed on the texture data within the voxel mesh to obtain visual complexity density features; Semantic annotation information of the 3D scene data is obtained, and weight assignment is performed based on the semantic annotation information to obtain semantic importance density features.

4. The method according to claim 1, characterized in that, The process of performing gradient calculation and centroid calculation on the comprehensive density field data to obtain the density gradient vector and density distribution centroid includes: The comprehensive density field data is subjected to three-dimensional smoothing filtering to obtain density gradient components in three orthogonal directions; The density gradient components in the three orthogonal directions are integrated to obtain the density gradient vector; The centroid location is determined by solving the density value distribution in the comprehensive density field data to obtain the density distribution centroid.

5. The method according to claim 1, characterized in that, The asymmetric segmentation of the space corresponding to the 3D scene data based on the spatial offset parameter to obtain an asymmetric octree spatial index includes: Obtain the variance of the voxel density of the current node to be segmented in the space corresponding to the three-dimensional scene data, wherein the current node to be segmented is the node space in the space corresponding to the three-dimensional scene data where the segmentation operation is being performed; If the variance satisfies the segmentation triggering condition, then the current node to be segmented is subjected to planar positioning and segmentation based on the spatial offset parameter to obtain asymmetric child nodes; The spatial volume of the asymmetric child nodes is normalized and recursively partitioned to obtain the asymmetric octree spatial index.

6. The method according to claim 1, characterized in that, The asymmetric segmentation of the space corresponding to the 3D scene data based on the spatial offset parameter to obtain an asymmetric octree spatial index includes: Based on the spatial offset parameters, the space corresponding to the three-dimensional scene data is initially segmented to obtain initial octree nodes; A sampling ray is emitted from a preset observation viewpoint to the space corresponding to the initial octree node, and an intersection test is performed on the sampling ray to obtain the visible leaf node; Based on the visible leaf nodes, the visual accessibility field is calculated, and the initial octree nodes are differentiated and merged according to the visual accessibility field to obtain the asymmetric octree spatial index.

7. The method according to claim 6, characterized in that, The step of differentially subdividing and merging the initial octree nodes based on the line-of-sight accessibility field to obtain an asymmetric octree spatial index includes: The visibility of leaf nodes is obtained by performing a ratio calculation on the number of visible leaf nodes. If the visibility of the leaf node meets the visibility condition and the average density of the corresponding region meets the density threshold condition, then the visible leaf node is further subdivided. If the visibility of the leaf node satisfies the invisible condition and the depth of the visible leaf node satisfies the depth condition, then the visible leaf nodes are merged to obtain an asymmetric octree spatial index.

8. The method according to claim 1, characterized in that, After performing tile conversion processing on the 3D scene data based on the asymmetric octree spatial index to obtain a 3D tile dataset, the method further includes: Acquire the current viewpoint state data, and perform view frustum range prediction processing based on the current viewpoint state data to obtain the predicted view frustum; Based on the predicted view frustum and the tile density data in the three-dimensional tile dataset, the target pre-fetched tile is obtained by filtering and prioritizing the data. The target prefetch tiles are dynamically adjusted and scheduled based on the current network bandwidth and rendering frame rate, so that the client can render the 3D scene data based on the target prefetch tiles.

9. A three-dimensional scene data processing device, characterized in that, The device includes: The data acquisition unit is used to acquire the three-dimensional scene data corresponding to the three-dimensional scene, and to perform multi-dimensional feature extraction on the three-dimensional scene data to obtain comprehensive density field data. The first data processing unit is used to perform gradient calculation and centroid calculation on the comprehensive density field data to obtain the density gradient vector and the density distribution centroid. The second data processing unit is used to determine a spatial offset parameter based on the density gradient vector and the centroid of the density distribution, wherein the spatial offset parameter is the amount of offset used to adjust the segmentation position from the spatial center. An index building unit is used to perform asymmetric segmentation processing on the space corresponding to the three-dimensional scene data based on the spatial offset parameter to obtain an asymmetric octree spatial index. The spatial volume of the child node in the asymmetric octree spatial index is inversely proportional to the comprehensive density of the region corresponding to the child node. The data generation unit is used to perform tile conversion processing on the three-dimensional scene data based on the asymmetric octree spatial index to obtain a three-dimensional tile dataset.

10. An electronic device, characterized in that, The electronic device includes a data acquisition unit, a memory, and a processor, wherein, The data acquisition device is used to collect 3D scene data corresponding to the 3D scene; The memory is used to store executable program code; The processor is configured to perform multi-dimensional feature extraction on the 3D scene data to obtain comprehensive density field data; perform gradient solving and centroid calculation on the comprehensive density field data to obtain density gradient vector and density distribution centroid; determine spatial offset parameters based on the density gradient vector and density distribution centroid; perform asymmetric segmentation on the space corresponding to the 3D scene data based on the spatial offset parameters to obtain an asymmetric octree spatial index; and perform tile conversion on the 3D scene data based on the asymmetric octree spatial index to obtain a 3D tile dataset.

Images