Point cloud model rendering optimization method and system based on cloud edge collaboration

Abstract

The invention discloses a point cloud model rendering optimization method and system based on cloud edge collaboration, and belongs to the field of computer graphic processing, and the method comprises the steps: carrying out the spatial partitioning of multi-stage point cloud data, generating a point cloud hierarchical structure which can be quickly accessed, building a user portrait model, and predicting the region of interest of the user portrait model; the multi-level LOD point cloud data structure is distributed to an edge computing node, the current network state is evaluated in real time, and if the local cache of the edge node cannot meet the rendering requirement, the edge node adaptively requests for cloud incremental data; the client receives the data blocks from the edge nodes and uploads real-time interaction data to the edge nodes; and the client interaction record and the edge node cache hit rate are transmitted back to the cloud end, and the cloud end performs iterative optimization on a point cloud data distribution strategy by using a reinforcement learning model. According to the method, priority loading and high-precision rendering of the region of interest are achieved by constructing the multi-level LOD point cloud data structure, the delay and jamming phenomena are effectively avoided, and the user interaction experience is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer graphics processing, more specifically, it relates to a point cloud model rendering optimization method and system based on cloud edge collaboration. BACKGROUND

[0002] Point cloud is a spatial data expression form based on discrete point set in three-dimensional space, and is widely used in three-dimensional surveying and mapping, building information modeling, digital twinning, autonomous driving, virtual reality and other fields. With the continuous improvement of the resolution of collection devices such as laser radar and three-dimensional scanners, the volume of point cloud data grows exponentially, often reaching hundreds of millions or even billions, bringing great challenges to storage, transmission and real-time rendering. The current point cloud rendering system mostly adopts a local rendering-based method, that is, all or part of the preprocessed point cloud data is downloaded to the client, and the rendering task is executed by the client device. However, this method has the following main technical bottlenecks: the data volume of the point cloud model is usually very large, and the terminal device is limited by network bandwidth, computing resources and memory capacity, making it difficult to load complete data at one time, resulting in rendering delay, lag and even crash. Current systems use LOD mechanism to improve rendering efficiency by pre-generating point cloud levels of different precision, but their switching strategies are mostly fixed rules, lacking the ability to dynamically schedule according to user behavior, viewing angle or area of interest, making it difficult to balance rendering quality and performance. Traditional point cloud rendering systems cannot recognize user behavior characteristics such as operation habits and browsing preferences, cannot predict area of interest and adaptively schedule resources, resulting in delayed loading of key areas and decreased user experience. Existing technologies mostly rely on central cloud servers to complete point cloud distribution and rendering decisions, failing to fully utilize the advantages of local computing and caching of edge nodes, resulting in large network transmission delay, high cloud pressure and inability to respond to user-side changes in real time. Cache and distribution strategies mostly use static rule configuration, which cannot be dynamically optimized according to environmental changes and user behavior, resulting in low resource utilization efficiency, slow response to hot areas and other problems.

[0003] In summary, the current point cloud rendering technology still has significant deficiencies in data scheduling intelligence, system response real-time and user experience adaptation. Therefore, it is urgent to propose a dynamic point cloud rendering optimization method that integrates cloud computing, edge intelligence and user behavior analysis to achieve efficient and adaptive three-dimensional point cloud visualization. SUMMARY

[0004] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] In view of the above or existing problems of the cloud edge collaboration based point cloud model rendering optimization method and system, the present application is proposed.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] The embodiment of the present application provides a cloud edge collaboration based point cloud model rendering optimization method, which comprises: spatially dividing multi-level point cloud data and indexing and organizing the same based on a KD tree to generate a point cloud hierarchical structure that can be quickly accessed; establishing a user portrait model according to the device performance, network bandwidth and interactive historical behavior of the user and predicting the region of interest of the user;

[0008] The multi-level LOD point cloud data structure is distributed to an edge computing node, the edge node dynamically loads and caches the corresponding ROI sub-block based on the user portrait, and the current network state is evaluated in real time; if the local cache of the edge node cannot meet the rendering requirement, the edge node adaptively requests incremental data from the cloud end according to the user perspective priority;

[0009] The client receives the data block from the edge node, combines the local GPU to perform point cloud projection, occlusion removal and dynamic LOD switching based on the perspective priority, uploads the real-time interactive data to the edge node, and the edge node updates the cache strategy and pre-fetches the required data for the next perspective according to the real-time interactive data;

[0010] The client interactive record and the edge node cache hit rate are fed back to the cloud end, the cloud end iteratively optimizes the point cloud data distribution strategy by using a reinforcement learning model, and the optimized scheduling model and weight are synchronized to the edge node.

[0011] As a preferred scheme of the cloud edge collaboration based point cloud model rendering optimization method, wherein: the multi-level point cloud data is spatially divided and indexed and organized based on a KD tree to generate a point cloud hierarchical structure that can be quickly accessed, which comprises:

[0012] The original point cloud data is subjected to noise removal and normal estimation to construct a basic point cloud model; the original point cloud data is subjected to multiple down-sampling by a voxel grid down-sampling method to obtain multiple levels of point cloud data; for each level of point cloud model, the entire point cloud space is divided into a plurality of cubic regions according to the spatial position thereof;

[0013] The user client perspective determines the current frustum, when traversing the KD tree, whether the minimum bounding box of each node intersects with the frustum is judged to remove the data block outside the field of view; for the data block within the field of view, a suitable LOD level is automatically selected according to the distance thereof from the user to realize the rendering strategy of far coarse and near fine; the point cloud model loading is performed in a coarse-to-fine manner, and the data block of a lower LOD level is preferentially loaded; when the user stays or zooms to a certain region, higher resolution blocks are gradually loaded for replacement.

[0014] As a preferred scheme of the cloud edge collaborative point cloud model rendering optimization method described in the application, wherein: a user portrait model is established according to the device performance, network bandwidth and interaction history behavior of the user, and the region of interest thereof is predicted, including:

[0015] The user portrait model U is constructed, which is represented as follows:

[0016] U=f(U device ,U behavior )∈R d

[0017] Wherein, f is the user portrait encoding function, R d is the user portrait feature space, U behavior is the user behavior vector, and U device is the device-network feature vector.

[0018] Based on the established user portrait model, the attention area of the user is predicted in combination with the current point cloud model structure and the user's view state, and the specific steps are as follows:

[0019] The entire point cloud space is pre-divided using a KD tree, and each sub-block represents a candidate interest area; the following scoring function is used to predict the interest degree S i of the user to each block:

[0020] S i =φ( U ,P i ,V)

[0021] Wherein, U is the user portrait, P i is the attribute feature of the point cloud i, V is the current user view parameter, and φ is the multi-modal interest degree scoring function.

[0022] All candidate regions are arranged in descending order of interest degree score S i , and the top K are selected as the key preloading targets; if the edge node has a cache capability, these regions are loaded in advance.

[0023] As a preferred scheme of the cloud edge collaborative point cloud model rendering optimization method described in the application, wherein: the multi-level LOD point cloud data structure is distributed to the edge computing node, the edge node dynamically loads and caches the corresponding ROI sub-block based on the user portrait, and the current network state is evaluated in real time, including:

[0024] For the identified ROI region, the edge node first searches the corresponding point cloud sub-block data in the local cache; if it hits, the sub-block is directly scheduled to the terminal for rendering; if it does not hit, an asynchronous request is initiated to the cloud to pull the sub-block data of the corresponding LOD level and perform local caching, and according to the user view angle movement trend and trajectory prediction result, the adjacent sub-blocks that are expected to be accessed soon are preloaded.

[0025] As a preferred scheme of the point cloud model rendering optimization method based on cloud-edge collaboration, if the local cache of the edge node cannot meet the rendering requirement, the edge node adaptively requests incremental data from the cloud according to the user view angle priority, including:

[0026] Before the system renders the point cloud, the point cloud space sub-blocks required for the current rendering are first traversed according to the above priority table; if a sub-block is already in the cache of the local edge node, it is marked as a “hit block”; if the sub-block is not in the cache or its LOD level is lower than the current requirement, it is marked as a “missing block”; all missing blocks are sorted according to the view angle priority to form a “candidate request list”;

[0027] When the edge node receives the point cloud incremental data returned by the cloud, the following operations are performed: store the sub-block data in the local cache and update the cache index state; if the sub-block is in the view angle of the current frame or the next frame, automatically trigger the rendering refresh; if the block is in the preloading state, wait for the user operation to trigger the rendering activation.

[0028] As a preferred scheme of the point cloud model rendering optimization method based on cloud-edge collaboration, the client receives the data block from the edge node, combines the local GPU to perform point cloud projection, occlusion removal, and dynamic LOD switching based on the view point priority, uploads the real-time interaction data to the edge node, and the edge node updates the cache strategy and pre-fetches the data required for the next view angle, including:

[0029] Convert the three-dimensional point cloud to the two-dimensional screen coordinates under the current camera view angle, and perform projection matrix transformation processing:

[0030] p screen =P·V·M· world

[0031] Where P, V, and M are the projection matrix, the view matrix, and the model matrix, respectively.

[0032] According to the view point distance, the view angle change rate, and the GPU load condition, the display resolution requirement of each space block is determined in real time, and the most suitable LOD level is dynamically selected for rendering; the client collects the following real-time user interaction data in each frame or each interaction period, and uploads it to the edge node in a lightweight structure.

[0033] As a preferred scheme of the cloud edge collaboration based point cloud model rendering optimization method, wherein: the client interaction record and the edge node cache hit rate are returned to the cloud, the cloud uses a reinforcement learning model to iteratively optimize the point cloud data distribution strategy, and the optimized scheduling model and weight are synchronized to the edge node, including:

[0034] The edge node continuously monitors the access request times and hit times of the locally cached point cloud data block, calculates the cache hit rate, counts the data access proportion and cache replacement frequency of different LOD levels, monitors the bandwidth utilization and data loading delay of the edge node, and uploads the cache state data and performance indicators to the cloud scheduling platform according to a preset period;

[0035] The cloud receives and aggregates the cache state data and client interaction logs from multiple edge nodes, constructs a joint system state vector, and fuses user behavior characteristics and edge node performance indicators; the reinforcement learning model generates a point cloud data distribution strategy action with the joint state vector as input.

[0036] A cloud edge collaboration based point cloud model rendering optimization system, comprising: a preprocessing module for spatially blocking multi-level point cloud data and indexing based on KD tree to generate a point cloud hierarchical structure that can be quickly accessed, establishing a user portrait model according to the device performance, network bandwidth and interaction history behavior of the user, and predicting the region of interest thereof;

[0037] A loading and caching module for distributing multi-level LOD point cloud data structure to edge computing nodes, the edge nodes dynamically loading and caching corresponding ROI sub-blocks based on the user portrait, and real-time evaluating the current network state, if the edge node local cache cannot meet the rendering requirements, the edge node adaptively requests the cloud for incremental data according to the user perspective priority;

[0038] A rendering scheduling module for the client to receive data blocks from the edge node, combine the local GPU for point cloud projection, occlusion culling and dynamic LOD switching based on the viewpoint priority, upload real-time interaction data to the edge node, and the edge node updates the cache strategy and pre-fetches the data required for the next perspective accordingly;

[0039] A feedback scheduling module for returning the client interaction record and the edge node cache hit rate to the cloud, the cloud using a reinforcement learning model to iteratively optimize the point cloud data distribution strategy, and the optimized scheduling model and weight are synchronized to the edge node.

[0040] A computing device, comprising:

[0041] At least one processor, memory and input output unit;

[0042] The memory is configured to store a computer program, and the processor is configured to invoke the computer program stored in the memory to execute the steps of the cloud-edge collaboration based point cloud model rendering optimization method.

[0043] A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the steps of the cloud-edge collaboration based point cloud model rendering optimization method.

[0044] The present application has the following beneficial effects: the present application realizes the preferential loading and high-precision rendering of the region of interest by constructing a multi-level LOD point cloud data structure and combining a user perspective dynamic loading mechanism, effectively avoids phenomena such as delay and lag, and improves user interaction experience. Relying on the edge node to undertake part of the data caching and preprocessing tasks, the terminal only needs to process a small amount of key data blocks and graphic projection tasks, adapts to low-performance devices, and realizes the operation of a 'light client'. Using user portrait modeling and interest region prediction mechanism, the system can intelligently predict the attention region according to the user historical behavior and perspective preference, dynamically adjust the data transmission and caching strategy, and enhance the system adaptability and user perception quality. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0046] Figure 1 The flowchart of the cloud-edge collaboration based point cloud model rendering optimization method provided by the embodiments of the present application.

[0047] Figure 2 The structure schematic diagram of the cloud-edge collaboration based point cloud model rendering optimization system provided by the embodiments of the present application.

[0048] Figure 3 The structure schematic diagram of a medium of the embodiments of the present application is schematically shown.

[0049] Figure 4 The structure schematic diagram of a computing device of the embodiments of the present application is schematically shown.

[0050] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0052] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be appreciated that the present application can be practiced in a variety of ways beyond the specific details set forth herein, having regard to the contents of the whole patent document, and that the present application can be practiced in different but but equivalent ways without departing from the scope of the present application. It is to be understood that other specific arrangements can be utilized and structural and functional modifications can be made without departing from the scope of the present application. At least some of the steps recited in the claims can be executed in different order than the order recited in the claims. All such possibilities are contemplated by the present application.

[0053] Secondly, the "one embodiment" or "embodiment" referred to herein means a specific feature, structure or characteristic that can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor does it mean an embodiment that is separate or alternative to other embodiments.

[0054] Embodiments

[0055] The following description refers to the accompanying drawings, which are meant to be exemplary and not limiting as to the scope of the present application. Figure 1 , Figure 1 The flowchart of the cloud-edge collaboration-based point cloud model rendering optimization method provided by an embodiment of the present application is shown. It should be noted that the embodiments of the present application can be applied to any applicable scenario.

[0056] Figure 1 The flowchart of the cloud-edge collaboration-based point cloud model rendering optimization method provided by an embodiment of the present application is shown. It should be noted that the embodiments of the present application can be applied to any applicable scenario.

[0057] S1: Spatially block the multi-level point cloud data, and index and organize based on the KD tree to generate a point cloud hierarchical structure that can be quickly accessed; establish a user portrait model according to the device performance, network bandwidth and interactive historical behavior of the user, and predict the region of interest of the user.

[0058] Preferably, the original point cloud data is subjected to noise removal and normal estimation to construct a basic point cloud model; the original point cloud data is subjected to multiple down-sampling through a voxel grid down-sampling method to obtain multiple levels of point cloud data; for each level of point cloud model, the entire point cloud space is divided into a plurality of cubic regions according to the spatial position thereof;

[0059] The user client determines the current frustum from the perspective, and when traversing the KD tree, determines whether the minimum bounding box of each node intersects with the frustum to remove the data blocks outside the field of view; for the data blocks within the field of view, a suitable LOD level is automatically selected according to the distance thereof from the user to realize the rendering strategy of far coarse and near fine; the point cloud model loading is performed in a coarse-to-fine manner, and the data blocks of lower LOD levels are preferentially loaded; when the user stays or zooms in on a certain region, higher resolution blocks are gradually loaded to replace them.

[0060] Preferably, a user portrait model U is constructed, which is represented as follows:

[0061] U = f(U device ,Ubehavior )∈R d

[0062] where f is a user profile encoding function, R d is a user profile feature space, U behavior is a user behavior vector, U device is a device-network feature vector;

[0063] Based on the established user profile model, combined with the current point cloud model structure and the user's perspective state, the attention area is predicted, and the specific steps are as follows:

[0064] The entire point cloud space is pre-divided using KD tree, and each sub-block represents a candidate interest area; the following scoring function is used to predict the interest degree S i of the user to each block:

[0065] S i = φ( U , P i , V)

[0066] where U is the user profile, P i is the attribute feature of point cloud i, V is the current user perspective parameter, and φ is a multi-modal interest degree scoring function;

[0067] All candidate areas are arranged in descending order of interest degree score S i , and the top K are selected as the key preloading targets; if the edge node has cache capability, these areas are loaded in advance.

[0068] Further, the original point cloud data is processed to remove noise, and statistical filtering, radius filtering, etc. can be used to remove isolated points and outliers. The normal vector of each point is estimated, for example, using K-neighbor fitting or PCA method, to provide basic attribute support for subsequent lighting and reconstruction tasks.

[0069] Based on the voxel grid method, the point cloud is down-sampled at multiple levels according to the preset voxel edge length, and point cloud levels with different precisions such as LOD0, LOD1, LOD2, etc. are constructed to form a pyramid data structure:

[0070] LOD k = Downsample(P orig , VoxelSize k )

[0071] where VoxelSize k is the voxel grid edge length of the k-level, VoxelSize0<VoxelSize1<…

[0072] <VoxelSizen ;

[0073] For each LOD level of point cloud data, the entire 3D space is divided into fixed-size cubic areas according to its spatial coordinate range, and the data points in each block are regarded as a logical subset; a multi-level KD tree index structure is constructed, and a node mapping is established for all sub-blocks. Each KD tree node stores the spatial bounding box information of the sub-block and points to the corresponding LOD data block;

[0074] When rendering on the client side, the camera's frustum is constructed based on the current viewing angle to determine the user's visible range. A top-down KD tree traversal strategy is used to perform the following judgment on each node: if the node's AABB does not intersect with the current frustum, the sub-block is eliminated; if it intersects, the sub-block is determined as a candidate loading block and added to the visible data queue. This step significantly reduces the loading of data in irrelevant areas and improves rendering efficiency.

[0075] For the data block entering the frustum, calculate its center distance d from the user's camera position i , based on the preset distance threshold and LOD level mapping table, dynamically select the appropriate LOD level to achieve a rendering strategy of coarse at far and fine at near;

[0076] The loading of all data blocks follows a coarse-to-fine priority strategy, with lower LODs being loaded first to quickly establish the scene framework. If the system detects that the user has stayed in a certain field of view for a long time (such as observation time > t seconds) or performs a zoom operation, the higher-resolution LODs of the corresponding sub-blocks will be loaded step by step, replacing the original coarse-grained data to achieve a seamless precision transition.

[0077] Furthermore, the following interestingness scoring function is used to score each sub-block. After calculating the scores for all sub-blocks, the system sorts them in descending order of scores and selects the first block:

[0078] in

[0079] If the edge node has caching capabilities, the data blocks corresponding to the above Top-K areas will be loaded or kept in the cache first; if the user is close to these areas, the system does not need to request data from the cloud, but directly retrieves it from the edge node cache to achieve the "nearest hit in the area of ​​interest" optimization; the system can perform reinforcement learning updates based on the prediction accuracy and user feedback to improve the accuracy of the next round of interest predictions.

[0080] S2: Distribute the multi-level LOD point cloud data structure to the edge computing node. The edge node dynamically loads the ROI sub-block corresponding to the cache based on the user portrait and evaluates the current network status in real time. If the local cache of the edge node cannot meet the rendering requirements, the edge node adaptively requests incremental data from the cloud based on the user's perspective priority.

[0081] Preferably, for the identified ROI region, the edge node first looks up the corresponding point cloud sub-block data in the local cache. If it hits, the sub-block is directly scheduled to the terminal rendering; if it does not hit, an asynchronous request is initiated to the cloud to pull the corresponding LOD level sub-block data and perform local caching, and according to the user view angle movement trend, trajectory prediction result, adjacent sub-blocks that are expected to be accessed soon are preloaded.

[0082] Preferably, the system first traverses the point cloud space sub-blocks required for current rendering according to the above priority table before rendering the point cloud; if a sub-block is already in the cache of the local edge node, it is marked as a “hit block”; if the sub-block is not in the cache or its LOD level is lower than the current requirement, it is marked as a “missing block”; all missing blocks are sorted according to the view angle priority to form a “candidate request list”;

[0083] When the edge node receives the point cloud incremental data returned by the cloud, the following operations are performed: store the sub-block data in the local cache and update the cache index state; if the sub-block is in the current frame or the next frame view angle, automatically trigger the rendering refresh; if the block is in the preloading state, wait for user operation to trigger the rendering activation.

[0084] Further, the system first identifies the region of interest ROI of the current frame based on the user's current view angle parameters (view cone direction, pitch angle, camera position) and user portrait model (including interaction behavior and historical preference); divides the point cloud space into a plurality of KD tree organized sub-blocks, performs intersection test on the view cone and the minimum bounding box of the sub-blocks, and preliminarily filters out the sub-block set within the field of view. Traverse all sub-blocks within the field of view, and query the local cache index table of the edge node in turn: if the sub-block number and its required LOD level hit the cache, mark it as a “hit block”; if the sub-block does not exist or the LOD level is lower than the current requirement, mark it as a “missing block”. Sort all “missing blocks” according to the following priority rules to form a candidate request list: sub-blocks with the closest distance in the current view angle are given priority; regions with high user interaction intensity are given priority (according to the user portrait); sub-blocks that are predicted to enter the view angle are given priority.

[0085] According to the user historical perspective moving track (time series of camera position and direction), the Kalman filter or GRU neural network is used for trend prediction; the future N frames of space regions that may enter the frustum are obtained, and the corresponding sub-blocks are marked as "pre-loaded blocks"; if the edge node resources allow, these blocks are pulled to the local cache in advance. For the "missing blocks" in the candidate request list, asynchronous requests are initiated to the cloud in batches according to the priority order; the cloud returns the corresponding LOD level incremental point cloud data block according to the request content. After the edge node receives the data from the cloud, the following processing is performed: the sub-block data is cached to the local cache area, and the index table is updated; if the block is in the current frame perspective, the client is immediately notified to perform graphic refresh rendering; if the block is in the "pre-loaded block" set, rendering is triggered after it enters the perspective; if the cache space is insufficient, the old block is eliminated according to the least recently used or perspective priority weight strategy.

[0086] S3: The client receives the data block from the edge node, combines the local GPU to perform point cloud projection, occlusion removal and dynamic LOD switching based on the viewpoint priority, uploads the real-time interaction data to the edge node, and the edge node updates the cache strategy and pre-fetches the data required for the next perspective according to the data.

[0087] Preferably, the three-dimensional point cloud is converted to a two-dimensional screen coordinate under the current camera perspective, and a projection matrix transformation process is performed:

[0088] p screen =P·V·M· world

[0089] P, V and M are respectively a projection matrix, a view matrix and a model matrix;

[0090] According to the viewpoint distance, the perspective change rate and the GPU load, the display resolution requirement of each space block is judged in real time, and the most suitable LOD level is selected for rendering; the client collects the following real-time user interaction data in each frame or each interaction period, and uploads the data to the edge node in a lightweight structure.

[0091] Further, the point cloud data is represented by a model point set X world in a world coordinate system, and each point needs to go through the following three-dimensional transformation process in turn, and is mapped to a two-dimensional image plane coordinate under the current camera perspective:

[0092] X screen =P·V·M·X world

[0093] M is a model transformation matrix, V is a view matrix, and P is a projection matrix;

[0094] The mapped point cloud is projected as two-dimensional pixel points in the screen for subsequent occlusion culling, color rendering and LOD selection.

[0095] To improve rendering efficiency and visual quality, the system designs a LOD level selection mechanism based on "GPU load + viewpoint distance + viewing angle change rate" multi-factor, the specific steps are as follows:

[0096] For each spatial block B i , calculate the Euclidean distance d i =||C i -C cam || of the center point C i to the current camera position C cam , record the last frame viewing angle vector v prev , the current frame is v curr , calculate the change rate of the included angle θ = arccos(v prev ·v curr ) / Δt; monitor the client GPU rendering queue length, frame rate jitter, memory usage and other indicators G i , extract the standardized load; for each sub-block B i , calculate the LOD level selection score:

[0097]

[0098] Where, α, β, γ are weight factors;

[0099] Finally, according to the score, each block is bound to its appropriate LOD level, avoiding performance bottleneck caused by full high-precision rendering.

[0100] S4: Return the client interaction record and edge node cache hit rate to the cloud, and use the reinforcement learning model to iteratively optimize the point cloud data distribution strategy. The optimized scheduling model and weights are synchronized to the edge node.

[0101] Preferably, the edge node continuously monitors the access request times and hit times of the locally cached point cloud data blocks, calculates the cache hit rate, and counts the data access proportion and cache replacement frequency of different LOD levels, monitors the bandwidth utilization and data loading delay of the edge node, and uploads the cache state data and performance indicators to the cloud scheduling platform according to the preset period;

[0102] The cloud receives and aggregates the cache state data and client interaction logs from multiple edge nodes, constructs a joint system state vector, and fuses user behavior features and edge node performance indicators; the reinforcement learning model generates point cloud data distribution strategy actions with the joint state vector as input.

[0103] Further, a deep reinforcement learning algorithm is selected to train the data distribution strategy. The model structure includes:

[0104] Input layer: state vector S t ; Hidden layer: LSTM+MLP structure for capturing temporal context; Output layer: action space A t , representing the sub-blocks and their LOD level selection for issuing / scheduling / preloading.

[0105] Maximize long-term cache hit rate and bandwidth utilization, while minimizing data loading delay and rendering stutter probability. The reward function is as follows:

[0106] R t = alpha * eta avg - beta * latency avg + gamma * preload success

[0107] Wherein, eta avg is the average hit rate under the current strategy, latency avg is the loading delay, preload success is the probability of the user entering the preloaded area, alpha, beta, gamma are weight hyperparameters;

[0108] The cloud packs and deploys the trained strategy model, and synchronizes its weight file and strategy configuration to each edge node; each edge node loads the strategy model using a local lightweight inference framework, and infers and schedules actions in real time according to the local state vector; after each round of strategy deployment, the client behavior and edge cache state are continuously collected and fed back to the cloud for the next round of training iteration of the reinforcement learning model.

[0109] After introducing the method of the exemplary embodiments of the present application, next, with reference to Figure 2 An exemplary embodiment of the cloud-edge collaborative point cloud model rendering optimization system of the present application is described, which includes:

[0110] A preprocessing module for spatially blocking multi-level point cloud data and indexing and organizing based on KD tree to generate a point cloud hierarchical structure that can be quickly accessed, establishing a user portrait model according to the user's device performance, network bandwidth and interactive historical behavior, and predicting the area of interest;

[0111] A loading and caching module for distributing multi-level LOD point cloud data structure to edge computing nodes, edge nodes dynamically loading and caching corresponding ROI sub-blocks based on user portrait, and real-time evaluating the current network state, if the edge node local cache cannot meet the rendering requirements, the edge node adaptively requests the cloud for incremental data according to the user's perspective priority;

[0112] The rendering scheduling module is used by the client to receive data blocks from edge nodes, perform point cloud projection, occlusion culling, and dynamic LOD switching based on viewpoint priority in combination with the local GPU, and upload real-time interactive data to the edge nodes. The edge nodes then update their cache strategies and pre-fetch the data required for the next viewpoint accordingly.

[0113] The feedback scheduling module is used to transmit client interaction records and edge node cache hit rates back to the cloud. The cloud uses a reinforcement learning model to iteratively optimize the point cloud data distribution strategy and synchronize the optimized scheduling model and weights to the edge nodes.

[0114] After introducing the method and system of the exemplary embodiment of the present invention, the following Figure 3 For a description of a computer-readable storage medium according to an exemplary embodiment of the present invention, please refer to Figure 3 , the computer-readable storage medium shown is a CD 30, on which a computer program (i.e., a program product) is stored. When the computer program is executed by the processor, the steps described in the above method implementation are implemented, for example, multi-level point cloud data is spatially divided into blocks, and indexed and organized based on the KD tree to generate a point cloud hierarchy structure that can be quickly accessed, and a user portrait model is established based on the user's device performance, network bandwidth, and historical interaction behavior, and the region of interest is predicted; the multi-level LOD point cloud data structure is distributed to the edge computing node, and the edge node dynamically loads the ROI sub-block corresponding to the cache based on the user portrait, and evaluates the current network status in real time. If The local cache of the edge node cannot meet the rendering requirements, and the edge node adaptively requests incremental data from the cloud based on the user's perspective priority; the client receives data blocks from the edge node, combines the local GPU to perform point cloud projection, occlusion culling and dynamic LOD switching based on viewpoint priority, and uploads real-time interaction data to the edge node. The edge node updates the cache strategy accordingly and pre-fetches the data required for the next perspective; the client interaction records and the edge node cache hit rate are transmitted back to the cloud, and the cloud uses the reinforcement learning model to iteratively optimize the point cloud data distribution strategy, and synchronizes the optimized scheduling model and weights to the edge node; the specific implementation methods of each step are not repeated here.

[0115] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0116] After introducing the method, apparatus and medium of the exemplary embodiment of the present invention, the following is a reference toFigure 4 A computing device optimized for rendering point cloud models based on cloud-edge collaboration according to an exemplary embodiment of the present invention.

[0117] Figure 4 A block diagram is shown of an exemplary computing device 40 , which may be a computer system or server, suitable for implementing embodiments of the present invention. Figure 4 The computing device 40 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0118] like Figure 4 As shown, the components of computing device 40 may include, but are not limited to, one or more processors or processing units 401 , a system memory 402 , and a bus 403 connecting various system components (including system memory 402 and processing unit 401 ).

[0119] The computing device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 40, including volatile and non-volatile media, removable and non-removable media.

[0120] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 4021 and / or cache memory 4022. Computing device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, ROM 4023 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 is not shown in the , usually referred to as "hard drive"). Although not in Figure 4 As shown in FIG403 , a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) can be provided. In these cases, each drive can be connected to bus 403 via one or more data media interfaces. System memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0121] A program / utility 4025 having a set (at least one) of program modules 4024 may be stored, for example, in system memory 402. Such program modules 4024 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 4024 generally implement the functions and / or methods of the embodiments described herein.

[0122] The computing device 40 can also communicate with one or more external devices 404 such as a keyboard, a pointing device, a display, etc. through an input / output (I / O) interface 405. Further, the computing device 40 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet, through a network adapter 406. As Figure 4 illustrated, the network adapter 406 communicates with the other modules of the computing device 40, such as the processing unit 401, etc., through the bus 403. It should be appreciated that the computing device 40 can be connected to other types of computing devices, such as a workstation, a server, a blade computer, etc., through the network adapter 406. ​ It should be appreciated that other hardware and / or software modules can be used in connection with the computing device 40, although not shown in FIG. 4.

[0123] The processing unit 401 performs various function applications and data processing by running programs stored in the system memory 402, such as spatially partitioning and indexing multi-level point cloud data based on KD tree to generate a fast-access point cloud hierarchy, establishing a user portrait model and predicting a region of interest of the user according to device performance, network bandwidth and interaction history of the user, distributing multi-level LOD point cloud data structure to an edge computing node, dynamically loading and caching a corresponding ROI sub-block based on the user portrait by the edge node, and real-time evaluating a current network state, and if the local cache of the edge node cannot meet the rendering requirement, the edge node adaptively requests incremental data from the cloud according to the user perspective priority; receiving data blocks from the edge node by the client, combining with the local GPU to perform point cloud projection, occlusion culling and dynamic LOD switching based on the viewpoint priority, uploading real-time interaction data to the edge node, updating the cache strategy and pre-fetching data required for the next perspective by the edge node according to the real-time interaction data, and returning the client interaction record and the cache hit rate of the edge node to the cloud, and iteratively optimizing the point cloud data distribution strategy by the cloud using a reinforcement learning model, and synchronizing the optimized scheduling model and weights to the edge node.

[0124] The specific implementation of each step will not be repeated here. It should be noted that although several units / modules or sub-units / sub-modules of the cloud-edge collaborative point cloud model rendering optimization system are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules.

[0125] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0126] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. The described device embodiments are merely schematic, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electric, mechanical or other forms.

[0127] The units described as separated components can or can not be physically separated, and components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0128] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit.

[0129] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium of a processor. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

[0130] Finally, it should be noted that the above-described embodiments are merely intended for describing the technical solutions of the present application, but not intended to limit the present application, and the protection scope of the present application is not limited to this. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any modification or easy-to-think change, or equivalent replacement of part of the technical features of the technical solutions recorded in the foregoing embodiments can be made within the technical range disclosed by the present application by any person skilled in the art. The modification, change or replacement does not make the corresponding technical solution deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0131] In addition, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or divided into multiple steps.

Claims

1. A point cloud model rendering optimization method based on cloud-edge collaboration, characterized by: include: Multi-level point cloud data is spatially partitioned and indexed based on a KD tree to generate a quickly accessible point cloud hierarchy. Based on the user's device performance, network bandwidth, and interaction history, a user profile model is established to predict their areas of interest. The multi-level LOD point cloud data structure is distributed to the edge computing node. The edge node dynamically loads the ROI sub-block corresponding to the cache based on the user profile and evaluates the current network status in real time. If the local cache of the edge node cannot meet the rendering requirements, the edge node adaptively requests incremental data from the cloud based on the user's perspective priority. The client receives data blocks from the edge node, combines it with the local GPU to perform point cloud projection, occlusion culling, and dynamic LOD switching based on viewpoint priority, and uploads real-time interactive data to the edge node. The edge node updates the cache strategy accordingly and pre-fetches the data required for the next viewpoint. The client interaction records and edge node cache hit rates are transmitted back to the cloud. The cloud uses a reinforcement learning model to iteratively optimize the point cloud data distribution strategy and synchronizes the optimized scheduling model and weights to the edge nodes.

2. The point cloud model rendering optimization method based on cloud-edge collaboration according to claim 1, characterized in that: The multi-level point cloud data is spatially partitioned and indexed based on a KD tree to generate a point cloud hierarchical structure that can be quickly accessed, including: The original point cloud data is subjected to noise removal and normal estimation to construct a basic point cloud model. The original point cloud data is downsampled multiple times using the voxel grid downsampling method to obtain multiple levels of point cloud data. For each level of the point cloud model, the entire point cloud space is divided into several cubic areas according to its spatial position. The user client's perspective determines the current viewing cone. When traversing the KD tree, the data blocks outside the field of view are eliminated by judging whether the minimum bounding box of each node intersects with the viewing cone. For data blocks within the field of view, the appropriate LOD level is automatically selected based on their distance from the user to implement a rendering strategy of coarse at far and fine at near. Point cloud models are loaded in a coarse-to-fine manner, with priority given to loading data blocks at lower LOD levels. When the user stays in or zooms to a certain area, higher-resolution blocks are gradually loaded for replacement.

3. The point cloud model rendering optimization method based on cloud-edge collaboration according to claim 1, characterized in that: The user profile model is established based on the user's device performance, network bandwidth, and historical interaction behavior, and the user's area of ​​interest is predicted, including: Construct a user portrait model U, which is expressed as follows: U=f(U device ,IN behavior )∈R d Among them, f is the user portrait encoding function, R d is the user portrait feature space, U behavior is the user behavior vector, U device is the device-network feature vector; Based on the established user portrait model, combined with the current point cloud model structure and the user's viewing angle, the focus area is predicted. The specific steps are as follows: Use the KD tree to pre-divide the entire point cloud space, and each sub-block represents a candidate area of ​​interest; use the following scoring function to predict the user's interest in each block S i : S i =φ( U ,P i ,V) Among them, U is the user portrait, P i is the attribute feature of point cloud block i, V is the current user's viewing angle parameter, and φ is the multimodal interest scoring function; Score all candidate regions according to their interest score i Arrange in descending order and select the top K as the key preloading targets; if the edge node has caching capabilities, load these areas in advance.

4. The point cloud model rendering optimization method based on cloud-edge collaboration according to claim 1, characterized in that: The multi-level LOD point cloud data structure is distributed to the edge computing node. The edge node dynamically loads the ROI sub-block corresponding to the cache based on the user profile and evaluates the current network status in real time, including: For the identified ROI area, the edge node first searches for the corresponding point cloud sub-block data in the local cache; if a hit is found, the sub-block is directly dispatched to the terminal for rendering; if a hit is not found, an asynchronous request is initiated to the cloud to pull the sub-block data of the corresponding LOD level and cache it locally. Based on the user's perspective movement trend and trajectory prediction results, the adjacent sub-blocks that are expected to be accessed are preloaded.

5. The point cloud model rendering optimization method based on cloud-edge collaboration according to claim 1, characterized in that: If the local cache of the edge node cannot meet the rendering requirements, the edge node adaptively requests incremental data from the cloud based on the user's perspective priority, including: Before rendering the point cloud, the system first traverses the point cloud spatial sub-blocks required for the current rendering according to the above priority table; if a sub-block is already in the cache of the local edge node, it is marked as a "hit block"; if the sub-block is not in the cache, or its LOD level is lower than the current requirement, it is marked as a "missing block"; all missing blocks are sorted by view priority to form a "candidate request list"; When the edge node receives the incremental point cloud data returned by the cloud, it performs the following operations: stores the sub-block data in the local cache and updates the cache index status; if the sub-block is in the perspective of the current frame or the next frame, it automatically triggers a rendering refresh; if the block is in the preloaded state, it waits for the user to trigger the rendering activation.

6. The point cloud model rendering optimization method based on cloud-edge collaboration according to claim 1, characterized in that: The client receives data blocks from the edge node, performs point cloud projection, occlusion culling, and dynamic LOD switching based on viewpoint priority in conjunction with the local GPU, and uploads real-time interactive data to the edge node. The edge node updates the cache strategy and pre-fetches the data required for the next viewpoint accordingly, including: Convert the 3D point cloud to the 2D screen coordinates under the current camera's perspective and perform the projection matrix transformation: p screen =P·V·M· world Among them, P, V, and M are projection matrix, view matrix, and model matrix respectively; Based on the viewpoint distance, viewing angle change rate and GPU load, the display resolution requirements of each spatial block are determined in real time, and the most appropriate LOD level is dynamically selected for rendering. In each frame or each interaction cycle, the client collects the following real-time user interaction data and uploads it to the edge node in a lightweight structure.

7. The point cloud model rendering optimization method based on cloud-edge collaboration according to claim 1, characterized in that: The client interaction records and edge node cache hit rates are transmitted back to the cloud. The cloud uses a reinforcement learning model to iteratively optimize the point cloud data distribution strategy and synchronizes the optimized scheduling model and weights to the edge node, including: The edge node continuously monitors the number of access requests and hits for the locally cached point cloud data blocks, calculates the cache hit rate, counts the data access ratios and cache replacement frequencies at different LOD levels, monitors the bandwidth utilization and data loading latency of the edge node, and uploads cache status data and performance indicators to the cloud scheduling platform according to a preset cycle. The cloud receives and aggregates cache status data and client interaction logs from multiple edge nodes, constructs a joint system state vector, and integrates user behavior characteristics with edge node performance indicators; using the joint state vector as input, the reinforcement learning model generates point cloud data distribution strategy actions.

8. A point cloud model rendering optimization system based on cloud-edge collaboration, characterized in that: include: The preprocessing module is used to spatially partition multi-level point cloud data and organize it into indexes based on a KD tree, generating a quickly accessible point cloud hierarchy. It also builds a user profile model based on the user's device performance, network bandwidth, and interaction history, and predicts their areas of interest. The loading and caching module is used to distribute the multi-level LOD point cloud data structure to the edge computing node. The edge node dynamically loads the ROI sub-block corresponding to the cache based on the user profile and evaluates the current network status in real time. If the local cache of the edge node cannot meet the rendering requirements, the edge node adaptively requests incremental data from the cloud based on the user's perspective priority. The rendering scheduling module is used by the client to receive data blocks from edge nodes, perform point cloud projection, occlusion culling, and dynamic LOD switching based on viewpoint priority in combination with the local GPU, and upload real-time interactive data to the edge nodes. The edge nodes then update their cache strategies and pre-fetch the data required for the next viewpoint accordingly. The feedback scheduling module is used to transmit client interaction records and edge node cache hit rates back to the cloud. The cloud uses a reinforcement learning model to iteratively optimize the point cloud data distribution strategy and synchronize the optimized scheduling model and weights to the edge nodes.

9. A computing device, comprising: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the steps of the point cloud model rendering optimization method based on cloud-edge collaboration as described in any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the steps of the point cloud model rendering optimization method based on cloud-edge collaboration as claimed in any one of claims 1 to 7.

Images