A point cloud data real-time hierarchical rendering method and system

Abstract

The application provides a point cloud data real-time hierarchical rendering method and system, relates to the technical field of data processing, and the method comprises the following steps: in response to a virtual viewpoint motion state change, acquiring a current viewpoint state and motion parameters of the virtual viewpoint; generating a predicted viewpoint state sequence of the virtual viewpoint within a preset time window based on the motion parameters; constructing a space-time visible envelope based on the predicted viewpoint state sequence, the space-time visible envelope being obtained by combining a plurality of local view sub-bodies distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local view sub-bodies changing with a steering intensity index of a corresponding predicted time; using the space-time visible envelope as a spatial index query condition, determining a target point cloud data block intersecting the space-time visible envelope in a hierarchical point cloud index structure, and performing hierarchical rendering. The application improves data loading hit rate, reduces invalid prefetching and resource occupation, and improves picture continuity and frame rate stability in a fast nonlinear roaming scene.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method and system for real-time hierarchical rendering of point cloud data. Background Technology

[0002] Point cloud data, as a crucial output format for 3D sensing technologies such as laser scanning and photogrammetry, has been widely used for 3D visualization and interactive analysis in scenarios such as smart cities, digital twins, and engineering construction and maintenance. With the increasing demand for immersive 3D interaction, the construction and content consumption of 3D spaces for virtual reality, augmented reality, and metaverse-like applications are also increasingly employing point clouds or point cloud reconstruction models as carriers for highly realistic scene representation. In these applications, users typically roam continuously in large-scale 3D scenes from a first-person or free-viewpoint perspective, placing higher demands on the frame rate stability and image continuity of real-time rendering. To support online browsing and editing of massive point clouds, existing systems typically employ block-based, layered details, and hierarchical spatial indexes, such as octrees and tile indexes, to organize point cloud data. At the rendering end, rules such as view frustum clipping, distance thresholds, or screen errors are combined to load and render data blocks related to the current viewpoint on demand.

[0003] However, during dynamic roaming in large-scale point cloud scenes, the user's viewpoint often exhibits nonlinear motion characteristics such as rapid acceleration and deceleration, sharp turns, frequent zooming, or sudden changes in path curvature. Simply constructing a static view domain based on the current frame's viewpoint state or using simple prediction strategies such as uniform linear extrapolation can easily lead to a mismatch between the predicted region and the actual view domain changes. On the one hand, if data blocks fail to cover the area about to enter the view domain in advance, it will cause loading lag, gaps in the image, or abrupt filling of details, affecting the continuity of interaction. On the other hand, blindly expanding the prefetch range to avoid loading lag will introduce a large number of data blocks unrelated to the future view domain for loading and decoding, resulting in redundant occupation of bandwidth, video memory, and computing resources, thereby exacerbating frame jitter.

[0004] The core contradiction thus arises from how to resolve the conflict between data loading delay and resource loading redundancy caused by the mismatch between the predicted region and the actual field of view during dynamic roaming of large-scale point cloud scenes, especially when the viewpoint undergoes rapid nonlinear motion. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a method and system for real-time hierarchical rendering of point cloud data.

[0006] Firstly, this application provides a method for real-time hierarchical rendering of point cloud data, including: In response to changes in the motion state of the virtual viewpoint, the current viewpoint state of the virtual viewpoint and motion parameters used to characterize the motion trend of the virtual viewpoint are obtained. Based on the motion parameters, a predicted viewpoint state sequence of the virtual viewpoint is generated within a preset time window; A spatiotemporal visual envelope is constructed based on the predicted viewpoint state sequence, wherein the spatiotemporal visual envelope is obtained by combining multiple local viewpoint sub-volumes distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local viewpoint sub-volume changes with the turning intensity index at the corresponding prediction time; wherein the turning intensity index is determined based on the rate of change of line of sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence. Using the spatiotemporal visual envelope as a spatial index query condition, the target point cloud data block intersecting with the spatiotemporal visual envelope is determined in the hierarchical point cloud index structure; Load and perform hierarchical rendering on the target point cloud data block.

[0007] Optionally, generating the predicted viewpoint state sequence of the virtual viewpoint within a preset time window includes: Obtain a sparse geometric proxy model to characterize the reachable boundaries of the current scene; Based on the current viewpoint state, a forward prediction direction is determined, and a forward safety margin is calculated in the sparse geometry proxy model along the forward prediction direction. The forward safety margin characterizes the reachability gap of the virtual viewpoint to the nearest geometric obstacle along the forward prediction direction. Based on the forward safety margin, boundary constraints are applied to the predicted motion increment within the preset time window to generate a predicted position sequence that does not cross the geometric obstacle, and the predicted viewpoint state sequence is determined based on the predicted position sequence.

[0008] Optionally, applying boundary constraints includes: The collision distance index is determined based on the forward safety margin and the motion parameters. In response to the collision distance index being less than a preset safety threshold, a constraint correction is triggered on the predicted motion increment, and when the collision distance index is greater than or equal to the preset safety threshold, the inertial predicted motion increment based on the motion parameters is maintained.

[0009] Optionally, the acquisition of the sparse geometric proxy model includes at least one of the following: A depth proxy generated from the depth buffer of the previous rendered frame; Low-resolution elevation grid generated from GIS terrain data; Simplified collision enclosure generated from the shell of BIM components; A set of coarse-level bounding volumes generated from a point cloud hierarchical index structure.

[0010] Optionally, the constraint correction includes: Determine the local surface normal corresponding to the nearest geometric obstacle; The environmental repulsion correction amount is determined based on the local surface normal, and the environmental repulsion correction amount is superimposed on the inertial predicted motion increment; The predicted motion increment after superposition correction is iteratively updated with discrete time steps, and a non-crossing constraint is applied to the predicted position at each discrete time step to generate the predicted position sequence. The determination of the local surface normal includes: when the sparse geometric proxy model includes a depth proxy, determining the local surface normal based on the depth change rate of the depth proxy; when the sparse geometric proxy model includes a simplified collision bounding body, determining the local surface normal based on the geometric patch normal of the simplified collision bounding body.

[0011] Optionally, the local view subvolume is a kissing cone; the construction of the spatiotemporal visual envelope based on the predicted viewpoint state sequence includes: For each predicted viewpoint state in the predicted viewpoint state sequence, a corresponding cutting cone is constructed with the corresponding predicted position as the cone apex and the corresponding predicted line of sight as the cone axis. At least one geometric parameter of the cutting cone is determined based on the steering strength index at the corresponding prediction time. The spatiotemporal visual envelope is obtained by combining multiple kissing cones constructed along the predicted viewpoint state sequence.

[0012] Optionally, determining at least one geometric parameter of the cutting cone based on the steering strength index includes: The angle adjustment amount of the cutting cone is determined based on the steering strength index; Obtain scene openness parameters that characterize the openness of the visible corridor at the predicted location, and determine the upper limit of the angle of the kissing cone based on the scene openness parameters, wherein the scene openness parameters are calculated by a geometric proxy used to characterize the terrain elevation surface, building shell and / or indoor passage structure, and used to characterize the reachable gap width and / or openness on both sides along the cone axis direction. The final angle of the cutting cone is determined based on the angle adjustment amount and the upper limit of the angle.

[0013] Optionally, the step of using the spatiotemporal visual envelope as a spatial index query condition to determine the target point cloud data block intersecting with the spatiotemporal visual envelope in the hierarchical point cloud index structure includes: Starting from the root node of the hierarchical point cloud index structure, a hierarchical traversal is performed. For the current index node, an intersection determination is performed based on the node bounding volume corresponding to the index node and the spatiotemporal visual envelope. In response to the intersection determination being non-intersection, the sub-level corresponding to the current index node is pruned; In response to the intersection determination being an intersection, based on the barrier geometry proxy used to characterize the building shell and / or terrain surface, a barrier determination is performed on at least two predicted viewpoint states corresponding to the predicted viewpoint state sequence, and a stable barrier certificate is determined based on the barrier determination result. In response to the establishment of the stability barrier certificate, prune the sub-level corresponding to the current index node; and In response to the failure of the stable isolation certificate, the intersection determination and stable isolation certificate determination are repeatedly performed on the child nodes corresponding to the current index node until the leaf node is traversed, so as to determine the data block corresponding to the leaf node that intersects with the spatiotemporal visual envelope and has not been pruned by the stable isolation certificate as the target point cloud data block.

[0014] Optionally, determining the stable barrier certificate includes: For at least K consecutive predicted viewpoint states in the predicted viewpoint state sequence, a line of sight is constructed pointing to the node bounding body, starting from the corresponding predicted position. The line of sight includes at least two types of directions, namely, the direction pointing to the center point of the node bounding body and the direction pointing to the preset extreme point of the node bounding body. Based on the blocking geometry agent, calculate the first intersection distance between the line of sight and the blocking geometry agent, and calculate the second intersection distance between the line of sight and the node bounding volume; In response to each of the at least K consecutive predicted viewpoint states satisfying that the first intersection distance is less than the second intersection distance, and the blocking object identifier corresponding to the first intersection remains consistent or falls within a preset consistency threshold range in the at least K consecutive predicted viewpoint states, the stable blocking certificate is determined to be established; wherein, the blocking object identifier is used to characterize at least one of the terrain grid cell identifier, building shell patch identifier, and / or floor slab patch identifier in the blocking geometry agent.

[0015] Secondly, this application provides a real-time hierarchical rendering system for point cloud data, comprising: The acquisition module is used to acquire the current viewpoint state of the virtual viewpoint and motion parameters used to characterize the motion trend of the virtual viewpoint in response to changes in the motion state of the virtual viewpoint. The first processing module is used to generate a predicted viewpoint state sequence of the virtual viewpoint within a preset time window based on the motion parameters. The second processing module is used to construct a spatiotemporal visual envelope based on the predicted viewpoint state sequence, wherein the spatiotemporal visual envelope is obtained by combining multiple local viewpoint sub-volumes distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local viewpoint sub-volume changes with the turning intensity index at the corresponding prediction time; wherein the turning intensity index is determined based on the rate of change of the line of sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence. The rendering module is used to use the spatiotemporal visual envelope as a spatial index query condition to determine the target point cloud data block that intersects with the spatiotemporal visual envelope in the hierarchical point cloud index structure; load and perform hierarchical rendering on the target point cloud data block.

[0016] Compared with existing technologies, the real-time hierarchical rendering method for point cloud data proposed in this application is geared towards the realistic interactive features of dynamic roaming. It introduces the viewpoint motion trend into the data block selection and loading boundary construction process. By generating a predicted viewpoint state sequence within a preset time window and constructing a spatiotemporal visual envelope covering the time dimension, the data block selection is transformed from a single-frame static view domain to a holistic coverage of short-term future view domain changes. Simultaneously, the geometric parameters of the local view domain sub-volumes of the envelope adaptively adjust with the steering intensity, enabling the system to better meet actual visual requirements in nonlinear maneuvering scenarios such as sharp turns and speed changes. This reduces missing data about to enter the view domain and suppresses invalid prefetching, thereby improving the loading hit rate, reducing redundant loading and decoding overhead under limited resource constraints, improving frame rate stability and image continuity, and enhancing the real-time interactive experience and engineering usability of massive point cloud scenes. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a real-time hierarchical rendering method for point cloud data provided in this application embodiment; Figure 2 A flowchart illustrating a method for generating a predicted viewpoint state sequence for virtual viewpoints, provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for constructing a spatiotemporal visual envelope, provided in an embodiment of this application; Figure 4 This is a schematic diagram of a real-time hierarchical rendering system for point cloud data provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0019] See Figure 1The diagram shown is a flowchart of a real-time hierarchical rendering method for point cloud data provided in an embodiment of this application, including steps S101 to S104, wherein: S101: In response to the change in the motion state of the virtual viewpoint, obtain the current viewpoint state of the virtual viewpoint and motion parameters used to characterize the motion trend of the virtual viewpoint; S102: Based on the motion parameters, generate a predicted viewpoint state sequence of the virtual viewpoint within a preset time window; S103: Construct a spatiotemporal visual envelope based on the predicted viewpoint state sequence, wherein the spatiotemporal visual envelope is obtained by combining multiple local viewpoint sub-volumes distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local viewpoint sub-volume changes with the turning intensity index at the corresponding prediction time; wherein the turning intensity index is determined based on the rate of change of the line of sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence; S104: Using the spatiotemporal visual envelope as a spatial index query condition, determine the target point cloud data block that intersects with the spatiotemporal visual envelope in the hierarchical point cloud index structure; load and perform hierarchical rendering on the target point cloud data block.

[0020] Regarding the above S101: In one implementation, a virtual viewpoint is used to represent the user's observation position and direction in a point cloud 3D scene. Changes in its motion state are typically triggered by interactive input, automatic roaming scripts, or changes in the posture of external devices. The system can periodically sample the virtual viewpoint in the main rendering loop or interactive event callbacks. For example, it can read the pose information of the virtual viewpoint once at a frame period corresponding to the display refresh rate (e.g., 16.7ms) or a fixed sampling period (e.g., 10ms to 50ms), and compare two adjacent sampling results to determine whether a change in motion state has occurred.

[0021] A change in motion state can be understood as a change in at least one of the following: the position of the virtual viewpoint, the direction of the line of sight, the attitude angle, or the field of view parameters, exceeding a preset change threshold, or a valid operation event that causes the viewpoint to be updated occurs on the input device side.

[0022] In one implementation, the current viewpoint state can be expressed using structured data objects, such as stored as a "ViewState" structure or a dictionary object, including: the viewpoint position in the world coordinate system (such as three-dimensional vectors x, y, z), the line of sight direction (such as a unit direction vector or an orientation characterized by yaw / pitch / roll), and optional field of view parameters, such as the field of view angle (FOV), the near and far clipping plane distance, and the projection type identifier.

[0023] The world coordinate system can be a geographic coordinate system or an engineering coordinate system commonly used in GIS scenarios, such as WGS-84 geocentric coordinate system, local east-north-sky coordinate system, or local coordinate system of BIM model. When there is a GIS and BIM integration in the scenario, the system can use a unified world coordinate system to store the current viewpoint status internally, and complete the conversion between input device coordinates and world coordinates through the coordinate transformation module to ensure the consistency of subsequent processing.

[0024] In one embodiment, motion parameters characterizing the motion trend of the virtual viewpoint can be derived from the current viewpoint state of adjacent samples and stored in the form of a "MotionParams" structure. The motion parameters include at least one of linear velocity, linear acceleration, and angular velocity. Linear velocity can be used to characterize the displacement trend of the viewpoint position per unit time, angular velocity can be used to characterize the rotation trend of the line of sight or attitude angle, and linear acceleration can be used to characterize the rate of velocity change to reflect behaviors such as sudden stops, sudden starts, or rapid speed changes. The above motion parameters can be obtained using a differential method, for example, by recording the viewpoint position and line of sight from the two or more most recent samples, converting the displacement difference and direction change difference under a fixed sampling period to obtain the corresponding trend quantity, and limiting or smoothing the trend quantity to avoid misjudgments caused by input jitter.

[0025] In one implementation, the determination of motion state changes can employ a threshold triggering mechanism. The threshold setting is related to the scene scale, interactive response requirements, and the latency characteristics of rendering and data loading. For example, in a city-level point cloud GIS scene, where the viewpoint translation speed varies significantly and the scene scale ranges from meters to kilometers, the position change threshold can be set to the order of 0.1m to 1m. In indoor BIM fusion roaming, where viewpoint movement is more nuanced and the passageway structure is sensitive to visual changes, the position change threshold can be set to the order of 0.01m to 0.1m. The orientation change threshold can be correlated with the frame rate and common interactive sensitivities. For instance, when the system runs at 60fps, the single-frame yaw or pitch change threshold can be set to 0.5° to 2° to distinguish between micro-jitter and effective steering.

[0026] As an example, it can be set that when the viewpoint position change between two adjacent samples exceeds 0.2m, or the line-of-sight change exceeds 1°, or the input device generates a valid scroll wheel zoom / drag event, the virtual viewpoint is determined to have changed motion and the data acquisition process in this step is initiated. The above thresholds can also be adaptively adjusted according to the device type. For example, the posture noise characteristics of a VR headset differ from those of mouse input, so the direction change threshold can be increased or decreased accordingly.

[0027] For example, taking a web-based 3D fusion rendering platform, the system can implement rendering based on WebGL or WebGPU. On the browser side, the camera controller is driven to update the virtual viewpoint via mouse, keyboard, touchpad, or gamepad. Before rendering each frame, the camera controller outputs a current viewpoint state (position, orientation, FOV, etc.) and writes it to a shared state cache. Motion parameters are calculated by the controller or rendering thread based on the state differences between adjacent frames and written to a motion parameter cache. When the system detects a continuous change in the viewing direction caused by rapid dragging by the user, or a continuous change in position caused by pressing a forward key, this step is triggered. The system reads the current viewpoint state and motion parameters to form a sampling record of a motion state change event. This sampling record can be stored in the form of log entries or memory queue elements, such as recording timestamps, ViewState, MotionParams, input event type identifiers, etc.

[0028] In another implementation, if the platform is a native desktop or mobile application, the rendering engine can be OpenGL, Vulkan, or DirectX, and the viewpoint state can be directly provided by the engine's camera object; if there are external sensor inputs such as IMU, gyroscope, or GNSS, the sensor output can be used as a supplementary source of motion trend parameters, for example, the output of a mobile gyroscope can be used to improve the stability of angular velocity estimation.

[0029] Regarding S102 above: Based on the current viewpoint state and motion parameters obtained in step S101, the system performs forward prediction of the future state of the virtual viewpoint within a preset time window, thereby generating a predicted viewpoint state sequence to characterize the short-term future roaming trend. The predicted viewpoint state sequence can be understood as providing multiple sets of predicted viewpoint states over a future period, starting from the current moment and following a preset prediction step size. Each set of predicted viewpoint states includes the predicted position and predicted line of sight, and may include fields such as a prediction time identifier, prediction attitude, and prediction field of view parameters, depending on implementation requirements.

[0030] In one implementation, a preset time window is used to limit the forward range of the prediction. Its setting is usually related to the end-to-end resource scheduling latency, data acquisition latency, and interaction continuity requirements of the system. For example, in a massive point cloud online browsing scenario, from determining that a certain area of ​​data is needed to the data block being available for rendering, it may involve network transmission, decoding and decompression, and data uploading from CPU to GPU. To improve the practical value of the prediction, the preset time window can be set to cover the magnitude of the above-mentioned comprehensive latency, so that the prediction results can match the actual processing link "ahead of time".

[0031] As an example, in a desktop local data loading scenario, the preset time window can be set to 0.2s to 0.8s; in a web-based or remote data source scenario, considering network jitter and download latency, the preset time window can be set to 0.5s to 2s. The preset time window can also be adaptively adjusted according to the running status. For example, when the system detects a decrease in current bandwidth or a backlog in the decoding queue, the time window can be shortened appropriately to reduce resource waste caused by invalid predictions; when the system is in a stable high-bandwidth state or the local cache hit rate is high, the time window can be extended appropriately to increase foresight.

[0032] In one embodiment, a sequence of predicted viewpoint states is generated using discrete sampling within a preset time window. For this purpose, a prediction step size (also known as the prediction sampling period or prediction resolution) can be preset, and the length of the prediction sequence can be determined accordingly. For example, when the preset time window is 1 second and the prediction step size is 100 ms, a sequence containing 10 predicted viewpoint states can be generated; when the prediction step size is aligned with the rendering frame period, frame-by-frame prediction states can be generated for several future frames. The setting of the prediction step size is usually related to the speed of change in motion trends and computational overhead: a smaller step size can express turning or speed change trends more precisely, but will result in higher prediction computation and storage overhead; a larger step size can reduce overhead, but may cause prediction "polyline" or "lag" during rapid turns. As an example, the prediction step size can be set to the same as or an integer multiple of the rendering frame period, such as 16.7 ms, 33 ms, 50 ms, or 100 ms.

[0033] In one implementation, the generation of the predicted viewpoint state can be achieved using an "inertial extrapolation" framework. This involves using the current viewpoint state as a starting point and recursively updating the pose at several discrete future moments based on motion parameters. Specifically, the predicted position can be updated by generating position increments based on motion parameters such as linear velocity and linear acceleration, and these increments are accumulated at each prediction step to obtain the corresponding predicted position. The predicted gaze direction can be updated by generating direction increments based on angular velocity or the trend of gaze direction changes, and the orientation is iteratively updated at each prediction step. To improve the stability of the direction update, the gaze direction can be represented as a unit direction vector or a quaternion / rotation matrix, and normalized or orthogonalized after each update to suppress numerical drift. If the motion parameters only include linear velocity and not acceleration, uniform extrapolation can be used to generate the predicted position sequence. If the motion parameters include linear acceleration, a variable acceleration trend can be introduced during the prediction process to make the predicted position more closely resemble the real-world interactive experience in scenarios such as sudden stops, sudden starts, or accelerated flight. Similarly, if the motion parameters include angular velocity, more continuous direction predictions can be given for scenarios such as rapid rotation and sharp turns; if only the trend of line-of-sight changes in adjacent frames can be obtained, direction extrapolation can also be performed based on this.

[0034] In one implementation, to prevent significant fluctuations in the predicted sequence caused by input noise, hand tremors, or device jitter, the system can perform smoothing and limiting processing on motion parameters before or during the generation of the predicted viewpoint state sequence. Smoothing can employ techniques such as sliding window averaging, exponential smoothing, or low-pass filtering, aiming to preserve the main motion trend and suppress high-frequency jitter. Limiting can impose upper limits on parameters such as linear velocity, linear acceleration, and angular velocity to prevent abnormal inputs from causing prediction jumps. For example, the upper limit for angular velocity can be set to the order of tens of degrees per second, or the upper limit for linear velocity can be set to a range of several meters per second to tens of meters per second that matches the scene scale. Specific upper limits can be set according to the product's roaming mode (walking, driving, flying); for example, the upper limit for linear velocity can be set lower in walking mode and appropriately higher in flying mode. The basis for setting these parameters can be explained as: limiting the extreme variation in the prediction to prevent the predicted sequence from deviating from the user's acceptable interaction inertia within a short time window, thereby improving the usability and consistency of the prediction results.

[0035] In one implementation, the predicted viewpoint state sequence can be stored and transmitted using a structured collection. For example, a "PredictedViewState" object can be defined, whose fields include the prediction time offset (e.g., t=100ms, 200ms, etc.), prediction position, prediction gaze direction, and optional prediction pose parameters; multiple "PredictedViewState" objects are arranged chronologically to form a "PredictedSequence" array or queue. To facilitate direct use by subsequent modules, the system can write the sequence to a shared memory cache after generation, or deliver it to the rendering scheduling queue as a message object; simultaneously, the start timestamp and sampling period of this prediction can be recorded so that a rolling update strategy can be executed when the next frame is updated, i.e., the prediction sequence is regenerated or incrementally updated with the latest current viewpoint state, thereby ensuring that the prediction always remains valid around the latest interaction state.

[0036] For example, in a web-based 3D visualization application, the camera controller can read the current camera pose and output motion parameters before the start of each rendering frame. The prediction module is implemented in JavaScript or WebAssembly and generates 20 predicted viewpoint states within a preset time window (e.g., 1 second) at fixed steps (e.g., 50 ms) and writes them to the "PredictedSequence" buffer. In a desktop engine, the same process can be implemented in C++ by the rendering thread or logic thread, using a high-precision timer to determine the time window and step size, and storing the prediction sequence in a circular buffer to reduce memory allocation overhead.

[0037] Optional, see Figure 2A flowchart of a method for generating a predicted viewpoint state sequence for virtual viewpoints, provided in an embodiment of this application, includes steps S201 to S203, wherein: S201: Obtain a sparse geometric proxy model to characterize the reachable boundaries of the current scene; S202: Determine the forward prediction direction based on the current viewpoint state, and calculate the forward safety margin in the sparse geometric proxy model along the forward prediction direction. The forward safety margin characterizes the reachability gap of the virtual viewpoint to the nearest geometric obstacle along the forward prediction direction. S203: Apply boundary constraints to the predicted motion increment within the preset time window based on the forward safety margin, generate a predicted position sequence that does not cross the geometric obstacle, and determine the predicted viewpoint state sequence based on the predicted position sequence.

[0038] Optionally, applying boundary constraints includes: The collision distance index is determined based on the forward safety margin and the motion parameters. In response to the collision distance index being less than a preset safety threshold, a constraint correction is triggered on the predicted motion increment, and when the collision distance index is greater than or equal to the preset safety threshold, the inertial predicted motion increment based on the motion parameters is maintained.

[0039] Optionally, the sparse geometry proxy model to be obtained includes at least one of the following: a depth proxy generated from the depth buffer of the previous rendering frame; a low-resolution elevation grid generated from GIS terrain data; a simplified collision bounding body generated from the shell of BIM components; and a set of coarse-level bounding bodies generated from a point cloud hierarchical index structure.

[0040] Research has found that in point cloud roaming scenarios that integrate GIS+BIM or connect indoor and outdoor environments, the movement of virtual viewpoints is often strongly constrained by geometric boundaries such as terrain undulations, building shells, and walls / door openings. If the predicted viewpoint state sequence is generated solely based on inertial extrapolation, the predicted trajectory may encounter unreachable situations such as crossing obstacles / walls, causing subsequent resource scheduling to preload unreachable areas, resulting in loading redundancy and a decrease in visual continuity.

[0041] To this end, this optional implementation introduces a sparse geometric surrogate model to characterize the reachability boundary of the current scene when generating the predicted viewpoint state sequence, and applies boundary constraints to the predicted motion increment based on the forward calculation safety margin, thereby improving the reachability and stability of the predicted sequence without significantly increasing the computational overhead.

[0042] In one optional implementation, the system acquires a sparse geometric proxy model to characterize the reachable boundaries of the current scene. A sparse geometric proxy model can be understood as a low-resolution, low-polygon, or low-sampling-density representation of the main geometrical obstructions in the scene. It does not strive for complete geometrical detail consistency with the original point cloud, but rather serves to quickly determine whether an insurmountable geometric obstacle exists within the short-term forward prediction range of the virtual viewpoint, and the approximate gap between the obstacle and the viewpoint. For example, the sparse geometric proxy model can be stored as a "ProxyModel" data object, which includes a set of geometric units for rapid intersection / ranging, such as a set of bounding volumes (axis-aligned bounding boxes, spheres, capsules, etc.), a collision shell composed of low-resolution mesh patches, or a terrain surface composed of low-resolution elevation meshes; and may include acceleration structures, such as hierarchical bounding volume trees, mesh block indexes, or raster hash indexes, to support real-time queries.

[0043] In one optional implementation, the system determines the forward prediction direction based on the current viewpoint state and calculates the forward safety margin along the forward prediction direction in a sparse geometric surrogate model. The forward prediction direction can be understood as the main forward trend direction when the prediction sequence is generated. It can be directly determined by the current line of sight, or it can be determined by combining motion parameters as a "weighted composite direction of the line of sight and the displacement trend direction" to adapt to different interaction modes where the line of sight turns first and the position follows, or the position moves first and the line of sight lags slightly. In implementation, the forward prediction direction can be normalized to a unit direction vector, and its update frequency can be limited to be consistent with the sampling period to avoid safety margin fluctuations caused by direction jitter.

[0044] The forward safety margin characterizes the reachability gap from a virtual viewpoint to the nearest geometric obstacle along the forward prediction direction, and can be obtained using a discrete detection method along the forward direction. For example, starting from the current viewpoint position, the system progressively samples along the forward prediction direction with a preset detection step size, and at each sampling position, queries whether there are intersecting or close geometric units in the sparse geometric proxy model; when an intersection with a geometric unit is detected for the first time, or when the distance to a geometric unit is detected to be less than a preset proximity threshold, the forward safety margin corresponding to that sampling position is determined as the current cumulative detection distance.

[0045] To reduce computational overhead, the detection step size can be set in relation to the scene scale: in indoor BIM corridor scenes, the detection step size can be set to 0.05m to 0.2m; in outdoor GIS city aerial browsing scenes, the detection step size can be set to 0.5m to 2m. The proximity threshold can be set based on the "collision radius" of the virtual viewpoint or the visual penetration risk caused by the camera's proximity to the clipping plane; an example value of 0.1m to 0.5m can be used. To avoid unnecessary overhead from long-distance detection, a maximum detection distance limit can also be set, which can be related to a preset time window and the current speed level. For example, in a mode with a preset time window of 1 second and a common roaming speed of 5m / s, the maximum detection distance limit can be set to 5m to 10m.

[0046] In one optional implementation, the system applies boundary constraints to the predicted motion increment within a preset time window based on a forward safety margin, generates a predicted position sequence that satisfies the condition of not crossing geometric obstacles, and determines a predicted viewpoint state sequence based on the predicted position sequence. Here, the predicted motion increment can be understood as the candidate displacement obtained by the inertial extrapolation frame at each discrete prediction step. The role of the boundary constraints is to: when a candidate displacement causes the predicted position to cross the reachable boundary, trim, project, or redirect the candidate displacement so that the updated predicted position falls before the geometric obstacle or slides along the obstacle surface, thereby avoiding crossing.

[0047] For example, boundary constraints can be implemented in one or more of the following ways: First, at each discrete prediction step, the component of the candidate displacement along the forward prediction direction is limited within the current safety margin, and a safety buffer distance, such as 0.1m to 0.5m, is reserved so that the predicted position remains before the obstacle; Second, when it is detected that the candidate displacement will cross the obstacle, the tangential component of the candidate displacement is maintained while the normal component is suppressed, so that the predicted position slides along the obstacle surface, which is suitable for roaming modes such as turning along the wall in indoor corridors or turning along the building facade in urban street valleys; Third, when an unreachable crossing is detected, a deceleration constraint is triggered, that is, the displacement amplitude is gradually reduced in the subsequent prediction steps, so that the prediction sequence naturally converges when approaching the obstacle. The core output of the above boundary constraints is the predicted position sequence, which can be combined with the direction prediction module generated by the aforementioned inertial extrapolation to form a predicted viewpoint state sequence containing the predicted position and the predicted line of sight direction.

[0048] In an optional implementation, to further improve the consistency and interpretability of the timing of boundary constraint triggering, a collision distance index can be determined based on the forward safety margin and motion parameters, and the comparison result between the collision distance index and a preset safety threshold can be used as the criterion for whether to trigger constraint correction. The collision distance index can be understood as: how long it will take for the viewpoint to reach the nearest geometric obstacle corresponding to the forward safety margin if it continues to move forward according to the current motion trend.

[0049] In implementation, the collision distance can be estimated based on the forward safety margin and the velocity component of the motion trend in the forward prediction direction. When the velocity component is too small or close to zero, the collision distance is set to a larger value or constraints are not triggered to avoid unnecessary corrections caused by stationary jitter. The setting of the preset safety threshold can be based on the minimum lead time of the system processing link and the requirements of interaction comfort: if the threshold is too small, the constraint is triggered too late, and penetration prediction and unnecessary prefetching may still occur; if the threshold is too large, the constraint is triggered too early, which may cause excessive conservatism for normal steering. For example, in indoor passage mode, the preset safety threshold can be set to 0.2s to 0.6s; in outdoor high-speed flight browsing mode, the preset safety threshold can be set to 0.1s to 0.4s, and adaptively adjusted according to the speed level. In response to the collision distance index being less than the preset safety threshold, the system triggers constraint correction for the predicted motion increment; and when the collision distance index is greater than or equal to the preset safety threshold, the inertial predicted motion increment based on motion parameters is maintained to reduce unnecessary correction overhead and prediction bias.

[0050] Regarding the acquisition method of sparse geometric proxy model, in an optional implementation, at least one of the following forms of geometric proxy can be acquired to adapt to the engineering conditions of different data sources and operating platforms.

[0051] First, a depth proxy generated from the depth buffer of the previous rendered frame. For example, the system can read a downsampled version of the depth buffer of the previous frame or a low-resolution layer of a multi-level depth pyramid from the GPU as a fast obstacle cue for the viewpoint neighborhood; this approach is applicable to WebGL / WebGPU or desktop rendering pipelines and has the advantage of consistent viewpoint coverage.

[0052] Secondly, low-resolution elevation grids generated from GIS topographic data. For example, DEM / DSM or tiled topographic data can be resampled at a coarser resolution to form regular grid surfaces, which are used to characterize the reachable boundaries formed by topographic undulations and are suitable for city-level or park-level navigation.

[0053] Third, simplified collision bounding bodies generated from the shells of BIM components. For example, mesh simplification or bounding body fitting can be performed on the shells of components such as walls, floors, and columns to generate collision shells for rapid intersection determination. This method can be implemented in conjunction with common BIM parsing tools or engine collision modules, such as using the collision shape generation interface of the physics engine on the desktop and using a pre-processed glTF collision mesh on the web.

[0054] Fourth, the set of coarse-level bounding boxes generated by the point cloud hierarchical index structure. For example, the system can directly reuse the set of bounding boxes of coarse-level nodes in the point cloud hierarchical index as an approximate expression of the distribution boundary of a large-volume point cloud, which is used to quickly determine whether there are "obstacles formed by a large number of point cloud entities in the forward direction". This is suitable for scenarios where there is a lack of explicit BIM shell or terrain data, but the point cloud itself has obvious occlusion boundaries.

[0055] For example, in a GIS+BIM integrated urban roaming scenario, a sparse geometric proxy model can be generated by combining a low-resolution elevation grid with simplified BIM collision bounding bodies: when the viewpoint is on an outdoor road or in low-altitude flight mode, the elevation grid constrains the viewpoint to prevent it from traversing the terrain; when the viewpoint approaches a building complex or enters an interior structure, the BIM collision shell constrains the viewpoint to prevent it from traversing walls and floors. Simultaneously, in the web implementation, a depth proxy from the previous frame's depth buffer can be supplemented to quickly capture nearby obstacles when the viewpoint makes a sharp turn, improving the real-time performance of forward safety margin calculations. The predicted viewpoint state sequence generated in this way better conforms to the scene's reachable boundary constraints, reducing invalid prefetching caused by predictive penetration and providing more stable input for subsequent processing.

[0056] Optionally, the constraint correction includes: Determine the local surface normal corresponding to the nearest geometric obstacle; The environmental repulsion correction amount is determined based on the local surface normal, and the environmental repulsion correction amount is superimposed on the inertial predicted motion increment; The predicted motion increment after superposition correction is iteratively updated with discrete time steps, and a non-crossing constraint is applied to the predicted position at each discrete time step to generate the predicted position sequence. The determination of the local surface normal includes: when the sparse geometric proxy model includes a depth proxy, determining the local surface normal based on the depth change rate of the depth proxy; when the sparse geometric proxy model includes a simplified collision bounding body, determining the local surface normal based on the geometric patch normal of the simplified collision bounding body.

[0057] This optional implementation addresses the problem of how to generate a predictive position sequence that neither crosses geometric obstacles nor traverses them, and which remains continuous and stable near the obstacle boundary, without significantly increasing the prediction computation overhead after triggering boundary constraints (e.g., when the collision distance index is below a safety threshold). This avoids trajectory abrupt changes, jitter, or oscillations along the boundary caused by relying solely on "stop / pruning," thereby improving the accuracy of reachability prediction and the consistency of interactive experience during dynamic roaming.

[0058] In one alternative implementation, after triggering a constraint correction to the predicted motion increment, the local surface normal corresponding to the nearest geometric obstacle is first determined. This local surface normal can be used as a boundary guide direction to indicate the direction in which the predicted trajectory should be pushed away from the obstacle, and further used to construct the environmental repulsion correction.

[0059] For example, a "ContactInfo" structure or object can be maintained in memory, containing obstacle identification, nearest distance / safety margin, contact point estimation, local surface normal, and the source type of the normal (deep proxy or simplified collision bounding volume) to support continuous correction and debouncing within subsequent prediction steps.

[0060] In one alternative implementation, when the sparse geometry proxy model includes a depth proxy, the system determines the local surface normal based on the depth change rate of the depth proxy. A depth proxy can be understood as a low-resolution representation or downsampled layer of the depth buffer from the previous rendered frame, which has the advantage of being consistent with the visible occlusion relationship of the current viewpoint and can be quickly generated by the GPU.

[0061] In practice, a low-resolution depth texture can be output after the rendering pipeline is completed. For example, the original depth buffer can be downsampled to 1 / 8 or 1 / 16 resolution, and the depth texture can be locally sampled on the CPU side or the GPU compute shader side to estimate the depth change rate.

[0062] Specifically, the landing area of ​​the sampling ray corresponding to the "forward prediction direction" in screen space or view space can be determined in the depth proxy. Within this area, the depth value difference between adjacent pixels (e.g., a 2×2 or 3×3 neighborhood) is taken to obtain the trend of depth variation with screen coordinates. This trend is then mapped to a local tilt direction in view space and, combined with camera intrinsics or projection parameters, restored to a normal direction in world coordinates. To reduce noise, the system can perform smoothing on the neighborhood sampling results, such as weighted averaging of normals estimated from multiple neighborhoods and normalizing the normal vector after each update, thereby obtaining stable local surface normals. This approach is particularly suitable for web-based or lightweight clients: for example, in WebGL / WebGPU, the depth proxy can be directly exported as a texture from the rendering target, normal estimation can be completed in the compute shader and written back to the buffer, and the CPU only needs to read the results of the local region.

[0063] In another alternative implementation, when the sparse geometry proxy model includes a simplified collision bounding body, the system determines the local surface normals based on the geometric patch normals of the simplified collision bounding body. The simplified collision bounding body can be a low-polygon mesh simplified from the shell of a BIM component, a collision shell composed of several bounding boxes / convex hulls, or a set of collision shapes generated by a physics engine. In implementation, the patch normals for each collision mesh patch can be pre-calculated and stored during the loading or preprocessing stage, or the contact normals returned by the collision query interface can be directly reused when using a physics engine (e.g., Bullet, PhysX, or a self-developed collision module).

[0064] Specifically, when performing forward safety margin calculations or boundary constraint judgments, the system has usually obtained a candidate set of "nearest geometric obstacles" and their nearest points or intersection information; at this time, the normal can be read from the intersection patch or the nearest patch as the local surface normal.

[0065] To improve continuity, a "normal locking / hysteresis" mechanism can be introduced: when the obstacle marker remains unchanged within several consecutive prediction steps and the contact distance fluctuates within a small range, the normal from the previous moment is reused first, and only a small, smooth update is made. This avoids trajectory jitter caused by normal jumps at corners, doorway edges, or complex component junctions. This method is suitable for BIM indoor corridors, stairwells, equipment rooms, and other scenarios with clearly defined wall / floor normals, and can provide a push-off direction that is more geometrically consistent.

[0066] In one alternative implementation, after determining the local surface normal, an environmental repulsion correction is determined based on the local surface normal, and this environmental repulsion correction is superimposed on the inertial predicted motion increment. The environmental repulsion correction can be understood as a directional compensation for the candidate displacement, used to push the predicted motion away from the obstacle normal direction, while preserving the motion trend along the obstacle tangential as much as possible, so as to achieve a more natural edge-sliding effect.

[0067] In implementation, the direction of the environmental repulsion correction can be taken as the opposite of the local surface normal, i.e., the side away from the obstacle, and its amplitude can be adaptively set according to the risk level. The risk level can be represented by previously obtained quantities, such as the remaining amount of the forward safety margin, the magnitude of the collision distance index, or the approximation component of the candidate displacement in the normal direction.

[0068] To avoid over-correction that could cause the viewpoint to "bounce away" or jump backward, the system can limit the correction amount and set a minimum safe buffer distance as a target. For example, when it is expected to collide with an obstacle in the next one or two steps, the correction amount should at least ensure that the predicted position maintains a preset buffer distance from the obstacle, such as 0.05m to 0.3m. A smaller value can be used indoors, and a larger value can be used appropriately on outdoor high-speed roads. At the same time, the system can apply the environmental repulsion correction amount progressively: gradually increase the correction amount as the collision risk gradually increases; and gradually decrease the correction amount as the risk is eliminated, in order to improve the continuity of the trajectory.

[0069] For example, during an indoor roaming where the user quickly turns toward a wall, the inertial prediction will lead the viewpoint toward the wall. The system cancels or weakens the normal component of the displacement by normal repulsion and retains the tangential component, so that the predicted trajectory extends along the wall, thus better matching the user's actual interaction behavior of turning close to the wall.

[0070] In one optional implementation, the system iteratively updates the superimposed and corrected predicted motion increments in discrete time steps, and applies non-crossing constraints to the predicted positions at each discrete time step, generating a sequence of predicted positions. The discrete time step can be consistent with the aforementioned prediction step, or a smaller internal integration step can be used to improve stability near the boundary; for example, when the external prediction step is 50ms, the internal sub-steps of 10ms to 25ms can be used for iterative updates to reduce the probability of an update crossing an obstacle. The iterative update process can be described using an engineering data structure: the system maintains a current predicted position variable. Starting from the current viewpoint, the "corrected predicted motion increment" is proportionally accumulated in each sub-step to obtain the next predicted position, and a non-crossing constraint check is performed immediately after accumulation. The non-crossing constraint check can be implemented based on the fast intersection judgment of a sparse geometric surrogate model: when it detects that the predicted position has entered the interior of an obstacle or that the distance to the obstacle is less than the minimum buffer threshold, the system performs a backtracking or projection process on the predicted position, returning it to outside the obstacle boundary, and can simultaneously update the motion increment of the current step, allowing subsequent steps to continue advancing tangentially instead of repeatedly hitting the boundary. Exemplary backoff methods include: pushing the predicted position outward along the local surface normal direction to a position that satisfies the minimum buffer distance; or projecting the predicted position onto the tangential plane of the obstacle surface and retaining the tangential displacement, thereby achieving "non-crossing and slip-able". After completing the above iterations, the system outputs the predicted positions corresponding to each sub-step or each prediction step in chronological order, forming a predicted position sequence, and thereby obtaining a predicted viewpoint state sequence that better conforms to the reachability boundary constraints within the scope defined by this subordinate scheme.

[0071] For example, in a web-based GIS+BIM fusion rendering platform, depth proxies can output low-resolution depth textures via the WebGPU rendering pipeline, and local normal estimation can be obtained by neighborhood sampling of a specified screen area using a computational shader. For BIM indoor collision shells, low-polygon collision meshes can be generated and facet normals pre-calculated using model processing tools during the resource import phase, and the nearest facet normals can be returned at runtime through lightweight collision queries. After triggering constraint correction, the system writes the normals, risk indicators, and correction amounts into the ContactInfo cache, and performs iterative updates and non-crossing constraints according to the step size within the prediction time window, outputting a stable predicted position sequence. Through this combination of normal guidance, progressive repulsion, and step-by-step non-crossing, the probability of predicted penetration and trajectory jitter can be significantly reduced when approaching walls, building shells, or terrain boundaries, while maintaining the continuity of the prediction sequence and engineering feasibility.

[0072] Regarding the above S103: Based on the predicted viewpoint state sequence obtained in step S102, a spatiotemporal visual envelope is constructed to characterize the "potential visible spatial range within a future period." This ensures that even during rapid nonlinear movements of the virtual viewpoint, such as sharp turns, sudden stops and starts, or cornering, the actual field of view can still be covered. Simultaneously, during smooth movements, the envelope converges as much as possible to reduce spatial redundancy in subsequent processing. The spatiotemporal visual envelope can be understood as a three-dimensional spatial volume obtained by constructing local viewpoint sub-volumes at multiple prediction moments corresponding to a preset time window and combining these sub-volumes (e.g., union or approximate union). Its "spatiotemporal" meaning lies in the fact that this spatial volume is formed by the accumulation of multiple viewpoint volumes over a time sequence, used to cover the visible sweep area in the short future period.

[0073] In one implementation, the predicted viewpoint state sequence can be represented as a time-ordered set, such as "PredictedSequence," where each element "PredictedViewState" includes at least a prediction time identifier, a prediction location, and a prediction viewing direction. Before constructing the spatiotemporal visual envelope, the predicted sequence can undergo consistency preprocessing, such as normalizing the predicted viewing direction, checking the coordinate system consistency of the predicted location, and removing or clamping outlier points (e.g., outliers where a single predicted point is far from its neighborhood) to avoid unnecessary geometric spikes in subsequent local view sub-volumes.

[0074] In one embodiment, a local field of view sub-volume is used to characterize the local visible range "at a certain predicted moment, with the predicted position as the observation point and the predicted line of sight as the central axis". The local field of view sub-volume can be represented by a geometric form that is easy to implement in engineering and for intersection testing, such as a frustum (camera frustum), a cone, a wedge, a spherical cap, a capsule, or a convex polyhedron composed of multiple half-space constraints. For example, when the system uses a perspective projection camera model, the local field of view sub-volume can be constructed according to the camera frustum: with the predicted position as the vertex of the frustum (or the center of the near clipping plane), and the predicted line of sight as the axis of the frustum, the opening and depth of the frustum are determined by combining the field of view angle and the near and far clipping distances; when the system is in a narrow passage scene such as an indoor corridor, a wedge in the line of sight direction combined with lateral safety margin can also be used to approximate the local field of view to reduce the coverage of irrelevant lateral spaces.

[0075] In one embodiment, the system further calculates a steering intensity index to characterize the degree of nonlinearity or uncertainty in viewpoint motion within the prediction sequence, and adaptively adjusts at least one geometric parameter of the local viewpoint sub-volume accordingly. This allows the local viewpoint to expand moderately to accommodate potential sweep directions when the steering intensity is high, and to converge when the steering intensity is low to reduce redundant coverage. The steering intensity index can be determined by the rate of change of line-of-sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence.

[0076] In implementation, the system can calculate the change in direction and the change in position between two adjacent predicted states, and normalize them by combining the time interval between adjacent predictions to obtain the strength of the change trend. To avoid jitter, the turning strength index can be stabilized by using sliding window smoothing or exponential smoothing, and a hysteresis interval can be set to avoid frequent switching near the threshold.

[0077] For example, the rate of change of gaze direction can be understood as the intensity of change in the rotation amplitude between adjacent predicted gaze directions per unit time, and the rate of change of position can be understood as the intensity of change in the displacement between adjacent predicted positions per unit time. Both can be used individually or fused together according to preset weights as a steering intensity index. In roaming modes primarily based on gaze dragging, such as right-clicking to rotate the view, the weight of the rate of change of gaze direction can be increased; in roaming modes primarily based on displacement movement, such as keyboard forward movement or joystick movement, the weight of the rate of change of position can be increased.

[0078] As an example, the following settings can be configured: when the line-of-sight direction changes significantly and continuously between adjacent items in the predicted sequence, the steering intensity is determined to be "high"; when the line-of-sight is mainly constant and the direction changes are small or discontinuous, the steering intensity is determined to be "low".

[0079] In one embodiment, the geometric parameters of the local field of view sub-void include at least one of the following: angular parameters (e.g., horizontal / vertical opening angle or equivalent opening angle), depth parameters (e.g., distance to the far end section or maximum viewing distance), lateral / vertical margin parameters (e.g., extended boundaries added on both sides of the viewing cone), and sub-void sampling density parameters (e.g., the number of segments when discretizing the sub-void). At least one of these geometric parameters varies with the steering intensity index at the corresponding prediction time.

[0080] For example, the angle parameter can be set to "base angle + angle adjustment amount", where the angle adjustment amount increases with the steering intensity; or the depth parameter can be set to "base depth × depth coefficient", where the depth is appropriately reduced to reduce ineffective coverage at long distances when the steering intensity is high, while increasing the lateral margin to cover the lateral sweep area during sharp turns. This combination is beneficial for reducing over-coverage of distant, unreachable, or unlikely-to-be-immediately-visible areas in sharp-turn edge-hugging scenarios, and for increasing the probability of coverage of newly entered nearby areas of view.

[0081] In one implementation, a tiered threshold strategy can be used to determine the geometric parameter adjustment rules. For example, the steering intensity can be divided into three levels: low, medium, and high. The low level uses a basic angle and basic depth; the medium level adds a small expansion to the basic angle while maintaining depth; and the high level significantly expands the angle or lateral margin, and can set an upper limit on the depth to avoid covering too far at once. The threshold setting can be based on the frame rate, common interaction sensitivity, and the response time of the data loading link: when the frame rate is high and the interaction is more refined, the threshold can be more sensitive to quickly respond to small steering movements; when the frame rate is low or the input noise is high, the threshold can be appropriately increased and smoothed to avoid redundancy caused by frequent expansion.

[0082] As an example, in an interactive scenario of around 60fps, "significant changes in direction within several consecutive prediction steps" can be used as a high-level trigger condition, and the high-level state (hysteresis) can be maintained for several steps after triggering to avoid coverage gaps caused by immediate convergence after the turn ends.

[0083] In one implementation, the system constructs a corresponding local view sub-volume for each predicted viewpoint state in the predicted viewpoint state sequence, and combines multiple local view sub-volumes to obtain a spatiotemporal visual envelope. The combination can be a strict geometric union or an approximate union oriented towards real-time performance.

[0084] In implementation, to reduce computational and storage overhead, the system can attach a fast bounding representation to each local view volume, such as an axis-aligned bounding box (AABB), an oriented bounding box (OBB), or a bounding sphere, and represent the spatiotemporal visual envelope as a hierarchical structure of multiple bounding sets and the overall bounding. For example, a "LocalViewBody" object can be defined with fields including: prediction time identifier, volume type identifier, key geometric parameters (angle, depth, margin, etc.), and the volume's bounding. A "SpatioTemporalEnvelope" object can be defined with fields including: a list of LocalViewBody objects, the overall bounding, and an optional hierarchical acceleration structure, such as a bounding tree or segmented index, for rapid use by subsequent modules.

[0085] For example, in web-based 3D visualization applications, this construction process can be implemented using JavaScript or WebAssembly: the prediction sequence is output by the camera controller, the turning intensity index is calculated in the main thread or worker, and the local view volume can be approximated using several planar parameters or bounding boxes and written to a shared buffer. In desktop engines, the same process can be implemented using C++ modules, using linear algebra libraries (such as the engine's built-in math library) to construct the view frustum plane or bounding volume, and writing the LocalViewBody to a circular buffer to reduce memory allocation. For GIS+BIM fusion scenarios, the system can uniformly construct local view volumes in the world coordinate system, thereby ensuring consistency in the view coverage calculation between city-level point clouds and indoor point clouds, avoiding envelope distortion or parameter mismatch caused by coordinate scale differences.

[0086] In this way, the "predicted motion trend" can be transformed into the "potential visible spatial range in the short term". Through the adaptive geometric parameters driven by the steering intensity, the spatiotemporal visible envelope has stronger coverage robustness during rapid nonlinear motion and stronger range convergence during steady motion, thus providing a stable and interpretable spatial representation basis for subsequent processing.

[0087] See Figure 3 A flowchart of a method for constructing a spatiotemporal visual envelope provided in this application embodiment includes steps S301 to S303, wherein: S301: For each predicted viewpoint state in the predicted viewpoint state sequence, construct a corresponding cutting cone with the corresponding predicted position as the cone apex and the corresponding predicted line of sight as the cone axis. S302: Determine at least one geometric parameter of the cutting cone based on the steering strength index at the corresponding prediction time; S303: Perform a union combination of multiple kissing cones constructed along the predicted viewpoint state sequence to obtain the spatiotemporal visual envelope.

[0088] During dynamic roaming of large-scale point clouds, especially in scenarios involving GIS+BIM integration, indoor-outdoor connectivity, and "corridor-type viewpoints" such as street valleys / corridors, the short-term future visible range of a viewpoint often exhibits a distinct geometric characteristic of extending along the line of sight and being laterally restricted. If a fixed view cone or a simple expanded volume is still used to represent the local viewpoint, lateral over-coverage can easily occur in narrow corridors, leading to the introduction of a large number of invalid areas for subsequent spatial queries and data prefetching. On the other hand, during sharp turns or corner turns, insufficient expansion can easily cause potential viewpoint omissions.

[0089] To balance coverage robustness and range convergence, this optional implementation defines the local view sub-void as a tangent cone and adjusts its sub-angle by adjusting the turning intensity and scene openness. This makes the local view sub-void fit the scene geometric constraints better near the corridor boundary and has a controllable expansion capability when the turning uncertainty increases.

[0090] In one optional implementation, the local view sub-volume is a kissing cone. A kissing cone can be understood as a cone-shaped view representation with the predicted position as the cone apex and the predicted viewing direction as the cone axis. The "kissing" semantics are reflected in the fact that the cone's sub-angle is not blindly set to a fixed field of view angle, but is constrained by the upper limit of the scene's visible corridor openness, making the cone's side boundaries statistically close to the corridor boundaries, thereby reducing lateral ineffective coverage. In implementation, the system can define a "TangentCone" data object to store the local view sub-volume corresponding to each prediction time. This data object typically includes: a prediction time identifier, cone apex coordinates, cone axis unit vector, final sub-angle, cone depth (e.g., maximum viewing distance or far-end section distance), and an outer bounding body for rapid intersection testing, such as a bounding sphere or axis-aligned bounding box.

[0091] In one optional implementation, the system constructs a corresponding tangent cone for each predicted viewpoint state in the predicted viewpoint state sequence, using the corresponding predicted position as the cone apex and the corresponding predicted line-of-sight direction as the cone axis. During construction, a basic sub-angle and a basic depth can be provided as initial values. The basic sub-angle can be an angle related to the camera's basic field of view, and the basic depth can be a depth matched to a preset time window, the current motion speed level, or a point cloud visibility distance strategy.

[0092] For example, the basic depth can be set to the order of 10m to 30m in indoor passage mode and to the order of 50m to 200m in outdoor low-altitude flight browsing mode; the initial value of the subtended angle can be taken as a certain proportion of the camera's horizontal field of view (e.g., 0.6 to 1.0 times) to take into account both the expression of the field of view and the prefetching convergence.

[0093] In one optional implementation, the system determines at least one geometric parameter of the slicing cone based on the steering intensity index at the corresponding prediction time. Specifically, the steering intensity index is used to characterize the "view sweep uncertainty" at that prediction time: when the rate of change of the line of sight direction between adjacent prediction states in the prediction sequence is high, or when the rate of change of position changes abruptly, such as sudden stop, sudden start, or sudden speed change, the potential visible directions in the short future are more dispersed, and the cone angle needs to be larger to cover the lateral areas that may be swept; when the steering intensity is low, the cone angle should converge to reduce the redundancy of subsequent processing. In implementation, a graded adjustment method can be adopted: for example, the steering intensity is divided into low, medium, and high, corresponding to the angle adjustment amount of 0°~5°, 5°~12°, and above 12°, respectively (the values ​​are only examples and can be combined with interactive sensitivity and frame rate parameter tuning), and the adjustment amount is smoothed and hysteresis is processed to avoid frequent jumps near the threshold that cause "breathing jitter" of the envelope.

[0094] In one optional implementation, determining at least one geometric parameter of the cutting cone based on the steering strength index includes: determining the angular adjustment amount of the cutting cone based on the steering strength index. The angular adjustment amount can be understood as an increment of the base angular angle, used to expand lateral coverage during sharp turns; its setting can be related to the system's end-to-end latency: when there is significant latency in system network download, decoding, or GPU upload, appropriately increasing the adjustment amount can reduce the risk of "actual field of view ahead of prefetch"; when the system's local cache hit rate is high and the latency is low, the adjustment amount can be appropriately reduced to improve convergence.

[0095] For example, in a web-based remote point cloud tile loading scenario, the high-level angle adjustment can be set to a relatively larger value; in a desktop-based local SSD loading scenario, the high-level adjustment can be set to a relatively smaller value.

[0096] In one optional implementation, the system acquires scene openness parameters characterizing the openness of the visible corridor at the predicted location, and determines the upper limit of the angle of the tangent cone based on the scene openness parameters. The scene openness parameters are calculated by a geometric proxy used to characterize the terrain elevation surface, building shell, and indoor passage structure, and their core semantics are: at the predicted location, in the forward space along the cone axis, the width or openness of the accessible gap on both sides of the cone axis.

[0097] The scene openness parameter can be expressed in one of the following forms: "accessible gap width", "minimum lateral clearance", "half width of corridor", "openness level", etc., and can be accompanied by a reference forward distance field so that the width-type value can be converted into the upper limit of the angle.

[0098] For example, in an indoor BIM access structure scenario, the openness parameter can be calculated based on local cross-sectional data of a simplified collision shell, navigation mesh, or corridor centerline: the predicted position is projected onto the plane where the navigation mesh is located, and lateral detection is performed in the left and right directions perpendicular to the cone axis to obtain the shortest reachable distance from the left and right sides to the wall or door frame, and the sum of the left and right distances is used as the reachable gap width; when a sudden change in clearance caused by a doorway, corner, or column is detected, a more conservative minimum clearance can be taken as the openness parameter.

[0099] In practice, the BIM parsing module (such as IFC parsing) can generate the component shell proxy, and the collision module (such as Bullet, PhysX or self-developed collision query) can provide the nearest distance and contact information; or the navigation mesh building tool such as Recast can generate the "corridor half-width field" in the offline stage, and the openness parameter can be obtained directly by looking up the table at runtime to reduce the real-time detection overhead.

[0100] In city-level GIS scenarios, the scene openness parameter can be calculated by low-resolution elevation grid and geometric proxy of building shell: For street valley areas, several forward sampling points can be selected along the cone axis at the predicted location, and the distance to the building shell or terrain shading boundary can be detected in the left and right directions at each sampling point in a ray or grid stepping manner, thereby estimating the average openness in that direction; for open squares or high-altitude views, a larger lateral clearance can be obtained.

[0101] In terms of implementation, a low-cost spatial index (such as raster hash or block bounding box index) can be built based on tiled terrain (DEM / DSM) and building white model shell, and an approximate openness parameter can be obtained at runtime using a small number of sampled rays; on the web, this calculation can be carried out in WebAssembly or Worker threads to avoid blocking the main rendering thread.

[0102] In one optional implementation, the system determines the upper limit of the angle of the scissor cone based on the scene openness parameter. The upper limit of the angle can be understood as "the maximum angle that the lateral boundary of the cone is allowed to expand to under the constraint of the corridor boundary," which is used to prevent excessive expansion due to increased turning intensity in narrow corridors or street canyon scenes, resulting in the cone covering a large area behind walls or inside the building that is not visible. A reference forward distance conversion strategy can be adopted: for example, take a reference distance related to the foundation depth (such as 5m, 10m or 20m), convert the lateral reachable gap width at the reference distance into the maximum allowable expansion angle, and apply minimum and maximum value clamps to the conversion result. For example, when the reachable gap width of the indoor corridor is about 2m to 3m, the upper limit of the angle can be limited to a relatively small range (e.g., on the order of 20° to 35°); when in an open square or outdoor high-altitude view, the upper limit of the angle can be relaxed to a larger range (e.g., above 60°) to adapt to a more open visible corridor.

[0103] In one optional implementation, the system determines the final angle of the cutting cone based on the angle adjustment amount and the upper limit of the angle. A strategy of adjustment followed by limiting can be adopted: candidate angles are obtained by superimposing the angle adjustment amount on the base angle, and then the upper limit of the angle is used to limit the candidate angles, thereby outputting the final angle. To maintain sequence consistency, the system can add a "gradual constraint" to the final angle, such as limiting the amplitude of angle changes between adjacent prediction times, or introducing a hysteresis interval, so that the angle does not frequently contract or expand due to slight fluctuations in the aperture parameter.

[0104] For example, when passing through a doorway or corner, the openness can change drastically over a short distance. If the system detects a sudden change in openness but the duration is very short, it can adopt a short-term holding strategy to make the angle transition smoothly within one or two prediction steps, so as to avoid the local field of view sub-volumes from changing drastically at the edge of the doorway.

[0105] In one alternative implementation, a union of multiple kissing cones constructed along the predicted viewpoint state sequence is performed to obtain a spatiotemporal visual envelope. The union can be geometrically exact or a real-time-oriented approximate union. For example, each kissing cone can be discretized into several "segmented frustums" or approximated by its bounding volume, and the multiple bounding volumes can be organized into a hierarchical structure, such as a bounding volume tree or a bucket index segmented by prediction time, thereby representing the spatiotemporal visual envelope as a set of geometric proxies that can be quickly used for subsequent spatial processing.

[0106] For example, an "EnvelopeNode" object can be defined to store the outer bounding volume of the cone and its key parameters at each prediction time, and an "EnvelopeBVH" can be defined as an acceleration structure. In WebGL / WebGPU scenarios, a TypedArray can be used to store the cone parameters (cone apex, cone axis, opening angle, depth) and maintain the AABB array synchronously. On the desktop, a C++ structure array plus a circular buffer can be used to achieve low-overhead updates.

[0107] For example, in an indoor-outdoor connectivity walkthrough integrated with GIS+BIM, when the viewpoint is in an outdoor street valley and makes a sharp turn, the increased turning intensity drives an increase in the angle adjustment. However, the calculated openness of the building shells on both sides of the street valley constrains the upper limit of the angle, preventing the tangent cone from expanding to cover the area behind the street block. When the viewpoint enters an indoor corridor, the openness parameter decreases significantly, and the upper limit of the angle converges accordingly, making the cone fit the corridor more closely. When the viewpoint reaches the lobby or atrium, the openness parameter increases, the upper limit of the angle widens, and the cone can recover a larger angle to cover a more open potential field of view. By combining the methods of "turning intensity driving expansion, openness limiting the upper limit, and sequence union forming the spatiotemporal range," the spatiotemporal visual envelope can better fit the physical access and occlusion boundary characteristics of the specific scene, and provide a more stable and convergent spatial expression input for subsequent processing.

[0108] Regarding S104 above: Using the spatiotemporal visual envelope obtained in step S103 as a unified query condition for range pruning and priority filtering of massive point cloud data, data blocks related to the future short-term visible range are quickly located in the hierarchical point cloud index structure. The located data blocks are then loaded into the memory / GPU side according to the hierarchical strategy and participate in rendering, thereby balancing rendering continuity and controllable resource overhead in dynamic roaming.

[0109] In one embodiment, a hierarchical point cloud index structure is used to organize the original point cloud into multi-level nodes according to spatial location and level of detail. Exemplarily, the index structure can be an octree, kd-tree, BVH-level bounding volume tree, or tile tree; each index node is associated with at least one point cloud data block and records the spatial bounding volume and hierarchical information of that data block. The spatial bounding volume can be implemented using one of axis-aligned bounding boxes (AABB), bounding spheres, or directed bounding boxes (OBB) for fast intersection testing; the hierarchical information may include a level identifier, point size, geometric error / level of detail representation value, data storage location identifier (e.g., file offset, object storage URL, tile ID), etc. The system can maintain an array of "IndexNode" structures or tree node objects in memory, storing a boundingVolume field and a children pointer / index list in each node; the point cloud data block can maintain a "BlockMeta" metadata object, containing blockId, lodLevel, byteSize, codecType (e.g., LAS / LAZ, Draco, Zstd, etc.), and an optional priority cache key.

[0110] In one implementation, the spatiotemporal visual envelope is converted into a set of geometric query volumes suitable for indexed queries. Since the envelope may be obtained by combining multiple local view sub-volumes, a conservative approximation strategy can be adopted to reduce the intersection judgment overhead: for example, generating an outer bounding volume (AABB or bounding sphere) for each local view sub-volume and combining these outer bounding volumes into a "QueryVolumes" set; or generating an overall outer bounding volume for the entire spatiotemporal visual envelope as a first-level coarse screening condition, and then performing a more refined intersection judgment on candidate nodes after the coarse screening passes.

[0111] The above approximate setup is based on the fact that a small amount of conservative redundancy is allowed in the query phase to ensure stable real-time performance, while subsequent hierarchical selection and rendering pruning can further eliminate unnecessary data blocks. For example, when the system runs at 60fps and the query time is to be controlled within the range of 1ms to 3ms, a two-level strategy of coarse screening of the overall bounding volume combined with fine screening of the local bounding volume can be used first; when running on mobile or web devices and the main thread overhead is sensitive, only coarse screening of the overall bounding volume can be used and the fine screening can be devolved to the worker thread or C++ side module for execution.

[0112] In one implementation, the spatiotemporal visual envelope is used as the spatial index query condition to determine the target point cloud data block that intersects with the spatiotemporal visual envelope in the hierarchical point cloud index structure. In practice, traversal can begin from the index root node: an intersection test is performed on the spatial bounding volume of the current node and QueryVolumes; if they do not intersect, the node and its subtrees are pruned and skipped; if they intersect, the node is marked as a candidate node, and a hierarchical strategy is used to determine whether to continue traversing the child nodes downwards. The hierarchical strategy can employ heuristic criteria: for example, based on the candidate node's hierarchical depth, point size, distance range from the predicted viewpoint location, and the loading budget of the current frame / current time window, it is determined whether to stop at that level and add its associated data block to the target set, or continue traversing to obtain data blocks at finer levels.

[0113] For example, the maximum number of nodes traversed per frame or the maximum number of output blocks per query can be set as a hard constraint, such as traversing a maximum of 500 to 2000 nodes and outputting a maximum of 50 to 300 candidate data blocks per query. This range can be configured according to CPU performance and index size, with higher values ​​on desktop and lower values ​​on web.

[0114] In one implementation, the identified target point cloud data blocks are organized into a loading queue, and then prioritized and deduplicated. A "QueryResultItem" object can be defined, containing blockId, lodLevel, boundingVolume, priorityScore, and a status indicator (not requested / requesting / ready). Priority settings are typically based on data blocks that are "more likely to affect the current or near-future view": for example, data blocks closer to the predicted viewpoint, located in the main line of sight, and with a coarser but larger coverage are prioritized to quickly fill the image; finer-level data blocks are added gradually as bandwidth and decoding budget allow.

[0115] In one implementation, the system loads and performs hierarchical rendering on the target point cloud data blocks. The loading process can be implemented using an asynchronous pipeline to avoid blocking the main rendering loop: for example, after the query thread generates a queue to be loaded, the I / O thread or network thread requests data blocks in parallel; after the data arrives, the decoding thread performs decompression / decoding and point attribute rearrangement, such as coordinate quantization inverse decoding and color / intensity field unpacking; subsequently, the point data is written to the vertex buffer or SSBO through the GPU upload module and registered as a renderable resource. The basic approach of hierarchical rendering is coarse-to-fine: when fine-level data blocks for a certain region are not yet ready, the system can first use its parent level or coarser level data blocks to participate in rendering to maintain continuity; when fine-level data blocks are ready, they are replaced or superimposed with even finer data blocks, thereby gradually improving detail without introducing obvious holes. To control video memory and memory usage, the system can maintain an LRU cache or a hierarchical cache pool: for example, setting a longer residence time for coarse-level blocks to stabilize the image, and setting a shorter residence time for fine-level blocks to avoid video memory exhaustion; when the cache reaches a threshold, eviction is triggered. For example, the GPU-side point cloud cache budget can be set to 512MB to 4GB (desktop) or 128MB to 512MB (mobile / lightweight), and the number of concurrent download requests per session can be limited to 4 to 16 to balance throughput and jitter.

[0116] For example, in a web-based city-level point cloud GIS walkthrough, the hierarchical point cloud index structure can correspond to a tile tree or 3DTiles-style hierarchical organization, and the spatial bounding volume can directly use the bounding box / bounding sphere of the tiles; the query module can be implemented using JavaScript + WebAssembly, node traversal and intersection testing are completed in the Worker, and the results are returned to the main thread in the form of messages; data blocks are pulled via HTTP Range or object storage fragmentation, and decoding can be performed using LAZ decompression or Draco decoding. On the GPU side, data is uploaded and rendered as point sprites via WebGL / WebGPU buffers. In the desktop engine (OpenGL / Vulkan / DirectX), index traversal and intersection testing can be executed in a C++ logic thread, I / O uses asynchronous file reading (SSD random access) or a network SDK, and after decoding, data is submitted to the rendering thread through an upload queue. A circular buffer manages GPU resources to ensure a continuous walkthrough experience at high frame rates.

[0117] The above methods enable the system to perform precise data block-level scheduling around the short-term visible range, reducing invalid loading and lowering the risk of screen latency in dynamic roaming, rapid turning, or non-linear motion scenarios.

[0118] Optionally, for the spatial index query in step S104, a further problem that needs to be solved is that during large-scale point cloud GIS+BIM fusion roaming, even if the node bounding volume of a certain index node intersects with the spatiotemporal visible envelope, the spatial area corresponding to that node may still be stably occluded by building shells, terrain surfaces, or floor slabs within a preset time window; if the sub-level is continued to be explored according to the conventional hierarchical traversal, a large number of repeated intersection judgments and invalid candidate block outputs of "inevitably invisible areas" will be generated, which will lead to increased CPU traversal overhead, I / O request redundancy, and loading queue congestion.

[0119] To this end, this alternative implementation introduces a barrier geometry proxy to characterize the building shell and / or terrain surface, in addition to the intersection determination, and performs barrier determination on candidate nodes based on the predicted viewpoint state sequence to generate a stable barrier certificate that can be used for pruning, thereby reducing unnecessary down-exploration traversal while maintaining conservative correctness.

[0120] In one optional implementation, a hierarchical traversal is performed starting from the root node in the hierarchical point cloud index structure. Exemplarily, the hierarchical point cloud index structure can be an octree or a BVH hierarchical bounding body tree, where each index node corresponds to a node bounding body (e.g., an AABB bounding box or bounding sphere) and a list of child nodes. During the traversal, for the current index node, the system performs an intersection determination based on the node bounding body corresponding to that index node and the spatiotemporal visual envelope. The intersection determination can be implemented using engineered, rapid testing methods, such as a coarse screening test of "intersection between the node bounding box and the bounding box outside the envelope" and a fine screening test of "intersection between the node bounding box and the set of child bodies formed by the envelope". In response to a non-intersection determination, the system directly prunes the sub-levels corresponding to the current index node and no longer visits its child nodes, thereby avoiding meaningless traversal.

[0121] In response to an intersection determination, instead of immediately probing down to child nodes, the system first performs an obstruction determination on at least two predicted viewpoint states corresponding to the predicted viewpoint state sequence based on the obstruction geometry proxy, and determines a stable obstruction certificate based on the obstruction determination result. The obstruction geometry proxy here provides a low-complexity occlusion representation that allows for rapid ray intersection. It can be constructed from the following data sources and either permanently or cached in chunks at runtime: First, a low-resolution elevation grid generated from GIS terrain data (e.g., DEM / DSM), where grid cells can serve as obstruction objects and have grid cell identifiers; Second, a simplified collision grid or collision bounding body set generated from BIM building shells or interior component shells, where patches or bounding bodies can serve as obstruction objects and have patch identifiers / component identifiers; Third, a simplified set of planar patches generated for large planar components such as floor slabs and walls, where patches can serve as obstruction objects and have floor slab patch identifiers.

[0122] For example, an "OccluderProxyModel" data object can be maintained, which consists of two parts: a proxy geometry array and an acceleration structure. The proxy geometry array stores the geometric descriptions of terrain grid cells, building shell patches, or floor slab patches and their blocking object identifiers. The acceleration structure can use BVH, raster hash, or block index to keep the query overhead of ray intersections within a controllable range.

[0123] In one optional implementation, a stable occlusion certificate is used to express that "in a series of consecutive predicted viewpoint states in the short future, the spatial region represented by the current index node is stably occluded by the same occluder object (or a set of occluder objects within a consistency threshold)," thereby allowing the system to conservatively prune the node sub-level within that time window. An "OcclusionCertificate" structure or object can be defined, including at least: nodeId (the current index node identifier), a set of occluderIds (occluder object identifiers or a set of identifiers), a K value, a certificate generation timestamp, and an optional valid time range, such as covering the first K time offsets of the predicted sequence. This certificate can be used only within the current query period or briefly reused in adjacent frames; when reused, a very short expiration condition can be set (e.g., viewpoint state changes exceeding a threshold or occluder object identifiers no longer being consistent) to avoid unintended pruning.

[0124] When a stable occlusion certificate is valid, the system prunes the sub-levels corresponding to the current index node, meaning it no longer performs intersection checks and downward traversals on the child nodes of that node. When a stable occlusion certificate is invalid, the system repeats the intersection checks and stable occlusion certificate determination on the child nodes corresponding to the current index node until a leaf node is reached. The data block corresponding to the leaf node that intersects with the spatiotemporal visible envelope and has not been pruned by the stable occlusion certificate is then identified as the target point cloud data block. By placing the stable occlusion check before the downward traversal, this optional implementation can truncate a large number of intersecting but stably invisible subtrees at higher levels, thereby significantly reducing the number of traversed nodes and the size of the candidate blocks.

[0125] In another alternative implementation, the system determines the stable blocking certificate as follows. First, for at least K consecutive predicted viewpoint states in the predicted viewpoint state sequence (e.g., selecting K consecutive moments from near to far according to the prediction step size), the system constructs a line-of-sight bundle pointing towards the node bounding body, starting from the corresponding predicted position. A line-of-sight bundle refers to a set of representative rays pointing from the predicted position towards the node bounding body, used to reduce accidental misjudgments of single rays at edges. The line-of-sight bundle includes at least two types of directions: one pointing towards the center point of the node bounding body; and the other pointing towards a preset extreme point of the node bounding body. The preset extreme points can be determined according to the node bounding body type: when the node bounding body is AABB, the preset extreme points can be selected from several corner points of the bounding box or extreme points along the principal axis direction, such as the four corner points on the front side; when the node bounding body is a bounding sphere, the preset extreme points can be selected from several extreme direction points on the sphere orthogonal to the line of sight; when the node bounding body is OBB, the preset extreme points can be selected from corner points in the OBB local coordinate system. To control computational load, the number of rays in the line of sight can be set to the order of 2 to 6, for example, 5 rays in the direction of the center point and the four corner points; and the number of rays can be reduced when the nodes are far away or the layer is coarser to maintain real-time performance.

[0126] Secondly, the system calculates the first intersection distance between the line of sight and the blocking geometry agent based on the blocking geometry agent, and the second intersection distance between the line of sight and the node bounding body. The first intersection distance can be understood as the distance at which the ray first intersects the blocking geometry agent along a ray starting from the predicted position; this distance can be returned by the ray projection interface, along with the blocking object identifier (e.g., terrain mesh cell identifier, building shell patch identifier, or floor slab patch identifier). The second intersection distance can be understood as the distance at which the ray first intersects the node bounding body along the same ray; this distance can be obtained by a fast intersection algorithm between the ray and the bounding box / bounding sphere, with stable computational overhead and independent of point cloud details. In engineering implementation, the system can maintain a "RayHitInfo" record for each ray, including rayId, d1 (first intersection distance), occluderId (blocking object identifier), d2 (second intersection distance), and a hit flag. Ray intersections that block geometric proxies can be implemented using CPU-side acceleration libraries or engine collision modules. For example, on the desktop, a C++ collision query module can be used to establish a BVH for the proxy mesh; on the web, a ray-triangle intersection module encapsulated by WebAssembly can be used, or a worker thread can be used to perform block ray testing to avoid blocking the main thread.

[0127] Furthermore, the system applies a consistent determination rule to at least K consecutive predicted viewpoint states: If each of the at least K consecutive predicted viewpoint states satisfies the condition that the first intersection distance is less than the second intersection distance, and the identifier of the obstructing object corresponding to the first intersection remains consistent or falls within a preset consistency threshold range across the at least K consecutive predicted viewpoint states, then a stable obstruction certificate is determined to be valid. Here, "the first intersection distance is less than the second intersection distance" indicates that at the predicted position, the obstructing object is located in front of the node's bounding volume, thus the node is occluded at that predicted position; "the identifier of the obstructing object remains consistent or falls within a consistency threshold range" is used to suppress identifier jitter caused by mesh tiling, patch boundaries, or terrain raster discretization.

[0128] For example, the consistency threshold range can be set according to a majority consistency approach: in K predicted viewpoint states, if the proportion of occurrences of the same obstruction object identifier is not less than 0.7 to 0.9, then the consistency threshold is satisfied. In terrain mesh scenarios, the same obstruction object can be extended to the same mesh cell or its adjacent mesh cell set to adapt to the case where the ray landing point crosses the cell boundary. In building shell scenarios, the consistency of facet identifiers can be extended to the consistency of the shell identifiers of the same component to adapt to the differences in triangulation of simplified meshes. The setting of K value can be combined with the prediction time window and prediction step size. For example, if the prediction step size is 50ms and it is desired that stable occlusion determination covers about 200ms, then K can be 4; if the prediction step size is 100ms and it is desired that it covers about 300ms, then K can be 3. If K is too small, it is easy to misjudge instantaneous occlusion as stable occlusion. If K is too large, it will reduce the pruning trigger rate and increase the computational overhead. Therefore, it can be configured in combination with platform performance and typical roaming speed.

[0129] In one alternative implementation, the input for the blocking determination can be explicitly defined as "predicted viewpoint state subsequence + node bounding volume + blocking geometric proxy", and the output can be a stable blocking certificate or a non-valid flag.

[0130] For example, the blocking determination function can accept the following parameters: the first K PredictedViewStates in PredictedSequence (including predicted position and time offset), the current node's boundingVolume, and the handle of the accelerated structure of OccluderProxyModel. It returns: the isOccludedStable boolean value, the dominantOccluderId, and a confidence count, such as the consistency percentage. If isOccludedStable is true, an OcclusionCertificate is generated, and the child levels of that node are pruned directly; if false, intersection determination and certificate confirmation are performed on the child nodes.

[0131] For example, in an outdoor urban point cloud roaming scenario, the blocking geometry proxy can consist of a low-resolution elevation mesh and a simplified collision mesh of building shells: when the predicted viewpoint moves at high speed along the street valley direction, many nodes facing away from the street valley, although intersecting with the bounding box outside the spatiotemporal visible envelope, will be permanently occluded by the building shell in front; after the system constructs line-of-sight bundles for these nodes at K consecutive predicted positions and calculates the first / second intersection distance, it can determine that the blocking object identifier is stable and consistent, thereby generating a stable blocking certificate and pruning the subtree, reducing subsequent traversal and candidate block output. In an indoor BIM corridor scenario, the blocking geometry proxy can consist of simplified collision bounding volumes of walls / floors: when the predicted viewpoint turns rapidly near a corridor corner, nodes in the room area outside the corridor will be stably occluded by the walls; through the stable blocking certificate, frequent attempts to load deep point cloud blocks in the room area can be avoided, thereby reducing network / disk jitter and improving roaming continuity. The above implementations do not rely on precise occlusion calculations at the point cloud detail level. Instead, they utilize the temporal consistency of fast-intersecting barrier geometric proxies and prediction sequences to achieve stable pruning effects, which meet the requirements of real-time systems.

[0132] Based on the same inventive concept, this application also provides a real-time hierarchical rendering system for point cloud data, corresponding to a real-time hierarchical rendering method for point cloud data. Since the principle of the system in this application is similar to the real-time hierarchical rendering method for point cloud data described above, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.

[0133] Reference Figure 4 The diagram shown is a schematic of a real-time hierarchical rendering system for point cloud data provided in an embodiment of this application. The system includes: The acquisition module 10 is used to acquire the current viewpoint state of the virtual viewpoint and motion parameters used to characterize the motion trend of the virtual viewpoint in response to changes in the motion state of the virtual viewpoint. The first processing module 20 is used to generate a predicted viewpoint state sequence of the virtual viewpoint within a preset time window based on the motion parameters. The second processing module 30 is used to construct a spatiotemporal visual envelope based on the predicted viewpoint state sequence, wherein the spatiotemporal visual envelope is obtained by combining multiple local viewpoint sub-volumes distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local viewpoint sub-volume changes with the turning intensity index at the corresponding prediction time; wherein the turning intensity index is determined based on the rate of change of the line of sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence. The rendering module 40 is used to use the spatiotemporal visual envelope as a spatial index query condition to determine the target point cloud data block that intersects with the spatiotemporal visual envelope in the hierarchical point cloud index structure; load and perform hierarchical rendering on the target point cloud data block.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for real-time hierarchical rendering of point cloud data, characterized in that, include: In response to changes in the motion state of the virtual viewpoint, the current viewpoint state of the virtual viewpoint and motion parameters used to characterize the motion trend of the virtual viewpoint are obtained. Based on the motion parameters, a predicted viewpoint state sequence of the virtual viewpoint is generated within a preset time window; A spatiotemporal visual envelope is constructed based on the predicted viewpoint state sequence, wherein the spatiotemporal visual envelope is obtained by combining multiple local viewpoint sub-volumes distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local viewpoint sub-volume changes with the turning intensity index at the corresponding prediction time; wherein the turning intensity index is determined based on the rate of change of line of sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence. Using the spatiotemporal visual envelope as a spatial index query condition, the target point cloud data block intersecting with the spatiotemporal visual envelope is determined in the hierarchical point cloud index structure; Load and perform hierarchical rendering on the target point cloud data block.

2. The method according to claim 1, characterized in that, The step of generating the predicted viewpoint state sequence of the virtual viewpoint within a preset time window includes: Obtain a sparse geometric proxy model to characterize the reachable boundaries of the current scene; Based on the current viewpoint state, a forward prediction direction is determined, and a forward safety margin is calculated in the sparse geometry proxy model along the forward prediction direction. The forward safety margin characterizes the reachability gap of the virtual viewpoint to the nearest geometric obstacle along the forward prediction direction. Based on the forward safety margin, boundary constraints are applied to the predicted motion increment within the preset time window to generate a predicted position sequence that does not cross the geometric obstacle, and the predicted viewpoint state sequence is determined based on the predicted position sequence.

3. The method according to claim 2, characterized in that, The applied boundary constraints include: The collision distance index is determined based on the forward safety margin and the motion parameters. In response to the collision distance index being less than a preset safety threshold, a constraint correction is triggered on the predicted motion increment, and when the collision distance index is greater than or equal to the preset safety threshold, the inertial predicted motion increment based on the motion parameters is maintained.

4. The method according to claim 2, characterized in that, The sparse geometry proxy model for obtaining the model includes at least one of the following: A depth proxy generated from the depth buffer of the previous rendered frame; Low-resolution elevation grid generated from GIS terrain data; Simplified collision enclosure generated from the shell of BIM components; A set of coarse-level bounding volumes generated from a point cloud hierarchical index structure.

5. The method according to claim 3, characterized in that, The constraint correction includes: Determine the local surface normal corresponding to the nearest geometric obstacle; The environmental repulsion correction amount is determined based on the local surface normal, and the environmental repulsion correction amount is superimposed on the inertial predicted motion increment; The predicted motion increment after superposition correction is iteratively updated with discrete time steps, and a non-crossing constraint is applied to the predicted position at each discrete time step to generate the predicted position sequence. The determination of the local surface normal includes: when the sparse geometric proxy model includes a depth proxy, determining the local surface normal based on the depth change rate of the depth proxy; when the sparse geometric proxy model includes a simplified collision bounding body, determining the local surface normal based on the geometric patch normal of the simplified collision bounding body.

6. The method according to claim 1, characterized in that, The local view subvolume is a kissing cone; the construction of the spatiotemporal visual envelope based on the predicted viewpoint state sequence includes: For each predicted viewpoint state in the predicted viewpoint state sequence, a corresponding cutting cone is constructed with the corresponding predicted position as the cone apex and the corresponding predicted line of sight as the cone axis. At least one geometric parameter of the cutting cone is determined based on the steering strength index at the corresponding prediction time. The spatiotemporal visual envelope is obtained by combining multiple kissing cones constructed along the predicted viewpoint state sequence.

7. The method according to claim 6, characterized in that, Determining at least one geometric parameter of the cutting cone based on the steering strength index includes: The angle adjustment amount of the cutting cone is determined based on the steering strength index; Obtain scene openness parameters that characterize the openness of the visible corridor at the predicted location, and determine the upper limit of the angle of the kissing cone based on the scene openness parameters, wherein the scene openness parameters are calculated by a geometric proxy used to characterize the terrain elevation surface, building shell and / or indoor passage structure, and used to characterize the reachable gap width and / or openness on both sides along the cone axis direction. The final angle of the cutting cone is determined based on the angle adjustment amount and the upper limit of the angle.

8. The method according to claim 1, characterized in that, The step of using the spatiotemporal visual envelope as a spatial index query condition to determine the target point cloud data block intersecting with the spatiotemporal visual envelope in the hierarchical point cloud index structure includes: Starting from the root node of the hierarchical point cloud index structure, a hierarchical traversal is performed. For the current index node, an intersection determination is performed based on the node bounding volume corresponding to the index node and the spatiotemporal visual envelope. In response to the intersection determination being non-intersection, the sub-level corresponding to the current index node is pruned; In response to the intersection determination being an intersection, based on the barrier geometry proxy used to characterize the building shell and / or terrain surface, a barrier determination is performed on at least two predicted viewpoint states corresponding to the predicted viewpoint state sequence, and a stable barrier certificate is determined based on the barrier determination result. In response to the establishment of the stability barrier certificate, prune the sub-level corresponding to the current index node; and In response to the failure of the stable isolation certificate, the intersection determination and stable isolation certificate determination are repeatedly performed on the child nodes corresponding to the current index node until the leaf node is traversed, so as to determine the data block corresponding to the leaf node that intersects with the spatiotemporal visual envelope and has not been pruned by the stable isolation certificate as the target point cloud data block.

9. The method according to claim 8, characterized in that, The determination of the stable barrier certificate includes: For at least K consecutive predicted viewpoint states in the predicted viewpoint state sequence, a line of sight is constructed pointing to the node bounding body, starting from the corresponding predicted position. The line of sight includes at least two types of directions, namely, the direction pointing to the center point of the node bounding body and the direction pointing to the preset extreme point of the node bounding body. Based on the blocking geometry agent, calculate the first intersection distance between the line of sight and the blocking geometry agent, and calculate the second intersection distance between the line of sight and the node bounding volume; In response to each of the at least K consecutive predicted viewpoint states satisfying that the first intersection distance is less than the second intersection distance, and the blocking object identifier corresponding to the first intersection remains consistent or falls within a preset consistency threshold range in the at least K consecutive predicted viewpoint states, the stable blocking certificate is determined to be established; wherein, the blocking object identifier is used to characterize at least one of the terrain grid cell identifier, building shell patch identifier, and / or floor slab patch identifier in the blocking geometry agent.

10. A real-time hierarchical rendering system for point cloud data, characterized in that, include: The acquisition module is used to acquire the current viewpoint state of the virtual viewpoint and motion parameters used to characterize the motion trend of the virtual viewpoint in response to changes in the motion state of the virtual viewpoint. The first processing module is used to generate a predicted viewpoint state sequence of the virtual viewpoint within a preset time window based on the motion parameters. The second processing module is used to construct a spatiotemporal visual envelope based on the predicted viewpoint state sequence, wherein the spatiotemporal visual envelope is obtained by combining multiple local viewpoint sub-volumes distributed along the predicted viewpoint state sequence, and at least one geometric parameter of the local viewpoint sub-volume changes with the turning intensity index at the corresponding prediction time; wherein the turning intensity index is determined based on the rate of change of the line of sight direction and / or the rate of change of position between adjacent predicted viewpoint states in the predicted viewpoint state sequence. The rendering module is used to use the spatiotemporal visual envelope as a spatial index query condition to determine the target point cloud data block that intersects with the spatiotemporal visual envelope in the hierarchical point cloud index structure; load and perform hierarchical rendering on the target point cloud data block.

Images