A Method and System for Loading 3D Models from Oblique Photogrammetry Based on Dynamic Trajectory Prediction

Abstract

This application relates to a method and system for loading 3D models using oblique photogrammetry based on dynamic trajectory prediction. The method includes the following steps: real-time acquisition of camera motion parameters and identification of camera motion modes; acquisition of moving region information and generation of a first preloaded tile set using a fan-shaped region prediction algorithm; generation of predicted spatiotemporal position information using a motion prediction model and collision detection with the scene octree index to generate a second preloaded tile set; acquisition of current load information and weight allocation to generate a preload list; identification of the priority order in the preload list and loading the 3D model target tiles into video memory using a multi-level cache scheduling mechanism. In summary, this application, by combining dynamic trajectory prediction with a multi-level cache scheduling mechanism, achieves intelligent response and efficient data loading for complex camera motion modes, significantly improving data loading efficiency and rendering smoothness in complex motion scenes.

Description

Technical Field

[0001] This application relates to the technical field of 3D model loading, and in particular to a method and system for loading 3D models by oblique photogrammetry based on dynamic trajectory prediction. Background Technology

[0002] In real-time visualization applications such as smart cities and 3D reality, the smooth loading and rendering of massive amounts of oblique photogrammetry 3D model data is a key challenge. Currently, mainstream preloading techniques typically rely on camera position and velocity information, predicting changes in the field of view through simple geometric extrapolation and preloading relevant data. These methods are relatively effective in scenarios with stable flight paths and slow directional changes. However, in complex real-world applications, such as drone inspections and virtual tours, camera motion is often rapidly changing, involving complex maneuvers like acceleration, deceleration, and sharp turns, exposing the limitations of traditional preloading methods.

[0003] First, because the dynamic characteristics of motion are not fully considered, prediction models based solely on instantaneous speed will experience a sharp drop in accuracy during maneuvers, leading to the misloading or failure to load large amounts of data in a timely manner, resulting in noticeable screen stuttering and model loss. Second, existing technologies lack the ability to intelligently identify and adapt to different motion modes, and cannot dynamically adjust prediction strategies according to the complexity of motion. Finally, the data scheduling mechanisms and cache management strategies of existing technologies are relatively static, making it difficult to deeply correlate the predictive value of data with cache priority, resulting in low overall resource utilization efficiency.

[0004] The aforementioned shortcomings severely restrict the user experience of large-scale 3D scenes in highly dynamic and interactive applications, especially in situations requiring real-time response and high-quality rendering, such as emergency command and flight simulation. Summary of the Invention

[0005] To address the aforementioned shortcomings, this application provides a method and system for loading 3D models using oblique photogrammetry based on dynamic trajectory prediction.

[0006] The above-mentioned objective of this application is achieved through the following technical solution:

[0007] A method for loading a 3D model from oblique photogrammetry based on dynamic trajectory prediction, comprising the following steps:

[0008] The camera motion parameters are acquired in real time, and the camera motion mode is identified based on the camera motion parameters, including position, velocity, acceleration and angular velocity.

[0009] Obtain moving area information, and based on camera motion parameters and moving area information, generate a first preloaded tile set using a preset sector region prediction algorithm;

[0010] Based on camera motion parameters and moving area information, predicted spatiotemporal location information is generated through a pre-trained motion prediction model. Collision detection is performed between the predicted spatiotemporal location information and the pre-built scene octree index to generate a second preloaded tile set.

[0011] Obtain current load information, and assign weights to the first preloaded tile set and the second preloaded tile set based on the camera motion mode and the current load information, and generate a preload list;

[0012] The priority order in the preload list is identified, and the corresponding 3D model target tiles are loaded into the video memory through a pre-set multi-level cache scheduling mechanism.

[0013] The multi-level cache scheduling mechanism includes: firstly searching for the target tile in the video memory cache; if the target tile is not found, then searching for the target tile in the memory cache and promoting it to video memory; if the target tile is still not found, then loading it from the storage medium into the memory cache, and determining whether to forward it to video memory according to the priority order.

[0014] In a preferred embodiment, this application can be further configured as follows: the step of acquiring moving region information and generating a first preloaded tile set based on camera motion parameters and the moving region information using a preset sector region prediction algorithm includes the following steps:

[0015] The three-dimensional sector prediction region is determined based on the position and velocity parameters in the camera motion parameters;

[0016] The three-dimensional sector prediction region is optimized based on the moving region information to generate an enhanced prediction region;

[0017] Identify all 3D model tiles whose spatial location is within the enhanced prediction region, and generate a first preloaded tile set based on the identified 3D model tiles.

[0018] In a preferred embodiment, this application can be further configured as follows: the step of optimizing the three-dimensional sector prediction region based on the moving region information to generate an enhanced prediction region includes the following steps:

[0019] Feature extraction is performed on the mobile area information to obtain passable area features and impassable area features. The passable area features include airspace constraint features and mobility constraint features.

[0020] Based on the characteristics of impassable regions, a spatial Boolean subtraction operation is performed on the 3D sector prediction region to obtain the initial optimized region;

[0021] Based on the spatial constraint features, a spatial Boolean intersection operation is performed on the initial optimization region to generate an intermediate optimization region;

[0022] Based on the mobility constraint feature domain, the accessibility of the intermediate optimization region is corrected to generate an enhanced prediction region.

[0023] In a preferred embodiment, this application can be further configured as follows: the step of generating predicted spatiotemporal location information based on camera motion parameters and moving area information using a pre-trained motion prediction model, and performing collision detection between the predicted spatiotemporal location information and a pre-constructed scene octree index to generate a second preloaded tile set includes the following steps:

[0024] Based on the position, velocity, acceleration, and angular velocity in the camera motion parameters, spatiotemporal trajectory information is generated through a pre-trained motion prediction model;

[0025] Using the mobile area information as a spatial constraint, the spatiotemporal trajectory information is modified for drivability, so as to exclude trajectory points located in impassable areas and obtain modified trajectory information.

[0026] The corrected trajectory information is mapped into a three-dimensional probability space volume that represents the spatial range of the future camera, and this three-dimensional probability space volume is used as the predicted spatiotemporal location information.

[0027] The predicted spatiotemporal location information is traversed layer by layer and collision detection is performed with the pre-built scene octree index to retrieve all intersecting octree nodes as target nodes.

[0028] Obtain the identifiers corresponding to the 3D model tiles associated with all target nodes, and generate a second preloaded tile set based on the identifiers.

[0029] In a preferred embodiment, this application can be further configured as follows: the step of using the mobile area information as a spatial constraint to correct the drivability of the spatiotemporal trajectory information to exclude trajectory points located in impassable areas, thereby obtaining corrected trajectory information, includes the following steps:

[0030] When an obstacle avoidance command is received from the flight control terminal, the command type and avoidance direction vector are identified. The command type includes emergency obstacle avoidance and normal avoidance.

[0031] Several auxiliary obstacle avoidance trajectories are generated based on the avoidance direction vector, and the auxiliary obstacle avoidance trajectories are adaptively adjusted based on the instruction type to obtain the optimized obstacle avoidance trajectory;

[0032] The generated set of trajectory points for the optimized obstacle avoidance trajectory is fused with the set of trajectory points for the spatiotemporal trajectory information after accessibility correction to generate corrected trajectory information.

[0033] In a preferred embodiment, this application can be further configured as follows: the step of mapping the corrected trajectory information into a three-dimensional probability space volume characterizing the spatial range of the future camera's existence, and using this three-dimensional probability space volume as the predicted spatiotemporal location information, includes the following steps:

[0034] Obtain the confidence level of camera motion state, and perform uncertainty modeling on discrete trajectory points in the corrected trajectory information based on the confidence level of camera motion state to generate several uncertainty ellipsoids;

[0035] The uncertain ellipsoid is transformed into a local probability density distribution, and a continuous spatiotemporal probability density field is generated based on the local probability density distribution.

[0036] Using the impassable areas in the mobile area information as hard constraints, the probability values ​​of those located within the impassable areas in the spatiotemporal probability density field are reset to zero to generate an optimized probability density field.

[0037] Connected regions with probability values ​​greater than a preset threshold are extracted from the optimized probability density field to form a three-dimensional probability space.

[0038] In a preferred embodiment, this application can be further configured as follows: the step of obtaining the camera motion state confidence level and performing uncertainty modeling on the discrete trajectory points in the corrected trajectory information based on the camera motion state confidence level to generate several uncertainty ellipsoids includes the following steps:

[0039] The signal quality of camera motion parameters is evaluated to obtain the signal confidence factor;

[0040] Obtain the model confidence factor of the motion prediction model, and calculate the camera motion state confidence based on the signal confidence factor and the model confidence factor;

[0041] The generation parameters of the uncertain ellipsoid are generated based on the confidence level of the camera motion state, and the generation parameters include size and orientation;

[0042] Each discrete trajectory point in the corrected trajectory information is used as the center, and the corresponding uncertain ellipsoid is generated by combining the generation parameters.

[0043] In a preferred embodiment, this application can be further configured as follows: the step of generating spatiotemporal trajectory information based on the position, velocity, acceleration, and angular velocity in the camera motion parameters using a pre-trained motion prediction model includes the following steps:

[0044] Acquire signal quality information, including normal, degraded, and lost signals;

[0045] When signal quality information is lost, the camera motion history sequence is obtained based on a preset sequence selection strategy;

[0046] The camera motion history sequence is input into a pre-trained temporal behavior recognition model to extract spatiotemporal features, and the dominant behavior pattern is identified based on the spatiotemporal features.

[0047] The dominant behavior pattern and camera motion history sequence are input into a pre-trained motion prediction model to generate spatiotemporal trajectory information.

[0048] The second objective of this invention is achieved through the following technical solution:

[0049] A system for loading oblique photogrammetric 3D models based on dynamic trajectory prediction, comprising:

[0050] The motion pattern recognition module is used to acquire camera motion parameters in real time and identify camera motion patterns based on the camera motion parameters, including position, velocity, acceleration and angular velocity.

[0051] The first tile set generation module is used to acquire moving area information and generate a first preloaded tile set based on camera motion parameters and moving area information using a preset sector region prediction algorithm.

[0052] The second tile set generation module is used to generate predicted spatiotemporal location information based on camera motion parameters and moving area information through a pre-trained motion prediction model, and to perform collision detection between the predicted spatiotemporal location information and the pre-built scene octree index to generate a second preloaded tile set.

[0053] The load list generation module is used to obtain the current load information, and to perform weight allocation on the first preloaded tile set and the second preloaded tile set based on the camera motion mode and the current load information, and generate a preload list;

[0054] The target tile loading module is used to identify the priority order in the preload list and load the corresponding 3D model target tiles into the video memory through a pre-set multi-level cache scheduling mechanism. The multi-level cache scheduling mechanism includes: firstly searching for the target tile in the video memory cache; if it is not found, then searching for the target tile in the memory cache and promoting it to the video memory; if it is still not found, then loading it from the storage medium into the memory cache, and determining whether to forward it to the video memory according to the priority order.

[0055] In a preferred embodiment, this application can be further configured such that the first tile set generation module includes:

[0056] The region determination submodule is used to determine the 3D sector prediction region based on the position and velocity in the camera motion parameters;

[0057] The region optimization submodule is used to optimize the 3D sector prediction region based on the moving region information and generate an enhanced prediction region.

[0058] The tile recognition submodule is used to identify all 3D model tiles whose spatial location is within the enhanced prediction area, and to generate a first preloaded tile set based on the identified 3D model tiles.

[0059] In summary, the method and system for loading oblique photogrammetry 3D models based on dynamic trajectory prediction provided in this application, by combining dynamic trajectory prediction with a multi-level cache scheduling mechanism, achieves intelligent response and efficient data loading for complex camera motion patterns, and has the advantages of significantly improving data loading efficiency and rendering smoothness in complex motion scenes. Attached Figure Description

[0060] Figure 1 This is a flowchart of an embodiment of an oblique photogrammetry 3D model loading method based on dynamic trajectory prediction according to this application;

[0061] Figure 2 This is a flowchart of step S20 in an embodiment of an oblique photogrammetry 3D model loading method based on dynamic trajectory prediction in this application. Detailed Implementation

[0062] The following is in conjunction with the appendix Figures 1-2 This application will be described in further detail.

[0063] In one embodiment, such as Figure 1 As shown, this application discloses a method for loading a 3D model from oblique photogrammetry based on dynamic trajectory prediction, which specifically includes the following steps:

[0064] S10: Acquire camera motion parameters in real time and identify camera motion modes based on camera motion parameters, including position, velocity, acceleration and angular velocity;

[0065] In this embodiment, camera motion parameters are a set of instantaneous physical quantities or composite motion characteristics that describe the camera's motion state in three-dimensional space. Specifically, they can be implemented using a fusion algorithm of inertial measurement unit and visual odometry to accurately describe the camera's instantaneous motion state. The camera motion parameters include position (three-dimensional coordinates), velocity (speed and direction of motion), acceleration (rate of change of velocity), and angular velocity (speed of turning). The camera motion mode is the result of high-level abstraction and classification of the camera's complex motion state. Based on the combined characteristics of the motion parameters, modes such as uniform linear mode, variable linear mode, and complex direction-changing mode can be identified.

[0066] S20: Obtain moving area information, and generate a first preloaded tile set based on camera motion parameters and moving area information using a preset sector region prediction algorithm;

[0067] In this embodiment, the mobile area information is spatial topology data that includes the boundaries of passable areas, the distribution of obstacles, and the conditions for maneuvering. Specifically, it can be extracted jointly by a digital elevation model and a semantic segmentation network to provide physical constraints for the predicted area. The fan-shaped area prediction algorithm is a method of constructing a three-dimensional fan-shaped space with the current position of the camera as the vertex and the velocity direction as the central axis. Specifically, it can be implemented using a dynamic angle adjustment mechanism, where the angle size is positively correlated with the velocity amplitude.

[0068] S30: Based on camera motion parameters and moving area information, predictive spatiotemporal location information is generated through a pre-trained motion prediction model, and collision detection is performed between the predicted spatiotemporal location information and the pre-built scene octree index to generate a second preloaded tile set.

[0069] In this embodiment, the motion prediction model is a machine learning model used to predict future spatiotemporal trajectories based on historical motion sequences. Specifically, it can be implemented using a spatiotemporal graph convolutional neural network architecture, which can capture nonlinear features in complex motion patterns. The predicted spatiotemporal location information is the output of the motion prediction model, which is a probabilistic description of the possible spatial location and time of the camera within a future period, such as a three-dimensional probabilistic space volume. The scene octree index is a multi-resolution spatial index structure established for massive oblique photogrammetry 3D model data. It recursively divides the 3D space into eight sub-cubes, greatly accelerating the efficiency of spatial range query and collision detection. Collision detection involves performing a spatial intersection test between the predicted spatiotemporal location information (such as the probabilistic space volume) and the scene octree index to quickly retrieve all 3D model tiles that may be located within the future field of view.

[0070] S40: Obtain current load information, and assign weights to the first preloaded tile set and the second preloaded tile set based on the camera motion mode and the current load information, and generate a preloaded list;

[0071] In this embodiment, the current load information is the real-time resource occupancy status of the system, including CPU utilization, memory usage, remaining video memory space, disk I / O throughput, etc., which is an important basis for the system to make dynamic decisions. The weight allocation is to assign appropriate weight ratios to the results (first and second preloaded tile sets) of the two prediction channels (sector algorithm and octree retrieval) based on the motion mode (decision basis) and the current load (resource constraint). Specifically, a fuzzy logic controller can be used to ensure that critical areas are prioritized for loading under high load conditions. The preloaded list is a queue of tiles to be loaded that is generated after weight allocation and fusion and sorted by priority. The order of the preloaded list takes into account factors such as prediction weight and system load.

[0072] S50: Identify the priority order in the preload list and load the corresponding 3D model target tile into the video memory through a pre-set multi-level cache scheduling mechanism; wherein, the multi-level cache scheduling mechanism includes: firstly searching for the target tile in the video memory cache; if not found, searching for the target tile in the memory cache and promoting it to the video memory; if still not found, loading it from the storage medium into the memory cache, and determining whether to forward it to the video memory according to the priority order.

[0073] In this embodiment, the priority order is the rule that determines the order of data loading and cache replacement, determined by the preload list. Its core is based on dynamic prediction of future visibility, rather than the traditional LRU principle. The multi-level cache scheduling mechanism is a strategy for data retrieval and migration according to the hierarchical structure of video memory-memory-storage media. Specifically, the Least Recently Used algorithm can be used to maintain the cache queue. Its core goal is to ensure that the highest priority data arrives in video memory with minimal latency, and preloading is achieved through asynchronous data transmission pipelines. The 3D model target tile is the management and scheduling unit of oblique photogrammetry 3D model data. It divides the large-scale 3D scene into a series of small tiles, which is the basis for realizing the dynamic loading of massive data.

[0074] Specifically, the system acquires motion parameters such as camera position, velocity, acceleration, and angular velocity in real time and identifies the camera's motion pattern. Then, it activates dual prediction channels in parallel: on one hand, based on the current motion parameters and moving area information, a lightweight sector region prediction algorithm quickly generates a first set of preloaded tiles. This step uses spatial Boolean operations to ensure the passability of the predicted area. On the other hand, based on the same input, a pre-trained motion prediction model generates predicted spatiotemporal position information, and collision detection is performed between this predicted spatiotemporal position information and the scene octree index to generate a second set of preloaded tiles. The system also acquires current CPU and memory load information and, based on the identified... The motion pattern and current load are used to dynamically weight and merge the first and second preloaded tile sets, ultimately generating a priority-sorted preloaded list. This process prioritizes efficiency when system resources are scarce and prioritizes accuracy when resources are abundant and motion is complex. Finally, based on the priority order in the preloaded list, a multi-level cache scheduling mechanism is driven: this mechanism first searches for the target tile in video memory; if it is not found, it searches for it in memory and promotes it to video memory; if it is still not found, it is loaded from storage medium into memory, and based on the tile's priority, a decision is made on whether to immediately forward it to video memory, thereby ensuring that the highest priority data is always ready with the lowest latency.

[0075] For example, a drone is performing an inspection mission. When it is flying in a straight line at a constant speed, the fan-shaped prediction results are adopted first, and the data of the first pre-loaded tile set is loaded quickly, with extremely low computational overhead. When it suddenly detects an obstacle and begins to make an emergency turn, the camera motion mode changes, and the weight of the second pre-loaded tile set needs to be increased to accurately predict the turning trajectory and load the scene data after the turn in advance. At the same time, if the current memory usage is high, the weight of the second pre-loaded tile set is appropriately reduced to avoid overloading. Finally, all data is intelligently scheduled to the video memory according to priority, ensuring smooth video throughout the entire emergency turn process.

[0076] Compared to existing technologies, traditional methods rely on single geometric extrapolation for prediction, resulting in insufficient coverage in maneuvering scenarios. This solution, however, maintains an effective prediction range even during sharp turns by fusing deterministic sector detection with probabilistic spatial volume detection. Existing static weight allocation strategies struggle to adapt to load fluctuations. This solution dynamically adjusts the preloading ratio by monitoring the status of video memory and computing resources in real time, reducing performance degradation caused by resource contention. Furthermore, compared to fixed-level caching mechanisms, this solution implements differentiated scheduling based on priority, ensuring that high-value tiles reside in video memory first, thereby improving cache hit rate.

[0077] Through the above technical solutions, this application effectively solves the model loading delay problem in complex maneuvering scenarios. When the UAV performs emergency obstacle avoidance and other sudden maneuvers, the dual prediction mechanism ensures the timely loading of tiles in key areas, reducing screen stuttering. The dynamic weight allocation strategy prioritizes the model residence of high-probability visible areas under limited video memory conditions, reducing the proportion of invalid data loading. The multi-level cache scheduling mechanism shortens the transmission delay of high-priority tiles from the storage medium to the video memory through intelligent data migration strategies, improving the smoothness of the user experience.

[0078] In one embodiment, such as Figure 2 As shown, step S20 includes:

[0079] S21: Determine the 3D sector prediction region based on the position and velocity in the camera motion parameters;

[0080] S22: Optimize the 3D sector prediction region based on the moving region information to generate an enhanced prediction region;

[0081] S23: Identify all 3D model tiles whose spatial location is within the enhanced prediction region, and generate a first preloaded tile set based on the identified 3D model tiles.

[0082] In this embodiment, the three-dimensional fan-shaped prediction region is a three-dimensional spatial region extending from the current position of the camera as the vertex and the direction of motion as the axis. Its range can be defined by the look-ahead distance and the fan-shaped angle, specifically achieved using geometric extrapolation combined with velocity vector calculation. This region characterizes the field of view that the camera may cover under inertial motion trends. The moving area information is a set of information or spatial constraint data describing the spatial constraints and rules of the camera's environment. It typically includes a digital terrain model (describing terrain undulations and the height of surface objects), an obstacle distribution map (identifying the precise three-dimensional positions and ranges of obstacles such as buildings, trees, and power lines), and airspace rule data (such as regulatory constraints like no-fly zones, restricted flight zones, and preset flight routes). Specifically, this can be achieved through… The prediction area is obtained through a geofencing system or scene semantic segmentation model and used to correct the spatial boundaries of the prediction area. The enhanced prediction area is a three-dimensional fan-shaped area optimized by spatial constraints. Specifically, it can be achieved by excluding impassable areas through Boolean operations to improve the physical rationality of the prediction area. The three-dimensional model tiles are oblique photogrammetry model data blocks cut according to spatial division rules. Specifically, they can be organized and managed using spatial grid indexes to achieve block loading of large-scale three-dimensional scenes. The first preloaded tile set is a list of tile IDs to be loaded generated by the fan-shaped area prediction algorithm. The first preloaded tile set represents the model data blocks that may enter the field of view in the future based on geometric rules in the current motion state.

[0083] Specifically, during camera movement, a three-dimensional fan-shaped prediction region is first constructed based on the position coordinates and velocity vector. The angular range of this region can be a preset value or a dynamically calculated value. Then, using obstacle distribution data in the moving area information, the prediction region is spatially clipped, for example, removing parts that overlap with buildings. Furthermore, combined with the spatial constraints of the passable area, such as flight altitude restrictions or terrain undulation features, the prediction region is further corrected to form an enhanced prediction region. Finally, the identification information of all three-dimensional model tiles in this region is queried through spatial indexing to form a first preloaded tile set. The first preloaded tile set serves as the basic prediction result and can effectively cover the scene area that the camera may reach under the inertial motion trend.

[0084] For example, in a scenario where a drone is flying at a constant speed of 10 m / s in the due north direction, a three-dimensional fan-shaped prediction region with an angle of 30° and a length of 200 meters is generated in the due north direction with the current position as the vertex; the moving region information is read, and a tall building is identified on the right side of the fan-shaped region, so the fan shape is optimized by removing the part that penetrates the building, and an enhanced prediction region that avoids the building is generated; all three-dimensional model tiles (such as the ground, the building in front, etc.) that are geometrically located within the enhanced prediction region are retrieved, and the IDs of these three-dimensional model tiles form the first preloaded tile set, and then these tiles are loaded preferentially.

[0085] Through the above technical solutions, this application can significantly improve the preloading accuracy in complex scenes, especially in environments with obstacles or spatial restrictions, effectively avoiding redundant loading of 3D model tiles in inaccessible areas; by dynamically optimizing the spatial boundary of the prediction area, it not only ensures the coverage of the basic motion trend, but also reduces the prediction error caused by environmental constraints, thereby reducing the waste of computing resources while ensuring the smoothness of rendering.

[0086] In one embodiment, step S22 includes:

[0087] S221: Extract features from the mobile area information to obtain passable area features and impassable area features. The passable area features include airspace constraint features and mobility constraint features.

[0088] S222: Perform spatial Boolean subtraction on the 3D sector prediction region based on the features of impassable areas to obtain the initial optimized region;

[0089] S223: Perform spatial Boolean intersection operation on the initial optimization region based on spatial constraint features to generate an intermediate optimization region;

[0090] S224: Based on the mobility constraint feature domain, the reachability of the intermediate optimization region is corrected to generate an enhanced prediction region.

[0091] In this embodiment, the passable area feature is the movable spatial range defined by both airspace constraint features and mobility constraint features. Specifically, it can be implemented using geofencing data combined with kinematic model parameters to limit the physical feasible domain of camera movement. The impassable area feature is the prohibited area defined by obstacle detection or preset no-fly zone data. Specifically, it can be implemented using a 3D point cloud segmentation algorithm combined with a digital elevation model to eliminate collision risk areas. Spatial Boolean subtraction operation is used to remove the overlapping part between the predicted area and the impassable area through 3D geometric operations. Specifically, it can be implemented using a voxelized spatial segmentation algorithm to eliminate invalid prediction ranges. Spatial Boolean intersection operation refers to retaining the overlapping part between the predicted area and the airspace constraint through spatial intersection operation. Specifically, it can be implemented using a hierarchical bounding box collision detection algorithm to ensure the physical accessibility of the predicted area. The initial optimized area is the intermediate result obtained after spatial Boolean subtraction operation, which has removed the part that penetrates absolute obstacles such as buildings and mountains, but within the range The initial optimization region may still contain areas that are not legally accessible or physically difficult to maneuver in. The intermediate optimization region is the result obtained after applying spatial constraints (spatial Boolean intersection) to the initial optimization region, ensuring that the predicted region not only avoids physical obstacles but also meets the regulatory requirements for airspace. The accessibility correction is a dynamic feasibility verification of the optimization region based on kinematic parameters. Specifically, it can be implemented using trajectory planning algorithms combined with maximum curvature constraints to ensure that the predicted region meets the camera's maneuverability. For example, even if an airspace is geometrically and legally accessible, if the aircraft cannot complete the required drastic maneuvers, such as sharp turns or high-angle climbs, due to performance limitations, the region will be corrected, such as by smoothing the edges or reducing the range. The enhanced prediction region is a highly reliable final prediction range region obtained after the above three optimizations of physical elimination, regulatory constraints, and dynamic correction. Every point in the enhanced prediction region simultaneously satisfies the following conditions: physically accessible, legally accessible, and dynamically achievable.

[0092] Specifically, after the 3D sector prediction region is generated, spatial Boolean subtraction is first performed using the inaccessible area features in the mobile area information to eliminate invalid spaces containing obstacles or no-fly zones within the prediction region. Then, spatial Boolean intersection is performed using airspace constraint features to restrict the prediction region to the flight permission range. Finally, accessibility analysis is performed on the optimized region in combination with maneuverability constraint features to exclude spatial ranges that exceed the camera's maximum turning angle or acceleration limits, forming the final enhanced prediction region. The above process gradually optimizes the geometry and physical feasibility of the prediction region through multi-level spatial calculations and dynamic verification.

[0093] For example, taking a drone flying in a city as a scenario, features are extracted from a 3D city model. Impassable areas include all building entities, while navigable areas include airspace constraints and maneuverability constraints. Airspace constraints include, for example, low-altitude passages below 200 meters approved by civil aviation authorities, and maneuverability constraints include, for example, the minimum turning radius of the drone being 50 meters. Spatial Boolean subtraction is used to remove all parts intersecting with buildings from the fan-shaped region, generating an initial optimized region. Spatial Boolean intersection is used to intersect the result of the previous step with the low-altitude passage region below 200 meters, eliminating areas above 200 meters, generating an intermediate optimized region. Based on the intermediate optimized region, a narrow right-angle bend is found, with a turning radius requirement of 30 meters, which is less than the 50-meter performance limit for the drone. Therefore, the prediction range of this area is smoothed or reduced accordingly, generating the final enhanced prediction region.

[0094] Through the above technical solutions, this application can accurately identify the feasible prediction area of ​​the camera under complex motion constraints, significantly reduce the preloading ratio of invalid tiles, and improve data scheduling efficiency; in scenarios with dynamic obstacles such as UAV inspection, it can avoid model loading errors caused by misjudgment of environmental constraints and improve the continuity and integrity of 3D scene rendering.

[0095] In one embodiment, step S30 includes:

[0096] S31: Based on the position, velocity, acceleration, and angular velocity in the camera motion parameters, spatiotemporal trajectory information is generated through a pre-trained motion prediction model;

[0097] S32: Use the mobile area information as a spatial constraint to correct the drivability of the spatiotemporal trajectory information, so as to exclude trajectory points located in the impassable area and obtain the corrected trajectory information.

[0098] S33: Map the corrected trajectory information into a three-dimensional probability space volume that represents the spatial range of the future camera, and use this three-dimensional probability space volume as the predicted spatiotemporal location information;

[0099] S34: Perform layer-by-layer traversal and collision detection between the predicted spatiotemporal location information and the pre-constructed scene octree index, and retrieve all intersecting octree nodes as the target node.

[0100] S35: Obtain the identifiers corresponding to the 3D model tiles associated with all target nodes, and generate a second preloaded tile set based on the identifiers.

[0101] In this embodiment, the spatiotemporal trajectory information is the trajectory prediction result obtained by training a kinematic model with historical data. Specifically, it can be implemented using a neural network model based on LSTM or Transformer to capture the nonlinear changes in the camera's motion state. Spatial constraints are environmental rules provided by the moving area information and must be followed. They define the "no-go zones" and "feasible regions" of trajectory planning, i.e., the geometric boundaries of inaccessible areas (such as buildings and mountains). Spatial constraints serve as hard rules to ensure that the prediction results conform to the reality of the physical world. Accessibility correction uses constraints such as terrain obstacles and no-fly zones in the moving area information to filter the predicted trajectory. Specifically, it can be implemented using spatial Boolean operations or geometric clipping algorithms to eliminate prediction errors in physically inaccessible areas. The corrected trajectory information is the reliable spatiotemporal trajectory obtained after accessibility correction, which is the direct basis for generating the final preloaded area. The three-dimensional probability space volume is the spatial range where the future camera may exist, modeled by a probability density field. It describes the probability that the future camera may appear at any location in space. High-probability areas are... The most likely location for the camera to appear is defined, while low-probability areas are even less likely. This can be achieved using Gaussian processes or Monte Carlo methods to quantify the uncertainty of the prediction results. The scene octree index is a data structure that divides the 3D scene into hierarchical spatial units. It recursively divides the entire 3D scene space into eight sub-cubes (called nodes) until each leaf node is associated with a small 3D model tile. This can be implemented using a bounding box-based node segmentation strategy to accelerate spatial queries and collision detection. The layer-by-layer traversal and collision detection algorithm uses the hierarchical structure of the octree for fast queries. Starting from the root node, it recursively checks whether the 3D probability space volume intersects with the bounding box of each octree node, i.e., a collision. If they do not intersect, all subtrees of that node are skipped; if they intersect, the traversal of its child nodes continues until the leaf node. This method avoids traversing the entire scene and reduces computational complexity. The target node is the leaf node of the octree that intersects with the 3D probability space volume, selected through collision detection. The set of tiles associated with the target node is the area that the camera's field of view may cover in the future.

[0102] Specifically, the multidimensional state variables in the camera motion parameters are input into the pre-trained motion prediction model to generate spatiotemporal trajectory information; the boundaries of impassable areas in the moving area information are transformed into spatial masks, and invalid points located inside obstacles in the spatiotemporal trajectory information are filtered out by point-by-point detection; the corrected trajectory point set generates a three-dimensional probability space volume through a probability diffusion algorithm, whose spatial shape can be represented as an ellipsoid or an irregular geometry. When the three-dimensional probability space volume is traversed hierarchically with the scene octree index, the axis-aligned bounding box detection algorithm is used to determine the intersection state of nodes and probability volumes. Finally, the tiles associated with octree nodes whose coverage volume exceeds the threshold are added to the second preloaded tile set.

[0103] For example, taking a drone performing a sharp turn avoidance maneuver as a scenario, the motion prediction model, based on the current high angular velocity, predicts a curved spatiotemporal trajectory. Reading the movement area information reveals that a small section of this trajectory passes through a building, so a passability correction is made, removing the trajectory point that passes through the wall and smoothly conforming the trajectory line to the building surface to bypass it. Considering the uncertainty of control, the corrected trajectory line is mapped to a three-dimensional probability space, with the pipe becoming thicker for sharper turns. Collision detection is performed between the entire three-dimensional probability space and the scene octree to quickly identify all intersecting leaf nodes as target nodes. The tile IDs corresponding to the target nodes are collected to generate a second pre-loaded tile set, which is immediately scheduled for loading, thus ensuring that all potentially visible model data is ready before the drone actually turns the curve.

[0104] Through the above technical solutions, this application solves the problem of insufficient prediction accuracy of traditional preloading methods under camera motion conditions, reduces rendering stuttering and model missing phenomena caused by misloading or missing loading; by integrating motion prediction and spatial constraint dual verification mechanism, the accuracy of preloaded tile sets is effectively improved, and the invalid occupation of video memory and memory resources is reduced.

[0105] In one embodiment, step S32 includes:

[0106] S321: When an obstacle avoidance command is received from the flight control terminal, the command type and avoidance direction vector of the obstacle avoidance command are identified. The command type includes emergency obstacle avoidance and normal avoidance.

[0107] S322: Generate several auxiliary obstacle avoidance trajectories based on the avoidance direction vector, and adaptively adjust the auxiliary obstacle avoidance trajectories based on the instruction type to obtain the optimized obstacle avoidance trajectory;

[0108] S323: The trajectory point set of the generated optimized obstacle avoidance trajectory is fused with the trajectory point set of the spatiotemporal trajectory information after accessibility correction to generate corrected trajectory information.

[0109] In this embodiment, the flight control terminal is the UAV's flight control system or ground control station, which is the issuer of flight decisions. It continuously perceives the environment through radar, visual sensors, etc., and generates corresponding control commands when it detects obstacles or receives avoidance commands. The obstacle avoidance command is a high-level semantic control command issued by the flight control terminal. It is not a low-level control parameter, but an instruction containing the avoidance intention. The instruction type of the obstacle avoidance command is the source of the decision to trigger trajectory correction. Specifically, it can be implemented by using the instruction encoding sent by the flight control system. For example, it can distinguish between emergency obstacle avoidance and normal obstacle avoidance by parsing the priority identifier in the instruction. Obstacle avoidance is categorized into several types. Emergency obstacle avoidance refers to encountering a sudden, close-range obstacle that requires an immediate change in course to avoid a collision; this typically corresponds to high-G maneuvers and high-overload avoidance maneuvers. Regular obstacle avoidance involves smooth, pre-planned adjustments to the course of known static obstacles (such as buildings or mountains) along a pre-planned path; this typically corresponds to smooth, gradual avoidance maneuvers. The avoidance direction vector is the spatial directional parameter of the obstacle avoidance maneuver, which can be represented by a unit vector in a three-dimensional coordinate system, for example, by generating a direction vector from the azimuth and pitch angles of the obstacle obtained from sensors. The auxiliary obstacle avoidance trajectory is an alternative trajectory generated to cope with sudden obstacles. The path can be generated using spline curve interpolation algorithms, producing multiple candidate trajectories. For example, random perturbations can be superimposed on the obstacle avoidance direction vector to generate diverse trajectories. Adaptive adjustment involves dynamically modifying trajectory parameters based on the obstacle avoidance type. This can be achieved by adjusting the trajectory curvature radius or velocity distribution. For instance, in emergency obstacle avoidance mode, the trajectory generation cycle can be shortened to improve response speed. Specifically, for emergency obstacle avoidance, parameters are adjusted to make the generated trajectory shorter or more aggressive, covering more direct escape paths. For conventional obstacle avoidance, parameters are adjusted to make the generated trajectory smoother and longer, conforming to cost-effective route planning. Optimized obstacle avoidance... The trajectory is the final adopted auxiliary obstacle avoidance trajectory obtained after adaptive adjustment. It is one or several high-quality candidate paths that best match the current avoidance intention. The trajectory point set is a collection of discrete three-dimensional spatial points with timestamps. In the subsequent trajectory fusion operation, the basic unit is the spatial point in the trajectory point set. Fusion is the process of merging the point set of the optimized obstacle avoidance trajectory with the original drivability-corrected trajectory point set. Its purpose is to expand the corrected trajectory information so that it not only includes the original planned path, but also all possible avoidance paths, thereby forming a trajectory cluster that covers all possibilities.

[0110] Specifically, when the flight control system detects an obstacle and triggers an obstacle avoidance command, it first identifies the type of the command, determining whether it is an emergency obstacle avoidance or a normal avoidance. For example, when the UAV approaches an obstacle at a distance less than a safety threshold, an emergency obstacle avoidance command is generated, carrying the corresponding avoidance direction vector. Subsequently, multiple candidate auxiliary trajectories are generated based on the avoidance direction vector, such as fan-shaped alternative paths extending from the direction vector on a horizontal plane. In normal avoidance mode, the candidate trajectories are smoothed to reduce energy consumption. In emergency obstacle avoidance mode, the trajectory length is shortened first to quickly escape the danger zone. Finally, the optimized obstacle avoidance trajectory points are spatiotemporally aligned and interpolated with the original spatiotemporal trajectory points to form corrected trajectory information covering all possible motion paths, thereby ensuring that the preloaded 3D model tiles can cover the potential visible area during the obstacle avoidance process.

[0111] For example, consider a scenario where a drone is cruising and the flight control system suddenly detects a moving obstacle directly in front of it. The flight control terminal immediately sends an obstacle avoidance command, which is an emergency obstacle avoidance command with an avoidance direction vector of "30 degrees to the upper left". Upon receiving the command, based on the "30 degrees to the upper left" vector, multiple auxiliary obstacle avoidance trajectories that turn sharply to the upper left are generated. Because it is an emergency command, these trajectories are adaptively adjusted to make them shorter and sharper, quickly deviating from the original flight path, resulting in an optimized obstacle avoidance trajectory. The trajectory points in the optimized obstacle avoidance trajectory are merged with the originally planned cruise trajectory points to generate a new corrected trajectory information. This corrected trajectory information includes multiple possibilities such as "continuing the original plan" and "emergency upper left avoidance".

[0112] Through the above technical solution, this application solves the problem of missing preloaded data caused by trajectory changes due to obstacle avoidance actions in the prior art. By dynamically fusing the obstacle avoidance trajectory and the original predicted trajectory, the loading timeliness of 3D model tiles in emergency maneuvering situations is significantly improved, reducing or even avoiding screen stuttering and model rendering loss.

[0113] In one embodiment, step S33 includes:

[0114] S331: Obtain the confidence level of camera motion state, and perform uncertainty modeling on discrete trajectory points in the corrected trajectory information based on the confidence level of camera motion state to generate several uncertainty ellipsoids;

[0115] S332: Transform the uncertain ellipsoid into a local probability density distribution, and generate a continuous spatiotemporal probability density field based on the local probability density distribution;

[0116] S333: Using the impassable areas in the mobile area information as hard constraints, the probability values ​​of areas located in the impassable areas in the spatiotemporal probability density field are reset to zero to generate an optimized probability density field.

[0117] S334: Extract connected regions with probability values ​​greater than a preset threshold from the optimized probability density field to form a three-dimensional probability space volume.

[0118] In this embodiment, the camera motion state confidence score is a quantitative evaluation index of the reliability of camera motion parameters. Specifically, it can be implemented using a weighted calculation of signal quality assessment and model prediction error, reflecting the credibility of the trajectory prediction results. Uncertainty modeling is a mathematical tool that transforms deterministic prediction into a probabilistic description. It acknowledges that any prediction has errors and quantifies the range and distribution of these errors based on the confidence score. In this embodiment, uncertainty modeling generates a geometric shape representing the possible position range of each trajectory point. The uncertainty ellipsoid is a three-dimensional probability distribution representation centered on the trajectory point. Its size is determined by the confidence score: a lower confidence score results in a larger ellipsoid, and a higher confidence score results in a smaller ellipsoid. Its direction is determined by the motion trend; for example, when moving at high speed, the ellipsoid elongates along the velocity direction, and when turning, it spreads out in the turning direction. Specifically, it can be implemented by determining the principal axis direction and semi-axis length using the covariance matrix, used to describe the camera position prediction. The spatial uncertainty range; the local probability density distribution is a probability density function constructed based on ellipsoidal parameters, which can be implemented by spatial interpolation using a Gaussian kernel function to establish a continuous probability distribution model; the spatiotemporal probability density field is a probability cloud covering the entire possible future space generated by superimposing the local probability density distributions of all trajectory points, which defines the probability that the camera will appear at any point in space at a certain future moment; hard constraints are inviolable constraint rules, which in this embodiment are impassable areas in the mobile area information, such as mountains, building interiors, etc.; the optimized probability density field is the probability distribution adjusted after the impassable area constraint, which can be implemented by setting the probability of no-fly zones to zero through spatial masking operations to eliminate prediction interference from illegal areas; the connected region is the region in the optimized probability density field where all probability values ​​are greater than the threshold and can be connected in three-dimensional space, ensuring that the extracted region is continuous and complete, rather than scattered fragments.

[0119] Specifically, discrete trajectory points in the corrected trajectory information are first assigned confidence weights, and a set of ellipsoids with directional characteristics is generated through the covariance matrix. These ellipsoids are then interpolated using kernel functions to form a continuous probability field, and the influence of no-fly zones is eliminated through spatial constraints. Finally, high-probability regions are extracted through regional connectivity analysis to form an accurate three-dimensional spatial volume as the basis for pre-loaded region judgment. The above process quantifies the credibility of the prediction results through probabilistic modeling and dynamically adjusts the prediction range in combination with spatial constraints, effectively balancing prediction accuracy and computational efficiency.

[0120] For example, taking a drone flying over a complex terrain area as a scenario, due to slight signal interference, the confidence level of the current motion state is calculated to be "medium". Based on this, an uncertainty ellipsoid is generated for each point on the corrected trajectory. At the turning point, the ellipsoid is stretched along the turning arc. Each ellipsoid is converted into a probability cloud, and then all probability clouds are superimposed to generate a continuous spatiotemporal probability density field that diffuses around the flight path. Reading the information of the moving area, it is found that part of the spatiotemporal probability density field has penetrated into the interior of the mountain. Therefore, the probability values ​​inside the mountain are all reset to zero to generate an optimized probability density field. The probability threshold is set to 0.6, and all areas with probability values ​​higher than 0.6 and connected together are extracted. Finally, a three-dimensional probability space volume with an irregular shape wrapped in the flyable airspace is obtained as the final output of the prediction, which is used to accurately load the required data.

[0121] Through the above technical solutions, this application can accurately identify high-probability areas that the camera may reach under complex motion conditions, reducing prediction deviations caused by sharp turns or obstacle avoidance actions; by controlling the preloading range through probability thresholds, redundant data loading is reduced while ensuring data integrity, significantly improving the utilization rate of video memory resources.

[0122] In one embodiment, step S331 includes:

[0123] S3311: Evaluate the signal quality of camera motion parameters and obtain the signal confidence factor;

[0124] S3312: Obtain the model confidence factor of the motion prediction model, and calculate the camera motion state confidence based on the signal confidence factor and the model confidence factor;

[0125] S3313: Generate parameters of an uncertain ellipsoid based on camera motion state confidence matching, wherein the generated parameters include size and orientation;

[0126] S3314: Take each discrete trajectory point in the corrected trajectory information as the center and combine it with the generation parameters to generate the corresponding uncertain ellipsoid.

[0127] In this embodiment, signal quality assessment is a quantitative analysis process of the reliability of raw sensor data. It evaluates the reliability of the data by calculating a series of indicators, such as signal-to-noise ratio, data update rate, positioning accuracy factor, and packet loss rate. The assessment result is a quantified numerical signal. The confidence factor is a quantitative evaluation indicator of the output signal quality of the camera motion parameter acquisition device. Specifically, it can be calculated using at least one parameter among signal-to-noise ratio, sampling frequency stability, or sensor calibration error, and is used to reflect the reliability of the sensor data. The model confidence factor is a quantitative evaluation indicator of the reliability of the motion prediction model's output results. Specifically, it can use historical... The prediction error statistics or validation set accuracy index calculated during the model training phase are used to reflect the accuracy of the model prediction; the camera motion state confidence is a comprehensive credibility index that integrates both signal and model factors. Specifically, it can be obtained by fusing the signal confidence factor and the model confidence factor through weighted summation or multiplication operations, and is used to dynamically adjust the intensity of uncertainty modeling; the generation parameters are configuration parameters that control the shape of the uncertainty ellipsoid. The size can be the length of the semi-axis of the ellipsoid along the three coordinate axes, and the direction can be the deflection angle of the ellipsoid's principal axis relative to the velocity vector. Specifically, it can be dynamically adjusted based on the confidence value through a lookup table method or parameter mapping function.

[0128] Specifically, the signal quality acquisition process involves real-time analysis of the raw sensor data using a pre-defined evaluation algorithm. For example, the noise level is calculated using the signal variance within a sliding window, and a signal confidence factor is output. The model confidence factor is obtained through the prediction error distribution of the offline statistical model on the validation set. For example, the average Euclidean distance between the predicted trajectory point and the actual trajectory point is calculated as an evaluation metric. The camera motion state confidence is fused with the signal and model confidence using a linear weighting method. For example, the signal confidence weight coefficient is reduced when the signal quality degrades. The generated parameters are dynamically matched according to the confidence value. For example, when the confidence is below a threshold, the ellipsoid size is increased to cover a wider range of potential locations, and the principal axis direction can be set as the tangent direction of the velocity vector. Each discrete trajectory point is assigned an independent uncertainty ellipsoid. For example, in a sharp turn scenario, a decrease in confidence causes the ellipsoid to expand along the turning direction, forming a probability space covering possible paths.

[0129] Through the above technical solution, this application can dynamically optimize the granularity of uncertainty modeling based on the real-time status of sensor data quality and model prediction capability, so as to accurately cover the spatial area that the camera may reach in the future when the camera motion state is unstable or the accuracy of the prediction model fluctuates. Through the above adaptive mechanism, the preloading omission or redundancy problem caused by sensor noise or model failure can be effectively reduced, and the accuracy of 3D model loading in complex maneuvering scenarios can be improved.

[0130] In one embodiment, step S31 includes:

[0131] S311: Obtain signal quality information, including normal, degraded, and lost signals;

[0132] S312: When signal quality information is lost, obtain the camera motion history sequence based on the preset sequence selection strategy;

[0133] S313: Input the camera motion history sequence into the pre-trained temporal behavior recognition model to extract spatiotemporal features, and identify the dominant behavior pattern based on the spatiotemporal features;

[0134] S314: Input the dominant behavior pattern and camera motion history sequence into the pre-trained motion prediction model to generate spatiotemporal trajectory information.

[0135] In this embodiment, signal quality information is an evaluation result of the working status of the camera motion parameter acquisition device. Specifically, it can be quantified using indicators such as signal strength, packet loss rate, and time continuity to determine the reliability of the current motion parameters. Signal quality information includes normal, degraded, and lost. Normal means that the signal strength, accuracy, and continuity are all in good condition, and it operates in the default high-performance mode. Degraded means that the signal fluctuates, noise increases, or accuracy decreases, but it has not been completely interrupted. Fault tolerance mechanisms can be enabled, such as filtering and smoothing the data and appropriately expanding the prediction range. Lost means that the signal is completely interrupted, and effective real-time motion parameters cannot be obtained. It can switch to emergency prediction mode and rely entirely on historical data for calculation. The sequence selection strategy is a rule for selecting effective motion sequences from historical data. Specifically, it can adopt a sliding window mechanism or dynamic time... The algorithm implements time-space regularization to ensure that the selected sequence length matches the input requirements of the motion prediction model. The camera motion history sequence is an array of motion parameters arranged in chronological order, recording the historical values ​​of parameters such as camera position, velocity, acceleration, and angular velocity during a period before signal loss. The temporal behavior recognition model is a neural network used to extract spatiotemporal features from the historical motion sequence. Specifically, it can be constructed using a 3D convolutional network or a spatiotemporal graph convolutional network to capture periodic or trend patterns in motion. The spatiotemporal features are feature vectors extracted from the camera motion history sequence that can characterize motion trends and patterns, used to reveal the deep-seated patterns of the sequence. The dominant behavior pattern is the behavior feature that appears repeatedly or lasts the longest in the historical motion sequence. Specifically, it can be identified through cluster analysis or attention mechanisms to provide prior knowledge for motion prediction.

[0136] Specifically, when signal quality information is detected to be lost, a sequence selection strategy is used to extract the camera's historical motion sequence containing position, velocity, acceleration, and angular velocity from the cache. This sequence is then input into the temporal behavior recognition model, which extracts high-level semantic information such as implicit acceleration variation patterns and turning frequency features through multiple spatiotemporal convolution operations. The model outputs the probability distribution of multiple candidate behavior patterns through fully connected layers and selects the behavior pattern with the highest probability as the dominant behavior pattern. The motion prediction model receives the dominant behavior pattern encoding vector and the historical motion sequence, and performs time series prediction through a gated recurrent unit network to generate predicted values ​​of position, velocity, and attitude angle for multiple future time steps, forming spatiotemporal trajectory information.

[0137] Through the above technical solutions, this application can still accurately reconstruct the camera's motion intent in scenarios of signal loss such as positioning failure and abnormal inertial navigation data, ensuring the continuity of 3D model preloading; the extraction of spatiotemporal behavioral features makes the predicted trajectory fit the actual maneuver characteristics, reducing model loading errors or delays caused by signal interruption and maintaining the smoothness of the visualization scene.

[0138] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0139] In one embodiment, a dynamic trajectory prediction-based oblique photogrammetry 3D model loading system is provided, which corresponds one-to-one with the dynamic trajectory prediction-based oblique photogrammetry 3D model loading method described in the previous embodiment. The dynamic trajectory prediction-based oblique photogrammetry 3D model loading system includes:

[0140] The motion pattern recognition module is used to acquire camera motion parameters in real time and identify camera motion patterns based on the camera motion parameters, including position, velocity, acceleration and angular velocity.

[0141] The first tile set generation module is used to acquire moving area information and generate a first preloaded tile set based on camera motion parameters and moving area information using a preset sector region prediction algorithm.

[0142] The second tile set generation module is used to generate predicted spatiotemporal location information based on camera motion parameters and moving area information through a pre-trained motion prediction model, and to perform collision detection between the predicted spatiotemporal location information and the pre-built scene octree index to generate a second preloaded tile set.

[0143] The load list generation module is used to obtain the current load information, and to perform weight allocation on the first preloaded tile set and the second preloaded tile set based on the camera motion mode and the current load information, and generate a preload list;

[0144] The target tile loading module is used to identify the priority order in the preload list and load the corresponding 3D model target tiles into the video memory through a pre-set multi-level cache scheduling mechanism. The multi-level cache scheduling mechanism includes: firstly searching for the target tile in the video memory cache; if it is not found, then searching for the target tile in the memory cache and promoting it to the video memory; if it is still not found, then loading it from the storage medium into the memory cache, and determining whether to forward it to the video memory according to the priority order.

[0145] Optionally, the first tile set generation module includes:

[0146] The region determination submodule is used to determine the 3D sector prediction region based on the position and velocity in the camera motion parameters;

[0147] The region optimization submodule is used to optimize the 3D sector prediction region based on the moving region information and generate an enhanced prediction region.

[0148] The tile recognition submodule is used to identify all 3D model tiles whose spatial location is within the enhanced prediction area, and to generate a first preloaded tile set based on the identified 3D model tiles.

[0149] Specific limitations regarding the oblique photogrammetry 3D model loading system based on dynamic trajectory prediction can be found in the limitations of the oblique photogrammetry 3D model loading method based on dynamic trajectory prediction described above, and will not be repeated here. Each module in the aforementioned oblique photogrammetry 3D model loading system based on dynamic trajectory prediction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0150] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A tilt photography three-dimensional model loading method based on dynamic trajectory prediction, characterized by, The method comprises the steps of: acquiring camera motion parameters in real time, and identifying a camera motion mode based on the camera motion parameters, wherein the camera motion parameters include position, speed, acceleration, and angular velocity; acquiring movement region information, and generating a first preloaded tile set through a preset sector region prediction algorithm based on the camera motion parameters and the movement region information; generating predicted spatiotemporal position information through a pre-trained motion prediction model based on the camera motion parameters and the movement region information, and performing collision detection on the predicted spatiotemporal position information and a pre-constructed scene octree index to generate a second preloaded tile set; acquiring current load information, and performing weight distribution on the first preloaded tile set and the second preloaded tile set based on the camera motion mode and the current load information, and generating a preloading list; identifying a priority order in the preloading list, and loading corresponding three-dimensional model target tiles to a video memory through a pre-set multi-level cache scheduling mechanism; wherein the multi-level cache scheduling mechanism comprises: preferentially searching for target tiles in a video memory cache; if no hit is found, searching for target tiles in a memory cache and promoting them to the video memory; if no hit is still found, loading them from a storage medium to the memory cache, and judging whether to forward them to the video memory according to the priority order. 2.The tilt-shift 3D model loading method based on dynamic trajectory prediction according to claim 1, wherein: The step of acquiring movement region information, and generating a first preloaded tile set through a preset sector region prediction algorithm based on the camera motion parameters and the movement region information, comprises the steps of: determining a three-dimensional sector prediction region based on the position and speed in the camera motion parameters; optimizing the three-dimensional sector prediction region based on the movement region information to generate an enhanced prediction region; identifying all three-dimensional model tiles whose spatial positions are located in the enhanced prediction region, and generating the first preloaded tile set based on the identified three-dimensional model tiles. 3.The method of claim 2, wherein the method further comprises: determining a dynamic trajectory of the camera based on the camera motion data; and determining a dynamic trajectory of the object based on the object motion data. The step of optimizing the three-dimensional sector prediction region based on the movement region information to generate an enhanced prediction region comprises the steps of: extracting features from the movement region information to obtain passable region features and impassable region features, wherein the passable region features include airspace constraint features and maneuverability constraint features; performing a spatial Boolean subtraction operation on the three-dimensional sector prediction region based on the impassable region features to obtain an initial optimization region; performing a spatial Boolean intersection operation on the initial optimization region based on the airspace constraint features to generate an intermediate optimization region; performing reachability correction on the intermediate optimization region based on the maneuverability constraint feature domain to generate the enhanced prediction region.

4. The method of claim 1, wherein the method further comprises: The step of generating predicted spatiotemporal position information through a pre-trained motion prediction model based on the camera motion parameters and the movement region information, and performing collision detection on the predicted spatiotemporal position information and a pre-constructed scene octree index to generate a second preloaded tile set, comprises the steps of: generating spatiotemporal trajectory information through the pre-trained motion prediction model based on the position, speed, acceleration, and angular velocity in the camera motion parameters; performing passability correction on the spatiotemporal trajectory information by taking the movement region information as a spatial constraint condition to exclude trajectory points located in impassable regions, to obtain corrected trajectory information; map the modified trajectory information into a three-dimensional probability space body representing a future camera existing space range, and take the three-dimensional probability space body as the predicted spatiotemporal position information; perform layer-by-layer traversal and collision detection on the predicted spatiotemporal position information and the pre-constructed scene octree index, and retrieve all intersected octree nodes as target nodes; obtain identifiers corresponding to three-dimensional model tiles associated with all target nodes, and generate a second preloaded tile set based on the identifiers.

5. The method of claim 4, wherein the method further comprises: The step of taking the moving area information as a spatial constraint condition to modify the passability of the spatiotemporal trajectory information to exclude trajectory points located in the unpassable area to obtain the modified trajectory information, comprises the steps of: When receiving an obstacle avoidance instruction from the flight control end, identifying the instruction type and avoidance direction vector of the obstacle avoidance instruction, the instruction type including emergency obstacle avoidance and regular avoidance; generating a plurality of auxiliary obstacle avoidance trajectories based on the avoidance direction vector, and adaptively adjusting the auxiliary obstacle avoidance trajectories based on the instruction type to obtain optimized obstacle avoidance trajectories; fusing the trajectory point set of the generated optimized obstacle avoidance trajectory with the trajectory point set of the passability-modified spatiotemporal trajectory information to generate the modified trajectory information.

6. The method of claim 4, wherein the method further comprises: The step of mapping the modified trajectory information into a three-dimensional probability space body representing a future camera existing space range, and taking the three-dimensional probability space body as the predicted spatiotemporal position information, comprises the steps of: obtaining a camera motion state confidence, and modeling the uncertainty of discrete trajectory points in the modified trajectory information based on the camera motion state confidence to generate a plurality of uncertainty ellipsoids; converting the uncertainty ellipsoids into local probability density distributions, and generating a continuous spatiotemporal probability density field based on the local probability density distributions; taking the unpassable area in the moving area information as a hard constraint, resetting the probability values in the spatiotemporal probability density field located in the unpassable area to zero to generate an optimized probability density field; extracting a connected region with a probability value greater than a preset threshold from the optimized probability density field to form a three-dimensional probability space body.

7. The method of claim 6, wherein the method further comprises: The step of obtaining a camera motion state confidence, and modeling the uncertainty of discrete trajectory points in the modified trajectory information based on the camera motion state confidence to generate a plurality of uncertainty ellipsoids, comprises the steps of: evaluating the signal quality of the camera motion parameters to obtain a signal confidence factor; obtaining a model confidence factor of a motion prediction model, and calculating the camera motion state confidence based on the signal confidence factor and the model confidence factor; matching the generation parameters of the uncertainty ellipsoids based on the camera motion state confidence, the generation parameters including size and direction; generating a corresponding uncertainty ellipsoid for each discrete trajectory point in the modified trajectory information based on the generation parameters.

8. The method of claim 4, wherein the method further comprises: The step of generating spatiotemporal trajectory information based on the position, velocity, acceleration, and angular velocity in the camera motion parameters through a pre-trained motion prediction model, comprises the steps of: obtaining signal quality information, the signal quality information including normal, degraded, and lost; when the signal quality information is lost, obtaining a camera motion history sequence based on a preset sequence selection strategy; The camera motion history sequence is input into a pre-trained time behavior recognition model to extract spatio-temporal features, and a dominant behavior pattern is recognized based on the spatio-temporal features; The dominant behavior pattern and the camera motion history sequence are input into a pre-trained motion prediction model to generate spatio-temporal trajectory information.

9. A dynamic trajectory prediction-based oblique photography three-dimensional model loading system, characterized in that, Comprise: A motion pattern recognition module for real-time acquisition of camera motion parameters, and recognition of camera motion patterns based on the camera motion parameters, the camera motion parameters including position, velocity, acceleration and angular velocity; A first tile set generation module for acquiring movement area information, generating a first preloaded tile set based on the camera motion parameters and the movement area information, and through a preset fan-shaped area prediction algorithm; A second tile set generation module for generating predicted spatio-temporal position information based on the camera motion parameters and the movement area information, and through a pre-trained motion prediction model, and performing collision detection on the predicted spatio-temporal position information and a pre-constructed scene octree index to generate a second preloaded tile set; A loading list generation module for acquiring current load information, and performing weight distribution on the first preloaded tile set and the second preloaded tile set based on the camera motion pattern and the current load information, and generating a preloading list; A target tile loading module for identifying the priority order in the preloading list, and loading the corresponding three-dimensional model target tile to the video memory through a pre-set multi-level cache scheduling mechanism; wherein the multi-level cache scheduling mechanism comprises: preferentially searching for the target tile in the video memory cache; if not hit, searching for the target tile in the memory cache and promoting it to the video memory; if still not hit, loading it from the storage medium to the memory cache, and judging whether to forward it to the video memory according to the priority order. 10.The oblique photography three-dimensional model loading system based on dynamic trajectory prediction according to claim 9, wherein, The first tile set generation module comprises: A region determination sub-module for determining a three-dimensional fan-shaped prediction region based on the position and velocity in the camera motion parameters; A region optimization sub-module for optimizing the three-dimensional fan-shaped prediction region based on the movement area information to generate an enhanced prediction region; A tile recognition sub-module for recognizing all three-dimensional model tiles whose spatial positions are located in the enhanced prediction region, and generating the first preloaded tile set based on the recognized three-dimensional model tiles.

Images