A VR-based method and system for drone scene modeling
By constructing a sampling density spatial variation field and optimizing the bundle adjustment with differentiated weights, the problem of uneven sampling density caused by wind speed was solved, ensuring the accuracy balance and geometric consistency of the 3D model in the VR drone training system, and improving the training effect.
Patent Information
- Application Number
- CN202611301459.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-26
- Publication Date
- 2026-11-17
AI Technical Summary
In existing VR drone training systems, uneven sampling density caused by wind speed leads to inconsistent accuracy of 3D models, affecting training effectiveness.
By constructing a sampling density spatial variation field, calculating local sampling density indices based on UAV pose data, and assigning differentiated reconstruction contribution weights to each frame of image according to a preset mapping function, a unified weighting is introduced in bundle adjustment to optimize the 3D reconstruction model.
It achieves uniformity of accuracy and geometric consistency of 3D models across the entire scene range, providing realistic and continuous virtual flight scenarios, enhancing the immersion of training and the effectiveness of operational training.
Smart Images

Figure CN123118903A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of 3D modeling technology, specifically to a VR-based method and system for modeling drone scenes. Background Technology
[0002] Traditional drone training relies on real flight training, which is not only costly and limited by airspace resources, but also carries the risk of crashes and other safety accidents due to trainee errors. In recent years, virtual reality (VR)-based drone training systems have gradually emerged as an alternative. By constructing highly realistic 3D flight scenarios, trainees can complete the entire process of takeoff, cruise, obstacle avoidance, and landing in a virtual environment, reducing training costs and risks while breaking the limitations of weather, airspace, and time imposed by real flight. However, the core of VR training systems lies in the quality of the 3D model of the flight scenario. Uneven model accuracy, blurred local textures, or geometric distortions can directly lead to trainees' errors in distance judgment and operational deviations in the VR environment, severely impacting training effectiveness.
[0003] In constructing 3D models of flight scenes using drone aerial photography, operators plan flight paths based on preset forward overlap rates and assume the drone flies at a constant speed. However, wind speed is a significant variable affecting drone speed in actual flight environments. Speed decreases when flying against the wind and increases when flying with the wind, leading to significant variations in the number of images collected per unit distance. Images are densely packed in the upwind section and sparsely packed in the downwind section, causing the actual forward overlap rate to fluctuate drastically along the flight path. This fluctuation is unpredictable during the flight path planning stage and is not corrected in subsequent processing by existing 3D reconstruction methods. More subtly, the uneven sampling density caused by wind speed is a systematic bias independent of image quality and observation accuracy. Densely sampled areas experience local overfitting due to over-constraints in 3D reconstruction, while sparsely sampled areas suffer geometric drift due to insufficient constraints. Ultimately, the accuracy of the 3D model exhibits a systematic imbalance in spatial distribution, highly correlated with wind speed.
[0004] In existing technologies, the reconstruction effect of specific algorithms is improved by actively planning the density-sparse acquisition path. However, the core idea is to actively design a non-uniform sampling strategy before acquisition. Another method assigns weights to observations based on GPS accuracy or image quality evaluation indicators in bundle adjustment. However, none of these methods recognize that wind speed-induced uneven sampling density is a systematic bias independent of image quality. Therefore, a VR-based UAV scene modeling method and system are proposed to address the above-mentioned problems. Summary of the Invention
[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a VR-based drone scene modeling method and system to solve the problems existing in the above-mentioned background technology.
[0006] This invention is implemented as follows: a VR-based drone scene modeling method, the method comprising the following steps: Acquire multiple frames of aerial images captured by the UAV in the target scene and their corresponding pose data, and calculate the actual ground projection spacing sequence between each adjacent image frame along the flight trajectory direction based on the pose data; A spatial variation field representing the distribution of full-scene image sampling density along the flight path and between flight paths is constructed based on the actual ground projection spacing sequence. The local sampling density index corresponding to each frame of the image is determined based on the spatial variation field, and an independent reconstruction contribution weight is assigned to each frame of the image according to the preset mapping function, so that the weight of dense sampling regions is reduced and the weight of sparse sampling regions is increased. The reconstruction contribution weight is introduced into the bundle adjustment objective function of 3D reconstruction to uniformly weight all observations in each frame of the image, solve the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame of the image, and output the target scene 3D model optimized by sampling density equalization constraint, which can be loaded into the VR drone training system as a virtual flight scene.
[0007] As a further aspect of the present invention: the step of calculating the actual ground projection spacing sequence between adjacent image frames along the flight trajectory direction based on pose data specifically includes: Obtain the digital elevation model of the target scene and mesh it to obtain the ground elevation value at each grid node; Extract the three-dimensional spatial coordinates and attitude angles corresponding to each frame of the image from the pose data, and calculate the angle between the principal optical axis of each frame of the image and the ground normal based on the attitude angles, in order to determine the projection direction of each frame of the image on the ground. The corresponding ground elevation value is queried based on the projection position of each frame image in the digital elevation model, and the actual ground distance along the surface between the current frame and its adjacent frames is calculated by combining the three-dimensional spatial coordinates and projection direction of each frame image, and this is used as the initial ground projection spacing. The initial ground projection spacing is corrected based on the elevation difference between the projection positions of two adjacent frames to obtain the actual ground projection spacing sequence after eliminating the influence of terrain.
[0008] As a further aspect of the present invention: the step of constructing a spatial variation field characterizing the distribution of the full-scene image sampling density along the flight path and between flight paths based on the actual ground projection spacing sequence specifically includes: Each spacing value in the actual ground projection spacing sequence is associated with the midpoint of the flight path between two adjacent frames corresponding to that spacing, and this is used as a control point. The wind direction and flight path angle, terrain slope, and flight altitude at that point are used as the local environmental feature vector of the control point. Using the local environmental feature vectors of all control points as independent variables and the distance between each control point as the dependent variable, a geographically weighted regression model is established to ensure that different control points have differentiated influences in the regression. Based on the results of the geographic weighted regression model, the spacing values at each location in the entire scene are interpolated and predicted. The reciprocal of the predicted spacing value is then used as the local sampling density index value at that location, generating a sampling density distribution surface covering the entire scene. For the extrapolated region in the sampling density distribution surface caused by the sparse control points, the interpolation confidence is calculated using the distance between the location to be evaluated and the control points and the environmental similarity as joint indices, and the spatial variation field containing the local sampling density index value and the interpolation confidence is output.
[0009] As a further aspect of the present invention: the step of assigning an independent reconstruction contribution weight to each frame of image according to a preset mapping function specifically includes: A first threshold and a second threshold greater than the first threshold are set, and the range of local sampling density index values is divided into high-density area, balanced area and low-density area by using the two as the judgment boundary. When the local sampling density index is less than the first threshold, the frame image is determined to be located in a high-density area, and the reconstruction contribution weight of the frame image is reduced according to the exponential decay law. When the local sampling density index is between the first threshold and the second threshold, the frame image is determined to be in the balance region, and the reconstruction contribution weight of the frame image remains at the baseline value. When the local sampling density index is greater than the second threshold, the image frame is determined to be located in a low-density area. The reconstruction contribution weight of the image frame is increased according to the logarithmic growth law, and it does not exceed the preset upper limit.
[0010] As a further aspect of the present invention: the step of setting a first threshold and a second threshold greater than the first threshold specifically includes: Scale-invariant feature transform feature points are extracted from each frame of the image. The number of feature points in each frame of the image is counted, and the mean and standard deviation of the number of feature points in all frames of the image are used as quantitative indicators of scene texture richness. The first threshold is calculated based on the mean and standard deviation, wherein the first threshold is equal to the mean minus the standard deviation by a first multiple; The second threshold is calculated based on the mean and standard deviation, wherein the second threshold is equal to the mean plus the standard deviation of the second multiple, and the first multiple and the second multiple are preset normal numbers; The calculated first and second thresholds are used as the judgment boundaries of the mapping function.
[0011] As a further aspect of the present invention: the step of introducing the reconstruction contribution weight into the objective function of the bundle adjustment of 3D reconstruction to uniformly weight all observations in each frame of the image specifically includes: Construct the objective function for bundle adjustment, which is the sum of squares of the reprojection errors of all observations; The reconstruction contribution weight of each frame is used as the common coefficient of all observations in that frame, and multiplied by the square of the reprojection error of each observation corresponding to that frame. The weighted objective function is formed by summing the squared terms of the reprojection errors of each observation after weighting. The weighted objective function is used to replace the equal-weight objective function in traditional bundle adjustment, and is used as the objective for iterative optimization.
[0012] Another object of the present invention is to provide a VR-based drone scene modeling system, the system comprising: The sequence acquisition module is used to acquire multiple frames of aerial images and their corresponding pose data collected by the UAV in the target scene, and to calculate the actual ground projection spacing sequence between each adjacent image frame along the flight trajectory direction based on the pose data. The density field construction module is used to construct a spatial variation field representing the distribution of the sampling density of the whole scene image along the flight path and between flight paths, based on the actual ground projection spacing sequence. The weight allocation module is used to determine the local sampling density index corresponding to each frame of the image based on the spatial variation field, and to assign an independent reconstruction contribution weight to each frame of the image according to the preset mapping function, so that the weight of dense sampling areas is reduced and the weight of sparse sampling areas is increased. The weighted output module is used to introduce the reconstruction contribution weight into the bundle adjustment objective function of 3D reconstruction, so as to uniformly weight all observations in each frame image, solve the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame image, and output the target scene 3D model optimized by sampling density equalization constraint, which can be loaded into the VR drone training system as a virtual flight scene.
[0013] As a further aspect of the present invention: the sequence acquisition module includes: Mesh building units are used to acquire the digital elevation model of the target scene and mesh it to obtain the ground elevation value at each grid node. The projection calculation unit is used to extract the three-dimensional spatial coordinates and attitude angles corresponding to each frame of the image from the pose data, and calculate the angle between the principal optical axis of each frame of the image and the ground normal based on the attitude angle, so as to determine the projection direction of each frame of the image on the ground. The distance calculation unit is used to query the corresponding ground elevation value based on the projection position of each frame image in the digital elevation model, and calculate the actual ground distance along the surface of the current frame and its adjacent frames by combining the three-dimensional spatial coordinates and projection direction of each frame image, and use it as the initial ground projection spacing. The deviation correction unit is used to correct the initial ground projection spacing based on the elevation difference between the projection positions of two adjacent frames, so as to obtain the actual ground projection spacing sequence after eliminating the influence of terrain.
[0014] As a further aspect of the present invention: the density field construction module includes: The control point calibration unit is used to associate each spacing value in the actual ground projection spacing sequence with the midpoint of the track between two adjacent frames of images corresponding to that spacing, and use it as a control point. The angle between the wind direction and the flight direction, the terrain slope, and the flight altitude at that point are used as the local environmental feature vector of the control point. The regression modeling unit is used to establish a geographically weighted regression model with the local environmental feature vectors of all control points as independent variables and the distance between each control point as the dependent variable, so that different control points have differentiated influence in the regression. The surface generation unit is used to interpolate and predict the spacing values of each location in the entire scene based on the results of the geographic weighted regression model, and then take the reciprocal of the predicted spacing value as the local sampling density index value of that location to generate a sampling density distribution surface covering the entire scene. The evaluation output unit is used to calculate the interpolation confidence score for the extrapolation region in the sampling density distribution surface caused by the sparse control points, using the distance of the location to be evaluated from the control points and the environmental similarity as joint indices, and outputs a spatial variation field containing local sampling density index values and interpolation confidence scores.
[0015] As a further aspect of the present invention: the weight allocation module includes: The partition setting unit is used to set a first threshold and a second threshold greater than the first threshold, and use the two as judgment boundaries to divide the range of local sampling density index values into high-density areas, balanced areas and low-density areas. The weight reduction processing unit is used to determine that the frame image is located in a high-density area when the local sampling density index is less than the first threshold, and to reduce the reconstruction contribution weight of the frame image according to the exponential decay law. The holding processing unit is used to determine that the frame image is located in the balance region when the local sampling density index is between the first threshold and the second threshold, and the reconstruction contribution weight of the frame image is kept at the baseline value; The weighting processing unit is used to determine that the frame image is located in a low-density area when the local sampling density index is greater than the second threshold, and to increase the reconstruction contribution weight of the frame image according to the logarithmic growth law, and the weighting does not exceed the preset upper limit.
[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention addresses the problem of uneven spatial distribution of sampling density caused by wind speed-induced ground speed fluctuations by constructing a sampling density spatial variation field and performing bundle adjustment based on differentiated weights. This effectively eliminates over-constraint in dense sampling areas and insufficient constraint in sparse sampling areas, ensuring the reconstructed model maintains uniform accuracy and geometric consistency across the entire scene. Furthermore, since the allocation of reconstruction contribution weights is not dependent on image content or human experience but is automatically calculated from sampling density indices driven by UAV pose data, it avoids the systematic accuracy loss caused by downwash interference in traditional equal-weight adjustment models. Therefore, the 3D model output by this invention has a balanced distribution of geometric details and stable scale consistency, directly providing VR UAV training systems with realistic, continuous, and visually abrupt virtual flight scenarios. This ensures trainees receive consistent visual feedback when making distance judgments and spatial perceptions in different areas of the scene, effectively enhancing the immersion and effectiveness of training. Attached Figure Description
[0017] Figure 1 A flowchart of a VR-based drone scene modeling method.
[0018] Figure 2 This is a flowchart for calculating the actual ground projection spacing sequence in a VR-based drone scene modeling method.
[0019] Figure 3 This is a flowchart illustrating the construction of a spatial variation field in a VR-based drone scene modeling method.
[0020] Figure 4 A flowchart illustrating how an independent reconstruction contribution weight is assigned to each frame of an image in a VR-based drone scene modeling method.
[0021] Figure 5 A flowchart for setting a first threshold and a second threshold in a VR-based drone scene modeling method.
[0022] Figure 6 This is a flowchart illustrating the uniform weighting of all observations in each frame of an image in a VR-based drone scene modeling method.
[0023] Figure 7 This is a schematic diagram of the structure of a VR-based drone scene modeling system.
[0024] Figure 8 This is a schematic diagram of the sequence acquisition module in a VR-based drone scene modeling system.
[0025] Figure 9 This is a schematic diagram of the density field construction module in a VR-based drone scene modeling system.
[0026] Figure 10 This is a schematic diagram of the weight allocation module in a VR-based drone scene modeling system. Detailed Implementation
[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0028] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0029] like Figure 1 As shown in the figure, this embodiment of the invention provides a VR-based drone scene modeling method, characterized in that the method includes the following steps: S100: Acquire multiple frames of aerial images captured by the UAV in the target scene and their corresponding pose data, and calculate the actual ground projection spacing sequence between each adjacent image frame along the flight trajectory direction based on the pose data. S200, based on the actual ground projection spacing sequence, constructs a spatial variation field characterizing the distribution of the full-scene image sampling density along the flight path and between flight paths; S300 determines the local sampling density index corresponding to each frame of the image based on the spatial variation field, and assigns an independent reconstruction contribution weight to each frame of the image according to the preset mapping function, so that the weight of dense sampling areas is reduced and the weight of sparse sampling areas is increased. S400 introduces the reconstruction contribution weight into the bundle adjustment objective function of 3D reconstruction to uniformly weight all observations in each frame of the image, solves the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame of the image, and outputs the target scene 3D model optimized by sampling density equalization constraint, which is then loaded into the VR drone training system as a virtual flight scene.
[0030] It should be noted that after the iterative solution of the weighted objective function is completed, the output of the bundle adjustment method is the camera exterior orientation parameters and 3D point cloud structure corresponding to the acquisition time of each frame image. The camera exterior orientation parameters include the 3D spatial position and attitude angle of the camera in the world coordinate system at the acquisition time of each frame image, and the 3D point cloud structure represents the sparse 3D point set and its spatial coordinates generated by dense matching in the target scene. On this basis, dense point cloud is generated by dense matching of sparse 3D point cloud, and surface reconstruction algorithm is used to convert dense point cloud into 3D mesh model. Then, each frame of aerial image is used as texture source, and texture information of multiple frames of images is projected onto the mesh surface according to the optimal view weight through texture mapping, thereby generating a 3D model of the target scene with realistic color and geometric details. Then, it is loaded into the scene database of VR UAV training system as the spatial base map of virtual flight scene, for trainees to train in operations such as take-off, cruise, hovering, obstacle avoidance and landing in virtual environment.
[0031] In this embodiment of the invention, the present invention constructs a sampling density spatial variation field and performs bundle adjustment based on differentiated weights. This solves the problem of uneven spatial distribution of sampling density caused by wind speed-induced ground speed fluctuations from the data source end, effectively eliminating over-constraint in dense sampling areas and insufficient constraint in sparse sampling areas. This allows the reconstructed model to maintain uniformity of accuracy and geometric consistency across the entire scene. Simultaneously, since the allocation of reconstruction contribution weights does not depend on image content or human experience but is automatically calculated by sampling density indices driven by UAV pose data, it avoids the systematic accuracy loss caused by downwash interference in traditional equal-weight adjustment modes. Therefore, the 3D model output by this invention has a balanced distribution of geometric details and stable scale consistency, directly providing VR UAV training systems with realistic, continuous, and visually abrupt virtual flight scenarios. This ensures that trainees receive consistent visual feedback when making distance judgments and spatial perceptions in different areas of the scene, thereby effectively improving the immersion of training and the effectiveness of operational training.
[0032] like Figure 2 As shown, in a preferred embodiment of the present invention, the step of calculating the actual ground projection spacing sequence between adjacent image frames along the flight trajectory direction based on pose data specifically includes: S101: Obtain the digital elevation model of the target scene and mesh it to obtain the ground elevation value at each grid node; S102, extract the three-dimensional spatial coordinates and attitude angles corresponding to each frame of the image from the pose data, and calculate the angle between the principal optical axis of each frame of the image and the ground normal based on the attitude angle, in order to determine the projection direction of each frame of the image on the ground. S103: Query the corresponding ground elevation value based on the projection position of each frame image in the digital elevation model, and calculate the actual ground distance along the surface between the current frame and its adjacent frames by combining the three-dimensional spatial coordinates and projection direction of each frame image, and use it as the initial ground projection spacing. S104, the initial ground projection spacing is corrected according to the elevation difference between the projection positions of two adjacent frames to obtain the actual ground projection spacing sequence after eliminating the influence of terrain.
[0033] In this embodiment of the invention, an existing digital elevation model (DEM) of the target scene is first acquired. This DEM can be derived from publicly available global DEM data (such as ASTERGDEM or SRTM) or pre-scanned using a lidar mounted on a UAV before modeling. The DEM is then meshed according to a preset resolution (e.g., a 1-meter or 5-meter grid matching the ground sampling interval of the aerial image) to obtain the ground elevation value at each grid node. The three-dimensional spatial coordinates and attitude angles (including pitch, roll, and yaw angles) corresponding to each frame of the image are extracted from the UAV POS data (i.e., pose data). The angle between the principal optical axis of the frame and the ground normal is calculated based on the attitude angle. This angle determines the projection direction of the image center point on the ground and the coverage area of the image on the ground, and is used to determine the projection direction of each frame of the image on the ground.
[0034] It should be noted that in traditional methods, the ground distance between adjacent image frames is usually calculated directly using the Euclidean distance or horizontal projection distance between the three-dimensional spatial coordinates in the UAV pose data. However, when there are significant terrain undulations in the target scene, neither of these two distances can accurately reflect the actual flight distance of the UAV along the surface morphology, leading to systematic deviations in subsequent sampling density calculations. To address this, this implementation introduces a digital elevation model (DEM). When calculating the actual ground distance along the surface surface between the current frame and its adjacent frames, the line segment between the projected positions of the current frame and the adjacent frames is divided into several small segments according to the DEM grid. The slope distance of each small segment is calculated based on the ground elevation values at its two endpoints. The slope distances of all small segments are summed to obtain the actual ground distance along the surface surface, which is used as the initial ground projection distance. Furthermore, considering the limited resolution of the digital elevation model grid, the actual ground distances along the surface curves calculated by the piecewise accumulation method may still have slight systematic deviations. This embodiment further corrects the initial ground projection spacing based on the elevation difference between adjacent projection positions. When adjacent projection positions are on the same elevation plane, the initial ground projection spacing is the horizontal distance and requires no correction. When there is a significant elevation difference between adjacent projection positions, the initial ground projection spacing is trigonometrically corrected based on this elevation difference to obtain the actual ground projection spacing sequence after eliminating terrain influence. This correction operation is a numerical compensation completed in the data post-processing stage. Unlike hardware solutions that avoid terrain influence from the data acquisition source through terrain-following flight, this method has the advantages of low implementation cost and applicability to any existing flight path data.
[0035] The above-mentioned terrain compensation process solves the spacing calculation error caused by directly projecting the UAV's 3D spatial coordinates onto the horizontal plane due to ignoring terrain undulations in traditional methods. This ensures that the subsequently constructed sampling density spatial variation field truly reflects the actual sampling density distribution of the UAV along the surface morphology, rather than false density fluctuations caused by terrain interference. This ensures that the 3D reconstruction accuracy is balanced and consistent across the entire scene, avoiding local model distortion caused by terrain undulations in mountainous or hilly areas.
[0036] like Figure 3 As shown, in a preferred embodiment of the present invention, the step of constructing a spatial variation field characterizing the distribution of the full-scene image sampling density along the flight path and between flight paths based on the actual ground projection spacing sequence specifically includes: S201, associate each spacing value in the actual ground projection spacing sequence with the midpoint of the track between two adjacent frames of images corresponding to that spacing, use it as a control point, and use the wind direction and flight direction angle, terrain slope and flight altitude at that point as the local environmental feature vector of the control point. S202 uses the local environmental feature vectors of all control points as independent variables and the distance between each control point as the dependent variable to establish a geographically weighted regression model, so that different control points have differentiated influence in the regression. S203, based on the results of the geographic weighted regression model, interpolates and predicts the spacing values at each location in the entire scene, and takes the reciprocal of the predicted spacing value as the local sampling density index value at that location, generating a sampling density distribution surface covering the entire scene; S204: For the extrapolation region in the sampling density distribution surface caused by the sparse control points, the interpolation confidence is calculated using the distance between the location to be evaluated and the control points and the environmental similarity as joint indices, and the spatial variation field containing the local sampling density index value and the interpolation confidence is output.
[0037] In this embodiment of the invention, in the local environmental feature vector of the control point, the angle between the wind direction and the flight path direction is used to characterize the spatial influence direction of wind speed on sampling density, the terrain slope is used to characterize the degree of local disturbance of terrain undulation on interpolation accuracy, and the flight altitude is used to characterize the scaling effect of the UAV's collection attitude on the ground sampling interval. These three factors together serve as the independent variable input for the subsequent geographically weighted regression model. Using the local environmental feature vectors of all control points as independent variables and the interval between each control point as the dependent variable, a geographically weighted regression model is established. Geographically weighted regression is a local regression method in the field of spatial statistics. Traditional global regression models assume that the relationship between the independent and dependent variables of all control points is constant, while geographically weighted regression allows control points at different spatial locations to have differentiated influences in the regression. Each control point is assigned a weight matrix that varies with its spatial location during regression fitting; control points closer to the location to be fitted are assigned higher weights, and control points farther away are assigned lower weights. This allows the regression parameters to change continuously in space, enabling a more accurate description of the non-stationarity of environmental factors such as wind speed, terrain, and flight altitude on the sampling density distribution.
[0038] Based on the regression results of the geographic weighted regression model, interpolation prediction is performed on the spacing values at each location in the entire scene to obtain continuous spacing prediction values covering the entire scene. The reciprocal of the predicted spacing value is then used as the local sampling density index value for that location, generating a sampling density distribution surface covering the entire scene. It should be noted that the specific implementation of the interpolation prediction is as follows: For any location to be predicted in the scene, the geographic weighted regression model is used to calculate the predicted spacing value for that location based on the local environmental feature vector (i.e., the angle between the wind direction and the flight path direction, the terrain slope, and the flight altitude). After traversing all locations in the entire scene, a continuous spacing prediction surface covering the entire scene is obtained, and the reciprocal of this value is the sampling density distribution surface. For the extrapolated regions in the sampling density distribution surface caused by sparse control points (i.e., regions far from all control points and lacking sufficient control points to support regression prediction), the interpolation confidence score is calculated using the spatial distance between the location to be evaluated and the nearest control point, and the similarity between the environmental features of the location to be evaluated and the environmental features of each control point, as joint indicators (the closer the spatial distance, the higher the confidence score; the higher the similarity of environmental features, the higher the confidence score; conversely, the greater the spatial distance, the lower the confidence score). This quantifies the reliability of the prediction results at each location in the extrapolated region. Finally, a spatial variation field containing the local sampling density index value and the corresponding interpolation confidence score at each location is output, providing a data basis for the allocation of contribution weights for subsequent image reconstruction in each frame. Through the above processing combining geographically weighted regression and confidence assessment, scenes with uneven control point distribution can be adaptively handled, effectively suppressing density prediction distortion caused by excessive extrapolation in sparse control point regions, and ensuring that the spatial variation field has reliable density indication capabilities across the entire scene.
[0039] It should be noted that the similarity is used to measure the degree of closeness between the environmental features of the location to be evaluated and the environmental features of each control point. Let the local environmental feature vector of any control point consist of three components: the angle between the wind direction and the flight path, the terrain slope, and the flight altitude. The location to be evaluated also extracts these three environmental parameters to form its local environmental feature vector. The environmental similarity between the location to be evaluated and a certain control point is quantified by calculating the weighted Euclidean distance between their feature vectors. Each of the three components is multiplied by a preset weight coefficient before the distance is calculated; the smaller the distance, the closer the environmental features. Since the angle between the wind direction and the flight path has the most direct impact on sampling density, its corresponding weight coefficient is set to the largest; the terrain slope mainly affects the local accuracy of interpolation prediction, and its corresponding weight coefficient is set to a medium level; the flight altitude has a relatively stable impact on sampling density, and its corresponding weight coefficient is set to the smallest level. After calculating the weighted Euclidean distance between the location to be evaluated and all control points, the similarity corresponding to the minimum distance value is taken as the environmental similarity index for that location. For example, if a location to be evaluated has a wind direction angle of 35°, a terrain slope of 8°, and a flight altitude of 120m, while a control point has a wind direction angle of 40°, a terrain slope of 10°, and a flight altitude of 115m, then the weighted Euclidean distance between the two is small, indicating a high degree of environmental similarity. This control point has high reference value for interpolation prediction of the location to be evaluated. Conversely, if another control point has a wind direction angle of 10°, a terrain slope of 2°, and a flight altitude of 150m, then the weighted Euclidean distance is large, indicating a low degree of environmental similarity, and the reference value of this control point is limited.
[0040] like Figure 4 As shown, in a preferred embodiment of the present invention, the step of assigning an independent reconstruction contribution weight to each frame of image according to a preset mapping function specifically includes: S301, set a first threshold and a second threshold greater than the first threshold, and use the two as the judgment boundary to divide the range of local sampling density index values into high-density area, balanced area and low-density area; S302, when the local sampling density index is less than the first threshold, the frame image is determined to be located in a high-density area, and the reconstruction contribution weight of the frame image is reduced according to the exponential decay law. S303, when the local sampling density index is between the first threshold and the second threshold, the frame image is determined to be in the balance region, and the reconstruction contribution weight of the frame image is kept at the baseline value; S304 When the local sampling density index is greater than the second threshold, the frame image is determined to be located in a low-density area. The reconstruction contribution weight of the frame image is increased according to the logarithmic growth law, and it does not exceed the preset upper limit.
[0041] In this embodiment of the invention, the range of local sampling density index values is divided into high-density, balanced, and low-density regions using a set first threshold and a second threshold. The first threshold corresponds to the lower limit of normal sampling density, and the second threshold corresponds to the upper limit of normal sampling density. When the local sampling density index value of a certain frame image is less than the first threshold, it indicates that the sampling density of the region where the frame image is located is too high (i.e., the actual ground projection spacing between adjacent images is too small, and the images are densely packed). When the local sampling density index value is between the first threshold and the second threshold, it indicates that the sampling density of the region where the frame image is located is within the normal range. When the local sampling density index value is greater than the second threshold, it indicates that the sampling density of the region where the frame image is located is too low (i.e., the actual ground projection spacing between adjacent images is too large, and the image distribution is sparse). The three processing methods are as follows: When the local sampling density index is less than the first threshold (i.e., located in a high-density region), it indicates that the sampling density of the region where the image frame is located is too high, and there is a high degree of redundancy between adjacent images. If it participates in the optimization with equal weights in bundle adjustment, this region will be over-constrained due to the superposition of a large number of redundant observations, leading to overfitting or local distortion of the model in this region, thereby destroying the global consistency of the model. To address this, this implementation reduces the reconstruction contribution weight of the image frame according to the exponential decay law. The characteristic of exponential decay is that after the density index is lower than the first threshold, the weight decreases rapidly as the density index decreases, thereby effectively suppressing the excessive influence of redundant observations in high-density regions on the adjustment objective function and avoiding local overconstraint. When the local sampling density index is between the first threshold and the second threshold (i.e., in the equilibrium region), it indicates that the sampling density of the region where the frame image is located is within the normal range, with neither excessive redundancy nor insufficient constraint. The reconstruction contribution weight of the frame image is kept at the baseline value to maintain the standard constraint strength of the region in bundle adjustment. When the local sampling density index exceeds the second threshold (i.e., it is located in a low-density region), it indicates that the sampling density of the region where the image frame is located is too low, the interval between adjacent images is too large, and the sparse sampling region has weak constraints in bundle adjustment due to insufficient available images, causing the model to be prone to geometric drift or deformation in this region. Therefore, this implementation increases the reconstruction contribution weight of the image frame according to the logarithmic growth law, and the increased weight does not exceed a preset upper limit. The characteristic of logarithmic growth is that after the density index exceeds the second threshold, the weight increases with the increase of the density index, but the growth rate gradually slows down, avoiding the amplification of the negative impact of observation noise that may exist in the sparse region due to excessive weight. At the same time, the setting of the upper limit of the weight further ensures the numerical stability of the optimization process. It should be noted that the preset upper limit can be set according to the scale of the target scene and the distribution of control points. For example, for scenes with fewer control points, the upper limit can be appropriately increased to enhance the constraint of sparse regions; for scenes with more control points, the upper limit can be appropriately decreased to avoid the optimization process becoming unstable due to excessive weight of a single image.
[0042] By using the differentiated weight allocation based on the three-segment mapping function, adaptive constraint control is achieved for regions with different sampling densities (reducing weight in dense regions to suppress over-constraint, maintaining balance in balanced regions to preserve normal constraints, and increasing weight in sparse regions to enhance constraints). This ensures that the contribution of the observed values of each frame in bundle adjustment matches its actual information content, avoiding the coexistence of overfitting in dense regions and under-constraint in sparse regions under the traditional equal-weight adjustment method.
[0043] like Figure 5 As shown, in a preferred embodiment of the present invention, the step of setting a first threshold and a second threshold greater than the first threshold specifically includes: S3011, extract scale-invariant feature transformation feature points of each frame image, count the number of feature points of each frame image, and use the mean and standard deviation of the number of feature points of all frames images as the quantitative index of scene texture richness. S3012, calculate the first threshold based on the mean and standard deviation, wherein the first threshold is equal to the mean minus the standard deviation by a first multiple; S3013, Calculate the second threshold based on the mean and standard deviation, wherein the second threshold is equal to the mean plus the standard deviation of the second multiple, and the first multiple and the second multiple are preset normal numbers; S3014, the calculated first threshold and second threshold are used as the judgment boundary of the mapping function.
[0044] In this embodiment of the invention, firstly, Scale Invariant Feature Transform (SIFT) feature points are extracted from each frame of the image, and the total number of feature points extracted in each frame is counted. SIFT feature points are local features that are robust to changes in image scale, rotation, and illumination. The extraction process is as follows: a Difference of Gaussian pyramid is constructed for the image; local extrema are detected in each layer of the pyramid as candidate feature points; and unstable points are eliminated by edge response suppression and contrast thresholding of the candidate points, ultimately obtaining a set of stable feature points with scale invariance. The number of the above feature points in each frame is counted, and the mean and standard deviation of the number of feature points in all frames are used as quantitative indicators of scene texture richness. The mean reflects the overall texture richness of the target scene; a higher mean indicates more stable feature points that can be extracted from the scene, i.e., richer texture. The standard deviation reflects the degree of fluctuation in texture distribution within the scene; a larger standard deviation indicates more obvious texture differences between different regions in the scene. Then, a first threshold and a second threshold are calculated based on the mean and standard deviation. It should be noted that the specific values of the first and second multipliers can be adjusted according to the actual application scenario. In most cases, setting the first and second multipliers to the same value (e.g., both set to 1.0), with one standard deviation on each side of the mean as the threshold interval, can identify and classify image frames with sampling densities that deviate significantly from the normal range into high-density or low-density areas based on the natural dispersion of the data distribution. If the overall texture richness of the target scene is extremely high or low, the width of the threshold interval can be controlled by adjusting the values of the first and second multipliers. The smaller the value, the narrower the threshold interval, and more frames will be identified as high-density or low-density areas, with a larger weight adjustment range; while the larger the value, the wider the threshold interval, and more frames will be identified as balanced areas, with a more conservative weight adjustment. In addition, the first and second multipliers can be set to different values. For example, when the feature points in the high-density area of the scene are more concentrated than those in the low-density area, the value of the first multiplier can be increased to narrow the judgment range of the high-density area, while the value of the second multiplier can be decreased to expand the judgment range of the low-density area. This makes the weight adjustment strategy more focused on the area that really needs to be corrected, rather than fixed to the same constant.
[0045] For example, a target scene has 100 frames. After extracting SIFT feature points, the number of feature points in each frame is calculated, with a mean of 800 and a standard deviation of 200. The first and second thresholds are both set to 1.0. Therefore, the first threshold = 800 - 1.0 × 200 = 600, and the second threshold = 800 + 1.0 × 200 = 1000. In this case, frames with fewer than 600 feature points are considered to be in high-density areas (indicating rich texture and dense features; densely sampled frames can provide sufficient constraints during reconstruction, and their weights should be reduced). Frames with between 600 and 1000 feature points are considered to be in balanced areas, and frames with more than 1000 feature points are considered to be in low-density areas (indicating poor texture and sparse features; their weights need to be increased to enhance constraints). It is important to emphasize that the terms "high-density area" and "low-density area" refer to the sampling density of the image frame itself, not the number of feature points. That is, image frames with fewer feature points correspond to densely sampled regions, while those with more feature points correspond to sparsely sampled regions. If the overall texture richness of a scene is low (e.g., a water scene), with a mean of only 300 and a standard deviation of 80, then the first threshold = 300 - 1.0 × 80 = 220, and the second threshold = 300 + 1.0 × 80 = 380. The threshold range is shifted downwards to ensure that the judgment boundary of the mapping function can adapt to the actual texture conditions of the scene, preventing all frames from being misclassified as the same region due to the overall low texture richness of the scene. If the feature points in high-density areas of a scene are more concentrated than those in low-density areas, the first multiplier is set to 1.2 (first threshold = 800 - 1.2 × 200 = 560, reducing the judgment range for high-density areas), and the second multiplier is set to 0.8 (second threshold = 800 + 0.8 × 200 = 960, expanding the judgment range for low-density areas). This makes the weight adjustment strategy more focused on the areas that truly need correction. Through the above adaptive threshold calculation method based on SIFT feature point statistics, scenes with different texture richness can automatically obtain appropriate threshold parameters, effectively avoiding the problem of fixed thresholds failing in different scenes.
[0046] like Figure 6 As shown, in a preferred embodiment of the present invention, the step of introducing the reconstruction contribution weight into the objective function of the bundle adjustment of 3D reconstruction to uniformly weight all observations in each frame of the image specifically includes: S401, Construct the objective function for bundle adjustment, where the objective function is the sum of squares of the reprojection errors of all observations; S402, the reconstruction contribution weight of each frame image is used as the common coefficient of all observations in that frame image, and multiplied by the square of the reprojection error of each observation corresponding to that frame image; S403, sum the squared terms of the reprojection errors of each weighted observation to form a weighted objective function; S404, the weighted objective function is used to replace the equal weight objective function in the traditional bundle adjustment as the objective of iterative optimization.
[0047] In this embodiment of the invention, bundle adjustment is a core step in 3D reconstruction used to simultaneously optimize camera parameters and 3D point coordinates. Its objective function is typically defined as the sum of squares of the reprojection errors of all observations, i.e., the sum of the squares of the differences between the pixel coordinates projected onto the image plane and the actual observed pixel coordinates of each 3D point in each frame. Since the sampling density varies across different frames, if all observations are treated equally in the objective function, redundant observations in densely sampled regions will dominate the optimization process, causing the geometric constraints of sparsely sampled regions to be overwhelmed. Therefore, this embodiment uses the reconstruction contribution weight of each frame as a common coefficient for all observations in that frame, multiplying it by the squared reprojection errors of each observation in that frame. This reduces the contribution of observations in high-density image frames to the objective function due to the reduced weight, while increasing the contribution of observations in low-density image frames to the objective function due to the increased weight. Then, the weighted squared reprojection errors of each observation are summed to form a weighted objective function, which replaces the equal-weight objective function in traditional bundle adjustment as the target for iterative optimization. The weighted objective function is mathematically consistent with the objective function of conventional bundle adjustment. The only difference is that each reprojection error is multiplied by the weight coefficient of its corresponding image frame. Therefore, the standard Levonburg-Marquardt algorithm can be used for iteration during optimization without additional algorithm adjustments. It has the advantages of simple implementation and strong compatibility. At the same time, the weighting mechanism allows the adjustment process to adaptively adjust the influence of the observations according to the sampling density of each region, effectively suppressing overfitting in densely sampled regions and geometric drift in sparsely sampled regions.
[0048] like Figure 7 As shown, this embodiment of the invention also provides a VR-based drone scene modeling system, the system comprising: The sequence acquisition module 100 is used to acquire multiple frames of aerial images and their corresponding pose data collected by the UAV in the target scene, and to calculate the actual ground projection spacing sequence between each adjacent image frame along the flight trajectory direction based on the pose data. Density field construction module 200 is used to construct a spatial variation field representing the distribution of the sampling density of the whole scene image along the flight path and between flight paths, based on the actual ground projection spacing sequence; The weight allocation module 300 is used to determine the local sampling density index corresponding to each frame of the image based on the spatial variation field, and to assign an independent reconstruction contribution weight to each frame of the image according to the preset mapping function, so that the weight of dense sampling areas is reduced and the weight of sparse sampling areas is increased. The weighted output module 400 is used to introduce the reconstruction contribution weight into the bundle adjustment objective function of the 3D reconstruction, so as to uniformly weight all observations in each frame image, solve the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame image, and output the target scene 3D model optimized by sampling density equalization constraint, which can be loaded into the VR drone training system as a virtual flight scene.
[0049] This system adopts a modular architecture design, including a sequence acquisition module 100, a density field construction module 200, a weight allocation module 300, and a weighted output module 400 connected in sequence. The sequence acquisition module 100 serves as the system's data input, acquiring multiple frames of aerial images captured by the UAV in the target scene and their corresponding pose data. Based on the pose data, it calculates the actual ground projection spacing sequence between adjacent image frames along the flight trajectory, providing basic data for subsequent density field construction. The density field construction module 200 receives the actual ground projection spacing sequence output by the sequence acquisition module 100 and uses it to construct a spatial variation field representing the distribution of image sampling density along and between flight paths, expanding the one-dimensional spacing sequence into two-dimensional density distribution information covering the entire scene. The weight allocation module 300 receives the spatial variation field output by the density field construction module 200 and uses it to determine the local sampling density index corresponding to each image frame based on the spatial variation field, and calculates the density index according to a preset mapping function. Each frame of the image is assigned an independent reconstruction contribution weight, which reduces the weight of densely sampled regions and increases the weight of sparsely sampled regions, realizing the mapping transformation of the sampling density distribution to the weight matrix. The weighted output module 400 serves as the output of the system and is connected to the weight allocation module 300 and the sequence acquisition module 100 respectively. It is used to receive the reconstruction contribution weights output by the weight allocation module 300 and the frame images and their corresponding pose data output by the sequence acquisition module 100. The reconstruction contribution weights are introduced into the bundle adjustment objective function of the 3D reconstruction to uniformly weight all observations in each frame image, solve the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame image, and output the target scene 3D model optimized by sampling density equalization constraints, which is then loaded into the VR UAV training system as a virtual flight scene.
[0050] like Figure 8 As shown, in a preferred embodiment of the present invention, the sequence acquisition module 100 includes: Mesh building unit 101 is used to acquire the digital elevation model of the target scene and mesh it to obtain the ground elevation value at each grid node. The projection calculation unit 102 is used to extract the three-dimensional spatial coordinates and attitude angles corresponding to each frame of the image from the pose data, and calculate the angle between the principal optical axis of each frame of the image and the ground normal based on the attitude angle, so as to determine the projection direction of each frame of the image on the ground. The distance calculation unit 103 is used to query the corresponding ground elevation value according to the projection position of each frame image in the digital elevation model, and calculate the actual ground distance along the surface of the current frame and its adjacent frames by combining the three-dimensional spatial coordinates and projection direction of each frame image, and use it as the initial ground projection spacing. The deviation correction unit 104 is used to correct the initial ground projection spacing based on the elevation difference between the projection positions of two adjacent frames, so as to obtain the actual ground projection spacing sequence after eliminating the influence of terrain.
[0051] like Figure 9 As shown, in a preferred embodiment of the present invention, the density field construction module 200 includes: The control point calibration unit 201 is used to associate each spacing value in the actual ground projection spacing sequence with the midpoint position of the track between two adjacent frames of images corresponding to that spacing, and use it as a control point. The angle between the wind direction and the flight direction, the terrain slope and the flight altitude at that point are used as the local environmental feature vector of the control point. The regression modeling unit 202 is used to establish a geographically weighted regression model with the local environmental feature vectors of all control points as independent variables and the distance between each control point as the dependent variable, so that different control points have differentiated influence in the regression. The surface generation unit 203 is used to interpolate and predict the spacing values of each location in the entire scene based on the results of the geographic weighted regression model, and take the reciprocal of the predicted spacing value as the local sampling density index value of that location to generate a sampling density distribution surface covering the entire scene. The evaluation output unit 204 is used to calculate the interpolation confidence level for the extrapolation region in the sampling density distribution surface caused by the sparse control points, using the distance between the location to be evaluated and the control points and the environmental similarity as joint indices, and outputs a spatial variation field containing local sampling density index values and interpolation confidence levels.
[0052] like Figure 10 As shown, in a preferred embodiment of the present invention, the weight allocation module 300 includes: The partition setting unit 301 is used to set a first threshold and a second threshold greater than the first threshold, and use the two as judgment boundaries to divide the range of local sampling density index values into high-density areas, balanced areas and low-density areas. The weight reduction processing unit 302 is used to determine that the frame image is located in a high-density area when the local sampling density index is less than the first threshold, and reduce the reconstruction contribution weight of the frame image according to the exponential decay law. The holding processing unit 303 is used to determine that the frame image is located in the balance region when the local sampling density index is between the first threshold and the second threshold, and the reconstruction contribution weight of the frame image is kept at the baseline value. The weighting processing unit 304 is used to determine that the frame image is located in a low-density area when the local sampling density index is greater than the second threshold, and to increase the reconstruction contribution weight of the frame image according to the logarithmic growth law, and the weighting does not exceed the preset upper limit.
[0053] The above description only details the preferred embodiments of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
[0054] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.
[0055] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the disclosure in the specification and embodiments. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.
Claims
1. A VR-based drone scene modeling method, characterized in that, The method includes the following steps: Acquire multiple frames of aerial images captured by the UAV in the target scene and their corresponding pose data, and calculate the actual ground projection spacing sequence between each adjacent image frame along the flight trajectory direction based on the pose data; A spatial variation field representing the distribution of full-scene image sampling density along the flight path and between flight paths is constructed based on the actual ground projection spacing sequence. The local sampling density index corresponding to each frame of the image is determined based on the spatial variation field, and an independent reconstruction contribution weight is assigned to each frame of the image according to the preset mapping function, so that the weight of dense sampling regions is reduced and the weight of sparse sampling regions is increased. The reconstruction contribution weight is introduced into the bundle adjustment objective function of 3D reconstruction to uniformly weight all observations in each frame of the image, solve the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame of the image, and output the target scene 3D model optimized by sampling density equalization constraint, which can be loaded into the VR drone training system as a virtual flight scene.
2. The VR-based drone scene modeling method according to claim 1, characterized in that, The step of calculating the actual ground projection spacing sequence between adjacent image frames along the flight trajectory direction based on pose data specifically includes: Obtain the digital elevation model of the target scene and mesh it to obtain the ground elevation value at each grid node; Extract the three-dimensional spatial coordinates and attitude angles corresponding to each frame of the image from the pose data, and calculate the angle between the principal optical axis of each frame of the image and the ground normal based on the attitude angles, in order to determine the projection direction of each frame of the image on the ground. The corresponding ground elevation value is queried based on the projection position of each frame image in the digital elevation model, and the actual ground distance along the surface between the current frame and its adjacent frames is calculated by combining the three-dimensional spatial coordinates and projection direction of each frame image, and this is used as the initial ground projection spacing. The initial ground projection spacing is corrected based on the elevation difference between the projection positions of two adjacent frames to obtain the actual ground projection spacing sequence after eliminating the influence of terrain.
3. The VR-based drone scene modeling method according to claim 1, characterized in that, The step of constructing a spatial variation field characterizing the distribution of the full-scene image sampling density along the flight path and between flight paths based on the actual ground projection spacing sequence specifically includes: Each spacing value in the actual ground projection spacing sequence is associated with the midpoint of the flight path between two adjacent frames corresponding to that spacing, and this is used as a control point. The wind direction and flight path angle, terrain slope, and flight altitude at that point are used as the local environmental feature vector of the control point. Using the local environmental feature vectors of all control points as independent variables and the distance between each control point as the dependent variable, a geographically weighted regression model is established to ensure that different control points have differentiated influences in the regression. Based on the results of the geographic weighted regression model, the spacing values at each location in the entire scene are interpolated and predicted. The reciprocal of the predicted spacing value is then used as the local sampling density index value at that location, generating a sampling density distribution surface covering the entire scene. For the extrapolated region in the sampling density distribution surface caused by the sparse control points, the interpolation confidence is calculated using the distance between the location to be evaluated and the control points and the environmental similarity as joint indices, and the spatial variation field containing the local sampling density index value and the interpolation confidence is output.
4. The VR-based drone scene modeling method according to claim 1, characterized in that, The step of assigning an independent reconstruction contribution weight to each frame of image according to a preset mapping function specifically includes: A first threshold and a second threshold greater than the first threshold are set, and the range of local sampling density index values is divided into high-density area, balanced area and low-density area by using the two as the judgment boundary. When the local sampling density index is less than the first threshold, the frame image is determined to be located in a high-density area, and the reconstruction contribution weight of the frame image is reduced according to the exponential decay law. When the local sampling density index is between the first threshold and the second threshold, the frame image is determined to be in the balance region, and the reconstruction contribution weight of the frame image remains at the baseline value. When the local sampling density index is greater than the second threshold, the image frame is determined to be located in a low-density area. The reconstruction contribution weight of the image frame is increased according to the logarithmic growth law, and it does not exceed the preset upper limit.
5. The VR-based drone scene modeling method according to claim 4, characterized in that, The steps of setting the first threshold and the second threshold greater than the first threshold specifically include: Scale-invariant feature transform feature points are extracted from each frame of the image. The number of feature points in each frame of the image is counted, and the mean and standard deviation of the number of feature points in all frames of the image are used as quantitative indicators of scene texture richness. The first threshold is calculated based on the mean and standard deviation, wherein the first threshold is equal to the mean minus the standard deviation by a first multiple; The second threshold is calculated based on the mean and standard deviation, wherein the second threshold is equal to the mean plus the standard deviation of the second multiple, and the first multiple and the second multiple are preset normal numbers; The calculated first and second thresholds are used as the judgment boundaries of the mapping function.
6. The VR-based drone scene modeling method according to claim 1, characterized in that, The step of incorporating reconstruction contribution weights into the objective function of bundle adjustment for 3D reconstruction to uniformly weight all observations in each frame of the image specifically includes: Construct the objective function for bundle adjustment, which is the sum of squares of the reprojection errors of all observations; The reconstruction contribution weight of each frame is used as the common coefficient of all observations in that frame, and multiplied by the square of the reprojection error of each observation corresponding to that frame. The weighted objective function is formed by summing the squared terms of the reprojection errors of each observation after weighting. The weighted objective function is used to replace the equal-weight objective function in traditional bundle adjustment, and is used as the objective for iterative optimization.
7. A VR-based drone scene modeling system, characterized in that, The system includes: The sequence acquisition module is used to acquire multiple frames of aerial images and their corresponding pose data collected by the UAV in the target scene, and to calculate the actual ground projection spacing sequence between each adjacent image frame along the flight trajectory direction based on the pose data. The density field construction module is used to construct a spatial variation field representing the distribution of the sampling density of the whole scene image along the flight path and between flight paths, based on the actual ground projection spacing sequence. The weight allocation module is used to determine the local sampling density index corresponding to each frame of the image based on the spatial variation field, and to assign an independent reconstruction contribution weight to each frame of the image according to the preset mapping function, so that the weight of dense sampling areas is reduced and the weight of sparse sampling areas is increased. The weighted output module is used to introduce the reconstruction contribution weight into the bundle adjustment objective function of 3D reconstruction, so as to uniformly weight all observations in each frame image, solve the camera exterior orientation parameters and 3D point cloud structure at the acquisition time corresponding to each frame image, and output the target scene 3D model optimized by sampling density equalization constraint, which can be loaded into the VR drone training system as a virtual flight scene.
8. The VR-based drone scene modeling system according to claim 7, characterized in that, The sequence acquisition module includes: Mesh building units are used to acquire the digital elevation model of the target scene and mesh it to obtain the ground elevation value at each grid node. The projection calculation unit is used to extract the three-dimensional spatial coordinates and attitude angles corresponding to each frame of the image from the pose data, and calculate the angle between the principal optical axis of each frame of the image and the ground normal based on the attitude angle, so as to determine the projection direction of each frame of the image on the ground. The distance calculation unit is used to query the corresponding ground elevation value based on the projection position of each frame image in the digital elevation model, and calculate the actual ground distance along the surface of the current frame and its adjacent frames by combining the three-dimensional spatial coordinates and projection direction of each frame image, and use it as the initial ground projection spacing. The deviation correction unit is used to correct the initial ground projection spacing based on the elevation difference between the projection positions of two adjacent frames, so as to obtain the actual ground projection spacing sequence after eliminating the influence of terrain.
9. The VR-based drone scene modeling system according to claim 7, characterized in that, The density field construction module includes: The control point calibration unit is used to associate each spacing value in the actual ground projection spacing sequence with the midpoint of the track between two adjacent frames of images corresponding to that spacing, and use it as a control point. The angle between the wind direction and the flight direction, the terrain slope, and the flight altitude at that point are used as the local environmental feature vector of the control point. The regression modeling unit is used to establish a geographically weighted regression model with the local environmental feature vectors of all control points as independent variables and the distance between each control point as the dependent variable, so that different control points have differentiated influence in the regression. The surface generation unit is used to interpolate and predict the spacing values of each location in the entire scene based on the results of the geographic weighted regression model, and then take the reciprocal of the predicted spacing value as the local sampling density index value of that location to generate a sampling density distribution surface covering the entire scene. The evaluation output unit is used to calculate the interpolation confidence score for the extrapolation region in the sampling density distribution surface caused by the sparse control points, using the distance of the location to be evaluated from the control points and the environmental similarity as joint indices, and outputs a spatial variation field containing local sampling density index values and interpolation confidence scores.
10. The VR-based drone scene modeling system according to claim 7, characterized in that, The weight allocation module includes: The partition setting unit is used to set a first threshold and a second threshold greater than the first threshold, and use the two as judgment boundaries to divide the range of local sampling density index values into high-density areas, balanced areas and low-density areas. The weight reduction processing unit is used to determine that the frame image is located in a high-density area when the local sampling density index is less than the first threshold, and to reduce the reconstruction contribution weight of the frame image according to the exponential decay law. The holding processing unit is used to determine that the frame image is located in the balance region when the local sampling density index is between the first threshold and the second threshold, and the reconstruction contribution weight of the frame image is kept at the baseline value; The weighting processing unit is used to determine that the frame image is located in a low-density area when the local sampling density index is greater than the second threshold, and to increase the reconstruction contribution weight of the frame image according to the logarithmic growth law, and the weighting does not exceed the preset upper limit.
