A UAV trajectory planning method for visual 3D reconstruction of urban scenes

Abstract

The present application relates to a method for UAV trajectory planning for visual three-dimensional reconstruction of urban scenes. The method comprises: constructing a dual-task learning neural network model; sampling the geometric proxy surface of the urban scene, obtaining all visible viewpoints passing through the sampling points and calculating the positions of the visible viewpoints relative to the sampling points, and calculating the reconstructability measurement results of the sampling points; constructing a large-scale pose-reconstructability training set based on the relative pose of the visible viewpoints and the reconstructability measurement results of each sampling point; training the dual-task learning neural network model based on the large-scale pose-reconstructability training set and a pre-set dual-task loss function, using the trained dual-task learning neural network model as a reconstructability predictor to adaptively optimize the number of visible viewpoints and the pose of the visible viewpoints, and generating the UAV flight trajectory based on the optimized set of visible viewpoints. This method can improve the quality of the three-dimensional reconstruction model of the target scene.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional scene information processing, and in particular to a UAV trajectory planning method for visual three-dimensional reconstruction of urban scenes. Background Art

[0002] Unmanned aerial vehicle (UAV) trajectory planning for 3D reconstruction of target scenes involves planning a UAV's flight path, capturing images of the target scene using the UAV's onboard visual sensors, and reconstructing a 3D model of the target scene. Compared to traditional methods, UAVs offer significantly greater flexibility in viewpoint selection in 3D space, enabling the acquisition of high-quality reconstruction data. Existing integrated flight planners, such as Altizure, DJI-Terra, and Pix4D capture, typically generate flight trajectories based on preset rules, such as circular and broken-line trajectories. While these provide a fast and easy-to-use solution, these preset flight patterns fail to fully consider the specific geometry and object distribution of the target scene. This can lead to oversampling of certain areas (e.g., rooftops) and undersampling of other areas (e.g., building facades, raised areas, and detailed features), compromising UAV acquisition efficiency and the quality of the final reconstruction. Recent research in the field of robotics has explored methods for real-time scene reconstruction and online exploration using UAVs, emphasizing the planning of the next optimal viewpoint during simultaneous localization and mapping. However, these methods fail to fully utilize the scene's geometric information, potentially limiting their accuracy and efficiency in complex scenes.

[0003] Trajectory planning methods based on scene geometry rely on a coarse representation of the scene geometry to plan the camera pose and drone trajectory. Some of these methods estimate the scene's reconstructability based on heuristics such as scene coverage and viewpoint correlation, but their effectiveness has not been rigorously verified. Other studies have proposed a learning-based reconstructability predictor that uses a two-stage optimization strategy to optimize accuracy and completeness losses, respectively. However, this two-stage training strategy can lead to suboptimal balances across different subtasks. Furthermore, when planning viewpoint poses, the problem is non-convex due to the correlation between viewpoints and the visibility between the viewpoint and the target. This makes these methods prone to falling into local optimal solutions, resulting in poor quality of the 3D reconstructed model of the target scene. Summary of the Invention

[0004] Based on this, it is necessary to provide a UAV trajectory planning method for visual 3D reconstruction of urban scenes, which can improve the quality of the 3D reconstruction model of the target scene, in order to address the above technical problems.

[0005] A UAV trajectory planning method for visual three-dimensional reconstruction of urban scenes, the method comprising:

[0006] Construct a dual-task learning neural network model; the dual-task learning neural network model includes a feature mapping module, an accuracy subtask encoder, a dual-task shared encoder, and a completeness subtask encoder;

[0007] The geometric proxy surface of the urban scene is sampled to obtain sampling points. All visible viewpoints of the sampling points are obtained and their positions relative to the sampling points are calculated to obtain the relative poses of the visual viewpoints. The reconstructability measurement results of the sampling points are calculated. Based on the relative poses of the visual viewpoints and the reconstructability measurement results of each sampling point, pose-reconstructability feature pairs are constructed. Multiple pose-reconstructability feature pairs are used to generate a large-scale pose-reconstructability training set.

[0008] The dual-task learning neural network model is trained according to a large-scale pose-reconstructability training set and a preset dual-task loss function to obtain a trained dual-task learning neural network model;

[0009] The trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number and pose of visual viewpoints to obtain an optimized visual viewpoint set; the UAV flight trajectory is generated based on the optimized visual viewpoint set.

[0010] In one embodiment, calculating the position of the visible viewpoint relative to the sampling point to obtain the relative pose of the visible viewpoint includes:

[0011] Calculate the 5-dimensional pose of the viewpoint relative to the sampling point and identify the spatial position of the viewpoint, that is, s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

[0012] In one embodiment, calculating the reconstructability metric result of the sampling point includes:

[0013] The reconstructability metric of the calculated sampling points is:

[0014]

[0015] in, and Represents the sampling point s j The reconstruction accuracy and completeness errors near the two subtasks are the true values ​​of the reconstruction accuracy and completeness errors of the two subtasks, and represents the point set on R and G within the distance threshold τ, d k Indicates the sampling point s j and point p k The distance between them, R and G represent the reconstructed model and the corresponding true value model, respectively. represents a point p on Rk The accuracy subtask error, Denotes a point p on G k The integrity subtask error.

[0016] In one embodiment, the preset dual-task loss function is

[0017]

[0018] in

[0019]

[0020] ω1 and ω2 represent the weight parameters of the two subtasks, L1 acc and L1 comp denote the L1 loss of the reconstructability metric prediction for the accuracy and completeness subtasks, respectively.

[0021] In one embodiment, the trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints, thereby obtaining an optimized set of visual viewpoints, including:

[0022] The trained dual-task learning neural network is used to predict the reconstructability of the current visual viewpoint for the sampling point, and the areas with poor reconstruction results are screened out. New viewpoints are added to the areas with poor reconstruction results. The distances between the new viewpoints and the selected sampling points conform to the Gaussian distribution. All visual viewpoints are filtered according to the pre-set scoring criteria to obtain the filtered visual viewpoints.

[0023] The viewpoints are optimized according to the contribution of the filtered visual viewpoints to the scene reconstructability, and an optimized visual viewpoint set is obtained.

[0024] In one embodiment, the pre-set scoring criteria are

[0025]

[0026] in

[0027]

[0028] v m Represents the visual viewpoint, x j , n j is the sampling point s j The position and normal vector, V v To be able to see through j A collection of viewpoints.

[0029] In one embodiment, the contribution of the filtered visual viewpoints to the scene reconstructability includes:

[0030] The contribution of filtered visual viewpoints to scene reconstructibility is

[0031]

[0032] Among them, {p m} represents the viewpoint v m The set of visible sampling points, Indicates removing the viewpoint v m The remaining viewpoints of point p j Reconstructability measurement results.

[0033] The above-mentioned drone trajectory planning method for visual three-dimensional reconstruction of urban scenes, this application designs a dual-task learning neural network model, jointly optimizes the reconstruction accuracy and completeness subtasks, and is used to predict the viewpoint's reconstructability metric for the scene, handles the suboptimal balance problem between the reconstruction accuracy and completeness subtasks, obtains high-precision reconstructability prediction results, and designs a corresponding large-scale pose-reconstructability training set and loss function for model training. The trained dual-task learning neural network model is then used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the pose of the visual viewpoints, obtaining visual viewpoints that contribute more to scene reconstruction, greatly improving the scene's reconstructability. The drone's flight trajectory is then generated based on the optimized set of visual viewpoints to capture target scene images and achieve high-quality three-dimensional reconstruction of the target scene. Compared with the trajectory planning method based on scene geometry, the three-dimensional reconstruction model generated by the captured images in this application has higher quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a UAV trajectory planning method for visual 3D reconstruction of urban scenes in one embodiment is provided;

[0035] Figure 2 A schematic diagram of the structure of a dual-task learning neural network model in one embodiment;

[0036] Figure 3 A schematic diagram of a process for constructing a pose-reconstructability training set in one embodiment;

[0037] Figure 4 is a schematic diagram of a 5-dimensional vector of a viewpoint relative to a sampling point in another embodiment;

[0038] Figure 5 A schematic diagram of a comparative experimental result of reconstructability prediction in one embodiment;

[0039] Figure 6 Schematic diagram of quantitative comparison experimental results of virtual scene reconstruction in one embodiment;

[0040] Figure 7 Schematic diagram of quantitative comparison experimental results of Church reconstruction in a real scene in one embodiment;

[0041] Figure 8 Schematic diagram of qualitative experimental results of the present application in a Church scene in one embodiment; wherein (a) is a viewpoint and trajectory result diagram, and (b) is a reconstruction result diagram;

[0042] Figure 9 Schematic diagram of qualitative experimental comparison results of Church scene reconstruction in one embodiment; (a) is the reconstruction result of the oblique photography method, and (b) is the reconstruction result of the present application;

[0043] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0045] In one embodiment, Figure 1 As shown, a UAV trajectory planning method for visual 3D reconstruction of urban scenes is provided, including the following steps:

[0046] Step 102: construct a dual-task learning neural network model; the dual-task learning neural network model includes a feature mapping module, an accuracy subtask encoder, a dual-task shared encoder, and an integrity subtask encoder.

[0047] This application designs a dual-task learning neural network model to predict the reconstructability metric of a viewpoint for a scene. The input data dimension of the model is K×5, consisting of 5-dimensional relative vectors of K visual viewpoints of rough geometric proxy sampling points. The output of the model is the reconstructability metric of the visual viewpoint for the sampling point in terms of accuracy and completeness subtasks. The detailed structure of the model is as follows Figure 2As shown in the figure, in order to improve the geometric representation ability of viewpoint features, the model first maps the viewpoint vector into 256-dimensional geometric features through a feature mapping module; then uses a soft parameter shared encoder to learn the feature representation of the accuracy and completeness sub-tasks, where the sub-encoders are composed of Transformer Encoder networks. At the same time, in order to capture the preferences of sub-tasks, two trainable vectors representing the relevant preferences of different sub-tasks are input into the structure together with the geometric features; finally, another feature mapping module is used to map the preference features of the two sub-tasks into reconstructability measurement results. The size of this value indicates how well the visual viewpoint reconstructs the sampling point area.

[0048] Step 104: Sample the geometric proxy surface of the urban scene to obtain sampling points, obtain all visible viewpoints of the sampling points, and calculate the positions of the visible viewpoints relative to the sampling points to obtain the relative poses of the visible viewpoints; calculate the reconstructability measurement results of the sampling points; construct pose-reconstructability feature pairs based on the relative pose of the visible viewpoints and the reconstructability measurement results of each sampling point, and use multiple pose-reconstructability feature pairs to generate a large-scale pose-reconstructability training set.

[0049] The training of dual-task learning neural network models requires the support of large-scale training datasets. However, there is currently no public dataset for viewpoint pose-reconstructibility. Therefore, this application proposes a method for constructing a training dataset for the above model. The construction process of training samples is as follows: Figure 3 As shown in the figure, the specific operation process of each step is as follows:

[0050] Sampling the geometric proxy: refers to sampling the rough geometric proxy of the scene to obtain sampling points on the surface of the scene geometric proxy, which is convenient for subsequent construction of the training set.

[0051] Calculate the relative pose of the viewpoint: for a sampling point s obtained after the above sampling j , we can get the sampling point s j All visible viewpoints and calculate their relative to s j By calculating the 5-dimensional pose of the viewpoint relative to the sampling point, the spatial position of the viewpoint is identified (such as Figure 4 As shown), that is, s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

[0052] Calculate the reconstructability metric of the sampling points: The reconstruction error can be evaluated by calculating the distance between the reconstructed model and the true value model. The inverse of the reconstruction error can be regarded as its reconstructability, which includes two subtasks: accuracy and completeness:

[0053]

[0054] Among them, R and G represent the reconstructed model and the corresponding true value model respectively; represents the accuracy subtask error of point p1 on R, represents the integrity subtask error of point p2 on G; d1 and d2 are the distances from point p1 to G and point p2 to R respectively; Dis(p, s) represents the distance between point p and point s. In order to obtain the sampled point s on the geometric proxy j The redevelopment of the surrounding area, and According to the principle of distance weighted summation, it is mapped to point s j :

[0055]

[0056] in, and Represents the sampling point s j The nearby reconstruction accuracy and completeness errors, i.e., the ground truth of the reconstructability of the two subtasks; and represents the point set on R and G within the threshold τ; d k Indicates point s j and point p k In this application, the distance threshold τ = 20 cm.

[0057] Constructing a pose-reconstructability training set: Through the above calculations, we can obtain the relative pose of each sampling point and its reconstruction metric. This allows us to construct pose-reconstructability feature pairs and generate a training set. In this step, we generated 100,000 training data points using the Town scene in the UrbanScene 3D dataset.

[0058] Step 106 : training the dual-task learning neural network model based on the large-scale pose-reconstructability training set and the preset dual-task loss function to obtain a trained dual-task learning neural network model.

[0059] Step 108 : Using the trained dual-task learning neural network model as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints, an optimized visual viewpoint set is obtained; and a UAV flight trajectory is generated based on the optimized visual viewpoint set.

[0060] In order to express the reconstructability more intuitively in the inference stage, this application uses the output of the dual-task learning network on the two subtasks and the learning weight results of the dual-task loss function to comprehensively calculate the reconstructability:

[0061]

[0062] It can be found that the reconstruction is finally expressed as the network prediction result and Afterwards, this application uses an iterative method to plan the number and pose of viewpoints based on the reconstructability prediction result.

[0063] First, based on the trained dual-task learning neural network model, the present invention outputs the reconstruction results of the current viewpoint with respect to the sampling points and filters out areas with poor reconstruction results. New viewpoints are then added to these areas with poor reconstruction results. The distances of these new viewpoints from the selected sampling points conform to a Gaussian distribution. The present invention uses the following scoring criteria to filter out new viewpoints of higher quality:

[0064]

[0065] in

[0066]

[0067] v m Indicates the visual viewpoint; x j , n j is the sampling point s j The position and normal vector of V v To be able to see through j The scoring criteria encourages the viewpoints to look straight ahead and observe the sampling points from multiple angles. This application then optimizes the viewpoints based on their contribution to the scene reconstructibility. The contribution of the viewpoint depends on its marginal benefit in improving the scene reconstructibility:

[0068]

[0069] Among them, {p m} represents the viewpoint v m The set of visible sampling points; Indicates removing the viewpoint v m The remaining viewpoints of point p j Reconstructability measurement results.

[0070] The adaptive viewpoint optimization strategy of this application is designed to improve the reconstructability of the scene and obtain viewpoints that contribute significantly to scene reconstruction. This strategy uses an iterative approach to optimize the viewpoints until a pre-set number of iterations (depending on the scene) is reached or the scene reconstructability is not significantly improved.

[0071] After obtaining the optimal set of visible viewpoints, the drone can capture images based on the poses of these viewpoints. In order to convert these viewpoints into a continuous path in the air, this application converts this problem into a traveling salesman problem. The set of viewpoints in three-dimensional space can be abstracted as a fully connected graph, where each viewpoint is represented as a node in the graph, and the length of the edge between nodes is the distance between the viewpoints in space. The goal of the traveling salesman is to determine a route with the shortest total distance, which should visit each node and eventually return to the starting point. By solving the traveling salesman problem, this application can efficiently determine a path with the minimum cost and generate the flight trajectory of the drone accordingly.

[0072] The above-mentioned drone trajectory planning method for visual three-dimensional reconstruction of urban scenes, this application designs a dual-task learning neural network model, jointly optimizes the reconstruction accuracy and completeness subtasks, and is used to predict the viewpoint's reconstructability metric for the scene, handles the suboptimal balance problem between the reconstruction accuracy and completeness subtasks, obtains high-precision reconstructability prediction results, and designs a corresponding large-scale pose-reconstructability training set and loss function for model training. The trained dual-task learning neural network model is then used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the pose of the visual viewpoints, obtaining visual viewpoints that contribute more to scene reconstruction, greatly improving the scene's reconstructability. The drone's flight trajectory is then generated based on the optimized set of visual viewpoints to capture target scene images and achieve high-quality three-dimensional reconstruction of the target scene. Compared with the trajectory planning method based on scene geometry, the three-dimensional reconstruction model generated by the captured images in this application has higher quality.

[0073] In one embodiment, calculating the position of the visible viewpoint relative to the sampling point to obtain the relative pose of the visible viewpoint includes:

[0074] Calculate the 5-dimensional pose of the viewpoint relative to the sampling point and identify the spatial position of the viewpoint, that is, s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

[0075] In one embodiment, calculating the reconstructability metric result of the sampling point includes:

[0076] The reconstructability metric of the calculated sampling points is:

[0077]

[0078] in, and Represents the sampling point s j The reconstruction accuracy and completeness errors near the two subtasks are the true values ​​of the reconstruction accuracy and completeness errors of the two subtasks, and represents the point set on R and G within the distance threshold τ, d k Indicates the sampling point s j and point p k The distance between them, R and G represent the reconstructed model and the corresponding true value model, respectively. represents a point p on R k The accuracy subtask error, Denotes a point p on G k The integrity subtask error.

[0079] In one embodiment, the preset dual-task loss function is

[0080]

[0081] in

[0082]

[0083] ω1 and ω2 represent the weight parameters of the two subtasks, L1 acc and L1 comp denote the L1 loss of the reconstructability metric prediction for the accuracy and completeness subtasks, respectively.

[0084] In one embodiment, the trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints, thereby obtaining an optimized set of visual viewpoints, including:

[0085] The trained dual-task learning neural network is used to predict the reconstructability of the current visual viewpoint for the sampling point, and the areas with poor reconstruction results are screened out. New viewpoints are added to the areas with poor reconstruction results. The distances between the new viewpoints and the selected sampling points conform to the Gaussian distribution. All visual viewpoints are filtered according to the pre-set scoring criteria to obtain the filtered visual viewpoints.

[0086] The viewpoints are optimized according to the contribution of the filtered visual viewpoints to the scene reconstructability, and an optimized visual viewpoint set is obtained.

[0087] In one embodiment, the pre-set scoring criteria are

[0088]

[0089] in

[0090]

[0091] v m Represents the visual viewpoint, x j , nj is the sampling point s j The position and normal vector, V v To be able to see through j A collection of viewpoints.

[0092] In one embodiment, the contribution of the filtered visual viewpoints to the scene reconstructability includes:

[0093] The contribution of filtered visual viewpoints to scene reconstructibility is

[0094]

[0095] Among them, {p m} represents the viewpoint v m The set of visible sampling points, Indicates removing the viewpoint v m The remaining viewpoints of point p j Reconstructability measurement results.

[0096] In a specific embodiment, Figure 5 This paper presents experimental results comparing the reconstructability prediction performance of a dual-task learning neural network model with previous methods. The Spearman correlation coefficient was used to evaluate the correlation between the predicted and true results. The results were tested on 12 trajectories and reconstructions from the School scene in the UrbanScene 3D dataset. As can be seen from the figure, this application achieves more accurate reconstructability prediction results.

[0097] Figure 6 The quantitative comparative experimental results of virtual scene reconstruction are presented. The experiment is based on three geometric proxies of different degrees of School, Bridge and Castle virtual scenes on the UrbanScene 3D dataset. It can be found that this application has better results than other methods in terms of both accuracy and completeness.

[0098] Figure 7 A schematic diagram shows the quantitative comparison experimental results of the method of the present invention and the oblique photography method in a real scene Church. Figure 8 The following figure shows the qualitative experimental results of this application in the Church scene; (a) is the viewpoint and trajectory result diagram, and (b) is the reconstruction result diagram. Figure 9 This paper presents qualitative experimental comparisons of this application with oblique photography in a church scene. These results demonstrate that the reconstruction results of this application achieve superior accuracy and completeness compared to oblique photography, particularly in areas with complex geometries and occlusions. Furthermore, these results demonstrate that, despite training based on virtual scenes, this application can achieve comparable results in real-world scenarios, demonstrating its potential for practical application.

[0099] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0100] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for drone trajectory planning for visual three-dimensional reconstruction of urban scenes is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0101] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0102] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0103] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A UAV trajectory planning method for visual 3D reconstruction of urban scenes, characterized by: The method comprises: Constructing a dual-task learning neural network model; the dual-task learning neural network model includes a feature mapping module, an accuracy subtask encoder, a dual-task shared encoder, and an integrity subtask encoder; Sampling the geometric proxy surface of the urban scene to obtain sampling points, obtaining all visible viewpoints of the sampling points and calculating the positions of the visible viewpoints relative to the sampling points to obtain relative poses of the visible viewpoints; calculating the reconstructability measurement results of the sampling points; constructing pose-reconstructability feature pairs based on the relative pose of the visible viewpoints and the reconstructability measurement results of each sampling point, and generating a large-scale pose-reconstructability training set using multiple pose-reconstructability feature pairs; Training the dual-task learning neural network model according to the large-scale pose-reconstructability training set and a preset dual-task loss function to obtain a trained dual-task learning neural network model; The trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints to obtain an optimized visual viewpoint set; and a UAV flight trajectory is generated according to the optimized visual viewpoint set; Calculating the reconstructability measurement result of the sampling point includes: The reconstructability metric result of the sampling point is calculated as in, and Represents the sampling point s j The reconstruction accuracy and completeness errors near the two subtasks are the true values ​​of the reconstruction accuracy and completeness errors of the two subtasks, and represents the point set on R and G within the distance threshold τ, d k Indicates the sampling point s j and point p k The distance between them, R and G represent the reconstructed model and the corresponding true value model, respectively. represents a point p on R k The accuracy subtask error, Denotes a point p on G k The integrity subtask error.

2. The method according to claim 1, characterized in that Calculate the position of the visible viewpoint relative to the sampling point to obtain the relative pose of the visible viewpoint, including: Calculate the 5-dimensional pose of the viewpoint relative to the sampling point and identify the spatial position of the viewpoint, that is, s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

3. The method according to claim 1, characterized in that The preset dual-task loss function is in ω1 and ω2 represent the weight parameters of the two subtasks, L1 acc and L1 comp denote the L1 loss of the reconstructability metric prediction for the accuracy and completeness subtasks, respectively.

4. The method according to claim 1, wherein The trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints, thereby obtaining an optimized set of visual viewpoints, including: Predicting the reconstructability of the current visual viewpoint to the sampling point based on the trained dual-task learning neural network, screening areas with poor reconstruction results, adding new viewpoints to the screened areas with poor reconstruction results, where the distances between the new viewpoints and the selected sampling points conform to a Gaussian distribution, and filtering all visual viewpoints based on a preset scoring criterion to obtain filtered visual viewpoints; The viewpoints are optimized according to the contribution of the filtered visual viewpoints to the scene reconstructability, and an optimized visual viewpoint set is obtained.

5. The method according to claim 4, characterized in that The preset scoring criteria are: in v m Indicates the visual viewpoint, xj , n j is the sampling point s j The position and normal vector, V v To be able to see through j A collection of viewpoints.

6. The method according to claim 4, characterized in that The contribution of the filtered visual viewpoints to the scene reconstructability includes: The contribution of filtered visual viewpoints to scene reconstructibility is Among them, {p m } represents the viewpoint v m The set of visible sampling points, Indicates removing the viewpoint v m The remaining viewpoints of point p j Reconstructability measurement results.

Images