Urban scene visual three-dimensional reconstruction-oriented unmanned aerial vehicle route planning method

Abstract

The invention relates to an unmanned aerial vehicle route planning method for urban scene visual three-dimensional reconstruction. The method comprises the following steps: constructing a dual-task learning neural network model; sampling the geometric agent surface of the urban scene, obtaining all visual viewpoints passing through a sampling point, calculating the position of the visual viewpoints relative to the sampling point, and calculating a reconstructability measurement result of the sampling point; constructing a large-scale pose-reconstructability training set according to the visual viewpoint relative pose of each sampling point and the reconstructability measurement result; and training a dual-task learning neural network model according to the large-scale pose-reconstructability training set and a preset dual-task loss function, and adaptively optimizing the number of visual viewpoints and the poses of the visual viewpoints by taking the trained dual-task learning neural network model as a reconstructability predictor. And generating the flight path of the unmanned aerial vehicle according to the optimized visual viewpoint set. By adopting the method, the quality of the three-dimensional reconstruction model of the target scene can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional scene information processing, and in particular to a method for unmanned aerial vehicle (UAV) trajectory planning for visual three-dimensional reconstruction of urban scenes. Background Art

[0002] The UAV trajectory planning method for target scene 3D reconstruction refers to the process of reconstructing the target scene 3D model by planning the UAV flight path, capturing the target scene image with the visual sensor carried by the UAV. Compared with the traditional method, UAV can greatly improve the flexibility of viewpoint selection in 3D space, which makes it possible to obtain high-quality reconstruction data. Existing integrated flight planners, such as Altizure, DJI-Terra, and Pix4D capture, usually generate flight trajectories based on preset rules, such as circular orbits, broken line orbits, etc. Although it can provide a fast and easy-to-operate solution, these preset flight modes do not fully consider the specific geometric structure and object distribution of the target scene, which may lead to oversampling of some areas (such as roofs) and undersampling of other areas (such as building facades, raised areas, and details), affecting the UAV acquisition efficiency and the final reconstruction quality. The field of robotics has recently explored methods for real-time reconstruction and online exploration of scenes using UAVs, emphasizing the planning of the next best viewpoint of the UAV during the process of simultaneous positioning and mapping, but the above methods cannot fully utilize the geometric information of the scene, which may limit the accuracy and efficiency of its algorithm in complex scenes.

[0003] Trajectory planning methods based on scene geometry rely on a rough representation of scene geometry to plan camera poses and drone trajectories. Some of these methods estimate the reconstructability of the scene based on heuristic rules such as scene coverage and viewpoint correlation, but their effectiveness has not been strictly verified; other studies have proposed a reconstructability predictor based on learning methods, using a two-stage optimization strategy to optimize accuracy and integrity losses respectively, but this two-stage training strategy may lead to suboptimal balance on different subtasks. At the same time, when planning viewpoint poses, the problem is non-convex due to the correlation between viewpoints and the visibility between viewpoints and targets, which makes the above methods easily fall into local optimal solutions, resulting in poor quality of the 3D reconstruction model of the target scene. Summary of the invention

[0004] Based on this, it is necessary to provide a UAV trajectory planning method for visual 3D reconstruction of urban scenes, which can improve the quality of the 3D reconstruction model of the target scene, in response to the above technical problems.

[0005] A UAV trajectory planning method for visual three-dimensional reconstruction of urban scenes, the method comprising:

[0006] Constructing a dual-task learning neural network model; the dual-task learning neural network model includes a feature mapping module, an accuracy subtask encoder, a dual-task shared encoder, and an integrity subtask encoder;

[0007] The geometric proxy surface of the urban scene is sampled to obtain sampling points, all visible viewpoints of the sampling points are obtained and the positions of the visible viewpoints relative to the sampling points are calculated to obtain the relative pose of the visible viewpoints; the reconstructability measurement results of the sampling points are calculated; the pose-reconstructability feature pairs are constructed according to the relative pose of the visible viewpoints of each sampling point and the reconstructability measurement results, and a large-scale pose-reconstructability training set is generated using multiple pose-reconstructability feature pairs;

[0008] The dual-task learning neural network model is trained according to a large-scale pose-reconstructability training set and a preset dual-task loss function to obtain a trained dual-task learning neural network model;

[0009] The trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number and pose of visual viewpoints to obtain an optimized set of visual viewpoints; the UAV flight trajectory is generated according to the optimized set of visual viewpoints.

[0010] In one embodiment, calculating the position of the visible viewpoint relative to the sampling point to obtain the relative position and posture of the visible viewpoint includes:

[0011] Calculate the 5D pose of the viewpoint relative to the sampling point and identify the spatial position of the viewpoint, i.e., s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

[0012] In one embodiment, calculating the reconstructability metric result of the sampling point includes:

[0013] The reconstructability metric result of the calculated sampling point is:

[0014]

[0015] in, and Represents the sampling point s j The reconstruction accuracy and completeness errors near the two subtasks are the true values ​​of the reconstruction accuracy and completeness errors of the two subtasks. and represents the point set on R and G within the distance threshold τ, d k Indicates the sampling point s j and point p k The distance between them, R and G represent the reconstructed model and the corresponding true value model respectively. represents a point p on Rk The accuracy subtask error, Denotes a point p on G k The integrity subtask error.

[0016] In one embodiment, the preset dual-task loss function is

[0017]

[0018] in

[0019]

[0020] ω1 and ω2 represent the weight parameters of the two subtasks, L1 acc and L1 comp denote the L1 loss of the reconstructibility metric prediction for the accuracy and completeness subtasks, respectively.

[0021] In one embodiment, the trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints to obtain an optimized set of visual viewpoints, including:

[0022] According to the trained dual-task learning neural network, the reconstruction results of the current visual viewpoint for the sampling point are predicted, and the area with poor reconstruction results is screened. New viewpoints are added to the screened area with poor reconstruction results. The distance between the new viewpoint and the selected sampling point conforms to the Gaussian distribution. All visual viewpoints are filtered according to the preset scoring criteria to obtain the filtered visual viewpoints.

[0023] The viewpoints are optimized according to the contribution of the filtered visual viewpoints to the scene reconstructability, and an optimized visual viewpoint set is obtained.

[0024] In one embodiment, the preset scoring criteria are:

[0025]

[0026] in

[0027]

[0028] v m represents the visible viewpoint, x j , n j is the sampling point s j The position and normal vector, V v To be able to see j A collection of viewpoints.

[0029] In one embodiment, the contribution of the filtered visible viewpoints to the scene reconstructability includes:

[0030] The contribution of the filtered visible viewpoints to the scene reconstructibility is

[0031]

[0032] Among them, {p m} represents the viewpoint v m The set of visible sampling points, Indicates that the viewpoint v is removed m The remaining viewpoints of point p j The reconstructability measurement results.

[0033] The above-mentioned UAV trajectory planning method for visual three-dimensional reconstruction of urban scenes, this application designs a dual-task learning neural network model, jointly optimizes the reconstruction accuracy and integrity subtasks, and is used to predict the viewpoint for the reconstruction metric of the scene, handle the suboptimal balance problem between the reconstruction accuracy and integrity subtasks, obtain high-precision reconstruction prediction results, and design a corresponding large-scale pose-reconstructability training set and loss function for model training, and then use the trained dual-task learning neural network model as a reconstructability predictor to adaptively optimize the number of visible viewpoints and the pose of the visible viewpoints, and obtain visual viewpoints that contribute more to scene reconstruction, greatly improving the scene Reconstructibility, and then generate the UAV flight trajectory based on the optimized visual viewpoint set, capture the target scene image, and achieve high-quality three-dimensional reconstruction of the target scene. Compared with the trajectory planning method based on scene geometry, the three-dimensional reconstruction model generated by the captured image in this application has higher quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a flow chart of a UAV trajectory planning method for visual three-dimensional reconstruction of urban scenes in one embodiment;

[0035] Figure 2 A schematic diagram of the structure of a dual-task learning neural network model in one embodiment;

[0036] Figure 3 A schematic diagram of a process for constructing a pose-reconstructability training set in one embodiment;

[0037] Figure 4 is a schematic diagram of a 5-dimensional vector of a viewpoint relative to a sampling point in another embodiment;

[0038] Figure 5 A schematic diagram of a comparative experimental result of reconstruction prediction in one embodiment;

[0039] Figure 6 A schematic diagram of quantitative comparison experimental results of virtual scene reconstruction in one embodiment;

[0040] Figure 7 A schematic diagram of quantitative comparison experimental results of Church reconstruction in a real scene in one embodiment;

[0041] Figure 8 Schematic diagram of qualitative experimental results of the present application in a Church scene in one embodiment; wherein (a) is a viewpoint and trajectory result diagram, and (b) is a reconstruction result diagram;

[0042] Fig. 9 Schematic diagram of qualitative experimental comparison results of Church scene reconstruction in one embodiment; (a) is a diagram of reconstruction results using the oblique photography method, and (b) is a diagram of reconstruction results using the present application;

[0043] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In one embodiment, Figure 1 As shown, a UAV trajectory planning method for visual 3D reconstruction of urban scenes is provided, comprising the following steps:

[0046] Step 102, constructing a dual-task learning neural network model; the dual-task learning neural network model includes a feature mapping module, an accuracy subtask encoder, a dual-task shared encoder and an integrity subtask encoder.

[0047] This application designs a dual-task learning neural network model to predict the reconstructability measurement results of viewpoints for scenes. The input data dimension of the model is K×5, which consists of 5-dimensional relative vectors of K visual viewpoints of rough geometric proxy sampling points. The output of the model is the reconstructability measurement of the visual viewpoint for the sampling point in the accuracy and completeness subtasks. The detailed structure of the model is as follows Figure 2As shown in the figure, in order to improve the geometric representation ability of viewpoint features, the model first maps the viewpoint vector into 256-dimensional geometric features through a feature mapping module; then the soft parameter shared encoder is used to learn the feature representation of the accuracy and completeness sub-tasks, where the sub-encoders are composed of Transformer Encoder networks. At the same time, in order to capture the preferences of sub-tasks, two trainable vectors representing the relevant preferences of different sub-tasks are input into the structure together with the geometric features; finally, another feature mapping module is used to map the preference features of the two sub-tasks into reconstructability metrics. The size of this value indicates how well the visual viewpoint reconstructs the sampling point area.

[0048] Step 104, sampling the urban scene geometric proxy surface to obtain sampling points, obtaining all visible viewpoints of the sampling points and calculating the positions of the visible viewpoints relative to the sampling points to obtain the relative poses of the visible viewpoints; calculating the reconstructability measurement results of the sampling points; constructing pose-reconstructability feature pairs according to the relative poses of the visible viewpoints and the reconstructability measurement results of each sampling point, and using multiple pose-reconstructability feature pairs to generate a large-scale pose-reconstructability training set.

[0049] The training of the dual-task learning neural network model requires the support of a large-scale training dataset. However, there is currently no public dataset for viewpoint pose-reconstructibility. Therefore, this application proposes a method for constructing a training dataset for the above model. The construction process of the training sample is as follows: Figure 3 As shown, the specific operation process of each step is as follows:

[0050] Sampling the geometric proxy: refers to sampling the rough geometric proxy of the scene to obtain sampling points on the surface of the scene geometric proxy, which is convenient for the subsequent construction of the training set.

[0051] Calculate the relative pose of the viewpoint: For a sampling point s obtained after the above sampling j , we can get the sampling point s j All visible viewpoints and calculate their relative to s j By calculating the 5-dimensional pose of the viewpoint relative to the sampling point, the spatial position of the viewpoint is identified (such as Figure 4 As shown), that is, j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

[0052] Calculate the reconstructability metric of the sampling points: The reconstruction error can be evaluated by calculating the distance between the reconstructed model and the true value model. The inverse of the reconstruction error can be regarded as its reconstructability, which includes two subtasks: accuracy and completeness:

[0053]

[0054] Among them, R and G represent the reconstruction model and the corresponding true value model respectively; represents the accuracy subtask error of point p1 on R, represents the integrity subtask error of point p2 on G; d1 and d2 are the distances from point p1 to G and point p2 to R respectively; Dis(p, s) represents the distance between point p and point s. In order to obtain the sampling point s on the geometric proxy j The rebuildability of the surrounding area, as mentioned above and According to the principle of distance weighted summation, it is mapped to point s j :

[0055]

[0056] in, and Represents the sampling point s j The nearby reconstruction accuracy and completeness errors, i.e., the true values ​​of the reconstructability of the two subtasks; and represents the point set on R and G within the threshold τ; d k Indicates point s j and point p k In this application, the distance threshold τ=20cm.

[0057] Construct pose-reconstructability training set: Through the above calculations, we can obtain the relative pose of each sampling point and its reconstruction measurement results, and then construct pose-reconstructability feature pairs to generate training sets. In this step, 100,000 training data were generated using the Town scene in the UrbanScene 3D dataset.

[0058] Step 106 , training the dual-task learning neural network model according to the large-scale pose-reconstructability training set and the preset dual-task loss function to obtain a trained dual-task learning neural network model.

[0059] Step 108, using the trained dual-task learning neural network model as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints to obtain an optimized set of visual viewpoints; and generating a UAV flight trajectory according to the optimized set of visual viewpoints.

[0060] In order to express the reconstructability more intuitively in the reasoning stage, this application uses the output of the dual-task learning network on the two subtasks and the learning weight results of the dual-task loss function to comprehensively calculate the reconstructability:

[0061]

[0062] It can be found that the reconstruction is finally expressed as the network prediction result and Afterwards, the present application adopts an iterative method to plan the number and position of viewpoints based on the reconstructability prediction result.

[0063] First, the present invention outputs the reconstruction results of the current viewpoint to the sampling point based on the trained dual-task learning neural network model and screens the areas with poor reconstruction results. Then, new viewpoints are added to these areas with poor reconstruction results. The distances of these new viewpoints from the selected sampling points conform to the Gaussian distribution, and the present application filters the new viewpoints with higher quality through the following scoring criteria:

[0064]

[0065] in

[0066]

[0067] v m Indicates the visible viewpoint; x j , n j is the sampling point s j The position and normal vector of V v To be able to see j The scoring criteria encourages the viewpoints to look straight ahead and observe the sampling points from multiple angles. This application then optimizes the viewpoints according to their contribution to the scene reconstructability, and the contribution of the viewpoint depends on its marginal benefit in improving the scene reconstructability:

[0068]

[0069] Among them, {p m} represents the viewpoint v m The set of visible sampling points; Indicates that the viewpoint v is removed m The remaining viewpoints of point p j The reconstructability measurement results.

[0070] The adaptive viewpoint optimization strategy of the present application is to improve the reconstructability of the scene and obtain viewpoints that contribute more to the scene reconstruction. The strategy optimizes the viewpoints in an iterative manner until a preset number of iterations (depending on the scene) is reached, or the reconstructability of the scene is not significantly improved.

[0071] After obtaining the optimal set of visible viewpoints, the drone can capture images based on these viewpoint poses. In order to convert these viewpoints into a continuous path in the air, the present application converts this problem into a traveling salesman problem. The set of viewpoints in three-dimensional space can be abstracted as a fully connected graph, each viewpoint is represented as a node in the graph, and the length of the edge between nodes is the distance between the viewpoints in space. The goal of the traveling salesman is to determine a route with the shortest total distance, which should visit each node and eventually return to the starting point. By solving the traveling salesman problem, the present application can efficiently determine a path with the minimum cost and generate the flight track of the drone accordingly.

[0072] The above-mentioned UAV trajectory planning method for visual three-dimensional reconstruction of urban scenes, this application designs a dual-task learning neural network model, jointly optimizes the reconstruction accuracy and integrity subtasks, and is used to predict the viewpoint for the reconstruction metric of the scene, handle the suboptimal balance problem between the reconstruction accuracy and integrity subtasks, obtain high-precision reconstruction prediction results, and design a corresponding large-scale pose-reconstructability training set and loss function for model training, and then use the trained dual-task learning neural network model as a reconstructability predictor to adaptively optimize the number of visible viewpoints and the pose of the visible viewpoints, and obtain visual viewpoints that contribute more to scene reconstruction, greatly improving the scene Reconstructibility, and then generate the UAV flight trajectory based on the optimized visual viewpoint set, capture the target scene image, and achieve high-quality three-dimensional reconstruction of the target scene. Compared with the trajectory planning method based on scene geometry, the three-dimensional reconstruction model generated by the captured image in this application has higher quality.

[0073] In one embodiment, calculating the position of the visible viewpoint relative to the sampling point to obtain the relative position and posture of the visible viewpoint includes:

[0074] Calculate the 5D pose of the viewpoint relative to the sampling point and identify the spatial position of the viewpoint, i.e., s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

[0075] In one embodiment, calculating the reconstructability metric result of the sampling point includes:

[0076] The reconstructability metric result of the calculated sampling point is:

[0077]

[0078] in, and Represents the sampling point s j The reconstruction accuracy and completeness errors near the two subtasks are the true values ​​of the reconstruction accuracy and completeness errors of the two subtasks. and represents the point set on R and G within the distance threshold τ, d k Indicates the sampling point s j and point p k The distance between them, R and G represent the reconstructed model and the corresponding true value model respectively. represents a point p on R k The accuracy subtask error, Denotes a point p on G k The integrity subtask error.

[0079] In one embodiment, the preset dual-task loss function is

[0080]

[0081] in

[0082]

[0083] ω1 and ω2 represent the weight parameters of the two subtasks, L1 acc and L1 comp denote the L1 loss of the reconstructibility metric prediction for the accuracy and completeness subtasks, respectively.

[0084] In one embodiment, the trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints to obtain an optimized set of visual viewpoints, including:

[0085] According to the trained dual-task learning neural network, the reconstruction results of the current visual viewpoint for the sampling point are predicted, and the area with poor reconstruction results is screened. New viewpoints are added to the screened area with poor reconstruction results. The distance between the new viewpoint and the selected sampling point conforms to the Gaussian distribution. All visual viewpoints are filtered according to the preset scoring criteria to obtain the filtered visual viewpoints.

[0086] The viewpoints are optimized according to the contribution of the filtered visual viewpoints to the scene reconstructability, and an optimized visual viewpoint set is obtained.

[0087] In one embodiment, the preset scoring criteria are:

[0088]

[0089] in

[0090]

[0091] v m represents the visible viewpoint, x j , nj is the sampling point s j The position and normal vector, V v To be able to see j A collection of viewpoints.

[0092] In one embodiment, the contribution of the filtered visible viewpoints to the scene reconstructability includes:

[0093] The contribution of the filtered visible viewpoints to the scene reconstructibility is

[0094]

[0095] Among them, {p m} represents the viewpoint v m The set of visible sampling points, Indicates that the viewpoint v is removed m The remaining viewpoints of point p j The reconstructability measurement results.

[0096] In a specific embodiment, Figure 5 The experimental results of the reconstruction prediction of the dual-task learning neural network model and the previous method are shown. The Spearman correlation coefficient is used to evaluate the correlation between the predicted results and the true results, and the test is conducted on 12 trajectories and reconstruction results in the School scene of the UrbanScene 3D dataset. As can be seen from the figure, the present application has more accurate results for the prediction of reconstruction.

[0097] Figure 6 The quantitative comparative experimental results of virtual scene reconstruction are shown. The experiment is based on three different degrees of geometric proxy expansion of the School, Bridge and Castle virtual scenes on the UrbanScene 3D dataset. It can be found that this application has better results than other methods in terms of both accuracy and completeness.

[0098] Figure 7 A schematic diagram showing the results of a quantitative comparison experiment between the method of the present invention and the oblique photography method in a real scene Church. Figure 8 The following is a schematic diagram showing the qualitative experimental results of this application in the Church scenario; (a) is a viewpoint and trajectory result diagram, and (b) is a reconstruction result diagram. Fig. 9 The qualitative experimental comparison results of this application and the oblique photography method in the Church scene are shown. The experimental results show that the reconstruction results of this application have better reconstruction accuracy and completeness than the oblique photography method, especially in areas with complex geometric shapes and occlusions. At the same time, the experimental results show that although this application is trained based on virtual scenes, it can achieve good results in real scenes and has the potential for practical application.

[0099] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0100] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a drone trajectory planning method for visual three-dimensional reconstruction of urban scenes is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0101] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0102] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0103] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A UAV trajectory planning method for visual 3D reconstruction of urban scenes, characterized in that: The method comprises: Constructing a dual-task learning neural network model; the dual-task learning neural network model includes a feature mapping module, an accuracy subtask encoder, a dual-task shared encoder and an integrity subtask encoder; Sampling the geometric proxy surface of the urban scene to obtain sampling points, obtaining all visible viewpoints of the sampling points and calculating the positions of the visible viewpoints relative to the sampling points to obtain the relative poses of the visible viewpoints; calculating the reconstructability measurement results of the sampling points; constructing pose-reconstructability feature pairs according to the relative poses of the visible viewpoints of each sampling point and the reconstructability measurement results, and generating a large-scale pose-reconstructability training set using multiple pose-reconstructability feature pairs; Training the dual-task learning neural network model according to the large-scale pose-reconstructability training set and a preset dual-task loss function to obtain a trained dual-task learning neural network model; The trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints to obtain an optimized visual viewpoint set; and a UAV flight trajectory is generated according to the optimized visual viewpoint set.

2. The method according to claim 1, characterized in that Calculate the position of the visible viewpoint relative to the sampling point to obtain the relative position and pose of the visible viewpoint, including: Calculate the 5D pose of the viewpoint relative to the sampling point and identify the spatial position of the viewpoint, i.e., s j The local spherical coordinates (ω, φ, d) of the origin, the angle α between the normal direction of the sampling point and the line connecting the sampling point and the viewpoint, and the angle β between the viewpoint direction and the line connecting the viewpoint and the sampling point.

3. The method according to claim 1, characterized in that Calculating the reconstructability measurement result of the sampling point includes: The reconstructability metric result of the sampling point is calculated as in, and Represents the sampling point s j The reconstruction accuracy and completeness errors near the two subtasks are the true values ​​of the reconstruction accuracy and completeness errors of the two subtasks. and represents the point set on R and G within the distance threshold τ, d k Indicates the sampling point s j and point p k The distance between them, R and G represent the reconstructed model and the corresponding true value model respectively. represents a point p on R k The accuracy subtask error, Denotes a point p on G k The integrity subtask error.

4. The method according to claim 1, characterized in that The preset dual-task loss function is: in ω1 and ω2 represent the weight parameters of the two subtasks, L1 acc and L1 comp denote the L1 loss of the reconstructibility metric prediction for the accuracy and completeness subtasks, respectively.

5. The method according to claim 1, characterized in that The trained dual-task learning neural network model is used as a reconstructability predictor to adaptively optimize the number of visual viewpoints and the positions of the visual viewpoints to obtain an optimized set of visual viewpoints, including: Predicting the reconstructability of the current visual viewpoint to the sampling point according to the trained dual-task learning neural network, screening the area with poor reconstruction results, adding new viewpoints to the screened area with poor reconstruction results, the distance between the new viewpoint and the selected sampling point conforms to the Gaussian distribution, and filtering all the visual viewpoints according to a preset scoring criterion to obtain filtered visual viewpoints; The viewpoints are optimized according to the contribution of the filtered visual viewpoints to the scene reconstructability, and an optimized visual viewpoint set is obtained.

6. The method according to claim 5, characterized in that The preset scoring criteria are: in v m represents the visible viewpoint, x j , n j is the sampling point s j The position and normal vector, V v To be able to see j A collection of viewpoints.

7. The method according to claim 5, characterized in that The contribution of the filtered visible viewpoints to the scene reconstructability includes: The contribution of the filtered visible viewpoints to the scene reconstructibility is Among them, {p m } represents the viewpoint v m The set of visible sampling points, Indicates that the viewpoint v is removed m The remaining viewpoints of point p j The reconstructability measurement results.

Images