Three-dimensional model management method and system based on unmanned aerial vehicle and grid technology

Abstract

The present application relates to model management, in particular to a three-dimensional model management method and system based on unmanned aerial vehicle and grid technology, model modeling: the latitude and longitude range is calculated by extracting the Exif information of the photo, and grouping is carried out according to the grid code, the oblique photography modeling is completed, the model is cut according to the grid boundary line, and the model and model auxiliary information are stored; model updating: the grid of the area needing to be updated is selected, the corresponding route is generated, the unmanned aerial vehicle multi-angle aerial photo is taken, the model is modeled, and the model and model auxiliary information of the grid of the area needing to be updated are stored and updated; model rendering: the underlying nodes are constructed and the adjacent grids are merged to form the bounding volume, the grid boundary line is converted into the node content, the model data is taken as the sub-node and the visual distance precision is configured, the rendering load is optimized through hierarchical merging and LOD simplification; the present application can overcome the defects that it is difficult to divide the unmanned aerial vehicle aerial survey model into spatial structure, automatically update and maintain and efficiently render and display.

Description

Technical Field

[0001] This invention relates to model management, specifically to a method and system for managing 3D models based on UAV and mesh technology. Background Technology

[0002] Currently, drone aerial surveying technology has matured considerably. By equipping drones with high-definition cameras to conduct multi-angle aerial photography, abundant ground imagery data can be acquired. Using mature modeling software, such as DJI Terra and ContextCaptureCenter, these aerial photographs can be processed efficiently to generate high-precision oblique photogrammetry models. These models have wide-ranging applications in fields such as digital twins, smart cities, and geographic information systems, providing intuitive and accurate 3D data support for urban planning, disaster monitoring, and environmental assessment.

[0003] However, current management of UAV aerial survey models is mostly done on a project-based basis, with each project managing its model data independently. This approach is adequate for overall model updates and maintenance, but falls short when partial updates are needed. Partial updates often require professionals to re-generate data, remodel, and update the model, which is not only inefficient but also costly. More importantly, the lack of a spatially based structural partitioning and management method prevents the effective organization and indexing of model data, hindering automated model updates and maintenance. This not only limits the application potential of UAV aerial survey models but also increases the complexity and cost of model management.

[0004] Therefore, developing a 3D model management method and system based on UAV and mesh technology to achieve spatial structural division, automated updating and maintenance, and efficient rendering and display of the model has become an urgent technical problem to be solved. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a three-dimensional model management method and system based on UAV and mesh technology, which can effectively overcome the shortcomings of the existing technology in that it is difficult to perform spatial structural division, automated update and maintenance, and efficient rendering and display of UAV aerial survey models.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] 3D model management methods based on UAVs and mesh technology include:

[0010] Model building: The UAV takes aerial photos from multiple angles, extracts the Exif information of the photos to calculate the latitude and longitude range, and groups them according to the grid code. The oblique photogrammetry modeling is completed by executing multiple parallel modeling tasks. The model is clipped according to the grid boundary line, and the model and its associated information are stored.

[0011] Model update: Select the grid of the area to be updated, generate flight path based on the selected grid, take multi-angle aerial photos from the UAV, perform model modeling, and store and update the model and model-related information of the grid of the area to be updated;

[0012] Model rendering: Extract mesh information, elevation information and piecewise path information contained in the model from the database, construct the bottom-level nodes and merge adjacent meshes to form a bounding volume, while converting the mesh boundary lines into node content, using the model data as child nodes and configuring the view distance precision geometricError, simplifying and optimizing the rendering load through layer merging and LOD, and finally displaying the model.

[0013] Preferably, the modeling includes the following steps:

[0014] S11. Data Acquisition: The UAV performs surveying and mapping tasks based on flight mission and route information. Based on the oblique photogrammetry modeling process, the UAV gimbal adjusts the camera to take photos from multiple angles.

[0015] S12. Photo Grouping: By extracting the Exif information of the photos, the latitude and longitude range of the photos is calculated. Based on the latitude and longitude range and the grid level selected when organizing the model, the grid code is calculated, and all photos are grouped according to the grid code.

[0016] S13. Parallel Modeling: Establish multiple parallel modeling tasks based on grouping information, and perform oblique photogrammetry modeling using modeling software;

[0017] S14. Model cropping: Due to the outward expansion of the model caused by photo grouping, the model edges are not aligned with the mesh edges. Model cropping technology is used to precisely crop the model edges so that the model matches the mesh.

[0018] S15. Data storage: Store the model and its associated information.

[0019] Preferably, data storage in S15 includes storing the model and its associated information, including:

[0020] The cropped model is converted into 3DTiles data format and stored using object storage software;

[0021] The model-related information, including elevation information and other related information, extracted during the execution of surveying and mapping tasks is indexed and stored in a database using the corresponding grid coding.

[0022] Preferably, the grid level adopts a 17-level 512m grid, and the updating and maintenance of model data are based on this level of grid;

[0023] To simplify data indexing and accelerate computation, the grid coding adopts a two-dimensional coding system based on the Earth spatial grid coding rules for planar division. Spatial information is expressed by combining two-dimensional Earth spatial grid coding with bottom and top elevation information.

[0024] Preferably, the model is stored and rendered using the 3DTiles data format. 3DTiles is a data format developed for streaming and rendering of 3D geospatial data, including photogrammetry, 3D architecture, BIM / CAD, instantiated features, and point clouds. It is based on a hierarchical data structure and tile format set that can be transmitted and rendered.

[0025] The model's associated information and grid index information are stored using the relational database MySQL and the non-relational database Elasticsearch.

[0026] Preferably, the model update includes the following steps:

[0027] S21. Region Selection: When the model needs to be locally updated, select the grid of the region to be updated.

[0028] S22. Route Generation: Generates corresponding route information based on the selected grid.

[0029] S23. Remodeling: The UAV performs surveying and mapping tasks based on the flight path information, takes aerial photos from multiple angles, and remodels the model according to the model modeling process.

[0030] S24. Storage Update: Update the grid code of the regional grid as needed, and perform storage update on the corresponding model and model-related information.

[0031] Preferably, in S22, route generation involves generating corresponding route information based on the selected grid, including:

[0032] The oblique photography model requires the drone camera to shoot at multiple angles. The flight path is adaptively expanded according to the camera parameters, aerial shooting altitude and gimbal tilt angle to ensure that the photos in each grid are captured completely.

[0033] Preferably, the model rendering includes the following steps:

[0034] S31. Data Extraction: Extract grid information, elevation information, and 3DTiles path information contained in the model from the database to prepare basic data for subsequent model rendering.

[0035] S32. Mesh Merging: Calculate the bottom-level node information of 3DTiles, merge adjacent meshes, and combine the mesh range, bottom height information and top height information to form a bounding volume to determine the spatial range of the model.

[0036] S33. Node Content Generation: Based on the mesh information contained in the node, the software draws the boundary line of each mesh to form GLB data as the node content, thus clarifying the mesh represented by the node.

[0037] S34. Model Data Import and Setting: Locate the model data based on the grid information contained in the node, import the model data as a child node of the corresponding node, and set the view distance precision geometricError to ensure that the model is displayed within a reasonable view distance.

[0038] S35. Top-level reconstruction and optimization: Merge the mesh information contained in the bottom-level nodes according to the mesh hierarchy to reconstruct the top-level of the model. According to the nesting relationship of the mesh hierarchy, draw boundary lines for the mesh to show whether model data exists in a certain space. Use LOD and simplify the display of mesh boundary lines to reduce the rendering pressure of the model.

[0039] S36. Model Display.

[0040] Preferably, the model display in S36 includes:

[0041] The browser uses rendering tools, including Cesium, to load the root node tileset.json data and finally display the model.

[0042] A 3D model management system based on UAV and mesh technology, including a model modeling unit, a model update unit, and a model rendering unit;

[0043] The modeling unit uses multi-angle aerial photos taken by UAVs to calculate the latitude and longitude range by extracting the Exif information of the photos, and groups them according to the grid code. It completes oblique photogrammetry modeling by executing multiple parallel modeling tasks, trims the model according to the grid boundary line, and stores the model and its associated information.

[0044] The model update unit selects the grid of the area to be updated, generates flight paths based on the selected grid, uses multi-angle aerial photos taken by the UAV to model the model, and stores and updates the model and model-related information of the grid of the area to be updated.

[0045] The model rendering unit extracts the mesh information, elevation information, and piecewise path information contained in the model from the database, constructs the bottom-level nodes, merges adjacent meshes to form a bounding volume, converts the mesh boundary lines into node content, uses the model data as child nodes and configures the view distance precision geometricError, and optimizes the rendering load through layer merging and LOD simplification, and finally displays the model.

[0046] (III) Beneficial Effects

[0047] Compared with existing technologies, the 3D model management method and system based on UAV and mesh technology provided by this invention have the following advantages:

[0048] 1) Spatial structure division

[0049] This invention defines a spatial organization method for 3D models based on mesh technology. By utilizing a nested mesh hierarchy, it changes the traditional project-based structure of model production. This innovative spatial partitioning method enables 3D models to be stored in an orderly manner according to their spatial location, forming a storable, indexable, computational, and displayable model, laying the foundation for subsequent automated updates, maintenance, and efficient rendering and display.

[0050] 2) Automated updates and maintenance

[0051] Based on the above data organization method, the production, updating and maintenance process of the model is innovated. Through grid tiling technology, the automated processing process is realized. When the model needs to be updated, only specific grid areas need to be operated, without the need to process the entire model. This greatly improves the model update efficiency, effectively solves the problem of slow updates for large city-level models, meets the update and iteration needs of large city-level models, and realizes rapid local updates and maintenance of 3D models.

[0052] 3) High-efficiency rendering and display

[0053] This invention achieves efficient rendering by using top-level model reconstruction technology, combined with model LOD (Level of Detail) and simplified mesh boundary lines. Specifically, during display, only the top-level mesh is loaded and the actual model data is not displayed. Instead, the data range is indicated by the mesh display. This method significantly reduces the model rendering pressure and improves the model rendering speed, making it particularly suitable for displaying large city-level models. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0055] Figure 1 This is a schematic diagram of the modeling process in this invention;

[0056] Figure 2 This is a schematic diagram of the model update process in this invention;

[0057] Figure 3 This is a schematic diagram of the model rendering process in this invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] The following section, with specific examples, describes the detailed process and technical effects of the 3D model management method based on UAV and mesh technology provided by this invention.

[0060] Model building: The UAV takes aerial photos from multiple angles, extracts the Exif information of the photos to calculate the latitude and longitude range, and groups them according to the grid code. The oblique photogrammetry modeling is completed by executing multiple parallel modeling tasks. The model is clipped according to the grid boundary line, and the model and its associated information are stored.

[0061] Model update: Select the grid of the area to be updated, generate flight path based on the selected grid, take multi-angle aerial photos from the UAV, perform model modeling, and store and update the model and model-related information of the grid of the area to be updated;

[0062] Model rendering: Extract mesh information, elevation information and piecewise path information contained in the model from the database, construct the bottom-level nodes and merge adjacent meshes to form a bounding volume, while converting the mesh boundary lines into node content, using the model data as child nodes and configuring the view distance precision geometricError, simplifying and optimizing the rendering load through layer merging and LOD, and finally displaying the model.

[0063] I. Model Building

[0064] like Figure 1 As shown, model building includes the following steps:

[0065] S11. Data Acquisition: The UAV performs surveying and mapping tasks based on flight mission and route information. Based on the oblique photogrammetry modeling process, the UAV gimbal adjusts the camera to take photos from multiple angles.

[0066] S12. Photo Grouping: By extracting the Exif information of the photos, the latitude and longitude range of the photos is calculated. The grid code is calculated based on the latitude and longitude range and the grid level selected when organizing the model. All photos are grouped according to the grid code (photos at the boundary may belong to multiple groups).

[0067] S13. Parallel Modeling: Establish multiple parallel modeling tasks based on grouping information, and perform oblique photogrammetry modeling using modeling software;

[0068] S14. Model cropping: Due to the outward expansion of the model caused by photo grouping, the model edges are not aligned with the mesh edges. Model cropping technology is used to precisely crop the model edges so that the model matches the mesh.

[0069] S15. Data storage: Store the model and its associated information.

[0070] Specifically, data storage in S15 includes storing the model and its associated information, including:

[0071] The cropped model is converted into 3DTiles data format and stored using object storage software;

[0072] The model-related information, including elevation information and other related information, extracted during the execution of surveying and mapping tasks is indexed and stored in a database using the corresponding grid coding.

[0073] In the technical solution of this application, the grid level adopts a 17-level 512m grid, and the updating and maintenance of model data are based on this level of grid.

[0074] To simplify data indexing and accelerate computation, the grid coding adopts a two-dimensional coding system based on the geospatial grid coding rules for planar division. Spatial information is expressed by combining two-dimensional geospatial grid coding with bottom and top elevation information.

[0075] In the technical solution of this application, the model is stored and rendered using the 3DTiles data format. 3DTiles is a data format developed for streaming and rendering of 3D geospatial data, including photogrammetry, 3D architecture, BIM / CAD, instantiated features, and point clouds. It is based on a hierarchical data structure and tile format set that can be transmitted and rendered.

[0076] Model-related information and grid index information are stored using the relational database MySQL and the non-relational database Elasticsearch.

[0077] Since digital twins and smart cities are mostly developed based on B / S architecture, 3DTiles' network data flow transmission method and tree-structured data organization method perfectly match the grid coding method adopted in this invention.

[0078] II. Model Update

[0079] like Figure 2 As shown, model updates include the following steps:

[0080] S21. Region Selection: When the model needs to be updated locally, select the grid of the region to be updated (if it spans multiple grids, select multiple adjacent grids).

[0081] S22. Route Generation: Generates corresponding route information based on the selected grid.

[0082] S23. Remodeling: The UAV performs surveying and mapping tasks based on the flight path information, takes aerial photos from multiple angles, and remodels the model according to the model modeling process.

[0083] S24. Storage Update: Update the grid code of the regional grid as needed, and perform storage update on the corresponding model and model-related information.

[0084] Specifically, in S22, route generation involves generating corresponding route information based on the selected grid, including:

[0085] The oblique photography model requires the drone camera to shoot at multiple angles. The flight path is adaptively expanded according to the camera parameters, aerial shooting altitude and gimbal tilt angle to ensure that the photos in each grid are captured completely.

[0086] III. Model Rendering

[0087] like Figure 3 As shown, model rendering includes the following steps:

[0088] S31. Data Extraction: Extract grid information, elevation information, and 3DTiles path information contained in the model from the database to prepare basic data for subsequent model rendering.

[0089] S32. Mesh Merging: Calculate the bottom-level node information of 3DTiles, merge adjacent meshes, and combine the mesh range, bottom height information and top height information to form a bounding volume to determine the spatial range of the model.

[0090] S33. Node Content Generation: Based on the mesh information contained in the node, the software draws the boundary line of each mesh to form GLB data as the node content, thus clarifying the mesh represented by the node.

[0091] S34. Model Data Import and Setting: Locate the model data based on the grid information contained in the node, import the model data as a child node of the corresponding node, and set the view distance precision geometricError to ensure that the model is displayed within a reasonable view distance.

[0092] S35. Top-level reconstruction and optimization: Merge the mesh information contained in the bottom-level nodes according to the mesh hierarchy to reconstruct the top-level of the model. According to the nesting relationship of the mesh hierarchy, draw boundary lines for the mesh to show whether model data exists in a certain space. Use LOD and simplify the display of mesh boundary lines to reduce the rendering pressure of the model.

[0093] S36. Model Display.

[0094] Specifically, the model shown in S36 includes:

[0095] The browser uses rendering tools, including Cesium, to load the root node tileset.json data and finally display the model.

[0096] Based on the above-disclosed method for managing 3D models using UAV and mesh technology, this invention also discloses a 3D model management system based on UAV and mesh technology, including a model modeling unit, a model updating unit, and a model rendering unit.

[0097] The modeling unit uses multi-angle aerial photos taken by UAVs to calculate the latitude and longitude range by extracting the Exif information of the photos, and groups them according to the grid code. It completes oblique photogrammetry modeling by executing multiple parallel modeling tasks, trims the model according to the grid boundary line, and stores the model and its associated information.

[0098] The model update unit selects the grid of the area to be updated, generates flight paths based on the selected grid, uses multi-angle aerial photos taken by the UAV to model the model, and stores and updates the model and model-related information of the grid of the area to be updated.

[0099] The model rendering unit extracts the mesh information, elevation information, and piecewise path information contained in the model from the database, constructs the bottom-level nodes, merges adjacent meshes to form a bounding volume, converts the mesh boundary lines into node content, uses the model data as child nodes and configures the view distance precision geometricError, and optimizes the rendering load through layer merging and LOD simplification, and finally displays the model.

[0100] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D model management method based on UAV and mesh technology, characterized by: include: Model building: The UAV takes aerial photos from multiple angles, extracts the Exif information of the photos to calculate the latitude and longitude range, and groups them according to the grid code. The oblique photogrammetry modeling is completed by executing multiple parallel modeling tasks. The model is clipped according to the grid boundary line, and the model and its associated information are stored. Model update: Select the grid of the area to be updated, generate flight path based on the selected grid, take multi-angle aerial photos from the UAV, perform model modeling, and store and update the model and model-related information of the grid of the area to be updated; Model rendering: Extract mesh information, elevation information and piecewise path information contained in the model from the database, construct the bottom-level nodes and merge adjacent meshes to form a bounding volume, while converting the mesh boundary lines into node content, using the model data as child nodes and configuring the view distance precision geometricError, simplifying and optimizing the rendering load through layer merging and LOD, and finally displaying the model.

2. The 3D model management method based on UAV and mesh technology according to claim 1, characterized in that: The model building process includes the following steps: S11. Data Acquisition: The UAV performs surveying and mapping tasks based on flight mission and route information. Based on the oblique photogrammetry modeling process, the UAV gimbal adjusts the camera to take photos from multiple angles. S12. Photo Grouping: By extracting the Exif information of the photos, the latitude and longitude range of the photos is calculated. Based on the latitude and longitude range and the grid level selected when organizing the model, the grid code is calculated, and all photos are grouped according to the grid code. S13. Parallel Modeling: Establish multiple parallel modeling tasks based on grouping information, and perform oblique photogrammetry modeling using modeling software; S14. Model cropping: Due to the outward expansion of the model caused by photo grouping, the model edges are not aligned with the mesh edges. Model cropping technology is used to precisely crop the model edges so that the model matches the mesh. S15. Data storage: Store the model and its associated information.

3. The 3D model management method based on UAV and mesh technology according to claim 2, characterized in that: Data storage in S15: This includes storing the model and its associated information, including: The cropped model is converted into 3DTiles data format and stored using object storage software; The model-related information, including elevation information and other related information, extracted during the execution of surveying and mapping tasks is indexed and stored in a database using the corresponding grid coding.

4. The 3D model management method based on UAV and mesh technology according to claim 2, characterized in that: The grid hierarchy adopts a 17-level 512m grid, and the updating and maintenance of model data are based on this level of grid. To simplify data indexing and accelerate computation, the grid coding adopts a two-dimensional coding system based on the Earth spatial grid coding rules for planar division. Spatial information is expressed by combining two-dimensional Earth spatial grid coding with bottom and top elevation information.

5. The 3D model management method based on UAV and mesh technology according to claim 3, characterized in that: The model is stored and rendered using the 3DTiles data format. 3DTiles is a data format developed for streaming and rendering of 3D geospatial data, including photogrammetry, 3D architecture, BIM / CAD, instantiated features, and point clouds. It is based on a hierarchical data structure and tile format set that can be transmitted and rendered. The model's associated information and grid index information are stored using the relational database MySQL and the non-relational database Elasticsearch.

6. The 3D model management method based on UAV and mesh technology according to claim 1, characterized in that: The model update includes the following steps: S21. Region Selection: When the model needs to be locally updated, select the grid of the region to be updated. S22. Route Generation: Generates corresponding route information based on the selected grid. S23. Remodeling: The UAV performs surveying and mapping tasks based on the flight path information, takes aerial photos from multiple angles, and remodels the model according to the model modeling process. S24. Storage Update: Update the grid code of the regional grid as needed, and perform storage update on the corresponding model and model-related information.

7. The 3D model management method based on UAV and mesh technology according to claim 6, characterized in that: S22 Route Generation: Generates corresponding route information based on the selected grid, including: The oblique photography model requires the drone camera to shoot at multiple angles. The flight path is adaptively expanded according to the camera parameters, aerial shooting altitude and gimbal tilt angle to ensure that the photos in each grid are captured completely.

8. The 3D model management method based on UAV and mesh technology according to claim 1, characterized in that: The model rendering includes the following steps: S31. Data Extraction: Extract grid information, elevation information, and 3DTiles path information contained in the model from the database to prepare basic data for subsequent model rendering. S32. Mesh Merging: Calculate the bottom-level node information of 3DTiles, merge adjacent meshes, and combine the mesh range, bottom height information and top height information to form a bounding volume to determine the spatial range of the model. S33. Node Content Generation: Based on the mesh information contained in the node, the software draws the boundary line of each mesh to form GLB data as the node content, thus clarifying the mesh represented by the node. S34. Model Data Import and Setting: Locate the model data based on the grid information contained in the node, import the model data as a child node of the corresponding node, and set the view distance precision geometricError to ensure that the model is displayed within a reasonable view distance. S35. Top-level reconstruction and optimization: Merge the mesh information contained in the bottom-level nodes according to the mesh hierarchy, and perform top-level reconstruction of the model. According to the nesting relationship of the mesh hierarchy, draw boundary lines for the mesh to show whether model data exists in a certain space. Use LOD and simplify the display of mesh boundary lines to reduce the rendering pressure of the model. S36. Model Display.

9. The 3D model management method based on UAV and mesh technology according to claim 8, characterized in that: The model shown in S36 includes: The browser uses rendering tools, including Cesium, to load the root node tileset.json data and finally display the model.

10. A 3D model management system based on UAV and mesh technology, used to execute the 3D model management method based on UAV and mesh technology as described in claim 1, characterized in that: It includes a modeling unit, a model updating unit, and a model rendering unit; The modeling unit uses multi-angle aerial photos taken by UAVs to calculate the latitude and longitude range by extracting the Exif information of the photos, and groups them according to the grid code. It completes oblique photogrammetry modeling by executing multiple parallel modeling tasks, trims the model according to the grid boundary line, and stores the model and its associated information. The model update unit selects the grid of the area to be updated, generates flight paths based on the selected grid, uses multi-angle aerial photos taken by the UAV to model the model, and stores and updates the model and model-related information of the grid of the area to be updated. The model rendering unit extracts the mesh information, elevation information, and piecewise path information contained in the model from the database, constructs the bottom-level nodes, merges adjacent meshes to form a bounding volume, converts the mesh boundary lines into node content, uses the model data as child nodes and configures the view distance precision geometricError, and optimizes the rendering load through layer merging and LOD simplification, and finally displays the model.

Images