Three-dimensional modeling and mapping method, system and device based on fixed-wing unmanned aerial vehicle

Abstract

The invention provides a three-dimensional modeling and mapping method, system and device based on a fixed-wing unmanned aerial vehicle, and relates to the technical field of geographic mapping. According to the method, through synchronous data acquisition of an inclined camera, a laser scanner and a position and attitude measurement system, and in combination with navigation trajectory calculation, point cloud coordinate transformation, image attitude calculation and multi-source data fusion processing, space collaboration of image data and point cloud data is realized under a unified coordinate system. And performing aerial triangulation based on the fused data, further carrying out spatial modeling and texture mapping, and effectively improving geometric consistency and modeling continuity among multi-source data. According to the method, ground control points do not need to be arranged, high-precision and high-consistency geographic space information expression can be achieved, and real scene modeling and multi-type surveying and mapping data result generation in a large-scale mapping scene are supported.

Description

Technical Field

[0001] The present disclosure relates to the technical field of geodetic surveying and mapping, and more particularly, to a three-dimensional modeling and mapping method, system and device based on a fixed-wing unmanned aerial vehicle. Background Art

[0002] Currently, using unmanned aerial vehicles to obtain geospatial information has become an important technical means in the fields of surveying and mapping, planning, emergency response, etc. Among them, fixed-wing unmanned aerial vehicles, with their advantages of long flight range, long endurance, high operation efficiency, etc., are gradually applied to large-scale, high-resolution three-dimensional modeling and mapping scenarios. To improve the modeling accuracy and efficiency, an image sensor and a lidar device are usually combined and carried on an unmanned aerial vehicle platform, and multi-source heterogeneous data is obtained to enhance the modeling ability.

[0003] In related technologies, the collaborative acquisition and processing of multi-source data often rely on setting a sufficient number of ground control points to constrain the attitude solution of images and the spatial registration of point clouds, so as to improve the positioning accuracy of the overall model. However, laying out ground control points in complex terrain, inconvenient transportation or restricted areas not only takes time and effort, but also has practical problems such as limited operation and high safety risks. At the same time, due to the differences in time stamps, coordinate systems, sampling densities, etc. of multi-source data, problems such as data inconsistency, large registration errors, and discontinuous modeling are likely to occur in the fusion processing process, limiting its applicability in high-precision, large-scale rapid modeling tasks. In addition, although some traditional solutions can achieve independent modeling of images or point clouds by means of a single sensor such as an oblique camera or a laser scanner, there are still limitations in the restoration of spatial geometric structures and the expression of ground object textures, and it is difficult to simultaneously take into account accuracy, details and wide-area coverage, affecting the integrity and accuracy of the mapping results.

[0004] Therefore, there is an urgent need for a new three-dimensional modeling and mapping solution to meet the actual needs of obtaining refined geographic information.

[0005] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] The purpose of the embodiments of the present disclosure is to provide a three-dimensional modeling and mapping method, system and device based on a fixed-wing unmanned aerial vehicle, which can effectively improve the spatial coordination consistency of multi-source data and enhance the modeling expression ability without relying on ground control points, so as to meet the actual needs of obtaining refined geographic information.

[0007] According to the first aspect of the embodiments of the present disclosure, a three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle is provided, including: Based on the internal orientation parameters of the oblique camera and the performance parameters of the airborne radar module, and in combination with the digital elevation model data and the survey area boundary range of the target area, calculate the route planning parameters; wherein, the airborne radar module includes a position and attitude measurement system and a laser scanner; Control the fixed-wing UAV to fly according to the route planning parameters. During the flight, collect the original navigation data of the fixed-wing UAV through the position and attitude measurement system, and at the same time collect image data through the oblique camera and collect original point cloud data through the laser scanner; Based on the original navigation data, the GNSS static data collected by the ground reference station, and the reference station coordinate data, perform navigation trajectory solution processing to obtain navigation trajectory data; Based on the navigation trajectory data, perform coordinate transformation on the original point cloud data to obtain registered point cloud data; calculate the exterior orientation elements of the exposure points based on the image exposure point information collected by the position and attitude measurement system, and combine the registered point cloud data and the image data to obtain multi-source fusion data; Based on the multi-source fusion data, perform aerial triangulation to obtain fused geometric data; Based on the fused geometric data, generate a three-dimensional geographical model through spatial modeling and texture mapping, and generate the target surveying and mapping results based on the three-dimensional geographical model.

[0008] In an exemplary embodiment of the present disclosure, before calculating the route planning parameters, the method further includes: Obtain the spatial offset of the laser scanner, the oblique camera, and the GNSS antenna relative to the inertial navigation system, and obtain the attitude installation angle error of the laser scanner relative to the inertial navigation system; Wherein, the spatial offset is determined by laboratory calibration, and the attitude installation angle error is determined by the difference iteration solution method based on the overlapping flight strip characteristics during flight.

[0009] In an exemplary embodiment of the present disclosure, the performing navigation trajectory solution processing based on the original navigation data, the GNSS static data collected by the ground reference station, and the reference station coordinate data to obtain navigation trajectory data includes: Synchronize the time of the original navigation data and the GNSS static data collected by the ground reference station; According to the spatial offset of the GNSS antenna relative to the inertial navigation system, correct the time-synchronized original navigation data to obtain intermediate navigation data; Based on the intermediate navigation data and the reference station coordinate data, calculate the three-dimensional position and attitude of the fixed-wing UAV at each moment to obtain the navigation trajectory data.

[0010] In an exemplary embodiment of the present disclosure, based on the navigation trajectory data, coordinate transformation is performed on the original point cloud data to obtain registered point cloud data, including: According to the spatial offset of the laser scanner relative to the inertial navigation system, the attitude installation angle error between the laser scanner and the inertial navigation system, and the navigation trajectory data, coordinate transformation is performed on the original point cloud data to obtain registered point cloud data.

[0011] In an exemplary embodiment of the present disclosure, based on the image exposure point information collected by the position and attitude measurement system, the exterior orientation elements of the exposure points are calculated, and multi-source fusion data is obtained by combining the registered point cloud data and the image data, including: According to the image exposure point information collected by the position and attitude measurement system, the position data of the fixed-wing UAV at the corresponding moment is extracted; According to the spatial offset of the tilt camera relative to the inertial navigation system, the position data is corrected to obtain the exterior orientation elements of the exposure points; The image data is subjected to spatial projection transformation by using the exterior orientation elements of the exposure points to obtain a georeferenced image; The registered point cloud data is converted to a coordinate system consistent with the georeferenced image and spatially aligned with the georeferenced image to obtain the multi-source fusion data.

[0012] In an exemplary embodiment of the present disclosure, based on the multi-source fusion data, aerial triangulation is performed to obtain fusion geometric data, including: Based on the exterior orientation elements of the exposure points and the interior orientation parameters of the tilt camera, initial spatial triangulation is performed on the georeferenced image by using a bundle adjustment model to obtain target exterior orientation elements; Combining the target exterior orientation elements, the registered point cloud data and the georeferenced image, a multi-source joint adjustment model is constructed, and by fusing the image reprojection residuals and the point cloud geometric residuals, weighted geometric solution is performed to obtain the joint spatial data aligned between the image and the point cloud; Based on the joint spatial data, local iterative registration of the registered point cloud data based on normal distribution transformation is performed to obtain the fusion geometric data.

[0013] In an exemplary embodiment of the present disclosure, the method further includes: Performing denoising processing and density equalization processing on the registered point cloud data, and performing radiometric correction and image enhancement processing on the georeferenced image, so as to obtain the multi-source fusion data according to the processed registered point cloud data and georeferenced image.

[0014] In an exemplary embodiment of the present disclosure, generating a three-dimensional geographic model based on the fused geometric data through spatial modeling and texture mapping, and generating a target surveying and mapping result based on the three-dimensional geographic model, includes: Constructing an irregular triangular network model based on the fused geometric data by using a triangulation algorithm; Mapping the texture information of the image data to the irregular triangular network model to generate a three-dimensional geographic model; Generating the target surveying and mapping result based on the spatial image data corresponding to the three-dimensional geographic model in combination with terrain expression data.

[0015] According to a second aspect of the embodiments of the present disclosure, there is provided a three-dimensional modeling and mapping system based on a fixed-wing unmanned aerial vehicle, including: A fixed-wing unmanned aerial vehicle; A power module, installed on the fixed-wing unmanned aerial vehicle, for supplying power to each airborne device; An airborne radar module, provided on the fixed-wing unmanned aerial vehicle, the airborne radar module includes a position and attitude measurement system and a laser scanner, the position and attitude measurement system is used to collect the original navigation data of the fixed-wing unmanned aerial vehicle during flight, and the laser scanner is used to collect original point cloud data during flight; An oblique camera, provided on the fixed-wing unmanned aerial vehicle, electrically connected to the airborne radar module, for collecting image data during flight; A communication module, provided on the fixed-wing unmanned aerial vehicle; A flight control system, electrically connected to the oblique camera and the airborne radar module; A ground base station system, including a ground reference station GNSS receiver, a first tripod and ground target control points, the ground reference station GNSS receiver is connected to the first tripod and erected on the ground target control points, for providing static reference positioning data; A ground control system, electrically connected to the communication module, the ground control system includes a dynamic RTK reference station, a radio antenna and a second tripod, the dynamic RTK reference station is electrically connected to the radio antenna, and the dynamic RTK reference station and the radio antenna are connected to the second tripod, for sending differential signals to the fixed-wing unmanned aerial vehicle in real time.

[0016] According to a third aspect of the embodiments of the present disclosure, there is provided a three-dimensional modeling and mapping device based on a fixed-wing unmanned aerial vehicle, including: A planning parameter determination module, configured to calculate route planning parameters based on the internal orientation parameters of the oblique camera and the performance parameters of the airborne radar module, in combination with the digital elevation model data and the survey area boundary range of the target area; wherein, the airborne radar module includes a position and attitude measurement system and a laser scanner; A multi-source data acquisition module, configured to control the flight of a fixed-wing UAV according to the route planning parameters, collect the original navigation data of the fixed-wing UAV through the position and attitude measurement system during the flight, and simultaneously collect image data through the tilt camera and collect original point cloud data through the laser scanner; A navigation trajectory calculation module, configured to perform navigation trajectory calculation processing based on the original navigation data, GNSS static data collected by a ground reference station, and reference station coordinate data to obtain navigation trajectory data; A multi-source data fusion module, configured to perform coordinate transformation on the original point cloud data to obtain registered point cloud data; calculate the exterior orientation elements of the exposure points based on the navigation trajectory data and image data, and combine the registered point cloud data and image data to obtain multi-source fusion data; An aerial triangulation module, configured to perform aerial triangulation based on the multi-source fusion data to obtain fused geometric data; A modeling and mapping module, configured to generate a three-dimensional geographic model through spatial modeling and texture mapping based on the fused geometric data, and generate a target mapping result based on the three-dimensional geographic model.

[0017] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: In the exemplary embodiments of the present disclosure, the three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle combines the internal orientation parameters of an oblique camera with the performance parameters of an airborne radar module, and introduces a digital elevation model and survey area boundary data for comprehensive calculation, enabling the acquisition of route planning parameters that better fit the terrain undulation and target area coverage requirements before task execution, ensuring the coverage integrity and spatial uniformity of subsequent data acquisition. During flight, navigation data is synchronously collected through a position and attitude measurement system, and is kept spatio-temporally coordinated with the image data and point cloud data collected by the oblique camera and the laser scanner, providing a prerequisite guarantee for the fusion of multi-source data in a unified reference system. The navigation trajectory solution depends on the joint processing of the original navigation data, GNSS static data collected by a ground reference station, and the reference station coordinate data, enabling the sequential position and attitude information of the flight platform to have high positioning reliability even without ground control points. On this basis, coordinate transformation and registration are performed on the point cloud data, which can significantly alleviate the spatial offset problem caused by heterogeneous data sources and improve the consistency of subsequent geometric processing. The introduction of image exposure point information enables the image data to be geometrically located in combination with the flight trajectory and the camera spatial relationship, thereby establishing an accurate spatial registration basis between the images and the point cloud. Performing aerial triangulation on the fused data, and then extracting the internal geometric relationships between the multi-source data, enables the original acquisition results to construct a spatially consistent fusion model in a unified framework, avoiding the problems of attitude incoordination and registration error accumulation caused by the separate processing of multi-source sensors in related technologies. Finally, through the modeling and texture mapping processes, the fused geometric data is transformed into a geospatial representation with continuous structure and consistent imagery, enabling the model to not only be measurable, but also support the expression and analysis of various terrain features, thereby meeting the multiple requirements for the accuracy and integrity of spatial information in large-scale scenarios.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0020] Figure 1 FIG. shows the system architecture diagram of a three-dimensional modeling and mapping system based on a fixed-wing unmanned aerial vehicle in an embodiment of the present disclosure.

[0021] Figure 2Shows a schematic structural diagram of a ground base station system in an embodiment of the present disclosure.

[0022] Figure 3 Shows a schematic structural diagram of a ground control system in an embodiment of the present disclosure.

[0023] Figure 4 Shows a schematic flowchart of a three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle in an embodiment of the present disclosure.

[0024] Figure 5 Shows a block diagram of a three-dimensional modeling and mapping device based on a fixed-wing unmanned aerial vehicle in an embodiment of the present disclosure.

[0025] Figure 6 Shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure.

[0026] In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0027] The main structural markings in the figure are explained as follows: 1. Fixed-wing unmanned aerial vehicle; 2. Power supply module; 3. Position and attitude measurement system; 4. Oblique camera; 5. Laser scanner; 6. Communication module; 7. Flight control system; 8. GNSS antenna; 9. Ground reference station GNSS receiver; 10. First tripod; 11. Ground target control point; 12. Dynamic RTK reference station; 13. Radio antenna; 14. Second tripod. Detailed implementation manners

[0028] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0029] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0030] Figure 1The system architecture diagram of a three-dimensional modeling and mapping system based on a fixed-wing unmanned aerial vehicle according to an embodiment of the present disclosure is shown.

[0031] As Figure 1 shown, the system includes a fixed-wing unmanned aerial vehicle 1, a power module 2, an airborne radar module, an oblique camera 4, a communication module 6, a flight control system 7, a ground base station system, and a ground control system.

[0032] Among them, the fixed-wing unmanned aerial vehicle 1 is used to carry various airborne devices and execute flight missions, and its structure is suitable for long-range flights, stable flights, and multi-source data collaborative acquisition. The power module 2 is installed on the fixed-wing unmanned aerial vehicle 1 and is used to provide continuous power support for airborne devices such as the carried airborne radar module, oblique camera 4, communication module 6, and flight control system 7. The oblique camera 4 is installed on the fixed-wing unmanned aerial vehicle 1 and is electrically connected to the airborne radar module. The oblique camera 4 can be a multi-lens oblique camera and is used to collect ground image data from multiple oblique angles during flight to enhance the modeling texture performance and the ability to reconstruct the facade structure.

[0033] The airborne radar module is connected to a GNSS (Global Navigation Satellite System) antenna 8 and is integrated by a Position and Orientation System (POS system) 3 and a laser scanner 5. The Position and Orientation System 3 includes a GNSS receiver, an inertial navigation system, a synchronization control module, and a data storage device, and can collect the original navigation data of the fixed-wing unmanned aerial vehicle 1 in real time during flight, including position coordinates, attitude angles, speed information, etc. The laser scanner 5 is used to emit laser pulses and receive echoes during flight to obtain the original point cloud data of the area covered by the flight path.

[0034] The communication module 6 is also installed on the fixed-wing unmanned aerial vehicle 1 and is used to transmit key data such as the device operation status to the ground control system in real time during flight to support remote monitoring and scheduling of the flight state. The communication module can also be used to store the original image data and the original point cloud data in the airborne system after the flight mission is completed, which is convenient for unified downloading and processing after the flight.

[0035] The flight control system 7 is arranged on the fixed-wing unmanned aerial vehicle 1 and is used to coordinate the operation status of each device and control the flight path. The flight control system 7 is electrically connected to the oblique camera 4 and the airborne radar module respectively to ensure that each device completes data synchronization acquisition according to the mission plan during flight.

[0036] Refer to Figure 2As shown in the figure, a schematic structural diagram of a ground base station system is presented. Among them, the ground base station system includes a ground reference station GNSS receiver 9, a first tripod 10, and a ground target control point 11. The ground reference station GNSS receiver 9 is connected to the first tripod 10 through a cable and is installed as a whole at the position of the preset ground target control point 11. The ground base station system is used to provide static reference positioning data, such as collecting GNSS static data in the flight area, providing reference positioning accuracy for subsequent navigation trajectory calculation, and enhancing the spatial consistency of the fused data. It can be understood that the ground base station system in the system shown in the exemplary embodiment of the present disclosure can be replaced by a CORS (Continuously Operating Reference Station) station to achieve reference data collection by accessing the existing CORS network within the coverage area, thereby reducing on-site layout work and improving the flexibility and automation level of system deployment.

[0037] Reference Figure 3 As shown in the figure, a schematic structural diagram of a ground control system is presented. Among them, the ground control system is electrically connected to the communication module 6. The ground control system includes a dynamic RTK (Real-Time Kinematic) reference station 12, a radio antenna 13, and a second tripod 14. There is an electrical signal connection between the dynamic RTK reference station 12 and the radio antenna 13. The dynamic RTK reference station 12 and the radio antenna 13 are installed on the second tripod 14 and are deployed on a visible high ground or open area outside the flight area, and are used to send differential signals to the fixed-wing UAV 1 in flight in real time to improve the position and attitude calculation accuracy of the navigation system during flight.

[0038] Through the collaborative configuration and functional cooperation of each component module, the system is enabled to possess comprehensive capabilities such as autonomous flight control, multi-source data synchronous acquisition, differential positioning enhancement, and reference data collection, providing a high-precision and multi-dimensional data basis for subsequent three-dimensional modeling and mapping processing.

[0039] Before flight, deploy the ground base station system at the ground target control point 11 and power it on. Start the dynamic RTK reference station 12, and transmit differential signals through the radio to provide a positioning reference for the fixed-wing UAV 1. During flight, the power module 2 supplies power to the on-board equipment. The flight control system 7 controls the attitude and speed of the fixed-wing UAV 1 according to the preset flight path, and at the same time controls the position and attitude measurement system 3 to record the position (latitude, longitude, altitude), attitude angles (pitch, roll, yaw) and timestamps of the fixed-wing UAV 1 in real time, and synchronously triggers the exposure of the tilt camera 4 and the pulse emission of the laser scanner 5 to ensure the spatio-temporal consistency of the image data and the point cloud data, and record the photo exposure point information. The communication module 6 transmits the navigation data and equipment status back to the ground control system to ensure real-time monitoring and emergency adjustment. After flight, the ground base station system statically observes and records GNSS data, and jointly processes the navigation data recorded by the position and attitude measurement system 3 to perform navigation trajectory solution, correct the spatial coordinates of the point cloud and the image, and then through multi-source data registration, joint adjustment and three-dimensional reconstruction, finally generate a high-precision real scene model and target mapping results. This system combines spatio-temporal synchronization, multi-sensor data fusion and ground-air collaborative positioning, taking into account both large-scale efficient operation and centimeter-level detail restoration capabilities.

[0040] An embodiment of the present disclosure provides a three-dimensional modeling and mapping method based on a fixed-wing UAV, referring to Figure 4 As shown, this method may include steps S410 to S460: Step S410, based on the internal orientation parameters of the tilt camera and the performance parameters of the on-board radar module, combined with the digital elevation model data and the survey area boundary range of the target area, calculate the flight path planning parameters; wherein, the on-board radar module includes a position and attitude measurement system and a laser scanner; Step S420, control the fixed-wing UAV to fly according to the flight path planning parameters. During flight, collect the original navigation data of the fixed-wing UAV through the position and attitude measurement system, and at the same time collect image data through the tilt camera and collect original point cloud data through the laser scanner; Step S430, based on the original navigation data, the GNSS static data collected by the ground reference station, and the reference station coordinate data, perform navigation trajectory solution processing to obtain navigation trajectory data; Step S440, based on the navigation trajectory data, perform coordinate transformation on the original point cloud data to obtain registered point cloud data; calculate the exterior orientation elements of the exposure points based on the image exposure point information collected by the position and attitude measurement system, and combine the registered point cloud data and the image data to obtain multi-source fusion data; Step S450, based on the multi-source fusion data, perform aerial triangulation to obtain fusion geometric data; Step S460: Generate a three-dimensional geographic model through spatial modeling and texture mapping based on the fused geometric data, and generate a target surveying and mapping result based on the three-dimensional geographic model.

[0041] Execute the three-dimensional modeling and mapping method based on a fixed-wing UAV provided by the present disclosure. During the data acquisition stage, through the combined operation of a position and attitude measurement system, an oblique camera, and a laser scanner, realize the spatio-temporal collaborative acquisition of navigation data, image data, and point cloud data during flight. During the data processing stage, combine navigation trajectory calculation, image exposure point positioning, and point cloud registration to achieve the spatial fusion of multi-source data in a unified coordinate system, and perform aerial triangulation operations based on the fusion result to construct a geometric calculation model with spatial consistency. The finally modeled three-dimensional geographic model has clear position reference and spatial structure information. The point cloud data formed by this model is dense and continuous, can supplement geometric information expression in scenarios lacking high-texture areas, and combined with the texture details carried by the image data, can meet the modeling requirements for expressing various types of spatial elements. The obtained target surveying and mapping result has high spatial resolution and coordinate accuracy, can support the extraction and map expression of topographic information at a large scale level, and is suitable for surveying and mapping application scenarios with coordinated requirements for accuracy, efficiency, and coverage.

[0042] Next, the three-dimensional modeling and mapping method based on a fixed-wing UAV in this exemplary embodiment will be described in detail.

[0043] In step S410, based on the internal orientation parameters of the oblique camera and the performance parameters of the airborne radar module, combine the digital elevation model data of the target area and the survey area boundary range to calculate the route planning parameters; wherein, the airborne radar module includes a position and attitude measurement system and a laser scanner.

[0044] In the exemplary embodiment of the present disclosure, the route planning parameters include, but are not limited to, route spacing, relative flight height, photographic baseline length, ground image resolution, forward overlap, side overlap, line scan speed and point density of the laser scanner, etc., which are used to guide the flight path design of the fixed-wing UAV.

[0045] The internal orientation parameters of the oblique camera include focal length, imaging width, pixel size, and resolution information, etc., to calculate the ground field of view range and ground image resolution at the target flight height. The performance parameters of the airborne radar module include laser scanning frequency, scanning angle, scanning speed, point spacing, etc. Combining the flight speed and laser trigger interval, the point cloud density achievable at different flight altitudes can be estimated.

[0046] The boundary range of the survey area can be defined by files in formats such as KML (Keyhole Markup Language) and Shapefile (vector graphics file), and can be projected to the same geospatial reference framework as the digital elevation model data through a coordinate system. The digital elevation model is used to provide terrain undulation information within the survey area, enabling flight path planning to not only consider planar coverage but also dynamically adjust the flight altitude according to the surface height difference, ensuring the imaging overlap degree and the uniformity of the point cloud density in the aerial photography area.

[0047] Exemplarily, load the digital elevation model data of the target area for analyzing the terrain undulation range, and determine the relative flight height by setting a flight altitude threshold on the premise of meeting flight safety and the working height of the sensor.

[0048] For example, calculate the relative flight height H , there is: (1) Among them, f is the camera focal length, a is the pixel size, GSD is the target ground image resolution.

[0049] Further combine the boundary range of the survey area, such as KML vector data containing the outer boundary contour, to calculate the flight line spacing and the photographic baseline length required for the forward overlap degree and the side overlap degree, ensure that the image data has the specified spatial overlap coverage ratio, and at the same time calculate whether the laser scanning bandwidth covers the lateral blank area.

[0050] On the premise of meeting coverage continuity and data accuracy, output flight line planning parameters such as the start and end points, spacing, heading angle, flight altitude parameters, image trigger interval, and laser scanning synchronization strategy of the flight line to generate a mission planning file that can be used by the flight control system to control a fixed-wing UAV to execute an automated flight line mission.

[0051] Jointly model the performance characteristics of the imaging sensor and the lidar system, and perform parameter resolution in combination with terrain information and the survey area boundary, so that the flight mission has adaptability and pertinence to the multi-source data acquisition conditions, laying a foundation for the high-quality acquisition of subsequent image and point cloud data, and avoiding problems such as modeling breaks or accuracy degradation caused by terrain occlusion, insufficient resolution, or missing coverage overlap. This planning method takes into account the three elements of spatial geometry, geographical coverage, and sensor capabilities, and has high scene adaptability and mission efficiency.

[0052] It should be noted that in the exemplary embodiments of the present disclosure, before calculating the route planning parameters, the installation parameter calibration of the lidar scanner, the tilt camera, and the GNSS antenna relative to the inertial navigation system can be completed first. This calibration process includes obtaining the spatial offset of the lidar scanner, the tilt camera, and the GNSS antenna relative to the inertial navigation system, as well as obtaining the attitude installation angle error of the lidar scanner relative to the inertial navigation system, for use in subsequent trajectory solution and geometric correction processing in multi-source data registration. Among them, the spatial offset is determined by laboratory calibration, and the attitude installation angle error is determined by differential iterative solution based on the characteristics of overlapping flight strips during flight.

[0053] The spatial offset refers to the three-dimensional vector distance of the installation centers of the lidar scanner, the tilt camera, and the GNSS antenna relative to the coordinate origin of the inertial navigation system in the flight platform coordinate system, and may include position offsets in the flight direction (X-axis), lateral (Y-axis), and vertical (Z-axis). For example, it can be recorded as: , , , where correspond to the offsets of the lidar scanner, the tilt camera, and the GNSS antenna to the coordinate origin of the inertial navigation system in the flight direction respectively, correspond to the offsets of the lidar scanner, the tilt camera, and the GNSS antenna to the coordinate origin of the inertial navigation system perpendicular to the flight direction respectively, correspond to the offsets of the lidar scanner, the tilt camera, and the GNSS antenna to the coordinate origin of the inertial navigation system in the vertical direction respectively. The spatial offset can be accurately measured in the laboratory by mechanical measurement, laser ranging, or a three-dimensional coordinate instrument.

[0054] The attitude installation angle error refers to the angular error of the coordinate system of the lidar scanner relative to the coordinate system of the inertial navigation system in three rotation directions, corresponding to the roll angle, pitch angle, and heading angle respectively. For example, it can be recorded as , which are the attitude installation angle errors of the lidar scanner relative to the inertial navigation system in the roll, pitch, and heading directions respectively. Due to the existence of small angular deviations in the actual installation process, this error cannot be ignored and can be dynamically estimated by field calibration.

[0055] Exemplarily, first, select a suitable calibration site, such as arranging a calibration field in an area with flat and exposed terrain, and require that the area contains buildings or obvious protruding features for assisting feature recognition to enhance the spatial saliency of feature points. The targets in the calibration area should have good laser reflectivity, such as setting target points with clear ground features such as road edges and corners as the reference objects for subsequent error calculation.

[0056] Secondly, evenly distribute ground target image control points or use precisely measured typical feature points of ground objects within the selected area, and the field measurement accuracy should meet the accuracy requirements of actual applications. Subsequently, set up a ground reference station GNSS receiver at the ground control point with known coordinates, and start the ground reference station GNSS receiver to record data. The sampling time interval is set to be greater than 1 second to obtain high-precision static reference data for track calculation.

[0057] In the data acquisition stage, flights can be carried out according to a 3×3 grid-like flight path design. Set the heading overlap to be not less than 70%, and the flight path spacing is 1 / 5 of the relative flight height. At the same time, set the scanning frequency of the airborne radar system to be greater than 400 kHz to ensure that the coverage density and accuracy of the point cloud data meet the requirements for attitude error inversion. During the flight process, synchronously obtain point cloud data, navigation data, and base station data.

[0058] After the data acquisition is completed, calculate the attitude installation angle error. This calculation process is based on the elevation difference of the same feature points in the overlapping flight strips, and analyzes the spatial offset of the point cloud caused by the installation angle error. Identify and measure the height difference of the same-name feature points in the overlapping area by visual and manual methods, and initially calculate the elevation difference residual. On this basis, adopt an iterative calculation strategy, and sequentially perform installation angle perturbations in the three angular directions of roll angle, pitch angle, and heading angle. Calculate the spatial error performance of the point cloud features in the overlapping area after the perturbation respectively, and optimize the angle value accordingly. After each iteration, update the spatial coordinates of the laser point cloud and recalculate the new residual. This loop iteration continues until the change values of the three installation angle errors converge to within the set threshold range respectively. At this time, the calculation is completed, and the final attitude installation angle error is obtained, as follows: (2) (3) (4) Among them, are the attitude installation angle errors of the laser scanner relative to the inertial navigation system in the roll, pitch, and heading directions respectively, is the height difference of a certain feature point between adjacent flight strips, r is the vertical distance from a point at the edge position to the center of the flight strip, d is the offset of the same-name feature point at the center position of the flight strip in two flight strips, H is the relative flight height, R is the distance between the center lines of two flight strips, is the arctangent function.

[0059] On the one hand, the spatial offset directly affects the consistency of multi-sensor data during coordinate transformation. On the other hand, if the attitude installation angle error is not corrected, it will cause systematic tilting or offset of the point cloud data in space, affecting the subsequent registration, fusion, and model construction accuracy. Therefore, completing the calibration of the installation parameters in advance can provide accurate external installation parameters for subsequent navigation trajectory calculation, multi-source data fusion, and aerial triangulation measurement, which helps to improve the spatial consistency and geometric accuracy of the final modeling results.

[0060] In step S420, the fixed-wing UAV is controlled to fly according to the route planning parameters. During the flight, the original navigation data of the fixed-wing UAV is collected through the position and attitude measurement system, and at the same time, image data is collected through the oblique camera, and the original point cloud data is collected through the laser scanner.

[0061] Exemplarily, at least 10 minutes before the UAV takes off, a ground reference station GNSS receiver, a dynamic RTK reference station, and a radio antenna are set up at the selected control points, and the equipment power-on, time synchronization, and data recording initialization operations are completed in sequence to provide reference coordinate support for subsequent navigation trajectory calculation and multi-source data unified registration.

[0062] After the ground system is started up, about 5 minutes before the fixed-wing UAV takes off, the power supply of the on-board equipment is started, and the power-on detection and status initialization of the position and attitude measurement system, oblique camera, laser scanner, communication module, and flight control system are completed to ensure that each part is in a normal working state and to achieve communication connection and data synchronization with the ground base station system and the ground control system.

[0063] During the flight, the original navigation data of the fixed-wing UAV is collected in real time through the position and attitude measurement system, including the three-dimensional position (latitude, longitude, and altitude) of the fixed-wing UAV in the global coordinate system and the attitude angle information (pitch angle, roll angle, yaw angle), and a unified time synchronization reference is provided for image acquisition and laser emission. The collected original navigation data will be used as the basic input for subsequent trajectory calculation and attitude analysis.

[0064] The oblique camera and the laser scanner also operate in coordination with the position and attitude measurement system. During the flight, the oblique camera performs image acquisition according to the preset trigger frequency, preset flight altitude, and overlap degree, and obtains multi-view image data including the front view, left view, right view, rear view, and vertical view. The oblique camera usually consists of multiple lenses, and its imaging directions are distributed along multiple spatial directions, which can enhance the geometric expression ability of structural elements such as the side facades of buildings and road edges, and significantly improve the texture integrity of 3D reconstruction.

[0065] Meanwhile, the laser scanner continuously emits laser pulses at a high frequency (e.g., ≥400 kHz) and records the flight time and intensity information of the laser echoes to obtain the spatial distance of ground targets, thereby forming high-density raw point cloud data. The point cloud data contains the spatial coordinates (X, Y, Z), reflection intensity, multiple echo information, etc. of each sampling point, and can be used for downstream tasks such as terrain modeling, object recognition, and high-precision surveying and mapping.

[0066] Through the spatio-temporal synchronization between sensors, ensure that the timestamps of the position and attitude measurement system are consistent with the pulse trigger of the laser scanner and the exposure time of the tilt camera, so that each image and point cloud segment can be associated with the corresponding flight position and attitude information, thereby supporting subsequent aerial triangulation measurement, coordinate transformation, and multi-source fusion operations. Further, by jointly collecting raw navigation data, image data, and point cloud data by the position and attitude measurement system, tilt camera, and laser scanner during flight, comprehensive coverage of the surveyed area and multi-dimensional data acquisition can be achieved, providing basic data support for subsequent trajectory solution, image positioning, point cloud registration, and 3D model construction.

[0067] In step S430, based on the raw navigation data, GNSS static data collected by the ground reference station, and the reference station coordinate data, perform navigation trajectory solution processing to obtain navigation trajectory data.

[0068] Among them, the GNSS static data collected by the ground reference station is static satellite observation data continuously collected by the ground reference station GNSS receiver installed on the target ground control point. This reference data has the characteristics of stability, high precision, and controllable error, and can be used as a differential correction source. The reference station coordinate data is the known geographical coordinate information of the GNSS reference station installation location, which can be obtained in advance through high-level measurement means such as third-order or fourth-order GPS (Global Positioning System) network and is used as the absolute coordinate reference for later differential solution.

[0069] Exemplarily, the raw navigation data and the GNSS static data collected by the ground reference station can be time-synchronized to unify the timestamp formats of the two types of data and ensure that the observation data within the same time period can correspond one by one. Subsequently, according to the spatial offset of the GNSS antenna relative to the inertial navigation system ( ), the original navigation data after time synchronization is corrected to obtain intermediate navigation data. This correction operation eliminates the position error caused by sensor installation offset and provides installation external parameter support for subsequent fusion. After completing time alignment and external parameter correction, the three-dimensional position and attitude of the fixed-wing UAV at each moment are calculated based on the intermediate navigation data and the reference station coordinate data to obtain navigation trajectory data. For example, the intermediate navigation data and the reference station coordinate data are used as input, and the differential GNSS solution is combined with the inertial navigation filter (such as the Kalman filter) to infer the three-dimensional position coordinates and attitude angle information of the fixed-wing UAV at each time point.

[0070] The navigation trajectory data finally generated may include information such as longitude, latitude, elevation, roll angle, pitch angle, heading angle, etc. throughout the flight, and has a timestamp corresponding to the point cloud data and image exposure events, which is used as a geometric reference for subsequent point cloud coordinate transformation, image attitude solution, and multi-source fusion steps.

[0071] Through the fusion processing of multi-source high-precision navigation data, a navigation trajectory solution result that can support high-precision modeling can be constructed without relying on ground image control points, which can significantly improve the geographic positioning accuracy and geometric constraint reliability of three-dimensional modeling.

[0072] In step S440, based on the navigation trajectory data, the coordinates of the original point cloud data are transformed to obtain registered point cloud data; based on the image exposure point information collected by the position and attitude measurement system, the external orientation elements of the exposure points are calculated, and the registered point cloud data and image data are combined to obtain multi-source fusion data.

[0073] In the example implementation of the present disclosure, when performing point cloud processing, the original point cloud data can be coordinate transformed according to the spatial offset of the laser scanner relative to the inertial navigation system, the attitude placement angle error between the laser scanner and the inertial navigation system, and the navigation trajectory data, so as to obtain registered point cloud data.

[0074] Specifically, firstly, according to the spatial offset of the laser scanner relative to the inertial navigation system ( ) and attitude placement angle error , establish the rigid transformation relationship between the laser scanner body coordinate system and the inertial navigation coordinate system, and then construct the transformation matrix from the inertial navigation coordinate system to the geographic reference coordinate system based on the three-dimensional position and attitude information at each moment in the navigation trajectory data. Use this transformation matrix to transform the original point cloud data to obtain the registered point cloud data after spatial alignment.

[0075] Meanwhile, based on the image exposure point information collected by the position and attitude measurement system, the position data of the fixed-wing UAV at the corresponding moment can be extracted, and the position data includes the position and attitude angle information of the UAV at the shooting moment. According to the spatial offset of the tilt camera relative to the inertial navigation system ( ), the position data is corrected to obtain the exterior orientation elements of the exposure point. Among them, the exterior orientation elements of the exposure point refer to the spatial position and orientation of the camera when each image is exposed, and are composed of six parameters: three-dimensional position (X, Y, Z) and three-dimensional attitude (Roll, Pitch, Yaw). By introducing the spatial offset of the tilt camera relative to the inertial navigation system, the reference position of the exposure point is accurately corrected from the IMU (Inertial Measurement Unit) coordinate system to the camera optical center position, and the spatial geometric description of each image in the geographic reference system is obtained by combining the attitude angles.

[0076] Furthermore, using the exterior orientation elements of the exposure point, the image data is subjected to spatial projection transformation to obtain a georeferenced image. The registered point cloud data is converted to a coordinate system consistent with the georeferenced image and spatially aligned with the georeferenced image to obtain multi-source fusion data. That is, each image in the image data is bound to its corresponding exterior orientation elements of the exposure point, and is aligned with the registered point cloud data through a spatial mapping relationship, realizing the unified expression of the image data and the point cloud data in the same coordinate system, and finally forming multi-source fusion data with consistent structure and corresponding space-time.

[0077] By driving the coordinate transformation process with the navigation trajectory data, and using the calibrated installation exterior parameters and exposure event information to achieve the geometric binding of the image and the point cloud, it lays a data foundation for subsequent aerial triangulation, model construction, and texture mapping. The fusion data has a consistent coordinate system, a joint spatial reference, and aligned attitude semantics, which can significantly improve the modeling consistency and accuracy guarantee ability of multi-source perception data in the subsequent processing process.

[0078] In step S450, based on the multi-source fusion data, aerial triangulation is performed to obtain fusion geometric data.

[0079] In some exemplary embodiments, based on the exterior orientation elements of the exposure point and the interior orientation parameters of the tilt camera, the initial spatial triangulation of the georeferenced image can be performed using a bundle adjustment model to obtain the target exterior orientation elements. Combining the target exterior orientation elements, the registered point cloud data, and the georeferenced image, a multi-source joint adjustment model is constructed. By fusing the image reprojection residuals and the point cloud geometric residuals, a weighted geometric solution is performed to obtain the aligned joint spatial data between the image and the point cloud. Based on the joint spatial data, local iterative registration of the registered point cloud data based on the normal distribution transformation is performed to obtain the fusion geometric data.

[0080] Specifically, in aerial triangulation, first, according to the initial values of the exterior orientation elements of the image exposure points and the interior orientation parameters, combined with the homologous feature points (such as corner points, line segment intersection points, etc.) between adjacent images, the image matching information is extracted. Based on the image matching relationship, an aerial triangulation network structure is constructed, and the exterior orientation elements are set as variables to be optimized, and a bundle adjustment model is constructed. Among them, the bundle adjustment model is a non-linear optimization method based on minimizing the image reprojection error. Utilizing the overlapping relationship between all images and the spatial consistency of homologous point pairs, it iteratively optimizes the exterior orientation parameters of the images to obtain a higher-precision solution for the spatial position and attitude of the images. During the optimization process, the initial values of the exterior orientation elements of the image exposure points are used as observational inputs and assigned certain weights to control their influence weights on the final result in the solution. After completing the initial spatial triangulation, the target exterior orientation elements can be obtained.

[0081] Furthermore, by combining the spatial geometric feature points or boundary constraint points (such as ground object broken lines, corner points, building foundation lines, etc.) extracted from the registered point cloud data, they are introduced into the bundle adjustment model to construct a multi-source joint adjustment model. In this model, both the image reprojection residuals and the point cloud geometric residuals are considered, and by setting appropriate weight coefficients, the two types of data act synergistically to enhance the geometric constraint ability of the final solution result for terrain and structural features.

[0082] For example, by combining image feature points and laser point cloud features, the constructed multi-source joint adjustment model is: (5) Wherein, n is the total number of image feature points, is the pixel coordinate residual of the th image feature point, represents the th image feature point, is the transpose of the pixel coordinate residual is the weight matrix of the image residuals, λ is the weight coefficient of the laser point cloud residuals, m is the number of laser point cloud features, is the geometric observation value of the laser point cloud features, T is the rigid body transformation matrix, is the original three-dimensional coordinate of the th point cloud directly observed by the lidar in the local coordinate system of the sensor.

[0083] The weight matrix of the image residuals in formula (5) is: (6) Wherein, is the iThe observed standard deviation of the horizontal direction (x-axis) of an image feature point in the image pixel coordinate system, is the i observed standard deviation of the vertical direction (y-axis) of the th image feature point in the image pixel coordinate system,

[0084] According to the multi-source joint adjustment model shown in formula (5), minimizing the image matching error and the registration error between the laser point cloud and the image features in space, by λ controlling the fusion weights of the two data sources, so as to achieve fusion optimization. In this process, by introducing the laser point cloud constraint, the initial spatial triangulation calculation results of the image and the point cloud registration matrix can be optimized.

[0085] For areas with weak image texture, occlusion, or sparse distribution of homologous points, the statistical features of the point cloud, such as normal vector consistency, point cloud mean covariance, etc., can be further used for local rigid body registration optimization to compensate for local registration errors and enhance the coordination of multi-source data in the local spatial structure.

[0086] For example, for the occluded area, local point cloud optimization registration can be carried out based on the normal distribution transformation algorithm, and there is: (7) where T is the rigid body transformation matrix, is the mean of the target point cloud in the local area corresponding to the i th point cloud, ∑ i is the covariance matrix of the target point cloud in the local area corresponding to the i th point cloud, is the inverse of the covariance matrix, is the three-dimensional coordinate vector of the i th point cloud in the source point cloud, and the superscript T is the transpose operator, N is the total number of point clouds participating in the registration in the source point cloud, is the exponential function, represents the overall matching degree score between the source point cloud after rigid body transformation and the local Gaussian model of the target point cloud. It can be understood that the mean of the local Gaussian model of the target point cloud is , and the covariance matrix is ∑ i .

[0087] Through the process of aerial triangulation, the optimized exterior orientation elements of all images, the geometric alignment information between the images and the point cloud, and the joint registration positions of each feature point in the three-dimensional space are output, constituting the fusion geometric data with stable structure and balanced errors. This data serves as the geometric basis for subsequent modeling and texture projection, and has strong spatial consistency, robustness, and modeling integrity.

[0088] It is understandable that before performing aerial triangulation, in order to improve the quality and usability of point cloud data and image data, the registered point cloud data can be denoised and density balanced, and the georeferenced image can be radiometrically corrected and image enhanced to obtain multi-source fusion data based on the processed registered point cloud data and georeferenced image.

[0089] Among them, for point cloud data, a noise removal operation is first performed. Point cloud noise mainly comes from factors such as sensor errors, atmospheric interference, and dynamic targets (such as birds and vehicles), and is manifested as isolated outliers or local high-frequency disturbances. During processing, a statistical outlier removal algorithm can be used to perform distribution analysis based on the mean and standard deviation of the distances of neighboring points of each point, and identify and remove outliers whose deviation exceeds a set threshold. Specifically: (8) (9) Among them, is the mean of the distances of neighboring point clouds, representing the average distance of k adjacent points around a certain point, is the standard deviation of the distances of neighboring point clouds, used to reflect the degree of dispersion of the distance distribution, is the Euclidean distance between the current point and the i th adjacent point, k is the preset number of neighboring points.

[0090] Subsequently, the moving least squares method can be introduced for surface reconstruction. By polynomial fitting of the local point cloud region, smooth processing of the geometric surface is achieved, weakening high-frequency interference while retaining structural features.

[0091] For example, there is: (10) Among them, K is the number of neighboring points, is the Gaussian weight function, P are the coordinates of the local surface points to be fitted, is the i th adjacent point coordinate, M is the normal vector of the fitting plane, and the superscript T is the transpose operator, D is the distance from the fitting plane to the origin of coordinates.

[0092] It can be seen from formula (10) that the optimization process is to find a local fitting plane with a normal vector of M and an intercept of D from the origin of coordinates, so that when the plane passes through point P, for all points in the neighborhood The sum of the weighted squared vertical distances is minimized. It should be noted that this plane does not fit the entire point cloud, but moves continuously as the point P to be fitted changes, thus forming a globally smooth surface representation.

[0093] To further control the data scale and improve efficiency, density equalization processing can be performed on the point cloud. Exemplarily, voxelized grid downsampling can be used, setting a larger voxel size (such as 0.5m×0.5m) in flat areas to reduce the number of points, and using a smaller voxel size (such as 0.1m×0.1m) in feature-dense areas to retain local boundary and contour information.

[0094] In addition, an edge-preserving bilateral filtering method can be adopted, comprehensively considering the weighted relationship between spatial position and reflection intensity, and effectively retaining high-frequency details such as building edges and power lines during the point cloud smoothing process. For example, there is: (11) Among them, is the reflection intensity value of point p after the point cloud passes through edge-preserving bilateral filtering. Point p is the pixel point to be filtered, point q is the position of the pixel point participating in the filtering in the neighborhood, is the filtering window range, such as a rectangular area centered on point p, is the pixel intensity value at point p, is the pixel intensity value at point q, is the Gaussian kernel function in the spatial domain, used to control the spatial distance weight, is the spatial standard deviation, is the Gaussian kernel function in the intensity domain, used to control the intensity difference weight, is the intensity standard deviation.

[0095] For image data, first perform radiometric calibration and color consistency processing. Use the histogram matching algorithm to unify the hues of images in different flight strips and eliminate the gray offset caused by lighting differences. The reference image can be selected by brightness statistics, and other images are based on the cumulative distribution function to match the brightness characteristics of the target image to achieve color balance among multiple images.

[0096] If the image has blurring or contrast degradation, the dark channel prior algorithm can be introduced to restore the image sharpness. By reconstructing the image transmittance map and atmospheric light intensity, defogging and edge enhancement processing are achieved to improve the texture expressiveness.

[0097] For example, for images such as clouds and fog, the dark channel prior algorithm is used to restore the sharpness, and there is: (12) Among them, is the intensity value of pixel x in the restored fog-free image, is the intensity value of a pixel in the actual observed image x , is the atmospheric light intensity is the transmittance

[0098] The processed point cloud data and image data respectively have high geometric expression accuracy and texture clarity, providing a reliable data basis and quality guarantee for subsequent aerial triangulation, structure modeling, and texture projection.

[0099] In step S460, based on the fused geometric data, a three-dimensional geographic model is generated through spatial modeling and texture mapping, and a target surveying and mapping result is generated based on the three-dimensional geographic model.

[0100] In some exemplary embodiments, an irregular triangular network model can be constructed based on the fused geometric data using a triangulation algorithm, and the texture information of the image data is mapped to the irregular triangular network model to generate a three-dimensional geographic model. Then, based on the spatial image data corresponding to the three-dimensional geographic model and combined with terrain expression data, a target surveying and mapping result is generated.

[0101] Specifically, in the spatial modeling stage, a surface reconstruction operation is performed based on the three-dimensional point set in the fused geometric data, such as using a triangulation algorithm to construct an irregular triangular network model of the ground surface. This model can cover the entire survey area, including spatial structures such as terrain, buildings, roads, and bridges, realizing a geometric expression of spatial continuity and topological integrity.

[0102] Based on the irregular triangular network model, a texture mapping operation is performed to project multi-view images with spatial attitude parameters after spatial triangulation onto the surface of the irregular triangular network model. Texture mapping adopts an image selection priority strategy, and the best-matching image is selected for texture projection according to the angle between the normal vector of each grid surface and the image shooting direction to ensure the realism and clarity of the three-dimensional model. This process can automatically achieve image stitching, brightness smoothing, and edge transition to generate a three-dimensional geographic model with real geographical semantics.

[0103] After the three-dimensional geographic model is constructed, multiple types of target surveying and mapping results can be further derived from the three-dimensional geographic model. For example, by orthographically projecting the model and combining with a known coordinate system, a digital orthophoto map is automatically generated; based on the point cloud classification result, ground points are extracted to generate a digital elevation model; according to the registered geometric boundaries and feature information in the three-dimensional geographic model, a digital line graph or a structured topographic map is automatically exported; urban-level real scene three-dimensional data with three-dimensional visualization, measurement, and query capabilities is directly provided through the model polygon structure, etc., and the present disclosure does not limit this.

[0104] It should be noted that the three-dimensional geographical model obtained by modeling has various capabilities such as visual display, spatial measurement, and feature extraction, can meet the requirements of large-scale mapping specifications such as 1:500 to 1:2000, and is applicable to various application scenarios such as urban and rural planning, cadastral survey, infrastructure management, and disaster assessment.

[0105] By converting the fused geometric data into a three-dimensional geographical model with texture, geometry, and geographical consistency, and generating various types of surveying and mapping data results on this basis, an integrated processing flow from data acquisition, solution to result output can be realized, ensuring double guarantees of mapping efficiency and result accuracy.

[0106] In the exemplary embodiment of the present disclosure, on the one hand, the fixed-wing unmanned aerial vehicle simultaneously carries an airborne radar and an oblique camera. Without laying ground control points, centimeter-level positioning accuracy can be achieved only relying on high-precision integrated navigation and joint adjustment algorithms, greatly reducing the difficulty of field operations and labor costs, and is particularly suitable for dangerous or inaccessible areas. On the other hand, through multi-sensor fusion technology, the rich texture information of the oblique photography images is deeply combined with the high-precision geometric data of the laser point cloud, which not only ensures the authenticity of the surface color of the model but also significantly improves the ability to restore details of complex terrains. On the other hand, multiple types of surveying and mapping results such as real scene three-dimensional models, digital line graphs, and digital elevation models are synchronously output in one flight, realizing the full-process automation from data acquisition to product delivery, and can meet the diverse needs of different engineering scenarios. Moreover, this method has strong environmental adaptability, can accurately extract surface terrain information through vegetation coverage, and at the same time is compatible with high-precision modeling of various terrains such as urban dense building groups, mountainous steep slopes, and complex landforms.

[0107] In the exemplary embodiment of this example, a three-dimensional modeling and mapping device based on a fixed-wing unmanned aerial vehicle is also provided. Refer to Figure 5 As shown, the three-dimensional modeling and mapping device 500 based on a fixed-wing unmanned aerial vehicle includes a planning parameter determination module 510, a multi-source data acquisition module 520, a navigation trajectory solution module 530, a multi-source data fusion module 540, an aerial triangulation module 550, and a modeling and mapping module 560, where: The planning parameter determination module 510 is used to calculate route planning parameters based on the internal orientation parameters of the oblique camera and the performance parameters of the airborne radar module, in combination with the digital elevation model data and the survey area boundary range of the target area; wherein, the airborne radar module includes a position and attitude measurement system and a laser scanner; The multi-source data acquisition module 520 is used to control the flight of the fixed-wing unmanned aerial vehicle according to the route planning parameters, collect the original navigation data of the fixed-wing unmanned aerial vehicle through the position and attitude measurement system during the flight, and simultaneously collect image data through the oblique camera and collect original point cloud data through the laser scanner; The navigation trajectory calculation module 530 is configured to perform navigation trajectory calculation processing based on the original navigation data, GNSS static data collected by a ground reference station, and reference station coordinate data, so as to obtain navigation trajectory data; The multi-source data fusion module 540 is configured to perform coordinate transformation on the original point cloud data to obtain registered point cloud data; calculate the exterior orientation elements of exposure points based on the navigation trajectory data and image data, and combine the registered point cloud data and image data to obtain multi-source fusion data; The aerial triangulation module 550 is configured to perform aerial triangulation based on the multi-source fusion data to obtain fused geometric data; The modeling and mapping module 560 is configured to generate a three-dimensional geographic model through spatial modeling and texture mapping based on the fused geometric data, and generate a target surveying and mapping result based on the three-dimensional geographic model.

[0108] The specific details of each module of the above three-dimensional modeling and mapping device based on a fixed-wing UAV have been described in detail in the corresponding three-dimensional modeling and mapping method based on a fixed-wing UAV, and thus will not be elaborated here.

[0109] The exemplary embodiments of the present disclosure further provide a computer-readable storage medium, on which a program product capable of implementing the above methods of this specification is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on an electronic device, the program code is used to cause the electronic device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of this specification.

[0110] This program product can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In the present disclosure, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device.

[0111] The program product may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the readable storage medium (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0113] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0114] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C#, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0115] In addition, an exemplary embodiment of the present disclosure further provides an electronic device capable of implementing the above-described three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle.

[0116] The following refers to Figure 6 to describe the electronic device 600 according to such an embodiment of the present disclosure. Figure 6The illustrated electronic device 600 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0117] As Figure 6 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one of the above-mentioned processing units 610, at least one of the above-mentioned storage units 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), and a display unit 640.

[0118] The storage unit 620 stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of this specification. For example, the processing unit 610 may execute the method steps in the exemplary embodiments of the present disclosure.

[0119] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 621 and / or a cache storage unit (Cache) 622, and may further include a read-only storage unit (ROM) 623.

[0120] The storage unit 620 may further include a program / utilities 624 having a set (at least one) of program modules 625. Such program modules 625 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0121] The bus 630 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0122] The electronic device 600 can also communicate with one or more external devices 670 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. As shown in the figure, the network adapter 660 communicates with other modules of the electronic device 600 through the bus 630. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0123] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or can be implemented by the way of software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0124] In addition, the above drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, rather than for limiting purposes. It can be easily understood that the processes shown in the above drawings do not indicate or limit the time sequence of these processes. Additionally, it can also be easily understood that these processes can be executed synchronously or asynchronously in, for example, multiple modules.

[0125] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or can be implemented by the way of software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0126] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0127] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle, characterized in that, Including: Based on the interior orientation parameters of the oblique camera and the performance parameters of the airborne radar module, combining the digital elevation model data of the target area and the survey area boundary range, calculate the route planning parameters; wherein, the airborne radar module includes a position and attitude measurement system and a laser scanner; Control the fixed-wing UAV to fly according to the route planning parameters. During the flight, collect the original navigation data of the fixed-wing UAV through the position and attitude measurement system, and at the same time collect image data through the oblique camera and collect original point cloud data through the laser scanner; Based on the original navigation data, the GNSS static data collected by the ground reference station, and the reference station coordinate data, perform navigation trajectory solution processing to obtain navigation trajectory data; Based on the navigation trajectory data, perform coordinate transformation on the original point cloud data to obtain registered point cloud data; calculate the exterior orientation elements of the exposure points based on the image exposure point information collected by the position and attitude measurement system, and combine the registered point cloud data and the image data to obtain multi-source fusion data; Based on the multi-source fusion data, perform aerial triangulation to obtain fused geometric data; Based on the fused geometric data, generate a three-dimensional geographic model through spatial modeling and texture mapping, and generate the target surveying and mapping results based on the three-dimensional geographic model.

2. The 3D modeling and mapping method based on a fixed-wing UAV according to claim 1, wherein Before calculating the route planning parameters, the method further includes: Obtain the spatial offset of the laser scanner, the oblique camera, and the GNSS antenna relative to the inertial navigation system, and obtain the attitude installation angle error of the laser scanner relative to the inertial navigation system; Wherein, the spatial offset is determined by laboratory calibration, and the attitude installation angle error is determined by a differential iteration solution method based on the characteristics of overlapping flight strips during flight.

3. The three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to claim 2, characterized in that, The performing navigation trajectory solution processing based on the original navigation data, the GNSS static data collected by the ground reference station, and the reference station coordinate data to obtain navigation trajectory data includes: Synchronize the time of the original navigation data and the GNSS static data collected by the ground reference station; Correct the time-synchronized original navigation data according to the spatial offset of the GNSS antenna relative to the inertial navigation system to obtain intermediate navigation data; Based on the intermediate navigation data and the reference station coordinate data, calculate the three-dimensional position and attitude of the fixed-wing UAV at each moment to obtain the navigation trajectory data.

4. The three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to claim 2, wherein Based on the navigation trajectory data, performing coordinate transformation on the original point cloud data to obtain registered point cloud data includes: Perform coordinate transformation on the original point cloud data according to the spatial offset of the laser scanner relative to the inertial navigation system, the attitude installation angle error between the laser scanner and the inertial navigation system, and the navigation trajectory data to obtain registered point cloud data.

5. The three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to claim 2, characterized in that, Calculating the exterior orientation elements of the exposure points based on the image exposure point information collected by the position and attitude measurement system, and combining the registered point cloud data and the image data to obtain multi-source fusion data includes: Extract the position data of the fixed-wing UAV at the corresponding moment according to the image exposure point information collected by the position and attitude measurement system; Correct the position data according to the spatial offset of the oblique camera relative to the inertial navigation system to obtain the exterior orientation elements of the exposure points; Perform spatial projection transformation on the image data by using the exterior orientation elements of the exposure points to obtain a georeferenced image; Convert the registered point cloud data to a coordinate system consistent with the georeferenced image and perform spatial alignment with the georeferenced image to obtain the multi-source fusion data.

6. The three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to claim 5, wherein Based on the multi-source fusion data, perform aerial triangulation to obtain fusion geometric data, including: Based on the exterior orientation elements of the exposure points and the interior orientation parameters of the oblique camera, use the bundle adjustment model to perform initial spatial triangulation on the georeferenced image to obtain the target exterior orientation elements; Combine the target exterior orientation elements, registered point cloud data, and georeferenced image to construct a multi-source joint adjustment model. By fusing the image reprojection residuals and point cloud geometric residuals, perform weighted geometric solution to obtain the joint spatial data aligned between the image and the point cloud; Based on the joint spatial data, perform local iterative registration on the registered point cloud data based on normal distribution transformation to obtain the fusion geometric data.

7. The three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to claim 5, characterized in that The method further includes: Perform denoising processing and density equalization processing on the registered point cloud data, and perform radiometric correction and image enhancement processing on the georeferenced image to obtain the multi-source fusion data according to the processed registered point cloud data and georeferenced image.

8. The three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to claim 1, characterized in that Based on the fusion geometric data, generate a three-dimensional geographic model through spatial modeling and texture mapping, and generate target surveying and mapping results based on the three-dimensional geographic model, including: Based on the fusion geometric data, use the triangulation algorithm to construct an irregular triangular network model; Map the texture information of the image data to the irregular triangular network model to generate a three-dimensional geographic model; Based on the spatial image data corresponding to the three-dimensional geographic model, combine the terrain expression data to generate the target surveying and mapping results.

9. A three-dimensional modeling and mapping system based on a fixed-wing unmanned aerial vehicle, characterized in that, Apply the three-dimensional modeling and mapping method based on a fixed-wing unmanned aerial vehicle according to any one of claims 1 to 8. The system includes: A fixed-wing unmanned aerial vehicle; A power supply module installed on the fixed-wing unmanned aerial vehicle for supplying power to each airborne device; An airborne radar module provided on the fixed-wing unmanned aerial vehicle. The airborne radar module includes a position and attitude measurement system and a laser scanner. The position and attitude measurement system is used to collect the original navigation data of the fixed-wing unmanned aerial vehicle during flight, and the laser scanner is used to collect the original point cloud data during flight; An oblique camera provided on the fixed-wing unmanned aerial vehicle, electrically connected to the airborne radar module, and used to collect image data during flight; A communication module provided on the fixed-wing unmanned aerial vehicle; A flight control system electrically connected to the oblique camera and the airborne radar module; A ground base station system including a ground reference station GNSS receiver, a first tripod, and a ground target control point. The ground reference station GNSS receiver is connected to the first tripod and erected on the ground target control point for providing static reference positioning data; The ground control system is electrically connected to the communication module. The ground control system includes a dynamic RTK reference station, a radio antenna, and a second tripod. The dynamic RTK reference station is electrically connected to the radio antenna. The dynamic RTK reference station and the radio antenna are connected to the second tripod and are used to transmit differential signals to the fixed-wing UAV in real time.

10. A three-dimensional modeling and mapping device based on a fixed-wing unmanned aerial vehicle, characterized in that, Comprising: A planning parameter determination module, configured to calculate route planning parameters based on the internal orientation parameters of the oblique camera and the performance parameters of the airborne radar module, in combination with the digital elevation model data of the target area and the survey area boundary range; wherein, the airborne radar module includes a position and attitude measurement system and a laser scanner; A multi-source data acquisition module, configured to control the flight of the fixed-wing UAV according to the route planning parameters, collect the original navigation data of the fixed-wing UAV through the position and attitude measurement system during the flight, and simultaneously collect image data through the oblique camera and collect original point cloud data through the laser scanner; A navigation trajectory calculation module, configured to perform navigation trajectory calculation processing based on the original navigation data, the GNSS static data collected by the ground reference station, and the reference station coordinate data to obtain navigation trajectory data; A multi-source data fusion module, configured to perform coordinate transformation on the original point cloud data to obtain registered point cloud data; calculate the exterior orientation elements of the exposure points based on the navigation trajectory data and the image data, and combine the registered point cloud data and the image data to obtain multi-source fusion data; An aerial triangulation module, configured to perform aerial triangulation based on the multi-source fusion data to obtain fused geometric data; A modeling and mapping module, configured to generate a three-dimensional geographic model through spatial modeling and texture mapping based on the fused geometric data, and generate a target surveying and mapping result based on the three-dimensional geographic model.

Images