Surveying and mapping method, device and equipment based on unmanned aerial vehicle, medium and program product

Abstract

The invention discloses a surveying and mapping method, device and equipment based on an unmanned aerial vehicle, a medium and a program product, and the method comprises the steps: collecting laser point cloud data and aerial image data of a to-be-measured region through a laser radar aerial survey module carried by the unmanned aerial vehicle, and constructing a three-dimensional model of the to-be-measured region; identifying and determining a target point in the three-dimensional model, controlling the unmanned aerial vehicle to fly to the target point, and identifying the base inclination of the unmanned aerial vehicle landing to the target point; and adjusting the gradient of the base until the gradient of the base is in a tolerance range, and controlling the image total station to map the target point to obtain an absolute three-dimensional coordinate of the target point. The laser radar aerial survey module and the image total station are carried on the unmanned aerial vehicle, after the target point is recognized, the base is sensed in real time and actively adjusted to be in the horizontal state, then sighting and surveying and mapping are carried out, and therefore small targets can be recognized, sighting and measured in the complex environment, and the precision of surveying and mapping operation in the complex dangerous area is improved.

Description

Technical Field

[0001] This invention relates to the field of surveying and mapping technology, and specifically to surveying and mapping methods, devices, equipment, media, and program products based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Traditional surveying methods, such as manual total station surveying and GNSS static surveying, while highly accurate, have limitations such as low operational efficiency, high labor intensity, and difficulty in covering complex and dangerous areas. In recent years, UAV surveying technology has been widely used due to its high efficiency and flexibility.

[0003] However, existing UAV mapping technology still has significant bottlenecks, such as insufficient absolute accuracy, photogrammetry relying on ground control points, difficulty in achieving millimeter-level accuracy in uncontrolled areas, and UAV lidar accuracy significantly decreasing in areas where GNSS signals are blocked; poor environmental adaptability, lack of high-precision absolute reference, making it difficult to achieve autonomous and accurate measurement in complex environments; and low levels of automation and intelligence, requiring a large amount of manual intervention for specific monitoring targets, making it impossible to achieve fully automated operations. Summary of the Invention

[0004] This invention provides a UAV-based surveying method, apparatus, equipment, medium, and program product to solve the problem of how to autonomously identify, aim at, and measure small targets in complex environments.

[0005] In a first aspect, the present invention provides a surveying method based on an unmanned aerial vehicle (UAV), wherein the UAV is equipped with a lidar aerial surveying module and an image total station, and the method includes:

[0006] The laser point cloud data and aerial image data of the area to be measured are collected by the laser radar aerial survey module carried by the UAV, and the laser point cloud data and aerial image data are fused to obtain a three-dimensional model of the area to be measured. The target point is identified in the three-dimensional model, the UAV is controlled to fly to the target point, and the tilt of the base of the image total station when it lands at the target point is identified; Adjust the base tilt until it is within the tolerance range, then control the UAV to survey the target point and obtain the absolute three-dimensional coordinates of the target point.

[0007] This invention, by equipping a drone with a lidar aerial survey module and an image total station, can identify target points, sense them in real time, and actively adjust the base to a horizontal state before aiming and mapping. This enables the identification, aiming, and measurement of small targets even in complex environments, thus improving the accuracy of surveying operations in complex and dangerous areas.

[0008] In an optional implementation, the method further includes: Obtain the historical three-dimensional coordinates of the target point, wherein the historical three-dimensional coordinates are the three-dimensional coordinates of the target point measured in multiple historical periods; Calculate the coordinate difference between the absolute three-dimensional coordinates and the historical three-dimensional coordinates, and determine the time displacement sequence of the target point based on the coordinate difference; The deformation index data of the target point is calculated based on the time displacement sequence, and the deformation index data is compared with the preset safety warning threshold to obtain the comparison result; A monitoring report for the target point is generated based on the three-dimensional coordinates, the time displacement sequence, and the comparison results.

[0009] This invention quantifies the deformation process by constructing a time-displacement sequence of a target point by comparing current and historical coordinates. Based on the time-displacement sequence, key deformation indicators such as cumulative displacement and deformation rate are further calculated and compared with preset safety thresholds to achieve automatic graded early warning of deformation status. Finally, by integrating coordinate data, displacement sequence, analysis indicators, and early warning results, a structured monitoring report is generated, directly transforming raw measurement data into safety information that can be used for decision-making, thus improving the timeliness, accuracy, and automation level of deformation monitoring.

[0010] In one alternative implementation, the UAV communicates with the ground station via a data link; The process of collecting laser point cloud data and aerial image data of the area to be measured through the lidar aerial survey module mounted on the UAV includes: The system presets an autonomous flight path for the UAV in the area to be tested, and controls the UAV to automatically fly according to the autonomous flight path. During flight, the lidar and aerial camera within the lidar aerial survey module are synchronized in time. After time synchronization, the laser point cloud data of the area to be tested is collected by the lidar, and the aerial image data of the area to be tested is acquired by the aerial camera. The laser point cloud data and the aerial image data are transmitted to the ground station or stored in the onboard memory of the UAV via the data link.

[0011] This invention ensures strict spatiotemporal alignment of multi-source data by performing hardware-level time synchronization between the lidar and the camera. At the same time, it transmits the collected data back in real time or stores it locally through a high-speed link, balancing the needs of real-time processing with the reliability of operation in complex terrain and areas with poor signal, and ensuring the integrity and consistency of the original data acquisition.

[0012] In one optional implementation, fusing the laser point cloud data and the aerial image data to obtain a three-dimensional model of the area to be measured includes: Extract the timestamp and the first pose information corresponding to the timestamp from the laser point cloud data, and determine the second pose information corresponding to the timestamp in the aerial image data; Based on the first pose information and the second pose information, the laser point cloud data and the aerial image data are spatially registered; A geometric surface model is established based on the registered laser point cloud data; The aerial image data is mapped onto the geometric surface model to generate a three-dimensional model of the area to be measured.

[0013] This invention utilizes synchronously acquired timestamps and high-precision pose information to achieve strict spatial registration between laser point clouds and aerial imagery, ensuring the geometric accuracy of the fusion. Subsequently, a geometric surface model is constructed based on the precisely registered laser point cloud, and image textures are accurately mapped and fused using this as a base, generating a 3D model that combines millimeter-level geometric details with realistic visual textures, providing a unified 3D model foundation.

[0014] In one optional implementation, the step of spatially registering the laser point cloud data with the aerial image data based on the first pose information and the second pose information includes: Based on the first pose information and the second pose information, the imaging center and imaging direction of the aerial image data are initially aligned with the laser point cloud data; Several two-dimensional feature points are extracted from the aligned aerial image data, and the several two-dimensional feature points are matched to generate a sparse three-dimensional feature point cloud. Spatial registration is performed between the sparse 3D feature point cloud and the laser point cloud data.

[0015] This invention utilizes synchronous pose information to achieve initial geometric alignment of data. Then, by extracting and matching two-dimensional feature points from aerial image data and generating a sparse three-dimensional feature point cloud, and using the sparse feature point cloud as a constraint, it performs fine registration of the laser point cloud, correcting the cumulative errors and system biases that may exist if the inertial navigation system is relied upon alone. This achieves pixel-level precise fusion of the laser point cloud and the aerial image in geometric space.

[0016] In one optional implementation, the geometric surface model comprises multiple geometric patches; the step of mapping the aerial image data onto the geometric surface model to generate a three-dimensional model of the area to be measured includes: Based on the spatial position and orientation of each geometric patch, at least one texture source image is determined from the aerial image data; The pixel region corresponding to the texture source image is projected onto the geometric patch and fused to generate a three-dimensional model of the region to be tested.

[0017] In one optional implementation, the UAV further includes an automatic leveling module; adjusting the base tilt until it is within tolerance, and controlling the total station to map the target point to obtain its absolute three-dimensional coordinates, includes: The tilt angle of the drone's base is obtained by the tilt sensor in the automatic leveling module. Based on the tilt angle, the base of the drone is leveled, and it is determined whether the residual tilt angle of the base after leveling is within the tolerance range. If so, a horizontal reference is established for the target point, and the image total station is controlled to survey the target point at the horizontal reference to obtain the absolute three-dimensional coordinates of the target point; If not, an alarm is triggered, and the UAV is controlled to re-execute the landing and leveling process at another location of the target point until the tilt of the base is within the tolerance range. A horizontal reference for the target point is established, and the total station is controlled to map the target point at the horizontal reference to obtain the absolute three-dimensional coordinates of the target point.

[0018] This invention utilizes a tilt sensor to sense and adjust the base to a level position that meets the measurement requirements in real time. If the adjustment fails once, an alarm is automatically triggered, and the UAV is controlled to relocate and re-land near the target point for leveling until a benchmark is successfully established. This ensures the absolute reliability of the image total station's measurement benchmark, thereby guaranteeing high accuracy and a high success rate of obtaining the absolute three-dimensional coordinates of the target point in complex field environments.

[0019] In one optional implementation, controlling the image total station to map the target point at the horizontal datum includes: Obtain the approximate three-dimensional coordinates of the target point in the three-dimensional model, and control the image total station to perform preliminary positioning of the target point based on the approximate three-dimensional coordinates; After initial positioning, the industrial camera configured on the total station is used to acquire an image of the target point, and the target point is repositioned in the image; The servo drive system of the total station is controlled to aim at and reposition the target point, and the horizontal angle, vertical angle and slant distance of the target point relative to the total station are measured. Obtain the absolute coordinate system of the total station, and determine the three-dimensional coordinates of the target point in the absolute coordinate system based on the horizontal angle, the vertical angle, and the slope distance.

[0020] After obtaining the horizontal angle, vertical angle, slant distance, and known reference parameters of the image total station, this invention can output the high-precision absolute coordinates of the target point in real time and automatically, realizing the seamless conversion of measurement data into engineering-usable coordinate information.

[0021] Secondly, the present invention provides a surveying device based on an unmanned aerial vehicle (UAV), wherein the UAV is equipped with a lidar aerial surveying module and an image total station, and the device includes: The data acquisition module is used to collect laser point cloud data and aerial image data of the area to be measured through the lidar aerial survey module carried by the UAV, and fuse the laser point cloud data and aerial image data to obtain a three-dimensional model of the area to be measured. The tilt calculation module is used to identify and determine the target point in the three-dimensional model, control the UAV to fly to the target point, and identify the tilt of the base where the UAV lands at the target point; The target mapping module is used to adjust the inclination of the base until the inclination of the base is within the tolerance range, and to control the image total station to map the target point to obtain the absolute three-dimensional coordinates of the target point.

[0022] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the UAV-based mapping method described in the first aspect or any corresponding embodiment thereof.

[0023] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the UAV-based mapping method of the first aspect or any corresponding embodiment described above.

[0024] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the UAV-based mapping method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a UAV-based surveying method according to an embodiment of the present invention. Figure 3 This is a structural block diagram of a UAV-based mapping device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0029] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0030] As an optional application scenario of this invention, such as Figure 1 As shown, the UAV mapping system includes a hardware integration unit and a data processing and control unit.

[0031] The hardware integration unit is mounted on a UAV platform and includes an image total station, a lidar aerial survey module, an automatic leveling module, and a synchronization control module. The UAV platform is used for flight and transport to the survey area, ensuring stable real-time downlink communication between the telemetry / remote control link and the mission data. The image total station integrates a high-precision angle and distance measurement unit, an industrial camera, and a servo drive unit for automatic aiming and absolute coordinate measurement of specific targets. The lidar aerial survey module includes a lidar, a high-precision IMU / GNSS system, and an aerial camera, used to quickly acquire large-area, high-precision 3D point cloud and image data. The automatic leveling module adjusts the image total station's base to a level position after the UAV lands, establishing a benchmark for high-precision measurements. The synchronization control module, based on GPS / PPS signals or a high-precision crystal oscillator, sends hardware trigger signals to the lidar, aerial camera, and image total station to achieve time synchronization of all sensor data.

[0032] In this embodiment, the data processing and control unit of the UAV mapping system is integrated into the ground workstation software platform. The data processing and control unit includes a multi-source data fusion module, a small target intelligent recognition module, and a fully automatic aiming control module. The multi-source data fusion module deeply fuses synchronously acquired laser point clouds and image data to generate a color point cloud with realistic texture information and a high-fidelity realistic 3D model. The small target intelligent recognition module, based on a lightweight deep learning model, automatically identifies and locates specific small targets in the image data or laser point cloud. The fully automatic aiming control module receives coarse positioning information from the small target recognition module, performs sub-pixel-level precise positioning using image processing algorithms, and guides the servo system of the total station to rotate to the precise aiming position, triggering measurement.

[0033] In this embodiment, the ground software platform of the UAV mapping system runs on a ground workstation. The ground software platform is used to automate the entire process from raw data import, spatiotemporal synchronization, coordinate transformation, fusion modeling to information extraction, as well as to perform deformation sequence analysis, trend prediction, and threshold warning by comparing multiple periods of measurement data, and to visualize the deformation field.

[0034] Traditional surveying methods, such as manual total station surveying and GNSS static surveying, while highly accurate, have limitations such as low operational efficiency, high labor intensity, and difficulty in covering complex and dangerous areas. In recent years, UAV surveying technology has been widely used due to its high efficiency and flexibility.

[0035] However, existing UAV mapping technology still has significant bottlenecks, such as insufficient absolute accuracy, photogrammetry relying on ground control points, difficulty in achieving millimeter-level accuracy in uncontrolled areas, and UAV lidar accuracy significantly decreasing in areas where GNSS signals are blocked; poor environmental adaptability, lack of high-precision absolute reference, making it difficult to achieve autonomous and accurate measurement in complex environments; and low levels of automation and intelligence, requiring a large amount of manual intervention for specific monitoring targets, making it impossible to achieve fully automated operations.

[0036] Based on this, the present invention provides an embodiment of a mapping method based on unmanned aerial vehicles (UAVs). It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] This embodiment provides a UAV-based surveying method, which can be used in the aforementioned UAV surveying system. The UAV is equipped with a lidar aerial surveying module and an image total station. Figure 2 This is a flowchart of a UAV-based mapping method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: The laser point cloud data and aerial image data of the area to be measured are collected by the laser radar aerial survey module carried by the UAV, and the laser point cloud data and aerial image data are fused to obtain a three-dimensional model of the area to be measured.

[0038] This embodiment first uses a lidar system in the lidar aerial survey module to emit laser pulses and receive echoes. Combined with a high-precision IMU / GNSS system, laser point cloud data is obtained. This laser point cloud data is a massive set of discrete points with precise three-dimensional spatial coordinates and reflection intensity, used to reflect the geometry of the land surface and features. On the other hand, aerial image data is obtained by a synchronously triggered aerial camera in the lidar aerial survey module. This aerial image data is a two-dimensional digital image with rich color and texture information.

[0039] Subsequently, the laser point cloud data and aerial imagery data are deeply fused using the multi-source data fusion module in the ground workstation. Specifically, in terms of time processing, microsecond-level hardware trigger signals provided by the synchronization control module ensure that the laser point cloud data and aerial imagery data are strictly synchronized at the time of acquisition. In terms of spatial processing, coordinate transformation and control registration are performed based on a unified IMU / GNSS system to assign the color and texture information of each pixel in the aerial imagery to the corresponding 3D points in the laser point cloud, thereby generating a color point cloud with both high-precision geometric features and realistic visual attributes. Based on this color point cloud, 3D reconstruction algorithms such as triangulation construction and surface modeling are used to further generate a 3D model of the area to be measured.

[0040] Step S202: Identify and determine the target point in the 3D model, control the UAV to fly to the target point, and identify the tilt of the base where the UAV lands at the target point.

[0041] Specifically, the small target intelligent recognition module in the data processing and control unit processes the colored point cloud and high-resolution texture in the fused 3D model. Specifically, it uses machine learning algorithms to identify and locate specific target points in the 3D model, such as monitoring prisms on slopes or surface cracks. After identification and locating the target point, it outputs the precise position of the target point in the 3D model, which is a 3D coordinate based on the 3D model's coordinate system.

[0042] In one embodiment, the UAV mapping system further includes a cooperative target. The cooperative target is engraved with characteristic patterns such as crosshairs and rings and is made of a highly reflective material. It is used to be attached to the object being measured to assist in measuring the position coordinates of the target. The industrial camera in the image total station is used to photograph the surrounding environment (i.e., the area to be measured) of the image total station and acquire images of the surrounding environment to identify the target point from the surrounding environment images.

[0043] Therefore, the surrounding environment of the total station is first photographed by an industrial camera to obtain the target environment image of the area to be measured. The target environment image includes at least one of wide-angle image and panoramic image. The target environment image is obtained by the industrial camera taking multiple photos of the surrounding environment at the target rotation angle and stitching them together.

[0044] In one embodiment, a cooperative target is identified from the wide-angle or panoramic image, and the center point of the cooperative target is determined as the target point; the cooperative target is then attached to the target being tested.

[0045] Furthermore, when it is necessary to measure the position coordinates (i.e., target point) of the target, the cooperative target is first pasted on the target, and the image total station is set up at an appropriate station position. After the setup is completed, the industrial part of the image total station host continuously takes pictures of the surrounding environment at the target rotation angle. By stitching together multiple images, a target environment image containing wide-angle or panoramic images is obtained.

[0046] Optionally, the target rotation angle is a rotation angle at certain intervals.

[0047] Furthermore, after acquiring a wide-angle or panoramic image of the environment (i.e., the target environment image), a machine learning algorithm is used to identify the cooperative target in the wide-angle or panoramic image and detect the three-dimensional coordinates of the target point (the center point of the cooperative target). Optionally, the machine learning algorithm can be a Convolutional Neural Network (CNN) series or a YOLO (You Only Look Once) series machine learning algorithm.

[0048] Furthermore, the fully automatic aiming control module in the data processing and control unit receives the three-dimensional coordinates of the target point, combines the current position and attitude information of the UAV, automatically plans a flight path from the current position to a suitable hovering position above the target point, and sends the flight command to the embedded flight control module on the UAV platform through a stable and reliable data transmission link, driving the UAV to fly autonomously to the designated position corresponding to the target point.

[0049] After the drone lands in the approximate area of ​​the target point, the dual-axis tilt sensor in the automatic leveling module of the hardware integration unit measures the current tilt state of the image total station's base in real time, obtaining the tilt data of the base. The tilt data is used to determine whether the base meets the requirements of the subsequent high-precision absolute measurement horizontal reference.

[0050] Step S203: Adjust the base inclination until it is within the tolerance range, and control the total station to survey the target point to obtain the absolute three-dimensional coordinates of the target point.

[0051] Specifically, the automatic leveling module receives real-time tilt data of the base, such as pitch and roll angles, from dual-axis tilt sensors. It compares this tilt data with a preset tolerance range, such as the horizontal reference tolerance. If the current tilt data exceeds this tolerance range, the motorized foot screw servo mechanism integrated into the base bottom is activated for fine-tuning. By changing the length of multiple support legs, the base's attitude is corrected until the tilt data fed back by the sensors stabilizes within the preset tolerance range. This establishes a stable horizontal physical reference that meets the requirements for high-precision angle measurement for subsequent target point mapping.

[0052] After the base is automatically leveled to the required level, the fully automatic aiming control module in the data processing and control unit uses the coarse positioning information of the target point provided by the small target intelligent recognition module, combined with the aerial images acquired in real time by the high-resolution industrial camera, to perform sub-pixel-level precise positioning of the target point. After positioning, the servo drive system of the total station is driven to control the telescope or aiming unit on the total station to automatically rotate until it is precisely aimed at the target center corresponding to the target point, so as to measure the angle and distance of the target point and obtain observation data. Furthermore, combined with the center point coordinates of the now leveled base and the azimuth orientation of the base, the observation values ​​are calculated into absolute three-dimensional coordinates through a spatial coordinate transformation mathematical model. The center point coordinates of the base can be obtained from the lidar scan and IMU / GNSS data during the previous flight.

[0053] The UAV-based surveying method provided in this embodiment collects laser point cloud data and aerial image data of the area to be measured through a lidar aerial surveying module. The lidar point cloud data and aerial image data are then fused to obtain a three-dimensional model of the area. Target points are identified and determined within the three-dimensional model. The UAV is then controlled to fly to the target point, and the tilt angle of the UAV's landing platform is identified. The platform tilt angle is adjusted until it is within tolerance. An image total station is then used to survey the target point, obtaining its absolute three-dimensional coordinates. This embodiment equips the UAV with a lidar aerial surveying module and an image total station. The lidar aerial surveying module collects laser point clouds and aerial images to identify target points. After real-time sensing and active adjustment of the platform to a horizontal state, the image total station is used for aiming and surveying. This allows for the identification, aiming, and measurement of small targets even in complex environments, improving the accuracy of surveying operations in complex and dangerous areas.

[0054] In one embodiment, the method further includes: acquiring the historical three-dimensional coordinates of the target point, wherein the historical three-dimensional coordinates are the three-dimensional coordinates measured for the target point within multiple historical periods; calculating the coordinate difference between the absolute three-dimensional coordinates and the historical three-dimensional coordinates, and determining the time displacement sequence of the target point based on the coordinate difference; calculating the deformation index data of the target point based on the time displacement sequence, comparing the deformation index data with a preset safety warning threshold to obtain a comparison result; and generating a monitoring report of the target point based on the three-dimensional coordinates, the time displacement sequence, and the comparison result.

[0055] Specifically, the system retrieves historical 3D coordinates of the target point from the absolute 3D coordinate dataset stored in the ground software platform, which was measured and recorded by this system in multiple previous monitoring cycles. Both the historical 3D coordinates and the latest acquired absolute 3D coordinates are based on a unified absolute coordinate system, ensuring the consistency of the coordinate data in terms of spatiotemporal reference.

[0056] Subsequently, the currently acquired absolute 3D coordinates are compared with the historical 3D coordinates of each period using 3D vector difference operations to obtain the displacement of the target point in each time interval and each 3D direction. These displacements are then arranged in chronological order to obtain a time displacement sequence that reflects the changing pattern of the target point over time.

[0057] Analyzing the time-displacement sequence yields deformation indices including, but not limited to, cumulative displacement, displacement rate, and acceleration. Specifically, the average rate (e.g., millimeters per day), or displacement rate, can be obtained by calculating the ratio of the displacement difference between two adjacent periods to the time interval. The displacement rate characterizes the rate of change of displacement relative to the corresponding time interval. Furthermore, the total displacement of the target point in one or more composite directions from a selected reference starting time to the current time is calculated, and the cumulative deformation is obtained by algebraically summing the relative displacements of each period.

[0058] The deformation index data will be compared with the preset safety warning threshold to determine whether the current deformation state triggers the alarm mechanism and generate corresponding qualitative or quantitative comparison results. When the deformation index data is greater than the preset safety warning threshold, the warning mechanism will be triggered, and relevant personnel will be notified through SMS, email or software interface to indicate the potential risk.

[0059] The ground-based software platform integrates three-dimensional coordinates, time-displacement sequences, and comparative results to generate structured monitoring reports. These reports are typically presented in a graphical format, including the target point's current absolute three-dimensional coordinates, graphs showing displacement over time in each direction, statistical tables of key deformation indicators, current safety status rating, and necessary textual analysis conclusions. This achieves full automation and intelligence throughout the monitoring data process, from acquisition and processing to analysis and output, providing timely and accurate data support for engineering safety assessments and decision-making.

[0060] Furthermore, this embodiment can also arrange the displacement amounts in each direction according to the monitoring time sequence to form displacement time subsequences for each direction. Based on the established displacement time subsequences, the built-in prediction algorithms, such as ARIMA and exponential smoothing, are used to estimate the displacement amount or displacement rate of the target point in the three-dimensional direction in the future period, obtaining prediction results that characterize the future deformation trend of the target point. The prediction results can be used to judge key indicators such as the structural or slope stability of the area under test.

[0061] This embodiment constructs a time displacement sequence of the target point by comparing current and historical coordinates, thereby quantifying its deformation process. Based on the time displacement sequence, key deformation indicators such as cumulative displacement and deformation rate are further calculated and compared with preset safety thresholds to achieve automatic hierarchical early warning of deformation status. Finally, by integrating coordinate data, displacement sequence, analysis indicators, and early warning results, a structured monitoring report is generated, directly transforming raw measurement data into safety information that can be used for decision-making, thus improving the timeliness, accuracy, and automation level of deformation monitoring.

[0062] In one embodiment, the UAV communicates with the ground station via a data link; the UAV's onboard LiDAR aerial survey module collects LiDAR point cloud data and aerial image data of the area to be measured. Specifically, this includes: pre-setting an autonomous flight path for the UAV in the area to be measured and controlling the UAV to automatically fly along the autonomous flight path; during flight, synchronizing the LiDAR and aerial camera within the LiDAR aerial survey module; after time synchronization, collecting LiDAR point cloud data of the area to be measured via the LiDAR and simultaneously acquiring aerial image data of the area to be measured via the aerial camera; and transmitting the LiDAR point cloud data and aerial image data to the ground station or storing them in the UAV's onboard memory via the data link.

[0063] Specifically, the UAV platform in this embodiment integrates a lidar aerial survey module comprising a lidar, aerial camera, and high-precision IMU / GNSS system. It establishes a bidirectional and stable communication connection with the ground control station via a data link for command uploading and data downloading. The UAV's flight path, altitude, speed, and the lidar and camera scanning parameters and image capture intervals are planned in advance on the ground workstation software platform based on the digital orthophoto map of the area to be measured or pre-loaded boundary information, generating flight path commands. After planning, the flight path commands are uploaded to the embedded flight control module on the UAV platform via the data link. The embedded flight control module then guides the UAV to fly automatically according to the preset flight path, altitude, and speed, without the need for continuous manual remote control.

[0064] During flight, the lidar and aerial camera within the lidar aerial survey module are synchronized to ensure the spatiotemporal consistency of lidar point cloud data and aerial image data. Specifically, the synchronization control module sends hardware trigger signals to the lidar's scanning motor / transmitter unit and the aerial camera's electronic shutter, achieving microsecond-level precise synchronization of their data acquisition actions.

[0065] After time synchronization, laser point cloud data of the area under test is collected by a lidar system, while aerial imagery data of the area is acquired by a drone camera. Specifically, under synchronization trigger, the lidar continuously emits laser pulses and receives reflected echoes from the ground and ground objects. Combined with real-time calculated IMU / GNSS position and attitude data, laser point cloud data with three-dimensional coordinates and reflection intensity is generated. At the same time, the drone camera performs exposures at preset shooting points or at equal time intervals to acquire two-dimensional high-resolution aerial imagery data of the area under test.

[0066] The laser point cloud data and aerial image data are transmitted to the ground station or stored in the UAV's onboard memory via a data link. Specifically, the collected laser point cloud data and aerial image data are transmitted to the ground workstation in real time via the data link for subsequent processing. Alternatively, to ensure the reliability of operations in areas with poor communication conditions, the data can be temporarily stored in the UAV's large-capacity onboard memory and then exported after the flight is completed, ensuring the integrity and reliability of the data acquisition.

[0067] This embodiment ensures strict spatiotemporal alignment of multi-source data by performing hardware-level time synchronization between the lidar and the camera. At the same time, it transmits the collected data back in real time or stores it locally through a high-speed link, taking into account both real-time processing requirements and reliability in complex terrain and areas with poor signal, thus ensuring the integrity and consistency of the original data acquisition.

[0068] In one embodiment, laser point cloud data and aerial image data are fused to obtain a three-dimensional model of the area to be measured. Specifically, this includes extracting timestamps and corresponding first pose information from the laser point cloud data, and determining second pose information corresponding to the timestamps in the aerial image data; spatially registering the laser point cloud data and aerial image data based on the first and second pose information; establishing a geometric surface model based on the registered laser point cloud data; and mapping the aerial image data to the geometric surface model to generate a three-dimensional model of the area to be measured.

[0069] Specifically, each laser footprint or scan line in the laser point cloud data carries a recorded timestamp. Therefore, the first pose information corresponding to that timestamp can be extracted, namely the three-dimensional position and attitude angle of the lidar emission center in space. Similarly, for each aerial image data, by reading the timestamp of its exposure time, the second pose information of that timestamp can be extracted from the same navigation system, namely the three-dimensional position and attitude angle of the aerial camera projection center. The first pose information and the second pose information correspond to the same timestamp.

[0070] Subsequently, each 3D coordinate point of the laser point cloud is geometrically associated with each pixel of the aerial image, that is, the imaging rays of the aerial image are matched with the spatial points of the laser point cloud. Based on the geometric spatial registration of the laser point cloud data and the aerial image data, the registered laser point cloud is processed, such as filtering to remove noise and adjusting sampling density. Then, using surface reconstruction algorithms such as triangulation, Poisson reconstruction, or digital elevation model interpolation, a geometric surface model composed of continuous triangular facets or regular grids is generated that accurately reflects the topography and surface undulations of the area under test. This embodiment does not specifically limit the choice of surface reconstruction algorithm. Further, the color information of the pixels in the aerial image data is attached to each triangular facet or grid unit of the geometric surface model. Through the stitching and color equalization of multiple images, a 3D model with both high-precision 3D geometry and realistic, continuous surface texture is generated.

[0071] This embodiment utilizes synchronously acquired timestamps and high-precision pose information to achieve strict spatial registration between laser point clouds and aerial imagery, ensuring the geometric accuracy of the fusion. Subsequently, a geometric surface model is constructed based on the precisely registered laser point cloud, and image textures are accurately mapped and fused using this as a base, generating a 3D model that combines millimeter-level geometric details with realistic visual textures, providing a unified 3D model foundation.

[0072] In one optional embodiment, the laser point cloud data and aerial image data are spatially registered based on the first pose information and the second pose information. Specifically, this includes: initially aligning the imaging center and imaging direction of the aerial image data with the laser point cloud data; extracting several two-dimensional feature points from the aligned aerial image data and matching these two-dimensional feature points to generate a sparse three-dimensional feature point cloud; and spatially registering the sparse three-dimensional feature point cloud with the laser point cloud data.

[0073] Specifically, the first pose information includes the three-dimensional position and three-axis attitude angles of the lidar transmission center in space, namely pitch, roll, and yaw. The second pose information includes the three-dimensional position and three-dimensional attitude angles of the aerial camera projection center in space, namely pitch, roll, and yaw. Based on the first and second pose information, and combined with the relative installation relationship between the aerial camera and lidar obtained through calibration, the projection center of each aerial image in space and the initial direction of its imaging beam are determined. This allows the imaging beam of the aerial image to be initially projected onto the three-dimensional space where the lidar point cloud is located, achieving initial alignment between the two.

[0074] Based on the initial alignment, two-dimensional feature points are extracted from the aerial image data. These two-dimensional feature points are pixel locations in the aerial image with significant local texture features, such as corner points and edge intersections. Subsequently, the two-dimensional feature points are matched between different aerial images to establish corresponding image point pairs. Based on these matched corresponding image point pairs, combined with the aerial camera's intrinsic parameters such as focal length and principal point coordinates, and the first and second pose information after initial alignment, the three-dimensional spatial coordinates corresponding to the corresponding image point pairs are determined using the principle of multi-view forward intersection. Thus, the laser point cloud corresponding to the three-dimensional spatial coordinates of the corresponding image point pairs is used as a sparse three-dimensional feature point cloud that is directly associated with the aerial image but has a lower density than the original laser point cloud.

[0075] Furthermore, point cloud registration algorithms such as the iterative nearest point algorithm are adopted. By determining the optimal three translation parameters and three rotation parameters, the sparse three-dimensional feature point cloud and the original laser point cloud have the highest degree of overlap in three-dimensional space, i.e., the smallest error. Thus, the spatial registration of the sparse three-dimensional feature point cloud and the laser point cloud is completed, and pixel-level fusion of image texture information and laser point cloud geometric information is achieved.

[0076] This embodiment utilizes synchronized pose information to achieve initial geometric alignment of data. Then, by extracting and matching two-dimensional feature points from aerial image data and generating a sparse three-dimensional feature point cloud, and using the sparse feature point cloud as a constraint, the laser point cloud is finely registered. This corrects the cumulative errors and system deviations that may exist if the inertial navigation system is relied upon alone, thereby achieving pixel-level precise fusion of the laser point cloud and the aerial image in geometric space.

[0077] In one optional embodiment, the geometric surface model comprises multiple geometric patches. Aerial image data is mapped onto the geometric surface model to generate a three-dimensional model of the area to be tested. Specifically, this includes determining at least one texture source image from the aerial image data based on the spatial position and orientation of each geometric patch; projecting the pixel region corresponding to the texture source image onto the geometric patch and performing a fusion process to generate a three-dimensional model of the area to be tested.

[0078] Specifically, based on the registered laser point cloud data, a geometric surface model is generated using surface reconstruction algorithms such as triangulation, Poisson reconstruction, or digital elevation model interpolation. This model contains a large number of continuous geometric patches covering the entire area to be measured. Each geometric patch has a definite spatial position and normal vector in three-dimensional space. The spatial position represents the vertex coordinates of the geometric patch, and the normal vector represents the orientation of the geometric patch.

[0079] For each geometric patch in the geometric surface model, based on its spatial location and normal vector, and using the established spatial registration relationship, the visible aerial images and their imaging quality are determined. Typically, the aerial image whose field of view is closest to the geometric patch's normal direction and whose resolution is greater than a preset resolution threshold is selected as the texture source image for that geometric patch. To obtain complete texture, the texture source image for a geometric patch may be determined from multiple overlapping images.

[0080] After determining the source images of the textures, the contours of the geometric patches in 3D space are precisely back-projected onto each selected source image of the textures through projection transformation, thereby determining the pixel regions corresponding to the geometric patches on the source images of the textures. Since one geometric patch may correspond to multiple source images, multiple pixel regions may be obtained. Subsequently, the pixel regions from different source images are fused. The fusion algorithm may include weighted averaging based on imaging angle and / or distance, selecting the clearest texture closest to the orthophoto viewpoint, etc., to eliminate visual discontinuities caused by differences in lighting, color deviations, or splicing misalignments. This embodiment does not specifically limit the selection of the fusion algorithm. Finally, the fused pixel regions are uniformly fitted onto each corresponding geometric patch of the geometric surface model, thereby generating a 3D model of the area to be tested.

[0081] In one embodiment, the UAV further includes an automatic leveling module that adjusts the base tilt until it is within tolerance, and controls a total station to map the target point to obtain its absolute three-dimensional coordinates. Specifically, this includes acquiring the UAV's base tilt using a tilt sensor in the automatic leveling module, leveling the UAV's base based on the tilt, and determining whether the residual tilt angle of the base after leveling is within tolerance. If so, a horizontal reference for the target point is established, and the total station is controlled to map the target point at the horizontal reference to obtain its absolute three-dimensional coordinates. If not, an alarm is triggered, and the UAV is controlled to re-execute the landing and leveling process at another location on the target point until the base tilt is within tolerance, a horizontal reference for the target point is established, and the total station is controlled to map the target point at the horizontal reference to obtain its absolute three-dimensional coordinates.

[0082] Specifically, after the drone lands at the target point, the tilt angle of the drone's base is obtained through a tilt sensor in the automatic leveling module. The tilt sensor is used to accurately measure the tilt angle of the carrier relative to the direction of gravity, and outputs the angle values ​​of the base in the pitch and roll directions in real time. Based on this real-time tilt angle data, this embodiment drives the electric foot screw servo mechanism in the automatic leveling module to adjust and counteract the tilt, achieving active leveling of the base. The electric foot screw servo mechanism is a support adjustment mechanism that is precisely controlled by a motor and can achieve linear extension and retraction.

[0083] After leveling the base, the residual tilt angle of the base after leveling is continuously monitored by the tilt sensor; that is, the residual tilt angle that still exists after adjustment. The residual tilt angle is compared and judged with a preset tolerance range, which is an angle range threshold set according to the measurement accuracy requirements, such as ±3 arcseconds.

[0084] If the residual tilt angle is within the tolerance range, it indicates that the base has reached a high-precision level. At this point, a horizontal benchmark is established for the target point based on the current angle of the base, providing a starting reference surface for subsequent image total station surveying. Subsequently, the guide servo system drives the telescope in the image total station to accurately aim at the target point, measure the angle and distance of the target point, and finally combine the absolute coordinates and azimuth of the base center point to determine the absolute three-dimensional coordinates of the target point.

[0085] If the residual tilt angle exceeds the tolerance range, an alarm is triggered and the operator is alerted. Then, the drone is controlled to another location at the target point, and the above-mentioned sensing, leveling, and verification process is repeated until the base tilt is within the tolerance range. At the new location, a horizontal reference is established and the absolute coordinate measurement is completed.

[0086] This embodiment ensures that, under complex and non-ideal surface conditions, a horizontal measurement benchmark that meets the accuracy requirements can be reliably established through closed-loop control and an adaptive retry strategy, thereby guaranteeing the high accuracy and reliability of the final obtained absolute three-dimensional coordinates.

[0087] In one optional embodiment, the image total station is controlled to map the target point at a horizontal reference. Specifically, this includes: acquiring approximate 3D coordinates of the target point in a 3D model; using these approximate 3D coordinates, controlling the image total station to perform preliminary positioning of the target point; after preliminary positioning, using an industrial camera configured on the image total station to acquire an image of the target point, and repositioning the target point within the image; controlling the servo drive system of the image total station to aim at the repositioned target point, and measuring the horizontal angle, vertical angle, and slope distance of the target point relative to the image total station; acquiring the absolute coordinate system of the image total station, and determining the 3D coordinates of the target point in the absolute coordinate system based on the horizontal angle, vertical angle, and slope distance.

[0088] Specifically, the small target intelligent recognition module in the data processing and control unit extracts and outputs the position information of the target point from the 3D model of the area to be measured, and uses the position coordinates as the approximate 3D coordinates of the target point. The fully automatic aiming control module in the data processing and control unit converts the approximate 3D coordinates into angle drive commands for the servo drive system in the image total station, thereby controlling the rotation of the telescope or aiming unit to bring the target point into the field of view of the telescope, thus shortening the time for subsequent fine-grained searching.

[0089] Once the target point enters the telescope's field of view, the high-speed, high-resolution industrial camera built into the total station captures a digital image of the current field of view. Image processing algorithms such as template matching, edge detection, and centroid methods are then used to process the digital image, repositioning the target point on the industrial camera's imaging surface with pixel-level or even sub-pixel-level accuracy, obtaining precise pixel coordinates, and eliminating errors caused by mechanical transmission and approximate coordinates.

[0090] Furthermore, based on the precise pixel coordinates obtained from repositioning in the digital image, the servo system is driven to make fine movements, aligning the image of the target point with the center of the crosshairs of the telescope or the preset reference position of the industrial camera, thus completing the aiming. Subsequently, using the high-precision angle and distance measuring unit inside the image total station, the horizontal angle, vertical angle, and slope distance of the target point relative to the image total station are measured.

[0091] The absolute coordinate system of the image total station is obtained. This system includes the three-dimensional coordinates of the total station's center point within the global absolute coordinate system and the total station's spatial orientation parameters. Specifically, the three-dimensional coordinates of the total station's center point in the global absolute coordinate system can be determined by calibrating the center of a leveled base; and the spatial orientation parameters of the total station can be obtained through a high-precision IMU / GNSS system. Therefore, based on the absolute coordinate system and the target point's horizontal angle, vertical angle, and slope distance relative to the total station, the three-dimensional coordinates of the target point in the absolute coordinate system can be calculated.

[0092] This embodiment utilizes a tilt sensor to sense and adjust the base to a level state that meets the measurement requirements in real time. If the adjustment fails once, an alarm is automatically triggered, and the UAV is controlled to relocate and re-land near the target point for leveling until a benchmark is successfully established. This ensures the absolute reliability of the benchmark for subsequent total station measurements, thereby guaranteeing the high accuracy and success rate of the absolute three-dimensional coordinates of the target point obtained in complex field environments.

[0093] Specifically, first determine the known three-dimensional coordinates of the center point of the total station in the absolute coordinate system. And determine the spatial orientation parameters of the image total station, that is, the azimuth angle of the zero direction of the image total station in the absolute coordinate system. Subsequently, the horizontal angle of the target point was obtained in real time using a total station image measurement. vertical angle and slant distance Coordinate calculation is then performed. The horizontal angle is the angle from the zero direction of the total station (clockwise) to the direction of the target; the vertical angle is the angle from the zenith direction to the direction of the target; and the slope distance is the straight-line distance from the center of the total station to the target point. The specific coordinate calculation formula is as follows: First, calculate the coordinate increments in the local coordinate system of the image total station:

[0094]

[0095]

[0096] in, This represents the x-axis increment in the local coordinate system. This represents the y-axis increment in the local coordinate system. This represents the z-axis increment in the local coordinate system. It is a horizontal angle; It is a vertical angle; This is the slope distance.

[0097] Then, by considering the azimuth angle The rotation matrix of the image total station, consisting of possible horizontal axis errors, vertical axis errors, and other calibration parameters. Transform the coordinate increments to the absolute coordinate system:

[0098] in, This represents the x-axis increment in the absolute coordinate system. This represents the y-axis increment in the absolute coordinate system. This represents the z-axis increment in the absolute coordinate system. This represents the x-axis increment in the local coordinate system. This represents the y-axis increment in the local coordinate system. This represents the z-axis increment in the local coordinate system.

[0099] Finally, the coordinate increments in the absolute coordinate system are added to the known three-dimensional coordinates of the center point of the image total station in the absolute coordinate system to obtain the absolute three-dimensional coordinates of the target point:

[0100]

[0101]

[0102] in, These are the absolute three-dimensional coordinates of the target point; These are the known three-dimensional coordinates of the center point of the image total station in the absolute coordinate system; This represents the x-axis increment in the absolute coordinate system. This represents the y-axis increment in the absolute coordinate system. This represents the z-axis increment in the absolute coordinate system.

[0103] Thus, after obtaining the horizontal angle, vertical angle, slope distance, and known reference parameters of the image total station, this embodiment can output the high-precision absolute coordinates of the target point in real time and automatically, realizing the seamless conversion of measurement data into engineering-usable coordinate information.

[0104] This embodiment also provides a UAV-based mapping device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0105] This embodiment provides a UAV-based surveying device, in which the UAV is equipped with a lidar aerial surveying module and an image total station, such as... Figure 3 As shown, it includes: The data acquisition module 301 is used to collect laser point cloud data and aerial image data of the area to be measured through the lidar aerial survey module carried by the UAV, and to fuse the laser point cloud data and aerial image data to obtain a three-dimensional model of the area to be measured.

[0106] The tilt calculation module 302 is used to identify and determine the target point in the 3D model, control the UAV to fly to the target point, and identify the tilt of the base where the UAV lands at the target point.

[0107] The target mapping module 303 is used to adjust the base inclination until it is within the tolerance range, and to control the image total station to map the target point and obtain the absolute three-dimensional coordinates of the target point.

[0108] In some alternative embodiments, the apparatus further includes: The historical data acquisition module is used to acquire the historical three-dimensional coordinates of the target point. The historical three-dimensional coordinates are the three-dimensional coordinates of the target point measured over multiple historical periods.

[0109] The displacement determination module is used to calculate the coordinate difference between the absolute three-dimensional coordinates and the historical three-dimensional coordinates, and to determine the time displacement sequence of the target point based on the coordinate difference.

[0110] The data comparison module is used to calculate the deformation index data of the target point based on the time displacement sequence, compare the deformation index data with the preset safety warning threshold, and obtain the comparison result.

[0111] The report generation module is used to generate monitoring reports for target points based on three-dimensional coordinates, time displacement sequences, and comparison results.

[0112] In some alternative implementations, the UAV communicates with the ground station via a data link; the data acquisition module 301 includes: The flight unit is used to preset the autonomous flight path of the UAV in the area to be tested, and to control the UAV to fly automatically according to the autonomous flight path.

[0113] The time synchronization unit is used to synchronize the time between the lidar and the aerial camera in the lidar aerial survey module during flight.

[0114] The data acquisition unit is used to acquire laser point cloud data of the area to be tested via LiDAR after time synchronization, and simultaneously acquire aerial image data of the area to be tested via aerial camera.

[0115] The data transmission unit is used to transmit laser point cloud data and aerial image data to a ground station or store them in the onboard memory of the UAV via a data link.

[0116] In some optional implementations, the data acquisition module 301 includes: The pose extraction unit is used to extract the timestamp and the first pose information corresponding to the timestamp from the laser point cloud data, and to determine the second pose information corresponding to the timestamp in the aerial image data.

[0117] The spatial registration unit is used to spatially register laser point cloud data with aerial image data based on the first pose information and the second pose information.

[0118] The surface model building unit is used to build a geometric surface model based on the registered laser point cloud data.

[0119] The 3D model building unit is used to map aerial image data onto a geometric surface model to generate a 3D model of the area to be measured.

[0120] In some optional implementations, the spatial registration unit is specifically used to initially align the imaging center and imaging direction of the aerial image data with the laser point cloud data based on the first pose information and the second pose information; extract several two-dimensional feature points from the aligned aerial image data, and match the several two-dimensional feature points to generate a sparse three-dimensional feature point cloud; and spatially register the sparse three-dimensional feature point cloud with the laser point cloud data.

[0121] In some optional implementations, the geometric surface model includes multiple geometric patches; a three-dimensional model building unit is specifically used to determine at least one texture source image from aerial imagery data based on the spatial position and orientation of each geometric patch; the pixel region corresponding to the texture source image is projected onto the geometric patch and fused to generate a three-dimensional model of the region to be tested.

[0122] In some alternative implementations, the UAV also includes an automatic leveling module; the target mapping module 303 includes: The error judgment unit is used to obtain the tilt angle of the drone's base through the tilt sensor in the automatic leveling module, level the drone's base based on the tilt angle, and determine whether the residual tilt angle of the base after leveling is within the tolerance range.

[0123] The surveying unit is used to establish a horizontal benchmark for the target point if the condition is met, and control the total station to survey the target point at the horizontal benchmark to obtain the absolute three-dimensional coordinates of the target point; if the condition is not met, an alarm is triggered, and the total station is controlled to re-execute the landing and leveling process at another location of the target point until the base tilt is within the tolerance range, establish a horizontal benchmark for the target point, and control the UAV to survey the target point at the horizontal benchmark to obtain the absolute three-dimensional coordinates of the target point.

[0124] In some alternative implementations, the UAV also includes an image total station; the mapping unit includes: The surveying subunit is used to obtain the approximate 3D coordinates of the target point in the 3D model. Based on the approximate 3D coordinates, it controls the image total station to perform preliminary positioning of the target point. After preliminary positioning, it uses the industrial camera configured on the image total station to acquire an image of the target point and repositions the target point in the image. It controls the servo drive system of the image total station to aim at the repositioned target point and measures the horizontal angle, vertical angle, and slope distance of the target point relative to the image total station. It obtains the absolute coordinate system of the image total station and determines the 3D coordinates of the target point in the absolute coordinate system based on the horizontal angle, vertical angle, and slope distance.

[0125] The UAV-based surveying device provided in this embodiment of the invention can execute the UAV-based surveying method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0126] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0127] The following is a detailed reference. Figure 4This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0128] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0129] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a memory 408, or installed from a ROM 402. When the computer program is executed by the processor 401, it performs the functions defined in the UAV-based mapping method of the embodiments of the present invention.

[0130] Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0131] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the UAV-based mapping method shown in the above embodiments is implemented.

[0132] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0133] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A surveying method based on unmanned aerial vehicles (UAVs), characterized in that, The UAV is equipped with a lidar aerial survey module and an image total station, and the method includes: The laser point cloud data and aerial image data of the area to be measured are collected by the laser radar aerial survey module carried by the UAV, and the laser point cloud data and aerial image data are fused to obtain a three-dimensional model of the area to be measured. The target point is identified in the three-dimensional model, the UAV is controlled to fly to the target point, and the tilt of the base where the UAV lands at the target point is identified; Adjust the inclination of the base until it is within the tolerance range, and control the total station to survey the target point to obtain the absolute three-dimensional coordinates of the target point.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the historical three-dimensional coordinates of the target point, wherein the historical three-dimensional coordinates are the three-dimensional coordinates of the target point measured in multiple historical periods; Calculate the coordinate difference between the absolute three-dimensional coordinates and the historical three-dimensional coordinates, and determine the time displacement sequence of the target point based on the coordinate difference; The deformation index data of the target point is calculated based on the time displacement sequence, and the deformation index data is compared with the preset safety warning threshold to obtain the comparison result; A monitoring report for the target point is generated based on the three-dimensional coordinates, the time displacement sequence, and the comparison results.

3. The method according to claim 1, characterized in that, The drone communicates with the ground station via a data link. The process of collecting laser point cloud data and aerial image data of the area to be measured through the lidar aerial survey module mounted on the UAV includes: The system presets an autonomous flight path for the UAV in the area to be tested, and controls the UAV to automatically fly according to the autonomous flight path. During flight, the lidar and aerial camera within the lidar aerial survey module are synchronized in time. After time synchronization, the laser point cloud data of the area to be tested is collected by the lidar, and the aerial image data of the area to be tested is acquired by the aerial camera. The laser point cloud data and the aerial image data are transmitted to the ground station or stored in the onboard memory of the UAV via the data link.

4. The method according to claim 1 or 3, characterized in that, The process of fusing the laser point cloud data and the aerial image data to obtain a three-dimensional model of the area to be measured includes: Extract the timestamp and the first pose information corresponding to the timestamp from the laser point cloud data, and determine the second pose information corresponding to the timestamp in the aerial image data; Based on the first pose information and the second pose information, the laser point cloud data and the aerial image data are spatially registered; A geometric surface model is established based on the registered laser point cloud data; The aerial image data is mapped onto the geometric surface model to generate a three-dimensional model of the area to be measured.

5. The method according to claim 4, characterized in that, The step of spatially registering the laser point cloud data with the aerial image data based on the first pose information and the second pose information includes: Based on the first pose information and the second pose information, the imaging center and imaging direction of the aerial image data are initially aligned with the laser point cloud data; Several two-dimensional feature points are extracted from the aligned aerial image data, and the several two-dimensional feature points are matched to generate a sparse three-dimensional feature point cloud. Spatial registration is performed between the sparse 3D feature point cloud and the laser point cloud data.

6. The method according to claim 5, characterized in that, The geometric surface model comprises multiple geometric patches; the process of mapping the aerial image data onto the geometric surface model to generate a 3D model of the area to be measured includes: Based on the spatial position and orientation of each geometric patch, at least one texture source image is determined from the aerial image data; The pixel region corresponding to the texture source image is projected onto the geometric patch and fused to generate a three-dimensional model of the region to be tested.

7. The method according to claim 1, characterized in that, The drone also includes an automatic leveling module; adjusting the base tilt until it is within tolerance, and controlling the total station to survey the target point to obtain its absolute three-dimensional coordinates, includes: The tilt angle of the drone's base is obtained by the tilt sensor in the automatic leveling module. Based on the tilt angle, the base of the drone is leveled, and it is determined whether the residual tilt angle of the base after leveling is within the tolerance range. If so, a horizontal reference is established for the target point, and the image total station is controlled to survey the target point at the horizontal reference to obtain the absolute three-dimensional coordinates of the target point; If not, an alarm is triggered, and the UAV is controlled to re-execute the landing and leveling process at another location of the target point until the tilt of the base is within the tolerance range. A horizontal reference for the target point is established, and the total station is controlled to map the target point at the horizontal reference to obtain the absolute three-dimensional coordinates of the target point.

8. The method according to claim 7, characterized in that, The process of controlling the total station to survey the target point at the horizontal datum includes: Obtain the approximate three-dimensional coordinates of the target point in the three-dimensional model, and control the image total station to perform preliminary positioning of the target point based on the approximate three-dimensional coordinates; After initial positioning, the industrial camera configured on the total station is used to acquire an image of the target point, and the target point is repositioned in the image; The servo drive system of the total station is controlled to aim at and reposition the target point, and the horizontal angle, vertical angle and slant distance of the target point relative to the total station are measured. Obtain the absolute coordinate system of the total station, and determine the three-dimensional coordinates of the target point in the absolute coordinate system based on the horizontal angle, the vertical angle, and the slope distance.

9. A mapping device based on an unmanned aerial vehicle (UAV), characterized in that, The UAV is equipped with a lidar aerial survey module and an image total station. The device includes: The data acquisition module is used to collect laser point cloud data and aerial image data of the area to be measured through the lidar aerial survey module carried by the UAV, and fuse the laser point cloud data and aerial image data to obtain a three-dimensional model of the area to be measured. The tilt calculation module is used to identify and determine the target point in the three-dimensional model, control the UAV to fly to the target point, and identify the tilt of the base where the UAV lands at the target point; The target mapping module is used to adjust the inclination of the base until the inclination of the base is within the tolerance range, and to control the image total station to map the target point to obtain the absolute three-dimensional coordinates of the target point.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the UAV-based mapping method according to any one of claims 1 to 8.

Images