A color point cloud model generation method and device

Abstract

The embodiment of the present application provides a kind of color point cloud model generation method and device, it is related to visual positioning technical field, method includes: the observation data of current frame acquisition of unmanned aerial vehicle in target area is collected;If it is detected that there is loop relationship between current frame and historical key frame, based on the position information of the unmanned aerial vehicle in current frame provided by the point cloud data of current frame, RTK data, and the relative pose relationship of the unmanned aerial vehicle current frame and historical key frame, determine the first pose of current frame unmanned aerial vehicle;Determine the conversion relationship from the second pose of the unmanned aerial vehicle to the first pose;Based on conversion relationship, each three-dimensional point in the point cloud data of adjacent frame is converted into the point cloud data of current frame, to obtain fusion point cloud data;Based on the color information of each three-dimensional point in the fusion point cloud data, generate the color point cloud model of target area, with high precision and real color.

Description

Technical Field

[0001] This invention relates to the field of visual positioning technology, and in particular to a method and apparatus for generating a color point cloud model. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) technology using lidar is a core technology for achieving real-time positioning and high-precision 3D modeling. LiDAR-SLAM algorithms rely on the geometric features of laser point clouds to achieve high-precision positioning and 3D modeling in scenes with rich structural textures. However, limited by the imaging principle of laser sensors, the generated 3D models lack color information, making it difficult to meet the needs of fields such as aerial surveying for scene visualization and refined analysis.

[0003] In related technologies, to overcome the aforementioned shortcomings, a LiDAR-SLAM algorithm that integrates visual sensors is employed to generate a colored point cloud model by combining color information from visual images. However, in aerial surveying applications, this approach still suffers from the following problems: Firstly, the vibrations of the UAV during flight can severely interfere with the Inertial Measurement Unit (IMU), reducing the accuracy of 3D modeling. Secondly, when the UAV flies over flat ground, the geometric features of the point cloud data collected by the LiDAR are limited, resulting in a lack of effective constraint information that can be extracted, leading to algorithm drift or even complete failure. This results in the inability to output a high-precision colored point cloud model that meets aerial surveying standards, thus reducing the accuracy and realism of the generated colored point cloud model. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for generating color point cloud models, so as to improve the accuracy and realism of the generated color point cloud models. The specific technical solution is as follows:

[0005] Firstly, in order to achieve the above objectives, embodiments of the present invention provide a method for generating a color point cloud model, the method comprising:

[0006] Acquire observation data collected by the UAV in the current frame in the target area; wherein, the observation data includes: point cloud data, image data, and RTK data;

[0007] If a loop relationship is detected between the current frame and historical keyframes, the first pose of the UAV in the current frame is determined based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and historical keyframes of the UAV; wherein, the first pose is the optimal solution of the pre-constructed target optimization function.

[0008] Determine the transformation relationship from the second pose of the UAV to the first pose; wherein the second pose is the pose of the adjacent frame of the current frame;

[0009] Based on the transformation relationship, each 3D point in the point cloud data of adjacent frames is transformed into the point cloud data of the current frame to obtain fused point cloud data;

[0010] Based on the color information of each three-dimensional point in the fused point cloud data, a color point cloud model of the target region is generated, wherein the color information of each three-dimensional point in the fused point cloud data is determined according to the image data of the current frame.

[0011] Optionally, determining the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and historical keyframes includes:

[0012] Based on the RTK data of the current frame, calculate the position data of the UAV in the current frame in the global coordinate system;

[0013] Using the UAV's position data in the current frame as a constraint, and the relative pose relationship between the current frame and historical keyframes as a constraint, the optimal solution of the pre-constructed objective optimization function is calculated based on the point cloud data of the current frame to obtain the first pose of the UAV in the current frame.

[0014] The target optimization function represents the functional relationship between the target error and the first pose of the current frame of the UAV; the target error includes the error of the relative pose relationship between adjacent frames of the UAV, the error of the relative pose relationship between the current frame of the UAV and historical key frames, and the error of the UAV's position data.

[0015] Optionally, before determining the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and the historical keyframes if a loop closure relationship is detected between the current frame and the historical keyframes, the method further includes:

[0016] Acquire the constructed target map data of the target area; wherein, the target map data includes: the third pose of each historical key frame, and the third pose of each historical key frame is: the pose of the UAV at the acquisition time corresponding to each historical key frame.

[0017] Based on the point cloud data of the current frame and the third pose of each historical keyframe, the current frame is matched with each historical keyframe.

[0018] If the current frame matches a historical keyframe, a loopback relationship is determined between the current frame and the historical keyframe.

[0019] Optionally, after acquiring the constructed target map data for the target area, the method further includes:

[0020] If, based on the first pose and the target map data, it is determined that the UAV has moved to a new surveying area in the target region, then, based on the point cloud data of the current frame, local map data of the surveying area is constructed. The new surveying area is: the unsurveyed area in the target region other than the surveyed area represented by the target map data.

[0021] The local map data is added to the target map data to obtain the updated target map data for the target area.

[0022] Optionally, before determining the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and the historical keyframes if a loop closure relationship is detected between the current frame and the historical keyframes, the method further includes:

[0023] Time synchronization is performed on the point cloud data and RTK data of each frame to obtain the time offset of the point cloud data and RTK data;

[0024] Based on the time offset, the RTK data of the current frame's point cloud data at the same time is determined from each frame's RTK data, thus obtaining the RTK data of the current frame.

[0025] Optionally, the step of determining the RTK data of the current frame point cloud data at the same moment from each frame of RTK data based on the time offset, to obtain the current frame RTK data, includes:

[0026] Using the time offset, the timestamps of each frame of RTK data under the RTK time are converted to timestamps under the radar time.

[0027] From the RTK data of each frame, determine the RTK data with the same timestamp as the point cloud data of the current frame, and obtain the RTK data of the current frame.

[0028] Secondly, in order to achieve the above objectives, embodiments of the present invention provide a color point cloud model generation apparatus, the apparatus comprising:

[0029] The observation data acquisition module is used to acquire observation data collected by the UAV in the target area in the current frame; wherein, the observation data includes: point cloud data, image data, and RTK data;

[0030] The first pose determination module is used to determine the first pose of the UAV in the current frame if a loop relationship is detected between the current frame and the historical keyframes, based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and the historical keyframes; wherein, the first pose is the optimal solution of a pre-constructed target optimization function.

[0031] A transformation relationship determination module is used to determine the transformation relationship from the second pose of the UAV to the first pose; wherein the second pose is the pose of the adjacent frame of the current frame;

[0032] The fused point cloud data generation module is used to convert each three-dimensional point in the point cloud data of adjacent frames to the point cloud data of the current frame based on the conversion relationship, so as to obtain fused point cloud data.

[0033] The color point cloud model generation module is used to generate a color point cloud model of the target area based on the color information of each three-dimensional point in the fused point cloud data, wherein the color information of each three-dimensional point in the fused point cloud data is determined according to the image data of the current frame.

[0034] Optionally, the first pose determination module is specifically used for:

[0035] Based on the RTK data of the current frame, calculate the position data of the UAV in the current frame in the global coordinate system;

[0036] Using the UAV's position data in the current frame as a constraint, and the relative pose relationship between the current frame and historical keyframes as a constraint, the optimal solution of the pre-constructed objective optimization function is calculated based on the point cloud data of the current frame to obtain the first pose of the UAV in the current frame.

[0037] The target optimization function represents the functional relationship between the target error and the first pose of the current frame of the UAV; the target error includes the error of the relative pose relationship between adjacent frames of the UAV, the error of the relative pose relationship between the current frame of the UAV and historical key frames, and the error of the UAV's position data.

[0038] Optionally, the device further includes: a target map data acquisition module, used for:

[0039] Before determining the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and the historical keyframes, if a loop relationship is detected between the current frame and the historical keyframes, the process of acquiring the constructed target map data of the target area is performed; wherein, the target map data includes: the third pose of each historical keyframe, and the third pose of each historical keyframe is: the pose of the drone at the acquisition time corresponding to each historical keyframe.

[0040] Based on the point cloud data of the current frame and the third pose of each historical keyframe, the current frame is matched with each historical keyframe.

[0041] If the current frame matches a historical keyframe, a loopback relationship is determined between the current frame and the historical keyframe.

[0042] Optionally, the device further includes:

[0043] The local map data construction module is used to, after the target map data acquisition module acquires the constructed target map data of the target area, execute the following: if, based on the first pose and the target map data, it is determined that the UAV has moved to a new mapping area in the target area, then, based on the point cloud data of the current frame, construct local map data of the mapping area, wherein the new mapping area is: the unmapped area in the target area other than the mapped area represented by the target map data;

[0044] The map data update module is used to add the local map data to the target map data to obtain the updated target map data of the target area.

[0045] Optionally, the device further includes:

[0046] The time synchronization module is used to perform time synchronization on the point cloud data and RTK data of each frame before determining the first pose of the UAV in the current frame based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and the historical key frames if a loop relationship is detected between the current frame and the historical key frames, so as to obtain the time offset of the point cloud data and the RTK data.

[0047] The data acquisition module is used to determine the RTK data of the point cloud data of the current frame at the same time from each frame of RTK data based on the time offset, and obtain the RTK data of the current frame.

[0048] Optionally, the data acquisition module is specifically used for:

[0049] Using the time offset, the timestamps of each frame of RTK data under the RTK time are converted to timestamps under the radar time.

[0050] From the RTK data of each frame, determine the RTK data with the same timestamp as the point cloud data of the current frame, and obtain the RTK data of the current frame.

[0051] This invention also provides an unmanned aerial vehicle (UAV) system, which includes: radar, camera, and data processor;

[0052] The radar is used to collect point cloud data during the movement of the UAV;

[0053] The camera is used to collect image data during the movement of the drone;

[0054] The data processor is used to execute any of the steps in the color point cloud model generation method described above.

[0055] This invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0056] Memory, used to store computer programs;

[0057] The processor, when executing a program stored in memory, implements any of the steps of the color point cloud model generation method described above.

[0058] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described color point cloud model generation methods.

[0059] This invention also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute any of the above-described color point cloud model generation methods.

[0060] Beneficial effects of the embodiments of the present invention:

[0061] The technical solution provided by this invention acquires observation data collected by a UAV in the current frame of a target area. The observation data includes point cloud data, image data, and RTK data. If a loop relationship is detected between the current frame and historical keyframes, the first pose of the UAV in the current frame is determined based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and historical keyframes. The transformation relationship from the second pose to the first pose of the UAV is determined. Based on the transformation relationship, each three-dimensional point in the point cloud data of adjacent frames is transformed into the point cloud data of the current frame to obtain fused point cloud data. Based on the color information of each three-dimensional point in the fused point cloud data, a color point cloud model of the target area is generated.

[0062] Based on the above processing, the UAV position information provided by RTK data can effectively suppress positioning drift caused by IMU jitter interference, improving the accuracy of 3D modeling. By leveraging the loopback relationship between the current frame and historical keyframes, pose errors accumulated by the SLAM algorithm during long-term, large-scale mapping are effectively corrected, ensuring global consistency of positioning results and providing a fundamental guarantee for high-precision modeling. Converting point cloud data from adjacent frames into the point cloud data of the current frame results in fused point cloud data containing both converted 3D points and the current frame's point cloud data. This retains more 3D points, preserving subtle geometric structures in the target area, ultimately generating a color point cloud model with both high accuracy and realistic colors. In other words, it improves the accuracy and realism of the generated color point cloud model.

[0063] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0065] Figure 1 A flowchart illustrating the first method for generating a color point cloud model provided in this embodiment of the invention;

[0066] Figure 2 A flowchart of the second method for generating a color point cloud model provided in an embodiment of the present invention;

[0067] Figure 3 A schematic diagram illustrating a backend optimization method provided in an embodiment of the present invention;

[0068] Figure 4A schematic diagram of the field of view of a radar and a camera provided for an embodiment of the present invention;

[0069] Figure 5 A flowchart illustrating the third method for generating a color point cloud model provided in this embodiment of the invention;

[0070] Figure 6 This is a structural diagram of a color point cloud model generation device provided in an embodiment of the present invention;

[0071] Figure 7 A structural diagram of an unmanned aerial vehicle (UAV) system provided in an embodiment of the present invention;

[0072] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0074] In related technologies, the LiDAR-SLAM algorithm, which integrates visual sensors, generates a color point cloud model by combining color information from visual images, thereby reducing the accuracy and realism of the generated color point cloud model.

[0075] To address the aforementioned problems, this invention provides a method for generating a color point cloud model, applicable to electronic devices. The electronic device can be a drone. Alternatively, the electronic device can be a server or other device that communicates with the drone.

[0076] The electronic device acquires observation data collected by the UAV in the target area in the current frame. This observation data includes point cloud data, image data, and RTK data. If a loop relationship is detected between the current frame and historical keyframes, the UAV's first pose in the current frame is determined based on the point cloud data, the UAV's position information provided by the RTK data, and the relative pose relationship between the current frame and historical keyframes. This first pose is the optimal solution of a pre-constructed target optimization function. The transformation relationship from the UAV's second pose to the first pose is then determined. The second pose is the pose of the adjacent frames. Based on this transformation relationship, each 3D point in the point cloud data of the adjacent frames is transformed into the point cloud data of the current frame, resulting in fused point cloud data. Based on the color information of each 3D point in the fused point cloud data, a color point cloud model of the target area is generated. The color information of each 3D point in the fused point cloud data is determined based on the image data of the current frame. Providing UAV position information via RTK data effectively suppresses positioning drift caused by IMU jitter interference, improving the accuracy of 3D modeling. Furthermore, by converting the point cloud data of adjacent frames into the point cloud data of the current frame, the resulting fused point cloud data contains the converted 3D points and the point cloud data of the current frame. This allows for the retention of more 3D points, preserving the subtle geometric structures in the target area, and ultimately generating a color point cloud model that combines high precision and realistic colors.

[0077] See Figure 1 The color point cloud model generation method provided in this embodiment of the invention includes the following steps:

[0078] S101: Acquire the observation data collected by the UAV in the target area in the current frame.

[0079] The observation data includes point cloud data, image data, and RTK data.

[0080] S102: If a loop relationship is detected between the current frame and the historical keyframe, the first pose of the drone in the current frame is determined based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and the historical keyframe.

[0081] Here, the first pose is the optimal solution of the pre-constructed objective function.

[0082] S103: Determine the transformation relationship from the second pose to the first pose of the UAV.

[0083] The second pose is the pose of the adjacent frame of the current frame.

[0084] S104: Based on the transformation relationship, each 3D point in the point cloud data of adjacent frames is transformed into the point cloud data of the current frame to obtain fused point cloud data.

[0085] S105: Generate a colored point cloud model of the target area based on the color information of each three-dimensional point in the fused point cloud data.

[0086] In this process, the color information of each three-dimensional point in the fused point cloud data is determined based on the image data of the current frame.

[0087] Based on the color point cloud model generation method provided in this invention, the location information of the UAV is provided by RTK data, which can effectively suppress the positioning drift caused by IMU jitter interference and improve the accuracy of 3D modeling. Furthermore, by converting the point cloud data of adjacent frames into the point cloud data of the current frame, the resulting fused point cloud data includes the converted 3D points and the point cloud data of the current frame, which can retain more 3D points to preserve the subtle geometric structure in the target area, ultimately generating a color point cloud model with both high accuracy and realistic colors.

[0088] For step S101, the target area is the area where the drone is located.

[0089] The drone is equipped with radar (such as lidar), a camera, and a data processor. The data processor includes a Real-Time Kinematic (RTK) module. During the drone's movement, the radar collects point cloud data, and the camera collects image data. The RTK module outputs RTK data containing the drone's position.

[0090] During the UAV's reconnaissance of the target area, the lidar collects point cloud data of the target area, and the camera captures image data of the target area. The RTK module obtains the UAV's RTK data. The RTK data includes the UAV's longitude, latitude, and altitude at various times. Correspondingly, the electronic equipment acquires the aforementioned point cloud data, image data, and RTK data.

[0091] Furthermore, since the frame rates of the LiDAR point cloud data acquisition, the camera image data acquisition, and the RTK data output by the RTK module are all different, time synchronization is required to obtain the same frame of point cloud data, image data, and RTK data. In the LiDAR-SLAM algorithm, time synchronization has already been performed on the LiDAR, camera, and IMU, meaning the point cloud data and image data are already synchronized. Therefore, time synchronization between the RTK module and the LiDAR is required, i.e., time synchronization between the point cloud data and the RTK data.

[0092] In some embodiments, after obtaining point cloud data, image data, and RTK data and before executing step S102, the method may further include the following steps: performing time synchronization on the point cloud data and RTK data of each frame to obtain the time offset of the point cloud data and RTK data; based on the time offset, determining the RTK data of the current frame point cloud data at the same time from each frame of RTK data to obtain the RTK data of the current frame.

[0093] Using the LiDAR-SLAM algorithm, the drone's trajectory can be obtained based on point cloud data acquired by LiDAR. The trajectory includes the drone's positions at multiple trajectory points. RTK data also includes the drone's position at various times.

[0094] Based on the positional deviation between the UAV's position obtained using the LiDAR-SLAM algorithm (which can be called the first position) and the UAV's position in the RTK data (which can be called the second position), the point cloud data and RTK data are synchronized in time to obtain the time offset.

[0095] Furthermore, to improve the accuracy of time synchronization, interpolation algorithms (such as linear interpolation) can be used to interpolate the RTK data, resulting in interpolated RTK data that contains more information about the drone's position at different times.

[0096] For example, at time i, the sum of squares of the trajectory point deviations (i.e., position deviations) of the UAV (denoted as ) This can be expressed as formulas (1) and (2) as follows:

[0097] (1);

[0098] (2);

[0099] Indicates lidar time The first position of the UAV at the i-th time step; Indicates RTK time The second position of the UAV at time j; Indicates RTK time At the j-th moment, Indicates lidar time At the i-th time, Indicates the time offset.

[0100] Theoretically, if the first and second positions of the drone are the same at the same moment, then... The value is 0. Therefore, based on the following formula (3), the positional deviation between the first and second positions of the UAV at the same time is calculated to minimize the deviation at multiple time points. The optimal solution when the minimum value is taken is used to obtain the time offset.

[0101] (3);

[0102] arg represents the calculated time offset (i.e., the optimal solution); arg represents the calculation... The optimal solution; Indicates This is a function that minimizes the independent variable; N represents the number of moments in the point cloud data. (The above...) and Both represent time offsets; This is the time offset that has already been solved; The time offset is unknown.

[0103] In some embodiments, after obtaining the time offset, when obtaining the RTK data of the current frame based on the time offset, the time offset can be used to convert the timestamp of each frame of RTK data under the RTK time to the timestamp under the radar time; from each frame of RTK data, the RTK data with the same timestamp of the point cloud data of the current frame is determined to obtain the RTK data of the current frame.

[0104] For the current frame's point cloud data, on one hand, the timestamp of the current frame's point cloud data can be determined; on the other hand, a time offset can be used to convert the timestamps of each frame's RTK data in RTK time to radar time timestamps. Thus, after obtaining the timestamp of the current frame's point cloud data and converting the timestamps of each frame's RTK data to radar time, the RTK data with the same timestamp as the current frame's point cloud data can be identified and used as the RTK data for the current frame.

[0105] For example, using the solved time offset, the RTK data is converted to the lidar time according to the following formula (4):

[0106] (4);

[0107] Indicates RTK time The time when the j-th moment is converted to the lidar time; Indicates RTK time At the j-th moment; Indicates the time offset.

[0108] Regarding step S102, after obtaining the observation data of the current frame by time synchronization of point cloud data and RTK data, for each historical key frame, based on the data collected in the current frame and the data collected in the historical key frame, it is possible to detect whether there is a loop relationship between the current frame and the historical key frame.

[0109] First, let's introduce the concept of loop relationships.

[0110] The drone collects data synchronously during flight. Therefore, there is a correspondence between the drone's position and the data it collects, or in other words, a correspondence between the drone's position and the data frames it collects. When the drone repeatedly passes through the same location within the target area, it is considered that the data frames repeatedly collected by the drone at that location have a loop relationship. Therefore, the drone positions corresponding to two frames with a loop relationship are the same, or in other words, data frames collected by the drone at the same location but at different times have a loop relationship.

[0111] The implementation method for detecting whether there is a loop relationship between the current frame and historical keyframes can be found in subsequent embodiments, which will not be described in detail here.

[0112] If a loop relationship is detected between the current frame and a certain historical key frame, the first pose of the UAV in the current frame can be calculated based on the point cloud data and RTK data of the current frame (i.e., the current moment) and the relative pose relationship between the current frame and the historical key frame.

[0113] In some embodiments, Figure 1 Based on this, see Figure 2 Step S102 may include the following steps:

[0114] S1021: Calculate the UAV's position data in the global coordinate system based on the RTK data of the current frame;

[0115] S1022: Using the current frame's UAV position data as a constraint, and the relative pose relationship between the current frame and historical keyframes as a constraint, calculate the optimal solution of the pre-constructed objective optimization function based on the point cloud data of the current frame to obtain the first pose of the UAV in the current frame.

[0116] The target optimization function represents the functional relationship between the target error and the first pose of the current frame of the UAV. The target error includes the error of the relative pose relationship between adjacent frames of the UAV, the error of the relative pose relationship between the current frame of the UAV and historical key frames, and the error of the position data of the current frame of the UAV.

[0117] The first pose mentioned above is the pose obtained by solving the objective optimization function when the above error is minimized.

[0118] The global coordinate system is the world coordinate system.

[0119] An adjacent frame can be any frame among the multiple frames preceding and following the current frame.

[0120] Let the pose of the drone in the i-th frame be denoted as It is expressed as the following formula (5):

[0121] (5);

[0122] in, ; Represents the rotation matrix; This represents a special orthogonal group, i.e., the set of rotation matrices; Represents the translation vector; Let x represent the set of translation vectors. The set of pose optimization variables consisting of all keyframes is denoted as x, and x is expressed as follows (6):

[0123] (6);

[0124] This represents the pose of the drone in the Nth frame; N represents the number of keyframes.

[0125] The RTK data of the i-th frame (denoted as...) ) is expressed as the following formula (7):

[0126] (7);

[0127] Represents the longitude of the UAV in the i-th frame; Represents the latitude of the drone in the i-th frame; This represents the altitude of the drone in the i-th frame.

[0128] Transform the RTK data of the i-th frame to the global coordinate system to obtain the three-dimensional coordinates (i.e., position data) of the UAV in the global coordinate system of the i-th frame as shown in the following formula (8):

[0129] (8);

[0130] The extrinsic parameters of the RTK antenna in the UAV's body coordinate system are denoted as... ,and The constraint equations for the RTK in the i-th frame are constructed as follows (9):

[0131] (9);

[0132] This represents the residual of the drone's position data in the i-th frame; This represents the position of the UAV in the i-th frame of the global coordinate system; Represents the rotation matrix; This represents the translation vector.

[0133] Considering the measurement noise of the RKT data, the noise covariance is denoted as... Then its information matrix is: Furthermore, the cost function of the RTK prior is expressed as follows (10):

[0134] (10);

[0135] This indicates the error in the drone's location data; This represents the residual of the position data of the UAV in the i-th frame; for The transpose of the matrix; express The information matrix of noise covariance.

[0136] The aforementioned cost function of RTK prior can effectively suppress the positioning drift caused by IMU jitter interference, providing stable and reliable positioning for the SLAM system and improving the overall positioning accuracy.

[0137] Furthermore, the cost function of adjacent frames can be expressed as the following formula (11):

[0138] (11);

[0139] The error representing the relative pose relationship between adjacent frames of the UAV; The observations represent the relative pose relationship between the UAV in frame i and frame i+1. for The transpose of the matrix; express The information matrix of noise covariance.

[0140] The objective optimization function is to minimize the sum of the cost functions of adjacent frames and the RTK prior cost function, with the pose of the UAV in the current frame as the independent variable. In other words, it calculates the optimal solution that minimizes the sum of the cost functions of adjacent frames and the RTK prior cost function, and obtains the pose of the UAV in the current frame.

[0141] Based on the above processing, the precise location information provided by RTK data effectively suppresses the positioning drift caused by IMU jitter interference, providing stable and reliable positioning for the SLAM system, improving the overall positioning accuracy, and thus improving the accuracy of the generated color point cloud model.

[0142] The following explains how to detect whether there is a loop relationship between the current frame and historical keyframes.

[0143] In some embodiments, before step S1022, the method may further include the following steps: acquiring constructed target map data of the target area. The target map data includes: the third pose of each historical keyframe, where the third pose of each historical keyframe is the pose of the UAV at the acquisition time corresponding to each historical keyframe. Based on the point cloud data of the current frame and the third poses of each historical keyframe, the current frame is matched with each keyframe;

[0144] Accordingly, step S1022 may include the following steps: If the current frame matches a key frame, determine that the key frame is a historical loopback frame. Using the current frame's UAV position data as a constraint, and the relative pose relationship between the current frame and the historical loopback frame as a constraint, calculate the optimal solution of the pre-constructed target optimization function based on the point cloud data of the current frame to obtain the first pose of the UAV in the current frame. The target error also includes the error in the relative pose relationship between the current frame and the loopback frame.

[0145] By embedding a loop closure detection process into the laser-vision SLAM system, historical loop closure frames can be identified by analyzing the relationship between the current position of the UAV and the modeled map.

[0146] Acquire the constructed target map data. The target map data contains the third pose of the UAV for keyframes of identified historical moments.

[0147] Based on the point cloud data of the current frame, the location of the UAV in the current frame is extracted in real time. For each keyframe, the third pose of the UAV in that keyframe is obtained, which gives the UAV's location. If the location of the UAV in the current frame is the same as the location of a UAV in a keyframe, it indicates that the UAV has scanned a covered, repetitive area. This triggers a loopback optimization process, determining that the current frame matches the keyframe and that the keyframe is a historical loopback frame.

[0148] Furthermore, the relative pose relationship between the current frame and the historical loop frames is used as a constraint, as is the position data of the UAV in the current frame. Based on the point cloud data of the current frame, the optimal solution of the pre-constructed objective optimization function is calculated to obtain the first pose of the UAV in the current frame.

[0149] For example, a loop closure is detected between frame i and frame j. By matching the point cloud data of frame i and frame j (historical loop closure frames), the relative pose relationship between frame i and frame j is obtained (denoted as ). The following formula (12) is shown:

[0150] (12);

[0151] Represents the rotation matrix between the i-th and j-th frames; Represents the translation vectors of the i-th and j-th frames; This represents a special Euclidean group.

[0152] Based on the UAV pose in the historical loopback frames, the relative pose relationship between the predicted i-th frame and the j-th frame can be calculated (denoted as ). ) is expressed as the following formula (13):

[0153] (13);

[0154] This represents the pose of the drone in the i-th frame; This represents the pose of the UAV in frame j.

[0155] Correspondingly, the residuals of the closure constraints in the i-th and j-th frames (denoted as...) ) is expressed as the following formula (14):

[0156] (14).

[0157] The measurement noise covariance of the relative pose relationship between frame i and frame j is denoted as . Then its information matrix is ​​represented as: .

[0158] Accordingly, the cost function for loop closure detection is expressed as follows (15):

[0159] (15);

[0160] This represents the error in the relative pose relationship between the i-th frame and the j-th frame (historical loop frame).

[0161] The objective optimization function is to minimize the sum of the cost functions of adjacent frames, the cost function of RTK prior, and the cost function of loop closure detection. The objective optimization function is expressed as the following formula (16). The optimal solution of formula (16) is calculated to obtain the first pose of the UAV in the current frame.

[0162] (16);

[0163] This represents the minimum function with x as the independent variable; x represents the set of optimization variables; the set of optimization variables includes the first pose of the UAV in the current frame; This represents the set of point cloud data collected by the drone; The observations represent the relative pose relationship between the UAV in frame i and frame i+1. for The transpose of the matrix; express The information matrix of noise covariance; This represents the set of location data for the drone; This represents the residual of the drone's position data in the i-th frame; for The transpose of the matrix; express The information matrix of noise covariance; Represents the set of historical loop frames; This represents the relative pose relationship between the i-th frame and the j-th frame of the UAV's historical loopback. for The transpose of the matrix; express The information matrix of noise covariance.

[0164] Based on the above processing, the loop closure detection process effectively corrects the pose error accumulated by the SLAM algorithm during long-term, large-scale mapping, ensuring the global consistency of the positioning results and providing a basic guarantee for high-precision modeling.

[0165] In some embodiments, if the current frame does not match any of the key frames in the target map data, it indicates that the UAV has moved to a new mapping area. There is no need to perform a loop closure detection process. Instead, the optimal solution that uses the minimum sum of formula (10) and formula (11) is calculated to obtain the first pose of the UAV in the current frame. The new mapping area is the unmapped area in the target area other than the mapped area represented by the target map data.

[0166] See Figure 3 , Figure 3 This is a schematic diagram illustrating the principle of backend optimization provided in an embodiment of the present invention.

[0167] Figure 3 The circles on the zigzag line represent multiple trajectory points of the drone. Among them, T... i T represents the trajectory point in the i-th frame, or the pose at time i. i+1 T represents the trajectory point in frame i+1, and so on. i+5 This represents the trajectory point in frame i+5. This represents the relative pose of the drone in frame i and frame i+1. This represents the relative pose of the drone in frame i+1 and frame i+2, and so on. This represents the relative pose relationship between the drone in frame i+4 and frame i+5.

[0168] Tj T represents the trajectory point in the j-th frame. j+1 This represents the trajectory point in frame j+1, and so on, T j+5 This represents the trajectory point in frame j+5. This represents the relative pose of the drone in frame j and frame j+1. This represents the relative pose of the drone in frame j+1 and frame j+2, and so on. This represents the relative pose relationship between the UAV in frame j+4 and frame j+5. There are adjacent frame constraints between the trajectory points of adjacent frames.

[0169] This represents the Global Navigation Satellite System (GNSS) observation information of the i-th frame (i.e., the RTK data of the i-th frame), or the GNSS observation information at time i. This represents the GNSS observation information for the (i+1)th frame, and so on. This represents the GNSS observation information for the (i+1)th frame. The GNSS observation information for each frame provides prior constraints for RTK.

[0170] During the drone's movement, each frame is matched with previously acquired keyframes to obtain the matching historical loopback frames. A loopback constraint exists between this frame and the historical loopback frames. For example, a loopback constraint exists between frame i+1 and frame j+4. There is a closure constraint between frame i+4 and frame j+1. .

[0171] The above adjacent frame constraints, loop closure constraints, and RTK prior constraints are used for optimization to obtain the UAV pose at each time step.

[0172] For steps S103 and S104, after obtaining the first pose of the current frame and the second pose of the adjacent frame, the relative pose relationship between the second pose and the first pose can be obtained based on the first pose and the second pose, that is, the transformation relationship including the rotation matrix and the translation vector.

[0173] Using the above transformation relationship, each three-dimensional point in the point cloud data of adjacent frames is rotated and translated to obtain the three-dimensional point corresponding to that point in the current frame. The transformed three-dimensional point is then merged with the point cloud data of the current frame to obtain fused point cloud data.

[0174] In step S105, for each 3D point in the fused point cloud data, based on the camera's intrinsic parameters and the extrinsic parameters of the LiDAR and camera (i.e., the transformation relationship between the camera coordinate system and the LiDAR coordinate system), the 3D point is projected onto the current frame image data to obtain the image projection coordinates of the 3D point in the current frame image data. Using the image projection coordinates, a unique pixel can be determined in the current frame image data.

[0175] Accordingly, the pixel value of the pixel at the image projection coordinates is obtained as the color information of the 3D point. Based on the color information of each 3D point, the fused point cloud data is colored to obtain a colored point cloud model of the target region. Alternatively, to further improve the geometric accuracy and visual consistency of the point cloud model, the colored point cloud model can be deduplicated and smoothed (e.g., filtered) to obtain the final colored point cloud model of the target region.

[0176] See Figure 4 In related technologies, only point cloud data of the shared field of view area of ​​the LiDAR and camera is retained, and the retained power data is colored to obtain a colored point cloud model. From Figure 4 As can be seen, the shared field of view between the effective area of ​​the lidar and the effective area of ​​the camera is small, and fewer 3D points can be retained, resulting in sparse point clouds in the modeling, which in turn leads to low precision and accuracy of the generated color point cloud model.

[0177] In the method provided by this embodiment of the invention, point cloud data from adjacent frames is converted into point cloud data of the current frame. The resulting fused point cloud data includes the converted 3D points and the point cloud data of the current frame, thus retaining more 3D points. Figure 4 In the above-mentioned adjacent frame point cloud data, the point cloud data is the point cloud at time t. Figure 4 The area marked with a slash represents the point cloud at time t within the camera's shared field of view at time t+1. After converting the point cloud in this area to the current frame, the color information of each laser point within this area can be determined based on the image data at time t+1. These laser points, containing color information, are then fused into the point cloud data of the current frame, resulting in a point cloud with more 3D points. This addresses the problem of sparse point clouds in modeling due to the high speed of UAV movement and the limited number of 3D points in a single frame, preserving the subtle geometric structures in the target area and ultimately generating a color point cloud model with both high accuracy and realistic colors.

[0178] In some embodiments, after obtaining the target map data, the method may further include the following steps:

[0179] If, based on the first pose and target map data, it is determined that the UAV has moved to a new mapping area in the target region, then, based on the point cloud data of the current frame, local map data of the mapping area is constructed; the local map data is added to the target map data to obtain the updated target map data of the target region. The new mapping area is: the unmapped area in the target region other than the mapped area represented by the target map data.

[0180] If the current frame does not match any of the keyframes, it indicates that the UAV has moved to a new mapping area, and map data for that area has not yet been generated. Accordingly, based on the feature information of each 3D point in the point cloud data of the current frame, local map data within a preset neighborhood of the UAV's location is constructed. Then, the newly constructed local map data is added to the target map data to obtain the updated target map data. The preset neighborhood range can be set according to requirements; for example, it can be the scanning range of the UAV's LiDAR.

[0181] Based on the above processing, a dynamic local map management strategy is adopted. Taking the real-time local positioning results of the UAV (optimized pose after fusion of RTK and SLAM) as the core, the local map region and all keyframe map regions are dynamically divided. This can significantly reduce the computational overhead of the system and improve the algorithm's capabilities in large-scale aerial mapping scenarios.

[0182] See Figure 5 A LiDAR-SLAM system consists of two inputs: a LiDAR and a camera. The LiDAR acquires raw point cloud data, while the camera acquires image data. Based on the point-to-plane constraints provided by the raw point cloud and the image constraints provided by the image data, optimization is performed to generate a local voxel map.

[0183] By searching keyframes using a local voxel map and combining them with the original point cloud for LiDAR-SLAM, the UAV's pose can be output. Based on the UAV pose output by the LiDAR-SLAM algorithm, and the original point cloud, combined with RTK data and loop closure detection, backend optimization is performed to finally obtain the UAV pose and a color point cloud model. Furthermore, a new local voxel map can be built based on the UAV pose from the keyframes and the original point cloud.

[0184] and Figure 1 Corresponding to the method embodiments, this invention also provides a color point cloud model generation device, see [link to relevant documentation]. Figure 6 The device includes:

[0185] The observation data acquisition module 601 is used to acquire observation data collected by the UAV in the current frame in the target area; wherein, the observation data includes: point cloud data, image data, and RTK data;

[0186] The first pose determination module 602 is used to determine the first pose of the UAV in the current frame if a loop relationship is detected between the current frame and the historical key frame, based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and the historical key frame; wherein, the first pose is the optimal solution of a pre-constructed target optimization function.

[0187] The transformation relationship determination module 603 is used to determine the transformation relationship from the second pose of the UAV to the first pose; wherein the second pose is the pose of the adjacent frame of the current frame;

[0188] The fused point cloud data generation module 604 is used to convert each three-dimensional point in the point cloud data of adjacent frames to the point cloud data of the current frame based on the conversion relationship, so as to obtain fused point cloud data.

[0189] The color point cloud model generation module 605 is used to generate a color point cloud model of the target area based on the color information of each three-dimensional point in the fused point cloud data, wherein the color information of each three-dimensional point in the fused point cloud data is determined according to the image data of the current frame.

[0190] Optionally, the first pose determination module 602 is specifically used for:

[0191] Based on the RTK data of the current frame, calculate the position data of the UAV in the current frame in the global coordinate system;

[0192] Using the UAV's position data in the current frame as a constraint, and the relative pose relationship between the current frame and historical keyframes as a constraint, the optimal solution of the pre-constructed objective optimization function is calculated based on the point cloud data of the current frame to obtain the first pose of the UAV in the current frame.

[0193] The target optimization function represents the functional relationship between the target error and the first pose of the current frame of the UAV; the target error includes the error of the relative pose relationship between adjacent frames of the UAV, the error of the relative pose relationship between the current frame of the UAV and historical key frames, and the error of the UAV's position data.

[0194] Optionally, the device further includes: a target map data acquisition module, used for:

[0195] Before the first pose determination module 602 determines the first pose of the UAV in the current frame based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and the historical keyframes, it performs the acquisition of the constructed target map data of the target area. The target map data includes the third pose of each historical keyframe, which is the pose of the UAV at the acquisition time corresponding to each historical keyframe.

[0196] The matching module is used to match the current frame with each historical keyframe based on the point cloud data of the current frame and the third pose of each historical keyframe.

[0197] If the current frame matches a historical keyframe, a loopback relationship is determined between the current frame and the historical keyframe.

[0198] Optionally, the device further includes:

[0199] The local map data construction module is used to, after the target map data acquisition module acquires the constructed target map data of the target area, execute the following: if, based on the first pose and the target map data, it is determined that the UAV has moved to a new mapping area in the target area, then, based on the point cloud data of the current frame, construct local map data of the mapping area, wherein the new mapping area is: the unmapped area in the target area other than the mapped area represented by the target map data;

[0200] The map data update module is used to add the local map data to the target map data to obtain the updated target map data of the target area.

[0201] Optionally, the device further includes:

[0202] The time synchronization module is used to perform time synchronization of the point cloud data and RTK data of each frame before the first pose determination module 602 performs the following steps: if a loop relationship is detected between the current frame and the historical key frame, based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and the historical key frame, to determine the first pose of the UAV in the current frame.

[0203] The data acquisition module is used to determine the RTK data of the point cloud data of the current frame at the same time from each frame of RTK data based on the time offset, and obtain the RTK data of the current frame.

[0204] Optionally, the data acquisition module is specifically used for:

[0205] Using the time offset, the timestamps of each frame of RTK data under the RTK time are converted to timestamps under the radar time.

[0206] From the RTK data of each frame, determine the RTK data with the same timestamp as the point cloud data of the current frame, and obtain the RTK data of the current frame.

[0207] Based on the color point cloud model generation device provided in this embodiment of the invention, the location information of the UAV is provided by RTK data, which can effectively suppress the positioning drift caused by IMU jitter interference and improve the accuracy of 3D modeling. Furthermore, by converting the point cloud data of adjacent frames into the point cloud data of the current frame, the resulting fused point cloud data includes the converted 3D points and the point cloud data of the current frame, which can retain more 3D points to preserve the subtle geometric structure in the target area, ultimately generating a color point cloud model with both high accuracy and realistic colors.

[0208] See Figure 7 The present invention also provides an unmanned aerial vehicle (UAV) system, which includes: radar 701, camera 702 and data processor 703;

[0209] The radar 701 is used to collect point cloud data during the movement of the UAV;

[0210] The camera 702 is used to collect image data during the movement of the drone;

[0211] The data processor 703 is used to execute any of the steps of the color point cloud model generation method described above.

[0212] Based on the UAV system provided in this embodiment of the invention, the location information of the UAV is provided through RTK data, which can effectively suppress the positioning drift caused by IMU jitter interference and improve the accuracy of 3D modeling. Furthermore, by converting the point cloud data of adjacent frames into the point cloud data of the current frame, the resulting fused point cloud data includes the converted 3D points and the point cloud data of the current frame, which can retain more 3D points to preserve the subtle geometric structure in the target area, ultimately generating a color point cloud model with both high accuracy and realistic colors.

[0213] This invention also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0214] Memory 803 is used to store computer programs;

[0215] When processor 801 executes a program stored in memory 803, it performs the following steps:

[0216] Acquire observation data collected by the UAV in the current frame in the target area; wherein, the observation data includes: point cloud data, image data, and RTK data;

[0217] If a loop relationship is detected between the current frame and historical keyframes, the first pose of the UAV in the current frame is determined based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and historical keyframes of the UAV; wherein, the first pose is the optimal solution of the pre-constructed target optimization function.

[0218] Determine the transformation relationship from the second pose of the UAV to the first pose; wherein the second pose is the pose of the adjacent frame of the current frame;

[0219] Based on the transformation relationship, each 3D point in the point cloud data of adjacent frames is transformed into the point cloud data of the current frame to obtain fused point cloud data;

[0220] Based on the color information of each three-dimensional point in the fused point cloud data, a color point cloud model of the target region is generated, wherein the color information of each three-dimensional point in the fused point cloud data is determined according to the image data of the current frame.

[0221] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0222] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0223] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0224] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0225] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described color point cloud model generation methods.

[0226] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the color point cloud model generation methods described above.

[0227] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0228] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0229] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for devices, unmanned aerial vehicle systems, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0230] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for generating a color point cloud model, characterized in that, The method includes: Acquire observation data collected by the UAV in the current frame in the target area; wherein, the observation data includes: point cloud data, image data, and RTK data; If a loop relationship is detected between the current frame and historical keyframes, the first pose of the UAV in the current frame is determined based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and historical keyframes of the UAV; wherein, the first pose is the optimal solution of the pre-constructed target optimization function. Determine the transformation relationship from the second pose of the UAV to the first pose; wherein the second pose is the pose of the adjacent frame of the current frame; Based on the aforementioned transformation relationship, each 3D point in the point cloud data of adjacent frames is transformed into the point cloud data of the current frame to obtain fused point cloud data; Based on the color information of each three-dimensional point in the fused point cloud data, a color point cloud model of the target region is generated, wherein the color information of each three-dimensional point in the fused point cloud data is determined according to the image data of the current frame.

2. The method according to claim 1, characterized in that, The determination of the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and historical keyframes includes: Based on the RTK data of the current frame, calculate the position information of the UAV in the current frame in the global coordinate system; Using the UAV's position information in the current frame as a constraint, and the relative pose relationship between the current frame and historical keyframes as a constraint, the optimal solution of the pre-constructed objective optimization function is calculated based on the point cloud data of the current frame to obtain the first pose of the UAV in the current frame. The target optimization function represents the functional relationship between the target error and the first pose of the current frame of the UAV; the target error includes the error of the relative pose relationship between adjacent frames of the UAV, the error of the relative pose relationship between the current frame of the UAV and historical key frames, and the error of the UAV's position information.

3. The method according to claim 1, characterized in that, Before determining the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and the historical keyframes, if a loop closure relationship is detected between the current frame and the historical keyframes, the method further includes: Acquire the constructed target map data of the target area; wherein, the target map data includes: the third pose of each historical key frame, and the third pose of each historical key frame is: the pose of the UAV at the acquisition time corresponding to each historical key frame. Based on the point cloud data of the current frame and the third pose of each historical keyframe, the current frame is matched with each historical keyframe. If the current frame matches a historical keyframe, a loopback relationship is determined between the current frame and the historical keyframe.

4. The method according to claim 3, characterized in that, After acquiring the constructed target map data for the target area, the method further includes: If, based on the first pose and the target map data, it is determined that the UAV has moved to a new surveying area in the target region, then, based on the point cloud data of the current frame, local map data of the surveying area is constructed. The new surveying area is: the unsurveyed area in the target region other than the surveyed area represented by the target map data. The local map data is added to the target map data to obtain the updated target map data for the target area.

5. The method according to claim 1, characterized in that, Before determining the first pose of the drone in the current frame based on the point cloud data of the current frame, the position information of the drone in the current frame provided by the RTK data, and the relative pose relationship between the drone in the current frame and the historical keyframes, if a loop closure relationship is detected between the current frame and the historical keyframes, the method further includes: Time synchronization is performed on the point cloud data and RTK data of each frame to obtain the time offset of the point cloud data and RTK data; Based on the time offset, the RTK data of the current frame's point cloud data at the same time is determined from each frame's RTK data, thus obtaining the RTK data of the current frame.

6. The method according to claim 5, characterized in that, The step of determining the RTK data of the current frame point cloud data at the same moment from each frame of RTK data based on the time offset, to obtain the current frame RTK data, includes: Using the time offset, the timestamps of each frame of RTK data under the RTK time are converted to timestamps under the radar time. From the RTK data of each frame, determine the RTK data with the same timestamp as the point cloud data of the current frame, and obtain the RTK data of the current frame.

7. A color point cloud model generation device, characterized in that, The device includes: The observation data acquisition module is used to acquire observation data collected by the UAV in the target area in the current frame; wherein, the observation data includes: point cloud data, image data, and RTK data; The first pose determination module is used to determine the first pose of the UAV in the current frame if a loop relationship is detected between the current frame and the historical key frame, based on the point cloud data of the current frame, the position information of the UAV in the current frame provided by the RTK data, and the relative pose relationship between the current frame and the historical key frame. The first pose is the optimal solution of a pre-constructed target optimization function. A transformation relationship determination module is used to determine the transformation relationship from the second pose of the UAV to the first pose; wherein the second pose is the pose of the adjacent frame of the current frame; The fused point cloud data generation module is used to convert each three-dimensional point in the point cloud data of adjacent frames to the point cloud data of the current frame based on the conversion relationship, so as to obtain fused point cloud data. The color point cloud model generation module is used to generate a color point cloud model of the target area based on the color information of each three-dimensional point in the fused point cloud data, wherein the color information of each three-dimensional point in the fused point cloud data is determined according to the image data of the current frame.

8. An unmanned aerial vehicle (UAV) system, characterized in that, The unmanned aerial vehicle system includes: radar, camera, and data processor; The radar is used to collect point cloud data during the movement of the UAV; The camera is used to collect image data during the movement of the drone; The data processor is configured to execute the method according to any one of claims 1-6.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images