Surveying and mapping system and method based on multi-source data fusion

Abstract

The invention relates to the technical field of surveying and mapping, and discloses a surveying and mapping system and method based on multi-source data fusion, and the method comprises the steps: collecting the image data of a to-be-surveyed area, collecting the illumination intensity of the to-be-surveyed area and the inertial data of an unmanned plane in real time, and judging whether to start LiDAR ranging; when it is judged that starting is conducted, image data and point cloud data are obtained, and the point cloud data are processed according to the timestamp of the image data; processing the image data to determine a high-contrast area, and converting image coordinates in the image data into global coordinates; denoising the point cloud data after time synchronization to generate a digital elevation model; and aligning the high-contrast area and fusing other areas in the image data according to the digital elevation model to obtain a surveying and mapping model. By fusing the optical image and the LiDAR data, the limitation of a single surveying and mapping means is overcome, and the precision and reliability of the surveying and mapping model are improved.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping technology, and in particular, to a surveying and mapping system and method based on multi-source data fusion. Background Art

[0002] Multi-source data fusion refers to integrating and integrating information from different data sources to obtain more comprehensive, accurate, and high-quality data results. In the field of land surveying and mapping, multi-source data fusion refers to fusing geospatial data from multiple measurement and observation means to generate more comprehensive and complete geographical information. Multi-source data fusion has important application value in land surveying and mapping. It can be used in fields such as cartography, geographic information system construction, resource management, and environmental monitoring. By fusing information from different data sources, it can provide more comprehensive, accurate, and reliable geographical information, providing a scientific basis for decision-making in land planning, urban construction, natural resource management, etc.

[0003] Traditional surveying and mapping methods mainly rely on a single data source, such as optical images or lidar data. Although these methods have their own advantages, they still have certain limitations in complex terrains or special environments. Optical image surveying and mapping rely on high-resolution cameras to obtain surface information and extract three-dimensional information through image processing techniques. However, when the lighting conditions are insufficient (such as at night or in shadow areas) or the attitude of the unmanned aerial vehicle changes greatly (such as vibration under the influence of wind), the quality of optical images may decline, resulting in limited surveying and mapping accuracy. In addition, it is difficult to directly obtain the precise elevation information of the ground surface from image data, especially in cases of vegetation coverage or ground object occlusion, which affects the accuracy of surveying and mapping.

[0004] Therefore, it is necessary to design a surveying and mapping system and method based on multi-source data fusion to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a surveying and mapping system and method based on multi-source data fusion, aiming to solve the problems of limited surveying and mapping accuracy and poor reliability of surveying and mapping results with a single data source.

[0006] On the one hand, the present invention proposes a surveying and mapping method based on multi-source data fusion, including:

[0007] Collecting image data of the area to be surveyed along a preset path by an image acquisition device carried by an unmanned aerial vehicle, and simultaneously collecting the illumination intensity of the area to be surveyed and the inertial data of the unmanned aerial vehicle, and determining whether to turn on the LiDAR ranging according to the illumination intensity and inertial data;

[0008] When it is determined to turn on the LiDAR ranging, obtaining the image data and point cloud data, and processing the point cloud data according to the time stamp of the image data to synchronize the data time;

[0009] Process the image data based on edge detection to determine the high-contrast region, and convert the image coordinates in the image data into global coordinates according to the high-contrast region;

[0010] Denoise the point cloud data after synchronization time based on the RANSAC algorithm, and generate a digital elevation model according to the denoised point cloud data;

[0011] Determine the high-contrast region in the digital elevation model, align the high-contrast region, and fuse the remaining regions in the image data according to the digital elevation model to obtain a mapping model.

[0012] Further, when judging whether to turn on LiDAR ranging according to the illumination intensity and inertial data, it includes:

[0013] Compare the illumination intensity with the lowest light intensity threshold, and compare the inertial data with the highest inertial threshold, and judge whether to turn on LiDAR ranging according to the comparison results;

[0014] When the illumination intensity is less than the lowest light intensity threshold, or, the inertial data is greater than or equal to the highest inertial threshold, it is determined to turn on LiDAR ranging;

[0015] When the illumination intensity is greater than or equal to the lowest light intensity threshold, and, the inertial data is less than the highest inertial threshold, it is determined not to turn on LiDAR ranging.

[0016] Further, when processing the point cloud data according to the timestamp of the image data and synchronizing the data time, it includes:

[0017] Collect the timestamp of the image data, and filter out the point cloud data at two moments close to the timestamp in the point cloud data;

[0018] Calculate the point cloud data at the timestamp based on linear interpolation;

[0019]

[0020] Wherein, Pt represents the point cloud data at timestamp t, Pt1 represents the point cloud data at time t1 close to the timestamp, Pt2 represents the point cloud data at time t2 close to the timestamp, and t1, t2 represent two moments close to the timestamp.

[0021] Further, when processing the image data based on edge detection to determine the high-contrast region, it includes:

[0022] Perform Gaussian filtering on the image data, and adjust the standard deviation of the Gaussian kernel to smooth the image;

[0023] Calculate the gradients of the image data in the horizontal and vertical directions, synthesize the total gradient G according to the horizontal and vertical gradients, and obtain the gradient direction;

[0024] Based on non-maximum suppression, check adjacent pixels according to the gradient direction, only retain the pixel points with the local maximum gradient value, and remove the remaining pixel points;

[0025] Set a high threshold Th and a low threshold Tl. When the total gradient G of the edge pixel is greater than the high threshold Th, define the edge pixel as a strong edge and retain it. When Th≥G≥Tl, define the edge pixel as a possible edge and retain it. When the total gradient G of the edge pixel is less than the low threshold Tl, define the edge pixel as a weak edge and remove it;

[0026] Perform edge connection according to the retained points, trace the possible edges. If the possible edge is connected to a strong edge, retain it, otherwise remove it, and connect the retained points to obtain the high-contrast region.

[0027] Further, when connecting the retained points to obtain the high-contrast region, it further includes:

[0028] Calculate the distance between any two of the retained points, connect the retained points according to the shortest distance. When there is only one area in the connected image, take this area as the high-contrast region;

[0029] When there are at least two areas in the connected image, obtain the area of each area, and determine the area corresponding to the largest area as the high-contrast region.

[0030] Further, when converting the image coordinates in the image data to global coordinates according to the high-contrast region, it includes:

[0031] Convert the image coordinates in the image data to camera coordinates;

[0032]

[0033]

[0034]

[0035] Among them, Xc, Yc, Zc represent the three-dimensional coordinates in the camera coordinate system, Sx represents the physical length corresponding to the pixel length, Sy represents the physical length corresponding to the pixel width, (u0, v0) represents the center pixel coordinates in the image data, (u, v) represents the position coordinates of the point to be converted, represents the camera focal length;

[0036] Determine the UAV position coordinates TGNSS based on UAV positioning;

[0037] Obtain the global coordinates according to the camera coordinates and the drone position coordinates TGNSS;

[0038]

[0039] Among them, RIMU represents the UAV rotation matrix.

[0040] Furthermore, when denoising the point cloud data after the synchronization time based on the RANSAC algorithm, it includes:

[0041] Randomly select three points in the point cloud data after the synchronization time to form a three-point plane;

[0042]

[0043]

[0044] Calculating the distances between all points in the point cloud data after the synchronization time and the three-point plane;

[0045] When the distance exceeds the distance threshold, the point cloud data is discarded.

[0046] Furthermore, when generating a digital elevation model based on the denoised point cloud data, it includes:

[0047] Constructing the denoised point cloud data into a non-overlapping triangle network;

[0048] Compute triangle interpolation:

[0049]

[0050]

[0051] Among them, SZ represents the terrain elevation calculated by interpolation, SZi represents the elevation of the known point cloud of the triangle, wi represents the i-th distance weight, and di represents the distance between the i-th point and the interpolation position.

[0052] Furthermore, when the rest of the regions in the image data are fused according to the digital elevation model to obtain a surveying and mapping model, the method includes:

[0053]

[0054] Among them, M represents the surveying and mapping model, DEM represents the digital elevation model, I represents the image data, and λ represents the fusion coefficient.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows: By carrying an image acquisition device on a drone, image data of the area to be surveyed and mapped is collected along a preset path, and combined with the real-time monitoring of the light intensity and the inertial data of the drone, it is judged whether to activate LiDAR ranging, so as to enhance the surveying and mapping accuracy and data integrity under complex lighting conditions or when the attitude of the drone changes greatly. The image data and the point cloud data are synchronized using timestamps to achieve precise matching of multi-source data and ensure the consistency of the surveying and mapping information. Edge detection technology is used to identify high-contrast regions in the image, and based on these regions, the conversion of image coordinates to global coordinates is performed to improve the spatial accuracy of geographical information. For the point cloud data, the RANSAC algorithm is used for denoising processing to improve the quality of the point cloud data, and on this basis, a digital elevation model is generated to achieve high-precision topographic surveying and mapping. By locating high-contrast regions in the digital elevation model and aligning and fusing them, the accuracy and reliability of the surveying and mapping model are effectively improved. Especially under conditions of vegetation coverage, ground object occlusion, or poor lighting, the integrity and accuracy of the surveying and mapping data can still be guaranteed.

[0056] On the other hand, the present application also provides a surveying and mapping system based on multi-source data fusion for applying the above-mentioned surveying and mapping method based on multi-source data fusion, including:

[0057] A drone, equipped with an image acquisition device, a LiDAR sensor, an inertial sensor, and a light sensor, and the image acquisition device includes an optical camera;

[0058] An acquisition unit, configured to collect the light intensity of the area to be surveyed and mapped and the inertial data of the drone in real time, and judge whether to activate LiDAR ranging according to the light intensity and the inertial data;

[0059] A processing unit, configured to when the acquisition unit determines to activate the LiDAR ranging, the processing unit acquires the image data and the point cloud data, and processes the point cloud data according to the timestamp of the image data to synchronize the data time;

[0060] The processing unit is further configured to process the image data based on edge detection to determine high-contrast regions, and convert the image coordinates in the image data into global coordinates according to the high-contrast regions;

[0061] The processing unit is further configured to denoise the point cloud data after synchronization time based on the RANSAC algorithm, and generate a digital elevation model according to the denoised point cloud data;

[0062] A fusion unit, configured to determine the high-contrast regions in the digital elevation model, align the high-contrast regions, and fuse the remaining regions in the image data according to the digital elevation model to obtain a surveying and mapping model.

[0063] It is understandable that the above-mentioned surveying and mapping system and method based on multi-source data fusion have the same beneficial effects, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered as a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0065] Figure 1 is a flowchart of a surveying and mapping method based on multi-source data fusion provided by an embodiment of the present invention;

[0066] Figure 2 is a structural block diagram of a surveying and mapping system based on multi-source data fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art. It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. Hereinafter, the present invention will be described in detail with reference to the drawings and in conjunction with the embodiments.

[0068] In some embodiments of the present application, referring to Figure 1 as shown, a surveying and mapping method based on multi-source data fusion includes:

[0069] S100: Based on the unmanned aerial vehicle (UAV) carrying an image acquisition device, collect image data of the area to be surveyed along a preset path, and collect the illumination intensity of the area to be surveyed and the inertial data of the UAV in real time. Determine whether to turn on the LiDAR ranging according to the illumination intensity and the inertial data.

[0070] S200: When it is determined to turn on the LiDAR ranging, obtain the image data and the point cloud data, process the point cloud data according to the timestamp of the image data, and synchronize the data time.

[0071] S300: Process the image data based on edge detection to determine the high-contrast area, and convert the image coordinates in the image data into global coordinates according to the high-contrast area.

[0072] S400: Denoise the point cloud data after synchronizing the time based on the RANSAC algorithm, and generate a digital elevation model according to the denoised point cloud data.

[0073] S500: Determine the high-contrast regions in the digital elevation model, align the high-contrast regions, and fuse the remaining regions in the image data according to the digital elevation model to obtain a mapping model.

[0074] Specifically, in S100, the image acquisition device carried by the UAV is used to collect the image data of the area to be mapped according to the preset path, and at the same time, the light intensity and the inertial data of the UAV are obtained in real time. The light intensity is used to evaluate the imaging quality, and the inertial data reflects the attitude stability of the UAV. When the light is insufficient or the UAV attitude fluctuates greatly, simply relying on optical images may lead to a decrease in mapping accuracy. These data are used to judge whether to turn on the LiDAR ranging to enhance the stability and integrity of the mapping data. In S200, if it is determined to turn on the LiDAR ranging, the UAV collects image data and point cloud data at the same time, and processes the point cloud data based on the time stamp of the image data to achieve data time synchronization. Since the acquisition frequencies of optical images and LiDAR data are different, direct fusion may lead to spatial misalignment. Therefore, time synchronization processing can ensure the matching accuracy of the two data sources. In S300, edge detection technology is used to process the image data to identify high-contrast regions (such as building edges, road signs, etc.). These regions often have clear contour information, which is convenient for coordinate conversion. Through the high-contrast regions, the image coordinates in the image are converted into global coordinates, so that the image information can accurately correspond to the actual geographical coordinates, thereby improving the mapping accuracy. In S400, before data fusion, the point cloud data collected by LiDAR contains noise (such as vegetation occlusion, equipment errors, etc.). The RANSAC (Random Sample Consensus) algorithm is used for denoising processing to remove outliers and improve the data quality. Subsequently, a digital elevation model is generated based on the denoised point cloud data, and this model can provide the elevation information of the ground surface. In S500, the digital elevation model is used to locate the high-contrast regions, and spatial matching of different data sources is achieved through coordinate alignment. The remaining regions of the image data are fused with the digital elevation model to finally generate a complete mapping model. This fusion process combines the texture information of the image and the three-dimensional elevation information of the point cloud, so that the mapping model not only has high-resolution image details but also contains accurate terrain information, thereby improving the authenticity and accuracy of the mapping results.

[0075] It can be understood that high-precision mapping is achieved based on the intelligent fusion of optical images and LiDAR point cloud data. By analyzing the illumination intensity and inertial data in real time, it adaptively determines whether to turn on LiDAR ranging, and can dynamically adjust the data acquisition method under complex illumination conditions and attitude change environments, improving the applicability and stability of mapping. The timestamp synchronization technology is used to accurately match the image data and the point cloud data in the time dimension, avoiding data misalignment problems. The edge detection technology is combined for image coordinate conversion to improve the geographical alignment accuracy of the image data. The RANSAC algorithm is used for denoising to ensure the quality of the point cloud data and generate an accurate digital elevation model. Through the alignment of high-contrast regions and the fusion of image data, a high-resolution and high-precision mapping model is obtained.

[0076] In some embodiments of the present application, when determining whether to turn on LiDAR ranging according to the illumination intensity and inertial data, it includes: comparing the illumination intensity with the lowest light intensity threshold, and comparing the inertial data with the highest inertial threshold, and determining whether to turn on LiDAR ranging according to the comparison results.

[0077] Specifically, when the illumination intensity is less than the lowest light intensity threshold, or, the inertial data is greater than or equal to the highest inertial threshold, it is determined to turn on LiDAR ranging. When the illumination intensity is greater than or equal to the lowest light intensity threshold, and, the inertial data is less than the highest inertial threshold, it is determined not to turn on LiDAR ranging.

[0078] Specifically, when the light intensity is too low (such as in shadow areas, at night, or under cloud cover), the images captured by the optical camera have problems such as noise, motion blur, or insufficient exposure, which affect the mapping accuracy. In this embodiment, the real-time measured light intensity is compared with this threshold. If the light intensity is less than this threshold, it is considered that the quality of the optical image cannot meet the mapping requirements, and LiDAR ranging is enabled to make up for the deficiencies of the optical image. During the mapping process, the unmanned aerial vehicle (UAV) may be affected by wind, vibration, or attitude changes, resulting in an unstable shooting angle of the camera, which in turn affects the spatial consistency of the image. The maximum inertia threshold includes an angular velocity threshold and an acceleration threshold. The inertial data of the UAV (such as acceleration, angular velocity, etc.) is compared with this threshold. If the inertial data is greater than or equal to this threshold, it means that the motion state of the UAV is unstable, which will cause the optical image to be blurred or deformed, thereby triggering LiDAR ranging to obtain more stable spatial mapping data. When the light intensity is less than the set minimum light intensity threshold, or when the inertial data is greater than or equal to the set maximum inertia threshold, when either condition is met, LiDAR ranging is triggered to enhance the stability of the data. Conditions for not enabling LiDAR ranging: When the light intensity is greater than or equal to the set minimum light intensity threshold, and when the inertial data is less than the set maximum inertia threshold, and both of the above conditions are met, the optical image is continued to be used for mapping without enabling LiDAR to reduce the energy consumption and data redundancy of LiDAR.

[0079] It can be understood that through the dual-threshold determination of light intensity and inertial data, the UAV can dynamically adjust the mapping method according to the real-time environment, improving the adaptability, stability, and reliability of the mapping data. In the case of good lighting conditions and a stable UAV attitude, optical images are preferentially used for mapping to reduce the data storage pressure and computational burden; while in the case of insufficient lighting or unstable UAV movement, LiDAR ranging is automatically triggered to obtain high-precision point cloud data, thereby making up for the limitations of the optical image.

[0080] In some embodiments of the present application, when processing the point cloud data according to the timestamp of the image data and synchronizing the data time, it includes:

[0081] Collect the timestamp of the image data, and screen the point cloud data at two moments close to the timestamp in the point cloud data.

[0082] Calculate the point cloud data at the timestamp based on linear interpolation.

[0083]

[0084] Among them, Pt represents the point cloud data at timestamp t, Pt1 represents the point cloud data at moment t1 close to the timestamp, Pt2 represents the point cloud data at moment t2 close to the timestamp, and t1, t2 represent two moments close to the timestamp.

[0085] Specifically, while collecting the image data, the timestamp t of the data is recorded, that is, the specific time point when the image is taken. In the LiDAR point cloud data stream, find the two point cloud data frames closest to this timestamp, which correspond to time points t1 and t2 respectively, where t1 < t < t2.

[0086] It can be understood that through timestamp comparison and linear interpolation calculation, the precise alignment of point cloud data and image data in the time dimension is achieved, avoiding the mapping error caused by time deviation during data fusion. Compared with the traditional nearest neighbor matching method (directly selecting the closest single point cloud data), the linear interpolation method is adopted in this embodiment, effectively improving the time accuracy of the point cloud data, making it more conform to the time characteristics of the image data, thereby optimizing the data fusion effect.

[0087] In some embodiments of the present application, when processing the image data based on edge detection to determine the high-contrast region, it includes: performing Gaussian filtering on the image data and adjusting the standard deviation of the Gaussian kernel to smooth the image.

[0088] Calculate the gradients of the image data in the horizontal and vertical directions, and synthesize the total gradient G according to the horizontal and vertical gradients and obtain the gradient direction.

[0089] Based on non-maximum suppression, check adjacent pixels according to the gradient direction, and only retain the pixel points with the local maximum gradient value, and remove the remaining pixel points.

[0090] Set the high threshold Th and the low threshold Tl. When the total gradient G of the edge pixel is greater than the high threshold Th, define the edge pixel as a strong edge and retain it. When Th ≥ G ≥ Tl, define the edge pixel as a possible edge and retain it. When the total gradient G of the edge pixel is less than the low threshold Tl, define the edge pixel as a weak edge and remove it.

[0091] Perform edge connection according to the retained points, trace the possible edges. If the possible edge is connected to the strong edge, retain it, otherwise remove it, and connect the retained points to obtain the high-contrast region.

[0092] In some embodiments of the present application, when connecting the retained points to obtain the high-contrast region, it further includes: calculating the distance between any two of the retained points, connecting the retained points according to the shortest distance. When there is only one region in the connected image, take this region as the high-contrast region.

[0093] When there are at least two regions in the connected image, obtain the area of each region, and determine the region corresponding to the largest region area as the high-contrast region.

[0094] It is understandable that the high-contrast regions in the image data are effectively extracted through Gaussian filtering, gradient calculation, non-maximum suppression, double-threshold screening, and edge tracking, ensuring the accurate conversion of image coordinates and improving the accuracy of point cloud data fusion. Compared with traditional edge detection methods, in this embodiment, Gaussian filtering reduces the influence of noise and improves the stability of edge detection. Redundant edges are eliminated through non-maximum suppression, making the detected edges more detailed and the positioning more accurate. The double-threshold method effectively distinguishes strong edges and weak edges, improving the reliability. By comparing the regional areas, the best high-contrast region is selected, enhancing the accuracy of data processing.

[0095] In some embodiments of the present application, when converting the image coordinates in the image data into global coordinates according to the high-contrast regions, it includes:

[0096] Converting the image coordinates in the image data into camera coordinates.

[0097]

[0098]

[0099]

[0100] Among them, Xc, Yc, Zc represent the three-dimensional coordinates in the camera coordinate system, Sx represents the physical length corresponding to the pixel length, Sy represents the physical length corresponding to the pixel width, (u0, v0) represents the central pixel coordinates in the image data, (u, v) represents the coordinates of the position to be converted, represents the camera focal length.

[0101] Determining the UAV position coordinates TGNSS based on UAV positioning.

[0102] Obtaining the global coordinates according to the camera coordinates and the UAV position coordinates TGNSS.

[0103]

[0104] Among them, RIMU represents the UAV rotation matrix. (XG, YG, ZG) represents the three-dimensional coordinates in the UAV coordinate system.

[0105] Specifically, the UAV rotation matrix is:

[0106] ;

[0107] Among them, ϕ represents the UAV roll angle, θ represents the UAV pitch angle, and ψ represents the UAV yaw angle.

[0108] It can be understood that by gradually converting the image coordinates into global coordinates, high-precision mapping from the pixel level to the actual geographic space is achieved, which effectively improves the surveying and mapping accuracy and data consistency. Based on the physical size information of the pixel, the image coordinates are converted into camera coordinates, and combined with the GNSS positioning information of the drone, it is ensured that the surveying and mapping data can accurately correspond to the global coordinates, avoiding the coordinate offset caused by the change of the drone's flight trajectory. By introducing the inertial measurement unit data of the drone and using the rotation matrix to perform attitude correction on the camera coordinates, the surveying and mapping errors caused by the change of the drone's attitude are compensated, thereby improving the accuracy and stability of the coordinate conversion. This embodiment can dynamically adapt to the motion state of the drone in complex terrain and changing environments, so that the surveying and mapping results can maintain a high degree of accuracy under different environmental conditions, especially for scenes with large terrain undulations or large changes in ambient light. At the same time, this method can be fused with LiDAR data to further improve the three-dimensional accuracy of the surveying and mapping model.

[0109] In some embodiments of the present application, when denoising the point cloud data after the synchronization time based on the RANSAC algorithm, it includes:

[0110] Randomly select three points in the point cloud data after synchronization to form a three-point plane.

[0111]

[0112]

[0113] Calculate the distance between all points in the point cloud data after synchronization time and the three-point plane.

[0114] When the distance exceeds the distance threshold, the point cloud data is discarded.

[0115] Among them, x, y, z represent the three-dimensional coordinates of a point in the point cloud, that is, the position of the point in the drone coordinate system or the global coordinate system. (a, b, c) are the direction cosines of the normal vector of the plane. d is the intercept of the plane, which represents the relative position of the plane to the origin.

[0116] It is understandable that the robustness of the RANSAC algorithm is used to achieve denoising of point cloud data, effectively improving surveying and mapping accuracy and data quality. Through random sampling and iterative optimization, the main terrain structure can be accurately extracted even when there are many noise points, avoiding the misjudgment of terrain edges and complex structures by traditional mean filtering or median filtering methods, and improving the authenticity of point cloud data. By calculating the distance from the point to the plane and setting a reasonable threshold, outliers can be adaptively removed, which is particularly suitable for point cloud processing in complex terrain or multi-source data fusion environments, ensuring the accuracy of the digital elevation model.

[0117] In some embodiments of the present application, when generating a digital elevation model based on the denoised point cloud data, it includes: constructing the denoised point cloud data into a non-overlapping triangular network.

[0118] Calculate the triangular interpolation:

[0119]

[0120]

[0121] Wherein, SZ represents the terrain elevation of the interpolation calculation, SZi represents the elevation of the known point cloud of the triangle, wi represents the i-th distance weight, and di represents the distance between the i-th point and the interpolation position.

[0122] In some embodiments of the present application, when obtaining a mapping model by fusing the remaining areas in the image data based on the digital elevation model, it includes:

[0123]

[0124] Wherein, M represents the mapping model, DEM represents the digital elevation model, I represents the image data, and λ represents the fusion coefficient.

[0125] Specifically, the fusion coefficient is preferably 0.7.

[0126] It can be understood that by constructing a high-precision digital elevation model and performing fusion in combination with image data, the accuracy and applicability of the mapping model are improved. Using Delaunay triangulation to construct a TIN grid effectively avoids the distortion phenomenon that may occur in traditional regular grid interpolation, making the expression of elevation data more accurate. Calculating the elevation by inverse distance weighted interpolation ensures that the elevation data of the interpolation points can accurately reflect the terrain features and improves the accuracy of terrain modeling. During the fusion process of the mapping model, the adjustment of the fusion coefficient λ enables the finally generated mapping model to balance the accuracy of elevation data and the clarity of image data, realizing the efficient expression of geographic information.

[0127] In the above embodiments, an image acquisition device is carried by a drone to collect image data of the area to be surveyed and mapped under a preset path. By combining real-time monitoring of the light intensity and the inertial data of the drone, it is determined whether to activate LiDAR ranging, so as to enhance the surveying and mapping accuracy and data integrity under complex lighting conditions or when the attitude of the drone changes greatly. The timestamp is used to synchronize the image data and the point cloud data to achieve precise matching of multi-source data and ensure the consistency of the surveying and mapping information. Edge detection technology is adopted to identify high-contrast regions in the image, and based on these regions, the transformation of image coordinates to global coordinates is performed to improve the spatial accuracy of geographical information. For the point cloud data, the RANSAC algorithm is used for denoising to improve the quality of the point cloud data, and on this basis, a digital elevation model is generated to achieve high-precision terrain surveying and mapping. By locating high-contrast regions in the digital elevation model and aligning and fusing them, the accuracy and reliability of the surveying and mapping model are effectively improved. Especially under conditions of vegetation coverage, ground object occlusion, or poor lighting conditions, the integrity and accuracy of the surveying and mapping data can still be ensured.

[0128] In another preferred embodiment based on the above embodiments, referring to Figure 2 as shown, this embodiment provides a surveying and mapping system based on multi-source data fusion for applying the above surveying and mapping method based on multi-source data fusion, including:

[0129] A drone equipped with an image acquisition device, a LiDAR sensor, an inertial sensor, and a light sensor, and the image acquisition device includes an optical camera.

[0130] An acquisition unit configured to collect the light intensity of the area to be surveyed and mapped and the inertial data of the drone in real time, and determine whether to activate LiDAR ranging according to the light intensity and the inertial data.

[0131] A processing unit configured to, when the acquisition unit determines to activate LiDAR ranging, the processing unit obtains the image data and the point cloud data, and processes the point cloud data according to the timestamp of the image data to synchronize the data time.

[0132] The processing unit is further configured to process the image data based on edge detection to determine high-contrast regions, and convert the image coordinates in the image data into global coordinates according to the high-contrast regions.

[0133] The processing unit is further configured to denoise the point cloud data after synchronizing the time based on the RANSAC algorithm, and generate a digital elevation model according to the denoised point cloud data.

[0134] A fusion unit configured to determine high-contrast regions in the digital elevation model, align the high-contrast regions, and fuse the remaining regions in the image data according to the digital elevation model to obtain a surveying and mapping model.

[0135] It can be understood that by carrying an image acquisition device on a drone, image data of the area to be surveyed and mapped is collected along a preset path, and by combining real-time monitoring of the light intensity and the inertial data of the drone, it is determined whether to activate LiDAR ranging, so as to enhance the surveying and mapping accuracy and data integrity under complex lighting conditions or when the attitude of the drone changes greatly. The image data and the point cloud data are synchronized using timestamps to achieve precise matching of multi-source data and ensure the consistency of the surveying and mapping information. Edge detection technology is used to identify high-contrast regions in the image, and based on these regions, the conversion of image coordinates to global coordinates is performed to improve the spatial accuracy of geographical information. For the point cloud data, the RANSAC algorithm is used for denoising to improve the quality of the point cloud data, and on this basis, a digital elevation model is generated to achieve high-precision topographic surveying and mapping. By locating high-contrast regions in the digital elevation model and aligning and fusing them, the accuracy and reliability of the surveying and mapping model are effectively improved. Especially in cases of vegetation coverage, ground object occlusion, or poor lighting conditions, the integrity and accuracy of the surveying and mapping data can still be ensured.

[0136] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0137] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0138] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for realizing the functions specified in one process or multiple processes and / or one block or multiple blocks in the flow Figure 1 one process or multiple processes and / or blocks Figure 1 or steps for realizing the functions specified in multiple blocks.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: it is still possible to modify the specific implementation manners of the present invention or make equivalent replacements, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A surveying and mapping method based on multi-source data fusion, characterized in that: include: Based on the image acquisition device carried by the drone, image data of the area to be surveyed is collected according to a preset path, the light intensity of the area to be surveyed and the inertial data of the drone are collected in real time, and whether to turn on LiDAR ranging is determined according to the light intensity and inertial data; When it is determined that the LiDAR ranging is turned on, the image data and the point cloud data are acquired, the point cloud data are processed according to the timestamp of the image data, and the data time is synchronized; Processing the image data based on edge detection to determine a high contrast area, and converting image coordinates in the image data into global coordinates according to the high contrast area; De-noising the point cloud data after synchronization based on the RANSAC algorithm, and generating a digital elevation model based on the de-noised point cloud data; The high contrast area is determined in the digital elevation model, the high contrast area is aligned, and the remaining areas in the image data are fused according to the digital elevation model to obtain a mapping model.

2. The surveying and mapping method based on multi-source data fusion according to claim 1, characterized in that: When judging whether to start LiDAR ranging according to the light intensity and inertial data, it includes: Comparing the light intensity with a minimum light intensity threshold, and comparing the inertial data with a maximum inertial threshold, and determining whether to turn on LiDAR ranging according to the comparison results; When the light intensity is less than the minimum light intensity threshold, or the inertial data is greater than or equal to the maximum inertial threshold, it is determined to start LiDAR ranging; When the light intensity is greater than or equal to the minimum light intensity threshold, and the inertial data is less than the maximum inertial threshold, it is determined that LiDAR ranging is not enabled.

3. The surveying and mapping method based on multi-source data fusion according to claim 2, characterized in that: Processing the point cloud data according to the timestamp of the image data to synchronize the data time includes: Acquire a timestamp of the image data, and select point cloud data at two moments close to the timestamp from the point cloud data; Calculate the point cloud data at the timestamp based on linear interpolation; ； Among them, Pt represents the point cloud data at timestamp t, Pt1 represents the point cloud data at time t1 close to the timestamp, Pt2 represents the point cloud data at time t2 close to the timestamp, and t1 and t2 represent two times close to the timestamp.

4. The surveying and mapping method based on multi-source data fusion according to claim 1, characterized in that: When the image data is processed based on edge detection to determine the high contrast area, it includes: Performing Gaussian filtering on the image data, and adjusting the standard deviation of the Gaussian kernel to smooth the image; Calculate the gradients of the image data in the horizontal direction and the vertical direction, synthesize the total gradient G according to the gradients in the horizontal direction and the vertical direction, and obtain the gradient direction; Based on non-maximum suppression, adjacent pixels are checked according to the gradient direction, only pixels with the local maximum gradient value are retained, and the rest are removed; Set a high threshold Th and a low threshold Tl, when the total gradient G of the edge pixel is greater than the high threshold Th, define the edge pixel as a strong edge and retain it, when Th≥G≥Tl, define the edge pixel as a possible edge and retain it, when the total gradient G of the edge pixel is less than the low threshold Tl, define the edge pixel as a weak edge and remove it; Edge connection is performed according to the reserved points, and possible edges are tracked. If the possible edge is connected to a strong edge, it is retained, otherwise it is removed, and the reserved points are connected to obtain the high contrast area.

5. The surveying and mapping method based on multi-source data fusion according to claim 4, characterized in that: When the reserved points are connected to obtain the high contrast area, the method further includes: Calculate the distance between any two points in the reserved points, connect the reserved points according to the shortest distance, and when there is only one area in the connected image, use the area as the high contrast area; When there are at least two regions in the connected image, the areas of the regions are obtained, and the region corresponding to the largest region area is determined as the high contrast region.

6. The surveying and mapping method based on multi-source data fusion according to claim 5, characterized in that: When converting the image coordinates in the image data into global coordinates according to the high contrast area, it includes: Convert image coordinates in image data into camera coordinates; ； ； ； Among them, Xc, Yc, Zc represent the three-dimensional coordinates in the camera coordinate system, Sx represents the physical length corresponding to the pixel length, Sy represents the physical length corresponding to the pixel width, (u0, v0) represents the center pixel coordinates in the image data, (u, v) represents the position coordinates of the point to be converted, Indicates the focal length of the camera; Determine the drone position coordinates TGNSS based on drone positioning; Obtain the global coordinates according to the camera coordinates and the drone position coordinates TGNSS; ； Among them, RIMU represents the UAV rotation matrix.

7. The surveying and mapping method based on multi-source data fusion according to claim 1, characterized in that: When denoising the point cloud data after synchronization time based on the RANSAC algorithm, it includes: Randomly select three points in the point cloud data after the synchronization time to form a three-point plane; ； ； Calculating the distances between all points in the point cloud data after the synchronization time and the three-point plane; When the distance exceeds the distance threshold, the point cloud data is discarded.

8. The surveying and mapping method based on multi-source data fusion according to claim 7, characterized in that: When generating a digital elevation model based on denoised point cloud data, it includes: Constructing the denoised point cloud data into a non-overlapping triangle network; Compute triangle interpolation: ； ； Among them, SZ represents the terrain elevation calculated by interpolation, SZi represents the elevation of the known point cloud of the triangle, wi represents the i-th distance weight, and di represents the distance between the i-th point and the interpolation position.

9. The surveying and mapping method based on multi-source data fusion according to claim 8, characterized in that: When the rest of the regions in the image data are fused according to the digital elevation model to obtain a surveying and mapping model, the method includes: ； Among them, M represents the surveying and mapping model, DEM represents the digital elevation model, I represents the image data, and λ represents the fusion coefficient.

10. A surveying and mapping system based on multi-source data fusion, used for applying the surveying and mapping method based on multi-source data fusion as claimed in any one of claims 1 to 9, characterized in that: include: A drone is provided with an image acquisition device, a LiDAR sensor, an inertial sensor and a light sensor, wherein the image acquisition device includes an optical camera; A collection unit is configured to collect the light intensity of the area to be surveyed and the inertial data of the drone in real time, and determine whether to turn on LiDAR ranging according to the light intensity and the inertial data; a processing unit configured to, when the acquisition unit determines to start the LiDAR ranging, obtain the image data and the point cloud data, process the point cloud data according to the timestamp of the image data, and synchronize the data time; The processing unit is further configured to process the image data based on edge detection to determine a high contrast area, and convert image coordinates in the image data into global coordinates according to the high contrast area; The processing unit is further configured to denoise the point cloud data after the synchronization time based on a RANSAC algorithm, and generate a digital elevation model according to the denoised point cloud data; The fusion unit is configured to determine the high contrast area in the digital elevation model, align the high contrast area and fuse the remaining areas in the image data according to the digital elevation model to obtain a mapping model.

Images