Intelligent processing method and system for multi-source data fusion of surveying and mapping equipment

Abstract

The invention discloses an intelligent processing method and system for multi-source data fusion of surveying and mapping equipment, and belongs to the technical field of surveying and mapping data processing.According to the method and system, an initial LiDAR point cloud is combined with position data and attitude data for joint calculation to obtain an accurate LiDAR point cloud, the accuracy of point cloud data is optimized, and therefore the accuracy of final registration is improved, and the registration accuracy is improved. And matching the point cloud multi-scale feature point set with the image multi-scale feature point set to obtain a point cloud homonymy point set and an image homonymy point set, analyzing a rotation matrix and a translation vector through a quaternion registration algorithm, and completing registration of the target panoramic image and the precise LiDAR point cloud according to the rotation matrix and the translation vector. And high-precision space registration is realized through a quaternion registration algorithm, the registration accuracy is further improved, and the surveying and mapping precision is improved.

Description

Technical Field

[0001] This invention relates to the field of surveying and mapping data processing technology, and in particular to an intelligent processing method and system for multi-source data fusion of surveying and mapping equipment. Background Technology

[0002] Surveying and mapping refers to the technology of measuring, collecting, processing, and analyzing spatial information about the Earth's surface or other objects, and expressing it in the form of graphics, data, or models. Traditional surveying methods often rely on a single data source for processing, such as using only LiDAR point cloud data or only image data, resulting in incomplete information coverage and limited accuracy. Therefore, existing technologies often use multi-source data fusion to improve surveying accuracy, such as registering and fusing images with LiDAR point clouds (laser radar point clouds). However, existing technologies typically employ simple overlay or rule-based fusion methods, resulting in insufficient accuracy in registration and fusion, which in turn affects the accuracy of surveying and mapping. Summary of the Invention

[0003] To address the technical problems existing in the prior art, this invention provides an intelligent processing method for multi-source data fusion in surveying and mapping equipment, comprising the following steps: Acquire the initial LiDAR point cloud of the target and multiple target images from different perspectives, as well as the position and attitude data of the surveying equipment during the surveying process; The initial LiDAR point cloud is combined with position and attitude data for joint calculation to obtain a precise LiDAR point cloud. Multi-scale feature point set is extracted from the precise LiDAR point cloud. The target images are stitched together to obtain a panoramic image of the target, and a multi-scale feature point set is extracted from the panoramic image of the target. Based on the feature descriptor vector, the multi-scale feature point set of the point cloud is matched with the multi-scale feature point set of the image to obtain the set of corresponding point pairs, and the set of corresponding point pairs is divided into the point cloud corresponding point set and the image corresponding point set. Based on the point cloud and image point sets, the rotation matrix and translation vector are analyzed using a quaternion registration algorithm. The registration of the target panoramic image and the accurate LiDAR point cloud is then completed based on the rotation matrix and translation vector.

[0004] Furthermore, the location data is GPS data or BeiDou location data, and the attitude data is IMU data.

[0005] Furthermore, the step of jointly solving the initial LiDAR point cloud with position and attitude data to obtain a precise LiDAR point cloud specifically involves: Align the initial LiDAR point cloud, position data, and attitude data; The device trajectory path is obtained by combining position and attitude data through Kalman filtering. Compensated LiDAR point clouds are obtained by compensating the initial LiDAR point cloud using a pre-built point cloud error compensation model. The feature complexity of the survey target is evaluated based on the equipment trajectory path, and the feature complexity includes comprehensive feature complexity and complexity threshold; The initial point weights of each initial point in the initial LiDAR point cloud and the compensation point weights of each compensation point in the compensated LiDAR point cloud are analyzed based on feature complexity. The initial LiDAR point cloud and the compensated LiDAR point cloud are fused using the initial point weight and the compensated point weight to obtain the accurate LiDAR point cloud. Furthermore, the step of evaluating the feature complexity of the mapping target based on the device trajectory path specifically includes: Divide the device trajectory path into multiple sub-paths; Evaluate the comprehensive feature complexity and complexity threshold of the mapping target based on the trajectory points on the sub-path: ; ; ; in, The comprehensive feature complexity of the mapping target mapped by the j-th sub-path segment. Let j be the number of trajectory points in the j-th sub-path. Let be the relative gradient value at the i-th trajectory point of the j-th sub-path segment. Let be the vertical distance mapped from the i-th trajectory point of the j-th sub-path to the surface of the target object. Let be the vertical distance mapped from the (i-1)th trajectory point of the j-th sub-path segment to the surface of the target object. Let be the straight-line distance between the (i-1)th trajectory point and the ith trajectory point in the j-th sub-path. The threshold for the complexity of the mapping target. and These are the maximum and minimum values ​​selected from the comprehensive feature complexity corresponding to each sub-path, respectively.

[0006] Furthermore, the step of analyzing the initial point weights of each initial point in the initial LiDAR point cloud and the compensation point weights of each compensation point in the compensated LiDAR point cloud based on feature complexity is as follows: Map each initial point in the initial LiDAR point cloud to the device trajectory path to obtain the sub-path where each initial point is mapped. Based on the comprehensive feature complexity corresponding to the sub-path of each initial point, calculate the compensation point weight corresponding to each initial point: ; ; For preset coefficients, Let the initial point weights be the weights of the i-th initial point in the initial LiDAR point cloud. To compensate for the weight of the i-th compensation point in the LiDAR point cloud, Let be the comprehensive feature complexity of the sub-path corresponding to the i-th initial point.

[0007] Furthermore, the extraction of multi-scale feature point sets from the accurate LiDAR point cloud specifically involves: Generate depth maps and normal vector maps based on accurate LiDAR point clouds; The depth map and normal vector map are converted into grayscale images, and the features of the depth map and normal vector map are obtained by the SIFT algorithm. The point cloud multi-scale feature point set is obtained by performing a union operation on the depth map features and the normal vector map features.

[0008] Furthermore, the step of stitching together the target images to obtain a panoramic target image, and extracting a multi-scale feature point set from the panoramic target image, specifically involves: Each target image is preprocessed to obtain a first image, including radiometric correction, geometric correction and color balance processing; Feature points are extracted from each first image using the SIFT algorithm, and feature point matching is performed between adjacent first images to obtain each matching point pair; The first homography matrix H1 is obtained based on the analysis of each matching point pair; Select a first image as a reference image, and transform the other first images one by one into the coordinate system of the reference image by applying the perspective transformation function and H1, and then stitch them together using a fusion algorithm to obtain the target panoramic image. The SIFT algorithm is used to obtain a multi-scale feature point set of the target panoramic image.

[0009] Furthermore, the first homography matrix H1 obtained from the analysis of each matching point pair is specifically as follows: S331. Select 4 sets of first matching point pairs from each matching point pair by random sampling, and calculate the corresponding 3×3 homography matrix H from the 4 sets of first matching point pairs; S332. Calculate the error E between the first matching point and H in each group, and count the number of second matching point pairs. The second matching point pair is the first matching point pair whose E is less than the preset error threshold. S333. Repeat steps S331 to S332 until the number of repetitions reaches the preset number, and select H0, which has the most second matching point pairs, as H1.

[0010] Furthermore, the step of analyzing the rotation matrix and translation vector using a quaternion registration algorithm based on the point cloud's corresponding point set and the image's corresponding point set specifically involves: Obtain the centroid coordinates of the point cloud point set and the image point set with the same name, respectively; Based on the centroid coordinates, the point cloud point set and the image point set are centered respectively; Construct a covariance matrix based on the centralized point cloud point set and the image point set; The covariance matrix is ​​converted into a symmetric matrix required for quaternion registration. The symmetric matrix is ​​then decomposed into eigenvalues ​​to obtain the eigenvector corresponding to the largest eigenvalue, which is then used as the quaternion vector. The rotation matrix and translation vector are calculated from the quaternion vector.

[0011] This invention also provides an intelligent processing system for multi-source data fusion in surveying and mapping equipment, comprising: The acquisition module is used to acquire the initial LiDAR point cloud of the surveying target and multiple target images from different perspectives, as well as the position and attitude data of the surveying equipment during the surveying process. The first extraction module is used to combine the initial LiDAR point cloud with position data and attitude data for joint calculation to obtain a precise LiDAR point cloud, and to extract a multi-scale feature point set from the precise LiDAR point cloud. The second extraction module stitches together the target images to obtain a target panoramic image, and extracts a multi-scale feature point set from the target panoramic image. The matching module is used to match the multi-scale feature point set of the point cloud with the multi-scale feature point set of the image based on the feature descriptor vector to obtain the set of corresponding point pairs, and then divide the set of corresponding point pairs into the point cloud corresponding point set and the image corresponding point set. The registration module is used to analyze the rotation matrix and translation vector based on the point cloud and image point sets using a quaternion registration algorithm, and then complete the registration between the target panoramic image and the accurate LiDAR point cloud. Compared with the prior art, the beneficial effects of the present invention are as follows: This invention obtains a precise LiDAR point cloud by jointly solving the initial LiDAR point cloud with position and attitude data, thus optimizing the accuracy of the point cloud data and improving the accuracy of the final registration. The multi-scale feature point set of the point cloud is matched with the multi-scale feature point set of the image to obtain the corresponding point set of the point cloud and the corresponding point set of the image. After analyzing the rotation matrix and translation vector using the quaternion registration algorithm, the registration of the target panoramic image and the precise LiDAR point cloud is completed based on the rotation matrix and translation vector. The quaternion registration algorithm achieves high-precision spatial registration, further improving the registration accuracy and the mapping accuracy. Kalman filtering is used to obtain the device trajectory path, which effectively handles the noise and uncertainty of the trajectory data. A pre-built point cloud error compensation model is introduced to solve the systematic error problem in the original point cloud data. Furthermore, through dynamic evaluation of feature complexity, adaptive allocation of point cloud weights is achieved, which significantly improves the point cloud accuracy. By assigning point cloud weights based on trajectory path segmentation, and mapping the initial point cloud points to the trajectory path, the weights are dynamically calculated based on the comprehensive feature complexity of the corresponding sub-paths, making the weight allocation more reasonable, effectively improving the point cloud accuracy, and thus improving the registration accuracy. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of an intelligent processing method for multi-source data fusion in surveying equipment according to the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0016] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0017] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0018] Example 1 See Figure 1 As shown, the present invention provides an intelligent processing method for multi-source data fusion in surveying and mapping equipment, which specifically includes the following steps: S1. Acquire the initial LiDAR point cloud of the target and multiple target images from different perspectives, as well as the position and attitude data of the surveying equipment during the surveying process; S2. Combine the initial LiDAR point cloud with position data and attitude data for joint calculation to obtain a precise LiDAR point cloud. Extract a multi-scale feature point set from the precise LiDAR point cloud. S3. Stitch together the target images to obtain a panoramic image of the target, and extract a multi-scale feature point set from the panoramic image of the target. S4. Based on the feature descriptor vector, match the multi-scale feature point set of the point cloud with the multi-scale feature point set of the image to obtain the set of corresponding point pairs, and divide the set of corresponding point pairs into the point cloud corresponding point set and the image corresponding point set. S5. Based on the point cloud and image point sets, analyze the rotation matrix and translation vector using a quaternion registration algorithm, and complete the registration of the target panoramic image with the accurate LiDAR point cloud based on the rotation matrix and translation vector.

[0019] Location data refers to the location data generated by the surveying equipment during the surveying of the target, which can be GPS data or BeiDou location data; attitude data refers to the inertial measurement unit (IMU) data generated by the surveying equipment during the surveying of the target.

[0020] In step S2, the initial LiDAR point cloud is combined with position data and attitude data for joint calculation to obtain a precise LiDAR point cloud, specifically as follows: Sa21. Align the initial LiDAR point cloud, position data, and attitude data. Sa22: Combine position and attitude data to obtain the device trajectory path through Kalman filtering; Sa23. The initial LiDAR point cloud is compensated by a pre-constructed point cloud error compensation model to obtain a compensated LiDAR point cloud. Sa24. Evaluate the feature complexity of the survey target based on the equipment trajectory path, wherein the feature complexity includes comprehensive feature complexity and complexity threshold; Sa25. Analyze the initial point weights of each initial point in the initial LiDAR point cloud and the compensation point weights of each compensation point in the compensated LiDAR point cloud based on feature complexity. Sa26. Based on the initial point weights and the compensation point weights, perform point cloud fusion on the initial LiDAR point cloud and the compensation LiDAR point cloud to obtain the accurate LiDAR point cloud: ;

[0021] in, To accurately determine the coordinates of the i-th point in the LiDAR point cloud, Let the initial point weights be the weights of the i-th initial point in the initial LiDAR point cloud. Let i be the coordinates of the i-th initial point in the initial LiDAR point cloud. To compensate for the weight of the i-th compensation point in the LiDAR point cloud, To compensate for the coordinates of the i-th compensation point in the LiDAR point cloud, the i-th compensation point is the point corresponding to the i-th initial point after compensation by the point cloud error compensation model.

[0022] In step Sa23, the point cloud error compensation model can be a deep learning model or a neural network, which is trained based on historical point cloud samples and historical correction point cloud samples.

[0023] In step Sa24, the step of evaluating the feature complexity of the mapping target based on the equipment trajectory path specifically involves: S241. Divide the device trajectory path into multiple sub-paths; S242. Evaluate the comprehensive feature complexity and complexity threshold of the mapping target based on the trajectory points on the sub-path: ; ; ; in, The comprehensive feature complexity of the mapping target mapped by the j-th sub-path segment. Let j be the number of trajectory points in the j-th sub-path. Let be the relative gradient value at the i-th trajectory point of the j-th sub-path segment. Let be the vertical distance mapped from the i-th trajectory point of the j-th sub-path to the surface of the target object. Let be the vertical distance mapped from the (i-1)th trajectory point of the j-th sub-path segment to the surface of the target object. Let be the straight-line distance between the (i-1)th trajectory point and the ith trajectory point in the j-th sub-path. The threshold for the complexity of the mapping target. and These are the maximum and minimum values ​​selected from the comprehensive feature complexity corresponding to each sub-path, respectively.

[0024] In step Sa25, the step of analyzing the initial point weights of each initial point in the initial LiDAR point cloud and the compensation point weights of each compensation point in the compensated LiDAR point cloud based on feature complexity is as follows: S251. Map each initial point in the initial LiDAR point cloud to the device trajectory path to obtain the sub-path where each initial point is mapped. S252. Based on the comprehensive feature complexity corresponding to the sub-path of each initial point, calculate the compensation point weight of each initial point: ; ; For preset coefficients, Let be the comprehensive feature complexity of the sub-path corresponding to the i-th initial point.

[0025] exp() is the natural exponential function.

[0026] In step S2, the extraction of multi-scale feature point sets from the accurate LiDAR point cloud specifically involves: Sb21, Generating depth maps and normal vector maps based on accurate LiDAR point clouds; Sb22. Convert the depth map and normal vector map into grayscale images respectively, and obtain the depth map features and normal vector map features respectively using the SIFT algorithm; Sb23. Perform a union operation on the depth map features and the normal vector map features to obtain a multi-scale feature point set of the point cloud.

[0027] SIFT (Scale Invariant Feature Transform) algorithm is an algorithm used to detect and describe image features.

[0028] In step S3, the process of stitching together the target images to obtain a panoramic target image and extracting a multi-scale feature point set from the panoramic target image specifically involves: S31. Perform correction preprocessing on each target image to obtain each first image, including radiometric correction, geometric correction and color balance processing. S32. Extract feature points from each first image using the SIFT algorithm, and perform feature point matching between adjacent first images to obtain each matching point pair; S33. Based on the analysis of each matching point pair, the first homography matrix H1 is obtained; S34. Select a first image as a reference image, and transform the other first images one by one into the coordinate system of the reference image by applying the perspective transformation function and H1, and then stitch them together using the fusion algorithm to obtain the target panoramic image. S35. Obtain the multi-scale feature point set of the target panoramic image using the SIFT algorithm.

[0029] In step S31, radiometric correction eliminates the influence of atmospheric scattering, uneven illumination, and other factors on the image; geometric correction eliminates distortion; and color balance processing eliminates illumination differences between different images.

[0030] In step S33, the first homography matrix H1 is obtained based on the analysis of each matching point pair, specifically as follows: S331. Select 4 sets of first matching point pairs from each matching point pair by random sampling, and calculate the corresponding 3×3 homography matrix H from the 4 sets of first matching point pairs; S332. Calculate the error E between the first matching point and H in each group, and count the number of second matching point pairs. The second matching point pair is the first matching point pair whose E is less than the preset error threshold. S333. Repeat steps S331 to S332 until the number of repetitions reaches the preset number, and select H0, which has the most second matching point pairs, as H1.

[0031] In step S331, the equation for the 3×3 homography matrix H0 is: ; ( , Let be the coordinates of one of the points in the first matching point pair of the i-th group in the corresponding first image. , ) represents the coordinates of the other point in the first matching point pair of the i-th group in the corresponding first image.

[0032] In step S332, the error E between each group of first matching points and H0 is calculated, and the expression is: ; Let H be the error between the first matching point pair in the i-th group and H. Let be the coordinates of one of the points in the first matching point pair of the i-th group in the corresponding first image. Let be the coordinates of the other point in the first matching point pair of the i-th group in the corresponding first image.

[0033] In step S34, the perspective transformation function is an existing transformation function, such as the warpPerspective function of OpenCV, and the fusion algorithm is an existing transformation function, such as weighted average fusion, gradient domain concatenation, or Laplacian pyramid fusion, etc.

[0034] In step S4, the multi-scale feature point set of the point cloud is matched with the multi-scale feature point set of the image based on the feature descriptor vector to obtain a set of corresponding point pairs. That is, the cosine similarity is calculated by comparing the feature descriptor vector of the feature point in the multi-scale feature point set of the point cloud with the feature descriptor vector of the corresponding feature point in the multi-scale feature point set of the image. If the similarity is greater than or equal to the preset similarity threshold, then the feature pair is a pair of corresponding feature points.

[0035] In step S5, the step of analyzing the rotation matrix and translation vector using a quaternion registration algorithm based on the point cloud point set and the image point set specifically involves: S51. Obtain the centroid coordinates of the point cloud point set and the image point set with the same name, respectively. S52. Based on the centroid coordinates, center the point cloud set of points with the same name and the image set of points with the same name respectively; S53. Construct a covariance matrix based on the centralized point cloud point set and the image point set; S54. Convert the covariance matrix into a symmetric matrix required for quaternion registration, and perform eigenvalue decomposition on the symmetric matrix to obtain the eigenvector corresponding to the largest eigenvalue as the quaternion vector. S55. Calculate the rotation matrix and translation vector based on the quaternion vector.

[0036] In step S51, the centroid coordinates of the point cloud set of corresponding points and the image set of corresponding points are obtained respectively: ; ; DTZ and YTZ are the centroid coordinates of the point cloud and image point sets, respectively, and TN is the number of point pairs. and These are the coordinates of the i-th point cloud corresponding point and the i-th image corresponding point, respectively.

[0037] In step S52, the point cloud point set and the image point set are centered according to the centroid coordinates: ; ; and These are the coordinates of the i-th point cloud corresponding point after centering and the coordinates of the i-th image corresponding point after centering, respectively.

[0038] The construction of the covariance matrix, the acquisition of the symmetric matrix, and the calculation of the rotation matrix and translation vector based on the quaternion vector are all performed using conventional methods of quaternion registration algorithms, and will not be elaborated further here.

[0039] Example 2 This invention also provides an intelligent processing system for multi-source data fusion in surveying and mapping equipment, specifically comprising: The acquisition module is used to acquire the initial LiDAR point cloud of the surveying target and multiple target images from different perspectives, as well as the position and attitude data of the surveying equipment during the surveying process. The first extraction module is used to combine the initial LiDAR point cloud with position data and attitude data for joint calculation to obtain a precise LiDAR point cloud, and to extract a multi-scale feature point set from the precise LiDAR point cloud. The second extraction module stitches together the target images to obtain a target panoramic image, and extracts a multi-scale feature point set from the target panoramic image. The matching module is used to match the multi-scale feature point set of the point cloud with the multi-scale feature point set of the image based on the feature descriptor vector to obtain the set of corresponding point pairs, and then divide the set of corresponding point pairs into the point cloud corresponding point set and the image corresponding point set. The registration module is used to analyze the rotation matrix and translation vector based on the point cloud and image point sets, and complete the registration of the target panoramic image with the accurate LiDAR point cloud based on the rotation matrix and translation vector.

[0040] Example 3 The present invention also provides an electronic device, including: a processor, a transmitting device, an input device, an output device, and a memory. The processor may be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory may be implemented using a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), and is used to store computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes a method as described in any of the above possible implementation methods.

[0041] Example 4 The present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor of an electronic device, cause the processor to perform a method as described in any of the above possible implementations.

[0042] The beneficial effects of this invention are as follows: This invention obtains a precise LiDAR point cloud by jointly solving the initial LiDAR point cloud with position and attitude data, thus optimizing the accuracy of the point cloud data and improving the accuracy of the final registration. The multi-scale feature point set of the point cloud is matched with the multi-scale feature point set of the image to obtain the corresponding point set of the point cloud and the corresponding point set of the image. After analyzing the rotation matrix and translation vector using the quaternion registration algorithm, the registration of the target panoramic image and the precise LiDAR point cloud is completed based on the rotation matrix and translation vector. The quaternion registration algorithm achieves high-precision spatial registration, further improving the registration accuracy and the mapping accuracy. Kalman filtering is used to obtain the device trajectory path, which effectively handles the noise and uncertainty of the trajectory data. A pre-built point cloud error compensation model is introduced to solve the systematic error problem in the original point cloud data. Furthermore, through dynamic evaluation of feature complexity, adaptive allocation of point cloud weights is achieved, which significantly improves the point cloud accuracy. By assigning point cloud weights based on trajectory path segmentation, and mapping the initial point cloud points to the trajectory path, the weights are dynamically calculated based on the comprehensive feature complexity of the corresponding sub-paths, making the weight allocation more reasonable, effectively improving the point cloud accuracy, and thus improving the registration accuracy.

[0043] In the description of this specification, the references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0044] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0045] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An intelligent processing method for multi-source data fusion of surveying equipment, characterized in that, The method comprises the following steps: obtaining initial LiDAR point cloud of a survey target, multiple target images of different perspectives, position data and attitude data of a survey device in a survey process; jointly solving the initial LiDAR point cloud in combination with the position data and the attitude data to obtain accurate LiDAR point cloud, and extracting a point cloud multi-scale feature point set from the accurate LiDAR point cloud; stitching the target images to obtain a target panoramic image, and extracting an image multi-scale feature point set from the target panoramic image; matching the point cloud multi-scale feature point set and the image multi-scale feature point set based on a feature descriptor vector to obtain a same-name point pair set, and dividing the same-name point pair set into a point cloud same-name point set and an image same-name point set; analyzing a rotation matrix and a translation vector through a quaternion registration algorithm according to the point cloud same-name point set and the image same-name point set, and completing registration of the target panoramic image and the accurate LiDAR point cloud according to the rotation matrix and the translation vector.

2. The intelligent processing method for surveying equipment multi-source data fusion according to claim 1, characterized in that, The position data is GPS data or Beidou position data, and the attitude data is IMU data.

3. The intelligent processing method for surveying equipment multi-source data fusion according to claim 1, characterized in that, The joint solving of the initial LiDAR point cloud in combination with the position data and the attitude data to obtain the accurate LiDAR point cloud specifically comprises: aligning data of the initial LiDAR point cloud, the position data and the attitude data; obtaining a device trajectory path through Kalman filtering in combination with the position data and the attitude data; compensating the initial LiDAR point cloud through a pre-constructed point cloud error compensation model to obtain compensated LiDAR point cloud; evaluating a feature complexity of the survey target according to the device trajectory path, wherein the feature complexity comprises a comprehensive feature complexity and a complexity threshold; analyzing an initial point weight of each initial point in the initial LiDAR point cloud and a compensated point weight of each compensated point in the compensated LiDAR point cloud according to the feature complexity; performing fusion point cloud calculation on the initial LiDAR point cloud and the compensated LiDAR point cloud according to the initial point weight and the compensated point weight to obtain the accurate LiDAR point cloud.

4. The intelligent processing method for multi-source data fusion of surveying equipment according to claim 3, characterized in that, The evaluation of the feature complexity of the survey target according to the device trajectory path specifically comprises: dividing the device trajectory path into multiple sub-paths; evaluating the comprehensive feature complexity and the complexity threshold of the survey target according to trajectory points on the sub-paths. ； ； ； wherein, is a complexity of the mapping target mapped to the jth sub-path, is a number of trajectory points of the jth sub-path, is a relative gradient value at the ith trajectory point of the jth sub-path, is a vertical distance from the ith trajectory point of the jth sub-path to the surface of the mapping target, is a vertical distance from the i-1th trajectory point of the jth sub-path to the surface of the mapping target, is a straight-line distance between the i-1th trajectory point and the ith trajectory point of the jth sub-path, is a complexity threshold of the mapping target, and are a maximum value and a minimum value, respectively, selected from the complexity of the mapping target.

5. The intelligent processing method of multi-source data fusion of surveying equipment according to claim 4, characterized in that, The analysis of the initial point weight of each initial point in the initial LiDAR point cloud and the compensated point weight of each compensated point in the compensated LiDAR point cloud according to the feature complexity specifically comprises: mapping each initial point in the initial LiDAR point cloud to the device trajectory path to obtain a sub-path where each initial point is mapped; calculating the compensated point weight of the compensated point corresponding to each initial point according to the comprehensive feature complexity corresponding to the sub-path corresponding to each initial point. ； ； is a preset coefficient, is an initial point weight of an i-th initial point in an initial LiDAR point cloud, is a compensated point weight of an i-th compensated point in a compensated LiDAR point cloud, is a corresponding comprehensive feature complexity of a corresponding sub-path of the i-th initial point.

6. The intelligent processing method for surveying equipment multi-source data fusion according to claim 1, characterized in that, The extraction of the point cloud multi-scale feature point set from the accurate LiDAR point cloud specifically comprises: generating a depth map and a normal vector map based on the accurate LiDAR point cloud; respectively converting the depth map and the normal vector map into grayscale maps, and respectively obtaining depth map features and normal vector map features through a SIFT algorithm; performing a set operation on the depth map features and the normal vector map features to obtain the point cloud multi-scale feature point set.

7. The intelligent processing method for surveying equipment multi-source data fusion according to claim 1, characterized in that, The stitching of the target images to obtain the target panoramic image, and the extraction of the image multi-scale feature point set from the target panoramic image specifically comprise: The target images are corrected and preprocessed to obtain each first image, including radiation correction, geometric correction and color balance processing; Feature points are extracted from each first image by using the SIFT algorithm, and feature point matching is performed between adjacent first images to obtain each matching point pair; A first homography matrix H1 is obtained according to the matching point pairs; One of the first images is selected as a reference image, and the other first images are sequentially subjected to perspective transformation by using a perspective transformation function and applying H1 to the coordinate system of the reference image, and splicing is completed by using a fusion algorithm to obtain a target panoramic image; Image multi-scale feature point sets of the target panoramic image are obtained by using the SIFT algorithm.

8. The intelligent processing method of multi-source data fusion of surveying equipment according to claim 7, characterized in that, The first homography matrix H1 is obtained according to the matching point pairs, and specifically: S331, 4 groups of first matching point pairs are selected from the matching point pairs by random sampling, and a corresponding 3*3 homography matrix H is calculated by using the 4 groups of first matching point pairs; S332, the error E of each group of first matching points and H is calculated, and the number of second matching point pairs is counted, the second matching point pair being the first matching point pair with an error E less than a preset error threshold; S333, steps S331 to S332 are repeated until the number of repetitions reaches a preset number, and H0 with the largest number of second matching point pairs is selected as H1.

9. The intelligent processing method for surveying equipment multi-source data fusion according to claim 1, characterized in that, According to the point cloud homonymic point set and the image homonymic point set, a rotation matrix and a translation vector are analyzed by using a quaternion registration algorithm, and specifically: The centroid coordinates of the point cloud homonymic point set and the image homonymic point set are obtained respectively; The point cloud homonymic point set and the image homonymic point set are centralized respectively according to the centroid coordinates; A covariance matrix is constructed based on the centralized point cloud homonymic point set and the centralized image homonymic point set; The covariance matrix is converted into a symmetric matrix required for quaternion registration, and the symmetric matrix is subjected to eigenvalue decomposition to obtain a feature vector corresponding to the maximum eigenvalue as a quaternion vector; The rotation matrix and the translation vector are calculated according to the quaternion vector.

10. An intelligent processing system for multi-source data fusion of surveying equipment, applying the intelligent processing method for multi-source data fusion of surveying equipment according to any one of claims 1 to 9, characterized in that, It comprises: An acquisition module is configured to acquire initial LiDAR point clouds of a surveying and mapping target, a plurality of target images with different viewing angles, position data and attitude data of a surveying and mapping device during a surveying and mapping process; A first extraction module is configured to jointly solve the initial LiDAR point clouds in combination with the position data and the attitude data to obtain accurate LiDAR point clouds, and extract a point cloud multi-scale feature point set from the accurate LiDAR point clouds; A second extraction module is configured to splice the target images to obtain a target panoramic image, and extract an image multi-scale feature point set from the target panoramic image; A matching module is configured to match the point cloud multi-scale feature point set and the image multi-scale feature point set based on a feature descriptor vector to obtain a homonymic point pair set, and divide the homonymic point pair set into a point cloud homonymic point set and an image homonymic point set; A registration module is configured to analyze a rotation matrix and a translation vector by using a quaternion registration algorithm according to the point cloud homonymic point set and the image homonymic point set, and complete registration of the target panoramic image and the accurate LiDAR point clouds according to the rotation matrix and the translation vector.

Images