A new basic surveying and mapping method and system based on multimodal data

Abstract

The present invention proposes a new basic surveying and mapping method and system based on multimodal data, which belongs to the field of surveying and mapping technology; the new basic surveying and mapping method includes: collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data; acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data; performing data fusion on the matched multimodal data to generate fused data, and integrating the fused data to form a comprehensive data set; performing feature extraction and classification recognition on the fused data using a deep learning model, and obtaining surveying and mapping results through the feature extraction and classification recognition results. By fusing multimodal data, the present invention can comprehensively utilize data from different sources to make up for the shortcomings of a single data source in information acquisition, thereby improving the accuracy of surveying and mapping.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping technology, and in particular to a novel basic surveying and mapping method and system based on multimodal data. Background Art

[0002] With the rapid development of remote sensing technology and geographic information system (GIS), surveying and mapping technology is also evolving. Traditional surveying and mapping methods mainly rely on a single data source, such as satellite remote sensing images or ground-based lidar scanning data. Although these data sources can provide rich geographic information, they often cannot fully reflect the complexity and variability of the surface due to their limitations. Therefore, developing a new surveying and mapping method that can comprehensively utilize multiple data sources and improve surveying and mapping accuracy and efficiency has become a hot topic in current research.

[0003] Multimodal data refers to multiple types of data from different sensors, different platforms or different time points. These data have different characteristics and information content, and can more comprehensively reflect the actual situation of the surface. The new basic surveying and mapping method based on multimodal data takes advantage of this and generates more accurate and comprehensive geographic information by collecting, processing, matching and fusing multimodal data.

[0004] At present, surveying and mapping methods based on multimodal data have been applied in some fields, but there are still some challenges and problems to be solved. For example, how to effectively collect and process multimodal data, how to extract useful feature information from it, how to achieve effective matching and fusion of multimodal data, and how to use deep learning models for feature extraction and classification recognition. Summary of the invention

[0005] The present invention provides a novel basic surveying and mapping method and system based on multimodal data to solve the problems mentioned in the above background technology:

[0006] The present invention proposes a new basic surveying and mapping method based on multimodal data, the new basic surveying and mapping method comprising:

[0007] Collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data;

[0008] Acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data;

[0009] Performing data fusion on the matched multimodal data to generate fused data, and integrating the fused data to form a comprehensive data set;

[0010] A deep learning model is used to perform feature extraction and classification recognition on the fused data, and the surveying and mapping results are obtained through the feature extraction and classification recognition results.

[0011] Further, collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data includes:

[0012] Collecting surveying and mapping data of the target area from multiple sources, wherein the surveying and mapping data includes satellite remote sensing images, aerial photography data, and ground-based laser radar scanning data;

[0013] Performing radiation calibration and atmospheric correction on the star remote sensing image to obtain pre-processed first data information;

[0014] Performing geometric correction on the aerial photography data to obtain pre-processed second data information;

[0015] Filtering the ground laser radar scanning data to obtain preprocessed third data information;

[0016] The preprocessed first data information, second data information and third data information are preprocessed multimodal data.

[0017] Further, acquiring feature information from the preprocessed multimodal data and performing data matching to obtain matched multimodal data includes:

[0018] Performing feature extraction on the preprocessed multimodal data to obtain feature information corresponding to the multimodal data, wherein the feature information includes elevation information, slope information, texture information, shape information, area range information and location information of the terrain;

[0019] The feature information corresponding to the multimodal data is matched and associated using a feature matching model to obtain matched multimodal data.

[0020] Furthermore, data fusion is performed on the matched multimodal data to generate fused data, and the fused data is integrated to form a comprehensive data set, including:

[0021] Performing data fusion on the matched multimodal data to generate fused data;

[0022] Performing data enhancement processing on the fused data to obtain fused data after the data enhancement processing;

[0023] The fused data after the data enhancement processing is integrated to form a comprehensive data set.

[0024] Furthermore, the deep learning model is used to perform feature extraction and classification recognition on the fused data, and the surveying and mapping results are obtained through the feature extraction and classification recognition results, including:

[0025] The deep learning model performs feature extraction and classification recognition on the fused data to obtain feature information and ground object recognition results and classification results corresponding to the classification recognition;

[0026] Converting the ground object recognition results and classification results into uniform formats to form ground object recognition results and classification results in uniform formats;

[0027] The object recognition results and classification results in the unified format are integrated into three-terrain models using visualization tools to output a three-dimensional terrain model.

[0028] The present invention proposes a new basic surveying and mapping system based on multimodal data, the new basic surveying and mapping system comprising:

[0029] Data processing module: collects multimodal data and preprocesses the multimodal data to obtain preprocessed multimodal data;

[0030] Data matching module: acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data;

[0031] Data fusion module: performs data fusion on the matched multimodal data to generate fused data, and integrates the fused data to form a comprehensive data set;

[0032] Result acquisition module: Use the deep learning model to perform feature extraction and classification recognition on the fused data, and obtain the surveying and mapping results through the feature extraction and classification recognition results.

[0033] Furthermore, the data processing module includes:

[0034] Data collection module: collects surveying and mapping data from various sources of the target area, wherein the surveying and mapping data includes satellite remote sensing images, aerial photography data, and ground laser radar scanning data;

[0035] The first data module is used to perform radiation calibration and atmospheric correction on the star remote sensing image to obtain pre-processed first data information;

[0036] The second data module is used to perform geometric correction on the aerial photography data to obtain pre-processed second data information;

[0037] The third data module is used to filter the ground laser radar scanning data to obtain the pre-processed third data information;

[0038] Multimodal data module: wherein the preprocessed first data information, second data information and third data information are preprocessed multimodal data.

[0039] Furthermore, the data matching module includes:

[0040] Feature extraction module: extracting features from the preprocessed multimodal data to obtain feature information corresponding to the multimodal data, wherein the feature information includes elevation information, slope information, texture information, shape information, area range information and location information of the terrain;

[0041] Matching and association module: using a feature matching model to match and associate feature information corresponding to the multimodal data to obtain matched multimodal data.

[0042] Furthermore, the data fusion module includes:

[0043] Data generation module: performing data fusion on the matched multimodal data to generate fused data;

[0044] Enhancement processing module: performing data enhancement processing on the fused data to obtain fused data after data enhancement processing;

[0045] Data integration module: integrates the fused data after the data enhancement processing to form a comprehensive data set.

[0046] Furthermore, the result acquisition module includes:

[0047] Classification and recognition module: The deep learning model performs feature extraction and classification and recognition on the fused data to obtain feature information and ground object recognition results and classification results corresponding to the classification and recognition;

[0048] Format conversion module: performing unified format conversion on the ground object recognition results and classification results to form ground object recognition results and classification results in a unified format;

[0049] Model output module: using visualization tools to integrate the object recognition results and classification results in the unified format into three-land models, and output a three-dimensional terrain model.

[0050] Beneficial effects of the present invention: By fusing multimodal data, the present invention can comprehensively utilize data from different sources to make up for the shortcomings of a single data source in information acquisition, thereby improving the accuracy of surveying and mapping. The present invention realizes automated and intelligent data processing and feature extraction, greatly reducing the workload of manual intervention and post-processing, thereby improving the efficiency of surveying and mapping. The deep learning model can automatically extract feature information from the fused data and perform classification and identification, greatly speeding up the speed of surveying and mapping; by integrating multimodal data to form a comprehensive data set, this technical solution can provide more comprehensive and richer geographic information to meet the needs of different fields and different application scenarios. The present invention applies advanced technologies such as deep learning to the field of surveying and mapping, promoting the innovation and development of surveying and mapping technology. At the same time, the present invention also provides technical ideas and solutions that can be used as reference for other fields, promoting the development and progress of related industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the method of the present invention;

[0052] Figure 2 This is a system module diagram of the present invention. DETAILED DESCRIPTION

[0053] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0054] Embodiment 1,

[0055] In this embodiment, Figure 1 As shown, a new basic surveying and mapping method based on multimodal data, the new basic surveying and mapping method includes:

[0056] S1. Collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data;

[0057] S2, acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data;

[0058] S3, performing data fusion on the matched multimodal data to generate fused data, and integrating the fused data to form a comprehensive data set;

[0059] S4. Use a deep learning model to perform feature extraction and classification recognition on the fused data, and obtain surveying and mapping results through the feature extraction and classification recognition results.

[0060] The working principle of the above technical solution is as follows: First, it is necessary to collect multimodal data from different sensors or sources, such as images, lidar data, etc. Then, preprocess these data, including denoising, correction, registration and other operations, to ensure the accuracy and consistency of the data; extract feature information from the preprocessed multimodal data, which may involve image features, spatial features, etc. Then, perform data matching, associate data of different modes, establish corresponding relationships, and facilitate subsequent data fusion and processing; fuse the matched multimodal data into a comprehensive data set. Data fusion can be a simple superposition or a more complex model fusion to obtain more comprehensive information; use a deep learning model to extract features and classify the fused data. Deep learning models can help identify patterns and features in the data, and then realize the classification and identification of objects or targets; by extracting features and classifying the data, the final mapping results can be obtained. These results may include topographic maps, three-dimensional models, target recognition and other information, providing important references for subsequent map making, urban planning, etc.

[0061] The effects of the above technical solutions are as follows: by combining multimodal data sources, more comprehensive and rich information can be obtained, which improves the integrity and accuracy of surveying and mapping data; through feature information extraction and data matching, the related information between different data sources can be better identified and matched, which improves the accuracy and efficiency of data matching; data fusion of the matched multimodal data helps to eliminate the limitations of a single data source, comprehensively utilize the advantages of various types of data, and improve the comprehensiveness and credibility of the data; using deep learning models to extract features and classify the fused data can better explore the hidden features and laws in the data, and improve the accuracy and efficiency of data analysis and identification; through the results of feature extraction and classification identification, more accurate and comprehensive surveying and mapping results can be obtained, providing reliable data support for subsequent map production, resource management and other tasks.

[0062] Embodiment 2,

[0063] In this embodiment, collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data includes:

[0064] Collecting surveying and mapping data of the target area from multiple sources, wherein the surveying and mapping data includes satellite remote sensing images, aerial photography data, and ground laser radar scanning data (LiDAR);

[0065] The star remote sensing image is subjected to radiometric calibration and atmospheric correction to obtain the first data information after preprocessing, so as to eliminate the data deviation caused by the atmospheric effect and the sensor characteristics. The radiometric calibration is the process of converting the original digital count value of the remote sensing sensor into a radiation amount or reflectivity with physical meaning. In this embodiment, the radiometric calibration of the star remote sensing image is completed by using the following formula:

[0066]

[0067] Wherein, L represents the radiation brightness or reflectivity; G represents the calibration coefficient; DN represents the original digital count value of the star remote sensing image; O represents the offset, which represents the digital count value corresponding to when the radiation brightness or reflectivity is zero; a, b and c represent the preset coefficient constants, respectively, the value range of a is 1.12-1.26; the value range of b is 0.57-0.72; the value range of c is 0.28-0.43;

[0068] Atmospheric correction aims to eliminate radiation errors caused by atmospheric absorption and scattering. This usually involves complex physical models and atmospheric parameter estimation, such as dark pixel method, 6S model, etc. And, the corrected surface reflectance (\rho) can be calculated by the following formula:

[0069]

[0070] Where ρ represents the surface reflectivity; d represents the distance between the sun and the earth; ESUN represents the solar irradiance; θ s and θ v denote the solar zenith angle and the observation zenith angle respectively; represents the angle between the sun and the sensor's line of sight; R represents the atmospheric reflectivity, and, represents the ratio of solar radiation reflected by the atmosphere to incident solar radiation under given solar and observation geometry conditions; α represents the compensation coefficient, and the expression of α is as follows:

[0071]

[0072] Among them, ρ s Represents the true reflectivity of the surface; ρ c represents the surface reflectivity detected by the sensor in the satellite remote sensing image; P represents the atmospheric transmittance; Δρ t It represents the atmospheric path radiation change rate corresponding to every two unit times, and the value range of unit time is 1h-3h;

[0073] When the difference between the corrected surface reflectivity and the preset surface reflectivity is less than a preset difference threshold, it is determined that the corrected surface reflectivity meets the requirements and the correction is qualified.

[0074] Performing geometric correction on the aerial photography data to obtain pre-processed second data information to ensure that the ground objects on the image have correct geometric shapes and positions;

[0075] Filtering the ground laser radar scanning data to obtain preprocessed third data information, removing noise and non-ground points, and retaining surface terrain information;

[0076] The preprocessed first data information, second data information and third data information are preprocessed multimodal data.

[0077] The working principle of the above technical solution is as follows: collect satellite remote sensing images, aerial photography data and ground laser radar scanning data (LiDAR) of the target area. These data sources contain information of different angles and resolutions; radiation calibration is to convert the original digital count value into a radiation amount or reflectivity with physical significance, and atmospheric correction is to eliminate the influence of atmospheric effects on the data to ensure the accuracy and comparability of the data; geometric correction is to ensure that the objects on the image have the correct geometric shape and position, and eliminate the distortion caused by factors such as shooting angle and height; filtering processing can remove noise and non-ground points, retain surface terrain information, and extract the true shape and characteristics of the ground; the first, second, and third data information obtained after the above steps are processed are the pre-processed multimodal data, which contain information processed by radiation calibration, atmospheric correction, geometric correction, filtering, etc., and can be used for further analysis and application.

[0078] The effects of the above technical solutions are as follows: integrating satellite remote sensing images, aerial photography data and ground-based laser radar scanning data can obtain all-round and multi-angle target area information, providing a more comprehensive and detailed data basis; by performing radiation calibration and atmospheric correction on satellite remote sensing images, data deviations caused by atmospheric effects and sensor characteristics are eliminated, and the accuracy and reliability of the data are improved; geometric correction of aerial photography data can ensure that the objects on the image have the correct geometric shape and position, providing a reliable basis for subsequent spatial analysis and measurement; filtering the ground-based laser radar scanning data to remove noise and non-ground points, retaining surface terrain information, and improving the quality and clarity of the data; integrating the processed first, second, and third data information to form pre-processed multi-modal data, which integrates information from different sources and has higher comprehensiveness and availability. The data preprocessing method described by the above formula can improve the quality and consistency of multi-modal data, retain key information and ensure that the data meets the preset requirements, and provide a reliable data basis and support for subsequent data analysis and application. At the same time, through radiation calibration and atmospheric correction, data deviations caused by atmospheric effects and sensor characteristics are eliminated, thereby improving the quality of satellite remote sensing images; through geometric correction of aerial photography data, it is ensured that the objects on the image have the correct geometric shape and position; through filtering of ground lidar scanning data, noise and non-ground points are removed and surface terrain information is retained; the data information after preprocessing has a unified standardized processing, ensuring the consistency and comparability between data from different sources; in the data processing process, key data features such as surface terrain information are retained to ensure the validity and availability of the final data; when the difference between the corrected surface reflectivity and the preset surface reflectivity is less than the preset difference threshold, it is judged that the corrected surface reflectivity meets the requirements and the correction is qualified, ensuring the validity and accuracy of data processing.

[0079] Embodiment 3,

[0080] In this embodiment, acquiring feature information from the preprocessed multimodal data and performing data matching to obtain matched multimodal data includes:

[0081] Performing feature extraction on the preprocessed multimodal data to obtain feature information corresponding to the multimodal data, wherein the feature information includes elevation information, slope information, texture information, shape information, area range information and location information of the terrain;

[0082] The feature information corresponding to the multimodal data is matched and associated using a feature matching model to obtain matched multimodal data.

[0083] Wherein, using the feature matching model to match and associate the feature information corresponding to the multimodal data to obtain the matched multimodal data includes:

[0084] Classifying the feature information according to the feature types contained in the feature information to obtain multiple feature information data sets, wherein each feature information data set corresponds to a feature type;

[0085] Using the feature matching model, for each two feature information data sets, obtain a correlation parameter value between a feature vector included in one of the two feature information data sets and each feature vector included in the other feature information data set;

[0086] Taking the feature data in the two feature information data sets with the smallest association parameter value as matching target feature data, and performing association matching on the matching target feature data;

[0087] Wherein, the feature matching model is as follows:

[0088] H=(1-λ 02 )·λ 01

[0089]

[0090]

[0091] Where H represents the correlation parameter value calculated by the feature matching model; λ 01 Represents the first matching parameter; 02 represents the second matching parameter; argmin represents the parameter value of a function that obtains the minimum value in its domain; A represents the feature vector contained in one of the two feature information data sets; B represents the feature vector contained in the other of the two feature information data sets; a i represents the elements contained in the eigenvector A, and i = 1, 2, ... n; b 01 、b 02i , ..., b mi They correspond to the elements contained in the m feature vectors B contained in another feature information data set; B a is the parameter between A and the m feature vectors B contained in another feature information data set The eigenvector corresponding to the minimum value of .

[0092] The working principle of the above technical solution is as follows: first, feature extraction is performed on the preprocessed multimodal data to obtain feature data including terrain elevation, slope, texture, shape, regional range and location. These feature information can be extracted from the original data through various methods and algorithms for subsequent matching and association analysis; according to the feature type contained in the feature information, the extracted feature information is classified to form multiple feature information data sets, each of which corresponds to a feature type. In this way, different types of feature data can be better organized and managed to prepare for the subsequent matching process; the feature matching model is used to match each two feature information data sets, and the associated parameter values ​​between the feature vectors in one of the data sets and all the feature vectors in the other data set are calculated. These parameter values ​​can reflect the similarity or correlation between the feature vectors; the two feature information data sets with the smallest associated parameter values ​​are selected as matching targets, and their feature data are used as matching target feature data. The matching target selected in this way is more likely to represent the best match between the two sets of data, which helps to improve the accuracy of the matching results; the matching target feature data are associated and matched, and they are combined to form the matched multimodal data, so as to achieve the association and integration between different feature information. This will enable more complete and comprehensive multimodal data to be obtained, providing a basis for further data analysis and application.

[0093] The effects of the above technical solution are as follows: by extracting and matching the features of the preprocessed multimodal data, different types of data can be integrated together to form a more comprehensive and integrated multimodal data set, which helps to improve the availability and application scope of the data; by using the feature matching model to match and associate feature information, more accurate data matching can be achieved, ensuring the consistency and accuracy between different data sources, and improving the efficiency and quality of data integration; by classifying feature information according to feature type, it is helpful to better understand and utilize different types of feature data, and improve the pertinence and effectiveness of data processing; by calculating the association parameter value between two feature information data sets, the similarity or correlation between different feature vectors can be quantified, providing a reliable basis for subsequent data matching and integration; selecting the two feature information data sets with the smallest association parameter value as the matching target helps to reduce errors and improve the accuracy of matching, ensuring that the matching results are more reliable and effective. The above feature matching model can effectively process the feature information of multimodal data, achieve data matching and association, and provide important data support and analysis methods for terrain analysis, regional planning and other fields. At the same time, by calculating the associated parameter values ​​between feature vectors, accurate feature matching can be achieved to ensure that the matched data has a high degree of correlation and consistency; the model can process a variety of feature information data sets, including different types of features such as terrain elevation information, slope information, texture information, shape information, etc., to achieve comprehensive matching of multiple feature information; by minimizing the associated parameter values ​​to determine the best matching target feature data, this algorithm can effectively find the most matching feature data, thereby improving the accuracy and efficiency of matching; after associating and matching different feature information data sets, a comprehensive matching result can be obtained, which provides a basis and support for subsequent data analysis and application.

[0094] Embodiment 4,

[0095] In this embodiment, data fusion is performed on the matched multimodal data to generate fused data, and the fused data is integrated to form a comprehensive data set, including:

[0096] Performing data fusion on the matched multimodal data to generate fused data;

[0097] Performing data enhancement processing on the fused data to obtain fused data after data enhancement processing; wherein the data enhancement processing includes but is not limited to interpolation processing, filtering processing and super-resolution processing, so as to improve the resolution and quality of the data and provide a more accurate data basis for subsequent surveying and mapping;

[0098] The fused data after the data enhancement processing is integrated to form a comprehensive data set.

[0099] The working principle of the above technical solution is: data fusion is performed on the matched multimodal data, and data of different modes are integrated together to form fused data. Data fusion can adopt various fusion algorithms and techniques, such as weighted fusion, feature-level fusion or decision-level fusion, to ensure that the fused data can more comprehensively reflect the information of the original data; data enhancement processing is performed on the fused data to improve the resolution and quality of the data. Data enhancement processing includes interpolation processing, filtering processing and super-resolution processing methods, which can effectively improve the details and clarity of the data and provide a more accurate data basis for subsequent mapping and analysis; the fused data after data enhancement processing is integrated into a comprehensive data set. In this step, the data set can be structured and integrated to ensure the consistency and integrity between the data, while providing a more convenient and efficient data basis for subsequent data analysis and application.

[0100] The effects of the above technical solutions are as follows: through data fusion, data from multiple modalities can be integrated together, so that the fused data contains richer information, thereby improving the comprehensive utilization value of the data; the fused data after data enhancement processing can improve the resolution and quality of the data through interpolation processing, filtering processing and super-resolution processing, thereby providing a more accurate data basis for subsequent surveying and analysis; by integrating the fused data after data enhancement processing to form a comprehensive data set, various data types can be effectively integrated, the consistency and integrity of the data can be improved, and a more convenient and efficient data basis can be provided for subsequent data analysis and application; the formation of a comprehensive data set can provide a more reliable and accurate data basis for data analysis and application in various fields, which helps to promote data-driven decision-making and innovation.

[0101] Embodiment 5,

[0102] In this embodiment, a deep learning model is used to perform feature extraction and classification recognition on the fused data, and a surveying and mapping result is obtained through the feature extraction and classification recognition results, including:

[0103] The deep learning model performs feature extraction and classification recognition on the fused data to obtain feature information and ground object recognition results and classification results corresponding to the classification recognition, wherein the ground object recognition results and classification results include but are not limited to recognition and classification results of roads, buildings, water bodies and plants;

[0104] Converting the ground object recognition results and classification results into uniform formats to form ground object recognition results and classification results in uniform formats;

[0105] The object recognition results and classification results in the unified format are integrated into three-terrain models using visualization tools to output a three-dimensional terrain model.

[0106] The working principle of the above technical solution is: using the deep learning model to extract features and classify the fused data. This step can obtain feature information and object classification results, including the identification and classification results of objects such as roads, buildings, water bodies and plants. The deep learning model can automatically learn the features in the data and perform efficient classification and identification; the object identification results and classification results are converted into a unified format to ensure that the identification results and classification results of different categories can be stored and managed in a unified format, which is convenient for subsequent processing and application; the unified format of the object identification results and classification results are integrated into a three-dimensional model using visualization tools to output a three-dimensional terrain model. By integrating the object identification results and classification results, a three-dimensional model of the real terrain can be generated to show the distribution and structure of the objects in space.

[0107] The effects of the above technical solutions are as follows: by using deep learning models to extract features and classify fused data, accurate identification and classification of land objects such as roads, buildings, water bodies and plants can be achieved, thereby improving the accuracy and reliability of surveying and mapping results; by converting the formats of land object identification results and classification results in a unified manner, identification results of different categories can be presented in a unified format, which is convenient for subsequent processing and analysis, and improves the operability and visualization of the data; by using visualization tools to integrate the land object identification results and classification results in a unified format into a three-dimensional model, and outputting a three-dimensional terrain model, the position and relationship of the land objects in space can be intuitively displayed, providing an intuitive reference for terrain analysis and planning; through the application of deep learning models, useful information can be extracted from the fused data more effectively, thereby generating high-quality land object identification results and classification results, providing more reliable data support for decision-making in areas such as map making and urban planning.

[0108] Embodiment 6,

[0109] In this embodiment, Figure 2 As shown, a new basic surveying and mapping system based on multimodal data, the new basic surveying and mapping system comprises:

[0110] Data processing module: collects multimodal data and preprocesses the multimodal data to obtain preprocessed multimodal data;

[0111] Data matching module: acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data;

[0112] Data fusion module: performs data fusion on the matched multimodal data to generate fused data, and integrates the fused data to form a comprehensive data set;

[0113] Result acquisition module: Use the deep learning model to perform feature extraction and classification recognition on the fused data, and obtain the surveying and mapping results through the feature extraction and classification recognition results.

[0114] The working principle of the above technical solution is as follows: First, it is necessary to collect multimodal data from different sensors or sources, such as images, lidar data, etc. Then, preprocess these data, including denoising, correction, registration and other operations, to ensure the accuracy and consistency of the data; extract feature information from the preprocessed multimodal data, which may involve image features, spatial features, etc. Then, perform data matching, associate data of different modes, establish corresponding relationships, and facilitate subsequent data fusion and processing; fuse the matched multimodal data into a comprehensive data set. Data fusion can be a simple superposition or a more complex model fusion to obtain more comprehensive information; use a deep learning model to extract features and classify the fused data. Deep learning models can help identify patterns and features in the data, and then realize the classification and identification of objects or targets; by extracting features and classifying the data, the final mapping results can be obtained. These results may include topographic maps, three-dimensional models, target recognition and other information, providing important references for subsequent map making, urban planning, etc.

[0115] The effects of the above technical solutions are as follows: by combining multimodal data sources, more comprehensive and rich information can be obtained, which improves the integrity and accuracy of surveying and mapping data; through feature information extraction and data matching, the related information between different data sources can be better identified and matched, which improves the accuracy and efficiency of data matching; data fusion of the matched multimodal data helps to eliminate the limitations of a single data source, comprehensively utilize the advantages of various types of data, and improve the comprehensiveness and credibility of the data; using deep learning models to extract features and classify the fused data can better explore the hidden features and laws in the data, and improve the accuracy and efficiency of data analysis and identification; through the results of feature extraction and classification identification, more accurate and comprehensive surveying and mapping results can be obtained, providing reliable data support for subsequent map production, resource management and other tasks.

[0116] Embodiment 7,

[0117] In this embodiment, the data processing module includes:

[0118] Data collection module: collects surveying and mapping data from various sources of the target area, wherein the surveying and mapping data includes satellite remote sensing images, aerial photography data, and ground laser radar scanning data (LiDAR);

[0119] The first data module: performs radiation calibration and atmospheric correction on the star remote sensing image to obtain the first data information after preprocessing; to eliminate the data deviation caused by the atmospheric effect and sensor characteristics; wherein the radiation calibration is the process of converting the original digital count value of the remote sensing sensor into a radiation amount or reflectivity with physical meaning. In this embodiment, the radiation calibration of the star remote sensing image is completed using the following formula:

[0120]

[0121] Wherein, L represents the radiation brightness or reflectivity; G represents the calibration coefficient; DN represents the original digital count value of the star remote sensing image; O represents the offset, which represents the digital count value corresponding to when the radiation brightness or reflectivity is zero; a, b and c represent the preset coefficient constants, respectively, the value range of a is 1.12-1.26; the value range of b is 0.57-0.72; the value range of c is 0.28-0.43;

[0122] Atmospheric correction aims to eliminate radiation errors caused by atmospheric absorption and scattering. This usually involves complex physical models and atmospheric parameter estimation, such as dark pixel method, 6S model, etc. And, the corrected surface reflectance (\rho) can be calculated by the following formula:

[0123]

[0124] Where ρ represents the surface reflectivity; d represents the distance between the sun and the earth; ESUN represents the solar irradiance; θ s and θ v denote the solar zenith angle and the observation zenith angle respectively; represents the angle between the sun and the sensor's line of sight; R represents the atmospheric reflectivity, and, represents the ratio of solar radiation reflected by the atmosphere to incident solar radiation under given solar and observation geometry conditions; α represents the compensation coefficient, and the expression of α is as follows:

[0125]

[0126] Among them, ρ s Represents the true reflectivity of the surface; ρ c represents the surface reflectivity detected by the sensor in the satellite remote sensing image; P represents the atmospheric transmittance; Δρ t It represents the atmospheric path radiation change rate corresponding to every two unit times, and the value range of unit time is 1h-3h;

[0127] When the difference between the corrected surface reflectivity and the preset surface reflectivity is less than a preset difference threshold, it is determined that the corrected surface reflectivity meets the requirements and the correction is qualified.

[0128] The second data module is used to perform geometric correction on the aerial photography data to obtain pre-processed second data information to ensure that the ground objects on the image have correct geometric shapes and positions;

[0129] The third data module is used to filter the ground laser radar scanning data to obtain pre-processed third data information, remove noise and non-ground points, and retain surface terrain information;

[0130] Multimodal data module: wherein the preprocessed first data information, second data information and third data information are preprocessed multimodal data.

[0131] The working principle of the above technical solution is as follows: collect satellite remote sensing images, aerial photography data and ground laser radar scanning data (LiDAR) of the target area. These data sources contain information of different angles and resolutions; radiation calibration is to convert the original digital count value into a radiation amount or reflectivity with physical significance, and atmospheric correction is to eliminate the influence of atmospheric effects on the data to ensure the accuracy and comparability of the data; geometric correction is to ensure that the objects on the image have the correct geometric shape and position, and eliminate the distortion caused by factors such as shooting angle and height; filtering processing can remove noise and non-ground points, retain surface terrain information, and extract the true shape and characteristics of the ground; the first, second, and third data information obtained after the above steps are processed are the pre-processed multimodal data, which contain information processed by radiation calibration, atmospheric correction, geometric correction, filtering, etc., and can be used for further analysis and application.

[0132] The effects of the above technical solutions are as follows: integrating satellite remote sensing images, aerial photography data and ground-based laser radar scanning data can obtain all-round and multi-angle target area information, providing a more comprehensive and detailed data basis; by performing radiation calibration and atmospheric correction on satellite remote sensing images, data deviations caused by atmospheric effects and sensor characteristics are eliminated, and the accuracy and reliability of the data are improved; geometric correction of aerial photography data can ensure that the objects on the image have the correct geometric shape and position, providing a reliable basis for subsequent spatial analysis and measurement; filtering the ground-based laser radar scanning data to remove noise and non-ground points, retaining surface terrain information, and improving the quality and clarity of the data; integrating the processed first, second, and third data information to form pre-processed multi-modal data, which integrates information from different sources and has higher comprehensiveness and availability. The data preprocessing method described by the above formula can improve the quality and consistency of multi-modal data, retain key information and ensure that the data meets the preset requirements, and provide a reliable data basis and support for subsequent data analysis and application. At the same time, through radiation calibration and atmospheric correction, data deviations caused by atmospheric effects and sensor characteristics are eliminated, thereby improving the quality of satellite remote sensing images; through geometric correction of aerial photography data, it is ensured that the objects on the image have the correct geometric shape and position; through filtering of ground lidar scanning data, noise and non-ground points are removed and surface terrain information is retained; the data information after preprocessing has a unified standardized processing, ensuring the consistency and comparability between data from different sources; in the data processing process, key data features such as surface terrain information are retained to ensure the validity and availability of the final data; when the difference between the corrected surface reflectivity and the preset surface reflectivity is less than the preset difference threshold, it is judged that the corrected surface reflectivity meets the requirements and the correction is qualified, ensuring the validity and accuracy of data processing.

[0133] Embodiment 8,

[0134] In this embodiment, the data matching module includes:

[0135] Feature extraction module: extracting features from the preprocessed multimodal data to obtain feature information corresponding to the multimodal data, wherein the feature information includes elevation information, slope information, texture information, shape information, area range information and location information of the terrain;

[0136] Matching and association module: using a feature matching model to match and associate feature information corresponding to the multimodal data to obtain matched multimodal data.

[0137] Wherein, using the feature matching model to match and associate the feature information corresponding to the multimodal data to obtain the matched multimodal data includes:

[0138] Classifying the feature information according to the feature types contained in the feature information to obtain multiple feature information data sets, wherein each feature information data set corresponds to a feature type;

[0139] Using the feature matching model, for each two feature information data sets, obtain a correlation parameter value between a feature vector included in one of the two feature information data sets and each feature vector included in the other feature information data set;

[0140] Taking the feature data in the two feature information data sets with the smallest association parameter value as matching target feature data, and performing association matching on the matching target feature data;

[0141] Wherein, the feature matching model is as follows:

[0142] H=(1-λ 02 )·λ 01

[0143]

[0144]

[0145] Where H represents the correlation parameter value calculated by the feature matching model; λ 01 Represents the first matching parameter; 02 represents the second matching parameter; argmin represents the parameter value of a function that obtains the minimum value in its domain; A represents the feature vector contained in one of the two feature information data sets; B represents the feature vector contained in the other of the two feature information data sets; a i represents the elements contained in the eigenvector A, and i = 1, 2, ... n; b 01 、b 02i , ..., b mi They correspond to the elements contained in the m feature vectors B contained in another feature information data set; B a is the parameter between A and the m feature vectors B contained in another feature information data set The eigenvector corresponding to the minimum value of .

[0146] The working principle of the above technical solution is as follows: first, feature extraction is performed on the preprocessed multimodal data to obtain feature data including terrain elevation, slope, texture, shape, regional range and location. These feature information can be extracted from the original data through various methods and algorithms for subsequent matching and association analysis; according to the feature type contained in the feature information, the extracted feature information is classified to form multiple feature information data sets, each of which corresponds to a feature type. In this way, different types of feature data can be better organized and managed to prepare for the subsequent matching process; the feature matching model is used to match each two feature information data sets, and the associated parameter values ​​between the feature vectors in one of the data sets and all the feature vectors in the other data set are calculated. These parameter values ​​can reflect the similarity or correlation between the feature vectors; the two feature information data sets with the smallest associated parameter values ​​are selected as matching targets, and their feature data are used as matching target feature data. The matching target selected in this way is more likely to represent the best match between the two sets of data, which helps to improve the accuracy of the matching results; the matching target feature data are associated and matched, and they are combined to form the matched multimodal data, so as to achieve the association and integration between different feature information. This will enable more complete and comprehensive multimodal data to be obtained, providing a basis for further data analysis and application.

[0147] The effects of the above technical solution are as follows: by extracting and matching the features of the preprocessed multimodal data, different types of data can be integrated together to form a more comprehensive and integrated multimodal data set, which helps to improve the availability and application scope of the data; by using the feature matching model to match and associate feature information, more accurate data matching can be achieved, ensuring the consistency and accuracy between different data sources, and improving the efficiency and quality of data integration; by classifying feature information according to feature type, it is helpful to better understand and utilize different types of feature data, and improve the pertinence and effectiveness of data processing; by calculating the association parameter value between two feature information data sets, the similarity or correlation between different feature vectors can be quantified, providing a reliable basis for subsequent data matching and integration; selecting the two feature information data sets with the smallest association parameter value as the matching target helps to reduce errors and improve the accuracy of matching, ensuring that the matching results are more reliable and effective. The above feature matching model can effectively process the feature information of multimodal data, achieve data matching and association, and provide important data support and analysis methods for terrain analysis, regional planning and other fields. At the same time, by calculating the associated parameter values ​​between feature vectors, accurate feature matching can be achieved to ensure that the matched data has a high degree of correlation and consistency; the model can process a variety of feature information data sets, including different types of features such as terrain elevation information, slope information, texture information, shape information, etc., to achieve comprehensive matching of multiple feature information; by minimizing the associated parameter values ​​to determine the best matching target feature data, this algorithm can effectively find the most matching feature data, thereby improving the accuracy and efficiency of matching; after associating and matching different feature information data sets, a comprehensive matching result can be obtained, which provides a basis and support for subsequent data analysis and application.

[0148] Embodiment 9,

[0149] In this embodiment, the data fusion module includes:

[0150] Data generation module: performing data fusion on the matched multimodal data to generate fused data;

[0151] Enhancement processing module: performing data enhancement processing on the fused data to obtain fused data after data enhancement processing; wherein the data enhancement processing includes but is not limited to interpolation processing, filtering processing and super-resolution processing, so as to improve the resolution and quality of the data and provide a more accurate data basis for subsequent surveying and mapping;

[0152] Data integration module: integrates the fused data after the data enhancement processing to form a comprehensive data set.

[0153] The working principle of the above technical solution is: data fusion is performed on the matched multimodal data, and data of different modes are integrated together to form fused data. Data fusion can adopt various fusion algorithms and techniques, such as weighted fusion, feature-level fusion or decision-level fusion, to ensure that the fused data can more comprehensively reflect the information of the original data; data enhancement processing is performed on the fused data to improve the resolution and quality of the data. Data enhancement processing includes interpolation processing, filtering processing and super-resolution processing methods, which can effectively improve the details and clarity of the data and provide a more accurate data basis for subsequent mapping and analysis; the fused data after data enhancement processing is integrated into a comprehensive data set. In this step, the data set can be structured and integrated to ensure the consistency and integrity between the data, while providing a more convenient and efficient data basis for subsequent data analysis and application.

[0154] The effects of the above technical solutions are as follows: through data fusion, data from multiple modalities can be integrated together, so that the fused data contains richer information, thereby improving the comprehensive utilization value of the data; the fused data after data enhancement processing can improve the resolution and quality of the data through interpolation processing, filtering processing and super-resolution processing, thereby providing a more accurate data basis for subsequent surveying and analysis; by integrating the fused data after data enhancement processing to form a comprehensive data set, various data types can be effectively integrated, the consistency and integrity of the data can be improved, and a more convenient and efficient data basis can be provided for subsequent data analysis and application; the formation of a comprehensive data set can provide a more reliable and accurate data basis for data analysis and application in various fields, which helps to promote data-driven decision-making and innovation.

[0155] Embodiment 10

[0156] In this embodiment, the result acquisition module includes:

[0157] Classification and recognition module: The deep learning model performs feature extraction and classification and recognition on the fused data, and obtains feature information and ground object recognition and classification results corresponding to the classification and recognition, wherein the ground object recognition and classification results include but are not limited to the recognition and classification results of roads, buildings, water bodies and plants;

[0158] Format conversion module: performing unified format conversion on the ground object recognition results and classification results to form ground object recognition results and classification results in a unified format;

[0159] Model output module: using visualization tools to integrate the object recognition results and classification results in the unified format into three-land models, and output a three-dimensional terrain model.

[0160] The working principle of the above technical solution is: using the deep learning model to extract features and classify the fused data. This step can obtain feature information and object classification results, including the identification and classification results of objects such as roads, buildings, water bodies and plants. The deep learning model can automatically learn the features in the data and perform efficient classification and identification; the object identification results and classification results are converted into a unified format to ensure that the identification results and classification results of different categories can be stored and managed in a unified format, which is convenient for subsequent processing and application; the unified format of the object identification results and classification results are integrated into a three-dimensional model using visualization tools to output a three-dimensional terrain model. By integrating the object identification results and classification results, a three-dimensional model of the real terrain can be generated to show the distribution and structure of the objects in space.

[0161] The effects of the above technical solutions are as follows: by using deep learning models to extract features and classify fused data, accurate identification and classification of land objects such as roads, buildings, water bodies and plants can be achieved, thereby improving the accuracy and reliability of surveying and mapping results; by converting the formats of land object identification results and classification results in a unified manner, identification results of different categories can be presented in a unified format, which is convenient for subsequent processing and analysis, and improves the operability and visualization of the data; by using visualization tools to integrate the land object identification results and classification results in a unified format into a three-dimensional model, and outputting a three-dimensional terrain model, the position and relationship of the land objects in space can be intuitively displayed, providing an intuitive reference for terrain analysis and planning; through the application of deep learning models, useful information can be extracted from the fused data more effectively, thereby generating high-quality land object identification results and classification results, providing more reliable data support for decision-making in areas such as map making and urban planning.

[0162] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A new basic surveying and mapping method based on multimodal data, characterized in that: The novel basic surveying and mapping method comprises: Collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data; Acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data; Performing data fusion on the matched multimodal data to generate fused data, and integrating the fused data to form a comprehensive data set; Using a deep learning model to perform feature extraction and classification recognition on the fused data, and obtaining surveying and mapping results through the feature extraction and classification recognition results; Acquiring feature information from the preprocessed multimodal data and performing data matching to obtain matched multimodal data includes: Performing feature extraction on the preprocessed multimodal data to obtain feature information corresponding to the multimodal data, wherein the feature information includes elevation information, slope information, texture information, shape information, area range information and location information of the terrain; Using a feature matching model to match and associate feature information corresponding to the multimodal data to obtain matched multimodal data; Wherein, using the feature matching model to match and associate the feature information corresponding to the multimodal data to obtain the matched multimodal data includes: Classifying the feature information according to the feature types contained in the feature information to obtain multiple feature information data sets, wherein each feature information data set corresponds to a feature type; Using the feature matching model, for each two feature information data sets, obtain a correlation parameter value between a feature vector included in one of the two feature information data sets and each feature vector included in the other feature information data set; Taking the feature data in the two feature information data sets with the smallest association parameter value as matching target feature data, and performing association matching on the matching target feature data; Wherein, the feature matching model is as follows: H=(1-λ 02 )·l 01 Where H represents the correlation parameter value calculated by the feature matching model; λ 01 Represents the first matching parameter; 02 represents the second matching parameter; argmin represents the parameter value of a function that obtains the minimum value in its domain; A represents the feature vector contained in one of the two feature information data sets; B represents the feature vector contained in the other of the two feature information data sets; a i represents the elements contained in the eigenvector A, and i = 1, 2, ... n; b 01 、b 02i , ..., b mi They correspond to the elements contained in the m feature vectors B contained in another feature information data set; B a is the parameter between A and the m feature vectors B contained in another feature information data set The eigenvector corresponding to the minimum value of .

2. The novel basic surveying and mapping method based on multimodal data according to claim 1, characterized in that: Collecting multimodal data and preprocessing the multimodal data to obtain preprocessed multimodal data includes: Collecting surveying and mapping data of the target area from multiple sources, wherein the surveying and mapping data includes satellite remote sensing images, aerial photography data, and ground-based laser radar scanning data; Performing radiation calibration and atmospheric correction on the star remote sensing image to obtain pre-processed first data information; Performing geometric correction on the aerial photography data to obtain pre-processed second data information; Filtering the ground laser radar scanning data to obtain pre-processed third data information; The preprocessed first data information, second data information and third data information are preprocessed multimodal data.

3. The novel basic surveying and mapping method based on multimodal data according to claim 1, characterized in that: Performing data fusion on the matched multimodal data to generate fused data, and integrating the fused data to form a comprehensive data set, including: Performing data fusion on the matched multimodal data to generate fused data; Performing data enhancement processing on the fused data to obtain fused data after the data enhancement processing; The fused data after the data enhancement processing is integrated to form a comprehensive data set.

4. The novel basic surveying and mapping method based on multimodal data according to claim 1, characterized in that: The deep learning model is used to perform feature extraction and classification recognition on the fused data, and the surveying and mapping results are obtained through the feature extraction and classification recognition results, including: The deep learning model performs feature extraction and classification recognition on the fused data to obtain feature information and ground object recognition results and classification results corresponding to the classification recognition; Converting the ground object recognition results and classification results into uniform formats to form ground object recognition results and classification results in uniform formats; The object recognition results and classification results in the unified format are integrated into three-terrain models using visualization tools to output a three-dimensional terrain model.

5. A new basic surveying and mapping system based on multimodal data, characterized in that: The new basic surveying and mapping system comprises: Data processing module: collects multimodal data and preprocesses the multimodal data to obtain preprocessed multimodal data; Data matching module: acquiring feature information from the preprocessed multimodal data, and performing data matching to obtain matched multimodal data; Data fusion module: performs data fusion on the matched multimodal data to generate fused data, and integrates the fused data to form a comprehensive data set; Result acquisition module: using a deep learning model to perform feature extraction and classification recognition on the fusion data, and obtaining the surveying and mapping results through the feature extraction and classification recognition results; The data matching module comprises: Feature extraction module: extracting features from the preprocessed multimodal data to obtain feature information corresponding to the multimodal data, wherein the feature information includes elevation information, slope information, texture information, shape information, area range information and location information of the terrain; Matching and associating module: using a feature matching model to match and associate feature information corresponding to the multimodal data to obtain matched multimodal data; Wherein, using the feature matching model to match and associate the feature information corresponding to the multimodal data to obtain the matched multimodal data includes: Classifying the feature information according to the feature types contained in the feature information to obtain multiple feature information data sets, wherein each feature information data set corresponds to a feature type; Using the feature matching model, for each two feature information data sets, obtain a correlation parameter value between a feature vector included in one of the two feature information data sets and each feature vector included in the other feature information data set; Taking the feature data in the two feature information data sets with the smallest association parameter value as matching target feature data, and performing association matching on the matching target feature data; Wherein, the feature matching model is as follows: H=(1-λ 02 )·l 01 Where H represents the correlation parameter value calculated by the feature matching model; λ 01 Represents the first matching parameter; 02 represents the second matching parameter; argmin represents the parameter value of a function that obtains the minimum value in its domain; A represents the feature vector contained in one of the two feature information data sets; B represents the feature vector contained in the other of the two feature information data sets; a i represents the elements contained in the eigenvector A, and i = 1, 2, ... n; b 01 、b 02i , ..., b mi They correspond to the elements contained in the m feature vectors B contained in another feature information data set; B a is the parameter between A and the m feature vectors B contained in another feature information data set The eigenvector corresponding to the minimum value of .

6. The novel basic surveying and mapping system based on multimodal data according to claim 5 is characterized in that: The data processing module comprises: Data collection module: collects surveying and mapping data from various sources of the target area, wherein the surveying and mapping data includes satellite remote sensing images, aerial photography data, and ground laser radar scanning data; The first data module is used to perform radiation calibration and atmospheric correction on the star remote sensing image to obtain pre-processed first data information; The second data module is used to perform geometric correction on the aerial photography data to obtain pre-processed second data information; The third data module is used to filter the ground laser radar scanning data to obtain the pre-processed third data information; Multimodal data module: wherein the preprocessed first data information, second data information and third data information are preprocessed multimodal data.

7. The novel basic surveying and mapping system based on multimodal data according to claim 5, characterized in that: The data fusion module comprises: Data generation module: performing data fusion on the matched multimodal data to generate fused data; Enhancement processing module: performing data enhancement processing on the fused data to obtain fused data after data enhancement processing; Data integration module: integrates the fused data after the data enhancement processing to form a comprehensive data set.

8. The novel basic surveying and mapping system based on multimodal data according to claim 5, characterized in that: The result acquisition module comprises: Classification and recognition module: The deep learning model performs feature extraction and classification and recognition on the fused data to obtain feature information and ground object recognition results and classification results corresponding to the classification and recognition; Format conversion module: performing unified format conversion on the ground object recognition results and classification results to form ground object recognition results and classification results in a unified format; Model output module: using visualization tools to integrate the object recognition results and classification results in the unified format into three-land models, and output a three-dimensional terrain model.

Images