Multi-source spatial data fusion processing system and method

Abstract

The invention discloses a multi-source spatial data fusion processing system and method, relates to the technical field of spatial data processing, and solves the problems that spectrum and spatial features extracted from satellite image data are difficult to fuse, multiple features of satellite images and aerial camera data are difficult to integrate to carry out target recognition in a complex environment, and the target recognition efficiency is high. The technical problem of lack of organic fusion of three different modalities of data of satellite images, aerial photography and point cloud data is solved. Comprising the following modules: a multi-source data acquisition module used for acquiring spatial data of different data sources; the data preprocessing module is used for preprocessing the collected original spatial data; the feature extraction and model construction module is used for extracting features of satellite images and aerial camera data, respectively constructing classifiers to generate classification results, and extracting point cloud data features; and the data fusion module is used for performing target identification by using the fused features and outputting a high-precision target identification result.

Description

Technical Field

[0001] The present invention relates to the technical field of spatial data processing, and specifically to a multi-source spatial data fusion processing system and method. Background Art

[0002] With the rapid development of technologies such as aerospace, satellite remote sensing, and geographic information systems, the means of obtaining spatial data have become increasingly rich, including various methods such as satellite images, aerial photography, and ground surveys. However, spatial data from different sources vary in data format, coordinate system, resolution, etc., which brings great difficulties to the comprehensive utilization of data.

[0003] Traditional multi-source spatial data fusion methods often have problems such as low fusion efficiency and poor accuracy, and cannot meet the requirements of practical applications. It is difficult to fuse the spectral and spatial features extracted from satellite image data to comprehensively and accurately describe ground objects, and it is also difficult to comprehensively utilize various features of satellite image data and aerial photography data for target recognition in complex environments. There is a lack of technology for organically fusing three different modalities of data, namely satellite images, aerial photography, and point cloud data, to improve the efficiency and accuracy of data fusion. Summary of the Invention

[0004] Traditional multi-source spatial data fusion methods often have problems such as low fusion efficiency and poor accuracy, and cannot meet the requirements of practical applications. It is difficult to fuse the spectral and spatial features extracted from satellite image data to comprehensively and accurately describe ground objects, and it is also difficult to comprehensively utilize various features of satellite image data and aerial photography data for target recognition in complex environments. There is a lack of technology for organically fusing three different modalities of data, namely satellite images, aerial photography, and point cloud data, to improve the efficiency and accuracy of data fusion.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A multi-source spatial data fusion processing system includes the following modules: Multi-source data acquisition module: used to collect spatial data from different data sources, including satellite image data, aerial photography data, and point cloud data; Data preprocessing module: performs preprocessing operations on the collected original spatial data, including data format conversion, coordinate system unification, and data denoising processing; Feature extraction and model construction module: extracts the features of satellite image data, constructs a classifier to generate a preliminary classification result, extracts the features of aerial photography data, constructs a multi-branch deep learning model to output a classification result, and extracts the features of point cloud data; Data fusion module: based on the existing multi-branch deep learning model, adds a point cloud data branch, uses the fused features for target recognition, and outputs a high-precision target recognition result.

[0006] Further, the method for collecting spatial data from different data sources, including satellite image data, aerial photography data, and point cloud data, includes the following steps: Start the satellite image acquisition process, collect satellite image data, transmit the collected data to the device for preliminary sorting and storage to generate original satellite image data; according to the comprehensive factors of the shooting target area, plan the flight route, use an aerial photography camera to shoot the target area, download the aerial photography data to the storage device, and label the downloaded aerial photography data, including shooting time, location, flight altitude, and camera parameter information; select appropriate control points around and inside the target area, use ground survey instruments to measure the control points to obtain the three-dimensional coordinates of the corresponding control points, and based on the control points, measure the feature points in the target area, where the feature points include terrain points and ground object corner points, create a unique identifier for each measurement point data, and generate point cloud data.

[0007] Further, the preprocessing operations on the collected original spatial data, including data format conversion, coordinate system unification, and data denoising processing, include the following steps: Use remote sensing image processing software, image editing software, and Python libraries to perform format conversion on the original satellite image data, original aerial photography data, and point cloud data respectively; Import the format-converted satellite image data into remote sensing image processing software for coordinate conversion to unify it to the WGS84 coordinate system; determine the original coordinate system corresponding to the original data of the format-converted aerial photography data, and perform conversion on the original coordinate system according to the relationship between the original coordinate system and the WGS84 coordinate system; determine the original coordinate system of the format-converted point cloud data, obtain the conversion parameters from the original coordinate system to WGS84, and save the converted data; Use the filtering tool in the remote sensing image processing software to perform filtering processing on the satellite image data; respectively use image enhancement algorithms and photogrammetry software to perform denoising processing on sensor noise and lens distortion noise; use Gaussian filtering to remove noise in the point cloud data.

[0008] Further, the method for extracting the features of satellite image data, constructing a classifier to generate a preliminary classification result, includes the following steps: Perform mathematical operations on each band of satellite imagery to extract the spectral characteristics of specific ground objects. After selecting the target area, extract the reflectance of this area in different bands to generate a spectral curve. Analyze the spectral characteristics of ground objects and calculate various spectral indices, including the normalized difference vegetation index, water index, and built-up index. Use a classifier trained with known ground object samples to automatically classify the satellite imagery. At the same time, extract the spectral characteristics of different ground objects, reduce the dimensionality of the multi-bands, and extract the main spectral characteristics; perform resampling processing on the satellite imagery data, use an edge detection algorithm to extract the ground object boundaries, calculate the texture characteristics of the satellite imagery, and then fuse the extracted spectral and spatial characteristics to form a high-dimensional feature vector, which is input into a support vector machine to train the classifier. Use the trained classifier to perform target recognition on the satellite imagery data and generate a preliminary classification result.

[0009] Further, the steps of extracting the features of aerial photography data and constructing a multi-branch deep learning model to output a classification result include the following: Perform resampling processing on the aerial photography data to ensure that its resolution is consistent with that of the satellite imagery data. Use a color histogram to extract color features from the aerial photography data. At the same time, use a gray-level co-occurrence matrix to extract texture features, use an edge detection algorithm to extract shape features, and use an object segmentation algorithm to extract the ground object contours. Extract the height information of the aerial photography data to generate three-dimensional features. Construct a multi-branch deep learning model. Take the satellite imagery data and the aerial photography data as inputs of different branches. The satellite imagery branch inputs spectral features and spatial features, and the aerial photography branch inputs high-resolution spatial features and three-dimensional features. At the last layer of the network, fuse the features of the two branches and use the fused features for target recognition to output a classification result.

[0010] Further, the steps of extracting the features of point cloud data include the following: Perform denoising, filtering, and removal of invalid points on the point cloud data. Extract the coordinate information of the point cloud data, calculate the elevation value of each point, and obtain the slope value and aspect of each point through gradient calculation methods. Classify the point cloud according to the coordinate, color, and intensity information of the point cloud. According to different categories of point clouds, extract their specific attribute information; use the preprocessed point cloud data to generate a three-dimensional model, extract the geometric and topological features of the three-dimensional model, and calculate the spatial distribution features of the point cloud data.

[0011] Further, based on the existing multi-branch deep learning model, add a point cloud data branch and use the fused features for target recognition to output a high-precision target recognition result. The steps include the following: Based on the existing multi-branch deep learning model, a point cloud data branch is added. The geometric features and attribute features of the point cloud data are input into the point cloud data branch, and the features of the three branches are fused at the last layer of the network. The features of satellite images, aerial photography, and point cloud data are stitched together into a high-dimensional feature vector. The attention mechanism is used to dynamically weight the features of different data sources, and feature fusion is performed at different network levels. The fused features are used for target recognition, and high-precision target recognition results are output.

[0012] The present invention also provides a multi-source spatial data fusion processing method, including the following steps: S1: Used to collect spatial data from different data sources, including satellite image data, aerial photography data, and point cloud data; S2: Perform preprocessing operations on the collected original spatial data, including data format conversion, coordinate system unification, and data denoising processing; S3: Extract the features of satellite image data, construct a classifier to generate a preliminary classification result, extract the features of aerial photography data, construct a multi-branch deep learning model to output a classification result, and extract the features of point cloud data; S4: Based on the existing multi-branch deep learning model, add a point cloud data branch, use the fused features for target recognition, and output high-precision target recognition results.

[0013] Advantages of the present invention: By connecting the output feature vectors of the satellite image branch, aerial photography branch, and point cloud data branch, the present invention forms a new fused feature vector, realizes the fusion of features from different data sources, reduces the limitations and uncertainties of a single data source, provides a richer basis for target recognition, introduces an attention mechanism between different branches to dynamically adjust the weights of features from different data sources, inputs the fused features into the target recognition module, and outputs the target recognition results in a visual form, realizing more effective utilization of the contributions of different data sources, improving the quality of the fused features, and the accuracy of target recognition. Description of the Drawings

[0014] Figure 1 It is a flow block diagram of a multi-source spatial data fusion processing system of the present invention; Figure 2 It is a flow block diagram of a multi-source spatial data fusion processing method of the present invention. Detailed Embodiments

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0016] Please refer to Figure 1 As shown, the present invention is a multi-source spatial data fusion processing system, including the following modules: Multi-source data acquisition module: used to collect spatial data from different data sources, including satellite image data, aerial photography data, and point cloud data; Data preprocessing module: perform preprocessing operations on the collected original spatial data, including data format conversion, coordinate system unification, and data denoising processing; Feature extraction and model construction module: extract the features of satellite image data, construct a classifier to generate a preliminary classification result, extract the features of aerial photography data, construct a multi-branch deep learning model to output a classification result, and extract the features of point cloud data; Data fusion module: based on the existing multi-branch deep learning model, add a point cloud data branch, use the fused features for target recognition, and output a high-precision target recognition result.

[0017] Specifically, start the image acquisition device and begin to collect satellite image data according to the set parameters. The aerial photography camera takes pictures at set time intervals or distance intervals. Meanwhile, the navigation system and positioning system on the aircraft record the position and attitude information of the aircraft in real time. Create a unique identifier for each data, and integrate the three-dimensional coordinate data of the measured control points and feature points with the corresponding identifier to form point cloud data. Start the remote sensing image processing software, import the satellite image data after format conversion, find the coordinate conversion tool in the software, select the target coordinate system as WGS84, determine the original coordinate system of the aerial photography data, calculate the coordinate conversion parameters according to the difference between the original coordinate system and the WGS84 coordinate system, and perform coordinate conversion on the aerial photography data according to the calculated conversion parameters. Determine the original coordinate system of the point cloud data, and then obtain the conversion parameters from the original coordinate system to WGS84 according to the relationship between the original coordinate system and the WGS84 coordinate system. Combine the standardized spectral features and spatial features into a feature vector, use the trained SVM classifier, take the fused feature vector as the input data to construct a target recognition model, and output the recognition results of the corresponding ground object categories or target objects. Construct another deep learning branch to process the aerial photography data. The input layer of this branch receives high-resolution spatial features and three-dimensional features as inputs, and set a feature fusion layer at the last layer of the network to fuse the output features of the satellite image branch and the aerial photography branch. Use the point cloud reconstruction algorithm to convert the preprocessed point cloud data into a triangular mesh model. Keep the input of the original satellite image branch unchanged, and also keep the input of the aerial photography branch unchanged. Create a new point cloud data branch, take the geometric features and attribute features of the previously extracted point cloud data as inputs, use the fused features for target recognition, and output high-precision target recognition results.

[0018] In one embodiment of the present invention, the method for collecting spatial data from different data sources, including satellite image data, aerial photography data, and point cloud data, comprises the following steps: Start the satellite image acquisition process, collect the satellite image data, transmit the collected data to the device for preliminary sorting and storage to generate the original satellite image data; according to the comprehensive factors of the shooting target area, plan the flight route, use the aerial photography camera to shoot the target area, download the aerial photography data to the storage device, and label the downloaded aerial photography data, including shooting time, location, flight altitude, and camera parameter information; select appropriate control points around and inside the target area, use ground measuring instruments to measure the control points to obtain the three-dimensional coordinates of the corresponding control points, and based on the control points, measure the feature points in the target area. Among them, the feature points include terrain points and ground object corner points. Create a unique identifier for each measurement point data to generate point cloud data.

[0019] Specifically, according to the predetermined flight orbit and the target shooting area, the operating orbit of the satellite is accurately calibrated. After the satellite enters the predetermined orbit, the ground control center monitors the operating state of the satellite in real time. Meanwhile, the image acquisition device is activated, and satellite image data is collected according to the set parameters. The collected satellite image data is transmitted back to the ground receiving station in real time through the communication system on the satellite. After receiving the data transmitted by the satellite, the ground receiving station first verifies the data and preliminarily sorts out the received original satellite image data, including classification and storage according to information such as shooting time, orbit number, and image coordinates. The sorted original satellite image data is stored in a large-capacity storage device; according to the range and shape of the target shooting area, professional route planning software is used to design the flight route, and the aircraft flies according to the planned route. During the flight, the aerial photography camera takes pictures at set time intervals or distance intervals. Meanwhile, the navigation system and positioning system on the aircraft record the position and attitude information of the aircraft in real time. After the aircraft completes the shooting task, the aerial photography data is downloaded to the storage device, and during the download process, the data is marked, including shooting time, location, flight altitude, and camera parameter information; appropriate control points are selected around and inside the target area, such as road intersections and building corners, and ground survey instruments are used to accurately measure the control points to obtain the three-dimensional coordinates of the corresponding control points. Based on the control points, the feature points in the target area are measured. The feature points include terrain points and ground object corner points. Ground survey instruments are used to measure the selected feature points to obtain their three-dimensional coordinates. Meanwhile, a unique identifier is created for each data, and the three-dimensional coordinate data of the measured control points and feature points is integrated with the corresponding identifiers to form point cloud data.

[0020] In one embodiment of the present invention, the preprocessing operation on the collected original spatial data, including data format conversion, coordinate system unification, and data denoising processing, comprises the following steps: Use remote sensing image processing software, image editing software, and Python library to perform format conversion on the original satellite image data, original aerial photography data, and point cloud data respectively; Import the format-converted satellite image data into remote sensing image processing software for coordinate conversion to unify it into the WGS84 coordinate system; determine the original coordinate system corresponding to the original data of the format-converted aerial photography data, and convert the original coordinate system according to the relationship between the original coordinate system and the WGS84 coordinate system; determine the original coordinate system of the format-converted point cloud data, obtain the conversion parameters from the original coordinate system to WGS84, and save the converted data; Filter the satellite image data using the filtering tool in the remote sensing image processing software; use the image enhancement algorithm and photogrammetry software to denoise the sensor noise and lens distortion noise respectively; use Gaussian filtering to remove the noise in the point cloud data.

[0021] Specifically, start the remote sensing image processing software, import the satellite image data after format conversion, find the coordinate conversion tool in the software, and select the target coordinate system as WGS84. The software will automatically perform the conversion according to the relationship between the original coordinate system and the target coordinate system; determine its original coordinate system by viewing the metadata or relevant documents of the aerial photography data, calculate the coordinate conversion parameters according to the difference between the original coordinate system and the WGS84 coordinate system, and use the coordinate conversion tool in the remote sensing image processing software or geographic information system software to perform coordinate conversion on the aerial photography data according to the calculated conversion parameters; determine its original coordinate system by viewing the metadata of the point cloud data or using relevant professional software, and then, according to the relationship between the original coordinate system and the WGS84 coordinate system, obtain the conversion parameters from the original coordinate system to WGS84, use professional point cloud processing software or library to perform coordinate conversion on the point cloud data according to the obtained conversion parameters, and use the Gaussian filtering function in the point cloud processing software or library to denoise the converted point cloud data; import the satellite image data after coordinate conversion into the remote sensing image processing software, find the filtering tool in the software, and set appropriate filtering parameters according to the characteristics and processing requirements of the satellite image data to perform the filtering operation; use the image editing software to convert the image from the spatial domain to the frequency domain, use a high-pass filter or band-pass filter to remove the noise in the low-frequency part, and then convert the image back from the frequency domain to the spatial domain, and use professional photogrammetry software to perform relative orientation and absolute orientation processing on the aerial photography data to reduce the noise caused by factors such as unstable camera attitude.

[0022] In one embodiment of the present invention, the extraction of the features of the satellite image data and the construction of the classifier to generate the preliminary classification result include the following steps: Perform mathematical operations on each band of satellite imagery to extract the spectral characteristics of specific ground objects. After selecting the target area, extract the reflectance of this area in different bands to generate a spectral curve. Analyze the spectral characteristics of ground objects and calculate various spectral indices, including the Normalized Difference Vegetation Index (NDVI), Water Index, and Built-up Index. Use a classifier trained with known ground object samples to automatically classify the satellite imagery. At the same time, extract the spectral characteristics of different ground objects, reduce the dimensionality of the multi-bands, and extract the main spectral characteristics. Resample the satellite imagery data, use an edge detection algorithm to extract the ground object boundaries, calculate the texture characteristics of the satellite imagery, and then fuse the extracted spectral and spatial characteristics to form a high-dimensional feature vector, which is input into a Support Vector Machine (SVM) trained classifier to perform target recognition on the satellite imagery data and generate a preliminary classification result.

[0023] Specifically, start a professional remote sensing image processing software. According to the research needs and the spectral characteristics of specific ground objects, select appropriate mathematical operation methods, such as addition, subtraction, multiplication, division, exponentiation, logarithm, etc., to perform operations on each band of the satellite imagery. The software generates a new image according to the selected operation method. In the remote sensing image processing software, use the drawing tool or region selection tool to select the target area for which the reflectance is to be extracted based on information such as geographical location and ground object distribution. Use the "pixel value query" or "region analysis" function of the software to extract the reflectance values of the target area in different bands. Import the extracted reflectance data of different bands into a data analysis software. Use the band as the abscissa and the reflectance as the ordinate to plot the spectral curve of the target area. Calculate the Normalized Difference Vegetation Index (NDVI), Water Index, and Built-up Index respectively. Collect the ground object samples of known categories in the study area, including sample data of land cover types such as vegetation, water bodies, and buildings. Use the "pixel value extraction" or "sample analysis" function of the remote sensing image processing software to extract the spectral characteristics of the known ground object samples in different bands, and use these characteristics as training data. Input the spectral characteristics of the extracted known ground object samples into the SVM classification algorithm for classifier training. Use the trained SVM classifier to automatically classify the entire satellite imagery data. In the remote sensing image processing software, import the multi-band satellite imagery data to be processed for dimensionality reduction. In the remote sensing image processing software, find the "resampling" or "image resampling" function module. Import the satellite imagery data to be resampled, set the resampling parameters, and then click the "OK" button. The software will perform resampling processing on the satellite imagery data to generate a new image with the target resolution. According to the characteristics of the ground object boundaries and application requirements, select an appropriate edge detection algorithm. Set the corresponding parameters according to the selected texture feature calculation method. Combine the standardized spectral and spatial characteristics into a feature vector. Use the trained SVM classifier, take the fused feature vector as the input data, construct a target recognition model, and output the recognition results of the corresponding ground object categories or target objects.

[0024] In one embodiment of the present invention, the steps of extracting the features of aerial camera data, constructing a multi-branch deep learning model, and outputting a classification result include the following steps: Perform resampling processing on the aerial camera data to make its resolution consistent with that of the satellite image data. Use the color histogram to extract color features from the aerial camera data. At the same time, use the gray-level co-occurrence matrix to extract texture features, use the edge detection algorithm to extract shape features, and use the object segmentation algorithm to extract the ground object contour. Extract the height information of the aerial camera data to generate three-dimensional features. Construct a multi-branch deep learning model. Use the satellite image data and the aerial camera data as inputs for different branches. The satellite image branch inputs spectral features and spatial features, and the aerial camera branch inputs high-resolution spatial features and three-dimensional features. At the last layer of the network, fuse the features of the two branches, and use the fused features for target recognition to output the classification result.

[0025] Specifically, first obtain the resolution parameters of satellite image data, including detailed information such as pixel pitch and pixel size. According to the characteristics of aerial photography data and the requirements of target recognition, determine the resampling method. Import the aerial photography data into professional image processing software, set the resampling parameters to the previously determined resolution, and perform the resampling operation to make the resolution of the aerial photography data consistent with that of the satellite image data. In the image processing software, select the resampled aerial photography data, calculate its color histogram, and convert the color image to a grayscale image. In the image processing software, set the parameters of the gray-level co-occurrence matrix, process the resampled aerial photography data to generate the gray-level co-occurrence matrix, and extract texture features from the gray-level co-occurrence matrix, including contrast, correlation, and entropy. In the image processing software, import the resampled aerial photography data, set the parameters of the edge detection algorithm, describe the features of the detected edges, and extract the length, curvature, and complexity features of the edges. According to the characteristics of the ground objects in the aerial photography data and the requirements of target recognition, select an appropriate object segmentation algorithm. In the image processing software, import the resampled aerial photography data, set the parameters of the object segmentation algorithm, use the metadata of the aerial photography data, including shooting angle, camera focal length, and flight altitude information, combine with the results of object segmentation, calculate the height information of each ground object, and combine the height information with other shape features to form a three-dimensional feature vector; construct a deep learning branch dedicated to processing satellite image data, the input layer of this branch receives the feature vector composed of the spectral features and spatial features of the satellite image as input, construct another deep learning branch to process the aerial photography data, the input layer of this branch receives the high-resolution spatial features and three-dimensional features as input, set a feature fusion layer at the last layer of the network to fuse the output features of the satellite image branch and the aerial photography branch, collect a large number of labeled data sets, including satellite image data and corresponding aerial photography data, as well as their classification labels, input the training set data into the constructed deep learning model for training, and use the trained model to output the classification results.

[0026] In one embodiment of the present invention, the extraction of the features of the point cloud data includes the following steps: Perform denoising, filtering, and removal of invalid points on the point cloud data, extract the coordinate information of the point cloud data, calculate the elevation value of each point, and obtain the slope value and aspect of each point through the gradient calculation method. Classify the point cloud according to the coordinate, color, and intensity information of the point cloud, and extract its specific attribute information according to different categories of point clouds; use the preprocessed point cloud data to generate a three-dimensional model, extract the geometric features and topological features of the three-dimensional model, and calculate the spatial distribution features of the point cloud data.

[0027] Specifically, directly read the three-dimensional coordinates of each point from the preprocessed point cloud data. Select the ground plane as the reference plane. For each point, calculate the height difference between its z coordinate and the reference plane, and this height difference is the elevation value of the point. For each point, within its 3×3 neighborhood, calculate the gradients in the x and y directions through the central difference method. Obtain the slope based on the calculated gradients. Calculate the aspect through aspect = atan2(-grad_x, -grad_y). Combine multiple information such as coordinates, colors, and intensities for more refined classification. For example, in urban scene reconstruction, classify the point clouds within a certain height range with specific color and intensity characteristics into different parts of buildings. Associate the extracted attribute information with the corresponding point cloud categories and store them. Use the point cloud reconstruction algorithm to convert the preprocessed point cloud data into a triangular mesh model. Extract geometric shape features from the three-dimensional model and analyze the topological structure of the three-dimensional model. Obtain the main axis by performing eigenvalue decomposition on the covariance matrix, and the main axis can reflect the distribution direction and shape characteristics of the point cloud data.

[0028] In one embodiment of the present invention, based on the existing multi-branch deep learning model, add a point cloud data branch, and use the fused features for target recognition to output high-precision target recognition results, including the following steps: Based on the existing multi-branch deep learning model, add a point cloud data branch. Input the geometric features and attribute features of the point cloud data into the point cloud data branch, and fuse the features of the three branches at the last layer of the network. Concatenate the features of satellite images, aerial images, and point cloud data into a high-dimensional feature vector. Use the attention mechanism to dynamically weight the features of different data sources, perform feature fusion at different network levels, and use the fused features for target recognition to output high-precision target recognition results.

[0029] Specifically, keep the original satellite image branch input unchanged, and also keep the aerial photography branch input unchanged. Create a new point cloud data branch, and use the geometric features and attribute features of the previously extracted point cloud data as inputs. Perform feature fusion at a relatively shallow level of the network, and splice the output features of the three branches at an earlier network layer and fuse them at an intermediate level of the network, usually after several convolutional layers and pooling layers. At this time, the features of each branch have undergone a certain degree of abstraction and extraction, and the fused features can better capture the correlations between different data sources. Fuse the features before the last layer or the last few layers of the network. In this case, the features of each branch have undergone sufficient learning and extraction, and the fused features are mainly used for the final target recognition task. Introduce an attention mechanism within each branch to dynamically weight the importance of different features, input the fused features into the target recognition module, and output the results of target recognition in a visual or text form. For example, display the positions and ranges of different target classes on a map.

[0030] Please refer to Figure 2 As shown, the present invention is a multi-source spatial data fusion processing method, including the following steps: S1: Used to collect spatial data from different data sources, including satellite image data, aerial photography data, and point cloud data; S2: Perform preprocessing operations on the collected original spatial data, including data format conversion, coordinate system unification, and data denoising processing; S3: Extract the features of the satellite image data, construct a classifier to generate a preliminary classification result, extract the features of the aerial photography data, construct a multi-branch deep learning model to output a classification result, and extract the features of the point cloud data; S4: Based on the existing multi-branch deep learning model, add a point cloud data branch, use the fused features for target recognition, and output a high-precision target recognition result.

[0031] The above embodiments are only used to illustrate the technical method of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A multi-source spatial data fusion processing system, characterized in that: Includes the following modules: Multi-source data acquisition module: used to collect spatial data from different data sources, including satellite image data, aerial photography data and point cloud data; Data preprocessing module: preprocessing the collected raw spatial data, including data format conversion, coordinate system unification and data denoising; Feature extraction and model building module: extract the features of satellite image data, build a classifier to generate preliminary classification results, extract the features of aerial photography data, build a multi-branch deep learning model to output classification results, and extract the features of point cloud data; Data fusion module: Based on the existing multi-branch deep learning model, a point cloud data branch is added, and the fused features are used for target recognition to output high-precision target recognition results.

2. A multi-source spatial data fusion processing system according to claim 1, characterized in that: The method for collecting spatial data from different data sources, including satellite image data, aerial photography data and point cloud data, comprises the following steps: Start the satellite image acquisition process, collect satellite image data, transfer the collected data to the device for preliminary sorting and storage, and generate original satellite image data; plan the flight route according to the comprehensive factors of the target area, use an aerial photography camera to shoot the target area, download the aerial photography data to the storage device, and annotate the downloaded aerial photography data, including shooting time, location, flight altitude and camera parameter information; select appropriate control points around and inside the target area, use ground surveying instruments to measure the control points, obtain the three-dimensional coordinates of the corresponding control points, and measure the feature points in the target area based on the control points, where the feature points include terrain points and corner points of objects, create a unique identifier for each measurement point data, and generate point cloud data.

3. A multi-source spatial data fusion processing system according to claim 1, characterized in that: The preprocessing operation of the collected raw spatial data, including data format conversion, coordinate system unification and data denoising, includes the following steps: Use remote sensing image processing software, image editing software and Python library to convert the formats of raw satellite image data, raw aerial photography data and point cloud data respectively; Import the satellite image data after format conversion into the remote sensing image processing software for coordinate conversion and unify it into the WGS84 coordinate system; determine the original coordinate system corresponding to the original data of the aerial photography data after format conversion, and convert the original coordinate system according to the relationship between the original coordinate system and the WGS84 coordinate system; determine the original coordinate system of the point cloud data after format conversion, obtain the conversion parameters from the original coordinate system to WGS84, and save the converted data; The satellite image data is filtered using the filtering tools in the remote sensing image processing software. The sensor noise and lens distortion noise are denoised using the image enhancement algorithm and aerial survey software respectively. The noise in the point cloud data is removed using Gaussian filtering.

4. The multi-source spatial data fusion processing system according to claim 1, characterized in that: The process of extracting features of satellite image data and constructing a classifier to generate preliminary classification results includes the following steps: Mathematical operations are performed on each band of the satellite image to extract the spectral characteristics of specific objects. After selecting the target area, the reflectivity of the area in different bands is extracted to generate a spectral curve. The spectral characteristics of the objects are analyzed and multiple spectral indices, including the normalized vegetation index, water index and building index, are calculated. The classifier trained with known object samples is used to automatically classify the satellite image and extract the spectral characteristics of different objects. The multi-band dimension is reduced to extract the main spectral characteristics. The satellite image data is resampled, and the edge detection algorithm is used to extract the boundaries of the objects. After calculating the texture characteristics of the satellite image, the extracted spectral and spatial features are fused to form a high-dimensional feature vector, which is input into the support vector machine training classifier. The trained classifier is used to identify the target of the satellite image data and generate preliminary classification results.

5. The multi-source spatial data fusion processing system according to claim 1, characterized in that: The method of extracting features of aerial photography data and constructing a multi-branch deep learning model to output classification results includes the following steps: Resample the aerial camera data to ensure that its resolution is consistent with the satellite image data. Use the color histogram to extract the color features of the aerial camera data. Use the gray-level co-occurrence matrix to extract the texture features and the edge detection algorithm to extract the shape features. Use the object segmentation algorithm to extract the contours of the objects and the height information of the aerial camera data to generate three-dimensional features. A multi-branch deep learning model is constructed, and satellite image data and aerial photography data are used as inputs of different branches. The satellite image branch inputs spectral features and spatial features, and the aerial photography branch inputs high-resolution spatial features and three-dimensional features. The features of the two branches are fused in the last layer of the network, and the fused features are used for target recognition and output classification results.

6. The multi-source spatial data fusion processing system according to claim 1, characterized in that: The feature extraction of point cloud data comprises the following steps: The point cloud data is denoised, filtered and invalid points are removed, the coordinate information of the point cloud data is extracted, the elevation value of each point is calculated, and the slope value and slope direction of each point are obtained through the gradient calculation method. The point cloud is classified according to the coordinate, color and intensity information of the point cloud, and specific attribute information is extracted according to the different categories of point clouds; the preprocessed point cloud data is used to generate a three-dimensional model, the geometric and topological features of the three-dimensional model are extracted, and the spatial distribution characteristics of the point cloud data are calculated.

7. The multi-source spatial data fusion processing system according to claim 1, characterized in that: The method adds a point cloud data branch on the basis of the existing multi-branch deep learning model, uses the fused features for target recognition, and outputs a high-precision target recognition result, including the following steps: Based on the existing multi-branch deep learning model, a point cloud data branch is added, the geometric features and attribute features of the point cloud data are input into the point cloud data branch, and the features of the three branches are fused in the last layer of the network. The features of satellite images, aerial photography and point cloud data are spliced ​​into a high-dimensional feature vector. The attention mechanism is used to dynamically weight the features of different data sources, and feature fusion is performed at different network levels. The fused features are used for target recognition, and high-precision target recognition results are output.

8. A multi-source spatial data fusion processing method, using a multi-source spatial data fusion processing system according to any one of claims 1 to 7, characterized in that: The following steps are involved: S1: used to collect spatial data from different data sources, including satellite image data, aerial photography data and point cloud data; S2: Preprocessing the collected raw spatial data, including data format conversion, coordinate system unification and data denoising; S3: Extract the features of satellite image data, build a classifier to generate preliminary classification results, extract the features of aerial photography data, build a multi-branch deep learning model to output classification results, and extract the features of point cloud data; S4: Based on the existing multi-branch deep learning model, a point cloud data branch is added, and the fused features are used for target recognition to output high-precision target recognition results.

Images