Road material automatic identification system and method based on multi-source remote sensing data fusion

Abstract

The invention provides a road material automatic identification system and method based on multi-source remote sensing data fusion. The system comprises a remote sensing data acquisition module, a data preprocessing module, a wave band selection module, an image enhancement module, an image fusion module, a road extraction module, a feature extraction module, a feature optimization module, a classification and recognition module, a result post-processing module and a user interaction module. According to the road material automatic identification system and method based on multi-source remote sensing data fusion, image data of different sensors are fused, and advanced image processing and machine learning technologies are combined, so that high-precision and automatic identification of road materials is realized; therefore, the requirements of the fields of urban planning, road construction, maintenance and the like can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of road material recognition, and in particular to a road material automatic recognition system and method based on multi-source remote sensing data fusion. Background Art

[0002] Accurate identification of road materials is of great significance to many fields such as urban planning, traffic management, road maintenance and environmental monitoring. Traditional road material identification methods mainly rely on ground surveys and manual measurements, which are not only time-consuming and labor-intensive, but also limited by geographical environment and human factors, making it difficult to achieve large-scale and efficient identification.

[0003] With the rapid development of remote sensing technology, especially the significant improvement in the ability to obtain high-resolution remote sensing data, it has become possible to use remote sensing data to identify road materials. Remote sensing technology can provide large-scale and efficient data collection, which has significant advantages in dynamically monitoring changes in road materials. However, existing technologies are still insufficient in processing complex land features and improving recognition accuracy, especially under different lighting conditions and different seasonal backgrounds, the spectral characteristics of road materials may change, affecting the recognition effect.

[0004] Remote sensing images from a single data source often have limitations in spatial resolution and spectral resolution, making it difficult to meet the needs of high-precision road material recognition. Therefore, how to fuse multi-source remote sensing data to improve the accuracy and robustness of road material recognition has become a hot topic in current research. Multi-source remote sensing data fusion technology can comprehensively utilize the advantages of different sensors, improve the spatial resolution and spectral resolution of images, and provide a new technical approach for automatic recognition of road materials.

[0005] There are many road material recognition methods in the prior art, mainly based on remote sensing image analysis. These methods usually rely on remote sensing images from a single data source, such as multispectral or hyperspectral images, to identify and classify road materials. For example, a common technical solution is to use multispectral images for road extraction and material classification. This method combines the red, green, and blue (RGB) bands of the image and identifies the road material based on the differences in the reflectance characteristics of these bands.

[0006] The technical solution of the prior art mainly includes the following steps: Use multispectral sensors carried by satellites or aircraft to obtain image data of road areas.

[0007] The acquired multispectral images are subjected to radiation correction and atmospheric correction to eliminate the influence of external factors on the image data.

[0008] By combining the RGB bands of the image and utilizing the reflectivity differences in different bands, the road area is extracted using threshold segmentation or classification algorithms.

[0009] According to the spectral characteristics of the road in the multispectral image, a simple machine learning algorithm, such as decision tree or naive Bayes, is used to classify the material of the extracted road area.

[0010] The accuracy of the classification results is evaluated by comparing them with the ground-truth data.

[0011] The shortcomings of existing technologies are mainly concentrated in the following key aspects: 1. Insufficient use of spectral information: Existing technologies are usually based on multispectral images, whose limited band information is insufficient to capture the subtle spectral differences of road materials. For example, different types of asphalt pavement may not be effectively distinguished in multispectral images because their reflectance differences in several bands are not large.

[0012] 2. Spatial resolution limitations: The spatial resolution limitations of remote sensing images used by existing technologies make it impossible to identify small or narrow road structures. This is particularly prominent in urban environments, where small road connections or alleys may be ignored or misclassified.

[0013] 3. Poor adaptability to environmental factors: Existing technologies lack effective adaptability strategies when dealing with cloud cover, shadows, and images under different lighting conditions. These environmental factors will significantly affect the spectral reflectance characteristics of road materials and reduce recognition accuracy.

[0014] 4. Single data processing method: Existing technologies mostly use basic image processing and classification algorithms for data processing, such as fixed threshold segmentation or simple machine learning models. These methods may not be accurate enough when dealing with complex road environments and have difficulty coping with changing road conditions.

[0015] 5. Lack of multi-source data fusion capabilities: Existing technologies fail to fully utilize the advantages of different sensor data, such as combining high spatial resolution images with high spectral resolution data to provide more comprehensive information on ground feature characteristics.

[0016] 6. Insufficient post-processing technology: The existing technology lacks effective post-processing steps after the classification results, such as morphological operations, edge optimization, etc., which may lead to fragmented and discontinuous road areas in the recognition results.

[0017] 7. Poor automation and user interactivity: Existing technologies require a lot of manual intervention during the operation process, from data selection to parameter adjustment, lack of automated processes, and the user interface is not friendly enough, which limits the ease of use and popularity of the technology.

[0018] 8. Poor generalization ability: Existing technologies may perform well under specific conditions, but their classification models may not work effectively under new or different environmental conditions and lack good generalization ability.

[0019] Therefore, it is necessary to provide a new road material automatic identification system and method based on multi-source remote sensing data fusion to solve the above technical problems. Summary of the invention

[0020] The technical problem solved by the present invention is to provide a road material automatic identification system and method based on multi-source remote sensing data fusion, which realizes high-precision and automatic identification of road materials by fusing image data from different sensors and combining advanced image processing and machine learning technologies, thereby meeting the needs of urban planning, road construction and maintenance and other fields.

[0021] To solve the above technical problems, the road material automatic identification system based on multi-source remote sensing data fusion provided by the present invention includes a remote sensing data acquisition module, a data preprocessing module, a band selection module, an image enhancement module, an image fusion module, a road extraction module, a feature extraction module, a feature optimization module, a classification and identification module, a result post-processing module and a user interaction module. The remote sensing data acquisition module, the data preprocessing module, the band selection module, the image enhancement module, the image fusion module, the road extraction module, the feature extraction module, the feature optimization module, the classification and identification module, the result post-processing module and the user interaction module are connected in sequence, and the feature extraction module is connected to the classification and identification module.

[0022] Preferably, the remote sensing data acquisition module is used to acquire hyperspectral and multispectral image data from different sensors; The data preprocessing module is used to perform radiation correction, atmospheric correction, geometric correction, and cloud occlusion and shadow processing; The band selection module is used to automatically select the optimal band combination based on spectral feature analysis; The image enhancement module is used to improve the distinguishability of road materials in the image through contrast enhancement and color correction; The image fusion module is used to improve spatial and spectral resolution by using pixel-level and feature-level fusion techniques; The road extraction module is used to extract the road area through mask technology and ground feature index; The feature extraction module is used to calculate the texture, spectrum and shape characteristics of the road area; The feature optimization module is used to select the optimal feature subset to enhance classification performance; The classification and recognition module is used to classify road materials using a support vector machine or other machine learning algorithms; The result post-processing module is used to optimize the classification results through morphological operations and other techniques; The user interaction module is used to provide a user interface to achieve parameter customization, data source selection and result visualization.

[0023] Preferably, the data preprocessing module adopts an image denoising step to reduce the impact of sensor noise on subsequent processing; The band selection module uses a statistical analysis-based method to determine the optimal combination of bands to maximize the spectral separation between different road materials.

[0024] Preferably, the image enhancement module uses a specific algorithm to perform color correction and contrast enhancement on the image to improve the visual effect of the road material; The image fusion module uses multi-scale fusion technology to combine high-resolution images and hyperspectral images to enhance the spatial details of road materials.

[0025] Preferably, the road extraction module further includes a semantic segmentation technology based on deep learning to improve the accuracy of road extraction in complex scenes; The feature extraction module uses multi-scale analysis and directional analysis to fully capture the texture characteristics of road materials. The feature optimization module uses a genetic algorithm or gradient boosted decision tree based approach to select the most discriminative features.

[0026] Preferably, the classification and recognition module also includes a model training and verification process to ensure that the classifier has high generalization ability for different road materials; The result post-processing module adopts adaptive threshold technology and conditional random field model to optimize the consistency and completeness of classification results.

[0027] The present invention also provides a method for automatically identifying road materials based on multi-source remote sensing data fusion, comprising the following steps: S1. Acquisition: Acquire multi-source remote sensing image data through remote sensing data acquisition module; S2, preprocessing: use the data preprocessing module to perform radiation correction, atmospheric correction, geometric correction, and cloud occlusion and shadow processing on the image data; S3, band selection: automatically select the band that is most critical for road material recognition through the band selection module; S4, image enhancement: using the image enhancement module to perform contrast enhancement and color correction on the image data; S5, Image fusion: Use the image fusion module to fuse multi-source image data at the pixel level and feature level to improve the resolution; S6, road extraction: extract road areas through the road extraction module combined with mask technology and object index; S7, feature extraction: using the feature extraction module to calculate texture, spectrum and shape features from the extracted road area; S8, feature optimization: select the optimal feature subset through the feature optimization module; S9, classification and recognition: Apply the machine learning algorithm in the classification and recognition module to classify and recognize the feature data; S10, result post-processing: use the result post-processing module to perform morphological operations and other technical optimizations on the classification results; S11. User interaction: The user interaction module allows users to customize parameters, select data sources, and visualize and export recognition results.

[0028] Compared with the related art, the road material automatic identification system and method based on multi-source remote sensing data fusion provided by the present invention has the following beneficial effects: 1. High-precision road material recognition capability: The present invention significantly improves the recognition accuracy of road materials by integrating multi-source remote sensing data and combining advanced image processing technology and machine learning algorithms. In particular, the band selection module can automatically select the bands that best represent the characteristics of different road materials, and with the image enhancement and feature extraction technology, the system can effectively distinguish between roads of different materials such as asphalt, cement, and soil. In addition, the support vector machine and other algorithms used in the classification and recognition module, and the feature data processed by the feature optimization module, further improve the accuracy of classification and ensure the high accuracy of the recognition results.

[0029] 2. Highly automated identification process: The automatic identification system of the present invention realizes the automation of the entire process from data acquisition to post-processing, greatly reducing the need for manual operation. The data preprocessing module automatically performs radiation correction, atmospheric correction and geometric correction, eliminating the impact of environmental factors on the data. The automated band selection and image fusion process reduces the dependence on professional knowledge and makes the system easier to operate. The user interaction module provides a simple and intuitive operating interface, allowing non-professionals to easily set parameters and view results, further improving the practicality and popularity of the system.

[0030] 3. Powerful data processing and fusion capabilities: The system and method of the present invention effectively integrate remote sensing data of different resolutions and spectral characteristics through the image fusion module, solving the limitations of a single data source in spatial resolution and spectral resolution. The pixel-level and feature-level fusion technology used not only improves the spatial details of the image, but also enhances the richness of spectral information, enabling the system to handle more complex road environments and conditions. In addition, multi-source data fusion also enhances the system's accuracy in identifying road materials under different seasons and lighting conditions, ensuring the stability and reliability of the identification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a principle block diagram of the road material automatic identification system based on multi-source remote sensing data fusion provided by the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0033] Please refer to Figure 1 ,in, Figure 1 The principle block diagram of the road material automatic identification system based on multi-source remote sensing data fusion provided by the present invention. The road material automatic identification system based on multi-source remote sensing data fusion includes: a remote sensing data acquisition module, a data preprocessing module, a band selection module, an image enhancement module, an image fusion module, a road extraction module, a feature extraction module, a feature optimization module, a classification recognition module, a result post-processing module and a user interaction module, wherein the remote sensing data acquisition module, the data preprocessing module, the band selection module, the image enhancement module, the image fusion module, the road extraction module, the feature extraction module, the feature optimization module, the classification recognition module, the result post-processing module and the user interaction module are connected in sequence, and the feature extraction module is connected to the classification recognition module.

[0034] Remote sensing data acquisition module, used to obtain hyperspectral and multispectral image data from different sensors; Data preprocessing module, used to perform radiation correction, atmospheric correction, geometric correction, and cloud occlusion and shadow processing; Band selection module, used to automatically select the optimal band combination based on spectral feature analysis; Image enhancement module, used to improve the distinguishability of road materials in images through contrast enhancement and color correction; Image fusion module, used to improve spatial and spectral resolution using pixel-level and feature-level fusion techniques; Road extraction module, used to extract road areas through mask technology and object index; Feature extraction module, used to calculate the texture, spectral and shape characteristics of the road area; Feature optimization module, used to select the optimal feature subset to enhance classification performance; Classification and recognition module, used to classify road materials using support vector machines or other machine learning algorithms; The result post-processing module is used to optimize the classification results through morphological operations and other techniques; The user interaction module is used to provide a user interface to achieve parameter customization, data source selection and result visualization.

[0035] Operation process Remote sensing data acquisition module: As the starting point of the entire system, it is responsible for acquiring hyperspectral and multispectral image data from different remote sensing sensors. These data may come from different times and different sensors, ensuring the diversity and comprehensiveness of the data.

[0036] Data preprocessing module: receives data from the remote sensing data acquisition module and performs radiation correction, atmospheric correction, geometric correction, and handles cloud occlusion and shadows. These preprocessing steps ensure the quality of the data and provide an accurate basis for subsequent analysis.

[0037] Band selection module: After data preprocessing, the band selection module automatically selects the optimal band combination based on spectral feature analysis. This step is critical because it determines which bands can provide the most useful information about road material identification.

[0038] Image Enhancement Module: Receives the output of the Band Selection Module and performs contrast enhancement and color correction to improve the distinguishability of road materials in the image. The enhanced image provides a clearer visual effect for subsequent feature extraction.

[0039] Image fusion module: It uses pixel-level and feature-level fusion technology to combine image data of different resolutions to improve spatial and spectral resolution. This enhances the spatial details of road materials and provides richer data for accurate identification.

[0040] Road Extraction Module: Use masking techniques and object index, possibly combined with deep learning semantic segmentation techniques, to extract road areas from the fused image. This step separates the road from the background and provides an accurate area for feature extraction.

[0041] Feature extraction module: After the road area is extracted, the texture, spectrum and shape features are calculated. These features are extracted from the road area for subsequent classification and recognition.

[0042] Feature Optimization Module: Receives the results of the Feature Extraction Module and selects the best subset of features to enhance classification performance. This is achieved by applying techniques such as genetic algorithms to ensure that the selected feature set can effectively distinguish different types of road materials.

[0043] Classification and recognition module: Apply support vector machine or other machine learning algorithms, combined with model training and verification process, to classify and recognize the feature data provided by the feature optimization module. This step ensures that the classifier has high generalization ability for different road materials.

[0044] Result post-processing module: morphological operations and other technical optimizations are performed on the output of the classification and recognition module to improve the consistency and completeness of the results. This includes adaptive threshold technology, conditional random field model, etc. to optimize classification boundaries and eliminate misclassified areas.

[0045] User interaction module: provides a user interface that allows users to customize parameters, select data sources, and visualize and export results. This module is the interface for users to interact with the system, allowing users to adjust system settings as needed and obtain the desired output results.

[0046] The data preprocessing module further includes an image denoising step to reduce the impact of sensor noise on subsequent processing.

[0047] Among them, the band selection module uses a method based on statistical analysis to determine the optimal combination of bands to maximize the spectral separation between different road materials.

[0048] Among them, the image enhancement module uses a specific algorithm to perform color correction and contrast enhancement on the image to improve the visual effect of the road material.

[0049] Among them, the image fusion module uses multi-scale fusion technology to combine high-resolution images and hyperspectral images to enhance the spatial details of road materials.

[0050] Among them, the road extraction module also includes semantic segmentation technology based on deep learning to improve the accuracy of road extraction in complex scenes.

[0051] Among them, the feature extraction module uses multi-scale analysis and directional analysis to comprehensively capture the texture characteristics of road materials.

[0052] Among them, the feature optimization module uses a method based on genetic algorithm or gradient boosting decision tree to select the most discriminative features.

[0053] Among them, the classification and recognition module also includes model training and verification processes to ensure that the classifier has high generalization ability for different road materials.

[0054] Among them, the result post-processing module adopts adaptive threshold technology and conditional random field model to optimize the consistency and completeness of classification results.

[0055] The present invention also provides a method for automatically identifying road materials based on multi-source remote sensing data fusion, the method comprising the following steps: S1. Collection: Acquire multi-source remote sensing image data through remote sensing data collection module; S2. Preprocessing: Use the data preprocessing module to perform radiation correction, atmospheric correction, geometric correction, and cloud occlusion and shadow processing on the image data; S3. Band selection: The band selection module automatically selects the band that is most critical for road material recognition; S4. Image enhancement: Use the image enhancement module to perform contrast enhancement and color correction on image data; S5. Image fusion: Use the image fusion module to fuse multi-source image data at the pixel level and feature level to improve resolution; S6. Road extraction: extract road areas through the road extraction module combined with mask technology and object index; S7. Feature extraction: using the feature extraction module to calculate texture, spectrum and shape features from the extracted road area; S88. Feature optimization: Select the optimal feature subset through the feature optimization module; S9. Classification and recognition: Apply the machine learning algorithm in the classification and recognition module to classify and recognize the feature data; S10. Result post-processing: Use the result post-processing module to perform morphological operations and other technical optimizations on the classification results; S11. User interaction: The user interaction module allows users to customize parameters, select data sources, and visualize and export recognition results.

[0056] The remote sensing data acquisition module is specially designed to collect and integrate image data from multiple remote sensing sensors. The module is highly compatible and adaptable, and can receive data from domestic high-resolution satellite sensors including but not limited to GF-1, GF-2 and GF-5. These sensors provide images of different resolutions. For example, the multispectral images provided by GF-1 and GF-2 satellites have a spatial resolution of 4 meters, while the hyperspectral images provided by GF-5 satellite have an original spatial resolution of 30 meters. The data types collected by the module cover panchromatic images, multispectral images, and hyperspectral images, with spectral ranges from visible light to near infrared, and the number of bands ranging from a few to hundreds, ensuring the diversity and comprehensiveness of the data. During the data collection process, the module automatically downloads and stores image data according to the preset time interval and spatial coverage. For example, the module can be set to automatically collect image data in a specified area every two weeks to ensure the timeliness and continuity of the data. The collected data will be stored in a specific data structure and format, such as HDF5 or NetCDF format, for subsequent processing and analysis. In addition, the module also includes a data quality check function to ensure that the collected image data meets certain quality standards, such as cloud coverage less than 10%, image resolution meeting specific requirements, etc., to ensure the accuracy and reliability of subsequent processing processes.

[0057] The data preprocessing module is responsible for converting the raw image data collected by the remote sensing data acquisition module into a format suitable for further analysis and processing. The module first performs radiometric correction to eliminate differences in the radiation response of the sensor and ensure that data acquired at different times or from different sensors are comparable. Next, atmospheric correction is performed, using models such as 6S or MODTRAN to adjust the image data to simulate the surface reflectance without the influence of the atmosphere, usually including an atmospheric transparency correction parameter of 2% to 5%. Then, the geometric correction process aligns the image data with a known geographic coordinate system, using ground control points (GCPs) or precise orbit information of satellites to ensure that the spatial accuracy after correction reaches 1 to 3 pixels. The data preprocessing module also involves cloud occlusion and shadow processing, by analyzing the spectral characteristics and spatial distribution of the image, identifying and removing pixels affected by cloud occlusion, and adjusting or removing shadow areas by calculating the length and direction of the shadow, ensuring that at least 90% of the data is not affected by these factors. The image denoising step uses a 3x3 or 5x5 median filter to eliminate random noise caused by sensor noise while maintaining the clarity of important features such as roads. The module also includes an adaptive histogram equalization process to enhance the contrast of the image, making the histogram of the image data closer to a uniform distribution, thereby improving the identifiability of road material features in subsequent processing steps. All pre-processing steps are performed through automated scripts to ensure consistency and repeatability of data processing.

[0058] The band selection module first performs a detailed spectral feature analysis of the hyperspectral image data, and determines the independence of the bands by calculating the correlation coefficient between each band and other bands. Then, statistical analysis methods such as principal component analysis (PCA) or linear discriminant analysis (LDA) are used to evaluate the contribution of different band combinations to the ability to distinguish road materials. This module pays special attention to selecting bands that can maximize the differences in spectral characteristics between different road materials. For example, through analysis, it was found that there are significant differences in the spectral responses of asphalt and concrete roads in the band range of 1000-1100 nanometers and 2200-2400 nanometers. During the automatic selection process, the band selection module will calculate the variance of each band and select bands with variances greater than a specific threshold (such as 0.05) to ensure that the selected bands carry enough information. In addition, the module will also adopt the mutual information criterion to select band combinations with mutual information values ​​higher than a preset threshold (such as 0.1) to ensure the information complementarity between bands. By setting these parameters, the band selection module finally determines an optimal band combination of 5 to 10 bands, with a spectral resolution of 5 nanometers, ensuring the best differentiation effect in subsequent feature extraction and classification. The process of the band selection module is automated, and the entire process from data input to band combination output does not require human intervention, greatly improving processing efficiency. Ultimately, the band combination output by this module will be used in the feature extraction module to provide a solid spectral foundation for the fine identification of road materials.

[0059] The image enhancement module receives the output of the band selection module and performs a series of enhancement operations on the image data of the selected band. First, color correction is performed to ensure that the color of roads of different materials is more realistic and consistent by adjusting the hue and saturation of the image. During the color correction process, the module adjusts the hue angle and chromaticity value in the RGB color space, and usually uses the color difference of ΔE<5 as the standard for correction accuracy. The image enhancement module optimizes the contrast of the image through contrast enhancement techniques such as histogram equalization or contrast-limited adaptive histogram equalization (CLAHE), making the differences between road materials more obvious. In histogram equalization, the module analyzes the histogram distribution of the image and adjusts the pixel values ​​to make the histogram more uniform and enhance the details of the dark and bright parts. In the CLAHE algorithm, the image is divided into several small areas, and each area is independently histogram equalized to maintain the natural transition of the image. In the parameter setting, the module selects the appropriate area size according to the image characteristics, such as 16x16 or 32x32 pixels. The image enhancement module also applies edge enhancement filters, such as Sobel or Canny operators, to highlight the edge information of the road and improve the clarity of the road boundary. During the edge enhancement process, the module adjusts the threshold parameters of the filter to control the sensitivity of edge detection. Usually, the threshold of the primary edge is set higher than that of the secondary edge to reduce noise interference. The entire image enhancement process is automated, and the module automatically adjusts the parameters of the enhancement algorithm according to the specific conditions of the image and the preset parameters to achieve the best enhancement effect. Ultimately, the enhanced image will have higher visual contrast and clearer road edges, providing a higher quality data foundation for feature extraction and road material recognition.

[0060] The image fusion module uses advanced multi-scale fusion technology to integrate image data of different resolutions to enhance the spatial details and spectral information of road materials. Specifically, the module first receives high-resolution panchromatic images (e.g., GF-2 satellite images with a spatial resolution of 0.5 meters) and hyperspectral images (e.g., images with a spatial resolution of 30 meters provided by the GF-5 satellite) from different sensors. The fusion process begins with preprocessing these images, including radiometric correction and spectral enhancement, to ensure data consistency. Subsequently, the module uses multi-scale decomposition techniques, such as Laplace pyramid or wavelet transform, to decompose the images into bands of different scales. For example, in the Laplace pyramid, the image is decomposed into four scale levels, and the spatial resolution of each level is halved, while the wavelet transform decomposes the image into approximate and detail bands based on the selection of wavelet functions (e.g., Haar wavelet). In the multi-scale fusion process, the module combines the detail bands of the high-resolution image with the approximate bands of the hyperspectral image. By calculating the correlation between the detail bands and the approximate bands, the module determines the best fusion strategy, such as correlation weight-based fusion or optimal band selection based on information entropy. Fusion parameters, such as the fusion coefficient α, are set to 0.7 to 0.9 to balance the contributions of different images and ensure that the fusion results retain both the spatial details of high-resolution images and the rich spectral features of hyperspectral images. The image fusion module reconstructs a hyperspectral image with enhanced spatial resolution by synthesizing the fused bands. For example, the fused image may have a spatial resolution of 1 meter while retaining more than 100 bands of the original hyperspectral image, greatly improving the accuracy and reliability of road material recognition. The entire fusion process is automated, and the module automatically adjusts the parameters of the fusion algorithm according to the specific conditions of the image and the preset parameters to achieve the best fusion effect. The fused image data will be used as the input of the feature extraction and classification recognition module, providing a solid foundation for the accurate recognition of road materials.

[0061] The road extraction module first applies masking techniques to identify high reflectivity areas in the image by setting specific thresholds, which usually correspond to road surfaces. For example, the masking technique may be based on the Normalized Difference Water Index (NDWI) and the Normalized Difference Building Index (NDBI), and distinguish between water and buildings by setting thresholds of NDWI > 0.15 and NDBI < 0.35 to identify road areas. The module uses a variety of ground feature indices, such as the Normalized Difference Vegetation Index (NDVI) and Spectral Angle Mapping (SAM), to further refine the extraction of roads. For example, the calculation formula of NDVI is (NIR - Red) / (NIR + Red), where NIR and Red represent the reflectance values ​​of the near-infrared and red bands, respectively. The threshold of NDVI < 0.2 is set to exclude vegetation-covered areas and focus on roads. The road extraction module integrates deep learning semantic segmentation technology and uses convolutional neural networks (CNN) to classify roads at the pixel level. The network is trained with a large number of labeled road and non-road pixels and is able to identify roads in complex scenarios, such as urban roads, rural roads, and roads under different lighting and weather conditions. The network structure may include multiple convolutional layers and pooling layers, as well as fully connected layers for classification, where the filter size of the convolutional layer may be 3x3 or 5x5 to capture road features of different scales. During the training process of the deep learning model, the module uses the cross entropy loss function and the Adam optimizer with a learning rate of 0.001 and a batch size of 32, and iterates the training until the performance of the model on the validation set is no longer significantly improved. After training, the module applies the trained model on new image data to achieve high-precision extraction of road areas. The output of the road extraction module is a binary road mask map, in which road pixels are marked as 1 and non-road pixels are marked as 0. This mask map will be used as the input of the subsequent feature extraction and classification recognition modules to ensure the accuracy and efficiency of the road material recognition process.

[0062] The feature extraction module is specially designed to calculate rich texture, spectral and shape features from the binary road mask obtained by the road extraction module for subsequent classification and recognition. The module adopts multi-scale analysis technology to capture features from local details to global patterns by analyzing road images at different resolution levels. For example, using the Gaussian pyramid decomposition technique, the image is decomposed into four scale levels, and the spatial resolution of each level is halved to extract features at each scale. In order to capture the texture characteristics of road materials, the module applies gray-level co-occurrence matrix (GLCM) and local binary pattern (LBP) to calculate statistics including energy, contrast, uniformity and directionality. The calculation of GLCM is performed in 8 main directions, and the quantization level of each direction is set to 16 to obtain comprehensive texture information. The extraction of spectral features utilizes the spectral curve of the road area. By calculating the reflectance value of each pixel on the selected band, indicators such as average reflectance, standard deviation and spectral slope are extracted. The selection of selected bands is based on the output of the previous band selection module, usually involving 5 to 10 bands that are spectrally most sensitive to distinguish different road materials. The extraction of shape features takes into account the geometric properties of the road, such as the width, length, curvature and area of ​​the road. Through morphological operations and geometric analysis, the module calculates these shape indicators and quantifies the topological structure of the road. Directional analysis identifies the anisotropic characteristics of road materials by using directional filters or Fourier transforms. For example, the frequency domain features extracted by Fourier transform can reveal the distribution characteristics of road materials in different directions. The output of the feature extraction module is a comprehensive feature set, which includes numerical descriptions of texture features, spectral features and shape features. These features provide rich input data for the classification and recognition module, making the automatic recognition of road materials more accurate and reliable. The whole process is automated, and the module automatically performs feature extraction tasks according to preset parameters and algorithms.

[0063] The feature optimization module focuses on selecting the most discriminative feature subsets from the large number of features generated by the feature extraction module. This module uses optimization techniques based on genetic algorithms (GA) and gradient boosting decision trees (GBDT) to ensure that the selected feature set can effectively distinguish different types of road materials. The feature optimization module initializes a feature cluster through the genetic algorithm, and each individual represents a possible feature subset. The initial population size of the algorithm is set to 100 individuals, and the number of features of each individual is 30 to ensure the diversity of the initial population. The number of iterations of the genetic algorithm is set to 50 generations to ensure that the algorithm has sufficient time to explore the solution space. In the genetic algorithm, the fitness function is based on the classification accuracy of the feature subset, which is evaluated by the cross-validation method. The algorithm uses three genetic operations: roulette wheel selection, single-point crossover, and random mutation to generate a new generation of feature subsets. The selection pressure is set to 0.7, the crossover rate is set to 0.8, and the mutation rate is set to 0.01. These parameters are set to balance exploration and exploitation to avoid premature convergence. The feature optimization module uses the gradient boosting decision tree to further optimize the feature subsets selected by the genetic algorithm. The GBDT model uses the F1 score as the criterion for splitting nodes, and the maximum depth of the tree is set to 6 to prevent overfitting. In each iteration, the GBDT model learns the residuals of the previous iteration and optimizes the feature weights by gradually reducing the classification error. The feature optimization module determines the final feature subset by combining the results of the genetic algorithm and the gradient boosting decision tree. The selected feature subset can not only maximize the performance of the classifier, but also reduce the feature dimension and improve the computational efficiency. For example, through the optimization process, the 10 most representative features may be selected in the end, which improves the accuracy of the classification task by 5% and reduces the computational time by about 30%. The entire feature optimization process is automated, and the module automatically performs feature selection and optimization tasks according to the preset parameters and algorithms, ensuring the consistency and repeatability of the feature selection process. The optimized feature subset will be used as the input of the classification and recognition module, providing a solid foundation for the accurate recognition of road materials.

[0064] The classification and recognition module ensures high generalization recognition capabilities for different road materials through a carefully designed machine learning algorithm. The module first uses the support vector machine (SVM) as the main classifier, which is a supervised learning algorithm known for its good performance in small sample cases and its ability to handle high-dimensional data. During the model training phase, the classification and recognition module uses the feature subset selected from the feature optimization module as input and evaluates the performance of the model through a cross-validation method. Specifically, the module uses a 5-fold cross-validation, dividing the dataset into 5 parts, using 4 of them for training in turn, and the remaining 1 for testing, and the cycle is repeated 5 times to obtain the average performance evaluation. The kernel function of SVM is selected as the radial basis function (RBF), which is a commonly used nonlinear kernel suitable for dealing with nonlinear problems in the feature space. The parameters C (penalty coefficient) and γ (kernel function coefficient) of the RBF kernel are optimized by grid search and 5-fold cross validation. The search range is [0.1, 1, 10, 100] for C and [0.001, 0.01, 0.1, 1] for γ, respectively. The search is performed with a step size of 0.1 to find the best parameter combination. In order to further improve the generalization ability of the model, the classification and recognition module also includes model regularization techniques such as L1 and L2 regularization to reduce the risk of overfitting of the model. The coefficient of the L1 regularization term is set to 0.01, while the coefficient of the L2 regularization term is set to 0.001. The values ​​of these coefficients are adjusted according to the performance on the validation set. In the model validation stage, the module uses an independent test set to evaluate the performance of the final model. Performance indicators include accuracy, recall, F1 score and confusion matrix analysis to ensure the effectiveness and reliability of the model in practical applications. The classification and recognition module outputs a classification result in which each pixel or object is assigned a road material category label. For example, the module can distinguish asphalt, concrete and dirt roads with an accuracy of more than 90%, showing its excellent generalization ability and practicality. The whole process is automated, and the module automatically performs model training, parameter optimization and verification processes according to preset parameters and algorithms, ensuring the consistency and repeatability of the classification and recognition process. In this way, the classification and recognition module of the present invention can provide stable and reliable road material automatic recognition services.

[0065] The results post-processing module improves the consistency and integrity of the classification results and ensures the accuracy and reliability of road material identification. The module uses adaptive thresholding technology to binarize the classification results. This technology dynamically adjusts the threshold according to local image characteristics to adapt to the contrast changes in different regions. Specifically, the optimal threshold for each image window is calculated using the Otsu method, which determines the optimal threshold by minimizing the intra-class variance. The window size is set to 50x50 pixels to ensure the local adaptability of the threshold calculation. The module applies the Conditional Random Field (CRF) model to optimize the classification results. As a probabilistic graph model, the CRF model can consider the spatial relationship between pixels and improve the consistency of classification. In the CRF, the module sets the σ value of the Gaussian kernel function to 3 to define the spatial proximity between pixels, and learns features such as color and texture between pixels to optimize the classification boundaries. The results post-processing module also contains morphological operations such as dilation and erosion to eliminate small, isolated misclassified areas while preserving the integrity of the road. The dilation operation uses a 3x3 pixel structural element, while the erosion operation uses a 5x5 pixel structural element. The number of these operations is adjusted according to the specific situation, usually 1 to 2 times. The module performs edge refinement operations, using the Sobel operator or the Canny operator to detect and optimize the road edges to improve the clarity and accuracy of the edges. The threshold parameters for edge refinement are adjusted according to the spectral characteristics of the road material to ensure the accuracy of edge detection. The entire result post-processing process is automated, and the module automatically performs optimization tasks according to preset parameters and algorithms, ensuring the stability and reliability of the classification results. Through this series of post-processing steps, the present invention can significantly improve the quality of road material recognition results and provide users with highly consistent and complete recognition results.

[0066] The user interaction module is used to provide a user-friendly operating experience and high customizability. Through an intuitive graphical user interface (GUI), the module allows users to easily interact with the system to achieve custom settings of parameters, selection of data sources, and visualization and export of recognition results. Users can select different remote sensing data sources, such as GF-1, GF-2 or GF-5 satellite images, and set the time range and spatial range of data acquisition through drop-down menus or sliders. Users can also select specific band combinations as needed and select bands of interest through simple drag and drop operations. The user interaction module provides a series of preset parameters that users can adjust to suit different application scenarios. For example, users can set the contrast enhancement factor in the image enhancement module (such as in the range of 1.2 to 2.0) and select the scale of texture analysis in the feature extraction module (such as 4x4, 8x8 pixel windows). Users can visually view the classification results through an interactive map view and use zoom, pan and mark tools to check and verify the accuracy of road material recognition. Users can also enhance visualization through color coding or transparency adjustment. In terms of result export, the user interaction module provides a variety of format options, including GeoTIFF, PNG and JPEG, to meet different application needs. The user can set the resolution (such as 300 DPI) and size (such as A4, A3) of the exported image, and can choose whether to include geographic coordinate information. The user interaction module also includes a help system that provides online documents, tutorial videos and real-time customer service support to ensure that users can quickly get started and effectively use the automatic identification system of the present invention. The entire user interaction process is designed to be intuitive and easy to use. The module will automatically update the system settings based on the user's input and provide real-time feedback on the processing results, ensuring that users can easily adjust parameters, select data and export results, thereby achieving efficient and personalized automatic recognition of road materials.

[0067] The road material automatic identification method of the present invention is an innovative technical process, which includes the following steps in detail: T1. Collection steps: First, the system starts the remote sensing data acquisition module and actively acquires multi-source remote sensing image data, which may come from different times and different sensors, including hyperspectral and multispectral image data.

[0068] T2. Preprocessing step: Subsequently, the data preprocessing module intervenes to perform key radiometric correction on the collected image data to eliminate sensor bias, atmospheric correction to simulate surface reflection without the influence of atmosphere, geometric correction to ensure the alignment of the data with the geographic coordinate system, as well as cloud occlusion and shadow processing to ensure data quality.

[0069] T3. Band selection step: The band selection module automatically analyzes the image data and selects the bands that are most critical for road material identification. This step is based on spectral feature analysis and may involve calculating the correlation coefficient between bands and selecting the band with the largest variance.

[0070] T4. Image enhancement step: The image enhancement module processes the image data of the selected band, enhances contrast and color, and uses techniques such as histogram equalization and CLAHE to improve the visual effect of the image.

[0071] T5. Image fusion step: Next, the image fusion module uses multi-scale fusion technology to combine image data of different resolutions, improve the spatial resolution through pixel-level and feature-level fusion, and enhance the spatial details of road materials.

[0072] T6. Road extraction step: The road extraction module uses masking technology and feature index to extract road areas, and may use deep learning semantic segmentation technology, such as CNN, for accurate pixel-level classification.

[0073] T7. Feature extraction step: The feature extraction module calculates texture, spectrum and shape features from the extracted road area, and uses multi-scale analysis and directional analysis to comprehensively capture the characteristics of road materials.

[0074] T8. Feature optimization step: The feature optimization module uses genetic algorithms or gradient boosting decision tree methods to select the optimal feature subset from a large number of features to enhance classification performance.

[0075] T9. Classification and recognition step: The classification and recognition module applies support vector machine or other machine learning algorithms, combined with model training and verification processes, to ensure that the classifier has high generalization ability for different road materials.

[0076] T10. Result post-processing step: The result post-processing module uses adaptive threshold technology and conditional random field model to perform morphological operations and other technical optimizations on the classification results to improve the consistency and completeness of the results.

[0077] T11. User Interaction Step: Finally, the user interaction module provides an intuitive user interface that enables users to customize parameters, select data sources, and visualize and export results, ensuring that users can easily adjust parameters, select data, and export results.

[0078] Through this coherent technical process, the present invention realizes high-precision, high-efficiency and high-automatic automatic identification of road materials, meeting the high-standard requirements in the fields of urban planning, road construction and maintenance.

[0079] Compared with the related art, the road material automatic identification system and method based on multi-source remote sensing data fusion provided by the present invention has the following beneficial effects: 1. High-precision road material recognition capability: The present invention significantly improves the recognition accuracy of road materials by integrating multi-source remote sensing data and combining advanced image processing technology and machine learning algorithms. In particular, the band selection module can automatically select the bands that best represent the characteristics of different road materials, and with the image enhancement and feature extraction technology, the system can effectively distinguish between roads of different materials such as asphalt, cement, and soil. In addition, the support vector machine and other algorithms used in the classification and recognition module, and the feature data processed by the feature optimization module, further improve the accuracy of classification and ensure the high accuracy of the recognition results.

[0080] 2. Highly automated identification process: The automatic identification system of the present invention realizes the automation of the entire process from data acquisition to post-processing, greatly reducing the need for manual operation. The data preprocessing module automatically performs radiation correction, atmospheric correction and geometric correction, eliminating the impact of environmental factors on the data. The automated band selection and image fusion process reduces the dependence on professional knowledge and makes the system easier to operate. The user interaction module provides a simple and intuitive operating interface, allowing non-professionals to easily set parameters and view results, further improving the practicality and popularity of the system.

[0081] 3. Powerful data processing and fusion capabilities: The system and method of the present invention effectively integrate remote sensing data of different resolutions and spectral characteristics through the image fusion module, solving the limitations of a single data source in spatial resolution and spectral resolution. The pixel-level and feature-level fusion technology used not only improves the spatial details of the image, but also enhances the richness of spectral information, enabling the system to handle more complex road environments and conditions. In addition, multi-source data fusion also enhances the system's accuracy in identifying road materials under different seasons and lighting conditions, ensuring the stability and reliability of the identification results.

[0082] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A road material automatic identification system based on multi-source remote sensing data fusion, characterized in that: include: A remote sensing data acquisition module, a data preprocessing module, a band selection module, an image enhancement module, an image fusion module, a road extraction module, a feature extraction module, a feature optimization module, a classification recognition module, a result post-processing module and a user interaction module. The remote sensing data acquisition module, the data preprocessing module, the band selection module, the image enhancement module, the image fusion module, the road extraction module, the feature extraction module, the feature optimization module, the classification recognition module, the result post-processing module and the user interaction module are connected in sequence, and the feature extraction module is connected to the classification recognition module.

2. The road material automatic identification system based on multi-source remote sensing data fusion according to claim 1 is characterized in that: The remote sensing data acquisition module is used to obtain hyperspectral and multispectral image data from different sensors; The data preprocessing module is used to perform radiation correction, atmospheric correction, geometric correction, and cloud occlusion and shadow processing; The band selection module is used to automatically select the optimal band combination based on spectral feature analysis; The image enhancement module is used to improve the distinguishability of road materials in the image through contrast enhancement and color correction; The image fusion module is used to improve spatial and spectral resolution by using pixel-level and feature-level fusion techniques; The road extraction module is used to extract the road area through mask technology and ground feature index; The feature extraction module is used to calculate the texture, spectrum and shape characteristics of the road area; The feature optimization module is used to select the optimal feature subset to enhance classification performance; The classification and recognition module is used to classify road materials using a support vector machine or other machine learning algorithms; The result post-processing module is used to optimize the classification results through morphological operations and other techniques; The user interaction module is used to provide a user interface to achieve parameter customization, data source selection and result visualization.

3. The road material automatic identification system based on multi-source remote sensing data fusion according to claim 2 is characterized in that: The data preprocessing module adopts an image denoising step to reduce the impact of sensor noise on subsequent processing; The band selection module uses a statistical analysis-based method to determine the optimal combination of bands to maximize the spectral separation between different road materials.

4. The road material automatic identification system based on multi-source remote sensing data fusion according to claim 2 is characterized in that: The image enhancement module uses a specific algorithm to perform color correction and contrast enhancement on the image to improve the visual effect of the road material; The image fusion module uses multi-scale fusion technology to combine high-resolution images and hyperspectral images to enhance the spatial details of road materials.

5. The road material automatic identification system and method based on multi-source remote sensing data fusion according to claim 2 is characterized in that: The road extraction module also includes a semantic segmentation technology based on deep learning to improve the accuracy of road extraction in complex scenes; The feature extraction module uses multi-scale analysis and directional analysis to fully capture the texture characteristics of road materials. The feature optimization module uses a genetic algorithm or gradient boosted decision tree based approach to select the most discriminative features.

6. The road material automatic identification system and method based on multi-source remote sensing data fusion according to claim 1 is characterized in that: The classification and recognition module also includes a model training and validation process to ensure that the classifier has high generalization ability for different road materials; The result post-processing module adopts adaptive threshold technology and conditional random field model to optimize the consistency and completeness of classification results.

7. A method for automatically identifying road materials using the road material automatic identification system based on multi-source remote sensing data fusion as described in any one of claims 1 to 6, characterized in that: The following steps are involved: S1. Acquisition: Acquire multi-source remote sensing image data through remote sensing data acquisition module; S2, preprocessing: use the data preprocessing module to perform radiation correction, atmospheric correction, geometric correction, and cloud occlusion and shadow processing on the image data; S3, band selection: automatically select the band that is most critical for road material recognition through the band selection module; S4, image enhancement: using the image enhancement module to perform contrast enhancement and color correction on the image data; S5, Image fusion: Use the image fusion module to fuse multi-source image data at the pixel level and feature level to improve the resolution; S6, road extraction: extract road areas through the road extraction module combined with mask technology and object index; S7, feature extraction: using the feature extraction module to calculate texture, spectrum and shape features from the extracted road area; S8, feature optimization: select the optimal feature subset through the feature optimization module; S9, classification and recognition: Apply the machine learning algorithm in the classification and recognition module to classify and recognize the feature data; S10, result post-processing: use the result post-processing module to perform morphological operations and other technical optimizations on the classification results; S11. User interaction: The user interaction module allows users to customize parameters, select data sources, and visualize and export recognition results.

Images