Method for intelligent recognition of road based on heterogeneous remote sensing image fusion by reparameterization optimization

Abstract

The application discloses a kind of based on reparameterization optimization's heterogeneous remote sensing image fusion road intelligent identification method, comprising: S1: collect high-resolution remote sensing image to station as the key area to construct label sample, to generate the image cut piece suitable for model training and verification;S2: adopt multi-polarization sample fusion strategy, fuse two kinds of heterogeneous remote sensing image;S3: based on reparameterization method constructs the model suitable for large-scale remote sensing road extraction, and carries out model training and prediction model reparameterization;S4: based on the feature of road network constructs a buffer grid search scheme, substantially reduces the amount of model calculation;S5: model result filtering and broken line connection.The application is trained by fusing the characteristics of heterogeneous remote sensing image, using larger semantic segmentation model, and using fewer parameters for prediction, while meeting the accuracy and efficiency of the model, can be effectively used for high-resolution satellite remote sensing image road rapid extraction.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method for intelligent road recognition based on heterogeneous remote sensing image fusion with reparameter optimization. Background Technology

[0002] Road network extraction is a challenging task. China's road network is vast, with complex urban and rural roads. Roadside vegetation, such as trees and shrubs, often obscures the roads, making accurate and efficient extraction extremely difficult. Furthermore, rapid urbanization leads to continuous development and changes in urban roads, further complicating extraction. Moreover, road network extraction requires substantial data and computational resources, as well as advanced algorithms. Other challenges arise during extraction. For example, roads are affected by environmental factors such as weather and seasons, which can interfere with extraction. Temporary road closures due to accidents or construction also impact extraction. Additionally, road quality must be considered. Poor road conditions, such as unevenness or narrowness, can result in inaccurate extractions and even threaten traffic safety. Therefore, road quality must be considered during extraction to ensure the safety and reliability of the extracted road network.

[0003] In recent years, remote sensing technology has been widely used in road network extraction. This technology collects remote sensing data from high altitudes or space and processes this data using specific software and algorithms to extract road network information. This method can extract road network information quickly and accurately, and has wide applications in many fields, such as urban planning, traffic management, and disaster emergency response. Furthermore, remote sensing technology eliminates the need for on-site surveys, thus helping to reduce environmental impact. Remote sensing technology can provide high-resolution remote sensing imagery containing information on roads, buildings, and terrain, facilitating accurate road network extraction. Additionally, remote sensing technology can provide real-time data, helping to monitor road network conditions and take timely countermeasures. Synthetic Aperture Radar (SAR) is a technology capable of providing high-resolution remote sensing imagery in dark and inclement weather conditions. It records multiple radar echo signals during the movement of a remote sensing detector and uses post-processing digital techniques to simulate a larger detector aperture, thus achieving a higher resolution than the actual detector aperture. Therefore, SAR technology has wide applications in road extraction from remote sensing imagery. It can provide more detailed and accurate road contour information, facilitating the rapid and accurate extraction of roads from remote sensing imagery. Furthermore, due to the high resolution of synthetic aperture radar technology, it can provide more detailed information, which helps to further identify the type, condition, and surrounding environment of roads.

[0004] Deep learning is a machine learning technique that simulates the learning process of the human brain by designing multi-layered neural networks, enabling it to automatically learn and extract features under the guidance of massive amounts of data. In road network extraction from remote sensing imagery, deep learning can learn road network features by training on large amounts of remote sensing image data, thereby automatically extracting road network information. Because deep learning can automatically learn features, it can effectively extract road network information under various conditions of remote sensing imagery, including different regions, different lighting conditions, and different remote sensing sensors. Furthermore, deep learning also boasts extremely high extraction accuracy and faster extraction speed. By optimizing the network structure and hyperparameter settings, deep learning models can achieve even higher accuracy, thus improving the precision of road network extraction. Moreover, once trained, deep learning models can be directly deployed to edge devices, exhibiting high real-time performance and meeting the needs of real-time road network extraction. Therefore, using deep learning to extract road network information from remote sensing imagery is an efficient and promising method.

[0005] In summary, with the continuous acquisition of high-resolution remote sensing imagery and synthetic aperture radar imagery, there is a significant amount of data redundancy. Therefore, there is an urgent need for an automated, high-precision algorithm to fuse features from heterogeneous remote sensing images and extract urban and rural roads. Summary of the Invention

[0006] This invention addresses the shortcomings of existing technologies by providing a method for intelligent road identification based on heterogeneous remote sensing image fusion using heavy parameter optimization. The aim is to construct a road network extraction model that integrates optical and SAR images, enabling efficient deployment and high accuracy.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for intelligent road recognition based on heterogeneous remote sensing image fusion using reparameter optimization includes the following steps:

[0009] Step S1: Collect high-resolution remote sensing images and construct labeled samples with the station as the key area to generate image patches suitable for model training and validation;

[0010] Step S2: Based on the powerful feature extraction and fusion capabilities of the convolutional neural network itself, an autoencoder is constructed to fuse sample features using multi-polar input. This is an excellent image dimensionality reduction method that can significantly save computing power.

[0011] Step S3: Using multi-GPU parallel computing, a model suitable for large-scale remote sensing road extraction is constructed based on the reparameterization method, and the model is trained.

[0012] Step S4: Construct a buffer grid search scheme based on the characteristics of the road network to significantly reduce the computational load of the model. Based on the continuity of roads, reduce the difficulty of extracting roads from remote sensing images through neighborhood search.

[0013] Step S5: Use the guided filtering algorithm to guide the extracted road network map with the gradient of the image itself, filter out non-edge noise, and use dynamic programming algorithm to stitch the roads together.

[0014] Furthermore, the specific steps of step S1 are as follows:

[0015] Step S11: Based on high-resolution optical satellite remote sensing images covered by road networks, analyze the cloud cover in the optical images and filter out images with cloud cover >5%.

[0016] Step S12: Extract the imaging time of the optical remote sensing image and supplement it with SAR images from adjacent dates. The optical image is the main one, and the SAR image is stitched together and subjected to joint processing of ascending and descending orbits.

[0017] Step S13: Resample the two images to the same resolution, and then construct an image tile containing the road network (urban and rural roads).

[0018] As a preferred option, in step S1, since the model uses reparameterization, the image block is 2048×2048 in order to maximize computing power and improve the model's receptive field.

[0019] Furthermore, the specific steps of step S2 are as follows:

[0020] Step S21: Using a convolutional neural network, construct an autoencoder based on the condition that the input and output are the same image.

[0021] Step S22: Combine the optical remote sensing image and the SAR remote sensing image into a single image by channel stitching.

[0022] Step S23: Register the two different remote sensing images using the mutual information method and sample the images to a consistent size.

[0023] Step S24: Input the autoencoder to perform image fusion in an unsupervised manner.

[0024] Furthermore, the specific steps of step S3 are as follows:

[0025] Step S31: Deploy 4 graphics cards and use PyTorch to connect them in parallel;

[0026] Step S32: A road extraction model is constructed using multiple residual connections by employing reparameterized methods;

[0027] Step S33: Model training and validation. The error function used in the model is composed of the following functions:

[0028]

[0029]

[0030]

[0031] In the formula, M is the image scaling ratio. These are the fused remote sensing images and the predicted results. These represent the mean values ​​of the fused remote sensing images and the prediction results, respectively. These represent the variances of the fused remote sensing images and the prediction results, respectively. [β] represents the covariance between the fused remote sensing image and the prediction result. m ,γ m [c1, c2] are two constants representing the importance of the mean and variance, and [c1, c2] are two very small constant factors to prevent the denominator from being zero. It is the norm gap between the fused remote sensing images and the prediction results. It is the structural similarity gap between the fused remote sensing images and the prediction results; This represents the final loss function used for fine-tuning during supervised training;

[0032] Step S34: Prediction model reparameterization, merging the branches from training into the prediction model structure, improving model speed and reducing computational requirements without affecting model performance; model reparameterization during prediction is performed according to the following formula:

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] In the formula, These represent 3×3 convolution and 1×1 convolution, respectively, where C1 is the number of channels in the input image and C2 is the number of channels in the output image; similarly, μ (i) ,σ (i) ,γ (i) ,β (i) These represent the mean, standard deviation, learning rate, and bias of the corresponding convolution, respectively; bn is a batch normalization method commonly used in deep learning.

[0039] Furthermore, the specific steps of step S4 are as follows:

[0040] Step S41: Establish the starting point of the grid search model, designate the starting point as the root node, and perform the search based on the four-neighborhood and eight-neighborhood of the node.

[0041] Step S42: When a road network is detected in the four-neighbor and eight-neighbor areas, further generate four-neighbor and eight-neighbor areas, and repeat this iteration until all road networks in the image are extracted.

[0042] Furthermore, the specific steps of step S5 are as follows:

[0043] Step S51: Perform morphological filtering based on the morphological closing operation method to reduce noise contamination of the results.

[0044] Step S52: Use the image after feature fusion to perform guided filtering on the extraction results to filter out some non-edge noise in the image.

[0045] Step S53: Construct road connections using a dynamic programming algorithm.

[0046] Preferably, a 10×10 operator is used for morphological filtering in step S51.

[0047] Compared with the prior art, the advantages of the present invention are as follows:

[0048] 1. By combining optical remote sensing imagery and SAR imagery, road information contained in remote sensing images can be better identified. When extracting roads, the model segments roads based on road network features; therefore, even correctly classified road vectors still have some discrepancy with their labels. However, overall, by combining SAR and optical remote sensing imagery, even medium-resolution global features can reduce the impact of vegetation occlusion, significantly suppressing false positive regions, resulting in a more complete road extraction compared to relying solely on optical remote sensing.

[0049] 2. This invention employs a reparameterization method for training and prediction, a novel algorithm highly suitable for large-scale remote sensing data processing. During training, this method uses numerous parameters and branches to achieve multi-level, multi-feature image feature extraction and fusion. During prediction, it significantly improves the model's computational speed and reduces its computational load by merging branches and combining batch normalization with convolution.

[0050] 3. A highly efficient grid neighborhood search algorithm was constructed. By observing that roads are linear targets with strong continuity, the algorithm manually sets the starting point and progressively traverses the road-containing portions of the entire image using four-neighbor and eight-neighbor methods. This method can reduce computational load by up to 90% on high-resolution remote sensing imagery, effectively expanding the model's application scenarios.

[0051] 4. Morphological filtering and guided filtering are used to reduce noise during road extraction, and dynamic programming is used to realize road connections, which greatly improves the overall applicability. Attached Figure Description

[0052] Figure 1 This is a flowchart of an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structured reparameterization in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of parametric pruning prediction in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of mutual information image registration and image channel fusion according to an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of the road extraction model according to an embodiment of the present invention.

[0057] Figure 6 This is a schematic diagram of dynamic planning road connection according to an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with... Figures 1-6 The present invention will be described in detail below.

[0059] Figure 1 This demonstrates the overall architecture design of the present invention, which is divided into five parts: S1, S2, S3, S4, and S5, proceeding sequentially. Each part has a separate output and serves as the input for the next part.

[0060] Figure 2 This demonstrates the modular architecture of a reparameterized network. The overall model is formed by stacking these modules, where the main branch is a 3×3 convolution, and the branches consist of a 1×1 convolution and an identity mapping, respectively. Both the main branch and the branches include a batch normalization process. Figure 2 The right side shows the number of parameters in the model, assuming that both the input and output have two channels.

[0061] Figure 3 This demonstrates the module used in the reparameterized model prediction. This module integrates the batch normalization layer, convolutional layer, and bias parameters through the method shown in step S34. In this way, the number of parameters is different during model training and prediction, but the effect is consistent, effectively saving the computational cost of the model.

[0062] Figure 4 This invention demonstrates a method for registering SAR imagery with optical remote sensing imagery. Since the two types of imagery use different bands and sensors, this is considered heterogeneous remote sensing image registration. Therefore, this invention employs inter-image information technology to calculate the information entropy between the two images and uses a slicing method for registration, reducing distortion that occurs during whole-scene image registration. Simultaneously, high-resolution optical imagery and low-resolution SAR imagery are fused to obtain the remote sensing image dataset required for training.

[0063] Figure 5 The model showcases the optical and SAR remote sensing images used in its design, demonstrating the accurate road vector layer obtained from these two different sources of remote sensing imagery.

[0064] Figure 6 This invention demonstrates a dynamic programming road connection scheme. Roads extracted from remote sensing images are prone to discontinuities. To address this issue, the invention employs a dynamic programming algorithm to automatically select the most suitable direction from multiple directional angles and connect the endpoints of two vector segments, thereby achieving connection of discontinuities. Typically, a certain pixel connection threshold is set to ensure the reliability of the connection.

[0065] This invention provides a method for intelligent road recognition based on heterogeneous remote sensing image fusion using reparameter optimization, comprising the following steps:

[0066] Step S1: Collect high-resolution remote sensing images and construct labeled samples with the station as the key area to generate image tiles suitable for model training and validation. Since the model uses reparameterization, in order to maximize computing power and improve the model's receptive field, the sampling tile is 2048×2048.

[0067] Step S11: Based on high-resolution optical satellite remote sensing images covered by road networks, analyze the cloud cover in the optical images and filter out images with cloud cover >5%.

[0068] Step S12: Extract the imaging time of the optical remote sensing image and supplement it with SAR images from adjacent dates. The optical image is the main one, and the SAR image is stitched together and subjected to joint processing of ascending and descending orbits.

[0069] Step S13: Resample the two images to the same resolution, and then construct a slice containing the road network (urban and rural roads). To maximize computing power and improve the model's receptive field, the sampling slice is constructed as 2048×2048 according to latitude and longitude.

[0070] Step S2: Based on the powerful feature extraction and fusion capabilities of convolutional neural networks, an autoencoder is constructed that uses multi-polar input to fuse sample features. This is an effective image dimensionality reduction method that can significantly save computational resources.

[0071] Step S21: Using a convolutional neural network, construct an autoencoder based on the condition that the input and output are the same image.

[0072] Step S22: Combine the optical remote sensing image and the SAR remote sensing image into a single image by channel stitching.

[0073] Step S23: Register the two different remote sensing images using the mutual information method and sample the images to a consistent size.

[0074] Step S24: Input the autoencoder to perform image fusion in an unsupervised manner.

[0075] Step S3: Using multi-GPU parallel computing, construct a model suitable for large-scale remote sensing road extraction based on the reparameterization method, and train the model:

[0076] Step S31: Deploy 4 NVIDIA 3080 graphics cards and use PyTorch to connect them in parallel.

[0077] Step S32: Use a reparameterized approach to construct a road extraction model with multiple residual connections.

[0078] Step S33: Model training and validation. The error function used in the model is mainly composed of the following functions:

[0079]

[0080]

[0081]

[0082] In the formula, M is the image scaling ratio. These are the fused remote sensing images and the predicted results. These represent the mean values ​​of the fused remote sensing images and the prediction results, respectively. These represent the variances of the fused remote sensing images and the prediction results, respectively. [β] represents the covariance between the fused remote sensing image and the prediction result. m ,γ m [c1, c2] are two constants representing the importance of the mean and variance, and [c1, c2] are two very small constant factors to prevent the denominator from being zero. It is the norm gap between the fused remote sensing images and the prediction results. It is the structural similarity gap between the fused remote sensing images and the prediction results; This represents the final loss function used for fine-tuning during supervised training.

[0083] Step S34: Prediction model reparameterization, merging the training branches into the prediction model structure, significantly improving model speed and reducing computational requirements without affecting model performance. Prediction model reparameterization is performed according to the following formula:

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] In the formula, These represent 3×3 convolution and 1×1 convolution, respectively, where C1 is the number of channels in the input image and C2 is the number of channels in the output image; similarly, μ (i) ,σ (i) ,γ (i) ,β (i) These represent the mean, standard deviation, learning rate, and bias of the corresponding convolution, respectively; bn is a batch normalization method commonly used in deep learning.

[0090] Step S4: Construct a buffer grid search scheme based on the characteristics of the road network to significantly reduce the computational load of the model. Based on the continuity of roads, reduce the difficulty of extracting roads from remote sensing images through neighborhood search.

[0091] Step S41: Establish the starting point of the grid search model, designate the starting point as the root node, and perform the search based on the four-neighborhood and eight-neighborhood of the node.

[0092] Step S42: When a road network is detected in the four-neighbor and eight-neighbor areas, further generate four-neighbor and eight-neighbor areas, and repeat this iteration until all road networks in the image are extracted.

[0093] Step S5: Use the guided filtering algorithm to guide the extracted road network map with the gradient of the image itself, filter out non-edge noise, and use dynamic programming algorithm to stitch the roads together.

[0094] Step S51: Based on the morphological closing operation method, a 10×10 operator is used for morphological filtering to reduce noise contamination of the results.

[0095] Step S52: Use the image after feature fusion to perform guided filtering on the extraction results to filter out some non-edge noise in the image.

[0096] Step S53: Construct road connections using a dynamic programming algorithm.

[0097] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.

Claims

1. A method for intelligent road recognition based on heterogeneous remote sensing image fusion using reparameter optimization, characterized in that, Includes the following steps: Step S1: Collect high-resolution remote sensing images and construct labeled samples with the station as the key area to generate image patches suitable for model training and validation; Step S2: Construct an autoencoder that fuses sample features using multi-polar input; The specific steps of step S2 are as follows: Step S21: Using a convolutional neural network, an autoencoder is constructed based on the condition that the input and output are the same image; Step S22: Combine the optical remote sensing image and the SAR remote sensing image into a single image using channel stitching; Step S23: Register the two different remote sensing images using the mutual information method and sample the images to a uniform size; Step S24: Input the autoencoder to perform image fusion in an unsupervised manner; Step S3: Using multi-GPU parallel computing, a model suitable for large-scale remote sensing road extraction is constructed based on the reparameterization method, and the model is trained. The specific steps of step S3 are as follows: Step S31: Deploy 4 graphics cards and use PyTorch to connect them in parallel; Step S32: Construct a road extraction model using a reparameterized approach; Step S33: Model training and validation; Step S34: Prediction model reparameterization, which merges the branches during training into the prediction model structure, improving the model speed and reducing the model's computational requirements without affecting the model's performance; Step S4: Construct a buffer grid search scheme based on the characteristics of the road network to significantly reduce the computational load of the model. Based on the continuity of roads, reduce the difficulty of extracting roads from remote sensing images through neighborhood search. The specific steps of step S4 are as follows: Step S41: Establish the starting point of the grid search model, specify the starting point as the root node, and perform the search based on the four-neighborhood and eight-neighborhood of the node; Step S42: When a road network is detected in the four-neighbor and eight-neighbor areas, further generate four-neighbor and eight-neighbor areas, and repeat this iteration until all road networks in the image are extracted; Step S5: Use the guided filtering algorithm to guide the extracted road network map with the gradient of the image itself, filter out non-edge noise, and use the dynamic programming algorithm to stitch the roads together; The specific steps of step S5 are as follows: Step S51: Perform morphological filtering based on the morphological closing operation method to reduce noise contamination of the results; Step S52: Apply guided filtering to the extracted results using the image after feature fusion to filter out some non-edge noise in the image; Step S53: Construct road connections using a dynamic programming algorithm.

2. The method for intelligent road recognition based on heterogeneous remote sensing image fusion using reparameter optimization as described in claim 1, characterized in that: The specific steps of step S1 are as follows: Step S11: Based on high-resolution optical satellite remote sensing images covered by road networks, analyze the cloud cover in the optical images and filter out images with cloud cover >5%. Step S12: Extract the imaging time of the optical remote sensing image and supplement it with SAR images from adjacent dates. The optical image is the main one, and the SAR image is stitched together and subjected to joint processing of the ascent and descent orbits. Step S13: Resample the two images to the same resolution, and then construct an image block slice containing the road network.

3. The method for intelligent road recognition based on heterogeneous remote sensing image fusion according to claim 2, characterized in that: In step S1, since the model uses reparameterization, the image block is 2048×2048 in order to maximize computing power and improve the model's receptive field.

4. The method for intelligent road recognition based on heterogeneous remote sensing image fusion according to claim 1, characterized in that: The error function used in step S33 is composed of the following functions: In the formula, M is the image scaling ratio. These are the fused remote sensing images and the predicted results. These represent the mean values ​​of the fused remote sensing images and the prediction results, respectively. These represent the variances of the fused remote sensing images and the prediction results, respectively. [β] represents the covariance between the fused remote sensing image and the prediction result. m ,γ m [c1, c2] are two constants representing the importance of the mean and variance, and [c1, c2] are constant factors of the two values ​​to prevent the denominator from being zero. It is the norm gap between the fused remote sensing images and the prediction results. It is the structural similarity gap between the fused remote sensing images and the prediction results; This represents the final loss function used for fine-tuning during supervised training.

5. The method for intelligent road recognition based on heterogeneous remote sensing image fusion according to claim 4, characterized in that: In step S34, the model reparameters are adjusted during prediction according to the following formula: bn(M*W,μ,σ,γ,β) :,i,:,: =(M*W′) :,i,:,: +b′ i (7) In the formula, These represent 3×3 convolution and 1×1 convolution, respectively, where C1 is the number of channels in the input image and C2 is the number of channels in the output image; similarly, μ (i) ,σ (i) ,γ (i) ,β (i) These represent the mean, standard deviation, learning rate, and bias of the corresponding convolution, respectively; bn is the batch normalization method in deep learning.

6. The method for intelligent road recognition based on heterogeneous remote sensing image fusion according to claim 1, characterized in that: In step S51, a 10×10 operator is used for morphological filtering.

Images