Remote sensing image road extraction method based on frequency domain feature extraction and fusion

Abstract

The invention relates to the technical field of remote sensing image road extraction, in particular to a remote sensing image road extraction method based on frequency domain feature extraction and fusion, which comprises a feature extractor, a bottleneck layer and a segmentation branch. The feature extractor adopts FCB to replace traditional convolution operation, and performs frequency domain filtering operation on a frequency domain to perform feature extraction; the bottleneck layer adopts a KAN Layer to learn complex road features; according to the segmentation branch, the feature map is recovered to the original size through up-sampling, multi-scale feature learning and fusion are achieved through a multi-scale pyramid, and finally the feature map is sent to an output layer to output a prediction result. According to the method, the frequency domain convolution with the global receptive field, the bottleneck layer capable of learning the activation function and the multi-scale image reasoning fusion mechanism are introduced, the extraction precision is improved, the model parameter quantity and the energy consumption are reduced, and the high-efficiency and high-precision remote sensing image road real-time extraction requirement can be met.

Description

Technical Field

[0001] This invention relates to the field of road extraction technology in remote sensing images, specifically a method for road extraction from remote sensing images based on frequency domain feature extraction and fusion. Background Technology

[0002] Remote sensing imagery has important applications in urban planning, traffic management, and environmental monitoring, with accurate extraction of road information being particularly crucial. Especially in urban areas, small, narrow, and obscured roads often form an important part of the transportation network, but these roads are easily overlooked or incompletely extracted using traditional methods, affecting the continuity and accuracy of the overall road network.

[0003] Traditional road extraction methods are limited by complex terrain, shadows, and noise interference, resulting in poor performance in identifying small and broken roads, and are prone to breaks or missed detections. In recent years, the development of deep learning technology has provided new ideas for improving road extraction accuracy. Extraction methods based on fully convolutional networks and U-shaped networks can automatically learn multi-level image features and perform well in identifying main roads. However, these models still face difficulties when dealing with small and broken roads, such as insufficient feature representation and insensitivity to local structures, leading to small roads being easily misclassified as background areas, resulting in broken or discontinuous extraction results.

[0004] Therefore, this invention proposes a method for road extraction from remote sensing images based on frequency domain feature extraction and fusion. Summary of the Invention

[0005] The purpose of this invention is to provide a remote sensing image road extraction method based on frequency domain feature extraction and fusion, focusing on improving the detection and continuity restoration capabilities of small and broken roads. By introducing frequency domain features to enhance local texture information and combining a multi-scale feature fusion strategy, detailed structures are effectively captured, improving the recognition accuracy and connectivity of small roads, and providing a better solution for the refined extraction of urban road networks.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The road remote sensing image to be processed is input into a multi-layer feature extractor constructed by the frequency domain convolution operation module FCB. The frequency domain convolution operation module FCB performs frequency domain filtering operation in the frequency domain using the global receptive field, while performing downsampling and dimensionality upsampling to output a high-dimensional feature map. The high-dimensional feature map is input into the bottleneck layer constructed by the KAN layer, and a nonlinear feature transformation is performed on the high-dimensional feature map using a learnable activation function parameterized by spline function to output the bottleneck layer feature map. The bottleneck layer features are input into the segmentation branch, and the bottleneck layer features are upsampled to restore the original image size and maintain the high number of dimensional channels. The upsampled feature map is then output. The upsampled feature map is input into a multi-scale feature fusion pyramid constructed by a graph inference module for multi-scale feature learning and fusion, and multi-scale fused features are output. The multi-scale fused features are input into the output layer for channel compression and normalization, and the road extraction results of the remote sensing image are output.

[0007] Preferably, the specific calculation steps of the frequency domain convolution operation module FCB include: The frequency domain convolution operation module FCB receives the input spatial domain feature map, and the spatial domain feature map received by the first-layer frequency domain convolution operation module FCB is the remote sensing image to be processed. The spatial domain feature map is subjected to a Fourier transform. A learned frequency domain convolution kernel is defined. The frequency domain convolution kernel is multiplied element-wise with the spectrum of the transformed spatial domain feature map. The result is then transformed by an inverse Fourier transform to output a new spatial domain feature map. The specific formula is as follows: ; in For the new spatial domain feature map, The input is a spatial domain feature map. For Fourier transform, For inverse Fourier transform, For frequency domain convolution kernels, In the frequency domain With frequency domain convolution kernel Perform element-wise multiplication.

[0008] Preferably, the specific definition of the frequency domain convolution kernel for learning includes: The frequency domain convolution kernel has a size of N*N, where N is the size of the input spatial domain feature map; the frequency domain convolution kernel Convolution kernel in the spatial domain The Fourier transform yields: ; ; Spatial domain convolution kernel It is a frequency domain convolution kernel The inverse Fourier transform is represented by a smaller kernel. The zero-fill version, with a smaller core The size is K, where K is a positive integer and 1 ≤ K ≤ N: .

[0009] Preferably, the specific parameter settings of the KAN Layer include: The KAN Layer employs a learnable activation function, and each weight parameter in the KAN Layer is replaced by a univariate function; the univariate function is parameterized in spline function form; the calculation formula for the KAN Layer is: ; in The number of KAN layers. For the first Layer The function matrix outputs the bottleneck layer feature map.

[0010] Preferably, the specific construction of the segmented branch includes: The segmentation branch consists of an upsampling layer, a multi-scale feature fusion pyramid, and an output layer. The upsampling layer is implemented using transposed convolution, which upscales the bottleneck layer features by X (x*x) to the original image size Y (y*y), outputting an upsampled feature map. The principle of transposed convolution is as follows: ; Where y is the original image size, s is the stride, p is the padding, k is the kernel size, and o is the output compensation. The multi-scale fusion pyramid structure contains three graph inference modules, two transposed convolutional upsampling layers, and two FCB modules forming a downsampling layer.

[0011] Preferably, the specific processing procedure of the multi-scale feature fusion pyramid includes: The upsampled feature map is fed into the multi-scale feature fusion pyramid constructed by the graph inference module. Specifically, the upsampled feature map is downsampled three times to obtain three different scale feature maps: 1 / 2, 1 / 4, and 1 / 8. Each type of feature map is fed into the graph inference module to obtain the enhanced multi-scale feature map. The learned 1 / 8 feature map is upsampled to 1 / 4 and fused with the learned 1 / 4 feature map. The fused 1 / 4 feature map is upsampled to 1 / 2 and fused with the learned 1 / 2 feature map to output the multi-scale fused feature.

[0012] Preferably, the specific processing procedure of the graph reasoning module includes: The upsampled feature map X (n*n) is input into the graph inference module and flattened into a node set x, where the i-th node is denoted as . Calculate the relationship weight matrix A between nodes, where nodes With nodes Relationship weights The relationship is calculated using the Gaussian similarity function, then: ; Using the relation weight matrix A, inference information is transmitted to the node set x to obtain the inference result y. ; In element form: ; y is the result of reasoning. It is the i-th element in y; Reshape the inference result y vector back into a two-dimensional image Y: ; The enhanced multi-scale feature map is obtained.

[0013] Preferably, the specific processing procedure of the output layer includes: The specific formula for inputting the multi-scale fused features into the output layer for channel compression and normalization is as follows: ; Where Z represents the multi-scale fusion feature. Here, b is a 1×1 convolution kernel used for channel compression, and b is compensation. is the Sigmoid activation function, and Y is the road extraction result from the remote sensing image.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention inputs the remote sensing image of the road to be processed into a multi-layer feature extractor constructed by a frequency domain convolution operation module (FCB). The FCB performs frequency domain filtering in the frequency domain using the global receptive field, while simultaneously performing downsampling and dimensionality upsampling. Because the global receptive field inherent in the frequency domain better preserves global features while achieving the same convolution operation as the spatial domain, it can capture global information faster and better. This results in more complete preservation of global features during downsampling, solving the problem that traditional convolution kernels easily lose global features in the original image during feature extraction, thus ensuring the continuity of the road network.

[0015] 2. This invention inputs the high-dimensional feature map into a bottleneck layer constructed from a KAN layer, and uses a learnable activation function parameterized by spline functions to perform nonlinear feature transformation on the high-dimensional feature map. Unlike traditional neural networks that use fixed activation functions, KAN uses learnable activation functions at the network edges. This design allows each weight parameter in KAN to be replaced by a univariate function, providing extremely high flexibility. This enables the model to simulate complex functions with fewer parameters, resulting in higher computational efficiency and meeting the needs of real-time road extraction.

[0016] 3. This invention inputs the upsampled feature map into a multi-scale feature fusion pyramid constructed by a graph inference module for multi-scale feature learning and fusion. By leveraging the interconnectedness between nodes through the graph inference module and the interactive propagation of neighbor information, each node can infer richer semantic or relational information. Combined with a multi-scale feature fusion strategy, it effectively captures detailed structures, improving the recognition accuracy and connectivity of narrow roads. This effectively solves the problems of easily broken or missed detections, especially for long, narrow, and obscured broken roads in urban areas. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a remote sensing image road extraction method based on frequency domain feature extraction and fusion proposed in this invention. Figure 2 A schematic diagram of the system structure of the model proposed in the embodiment; Figure 3 This is a schematic diagram of the FCB module proposed in the embodiment; Figure 4 This is a schematic diagram of the structure of the KAN Layer module proposed in the embodiment; Figure 5 This is a schematic diagram of the multi-scale feature fusion pyramid structure proposed in the embodiment; Figure 6 This is a schematic diagram of the road extraction results from remote sensing images proposed in the example. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be understood that the terms "center," "lateral," "upper," "lower," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or component referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Additionally, the term "comprising" and any variations thereof mean "at least comprising."

[0020] This invention provides a method for road extraction from remote sensing images based on frequency domain feature extraction and fusion. The technical solution is as follows: The road remote sensing image to be processed is input into a multi-layer feature extractor constructed by the frequency domain convolution operation module FCB. The frequency domain convolution operation module FCB performs frequency domain filtering operation in the frequency domain using the global receptive field, while performing downsampling and dimensionality upsampling to output a high-dimensional feature map. The high-dimensional feature map is input into the bottleneck layer constructed by the KAN layer, and a nonlinear feature transformation is performed on the high-dimensional feature map using a learnable activation function parameterized by spline function to output the bottleneck layer feature map. The bottleneck layer features are input into the segmentation branch, and the bottleneck layer features are upsampled to restore the original image size and maintain the high number of dimensional channels. The upsampled feature map is then output. The upsampled feature map is input into a multi-scale feature fusion pyramid constructed by a graph inference module for multi-scale feature learning and fusion, and multi-scale fused features are output. The multi-scale fused features are input into the output layer for channel compression and normalization, and the road extraction results of the remote sensing image are output.

[0021] Example 1: This embodiment provides a specific application of a remote sensing image road extraction method based on frequency domain feature extraction and fusion, referencing... Figure 1The specific steps are as follows: The remote sensing image of the road to be processed is input into a multi-layer feature extractor constructed by a frequency domain convolution operation module (FCB). The FCB performs frequency domain filtering in the frequency domain using the global receptive field, while simultaneously downsampling and upsampling, outputting a high-dimensional feature map. The high-dimensional feature map is input into a bottleneck layer constructed by a KAN layer. A learnable activation function parameterized by spline functions is used to perform nonlinear feature transformation on the high-dimensional feature map, outputting a bottleneck layer feature map. The bottleneck layer features are input into a segmentation branch, where they are upsampled to restore the original image size while maintaining the number of high-dimensional channels, outputting an upsampled feature map. The upsampled feature map is input into a multi-scale feature fusion pyramid constructed by a graph inference module, where multi-scale feature learning and fusion are performed, outputting multi-scale fused features. The multi-scale fused features are input into the output layer for channel compression and normalization, outputting the road extraction result from the remote sensing image.

[0022] Furthermore, a feature extractor is established. The feature extractor is designed based on the set input data size and consists of one or more FCB modules, each module being a downsampling layer. The function of each downsampling layer (i.e., FCB module) is to downsample the feature map size to half the size of the input feature map at that layer and increase the number of channels to the preset number of layers, thereby achieving feature extraction and dimensionality enhancement of the source data. The specific process is as follows: The principle of FCB is as follows: input the spatial domain feature map, perform Fourier transform, define a learned frequency domain convolution kernel, perform element-wise multiplication between the frequency domain convolution kernel and the spectrum of the transformed feature map, and then output the result in the 2D spatial domain through inverse Fourier transform.

[0023] The entire feature extractor process is as follows: design X FCB module components and X downsampling layers based on the input size, and determine the dimension of each layer based on the device memory. , , … The first layer receives the remote sensing image of the road to be processed, performs a downsampling operation, and downsamples the image to its original size. Figure 1 / 2, channel count increased to The second layer receives the feature map data output from the first layer and performs downsampling again, downsampling the image to its original value. Figure 1 / 4, channel count increased to After passing through X layers, the image size becomes 1 / (2*X), and the number of image channels increases to the set dimension. This yields a high-dimensional feature map.

[0024] The feature extractor utilizes a multi-layer FCB module to downsample the image to the original... Figure 1During the process of / 8, the number of channels can be smoothly increased in a preset dimension simultaneously, thereby efficiently extracting and upsizing the features of the source data while reducing the spatial resolution, providing a rich high-dimensional data foundation for subsequent learning of complex road features.

[0025] refer to Figure 2 In this embodiment, the model consists of three FCB modules. Each downsampling operation extracts features and upscales the feature map to a higher dimension. Each downsampling operation reduces the size of the input feature map to half. After three FCB modules, the final feature map is 1 / 8 the size of the original image. Because the frequency domain inherently possesses a global receptive field, it better preserves global features while achieving the same convolutional operations as the spatial domain.

[0026] FCB Module Reference Figure 3 The principle is as follows: A spatial domain feature map is input and subjected to a Fourier transform. A learned frequency domain convolution kernel is defined. The frequency domain convolution kernel is multiplied element-wise with the spectrum of the transformed feature map. Finally, the result is output in the 2D spatial domain through an inverse Fourier transform. The specific formula is: ; in To output the spatial domain feature map, For input and output spatial domain feature maps, For Fourier transform, For inverse Fourier transform, Fourier convolution kernel, In the frequency domain With frequency domain convolution kernel Perform element-wise multiplication.

[0027] The kernel is a Fourier convolution kernel of size N*N, where N is the size of the input feature map. It is obtained by spatial domain convolution through kernel Fourier transform. ; ; It is the frequency domain core in the FCB module. The inverse Fourier transform is represented by a relatively small kernel. Zero-padded version of (size K, where 1≤K≤N): ; The frequency domain convolution W has the same size as the input image, N*N, which translates to N*N in the frequency domain. Performing an inverse Fourier transform on the frequency domain convolution W can be viewed as a small spatial domain convolution kernel w (which can be understood as the frequency domain representation in the spatial domain / the frequency domain kernel being converted to a spatial domain representation). However, w differs from traditional convolution kernels; it might be a full-scale kernel N*N, or... An N*N kernel consisting of a small kernel and an outer layer padded with zeros. When it is a full-scale N*N kernel, it represents the global receptive field; when it is a small kernel, it represents the global receptive field. When there is an N*N core consisting of a small core and an outer layer filled with zeros, it is equivalent to having only one It is at work, that is, the local receptive field.

[0028] By adjusting the size of the frequency domain kernel mapped to the spatial domain in the FCB module, when the kernel w is full-scale... When mapped to a small kernel and zero-padding is applied, a global receptive field can be utilized; conversely, when mapped to a small kernel and zero-padding is applied, a local receptive field can be obtained. This characteristic allows the feature extraction stage to flexibly adjust the focus scale as needed, thereby enabling targeted extraction of road features of different scales.

[0029] Furthermore, a bottleneck layer is designed for feature learning, and the high-dimensional feature map output by the feature extractor is used to learn more complex road features, referencing... Figure 2 The bottleneck layer is specifically replaced by a KAN layer instead of the traditional bottleneck layer in UNET. Unlike traditional neural networks that use fixed activation functions, KAN uses learnable activation functions at the network edges. The specific process is as follows: Traditional bottleneck layers typically employ an MLP structure, where each connection has a linear weight, and each node has a fixed activation function (ReLU, sigmoid, etc.) applied to the weighted sum. In other words: Input → Linear Transformation (Weights + Bias) → Activation Function → Next Layer. In KAN, however, the activation function is fixed at the node, and each connection uses a learnable one-dimensional function instead of the traditional "weights × input". The KAN structure combines "weights × activation" into one, allowing each edge to have a complex one-dimensional nonlinear effect, enabling more flexible fitting of complex relationships.

[0030] KAN layer design reference Figure 4 This design allows each weight parameter in a KAN to be replaced by a univariate function, typically parameterized as a spline function, thus providing high flexibility and enabling the simulation of complex functions with fewer parameters. The computational formula for a KAN with L layers is as follows: ; in For the l-th layer Function matrix.

[0031] The high-dimensional feature map is transformed nonlinearly using a function to output a bottleneck layer feature map.

[0032] Furthermore, a segmentation branch is constructed to achieve multi-scale feature learning and output. The segmentation branch consists of an upsampling layer, a multi-scale feature fusion pyramid, and an output layer. The specific process is as follows: The upsampling layer is implemented using transposed convolution, which upscales the bottleneck layer feature map X (x*x) obtained by the feature extractor and the bottleneck layer to the original image size Y (y*y), resulting in the upsampled feature map. The principle of transposed convolution is as follows: ; Where s is the stride, p is the padding, k is the kernel size, and o is the output compensation; The multi-scale fusion pyramid structure comprises three graph inference modules, two transposed convolutional upsampling layers, and two downsampling layers composed of FCB modules. The multi-scale feature fusion pyramid first receives the upsampled feature map, then passes it through transposed convolution to obtain a high-dimensional feature map X at the original image size. It then passes through two downsampling layers constructed by the FCB modules to perform downsampling operations, each downsampling to half the size of the road remote sensing image, thus obtaining... and X, and The data are then fed into the graph reasoning module for processing to obtain Y. and Then Obtained by upsampling via transposed convolution , and Feature fusion is performed using average summation to obtain... Then... Z' is obtained by transposing convolution and upsampling. Z' is then fused with Y to finally output the multi-scale fused feature Z.

[0033] The principle of graph reasoning is that, given the connections between nodes (graph structure), each node can infer richer semantic or relational information through the interaction and propagation of neighbor information. The specific steps are flattening, relation building, reasoning, and restoration. The input feature map X (n*n) of the graph reasoning module is treated as an n*n node, with each pixel considered as a node. The first step is to flatten X into a one-dimensional shape: ; Let x be a set of nodes, and let the i-th node be denoted as . ; The second step is to calculate the relationship between every two nodes, usually using a similarity-based method, denoted as... for and The relationship between two nodes is as follows: ; Depend on The resulting matrix A is the weight matrix representing the relationships between the nodes.

[0034] The third step is reasoning (information transmission). Each node receives information from all its neighboring nodes, written in matrix-vector form: ; In element form: ; y is the result of reasoning. It is the i-th element in y.

[0035] The final step is to reshape the y-vector obtained from the inference back into a two-dimensional image Y: ; The multi-scale fused features, processed by the multi-scale feature fusion pyramid, are fed into the output layer. The output layer transforms the multi-scale fused features Z into the final output result. First, a 1*1 convolution is used to compress the channels to 1, and then a sigmoid activation function is used to obtain the final remote sensing image road extraction result Y, i.e.: ; in The kernel is 1*1, and b is the compensation. This is the Sigmoid activation function.

[0036] For reference Figure 2 The segmentation branch is described in this example. The feature extractor has downsampled the road remote sensing image to be processed to 1 / 8 and performed feature enhancement using a KAN layer. In the segmentation branch, a transposed convolution with a stride of 8, a kernel size of 16, padding of 4, and compensation of 0 is first used to output the bottleneck layer features to the original image size while maintaining a high number of channels, resulting in an upsampled feature map. This upsampled feature map is then fed into a multi-scale feature fusion pyramid constructed by the graph inference module. (For reference, see [link to multi-scale feature fusion pyramid]). Figure 5 The specific operation is as follows: The feature map upsampled by transposed convolution is downsampled twice by two FCB modules, each downsampling to half the input of the module. This yields three different scale feature maps: original size, half size, and quarter size. Each type of feature map is fed into the graph inference module to obtain enhanced multi-scale feature maps. Then, the learned quarter size is upsampled to half size using transposed convolution with a stride of 4, a kernel size of 8, padding of 2, and compensation of 0, and fused with the learned half size feature map. The fused half size feature map is then upsampled to the original size using transposed convolution with a stride of 2, a kernel size of 4, padding of 1, and compensation of 0, and fused with the learned original size feature map. Finally, it is fed into the output layer for normalization and output to obtain the reference road extraction result of the remote sensing image. Figure 6 .

[0037] In summary, the embodiments of this invention provide a remote sensing image road extraction method based on frequency domain feature extraction and fusion. First, a feature extractor is constructed using the frequency domain convolution module (FCB). Frequency domain filtering is performed using the built-in global receptive field, achieving downsampling and dimensionality upsampling while preserving global features. This design captures global contextual information more quickly, thus more completely preserving the global connectivity features of the road during the feature extraction stage, effectively solving the common problem of road fragmentation in road extraction. Furthermore, a learnable activation function parameterized by spline functions is used to construct a KAN layer at the bottleneck layer, replacing the weight parameters with univariate functions. This gives the model extremely high flexibility, enabling it to simulate complex nonlinear road features with fewer parameters, reducing the number of model parameters while improving feature fitting ability. A multi-scale feature fusion pyramid is constructed using a graph inference module, utilizing graph inference mechanisms for information transfer and reasoning between nodes. Combined with the multi-scale fusion strategy, the model can effectively mine deep semantic relationships between pixels, accurately capturing the detailed structures of small, fragmented, and occluded roads, thereby improving the recognition accuracy and connectivity of small target roads.

[0038] Example 2: Under the same parameters and dataset configuration, the Unet network was trained, and the results were compared. (Results are referenced.) Figure 6 Compared to the results from the Unet network, it has certain advantages in extracting narrow and fragmented roads.

[0039] Here is a specific data example: 1. Dataset The dataset used is the DeepGlobe Roads dataset, which includes 8570 RGB three-channel satellite remote sensing images. The images have a resolution of 1024×1024 pixels and a spatial resolution of 0.5 meters per pixel, covering a land area of ​​2220 square kilometers and encompassing various road types in both urban and rural areas. Of the 8570 RGB three-channel satellite remote sensing images, 6226 are open-source labeled images. Therefore, from these 6226 labeled images, 4696 images are used as the training set, and 1530 images are used as the test set.

[0040] 2. Evaluation Indicators mIoU is the average intersection-union ratio, calculated as the average ratio of the intersection and union of all classes. ; The F1 score is the harmonic mean of precision and recall, used to comprehensively reflect the algorithm's performance. ; ; ; Where N represents the number of categories, TP stands for true positive (the number of correctly identified road pixels), FN stands for false negative (the number of road pixels misidentified as non-road pixels), and FP stands for false positive (the number of non-road pixels misidentified as road pixels). Precision is the number of samples correctly predicted as positive (TP) divided by the total number of samples classified as positive (TP+FP). Recall is the number of samples correctly predicted as positive (TP) divided by the total number of actual positive samples (TP+FN).

[0041] These two evaluation metrics are used because they are widely used in the fields of computer vision and pattern recognition. The F1 score is robust to imbalanced data, while the mIoU score calculates the value of the overlap between the predicted result and the ground truth label divided by the joint region, providing a global assessment of segmentation quality.

[0042] 3. Implementation details The training configuration used a stochastic gradient descent optimizer with a momentum of 0.9 and a loss function of BCE_Dice_Loss.

[0043] Training parameters: Input image size is 1024*1024, initial learning rate is set to 1e. -4 The weight decay is set to 1e. -5 The initial weights of the KAN layer are set to 1e. -2 The weight decay is set to 1e. -3 The training rounds are set to 300, and the batch size is set to 4.

[0044] Model comparison: Under the same parameters and dataset configuration, the Unet network was trained, and the results were compared. (See reference for details.) Figure 6 .

[0045] Data augmentation involves using methods such as random rotation, horizontal flipping, Gaussian blur, and rescaling to improve the generalization ability of the model, and using the mean and variance of the ImageNet dataset to normalize the training data.

[0046] Example 3 This third embodiment constructs a complete experimental verification environment based on the standard remote sensing dataset disclosed in embodiment two, and demonstrates the practical ability of the method to process complex remote sensing images through a rigorous training and testing process; the specific implementation process is as follows: The experimental data used was from the DeepGlobe Road Extraction Challenge dataset, one of the most widely recognized benchmark datasets in the field of remote sensing road extraction. This dataset contains 8570 RGB three-channel satellite remote sensing images. These images have extremely high resolution, with a uniform resolution of 1024×1024 pixels and a spatial resolution of 0.5 meters per pixel, clearly presenting surface details. The dataset covers a total land area of ​​approximately 2220 square kilometers, encompassing various geographical environments including urban, rural, and wilderness areas, and including a variety of complex road types, from wide main roads to narrow rural dirt roads. This allows for a thorough evaluation of the model's feature extraction and generalization capabilities across different geographical scenarios. Regarding the data partitioning, to ensure sufficient model parameter learning and objective performance evaluation, 4696 images were randomly selected from the 6226 open-source images with pixel-level labels to construct the training set for gradient updates and parameter optimization. The remaining 1530 images served as the test set, used to evaluate the model's predictive performance on unseen data after training.

[0047] Before feeding remote sensing image data into the network for training, a rigorous data preprocessing and augmentation strategy was formulated to improve the model's adaptability to complex environments and prevent overfitting. Specifically, the training data underwent various augmentation operations: random rotation and horizontal flipping were used to simulate road morphology under different shooting angles, enabling the model to learn rotation-invariant characteristics; Gaussian blur was applied to simulate atmospheric interference or sensor noise, enhancing the model's noise resistance in low-quality images; and rescaling was used to help the model adapt to road targets of different widths, ensuring the multi-scale feature fusion mechanism could achieve maximum effectiveness. After augmentation, to accelerate model convergence and maintain data distribution consistency, all input images were standardized using the mean and variance of the ImageNet dataset, mapping the data to a unified feature space and laying a solid data foundation for subsequent frequency domain feature extraction.

[0048] During the model training and configuration phase, a training environment based on a deep learning framework was built, and the network structure and hyperparameters were finely configured. The optimizer selected was the stochastic gradient descent (SGD) algorithm, which exhibits good convergence stability when processing large-scale high-resolution image data. The momentum parameter was set to 0.9 to accelerate the gradient descent process and suppress oscillations. For the loss function, considering that road pixels often account for only a small proportion in remote sensing images, resulting in extreme foreground-background class imbalance, the BCE_Dice_Loss joint loss function was adopted. This loss function combines binary cross-entropy loss and Dice coefficient loss, focusing on pixel-level classification accuracy while optimizing the overlap between predicted results and true labels from a global perspective, thus effectively driving the model to focus on small road targets.

[0049] The specific training parameters are closely integrated with the network characteristics: the input image size is kept at its original 1024×1024 pixels to preserve the richest spatial details for the frequency domain convolutional module to capture global features. The initial global learning rate is set to 1e. -4 Combined with a weight decay strategy (set to 1e) -5 To control model complexity, a differentiated parameter configuration was applied to the KAN Layer, with its initial weights set to 1e. -2 The weight decay is set to 1e. -3 This setup takes into account that the KAN layer uses a learnable spline function as its activation function. The higher initial weights and specific decay strategy help the layer quickly adapt to the feature distribution in the early stages of training, thus more efficiently fitting complex nonlinear road features. The total number of training epochs is set to 300 to ensure sufficient model convergence; the batch size is set to 4 to accommodate GPU memory limitations and ensure gradient stability. The entire training process is performed on a high-performance GPU computing platform, where the model iterates repeatedly to continuously optimize the frequency domain convolution kernel parameters of the FCB module and the spline function parameters of the KAN layer.

[0050] After model training, the model was objectively evaluated using test set data, with mean Intersection over Union (mIoU) and F1 score selected as core evaluation metrics. mIoU primarily calculates the ratio of the intersection to the union of the predicted results and the true labels, providing a global assessment of segmentation quality; a higher mIoU value indicates higher overlap. The F1 score is the harmonic mean of precision and recall, exhibiting good robustness to imbalanced datasets. Experimental results show that, through the complete implementation process described above, the model achieved high mIoU and F1 scores on the test set. This indicates that the present invention, by introducing a frequency domain convolution module, effectively preserves the long-distance dependency features of roads using its global receptive field, resulting in good connectivity of the extracted road network. Simultaneously, thanks to the learnable activation function of the bottleneck layer and the multi-scale graph inference mechanism, the model's ability to capture local texture and detailed structures is enhanced. Even for long, narrow, and occluded broken roads in the test set, the model can accurately identify and recover them.

[0051] Example 4 This embodiment takes the processing of a standard high-resolution remote sensing road image as an example to illustrate in detail the specific data flow paths, dimensional changes, and structural configuration details between different network layers during the actual operation of the present invention. The specific implementation process is as follows: The input remote sensing image to the network is an RGB three-channel image with a normalized resolution of 1024×1024 pixels. First, the data enters the feature extractor stage. This stage is configured with three cascaded frequency domain convolutional (FCB) modules, each performing a downsampling and feature upscaling task. The image first enters the first-layer FCB module, where global convolution in the frequency domain reduces the spatial resolution to half of the original image, i.e., 512×512, while simultaneously increasing the number of channels from the input 3 channels to a preset first-layer dimension (e.g., 64 channels), completing the initial shallow texture feature extraction. Subsequently, the data flows into the second-layer FCB module, where the resolution is further downsampled to 256×256, and the number of channels doubles (e.g., 128 channels). Finally, after processing by the third-layer FCB module, a high-dimensional feature map is obtained. At this point, the feature map size is compressed to one-eighth of the original image, i.e., 128×128, and the number of channels reaches its maximum value (e.g., 256 channels). This process fully utilizes the global receptive field characteristics of FCB, which significantly reduces spatial resolution while highly condensing the semantic information of the image into high-dimensional feature channels.

[0052] A 128×128 high-dimensional feature map enters the bottleneck layer. At this stage, the data dimension remains unchanged at 128×128 pixels and 256 channels. The KAN Layer utilizes its unique learnable spline activation function. In specific implementation, each weight parameter in the KANLayer is replaced with a learnable univariate function, which is a weighted combination of a basic activation function and a spline function. Specifically, the basic activation function uses the SiLU function to provide basic nonlinear features, while the spline part is parameterized using a B-spline function to capture high-frequency or complex nonlinear fluctuations. This B-spline function is defined on a preset fixed grid interval. In this embodiment, the spline order is set to 3 (i.e., cubic spline), and the number of grid intervals is set to 5. By dynamically updating the spline coefficients during network training, the activation function can adaptively adjust its shape according to the complexity of the road features, thereby achieving a deep abstraction of the road topology at the bottleneck layer and outputting a further refined bottleneck layer feature map.

[0053] The data enters a uniquely structured segmentation branch, employing a "restore first, then fuse" strategy. The 128×128 feature map output from the bottleneck layer first passes through a large-span transposed convolutional upsampling layer with a stride of 8, directly restoring the spatial size of the feature map to the original image size, 1024×1024, while maintaining the number of high-dimensional channels (256 channels). This step generates a high-dimensional feature map containing rich high-level semantics but with spatial details still needing enhancement. To address the issue of detail loss, this high-dimensional map is then fed into a multi-scale feature fusion pyramid. Inside this pyramid structure, the data stream is split and undergoes three more downsampling operations of varying degrees, generating three sets of multi-scale feature maps at the original image size (1024×1024), half the size (512×512), and a quarter the size (256×256), respectively. These three sets of feature maps are then independently enhanced through a graph inference module, utilizing the node interaction mechanism of the graph structure to uncover long-distance dependencies between pixels. For feature maps of different scales, the graph inference module introduces a node projection mechanism to optimize computational efficiency. Specifically, for high-resolution feature maps of the original size (1024×1024) or half the size (512×512), the module does not directly construct a massive adjacency matrix on all pixels. Instead, it compresses the feature space using a linear projection operator to extract a fixed number (e.g., 256 in this embodiment) of representative latent nodes. During graph inference, only the weights of the relationships between these 256 latent nodes are calculated, thus successfully decoupling the computational complexity from the square of the total number of pixels and limiting it to a constant level of computational overhead. After information transfer and feature enhancement are completed in the latent space, the enhanced semantic information is compensated back to the original spatial dimension through back-projection transformation. This approach enables the multi-scale pyramid to capture long-distance dependencies of narrow roads during fusion while reducing memory usage.

[0054] In the pyramid fusion stage, the enhanced multi-scale feature maps are fused step-by-step in ascending order: the 256×256 feature map is first upsampled and fused with the 512×512 feature map, and the fused result is then upsampled and fused with the original 1024×1024 feature map. After this series of complex internal pyramid processes, a fused feature map that integrates multi-scale information is finally output. This feature map is fed into the output layer, where 256 channels are compressed into one channel using a 1×1 convolutional kernel, and the values ​​are mapped to the 0-1 range using a Sigmoid activation function. Finally, a 1024×1024 binarized road prediction map is output, completing the end-to-end road extraction task.

[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for road extraction from remote sensing images based on frequency domain feature extraction and fusion, characterized in that, include: The road remote sensing image to be processed is input into a multi-layer feature extractor constructed by the frequency domain convolution operation module FCB. The frequency domain convolution operation module FCB performs frequency domain filtering operation in the frequency domain using the global receptive field, while performing downsampling and dimensionality upsampling to output a high-dimensional feature map. The high-dimensional feature map is input into the bottleneck layer constructed by the KAN layer, and a nonlinear feature transformation is performed on the high-dimensional feature map using a learnable activation function parameterized by spline function to output the bottleneck layer feature map. The bottleneck layer feature map is input into the segmentation branch, and the bottleneck layer feature map is upsampled to restore the original image size and maintain the high number of dimensional channels. The upsampled feature map is then output. The upsampled feature map is input into a multi-scale feature fusion pyramid constructed by a graph inference module to perform multi-scale feature learning and fusion, and output multi-scale fused features. The multi-scale fusion features are input into the output layer for channel compression and normalization processing, and the road extraction results of the remote sensing image are output.

2. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 1, characterized in that, The specific calculation steps of the frequency domain convolution operation module FCB include: The frequency domain convolution operation module FCB receives the input spatial domain feature map, and the spatial domain feature map received by the first-layer frequency domain convolution operation module FCB is the remote sensing image to be processed. The spatial domain feature map is subjected to a Fourier transform. A learned frequency domain convolution kernel is defined. The frequency domain convolution kernel is multiplied element-wise with the spectrum of the transformed spatial domain feature map. The result is then transformed by an inverse Fourier transform to output a new spatial domain feature map. The specific formula is as follows: ； in For the new spatial domain feature map, The input is a spatial domain feature map. For Fourier transform, For inverse Fourier transform, For frequency domain convolution kernels, In the frequency domain With frequency domain convolution kernel Perform element-wise multiplication.

3. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 2, characterized in that, The specific definition of the frequency domain convolution kernel for learning includes: The frequency domain convolution kernel has a size of N*N, where N is the size of the input spatial domain feature map; the frequency domain convolution kernel Convolution kernel in the spatial domain The Fourier transform yields: ； ； Spatial domain convolution kernel It is a frequency domain convolution kernel The inverse Fourier transform is represented by a smaller kernel. The zero-fill version, with a smaller core The size is K, where K is a positive integer and 1 ≤ K ≤ N: 。 4. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 1, characterized in that, The specific parameter settings for the KAN Layer include: The KAN Layer employs a learnable activation function, and each weight parameter in the KAN Layer is replaced by a univariate function; the univariate function is parameterized in spline function form; the calculation formula for the KAN Layer is: ； in The number of KAN layers. For the first Layer The function matrix is ​​used to obtain the bottleneck layer feature map.

5. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 1, characterized in that, The specific construction of the split branch includes: The segmentation branch consists of an upsampling layer, a multi-scale feature fusion pyramid, and an output layer. The upsampling layer is implemented using transposed convolution, which upscales the bottleneck layer features by X (x*x) to the original image size Y (y*y), outputting an upsampled feature map. The principle of transposed convolution is as follows: ； Where y is the original image size, s is the stride, p is the padding, k is the kernel size, and o is the output compensation. The multi-scale fusion pyramid structure contains three graph inference modules, two transposed convolutional upsampling layers, and two FCB modules forming a downsampling layer.

6. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 1, characterized in that, The specific processing steps of the multi-scale feature fusion pyramid include: The upsampled feature map is fed into the multi-scale feature fusion pyramid constructed by the graph inference module. Specifically, the upsampled feature map is downsampled three times to obtain three different scale feature maps: 1 / 2, 1 / 4, and 1 / 8. Each type of feature map is fed into the graph inference module to obtain the enhanced multi-scale feature map. The learned 1 / 8 feature map is upsampled to 1 / 4 and fused with the learned 1 / 4 feature map. The fused 1 / 4 feature map is upsampled to 1 / 2 and fused with the learned 1 / 2 feature map to output the multi-scale fused feature.

7. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 6, characterized in that, The specific processing steps of the graph reasoning module include: The upsampled feature map X (n*n) is input into the graph inference module and flattened into a node set x, where the i-th node is denoted as . Calculate the relationship weight matrix A between nodes, where nodes With nodes Relationship weights The relationship is calculated using the Gaussian similarity function, then: ； Using the relation weight matrix A, inference information is transmitted to the node set x to obtain the inference result y. ； In element form: ； y is the result of reasoning. It is the i-th element in y; Reshape the inference result y vector back into a two-dimensional image Y: ； The enhanced multi-scale feature map is obtained.

8. The method for road extraction from remote sensing images based on frequency domain feature extraction and fusion according to claim 1, characterized in that, The specific processing steps of the output layer include: The specific formula for inputting the multi-scale fused features into the output layer for channel compression and normalization is as follows: ； Where Z represents the multi-scale fusion feature. Here, b is a 1×1 convolution kernel used for channel compression, and b is compensation. is the Sigmoid activation function, and Y is the road extraction result from the remote sensing image.

Images