A remote sensing image road extraction method based on directional wavelet convolution

Abstract

The application provides a remote sensing image road extraction method based on directional wavelet convolution, and the steps comprise: step S01. Constructing a remote sensing image road extraction model, wherein a wavelet enhancement decoder in the model uses wavelet transform to decompose each level of feature into a plurality of different frequency components, and different enhancement strategies are used for each frequency component, wherein a level-adjustable strip hollow convolution with a changeable receptive field is used for a low-frequency component, and different directional strip hollow convolutions are used for each high-frequency component; step S02. Obtaining a remote sensing road training data set, and training the remote sensing image road extraction model; step S03. Obtaining a real-time remote sensing road image, inputting the real-time remote sensing road image into the trained remote sensing image road extraction model, and extracting road data output. The application can realize decoupling processing of road details and overall structure, improve the connectivity and integrity of road extraction, and enhance the direction sensitivity and long-distance geometric relationship.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image road detection and recognition, and particularly relates to a remote sensing image road extraction method based on directional wavelet convolution. BACKGROUND

[0002] Automatic extraction of roads from high-resolution remote sensing images is an important task for many applications, such as city planning, updating geographic information systems, urban planning, autonomous driving, etc. Extracting road information from high-resolution remote sensing images can provide a large amount of real-time road data, meet the demand for large-scale real-time road data, greatly reduce the cost of manpower, and have higher extraction accuracy.

[0003] For remote sensing image road extraction, the existing technology usually adopts a model mainly based on an encoder-decoder architecture, extracts image features through a pre-trained encoder, uses a fixed convolution kernel for feature extraction, and then uses convolution, self-attention, etc. on the decoder side to realize road segmentation. However, the above remote sensing image road extraction method based on the encoder-decoder architecture in the prior art still has the following problems:

[0004] (1) Using a fixed convolution kernel for feature extraction will result in a uniform receptive field, making it difficult to cooperatively capture the global topological structure (high-frequency information) and local edge detail features (low-frequency information) of the road, and the problems of discontinuous segmentation of the road and unclear edges still exist.

[0005] (2) Self-attention will have a quadratic growth in complexity as the channel dimension increases, and the feature spectrum formed by the superposition of self-attention is similar to low-pass filtering, which is not sensitive to detecting weak roads in remote sensing images. SUMMARY

[0006] The present application provides a remote sensing image road extraction method based on directional wavelet convolution, device and medium, through the decomposition-reconstruction mechanism of wavelet transform and the direction-sensitive strip hole convolution in the decoder, the decoupling processing of road details and overall structure is realized, the connectivity and integrity of road extraction are improved, and at the same time, according to the frequency component characteristics, a differentiated enhancement strategy is adopted, which can enhance the direction sensitivity and long-distance geometric relationship.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] On the one hand, the present application provides a remote sensing image road extraction method based on directional wavelet convolution, the steps comprising:

[0009] Step S01. Model construction: constructing a remote sensing image road extraction model, the remote sensing image road extraction model comprising a multi-scale feature encoder, a wavelet enhancement decoder and a prediction head connected in sequence;

[0010] The multi-scale feature encoder is configured to extract multi-scale features of the input image, the wavelet enhancement decoder is configured to use wavelet transform to respectively decompose features output by the multi-scale feature encoder into a plurality of different frequency components, the frequency components include a low frequency component and a plurality of high frequency components in different directions, different enhancement strategies are used for enhancement processing of each frequency component, wherein the low frequency component adopts a level-adjustable strip hollow convolution with a receptive field (RF) changing with level, each high frequency component respectively uses a strip hollow convolution in a different direction, and each frequency component after enhancement is reconstructed by inverse wavelet transform to obtain reconstructed features, and a prediction head predicts a road segmentation result according to the reconstructed features.

[0011] Step S02. Model training: Obtain a remote sensing road training data set and input it into the constructed remote sensing image road extraction model for training.

[0012] Step S03. Real-time road extraction: Obtain a real-time remote sensing road image, input it into the trained remote sensing image road extraction model, and extract a road segmentation result output.

[0013] As a further improvement of the present application, the multi-scale feature encoder is a Swin TransFormer encoder, the Swin TransFormer encoder includes four or more Swin encoding modules for extracting four or more levels of multi-scale features with different resolutions and receptive fields, the wavelet enhancement decoder includes four or more wavelet enhancement modules, each wavelet enhancement module is connected in series, each wavelet enhancement module corresponds to a Swin encoding module connected thereto for decomposing features output by the Swin encoding module into a plurality of different frequency components, using different enhancement strategies for each frequency component, and reconstructing features by inverse wavelet transform.

[0014] As a further improvement of the present application, each wavelet enhancement module decomposes the input features into a plurality of different frequency components through a low-pass filter and a high-pass filter The calculation expression of each frequency component is:

[0015]

[0016] wherein, represents the input feature, represents a convolution operation, is an outer product to generate a two-dimensional convolution kernel, represents a low frequency component, represents a vertical high frequency component, represents a horizontal high frequency component, represents a diagonal high frequency component.

[0017] As a further improvement of the application, the different direction high frequency components include horizontal high frequency components, vertical high frequency components and diagonal high frequency components, the horizontal high frequency components use horizontal direction strip hole convolution to strengthen longitudinal edges, the vertical high frequency components use vertical direction strip hole convolution to enhance transverse texture, and the diagonal high frequency components use convolution along diagonal 45° and / or 135° direction to sharpen diagonal details.

[0018] As a further improvement of the application, the low frequency component uses a receptive field In the hierarchical adjustable strip hole convolution varying with the hierarchy, further comprising a calculation model constructed as follows:

[0019]

[0020] Wherein, represents the feature hierarchy serial number, represents the receptive field of the feature hierarchy, represents the feature hierarchy convolution kernel size, represents the minimum convolution kernel, represents the feature hierarchy hole rate, represents the minimum hole rate, and are the minimum receptive field and the receptive field change amplitude respectively, represents the minimum Euclidean distance, represents the total number of feature hierarchies, represents the modulo operation;

[0021] In the calculation model, is the target, is the constraint condition, and the convolution kernel and the hole rate of each hierarchy are determined.

[0022] As a further improvement of the application, further comprising configuring , , and presetting and to 5 and 4 respectively, and then screening out continuous , , , and selecting when the distances are the same. , value corresponding to .

[0023] As a further improvement of the application, the enhanced frequency components are reconstructed by inverse wavelet transform to obtain reconstructed features.

[0024] reconstruction low-pass filter and reconstruction high-pass filter and satisfy the biorthogonal condition as follows: wherein , are a low-pass filter and a high-pass filter used in the wavelet transform process, respectively;

[0025] Each frequency component is up-sampled to restore the original resolution, and the up-sampled sub-band features are respectively convolved with the reconstruction low-pass filter and the reconstruction high-pass filter in the row direction to obtain the row direction reconstructed features

[0026] The row direction reconstructed features

[0027] are convolved with and to obtain the final reconstructed features

[0028]

[0029] wherein: , represent low frequency features in the row direction reconstructed features and high frequency features in the row direction reconstructed features, respectively, represents a transpose operation.

[0030] As a further improvement of the present application, in the wavelet enhancement decoder, after each wavelet enhancement module obtains the reconstructed features, a 3x3 transpose convolution is used for 2 times up-sampling, and after up-sampling, a channel adjustment and feature standardization are performed by a CBR module to obtain the enhanced feature output of each level, the CBR module includes a 1x1 convolution unit, a batch normalization unit and a ReLU activation unit connected in sequence; then the high-level features and the low-level features in each enhanced feature are spliced in the channel, and then processed by the CBR module to complete the preliminary fusion, and input into the next wavelet enhancement module for iterative optimization.

[0031] As a further improvement of the present application, in the process of model training in step S02, the following loss function is used:

[0032]

[0033]

[0034] ​​

[0035] wherein F is a predicted image, GT is a ground truth image, denotes the true label of the i-th position, wherein 0 denotes a non-road pixel, and 1 denotes a road pixel, denotes the predicted probability of the i-th position output by the model, denotes the total number of pixels, N denotes a smoothing factor, denotes a total loss function, denotes a BCE loss function, denotes a Dice loss function.

[0036] Compared with the prior art, the present application has the following beneficial effects:

[0037] 1. The present application combines the decomposition-reconstruction mechanism of wavelet transform and the direction-sensitive strip hole convolution in the decoder of the remote sensing image road extraction model, uses wavelet transform to decompose the features extracted by the encoder at each level into different frequency components, and then uses different types of strip hole convolution on each frequency component for enhancement processing, realizes frequency-space joint optimization, effectively decouples the processing of road details and overall structure, solves the problem of uniform receptive field caused by the fixed convolution kernel of the traditional encoder-decoder architecture, and makes it possible to detect long continuous and small and weak roads from remote sensing images, thereby improving the connectivity and integrity of road extraction.

[0038] 2. In the enhancement processing process, the present application adopts a differentiated enhancement strategy according to the characteristics of the frequency components, uses a hierarchically adjustable strip hole convolution on the low-frequency components, so that the high-level features and the low-level features can both maintain similar large receptive fields, which is beneficial to accurately capturing the global road shape, and the high-frequency components use strip hole convolutions of different directions, which can strengthen the edge texture details in different directions and effectively enhance the direction sensitivity. BRIEF DESCRIPTION OF DRAWINGS

[0039] The present application will be described in more detail below based on the embodiments and with reference to the accompanying drawings. Among them:

[0040] Figure 1 is the implementation flowchart of the remote sensing image road extraction method based on directional wavelet convolution of the embodiment of the present application.

[0041] Figure 2 is the overall architecture diagram of the remote sensing image road extraction model constructed in the embodiment of the present application.

[0042] Figure 3 is the implementation principle diagram of the wavelet convolution kernel in the embodiment of the present application. ​​

[0043] Figure 4 Figure 1 is a schematic diagram of the internal structure and principle of a wavelet enhancement module WEM in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The present application will be further described in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present application is not limited thereby.

[0045] In addition to convolution and self-attention, frequency domain learning can provide another paradigm to design decoder structure, such as fast Fourier convolution, adaptive Fourier U-Net, adaptive frequency filtering, wavelet transform, etc. Considering that the road region in remote sensing image has obvious low-frequency characteristics and shows continuity, while the region beside the road has different frequency domain characteristics, the pixel changes more rapidly and shows higher frequency signals, therefore, the road segmentation of remote sensing image is very suitable to introduce frequency learning. Applying low-pass filter can suppress the change of low-frequency information and capture small-size objects, thereby narrowing the intra-class difference between roads, on the other hand, using high-pass filter can enhance high-frequency information, expand the inter-class difference of similar objects, thereby combining the feature representation method of frequency domain operator with convolution.

[0046] Based on the encoder-decoder architecture for building a remote sensing image road extraction model, the present application introduces frequency learning method, combines the decomposition-reconstruction mechanism of wavelet transform and direction-sensitive strip hole convolution in the decoder of the model, uses wavelet transform to decompose the features extracted by the encoder at each level into different frequency components, then uses different types of strip hole convolution on each frequency component for enhancement processing, realizes frequency-space joint optimization with the mechanism of "decomposition-enhancement-reconstruction", effectively decouples the processing of road details (such as edges) and overall structure, solves the problem of uniform receptive field caused by fixed convolution kernel in traditional encoder-decoder architecture, so that long continuous and small and weak roads can be detected from remote sensing image, and the connectivity and integrity of road extraction are improved, and in the enhancement process, different types of strip hole convolution are used for low-frequency and high-frequency according to the characteristics of frequency components to form differentiated enhancement strategy, which can not only accurately capture the global road shape, but also strengthen the edge texture details in different directions, and enhance the direction sensitivity.

[0047] Referring to Figure 1 The steps of the remote sensing image road extraction method based on directional wavelet convolution in the embodiment include:

[0048] Step S01. Model construction: a remote sensing image road extraction model is constructed, the remote sensing image road extraction model comprising a multi-scale feature encoder, a wavelet enhancement decoder and a prediction head connected in sequence; the multi-scale feature encoder is used to extract multi-scale features of an input image, the wavelet enhancement decoder is used to use wavelet transform to respectively decompose features at each level output by the multi-scale feature encoder into a plurality of different frequency components, the frequency components including a low-frequency component and a plurality of high-frequency components in different directions, different enhancement strategies are used for enhancement processing on each frequency component, wherein the low-frequency component adopts a level-adjustable strip-like empty convolution with a receptive field RF changing with level, each high-frequency component uses a strip-like empty convolution in a different direction, and each frequency component after enhancement is reconstructed into a reconstructed feature through inverse wavelet transform, and the prediction head predicts a road segmentation result according to the reconstructed feature;

[0049] Step S02. Model training: a remote sensing road training data set is obtained and input into the constructed remote sensing image road extraction model for training.

[0050] Step S03. Real-time road extraction: a real-time remote sensing road image is obtained and input into the trained remote sensing image road extraction model to extract a road segmentation result output.

[0051] Through the above steps, the remote sensing image road extraction model adopts a decoder based on directional wavelet convolution, combines the decomposition-reconstruction mechanism of wavelet transform and the direction-sensitive strip-like empty convolution, first uses wavelet transform to decompose the multi-scale features of the encoder, then uses the strip-like empty convolution to enhance each frequency component after decomposition, and then uses inverse wavelet transform to reconstruct the features, realizes the mechanism of “decomposition-enhancement-reconstruction”, and based on this mechanism, frequency-space joint optimization can be realized, which effectively decouples the processing of road details (such as edges) and overall structure, and improves the connectivity and integrity of road extraction; at the same time, different enhancement strategies are used for different frequency components, the low-frequency component uses a level-adjustable strip-like empty convolution with a receptive field RF changing with level, which can make the high-level features and the low-level features both maintain a similar large receptive field, thereby capturing the global road shape, and each high-frequency component uses a strip-like empty convolution in a different direction, which can strengthen the edge texture details from different directions, thereby enhancing the direction sensitivity.

[0052] In this embodiment, the multi-scale feature encoder specifically adopts a Swin TransFormer encoder, and the Swin TransFormer encoder includes four or more Swin encoding modules for extracting multi-scale features with different resolutions and receptive fields at four or more levels. The wavelet enhancement decoder includes four or more wavelet enhancement modules (WEMs), each of which is connected in series. Each wavelet enhancement module is connected to a Swin encoding module for decomposing the features output by the Swin encoding module into multiple different frequency components, using different enhancement strategies for each frequency component, and reconstructing the features through inverse wavelet transform. Swin Tansformer is a deep learning model based on Transformer architecture. By introducing hierarchical design and sliding window self-attention mechanism, it can balance local sensitivity and long-range dependency capturing ability, effectively extract multi-scale features, and reduce computational complexity. The pre-trained Swin Transformer encoder is used in this application to extract multi-scale features of the input image. The channel number and size transformation of the features are shown in Figure 2 .

[0053] As shown in Figure 3 , the wavelet convolution in the wavelet transform process can be described as follows:

[0054] Let the input feature be , define a one-dimensional low-pass filter and a high-pass filter , and the two-dimensional wavelet decomposition can be represented as:

[0055] (1)

[0056] wherein, represents the convolution operation, is the outer product to generate a two-dimensional convolution kernel, represents the low-frequency component, represents the vertical high-frequency component, represents the horizontal high-frequency component, represents the diagonal high-frequency component.

[0057] Taking each level of feature output by the multi-scale feature encoder as the input feature, the input feature can be decomposed into four orthogonal components, i.e., the low-frequency component , the vertical high-frequency component , the horizontal high-frequency component , and the diagonal high-frequency component , through equation (1). The channel number of each component is consistent with the original feature, and the spatial size is reduced to 1 / 4.

[0058] In this embodiment, the wavelet enhancement decoder decodes and enhances features through a plurality of wavelet enhancement modules connected in series. In the decoding process, wavelet transform and strip hole convolution are combined to realize the decoupling mechanism of "decomposition-enhancement-reconstruction". Considering that the strip attribute is similar to the road, the hole attribute can effectively adapt to the road under occlusion and has stronger generalization. In this embodiment, different enhancement strategies are adopted for different frequency components. For low-frequency components, a hierarchical adjustable strip hole convolution is used, and the receptive field RF changes linearly with the level i, that is, the receptive field RF is adjustable and increases with the feature size, which is easy to capture the global road structure in a large range. For high-frequency components, a directional selective N*N strip hole convolution kernel is used, where the horizontal high-frequency component uses horizontal strip hole convolution to strengthen the longitudinal edge, the vertical high-frequency component uses vertical strip hole convolution to enhance the horizontal texture, and the diagonal high-frequency component uses convolution along the diagonal 45° and / or 135° direction to sharpen the diagonal details. In this embodiment, N*N is specifically 5*5, that is, 5*5 strip hole convolution kernel is used for high-frequency components for enhancement processing. It can be understood that other size strip hole convolution kernels can also be used according to actual needs.

[0059] It can be understood that the wavelet enhancement decoder described above can be flexibly configured according to the extracted feature dimension to adapt to different encoders (such as ResNet, Transformer, etc. popular encoders), realize the flexibility of "plug and play", can significantly reduce the difficulty of algorithm deployment, so as to support rapid integration of the latest backbone network.

[0060] Specifically, as shown in Figure 2 , the input remote sensing image generates four-level features with different resolutions and receptive fields after the Swin encoder. Starting from the highest-level feature, the four-level features are optimized by a wavelet enhancement module WEM level by level: first, the wavelet transform is used to decompose the feature into four orthogonal components of low-frequency ( ), vertical high-frequency ( ), horizontal high-frequency ( ) and diagonal high-frequency ( ), and the channel number of each component is consistent with the original feature. The low-frequency component ( ) uses a hierarchical adjustable strip hole convolution, and the high-frequency component ( ) uses a directional selective 5*5 strip hole convolution kernel, the vertical direction (90°) convolution is used for the vertical high-frequency component to strengthen the longitudinal edge, the horizontal direction (0°) convolution is used for the horizontal high-frequency component to enhance the horizontal texture, and the diagonal direction (45° and 135°) convolution is used for the diagonal high-frequency component to sharpen the diagonal details, and the anisotropic convolution kernel is used to realize the maximization of edge response.

[0061] In this embodiment, the low-frequency component ( In the use of hierarchically adjustable strip-shaped dilated convolution, in order to obtain a linear receptive field, the selection of the convolution kernel and dilation rate is regarded as a goal planning process, which also includes constructing the following computational model to select the convolution kernel and dilation rate:

[0062] (2)

[0063] in, Indicates the feature level number. Indicates the first Receptive field at the feature level Indicates the first Feature level convolution kernel size, Represents the smallest convolution kernel. Indicates the first Feature level void ratio Indicates the minimum void ratio. and These represent the minimum receptive field and the magnitude of the receptive field change, respectively, for linearly changing receptive fields. Indicates the total number of feature levels. This indicates a modulo operation.

[0064] As shown in equation (2), the receptive field of each layer with hierarchy They satisfy a linear relationship At the same time, the sensory fields of each layer With convolution kernel The relationship between void ratio and void ratio is satisfied Then minimize the Euclidean distance. For the goal, that is In a linear relationship and feel the wild With convolution kernel The relationship between void ratio and void ratio As a constraint, the convolution kernels at each level are solved. With void ratio To find the optimal solution, and to ensure that the receptive field is symmetric and reasonable, the following requirements must be met. , It is an odd number. If the number is even, then the constraints also include... This allows for the effective determination of the convolution kernel and dilation rate at each level, thus enabling the acquisition of hierarchically adjustable strip-shaped dilated convolutions with receptive fields that vary linearly between levels. For low-frequency ( The component adopts the hierarchical adjustable strip hollow convolution, a better continuity can be obtained by using the linearly changed receptive field between the hierarchies, and the part of details can be ignored to pursue a larger receptive field, the overall structure is fully considered, the high and low hierarchical features maintain similar large receptive fields, so that different hierarchical features can capture large-scale global road shapes.

[0065] For example, in the strip hollow convolution, the minimum receptive field is 5, the corresponding , , , , and and are respectively preset as 5 and 4, the corresponding receptive field dictionary is , the strip hollow convolution is initialized according to the receptive field dictionary, then the continuous , , are screened out according to the minimum Euclidean distance , the value of , corresponding to the same distance is selected when the effective elements are too sparse, the best , value can be quickly and accurately determined, and the high and low hierarchical features maintain similar large receptive fields as much as possible, so that different hierarchical features can capture large-scale global road shapes.

[0066] In the embodiment, the enhanced frequency components are further subjected to inverse wavelet transform to reconstruct features to obtain reconstructed features, so that the enhanced components are reconstructed into optimized features by inverse wavelet transform, and the specific steps include:

[0067] The reconstruction low-pass filter and the reconstruction high-pass filter are defined, and satisfy the following biorthogonal condition: , wherein , are the low-pass filter and the high-pass filter used in the wavelet transform process, denotes the unit impulse function, in the discrete time, the unit impulse function denotes a pulse that is 1 at the discrete index and 0 at other points;

[0068] Each frequency component , , , is up-sampled (for example, 2 times up-sampling can be performed by transpose convolution) to restore the original resolution, and the up-sampled sub-band features are respectively multiplied by the reconstruction low-pass filter and a reconstruction high-pass filter convolution to obtain a row direction reconstructed feature

[0069] (3)

[0070] convolution to obtain a row direction reconstructed feature convolution to obtain a row direction reconstructed feature and convolution to obtain a row direction reconstructed feature :

[0071] (4)

[0072] wherein: , respectively represent low-frequency features in the row direction reconstructed feature, high-frequency features in the row direction reconstructed feature, represents a transpose operation, and respectively represent a low-pass filter and a transpose filter of the reconstruction high-pass filter.

[0073] After the multi-scale features are decomposed into different frequency components by wavelet transform in the wavelet enhancement decoder, different strip hole convolutions are used for each frequency component according to the frequency characteristics, which not only enables the high and low level features to maintain similar large receptive fields and capture large-scale global road shapes, but also enhances the edges in different directions and enhances the direction, so that the optimized features can be reconstructed after the above inverse wavelet transform, which can not only accurately capture the global road shape and enhance the road connectivity, but also maintain the edge accuracy.

[0074] Specifically, in the wavelet enhancement decoder, after each wavelet enhancement module obtains the reconstructed feature, upsampling fusion is performed. First, 3x3 transpose convolution is used for 2 times upsampling, which can reduce information loss compared with interpolation method, and the reconstruction quality can also be improved through parameter learning of transpose convolution. After upsampling, the channel adjustment and feature standardization are performed through the CBR module to obtain the enhanced feature output of each level, wherein the CBR module includes a 1x1 convolution unit, a batch normalization unit and a ReLU activation unit connected in turn, that is, the CBR module performs channel adjustment and feature standardization through the architecture of 1x1 convolution+batch normalization+ReLU to relieve gradient anomaly and accelerate convergence. Then, the high-level features and low-level features in each enhanced feature are spliced in the channel, and then processed by the CBR module to complete the preliminary fusion and input into the next wavelet enhancement module for iterative optimization.

[0075] ​​In this embodiment, in step S02, after obtaining the remote sensing road extraction original data set, the data set is divided into training set and validation set according to the specified proportion, the remote sensing road extraction model is trained using the training set, and the performance of the trained remote sensing road extraction model is verified by the validation set. Further, each image in the data set can be randomly cropped to a specified size (such as 512x512) picture, and random rotation and other spatial transformations can be applied to improve the generalization ability and stability of the model.

[0076] In the process of model training in step S02 in this embodiment, the following loss function is used:

[0077] (5)

[0078] (6)

[0079] (7)

[0080] Where F is the predicted image, GT is the true value image, represents the true label of the i-th position, where 0 represents a non-road pixel and 1 represents a road pixel, represents the predicted probability of the i-th position output by the model, represents the total number of pixels, represents a smoothing factor (for example, can be taken as N ), represents the total loss function, represents the BCE loss function, represents the Dice loss function.

[0081] After obtaining the real-time remote sensing road image, it is input into the remote sensing image road extraction model trained as described above, and the road segmentation result can be extracted in real time.

[0082] ​​In summary, the application realizes the decoder of the "decomposition-enhancement-reconstruction" mechanism through the decomposition-reconstruction mechanism of wavelet transform and the direction-sensitive strip hollow convolution, can realize the frequency-space joint optimization, effectively decouples the processing of road details and overall structure, solves the problem of uniform receptive field caused by fixed convolution kernel in traditional encoder-decoder architecture, and improves the connectivity and integrity of road extraction; at the same time, after the decomposition stage, according to the frequency component characteristics, a differentiated enhancement strategy is adopted, the low frequency (LL) uses the hierarchical adjustable strip hollow convolution, so that the high level features and the low level features can maintain similar large receptive field, capture the global road shape, and each high frequency (LH / HL / HH) component uses N×N strip hollow convolution to respectively strengthen the edge texture details in the vertical, horizontal and diagonal directions, and enhance the direction sensitivity, which can not only break through the limitation of uniform receptive field of traditional CNN and improve the multi-scale feature interaction effect, but also can enhance the road connectivity while maintaining the edge accuracy, especially suitable for high-precision end-to-end remote sensing image road segmentation in complex scenes.

[0083] The embodiment further simulates the above method to verify the effect. The simulation experiment adopts the Pytorch deep learning framework to build the network model, the NVIDIA Tesla P100 GPU is used for GPU acceleration training, and the operating system is 64-bit The simulation experiment adopts two data sets:

[0084] 1) DeepGlobe data set, which contains 8570 satellite images from India, Indonesia and Thailand, each image size is 1024 × 1024 pixels, spatial resolution is 0.5 m, and the data set covers various scenes of land cover, including rural and urban areas. Among the 8570 satellite images in the DeepGlobe data set, 6226 labeled images are provided as a training set, and 2344 images are provided as a validation set and a test set. In this embodiment, the 6226 labeled images are divided into 4696 sample pairs as a training set and 1530 sample pairs as a test set.

[0085] 2) Jilin-1 high-resolution remote sensing data road extraction data set (Jilin-1 Website of CGSTL), which selects urban, rural and other regions for road labeling, a total of 4000 images, each image size is 512 × 512 pixels, spatial resolution is 0.5 m. In this data set, the embodiment divides the 3200 labeled images into 2400 sample pairs as a training set and 800 sample pairs as a test set.

[0086] To verify the effectiveness of this application, the method was compared with five existing road extraction methods (DBRANet, AD-RoadNet, RCFSNet, MCCA, and OARENet). The evaluation metrics used were precision (P), recall (R), F1 score, and intersection over union (IoU), each defined as follows:

[0087] (8)

[0088] (9)

[0089] (10)

[0090] (11)

[0091] Wherein, TP, TN, FP, and FN represent the number of true positive, true negative, false positive, and false negative pixels, respectively. The simulation results are shown in Table 1.

[0092]

[0093] As shown in Table 1, our application achieves an F1 score 0.6%-1.7% higher than the other five methods on the DeepGlobe dataset. It is higher than the other 5 methods. On the Jlin-1 dataset, the F1 score was 0.3%–8.7% higher than the other five methods. It is higher than the other 5 methods. Compared to existing road extraction methods, this application achieves better segmentation performance. This is because it combines wavelet transform decomposition-reconstruction with direction-sensitive strip-dilated convolution to achieve joint frequency-space optimization through a "decomposition-enhancement-reconstruction" mechanism. This effectively decouples road details (such as edges) from the overall structure. Furthermore, after the decomposition stage, this application employs a differentiated enhancement strategy based on the characteristics of the frequency components. Low-frequency (LL) features hierarchically adjustable strip-dilated convolution, ensuring that higher-level and lower-level features maintain similar large receptive fields to capture the global road shape. High-frequency (LH / HL / HH) features use 5×5 strip-dilated convolution, which respectively enhances edge texture details in the vertical, horizontal, and diagonal directions, improving the ability to distinguish between fragmented roads and complex backgrounds. Therefore, this application achieves better road extraction performance from remote sensing images.

[0094] In the embodiments of the present disclosure, it should be understood that the disclosed apparatus and method can also be implemented in other manners. The described apparatus embodiments are merely exemplary. For example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operation of the apparatus, method and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks can actually be executed in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and flowcharts, and the combination of blocks in the block diagrams and flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0095] Although the present application has been described with reference to the preferred embodiments, various modifications and changes can be made thereto without departing from the scope of the present application. In particular, the technical features mentioned in each of the embodiments can be combined in any manner as long as there is no structural conflict. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A remote sensing image road extraction method based on directional wavelet convolution, characterized by the steps of The application relates to a remote sensing image road extraction method and device. Step S01. Model construction: a remote sensing image road extraction model is constructed, which comprises a multi-scale feature encoder, a wavelet enhancement decoder and a prediction head connected in sequence; The multi-scale feature encoder is used to extract multi-scale features from the input image. The wavelet enhancement decoder is used to decompose the features output by the multi-scale feature encoder into multiple different frequency components using wavelet transform. These frequency components include low-frequency components and multiple high-frequency components in different directions. Different enhancement strategies are applied to each frequency component. The low-frequency components employ a layer-adjustable strip-shaped dilated convolution with a receptive field (RF) varying with the layer. Each high-frequency component uses a strip-shaped dilated convolution in a different direction. These high-frequency components include horizontal, vertical, and diagonal high-frequency components. The horizontal high-frequency components use a horizontal strip-shaped dilated convolution to enhance vertical edges. The vertical high-frequency components use a vertical strip-shaped dilated convolution to enhance horizontal texture. The diagonal high-frequency components use convolution along the diagonal at 45° and / or 135° to sharpen diagonal details. The enhanced frequency components are then reconstructed using inverse wavelet transform. The prediction head predicts the road segmentation result based on the reconstructed features. In the layer-adjustable strip-shaped dilated convolution, the... With the goal, To constrain the conditions, the convolution kernel and dilation rate for each level are determined, where, Indicates the feature level number. Indicates the first Receptive fields at the feature level, Indicates the first kernel size at the feature level, Represents the smallest convolution kernel. Indicates the first Hollow rate at the feature level Indicates the minimum void ratio. and These represent the minimum receptive field and the magnitude of the receptive field change, respectively, for linearly changing receptive fields. Indicates the minimum Euclidean distance. Indicates the total number of feature levels. Indicates a modulo operation; Step S02. Model training: a remote sensing road training data set is obtained and input into the constructed remote sensing image road extraction model for training; Step S03. Real-time road extraction: real-time remote sensing road images are obtained and input into the trained remote sensing image road extraction model to extract road segmentation results. 2.The method of claim 1, wherein, The multi-scale feature encoder is a Swin TransFormer encoder, the Swin TransFormer encoder comprises four or more Swin encoding modules for extracting multi-scale features with different resolutions and receptive fields at four or more levels, the wavelet enhancement decoder comprises four or more wavelet enhancement modules, each wavelet enhancement module is connected in series, each wavelet enhancement module is connected with a Swin encoding module for decomposing the features output by the Swin encoding module into a plurality of different frequency components, using different enhancement strategies for each frequency component, and reconstructing the features through inverse wavelet transform. 3.The method of claim 2, wherein, Each of the wavelet enhancement modules is passed through a low pass filter and a high pass filter to decompose the input feature into a plurality of different frequency components, the computational expression of each frequency component being wherein, represents an input feature, represents a convolution operation, generates a two-dimensional convolution kernel for the outer product, represents a low frequency component, represents a vertical high frequency component, represents a horizontal high frequency component, represents a diagonal high frequency component. 4.The method of claim 1, wherein, The low-frequency component adopts a level-adjustable strip-shaped hollow convolution with a receptive field RF changing with a level, and a calculation model is constructed as follows: 。 5.The remote sensing image road extraction method based on directional wavelet convolution according to claim 1, characterized in that, Also include configuration , , and set and to 5, 4 respectively, according to the minimum Euclidean distance Screening out the convolution kernel of each level , the void rate , wherein the same distance is preferred The corresponding value of , .

6. The method according to any one of claims 1 to 5, wherein, The reconstructed features obtained by inversely wavelet transforming the enhanced frequency components include: defining a reconstruction low-pass filter and a reconstruction high-pass filter and satisfy the biorthogonal condition as follows: where , are a low-pass filter, a high-pass filter used in the wavelet transform process, respectively, is a unit impulse function; The frequency components are up-sampled to restore the original resolution, and the up-sampled sub-band features are respectively convolved with a reconstruction low-pass filter and a reconstruction high-pass filter in the row direction to obtain the reconstructed features in the row direction : wherein represents a low frequency component, represents a vertical high frequency component, represents a horizontal high frequency component, represents a diagonal high frequency component; reconstructing features in the row direction convolving with and to obtain the final reconstructed features : wherein: , respectively denote a low frequency feature in the row direction reconstruction feature, a high frequency feature in the row direction reconstruction feature, denotes a transpose operation, and respectively denote a low pass filter and a transpose filter of the reconstruction high pass filter .

7. The convolution-based directional wavelet method for road extraction from remote sensing images according to any one of claims 1 to 5, characterized in that, In the wavelet enhancement decoder, after each wavelet enhancement module obtains the reconstructed features, 2x upsampling is performed through 3x3 transpose convolution, and after upsampling, channel adjustment and feature standardization are performed through a CBR module to obtain the enhanced feature output of each level, the CBR module comprises a 1x1 convolution unit, a batch normalization unit and a ReLU activation unit connected in sequence; then, the high-level features and the low-level features in each enhanced feature are spliced, and then the CBR module is used for processing to complete preliminary fusion, and the next wavelet enhancement module is input for iterative optimization.

8. The method according to any one of claims 1 to 5, wherein the method is a method for extracting a road from a remote sensing image based on directional wavelet convolution, characterized by, In the model training process in step S02, the following loss function is used: wherein, is a predicted image, is a ground truth image, denotes the true label of the -th position, where 0 denotes a non-road pixel and 1 denotes a road pixel, denotes the predicted probability of the -th position output by the model, denotes the total number of pixels, denotes a smoothing factor, denotes the total loss function, denotes the BCE loss function, denotes the Dice loss function.

Images