U-Net remote sensing image road segmentation method based on adaptive double filters

Abstract

The invention discloses a U-Net remote sensing image road segmentation method based on self-adaptive double filters, which relates to the technical field of remote sensing image processing, and comprises the following steps: step 1, collecting and preprocessing a remote sensing image, and constructing a training sample set; 2, constructing a self-adaptive double-filtering U-Net network which comprises an encoder and a decoder, setting two fusion layers in an up-sampling module of the decoder, generating a self-adaptive low-pass filter and a self-adaptive high-pass filter in each fusion layer, and performing filtering fusion on the extracted feature maps; 3, performing training optimization on the self-adaptive double-filtering U-Net network by using the training sample set to obtain a remote sensing image segmentation model; and 4, collecting a remote sensing image to be segmented, and inputting the remote sensing image to the remote sensing image segmentation model to obtain a road segmentation result. According to the invention, by introducing the double-filtering frequency domain perception fusion technology, the feature consistency can be effectively enhanced, the object boundary can be sharpened, and the image segmentation precision can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more specifically to a U-Net remote sensing image road segmentation method based on an adaptive dual filter. Background Art

[0002] With the rapid development of remote sensing technology, the ability to obtain high-resolution remote sensing images has been significantly enhanced, which provides rich data support for urban planning, agricultural monitoring, environmental protection and other fields. However, the segmentation task of remote sensing images still faces many challenges, especially in complex scenes, such as when the boundaries of targets such as roads, buildings and vegetation are blurred. Traditional segmentation methods often rely on pixel-level feature extraction, which is easily affected by changes in illumination, perspective and scene complexity, resulting in insufficient segmentation accuracy.

[0003] Although existing deep learning models, such as U-Net, have shown excellent performance in image processing in certain specific fields, their application in remote sensing image processing still needs further improvement. The encoder-decoder structure of U-Net can effectively integrate low-level detail information and high-level semantic information, which makes it perform well in some tasks. However, in the process of feature fusion, it often faces interference from intra-category inconsistency and inter-category similarity. This interference can lead to blurred target boundaries and inaccurate positioning in the processing results, especially in remote sensing images with complex backgrounds. In order to solve these problems, researchers began to focus on the extraction and fusion of frequency domain features. In the frequency domain, the high-frequency information of the image corresponds to details and edges, while the low-frequency information represents the overall structure of the image. By performing feature processing in the frequency domain, the boundary information of the target can be more effectively captured and enhanced, and the segmentation effect can be improved. Based on this, feature processing technology based on the frequency domain has gradually been introduced into image segmentation tasks. However, existing methods often lack adaptive processing of frequency domain features, and it is difficult to adjust feature fusion strategies for different image contents.

[0004] Therefore, how to adaptively combine spatial features to achieve image segmentation and improve the accuracy of remote sensing image segmentation is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a U-Net remote sensing image road segmentation method based on adaptive dual filters, aiming to solve the problems of low spatial resolution and inaccurate boundary positioning caused by insufficient feature fusion in traditional remote sensing image segmentation. By introducing the frequency domain perception fusion technology of dual filtering, it can effectively enhance feature consistency and sharpen object boundaries, improve segmentation accuracy, and thus support applications such as high-precision agricultural monitoring, urban planning, and environmental monitoring.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A U-Net remote sensing image road segmentation method based on adaptive dual filters comprises the following steps:

[0008] Step 1: Collect remote sensing images and preprocess them to construct a training sample set;

[0009] Step 2: Construct an adaptive dual-filter U-Net network, including an encoder and a decoder. Two fusion layers are set in the upsampling module of the decoder. Each fusion layer generates an adaptive low-pass filter and an adaptive high-pass filter to perform filtering fusion on the extracted feature maps.

[0010] Step 3: Use the training sample set to train and optimize the adaptive dual-filter U-Net network to obtain a remote sensing image segmentation model;

[0011] Step 4: Collect the remote sensing image to be segmented, input it into the remote sensing image segmentation model, and obtain the road segmentation result.

[0012] Preferably, the preprocessing includes image annotation, which uses pixel-level labels to annotate the category of each pixel in the remote sensing image.

[0013] Preferably, the encoder extracts low-level features from the remote sensing image of the training sample set; the low-level features include edge information, texture information, etc.; the encoder includes 4 downsampling layers, the remote sensing image is used as input, each downsampling layer outputs a feature map, the output of the previous downsampling layer is used as the input of the next downsampling layer, and the first input feature map, the second input feature map, the third input feature map and the fourth input feature map are obtained respectively, and the sizes of the four feature maps decrease in sequence. The initial image I is processed by a series of convolutional layers, each convolutional layer applies a set of convolution kernels (filters) to extract the feature map of the image; a simple CNN structure can be selected to realize feature map extraction, which includes multiple convolutional layers, activation functions (ReLU) and pooling layers (maximum pooling). After processing by these convolutional layers, an original feature map X (first input feature map) is obtained, and its size is C×H×W, where C is the number of feature channels, and H and W are the height and width of the feature map.

[0014] Preferably, the decoder includes an upsampling module, a jump connection layer, a convolution module and a segmentation module connected in sequence; the upsampling module includes 4 upsampling layers and two fusion layers, wherein the fourth input feature map is used as input, each upsampling layer outputs a feature map, and the output of the previous upsampling layer is used as the input of the next upsampling layer to obtain the fourth output feature map, the third output feature map, the second output feature map and the first output feature map respectively, and the sizes of the four feature maps are increased in sequence; the first fusion layer is arranged in parallel with the third upsampling layer, and is used to fuse the third output feature map and output the first fused feature map; the second fusion layer is arranged in parallel with the fourth upsampling layer, and is used to fuse the second output feature map and output the second fused feature map; the fourth output feature map, the first fused feature map, the third fused feature map and the first output feature map are merged with the corresponding input feature map in the jump connection layer and then input into the convolution module for fusion to obtain the third fused feature map, and the third fused feature map is input into the segmentation module for road classification to obtain a road segmentation result.

[0015] Preferably, each fusion layer includes an expansion module, two convolutional layers, a filter generation module, a low-pass filter module, a high-pass filter module and a feature fusion module;

[0016] The expansion module expands the size of the input output feature map by 2 times to obtain an expanded output feature map;

[0017] The first convolution layer performs a convolution operation on the input output feature map to obtain the first convolution feature map, and the second convolution layer performs a convolution operation on the expanded output feature map to obtain the second convolution feature map;

[0018] The filter generation module generates an adaptive low-pass filter and an adaptive high-pass filter according to the second convolution feature map;

[0019] The low-pass filtering module uses an adaptive low-pass filter to filter the first convolution feature map to obtain a low-pass filtering feature map, and performs pixel random fusion with the first convolution feature map to obtain a low-frequency feature map;

[0020] The high-pass filtering module uses an adaptive high-pass filter to filter the second convolution feature map to obtain a high-pass filtering feature map, and adds and fuses the high-frequency feature map with the second convolution feature map pixel by pixel to obtain a high-frequency feature map;

[0021] The feature fusion module fuses the low-frequency feature map and the high-frequency feature map to obtain a fused feature map, including a first fused feature map or a second fused feature map.

[0022] Preferably, the filter generation module includes a convolution layer, a pixel reordering layer, a softmax layer and a filter inversion layer connected in sequence; the second convolution feature map is obtained through the convolution operation of the convolution layer to obtain a third convolution feature map; the pixel reordering layer performs pixel reordering on the third convolution feature map to obtain a reordered feature map; the reordered feature map is activated by the softmax layer to obtain a low-pass filter; the filter inversion layer inverts the low-pass filter according to the input identity core, the identity core is a 3×3 convolution core, and its essence is to use the 3×3 convolution core to convolve the low-pass filter to achieve inversion and obtain a high-pass filter. The convolution layer uses a 3×3 convolution layer, and the softmax layer uses a Softmax function.

[0023] Preferably, the specific process of realizing adaptive low-pass filter generation and low-pass filtering through the second convolution layer, the filter generation module and the low-pass filtering module is as follows:

[0024] Step 311: the second convolutional layer uses a 1×1 convolutional layer to perform a convolution operation on the extended output feature map to obtain a second convolutional feature map X;

[0025] Step 312: The filter generation module performs convolution, pixel rearrangement and activation operations on the second convolution feature map in sequence to generate an adaptive low-pass filter It is expressed as:

[0026]

[0027] Among them, i and j represent the row and column coordinates in the feature map respectively; p and q represent the relative positions in the low-pass filter kernel respectively; Ω represents a 3×3 filter kernel; PixelShuffle represents pixel shuffling to achieve pixel rearrangement; Softmax represents the Softmax function to achieve activation operation; the specific process is:

[0028] Step 3121: Perform a convolution operation on the second convolution feature map to obtain a third convolution feature map

[0029] Step 3122: reshape the third convolution feature map by pixel de-shuffling using a pixel shuffle operation, reducing the height and width by half, and expanding the channel by 4 times;

[0030] Step 3123: Apply the Softmax function to each filter kernel separately to ensure that the weight of each filter kernel still satisfies the properties of the probability distribution after applying the Softmax function. Divide the channels into 4 groups, each with a spatially transformed adaptive low-pass filter, expressed as Among them, g∈1, 2, 3, 4 represents the group;

[0031] Step 313: The low-pass filtering module uses an adaptive low-pass filter to smooth and upsample the first convolution feature map; uses a low-pass filter to filter the filtered low-pass filter feature map using a sub-pixel upsampling technique. l+1 Rearrange and upsample to obtain 2 times upsampled feature values Specifically:

[0032] Step 3131: The first convolution feature map is filtered using an adaptive low-pass filter to obtain four sets of low-pass filter features. g=1,2,3,4, expressed as:

[0033]

[0034] Step 3132: shuffle the pixels of the four sets of low-pass filter features and rearrange them into 2x upsampled feature values It is expressed as:

[0035]

[0036] Among them, PixelShuffle() represents pixel shuffling, which is an operation of converting channel information into spatial information.

[0037] Preferably, the filtering inversion layer inverts the low-pass filter to generate an adaptive high-pass filter, and the high-pass filtering module uses the adaptive high-pass filter to perform filtering to enhance the detailed boundary information lost during the downsampling process. The process is:

[0038] Step 321: Invert the low-pass filter to generate an adaptive high-pass filter It is expressed as:

[0039]

[0040] Where E represents the unit core; in order to ensure that the final generated core It is high-pass. First, use per-core softmax to obtain low-pass kernels, and then subtract these kernels from the unit kernel E. The weights of the unit kernel E are [[0,0,0],[0,1,0],[0,0,0]];

[0041] Step 322: After filtering the second convolution feature map using an adaptive high-pass filter, the residual is added to the second convolution feature map to obtain an enhanced result and generate a high-frequency feature map. It is expressed as:

[0042]

[0043] Represents the feature value of the (i, j) position in the high-frequency feature map of the lth layer.

[0044] Preferably, the feature fusion module upsamples the feature values ​​by a factor of 2 And high frequency feature map Perform pixel-by-pixel addition to obtain a fused feature map; specifically:

[0045] Upsample the feature values ​​by a factor of 2 And high frequency feature map A 1×1 convolutional layer is used to compress the channels, and the compressed features are added pixel by pixel. Then, the fused feature map is obtained by upsampling, which is expressed as:

[0046]

[0047] in, represents the compressed features after fusion, where r is the channel reduction rate, which is used to reduce the subsequent calculation cost of the three; Represents an upsampling operation, applying nearest neighbor interpolation to upsample the low-resolution feature map to a high resolution; Conv 1×1 Represents a 1×1 convolution operation.

[0048] Preferably, the segmentation module uses the activation function Softmax to perform road target segmentation.

[0049] The technical effect of the above technical solution is to use an adaptive low-pass filter generator to predict a spatially varying low-pass filter, adaptively use a spatially transformed low-pass filter to smooth high-level features, resample nearby low-frequency features of the same category to replace inconsistent features in high-level features, and enhance high-frequency boundary details of low-level features, thereby solving the problems of category inconsistency and boundary displacement, reducing high-frequency components inside objects in remote sensing images, and reducing intra-class inconsistency during upsampling; using an adaptive high-pass filter generator to predict a high-pass filter, extract high-frequency edges and detail features from low-level features, enhance high-frequency information lost during downsampling, and achieve more accurate boundary delineation; and further optimize feature fusion strategies through dual filters. The adaptive low-pass filter generator and the self-use high-pass filter generator separate information into low-frequency and high-frequency parts in subsequent feature processing through parameter learning or fixed filter design. During the training process of the U-Net network, different types of features are filtered through two generators, and the generated low-pass and high-pass filters work together to extract and separate features, which increases the similarity of features within a category and improves the distinction of features between categories. This allows the network to learn information of different categories more specifically during training, improves the model's discrimination and separation capabilities, thereby reducing the misclassification rate and improving the classification effect.

[0050] Preferably, the process of constructing the optimization target to realize model training is:

[0051] S331: Fusion feature map Y l The predicted segmentation result is obtained after passing through the skip connection layer, convolution module and segmentation head of the segmentation module in sequence. The training sample set includes the real segmentation results y annotated by humans i ;

[0052] S332: Segmentation results based on prediction And the true segmentation result y i Calculate the loss value and optimize the network. The optimization goal is to make the predicted segmentation result consistent with the actual segmentation result. According to the optimization goal, terminate the model training and obtain the remote sensing image segmentation model.

[0053] Preferably, in step 3, a multi-loss function is used to optimize the network, and the multi-loss function L is expressed as:

[0054] L=L ce +λL b

[0055]

[0056]

[0057] Among them, L ce represents the cross entropy loss; L b represents the boundary loss; λ represents the model parameter.

[0058] Preferably, in step 3, the network is optimized using an Adam optimizer, and the parameter update rule of the Adam optimizer is:

[0059]

[0060] Among them, θ t is the model parameter at time step t; α is the learning rate; β is the exponential decay rate of the calculated gradient, and β is the exponential decay rate of the calculated gradient square; g t is the gradient at time step t; m t is the cumulative average of the squared gradients at time step t.

[0061] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a U-Net remote sensing image road segmentation method based on an adaptive dual filter, which optimizes feature extraction and fusion strategies by combining adaptive low-pass and high-pass filters and integrating feature fusion layers in the upsampling path, thereby enhancing the model's ability to capture details and improving the accuracy of target boundary positioning. This method not only improves segmentation accuracy, but also effectively solves the shortcomings of traditional methods in complex scenes, provides a more accurate solution for the application of remote sensing images, has higher accuracy and better boundary alignment, and meets the application needs of remote sensing image segmentation in multiple fields. Specifically:

[0062] 1) The present invention optimizes the feature extraction and fusion strategy by introducing an adaptive dual filter, significantly enhancing the ability to recognize road boundaries, especially under complex backgrounds and variable lighting conditions, and can achieve more refined and accurate road segmentation, effectively improving the precision and accuracy of remote sensing image road segmentation;

[0063] 2) Through the adaptive processing of frequency domain features, the present invention can maintain stable segmentation performance under different conditions, effectively reduce the misclassification caused by intra-category inconsistency and boundary displacement, enhance the robustness and generalization ability of the model, and make the model more adaptable and reliable in changing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0065] Figure 1 A schematic diagram of the adaptive dual-filter U-Net network structure provided by the present invention;

[0066] Figure 2 A schematic diagram of the structure of the filter generation module provided by the present invention;

[0067] Figure 3 This is a schematic diagram of the fusion layer structure provided by the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The embodiment of the present invention discloses a U-Net remote sensing image road segmentation method based on an adaptive dual filter, comprising the following steps:

[0070] S1: Collect remote sensing images and preprocess them to construct training sample sets;

[0071] S2: Construct an adaptive dual-filter U-Net network, including an encoder and a decoder. Two fusion layers are set in the upsampling module of the decoder. Each fusion layer generates an adaptive low-pass filter and an adaptive high-pass filter to perform filtering fusion on the extracted feature maps.

[0072] S3: Use the training sample set to train and optimize the adaptive dual-filter U-Net network to obtain a remote sensing image segmentation model;

[0073] S4: Collect the remote sensing image to be segmented, input it into the remote sensing image segmentation model, and obtain the road segmentation result.

[0074] Furthermore, preprocessing includes image annotation, which annotates the category of each pixel in the remote sensing image with a pixel-level label.

[0075] Furthermore, the encoder extracts low-level features in the remote sensing images of the training sample set; the low-level features include edge information, texture information, etc.; the encoder includes 4 downsampling layers, the remote sensing image is used as input, each downsampling layer outputs a feature map, the output of the previous downsampling layer is used as the input of the next downsampling layer, and the first input feature map, the second input feature map, the third input feature map and the fourth input feature map are obtained respectively, and the sizes of the four feature maps decrease successively.

[0076] Furthermore, the decoder includes an upsampling module, a skip connection layer, a convolution module and a segmentation module connected in sequence; the upsampling module includes 4 upsampling layers and two fusion layers, wherein the fourth input feature map is used as input, each upsampling layer outputs a feature map, and the output of the previous upsampling layer is used as the input of the next upsampling layer to obtain the fourth output feature map, the third output feature map, the second output feature map and the first output feature map respectively, and the sizes of the four feature maps are increased in sequence; the first fusion layer is arranged in parallel with the third upsampling layer to fuse the third output feature map and output the first fused feature map; the second fusion layer is arranged in parallel with the fourth upsampling layer to fuse the second output feature map and output the second fused feature map; the fourth output feature map, the first fused feature map, the third fused feature map and the first output feature map are merged with the corresponding input feature map in the skip connection layer and then input into the convolution module for fusion to obtain the third fused feature map, and the third fused feature map is input into the segmentation module for road classification to obtain the road segmentation result. The processing flow of the upsampling layer is as follows: Figure 1 shown.

[0077] Furthermore, each fusion layer includes an expansion module, two convolutional layers, a filter generation module, a low-pass filter module, a high-pass filter module and a feature fusion module; Figure 3 As shown;

[0078] The expansion module expands the size of the input output feature map by 2 times to obtain an expanded output feature map;

[0079] The first convolution layer performs a convolution operation on the input output feature map to obtain the first convolution feature map, and the second convolution layer performs a convolution operation on the expanded output feature map to obtain the second convolution feature map;

[0080] The filter generation module generates an adaptive low-pass filter and an adaptive high-pass filter according to the second convolution feature map;

[0081] The low-pass filtering module uses an adaptive low-pass filter to filter the first convolution feature map to obtain a low-pass filtering feature map, and performs pixel random fusion with the first convolution feature map to obtain a low-frequency feature map;

[0082] The high-pass filtering module uses an adaptive high-pass filter to filter the second convolution feature map to obtain a high-pass filtering feature map, and adds and fuses the high-frequency feature map with the second convolution feature map pixel by pixel to obtain a high-frequency feature map;

[0083] The feature fusion module fuses the low-frequency feature map and the high-frequency feature map to obtain a fused feature map, including a first fused feature map or a second fused feature map.

[0084] Furthermore, the filter generation module includes a convolution layer, a pixel reordering layer, a softmax layer and a filter inversion layer connected in sequence, such as Figure 2 As shown in the figure, the second convolution feature map is obtained through the convolution operation of the convolution layer to obtain the third convolution feature map; the pixel reordering layer reorders the pixels of the third convolution feature map to obtain the reordered feature map; the reordered feature map is reshaped or activated by the softmax layer to obtain a low-pass filter; the filter inversion layer inverts the low-pass filter according to the input identity kernel, and the identity kernel is a 3×3 convolution kernel. In essence, it uses the 3×3 convolution kernel to convolve the low-pass filter to achieve inversion and obtain a high-pass filter. The convolution layer uses a 3×3 convolution layer, and the softmax layer uses the Softmax function.

[0085] Furthermore, the filter generation module predicts a spatially varying low-pass filter to reduce intra-class inconsistency during upsampling. The specific process of implementing adaptive low-pass filter generation and low-pass filtering through the second convolutional layer, the filter generation module, and the low-pass filtering module is as follows:

[0086] S311: The second convolution layer uses a 1×1 convolution layer to perform a convolution operation on the extended output feature map to obtain a second convolution feature map X;

[0087] S312: The filter generation module performs convolution, pixel rearrangement, reshaping or activation operations on the second convolution feature map in sequence to generate an adaptive low-pass filter It is expressed as:

[0088]

[0089] in, Represents the kernel size of the low-pass filter. After passing through the softmax layer, a smooth low-pass filter is obtained. i and j represent the row and column coordinates in the feature map respectively; p and q represent the relative positions in the low-pass filter kernel respectively; Ω represents a 3×3 filter kernel; PixelShuffle represents pixel shuffling to achieve pixel rearrangement; Softmax represents the Softmax function to achieve activation operation; the specific process is:

[0090] Step 3121: Perform a convolution operation on the second convolution feature map to obtain a third convolution feature map

[0091] Step 3122: Use pixel shuffling operation to perform pixel deshuffling on the third convolution feature map W l Reshape it, reducing the height and width by half, while expanding the channel by 4 times;

[0092] Step 3123: Apply the Softmax function to each filter kernel separately to ensure that the weight of each filter kernel still satisfies the properties of the probability distribution after applying the Softmax function. Divide the channels into 4 groups, each with a spatially transformed adaptive low-pass filter, expressed as Among them, g∈1, 2, 3, 4 represents the group;

[0093] S313: The low-pass filtering module uses an adaptive low-pass filter to smooth and upsample the first convolution feature map; uses a low-pass filter to use a sub-pixel upsampling technique to filter the filtered low-pass filter feature ψ l+1 Rearrange and upsample to obtain 2 times upsampled feature values Specifically:

[0094] Step 3131: The first convolution feature map is filtered using an adaptive low-pass filter to obtain four sets of low-pass filter features. g=1,2,3,4, expressed as:

[0095]

[0096] S3132: Rearrange the 4 sets of low-pass filter features into 2 times upsampled feature values ​​to obtain low-frequency feature maps It is expressed as:

[0097]

[0098] Among them, PixelShuffle() represents pixel shuffling, which is an operation of converting channel information into spatial information.

[0099] Furthermore, the filter inversion layer inverts the low-pass filter to generate an adaptive high-pass filter, and the high-pass filter module uses the adaptive high-pass filter to filter, extract high-frequency detail information to enhance the detailed boundary information lost during the downsampling process, and achieve more accurate boundary delineation; the specific process is:

[0100] S321: Invert the low-pass filter to generate an adaptive high-pass filter It is expressed as:

[0101]

[0102] Where E represents the unit core; in order to ensure that the final generated core It is high-pass. First, use per-core softmax to obtain low-pass kernels, and then subtract these kernels from the unit kernel E. The weights of the unit kernel E are [[0,0,0],[0,1,0],[0,0,0]];

[0103] S322: After filtering the second convolution feature map using an adaptive high-pass filter, the residual is added to the second convolution feature map to obtain a high-frequency feature map It is expressed as:

[0104]

[0105] Represents the feature value of the (i, j) position in the high-frequency feature map of the lth layer.

[0106] Furthermore, the feature fusion module upsamples the feature values ​​by a factor of 2. (Low frequency feature map) and high frequency feature map Perform pixel-by-pixel addition to obtain a fused feature map; specifically:

[0107] Upsample the feature values ​​by a factor of 2 And high frequency feature map A 1×1 convolutional layer is used to compress the channels, and the compressed features are added pixel by pixel. Then, the fused feature map is obtained by upsampling, which is expressed as:

[0108]

[0109] in, represents the compressed features after fusion, where r is the channel reduction rate, which is used to reduce the subsequent calculation cost of the three; Represents an upsampling operation, applying nearest neighbor interpolation to upsample the low-resolution feature map to a high resolution; Conv 1×1 Represents a 1×1 convolution operation.

[0110] Furthermore, the segmentation module uses the activation function Softmax to perform road target segmentation.

[0111] Furthermore, the process of constructing the optimization target to realize model training is as follows:

[0112] S331: Fusion feature map Y l The predicted segmentation result is obtained after passing through the skip connection layer, convolution module and segmentation head of the segmentation module in sequence. The training sample set includes the real segmentation results y annotated by humans i ;

[0113] S332: Segmentation results based on prediction And the true segmentation result y i Calculate the loss value and optimize the network. The optimization goal is to make the predicted segmentation result consistent with the actual segmentation result. According to the optimization goal, terminate the model training and obtain the remote sensing image segmentation model.

[0114] Furthermore, S3 uses multiple loss functions to optimize the network to calculate the loss value, and the loss value L is expressed as:

[0115] L=L ce +λL b

[0116]

[0117]

[0118] Among them, L ce represents the cross entropy loss; L b represents the boundary loss; λ represents the model parameter; N represents the number of samples.

[0119] Furthermore, the Adam optimizer is used in S3 to optimize the network. The parameter update rule of the Adam optimizer is:

[0120]

[0121] Among them, θ t is the model parameter at time step t; α is the learning rate; β is the exponential decay rate of the calculated gradient, and β is the exponential decay rate of the calculated gradient square; g tis the gradient at time step t; m t is the cumulative average of the squared gradients at time step t.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A U-Net remote sensing image road segmentation method based on adaptive dual filters, characterized in that: The following steps are involved: Step 1: Collect remote sensing images and preprocess them to construct a training sample set; Step 2: Construct an adaptive dual-filter U-Net network, including an encoder and a decoder. Two fusion layers are set in the upsampling module of the decoder. Each fusion layer generates an adaptive low-pass filter and an adaptive high-pass filter to perform filtering fusion on the extracted feature maps. Step 3: Use the training sample set to train and optimize the adaptive dual-filter U-Net network to obtain a remote sensing image segmentation model; Step 4: Collect the remote sensing image to be segmented, input it into the remote sensing image segmentation model, and obtain the road segmentation result.

2. According to claim 1, a U-Net remote sensing image road segmentation method based on adaptive dual filters is characterized in that: The encoder extracts low-level feature maps from the remote sensing images of the training sample set; the low-level feature maps include edge information and texture information; the encoder includes 4 downsampling layers, the remote sensing image is used as input, each downsampling layer outputs a feature map, the output of the previous downsampling layer is used as the input of the next downsampling layer, and the first input feature map, the second input feature map, the third input feature map and the fourth input feature map are obtained respectively, and the sizes of the four feature maps decrease successively.

3. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 2, characterized in that: The decoder includes an upsampling module, a skip connection layer, a convolution module and a segmentation module connected in sequence; the upsampling module includes four upsampling layers and two fusion layers, wherein the fourth input feature map is used as input, each upsampling layer outputs a feature map, and the output of the previous upsampling layer is used as the input of the next upsampling layer, and the fourth output feature map, the third output feature map, the second output feature map and the first output feature map are obtained respectively, and the sizes of the four feature maps are increased in sequence; the first fusion layer is arranged in parallel with the third upsampling layer, and is used to fuse the third output feature map and output the first fused feature map; the second fusion layer is arranged in parallel with the fourth upsampling layer, and is used to fuse the second output feature map and output the second fused feature map; The fourth output feature map, the first fused feature map, the third fused feature map and the first output feature map are merged with the corresponding input feature map in the jump connection layer and input into the convolution module for fusion to obtain the third fused feature map. The third fused feature map is input into the segmentation module for road classification to obtain the road segmentation result.

4. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 3 is characterized in that: Each fusion layer includes an expansion module, two convolutional layers, a filter generation module, a low-pass filter module, a high-pass filter module, and a feature fusion module; The expansion module expands the size of the input output feature map by 2 times to obtain an expanded output feature map; The first convolution layer performs a convolution operation on the input output feature map to obtain the first convolution feature map, and the second convolution layer performs a convolution operation on the expanded output feature map to obtain the second convolution feature map; The filter generation module generates an adaptive low-pass filter and an adaptive high-pass filter according to the second convolution feature map; The low-pass filtering module uses an adaptive low-pass filter to filter the first convolution feature map to obtain a low-pass filtering feature map, and performs pixel random fusion with the first convolution feature map to obtain a low-frequency feature map; The high-pass filtering module uses an adaptive high-pass filter to filter the second convolution feature map to obtain a high-pass filtering feature map, and adds and fuses the high-frequency feature map with the second convolution feature map pixel by pixel to obtain a high-frequency feature map; The feature fusion module fuses the low-frequency feature map and the high-frequency feature map to obtain a fused feature map.

5. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 4, characterized in that: The filter generation module includes a convolution layer, a pixel reordering layer, a softmax layer and a filter inversion layer connected in sequence; the second convolution feature map is obtained by the convolution operation of the convolution layer to obtain a third convolution feature map; the pixel reordering layer reorders the pixels of the third convolution feature map to obtain a reordered feature map; the reordered feature map is activated by the softmax layer to obtain a low-pass filter; The filter inversion layer convolves the low-pass filter with a 3×3 convolution kernel to obtain a high-pass filter.

6. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 4, characterized in that: The specific process of realizing adaptive low-pass filter generation and low-pass filtering through the second convolution layer, filter generation module and low-pass filtering module is as follows: Step 311: the second convolutional layer uses a 1×1 convolutional layer to perform a convolution operation on the extended output feature map to obtain a second convolutional feature map; Step 312: The filter generation module performs convolution, pixel rearrangement, and activation operations on the second convolution feature map in sequence to generate an adaptive low-pass filter. It is expressed as: Among them, i and j represent the row and column coordinates in the feature map respectively; p and q represent the relative positions in the low-pass filter kernel respectively; PixelShuffle represents pixel shuffling to achieve pixel rearrangement; Softmax represents the Softmax function; the specific process is: Step 3121: Perform a convolution operation on the second convolution feature map to obtain a third convolution feature map Step 3122: Use pixel shuffle operation to perform convolution on the third convolution feature map Reshape it, reducing the height and width by half, while expanding the channel by 4 times; Step 3123: Apply the Softmax function to each filter kernel separately, ensure that the weight of each filter kernel still satisfies the property of probability distribution after applying the Softmax function, and divide the channels into 4 groups, each group having a spatially transformed adaptive low-pass filter; Step 313: The low-pass filtering module filters the first convolution feature map using an adaptive low-pass filter, and performs an adaptive low-pass filtering on the filtered low-pass filter feature ψ l+1 Rearrange and upsample to obtain 2 times upsampled feature values Specifically: Step 3131: The first convolution feature map is filtered using an adaptive low-pass filter to obtain four sets of low-pass filter features. g=1,2,3,4, expressed as: Step 3132: Rearrange the 4 sets of low-pass filter features into 2 times upsampled feature values It is expressed as: Among them, PixelShuffle() represents pixel shuffling, which is an operation of converting channel information into spatial information.

7. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 6, characterized in that: The filtering inversion layer inverts the low-pass filter to generate an adaptive high-pass filter, and the high-pass filtering module uses the adaptive high-pass filter to filter as follows: Step 321: Invert the low-pass filter to generate an adaptive high-pass filter It is expressed as: Among them, E represents the unit nucleus; Step 322: After filtering the second convolution feature map using an adaptive high-pass filter, the residual is added to the second convolution feature map to obtain a high-frequency feature map. It is expressed as: in, Represents the feature value of the (i, j) position in the high-frequency feature map of the lth layer.

8. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 7, characterized in that: The feature fusion module upsamples the feature values ​​by a factor of 2 And high frequency feature map Perform pixel-by-pixel addition to obtain a fused feature map; specifically: Upsample the feature values ​​by a factor of 2 And high frequency feature map A 1×1 convolutional layer is used to compress the channels, and the compressed features are added pixel by pixel. Then, the fused feature map is obtained by upsampling, which is expressed as: in, represents the fused feature map, where r is the channel reduction rate; Represents upsampling operation; Conv 1×1 Represents a 1×1 convolution operation.

9. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 2, characterized in that: The process of constructing the optimization target to implement model training is: Step 331: The segmentation module outputs the predicted segmentation result, and the training sample set is annotated with the real segmentation result; Step 332: Calculate the loss value based on the predicted segmentation result and the true segmentation result, take the consistency between the predicted segmentation result and the true segmentation result as the optimization goal, terminate the model training based on the optimization goal, and obtain the remote sensing image segmentation model.

10. The method for road segmentation of remote sensing images based on U-Net with adaptive dual filters according to claim 1, characterized in that: In step 3, a multi-loss function is used to optimize the network. The multi-loss function L is expressed as: L=L ce +λL b Among them, L ce represents the cross entropy loss; L b represents the boundary loss; λ represents the model parameter; Represents the predicted segmentation result; y i represents the actual segmentation result; N represents the number of samples.

Images