A remote sensing image domain adaptation semantic segmentation method based on flow matching

Abstract

This invention discloses a domain-adaptive semantic segmentation method for remote sensing images based on flow matching. Firstly, a frequency domain feature decoupling module based on Haar wavelet transform fundamentally avoids alignment interference caused by mixed processing, effectively suppressing cross-domain segmentation degradation phenomena such as blurred ground object boundaries and loss of small targets. Secondly, a pre-trained depth estimation model is introduced to generate pseudo-depth maps for remote sensing images, enhancing the model's ability to perceive cross-domain spatial structures under different imaging conditions and viewpoint changes, significantly improving the consistency of ground object spatial structures in cross-domain scenes. Thirdly, precise direction and distance guidance is provided for the alignment process of deep semantic features, effectively alleviating the problem of cross-domain category confusion, and providing semantic alignment gain for the overall domain adaptation system with extremely low additional computational overhead. Fourthly, the overall domain adaptation performance is systematically improved, and the effectiveness and superiority of the proposed method are verified on several representative cross-domain transfer benchmarks.

Description

Technical Field

[0001] This invention belongs to the field of intelligent interpretation of remote sensing images and unsupervised domain-adaptive semantic segmentation technology, and specifically relates to a domain-adaptive semantic segmentation method for remote sensing images based on flow matching. Background Technology

[0002] Semantic segmentation of remote sensing images aims to assign a semantic category label to each pixel in an image. It is a core technology for achieving detailed interpretation of ground features in fields such as urban planning, disaster emergency response, land use monitoring, and planetary topography mapping. The rise of deep learning methods has significantly improved the performance of semantic segmentation of remote sensing images. However, training high-quality models heavily relies on large-scale, accurate pixel-level labeled data. This labeling process requires not only professional geoscientific knowledge but also extremely high manpower and time costs. More importantly, due to the interplay of systematic factors such as the physical characteristics of different sensors, differences in imaging resolution, geographical location variations, and atmospheric scattering conditions, high-performance segmentation models trained in a specific data domain (source domain) often experience a sharp drop in prediction accuracy when facing another data domain (target domain), i.e., they fall into a serious domain shift problem. Unsupervised domain adaptation technology, by collaboratively utilizing labeled source domain data and unlabeled target domain data, achieves cross-domain knowledge transfer without the need for manual labeling of the target domain, providing an economical and efficient solution to the above problems.

[0003] However, existing unsupervised domain adaptive segmentation methods for remote sensing images still suffer from the following three significant technical shortcomings that urgently need to be addressed: First, existing mainstream methods generally focus on global distribution alignment in high-level feature space, making it difficult to simultaneously take into account and retain local spatial details that are crucial for accurate segmentation, resulting in blurred feature boundary segmentation and loss of small target features. Moreover, these methods are limited to single-dimensional spatial domain operations when performing domain alignment, failing to effectively distinguish between domain-invariant low-frequency global semantic components and domain-specific high-frequency texture noise components in the features.

[0004] Second, existing methods neglect the important multidimensional complementary prior information of three-dimensional spatial geometric structure contained in remote sensing images. The three-dimensional geometric priors such as the height distribution, three-dimensional shape, and topographic relief of ground features have high stability in cross-sensor and cross-view scenarios and are important auxiliary information for mitigating domain offset. Existing methods have failed to utilize this information, which makes it difficult to fundamentally alleviate problems such as inconsistent spatial structure of ground features and shape distortion in cross-domain scenarios.

[0005] Third, the discriminator in traditional adversarial training methods can only provide coarse-grained binary supervision signals on whether features come from the source domain. It lacks precise quantitative guidance on feature alignment direction and distance, which is particularly inadequate in the deepest semantic feature space of the network. Once the deep semantic alignment is deviated, subsequent segmentation decisions will produce systematic category confusion, resulting in limited generalization performance of the model.

[0006] In summary, existing remote sensing image domain adaptive segmentation techniques have significant shortcomings in three aspects: frequency domain feature decoupling, utilization of multi-dimensional prior information, and fine alignment of deep semantics, which restricts the further improvement of cross-domain segmentation accuracy.

[0007] Therefore, it is necessary to propose a domain-adaptive semantic segmentation method for remote sensing images based on flow matching. Summary of the Invention

[0008] The purpose of this invention is to provide a remote sensing image domain adaptive semantic segmentation method based on flow matching, so as to solve the technical problems of existing methods mentioned in the background art in terms of frequency domain feature decoupling, insufficient utilization of multi-dimensional complementary priors, and coarse granularity of deep semantic alignment.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a remote sensing image domain-adaptive semantic segmentation method based on flow matching, comprising the following steps: Step 1: Construct a hierarchical feature encoder using Swin Transformer V2 as the backbone. Extract multi-scale feature maps from source domain remote sensing images and target domain remote sensing images through the hierarchical feature encoder to form a multi-scale feature map pyramid. Then, use the pre-trained depth estimation base model to generate a single-channel pseudo-depth map D. Step 2: Apply a two-dimensional Haar wavelet transform to the output feature map of each level, where it is orthogonally decomposed into low-frequency approximate subbands and horizontally, vertically and diagonally decomposed into high-frequency detail subbands. Step 3: The deepest output features of the encoder are used as input and fed into the deep semantic feature alignment module based on conditional flow matching; at the same time, the deepest semantic features of the target domain are used as the set input conditions of the module; the module learns to estimate the continuous probability flow mapping from the feature distribution of the target domain to the feature distribution of the source domain through a vector field estimation network with a lightweight inverse residual network as the backbone, thereby providing alignment direction and distance estimation for each deep feature vector of the target domain in the form of a vector field; Step four: The multi-scale features of the encoder are aggregated by a multi-scale global-local decoder, and the main branch segmentation prediction map P is generated by a dual-head prediction structure. main With auxiliary branch splitting prediction graph P aux ; Step 5: Split the main branch into the prediction graph P. mainThe spatial feature fusion module, which integrates the pseudo-depth map D with depth-guided input, injects 3D geometric priors into the segmentation prediction process through channel-dimensional stitching and attention-gated weighting, generating a depth-fused segmentation prediction map P. depth ; Step 6: Construct a multi-level, multi-dimensional collaborative adversarial system consisting of four discriminators. One spatial domain discriminator is set up for the original segmentation prediction of the main branch and the auxiliary branch, and two deep domain discriminators are set up for the deep fusion segmentation prediction. The four discriminators are trained collaboratively against the feature distributions of the source domain and the target domain from two orthogonal dimensions of the original segmentation prediction and the deep fusion segmentation prediction, forming an end-to-end overall optimization objective. The encoder-decoder network and the discriminator system are jointly trained with an alternating iterative strategy.

[0010] According to the above technical solution, the specific operation method of step one is as follows: The source domain remote sensing image and the target domain remote sensing image are respectively input into a hierarchical feature encoder with Swin Transformer V2 as the backbone to extract a multi-scale feature map pyramid with four hierarchical scales. The spatial resolution of each level feature map is halved as the level increases, and the channel dimension is doubled as the level increases. Simultaneously, a pre-trained depth estimation model is used to perform monocular depth estimation on the input image, generating a single-channel pseudo-depth map D with the same resolution as the image, which encodes the three-dimensional spatial geometric structure information of the corresponding remote sensing scene with relative depth values.

[0011] Based on the above technical solution, the specific operation method of step two is as follows: At the four levels of the encoder, a two-dimensional Haar wavelet transform is applied to the output feature map of each level, orthogonally decomposing it into low-frequency approximate subbands and high-frequency detail subbands in the horizontal, vertical, and diagonal directions. The low-frequency subbands correspond to domain-invariant global semantic structure features, while the high-frequency subbands correspond to domain-specific local texture and edge information. Frequency domain attention enhancement processing, which combines channel attention and spatial attention, is applied to the low-frequency subbands. Instance-normalized domain-specific filtering is applied to the high-frequency subbands. Based on the feature distribution differences between the source and target domains in the low-frequency subbands at each level, a multi-scale frequency domain consistency loss function L is constructed. freq This guides the encoder to learn frequency domain feature representations that are invariant to the learning domain.

[0012] According to the above technical solution, the two-dimensional Haar wavelet transform achieves four-subband frequency domain orthogonal decomposition by sequentially applying separable convolution operations of low-pass and high-pass filters in the horizontal and vertical directions to the input feature map. A low-pass filter calculates the mean of adjacent elements in a row or column and then divides it by the square root of two to extract a smooth, low-frequency approximate component. A high-pass filter extracts high-frequency detail components of edges and textures by subtracting the difference between adjacent elements in a row or column and then dividing by the square root of two. Each sub-band has a spatial resolution that is half that of the original feature map, the number of channels remains unchanged, and the decomposition operation is performed independently on the four levels of the encoder, forming a multi-scale parallel frequency domain decoupling structure.

[0013] According to the above technical solution, the frequency domain attention enhancement processing applies a joint enhancement operation of channel attention and spatial attention to each level of low-frequency sub-bands: A) Extract dual-channel statistics through global average pooling and global max pooling, and generate channel attention weight vectors through a multilayer perceptron with shared parameters to weight the low-frequency subband features by channel dimension. B) Perform mean and maximum operations on the channel-weighted features along the channel axis and concatenate them. Then, generate a spatial attention weight map by 3×3 convolution and Sigmoid activation. C) Two levels of attention are applied sequentially to the low-frequency subband, outputting an enhanced domain-invariant low-frequency feature representation; For the high-frequency subband, an instance normalization operation is applied to normalize it to zero mean and unit variance in the spatial dimension, thereby removing domain-specific high-frequency style interference introduced by differences in sensor characteristics, illumination conditions and imaging parameters, while preserving edge structure information. D) The multi-scale frequency domain consistency loss function L freq The optimization objective is to use the sum of the weighted mean square differences between the global statistics of low-frequency enhanced features in the source and target domains at each level. The weight coefficients of each level increase with the level depth to apply stronger alignment constraints to the deeper low-frequency features with richer semantic information.

[0014] Based on the above technical solution, the specific operation method of step three is as follows: The training method of the deep semantic feature alignment module based on conditional flow matching is as follows: with the deepest semantic feature of the target domain as the conditional input, under the uniform sampling time step t (t∈[0,1]), the intermediate transition state is constructed by the linear interpolation of the deep features of the source domain and the deep features of the target domain. The time step t = 0 corresponds to the target domain feature, the time step t = 1 corresponds to the source domain feature, and the corresponding true target direction vector is the difference between the source domain feature and the target domain feature. The training objective is to minimize the mean square error between the predicted direction of the vector field estimation network and the true target direction, so that the network learns a deterministic continuous probability flow mapping from the feature distribution of the target domain to the feature distribution of the source domain. The vector field estimation network backbone adopts a lightweight inverted residual network architecture, with an inverted bottleneck structure that performs pointwise convolution for dimensionality upscaling, depthwise separable convolution for spatial feature extraction, and pointwise convolution for dimensionality downscaling as the basic unit. The overall computational cost is negligible compared to the backbone encoder.

[0015] According to the above technical solution, the specific operation method of step four is as follows: The multi-scale global-local decoder adopts a top-down feature aggregation path, and aggregates multi-scale features step by step at each scale fusion node through the parallel combination of global self-attention branch and local depth-separable convolution branch to capture long-distance semantic dependencies and fine spatial details. The weighted feature fusion module dynamically weights and sums features at different levels using learnable scalar weights. The dual-head prediction structure at the end of the decoder outputs the main branch split prediction map P. main With auxiliary branch splitting prediction graph P aux It provides multiple supervision signals during the training phase and provides hierarchical input representations for the multi-level discriminator during the domain adaptation phase.

[0016] Based on the above technical solution, the specific operation method of step five is as follows: The depth-guided spatial feature fusion module achieves 3D geometric prior injection through the following operations: First, the pseudo-depth map D is scaled to the same spatial resolution as the segmentation prediction map by bilinear interpolation, and then concatenated with the segmentation prediction features in the channel dimension and projected to the original number of channels by 1×1 convolution. Subsequently, channel-level Sigmoid-gated weight vectors are generated through global average pooling and a multilayer perceptron. These vectors are then adaptively weighted channel-by-channel to form geometrically enhanced segmentation features. After normalization, the resulting deep fusion segmentation prediction map P is output. depth ; For deep fusion output P depth Adversarial alignment training is carried out in conjunction with an independently designed depth domain discriminator.

[0017] Based on the above technical solution, the specific method for the mid-to-end overall optimization objective in step six is ​​as follows: The end-to-end overall optimization objective L total Defined as: L total =L seg +α·L freq +λ fm ·L fm +λ adv ·L adv L seg For source domain supervised segmentation cross-entropy loss, L freq For multi-scale frequency domain consistency loss, L fm For deep semantic alignment loss in conditional flow matching, L adv For four-way cooperative adversarial regularization loss, α, λ fm , λ adv These are the balance coefficients for the corresponding loss terms; During the training phase, an end-to-end strategy is employed, using an encoder-decoder network and four discriminators for iterative optimization. During the inference phase, only the main branch P is used. main The output achieves high-precision cross-domain semantic segmentation of remote sensing images in the target domain.

[0018] According to the above technical solution, the pre-trained depth estimation base model adopts Depth Anything ModelV2 to perform monocular depth estimation on the input remote sensing image and generate a single-channel pseudo-depth map D with the same resolution as the input image in an offline pre-computation manner without introducing additional online computing overhead.

[0019] Compared with the prior art, the beneficial effects of the present invention are: First, the frequency domain feature decoupling module based on Haar wavelet transform proposed in this invention innovatively performs orthogonal decoupling of the features of the four levels of the encoder in the frequency domain space, explicitly separating the domain-invariant low-frequency global semantic components from the domain-specific high-frequency texture detail components, and applying targeted domain adaptation strategies to the two types of components respectively, fundamentally avoiding alignment interference caused by mixed processing; the multi-scale frequency domain consistency loss function simultaneously applies frequency domain feature alignment constraints at the four levels, effectively suppressing cross-domain segmentation degradation phenomena such as blurred ground feature boundary segmentation and loss of small targets.

[0020] Secondly, this invention introduces a pre-trained depth estimation model to generate pseudo-depth maps for remote sensing images, and injects three-dimensional spatial geometric structure information into the segmentation prediction process in an adaptive gated fusion manner. This breaks through the single-dimensional bottleneck of existing methods that are limited to two-dimensional spatial domain feature alignment, and enhances the model's ability to perceive cross-domain spatial structures under different imaging conditions and viewpoint changes. The introduction of a dedicated discriminator in the depth domain further drives adversarial alignment in the geometric fusion dimension, which significantly improves the consistency of the spatial structure of ground objects in cross-domain scenarios.

[0021] Furthermore, this invention introduces a conditionally optimal transport stream matching mechanism into the remote sensing image domain adaptation task. In the deepest semantic feature space of the encoder, a continuous probability stream mapping is used to replace the coarse-grained binary supervision of the traditional adversarial discriminator, providing precise direction and distance guidance for the alignment process of deep semantic features and effectively alleviating the cross-domain category confusion problem. The backbone of this module adopts a lightweight inverse residual network, which provides semantic alignment gain for the overall domain adaptation system with extremely low additional computational overhead.

[0022] Finally, the multi-level, multi-dimensional collaborative domain discriminator system constructed in this invention simultaneously drives the distribution alignment of multiple feature spaces from two orthogonal dimensions: the original segmentation prediction and the deep fusion segmentation prediction. Together with the frequency domain consistency loss, the flow matching alignment loss, and the segmentation supervision loss, it constitutes a multi-objective collaborative optimization system, which systematically improves the overall domain adaptation performance. The effectiveness and superiority of the proposed method have been verified on several representative cross-domain transfer benchmarks. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a structural diagram of the frequency domain attention enhancement module of the present invention; Figure 3 This is a schematic diagram of the deep semantic feature alignment module based on conditional flow matching of the present invention; Figure 4 This is a schematic diagram of the deep-guided spatial feature fusion module of the present invention; Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example Please see Figure 1-4 This invention provides a technical solution: a remote sensing image domain-adaptive semantic segmentation method based on flow matching, comprising the following steps: Step 1, Multi-scale Feature Extraction and Deep Prior Generation Stage: The source and target domain remote sensing images are input into a hierarchical feature encoder based on the Swin Transformer V2 framework. The input images are first segmented and projected into the embedding space using a non-overlapping 2D convolution with a stride of 4 in the embedding layer. Subsequently, they undergo four processing stages based on shift-window cosine self-attention and residual post-normalization. A downsampling embedding layer is placed between each adjacent stage to achieve 2x spatial compression and 2x channel expansion. Figure 1 As shown; The encoder outputs a feature map pyramid with four levels, denoted as F. i (i = 1, 2, 3, 4), where F iThe output feature map of the i-th level is represented by the spatial resolution of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively, with the channel dimension increasing sequentially. At the same time, the pre-trained Depth Anything Model V2 is used to generate a single-channel pseudo-depth map D with the same spatial resolution as each image in the source and target domain images through offline pre-computation. The D encodes the three-dimensional spatial geometric topology of the remote sensing scene with relative depth values. This process is completed before training and does not introduce additional online computation overhead.

[0026] Step 2, Multi-scale frequency domain feature decoupling and alignment constraint construction stage: At the four levels of the encoder, for the output feature map F of each level i Applying a two-dimensional Haar wavelet transform, orthogonal decomposition in the frequency domain is achieved, such as... Figure 2 As shown, the feature map of the i-th level, i.e., the spatial size H i ×W i Number of channels C i For example, where H i W i C represents the height and width respectively. i Indicates the number of channels; The low-pass filter calculates the mean of adjacent elements in a row or column and then divides it by . A high-pass filter calculates the difference between adjacent elements in a row or column and then divides by... ; Through four combinations of low-pass and high-pass filters in both horizontal and vertical directions, F i Orthogonal decomposition into: low-frequency approximate subband LL i Horizontal high-frequency subband LH i Vertical high-frequency subband HL i With diagonal high-frequency subband HH i , of which LL i LH i HL i HH i The spatial resolution of each subband is (H). i / 2)×(W i / 2), the number of channels remains C i constant; LL i It gathers semantic structural information with high domain invariance, such as the spatial layout and global shape of ground features; LH i HL i HH i They respectively carry local detail information that is sensitive to domain offset, such as horizontal edges, vertical edges, and diagonal textures; The above decomposition operations are performed independently at four levels, such as Figure 2As shown. For the low-frequency subband LL_i, frequency domain attention enhancement processing is applied: global average pooling and global max pooling are performed on LL_i to extract dual-channel statistics, which are then summed after passing through a two-layer fully connected perceptron with shared parameters and activated by Sigmoid to generate a channel attention weight vector; The mean and maximum values ​​are calculated along the channel axis on the channel-weighted low-frequency features and then concatenated. A spatial attention weight map is generated by 3×3 convolution and sigmoid activation. Channel attention and spatial attention are then sequentially applied to the LL array. i Multiplying, the output is the enhanced low-frequency characteristic, denoted as LL. i enh , represents the low-frequency subband feature of the i-th level after attention enhancement; For high-frequency subband LH i HL i HH i Instance normalization is applied to normalize the spatial dimension to zero mean and unit variance, removing domain-specific high-frequency style interference while preserving edge topology. In loss construction, LL is set... i src_enh With LL i tgt_enh Let L represent the low-frequency enhancement features of the source and target domains at the i-th level, respectively. Global average pooling is performed on both, compressing them to channel-level statistics. The mean square error is calculated as the frequency domain alignment loss term at the i-th level, and the multi-scale frequency domain consistency loss L is calculated. freq Defined as:

[0027] Where λ i For the i-th level, the adaptive weighting coefficients are λ1 = 0.5, λ2 = 0.75, λ3 = 1.0, and λ4 = 1.25.

[0028] Step 3, Deep semantic feature alignment stage based on conditional flow matching: The deepest output feature F4 of the encoder is input into the deep semantic feature alignment module based on conditional flow matching, such as... Figure 3 As shown; where F4 represents the 4th layer, i.e., the deepest layer's output features, the core of this module is the vector field estimation network. Where θ represents the network parameters, and F4 is the deepest semantic feature of the target domain. tgt As a conditional input, its backbone adopts a lightweight inverted residual network architecture, with an inverted bottleneck structure of pointwise convolution for dimensionality upscaling, depthwise separable convolution for spatial feature extraction, and pointwise convolution for dimensionality downscaling as the basic unit. The overall number of parameters is negligible compared to the backbone encoder. During the training phase, based on the conditionally optimal transport theory, the source domain deep feature F4 is used. srcF4 with deep features of the target domain tgt The intermediate transition state F is constructed by linear interpolation at uniform sampling time step t (t∈[0,1]). t Its definition is:

[0029] Wherein, time step t=0 corresponds to the target domain features, and t=1 corresponds to the source domain features; the corresponding true target direction vector u t Defined as:

[0030] The stream matching alignment loss L_fm is defined as:

[0031] Where E[·] represents the expectation, and ||·|| represents the Euclidean norm of the vector, in order to minimize L fm Driven by a goal At each feature point, predict the optimal path for its migration to the feature manifold of the source domain with precise direction and magnitude; During the inference phase, the deep features of the target domain are mapped to the vicinity of the source domain feature space by a numerical solver of ordinary differential equations with a few integration steps, thereby achieving semantic-level fine-grained distribution alignment and replacing the coarse-grained binary supervision of traditional adversarial discriminators.

[0032] Step 4, Multi-scale Decoding and Dual-Branch Prediction Stage: The multi-scale feature map pyramid F output by the encoder i The data is fed into a multi-scale global-local decoder. The decoder adopts a top-down feature aggregation path and aggregates multi-scale features step by step at each scale fusion node through the parallel combination of global self-attention branch and local depthwise separable convolution branch. The weighted feature fusion module dynamically weights and sums features at different levels using learnable scalar weights, balancing the contribution weights of semantic information at each level. The resulting main branch segmentation prediction map P is output via a dual-head prediction structure. main With auxiliary branch splitting prediction graph P aux , where P main P aux All are semantic category probability distribution maps with the same resolution as the input image.

[0033] Step 5, Deep-guided spatial feature fusion stage: P, the output of step four main The spatial feature fusion module, which integrates the pre-calculated pseudo-depth map D from step one with the input depth-guided depth, is as follows: Figure 4 As shown: A) Scale D to be similar to P using bilinear interpolation. mainCompletely consistent spatial resolution, in the channel dimension and P main After splicing, the data is projected back to the original number of channels via a 1×1 convolution to form preliminary fusion features; B) The initial fused features are compressed into global channel statistics through global average pooling, and then channel-level gated weight vector g is generated through two layers of fully connected perceptron and Sigmoid activation, where g represents adaptive gated weights; C) Weight the initial fusion features channel by channel using g to form a geometrically enhanced deep fusion segmentation feature, and output the deep fusion segmentation prediction map P after normalization. depth The P depth By using an adaptive gating method to tightly couple 3D spatial geometric information with semantic segmentation decisions, the model can have a more robust spatial structure perception capability under different terrain feature height distributions and terrain undulations.

[0034] Step Six: Multi-level, Multi-dimensional Collaborative Adversarial Training and End-to-End Joint Optimization Phase: A multi-level, multi-dimensional collaborative domain discriminator system is constructed, consisting of four fully convolutional discriminator instances. The four discriminators have identical structures, all employing a fully convolutional architecture with multi-layer stride convolutions and LeakyReLU activation. They take the segmentation prediction probability map as input and output a pixel-level domain source probability map. The specific configurations of the four discriminators are as follows: Discriminator D main With P main As input, drive spatial domain adversarial alignment in the original segmentation prediction dimension; Discriminator D aux With P aux Provides additional gradient supervision for the input; Discriminator D depth_1 With D depth_2 With P depth As input, adversarial alignment is implemented in the deep fusion multimodal prediction dimension; All four discriminators employ least-squares adversarial loss, with source domain segmentation output as positive examples and target domain segmentation output as negative examples during training. The four adversarial losses are weighted and summed to form the cooperative adversarial regularization loss L. adv In this embodiment, the weights of each path are set to 1.0, and the overall supervised segmentation loss L seg Defined as:

[0035] Among them, L seg_main With L seg_aux , respectively, represent the splitting cross-entropy loss of the main branch and the auxiliary branch, and 0.4 is the loss weight coefficient of the auxiliary branch.

[0036] End-to-end overall optimization objective L totalDefined as:

[0037] Where L seg For source domain supervised segmentation cross-entropy loss, L freq For multi-scale frequency domain consistency loss, L fm For deep semantic alignment loss in conditional flow matching, L adv The four-way cooperative adversarial regularization loss is represented by α=0.1, which is the balance coefficient for frequency domain consistency loss, and λ is the coefficient for balancing the loss. fm =0.1 is the balancing coefficient for flow matching alignment loss, λ adv =1.0 is the balance coefficient against regularization loss; During the training phase, an end-to-end joint training strategy is employed, using an alternating iterative optimization of the encoder-decoder network and four discriminators. During the inference phase, only the main branch P is used. main The output enables high-precision cross-domain semantic segmentation of remote sensing images in the target domain.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A domain-adaptive semantic segmentation method for remote sensing images based on flow matching, characterized in that: Includes the following steps: Step 1: Construct a hierarchical feature encoder using Swin Transformer V2 as the backbone. Extract multi-scale feature maps from source domain remote sensing images and target domain remote sensing images through the hierarchical feature encoder to form a multi-scale feature map pyramid. Then, use the pre-trained depth estimation base model to generate a single-channel pseudo-depth map D. Step 2: Apply two-dimensional Haar wavelet transform to the output feature map of each level, where it is orthogonally decomposed into low-frequency approximate subbands and horizontally, vertically and diagonally decomposed into high-frequency detail subbands. Step 3: The deepest output feature of the encoder is used as input and fed into the deep semantic feature alignment module based on conditional flow matching. Meanwhile, the deepest semantic features of the target domain are used as the set input conditions of the module; the module learns to estimate the continuous probability flow mapping from the feature distribution of the target domain to the feature distribution of the source domain through a vector field estimation network with a lightweight inverse residual network as the backbone, thereby providing alignment direction and distance estimation for each deep feature vector of the target domain in the form of a vector field. Step four: The multi-scale features of the encoder are aggregated by a multi-scale global-local decoder, and the main branch segmentation prediction map P is generated by a dual-head prediction structure. main With auxiliary branch splitting prediction graph P aux ; Step 5: Split the main branch into the prediction graph P. main The spatial feature fusion module, which integrates the pseudo-depth map D with depth-guided input, injects 3D geometric priors into the segmentation prediction process through channel-dimensional stitching and attention-gated weighting, generating a depth-fused segmentation prediction map P. depth ; Step 6: Construct a multi-level, multi-dimensional collaborative adversarial system consisting of four discriminators. One spatial domain discriminator is set up for the original segmentation prediction of the main branch and the auxiliary branch, and two deep domain discriminators are set up for the deep fusion segmentation prediction. The four discriminators are trained collaboratively against the feature distributions of the source domain and the target domain from two orthogonal dimensions of the original segmentation prediction and the deep fusion segmentation prediction, forming an end-to-end overall optimization objective. The encoder-decoder network and the discriminator system are jointly trained with an alternating iterative strategy.

2. The remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 1, characterized in that, The specific operation method for step one is as follows: The source domain remote sensing image and the target domain remote sensing image are respectively input into a hierarchical feature encoder with Swin Transformer V2 as the backbone to extract a multi-scale feature map pyramid with four hierarchical scales. The spatial resolution of each level feature map is halved as the level increases, and the channel dimension is doubled as the level increases. Simultaneously, a pre-trained depth estimation model is used to perform monocular depth estimation on the input image, generating a single-channel pseudo-depth map D with the same resolution as the image, which encodes the three-dimensional spatial geometric structure information of the corresponding remote sensing scene with relative depth values.

3. The remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 1, characterized in that, The specific operation method for step two is as follows: At the four levels of the encoder, a two-dimensional Haar wavelet transform is applied to the output feature map of each level, orthogonally decomposing it into low-frequency approximate subbands and high-frequency detail subbands in the horizontal, vertical, and diagonal directions. The low-frequency subbands correspond to domain-invariant global semantic structure features, while the high-frequency subbands correspond to domain-specific local texture and edge information. Frequency domain attention enhancement processing, which combines channel attention and spatial attention, is applied to the low-frequency subbands. Instance-normalized domain-specific filtering is applied to the high-frequency subbands. Based on the feature distribution differences between the source and target domains in the low-frequency subbands at each level, a multi-scale frequency domain consistency loss function L is constructed. freq This guides the encoder to learn frequency domain feature representations that are invariant to the learning domain.

4. The remote sensing image domain adaptive semantic segmentation method based on flow matching according to claim 2, characterized in that, The two-dimensional Haar wavelet transform achieves four-subband frequency domain orthogonal decomposition by sequentially applying separable convolution operations of low-pass and high-pass filters in the horizontal and vertical directions to the input feature map. A low-pass filter calculates the mean of adjacent elements in a row or column and then divides it by the square root of two to extract a smooth, low-frequency approximate component. A high-pass filter extracts high-frequency detail components of edges and textures by subtracting the difference between adjacent elements in a row or column and then dividing by the square root of two. Each sub-band has a spatial resolution that is half that of the original feature map, the number of channels remains unchanged, and the decomposition operation is performed independently on the four levels of the encoder, forming a multi-scale parallel frequency domain decoupling structure.

5. The remote sensing image domain adaptive semantic segmentation method based on flow matching according to claim 3, characterized in that, The frequency domain attention enhancement process applies a combined enhancement operation of channel attention and spatial attention to each level of low-frequency sub-bands: A) Extract dual-channel statistics through global average pooling and global max pooling, and generate channel attention weight vectors through a multilayer perceptron with shared parameters to weight the low-frequency subband features by channel dimension. B) Perform mean and maximum operations on the channel-weighted features along the channel axis and concatenate them. Then, generate a spatial attention weight map by 3×3 convolution and Sigmoid activation. C) Two levels of attention are applied sequentially to the low-frequency subband, outputting an enhanced domain-invariant low-frequency feature representation; For the high-frequency subband, an instance normalization operation is applied to normalize it to zero mean and unit variance in the spatial dimension, thereby removing domain-specific high-frequency style interference introduced by differences in sensor characteristics, illumination conditions and imaging parameters, while preserving edge structure information. D) The multi-scale frequency domain consistency loss function L freq The optimization objective is to use the sum of the weighted mean square differences between the global statistics of low-frequency enhanced features in the source and target domains at each level. The weight coefficients of each level increase with the level depth to apply stronger alignment constraints to the deeper low-frequency features with richer semantic information.

6. The remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 1, characterized in that, The specific operation method for step three is as follows: The training method of the deep semantic feature alignment module based on conditional flow matching is as follows: with the deepest semantic feature of the target domain as the conditional input, under the uniform sampling time step t (t∈[0,1]), the intermediate transition state is constructed by the linear interpolation of the deep features of the source domain and the deep features of the target domain. The time step t = 0 corresponds to the target domain feature, the time step t = 1 corresponds to the source domain feature, and the corresponding true target direction vector is the difference between the source domain feature and the target domain feature. The training objective is to minimize the mean square error between the predicted direction of the vector field estimation network and the true target direction, so that the network learns a deterministic continuous probability flow mapping from the feature distribution of the target domain to the feature distribution of the source domain. The vector field estimation network backbone adopts a lightweight inverted residual network architecture, with an inverted bottleneck structure that performs pointwise convolution for dimensionality upscaling, depthwise separable convolution for spatial feature extraction, and pointwise convolution for dimensionality downscaling as the basic unit. The overall computational cost is negligible compared to the backbone encoder.

7. The remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 1, characterized in that, The specific operation method for step four is as follows: The multi-scale global-local decoder adopts a top-down feature aggregation path, and aggregates multi-scale features step by step at each scale fusion node through the parallel combination of global self-attention branch and local depth-separable convolution branch to capture long-distance semantic dependencies and fine spatial details. The weighted feature fusion module dynamically weights and sums features at different levels using learnable scalar weights. The dual-head prediction structure at the end of the decoder outputs the main branch split prediction map P. main With auxiliary branch splitting prediction graph P aux It provides multiple supervision signals during the training phase and provides hierarchical input representations for the multi-level discriminator during the domain adaptation phase.

8. The remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 1, characterized in that, The specific operation method for step five is as follows: The depth-guided spatial feature fusion module achieves 3D geometric prior injection through the following operations: First, the pseudo-depth map D is scaled to the same spatial resolution as the segmentation prediction map by bilinear interpolation, and then concatenated with the segmentation prediction features in the channel dimension and projected to the original number of channels by 1×1 convolution. Subsequently, channel-level Sigmoid-gated weight vectors are generated through global average pooling and a multilayer perceptron. These vectors are then adaptively weighted channel-by-channel to form geometrically enhanced segmentation features. After normalization, the resulting deep fusion segmentation prediction map P is output. depth ; For deep fusion output P depth Adversarial alignment training is carried out in conjunction with an independently designed depth domain discriminator.

9. The remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 1, characterized in that, The specific method for the mid-to-end overall optimization objective in step six is ​​as follows: The end-to-end overall optimization objective L total Defined as: L total =L seg +α·L freq +λ fm ·L fm +λ adv ·L adv L seg For source domain supervised segmentation cross-entropy loss, L freq For multi-scale frequency domain consistency loss, L fm For deep semantic alignment loss in conditional flow matching, L adv For four-way cooperative adversarial regularization loss, α, λ fm , λ adv These are the balance coefficients for the corresponding loss terms; During the training phase, an end-to-end strategy is employed, using an encoder-decoder network and four discriminators for iterative optimization. During the inference phase, only the main branch P is used. main The output achieves high-precision cross-domain semantic segmentation of remote sensing images in the target domain.

10. A remote sensing image domain-adaptive semantic segmentation method based on flow matching according to claim 2, characterized in that, The pre-trained depth estimation base model adopts Depth Anything Model V2 to perform monocular depth estimation on the input remote sensing image and generate a single-channel pseudo-depth map D with the same resolution as the input image in an offline pre-computation manner without introducing additional online computation overhead.

Images