Alluvial island instantaneous waterline segmentation method fusing MobileViT and dynamic resampling

Abstract

The invention discloses an alluvial island instantaneous waterline segmentation method fusing MobileViT and dynamic resampling. The method comprises the steps that a remote sensing image, shot by an unmanned aerial vehicle, of an alluvial island area is acquired and preprocessed; and inputting the preprocessed remote sensing image into the trained semantic segmentation model for processing, and accurately and efficiently obtaining a segmentation result of the instantaneous waterline in the remote sensing image. The semantic segmentation model is a lightweight ALISEg model constructed based on a DeepLabv3 + framework, a MobileViT structure is introduced into an encoder part, local modeling advantages and global perception capability are integrated, so that local detail recognition capability and global context perception capability are greatly enhanced, a DSC-ASPP module is designed, depth separable cavity convolution and a CBAM attention mechanism are combined, and the semantic segmentation model has the advantages of being high in robustness, high in robustness and high in robustness. And multi-scale spatial features can be fully captured. And meanwhile, a DySample module and an H-RAMi module are introduced into a decoder part, so that the up-sampling precision and the multi-scale feature fusion effect are improved, and the details of the water edge line are better reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to an alluvial island instantaneous waterline segmentation method fusing MobileViT and dynamic resampling. BACKGROUND

[0002] As a key geographical unit in the marine environment, islands have multiple values such as strategy, ecology and resources. Among many types of islands, alluvial islands are formed by the deposition of sediment transported by water flow, tides and wind power. The geological structure of alluvial islands is loose and the morphology changes frequently, so alluvial islands become a special object in the study and management of dynamic evolution of the coastal zone. Compared with bedrock islands composed of hard bedrock and relatively stable structure, alluvial islands are more sensitive to natural dynamic processes, and the waterline is often affected by tides, waves and seasonal deposition, resulting in obvious fluctuations. This characteristic not only increases the difficulty of waterline information extraction and dynamic analysis, but also puts higher requirements on the timeliness and accuracy of monitoring.

[0003] In recent years, remote sensing technology and deep learning methods have provided a new technical path for the dynamic monitoring of alluvial islands, especially in the extraction of instantaneous waterline under high-resolution images. At present, such methods can be divided into manual visual interpretation and computer automatic recognition. Manual visual interpretation method has strong intuitiveness, high flexibility, good accuracy and strong anti-noise ability, and can make judgments by integrating multi-source information, so it is often used as a supplement or accuracy verification for automatic methods. However, this method relies on the experience of interpreters, has high labor intensity and limited efficiency, and is difficult to meet the needs of large-scale and high-frequency dynamic monitoring.

[0004] In contrast, computer automatic recognition methods have become the focus of current research due to their high efficiency and scalability. Related methods mainly include traditional image processing methods and semantic segmentation models based on deep learning. Traditional methods such as threshold segmentation, edge detection, object-oriented analysis and level set C-V model. Among them, threshold segmentation sets threshold according to image gray or spectral features, which is simple and efficient, but sensitive to image contrast and easily disturbed by noise; edge detection extracts boundaries based on image gradient changes, which is suitable for areas with clear edges; object-oriented analysis takes image objects as processing units, which has high accuracy but requires complex rule setting; level set C-V model drives curve evolution through regional gray difference, which can handle fuzzy boundaries, but is sensitive to initial conditions and has high computational complexity.

[0005] With the rapid development of deep learning, semantic segmentation-based deep learning methods, leveraging their end-to-end feature learning capabilities, have demonstrated stronger feature modeling and generalization abilities in water boundary line extraction. Their core objective is to achieve pixel-level fine-grained classification of water and land. Typical network architectures include DANet, SegNet, SegFormer, PSPNet, HRNet, U-Net, and the DeepLab series. For example, DANet enhances the discriminative power of spatial and channel features by introducing a self-attention mechanism; SegNet utilizes max-pooling indexing for efficient upsampling, making it suitable for resource-constrained scenarios; and SegFormer integrates Transformer with lightweight design.

[0006] MLP achieves a balance between speed and accuracy; PSPNet acquires multi-scale contextual information through pyramid pooling; HRNet utilizes a multi-resolution parallel structure to preserve spatial and semantic features; U-Net enhances detail restoration capabilities through an encoder-decoder structure and skip connections; and the DeepLab series enhances multi-scale feature extraction capabilities through dilated convolutions and spatial pyramid pooling. Compared to traditional methods that rely on empirical rules, these deep learning models exhibit more significant advantages in extraction accuracy and robustness.

[0007] Despite advancements in various methods, the complex and dynamically changing shoreline characteristics of alluvial islands still present unique challenges: the shoreline is influenced by multiple factors such as tides, waves, and sediment deposition, exhibiting heterogeneity and blurred boundaries. These characteristics increase the difficulty of automated extraction of instantaneous waterline and place higher demands on the model's segmentation accuracy, robustness, and detail sensitivity. Summary of the Invention

[0008] This invention provides a method for instantaneous waterline segmentation of alluvial islands that integrates MobileViT and dynamic resampling, aiming to effectively solve the above-mentioned technical problems.

[0009] According to a first aspect of the present invention, the present invention provides a method for instantaneous waterline segmentation of alluvial islands that integrates MobileViT and dynamic resampling, characterized in that it includes: acquiring remote sensing images of an alluvial island region taken by a UAV and performing preprocessing; inputting the preprocessed remote sensing images into a trained semantic segmentation model for processing to obtain the segmentation result of the instantaneous waterline in the remote sensing images; wherein the semantic segmentation model is built based on DeepLabv3+, and the encoder of the model includes a MobileViT structure and a DSC-ASPP module.

[0010] Furthermore, the decoder of the semantic segmentation model includes a DySample upsampling module and an H-RAMi feature fusion module.

[0011] Furthermore, the MobileViT structure includes shallow convolutions, an MV2 module, and a MobileViT Block module. The MV2 module employs an inverted residual structure, using depthwise separable convolutions and skip connections to achieve downsampling and feature transformation, thereby building a lightweight network foundation and enhancing feature expressiveness. The MobileViT Block module integrates convolutional neural networks and Transformers to capture global semantic features and fuse them with local features, achieving high-precision semantic segmentation.

[0012] Furthermore, the MobileViT Block module sequentially performs three stages: local representation, global representation, and feature fusion, thereby achieving the fusion of local details and global features. The specific process is as follows:

[0013] In the local representation stage, local spatial structure information in the input features is extracted through convolution operations, preserving fine-grained details;

[0014] In the global representation stage, the feature map is flattened into a sequence form by the unfold operation and input into the Transformer module. The long-distance contextual dependencies are effectively modeled through the multi-head self-attention mechanism. Then, the sequence is reconstructed back to the original spatial structure by the fold operation to restore the position information.

[0015] In the feature fusion stage, a residual connection method is used to fuse local and global features, further improving the completeness and semantic richness of feature representation.

[0016] Furthermore, the DSC-ASPP module is used to extract and fuse multi-scale contextual information around the waterline of the alluvial island in the remote sensing image;

[0017] The DSC-ASPP module extracts multi-scale features through parallel multi-branch depthwise separable dilated convolution. The depthwise separable dilated convolution is composed of depthwise dilated convolution and pointwise convolution in series. The depthwise dilated convolution performs dilated convolution on each input channel to extract features, and the pointwise convolution is used to achieve cross-channel information integration.

[0018] Furthermore, after each depth-separable dilated convolution branch, the DSC-ASPP module also introduces the channel attention module and spatial attention module from the CBAM attention mechanism in series, thereby performing channel- and spatial weighted reconstruction of the extracted multi-scale feature maps.

[0019] Furthermore, the DySample upsampling module guides the sampling points to focus on semantically key regions by adaptively predicting the sampling offset based on the input features, thereby improving the upsampling accuracy of the waterline edge.

[0020] Furthermore, the H-RAMi feature fusion module is used to perform spatial resolution alignment and channel concatenation on the input multi-scale features. The concatenated features are mixed and enhanced by an efficient convolution module that integrates hierarchical residual mechanism and attention mechanism, thereby strengthening the collaborative expression of local details and global semantics.

[0021] According to a second aspect of the present invention, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the steps of the above-described method for instantaneous waterline segmentation of alluvial islands that integrates MobileViT and dynamic resampling.

[0022] According to a third aspect of the invention, the invention also provides a storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps of any of the above-described methods for instantaneous waterline segmentation of alluvial islands that integrates MobileViT and dynamic resampling.

[0023] Through one or more embodiments of the above-described embodiments of the present invention, at least the following technical effects can be achieved: The present invention proposes a method for segmenting the instantaneous waterline of alluvial islands by integrating MobileViT and dynamic resampling. The lightweight semantic segmentation model ALISeg, built on the DeepLabv3+ framework, accurately and efficiently extracts the instantaneous waterline of alluvial islands from UAV remote sensing images. Because the ALISeg model introduces a MobileViT structure in the encoder, it combines the advantages of local modeling with global perception capabilities, thus greatly enhancing the ability to identify local details and the ability to perceive global context. Furthermore, a DSC-ASPP module is designed, combining depthwise separable dilated convolution with the CBAM attention mechanism, which can fully capture multi-scale spatial features. Simultaneously, in the decoder, DySample and H-RAMi modules are introduced to improve upsampling accuracy and multi-scale feature fusion effects, thereby better restoring the details of the waterline. Attached Figure Description

[0024] The technical solution and other beneficial effects of the present invention will become apparent from the following detailed description of specific embodiments of the invention, in conjunction with the accompanying drawings.

[0025] Figure 1 This is a schematic diagram of the structure of the lightweight semantic segmentation model ALISeg provided in the embodiments of the present invention;

[0026] Figure 2 This is a schematic diagram of the MobileViT Backbone main structure provided in an embodiment of the present invention;

[0027] Figure 3This is a schematic diagram comparing standard convolution and depthwise separable convolution provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the MobileViT Block structure provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram comparing the receptive fields of standard convolution and dilated convolution provided in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the CBAM attention mechanism module provided in an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of the structure of the DySample module provided in an embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram of the structure of the H-RAMi module provided in an embodiment of the present invention;

[0033] Figure 9 This is a typical example image of the instantaneous waterline of an alluvial island under UAV remote sensing imagery provided in this embodiment of the invention;

[0034] Figure 10 These are the instantaneous waterline extraction results of alluvial islands using different models provided in the embodiments of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0036] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0037] To address the aforementioned technical issues, this invention provides a method for segmenting the instantaneous waterline of alluvial islands by integrating MobileViT and dynamic resampling. The method utilizes the lightweight semantic segmentation model ALISeg (Alluvial Island Segmentation) to extract the instantaneous waterline of alluvial islands from UAV remote sensing imagery. The method and the ALISeg model are described below with reference to the accompanying drawings.

[0038] The instantaneous waterline segmentation method for alluvial islands that integrates MobileViT and dynamic resampling according to embodiments of this application includes the following steps:

[0039] Step S1: Acquire remote sensing images of the alluvial island area taken by the drone and perform preprocessing.

[0040] In this step, the acquired raw remote sensing images underwent the following preprocessing:

[0041] First, the image data is standardized, converting pixel values ​​from the range [0, 255] to the range [-1, 1]. Next, dimensionality adjustment is performed, converting the image data from (H, W, C) format to (C, H, W) format to meet PyTorch input requirements, and adding a batch dimension, ultimately forming an input format of (B, C, H, W). Finally, the corresponding segmentation label image is read, and the labels are converted to one-hot encoding format, ensuring that the label values ​​do not exceed the range of the number of categories.

[0042] The above preprocessing ensures the consistency and diversity of input data, thereby improving the training effect and generalization ability of the model.

[0043] Step S2: The preprocessed remote sensing image is input into the trained lightweight semantic segmentation model ALISeg for processing. Multi-scale features are extracted based on the encoder, and multi-scale features are fused after high-precision sampling by the decoder, thereby obtaining and outputting the segmentation result of the instantaneous waterline in the remote sensing image.

[0044] The data set acquisition process for training the ALISeg model in this embodiment is as follows:

[0045] A DJI Phantom 4 RTK drone equipped with a high-performance CMOS sensor was used to collect drone-based remote sensing images of alluvial islands. The data collection period covered July and August 2023, and the acquired images have a spatial resolution of approximately 1 cm, meeting the requirements for high-precision surface feature extraction. These raw images were processed using ArcMap software, including data augmentation, manual annotation and cropping of waterline lines, to construct a standardized dataset containing 2120 high-quality samples. To ensure the scientific rigor and generalization ability of the model training, this dataset was divided into a training set and a validation set in an 8:2 ratio, used for model parameter optimization and subsequent performance evaluation, respectively. Data augmentation operations in this embodiment include, but are not limited to, random scaling and distortion, random flipping, filling gray bars, Gaussian blur, random rotation, and color gamut transformation.

[0046] Figure 1 This is a schematic diagram of the lightweight semantic segmentation model ALISeg shown in the embodiments of this application. The ALISeg model in the embodiments of this application is optimized and extended based on the DeepLabv3+ network framework, such as... Figure 1 As shown, in the encoder stage, MobileViT is introduced as the backbone feature extraction network to improve the model's feature modeling ability in complex scenes and its ability to capture long-range dependencies. The dilated convolutions in the original ASPP module are replaced with depthwise separable dilated convolutions, and a CBAM attention mechanism is introduced in series. This enhances the expressive power of features and the discriminative power of the attention mechanism while maintaining the network's lightweight nature. In the decoder stage, the DySample module is introduced to implement flexible resampling operations, restoring edge structures and detailed information. An H-RAMi module is also added to further improve the model's ability to recognize small boundaries and blurred regions while reducing feature information loss. The following sections will provide a detailed introduction to each module in the ALISeg model, with accompanying figures.

[0047] For local information extraction, Convolutional Neural Networks (CNNs) extract local features layer by layer through convolution. However, feature extraction relies heavily on the local receptive field of the convolutional kernels, which to some extent limits the network's ability to perceive global information. Vision Transformers utilize self-attention mechanisms to capture the dependencies between any two locations in an image, which is beneficial for obtaining global features and contextual information, but its ability to process local information is poor. Therefore, this application's embodiment replaces the original Xception network with MobileViT as the backbone feature extraction module in the DeepLabv3+ framework. This combines the local modeling advantages of CNNs with the global perception capabilities of ViT, and features a lightweight structural design that balances accuracy and efficiency. The structure of MobileViT is as follows: Figure 2As shown, it utilizes shallow convolution and MV2 modules to extract fine local information, and combines deep MobileViT Block to capture global semantics, thereby achieving joint modeling of multi-scale features from local to global, balancing model performance and inference efficiency.

[0048] The MV2 module employs an inverted residual block structure, which enhances feature representation while reducing computational cost through depthwise separable convolutions and skip connections. It is a fundamental component for building the lightweight MobileViT network. While standard residual structures extract features using a "compression-convolution-expansion" strategy, the inverted residual structure uses an "expansion-depthwise convolution-compression" approach. Depthwise convolution extracts features independently per channel, while pointwise convolution (1×1 convolution) is used for inter-channel information fusion. The combination of these two forms a depthwise separable convolution.

[0049] Figure 3 This example demonstrates a comparison of parameter count between standard convolution and depthwise separable convolution. Taking a 256×256 pixel, 4-channel input image as an example, to expand the number of channels to 4 while maintaining the same resolution, standard convolution requires four 3×3×3 kernels, resulting in a total of 3×3×3×4 = 108 parameters. However, depthwise separable convolution first performs a channel-wise 3×3 depthwise convolution (3×3×1×3), followed by four pointwise 1×1×3 convolutions (1×1×3×4), resulting in a total of 27 + 12 = 39 parameters, only about 36% of the standard convolution. Clearly, depthwise separable convolution has a significant advantage over standard convolution in reducing parameter count and computational overhead.

[0050] The MobileViT Block module in this application combines the advantages of Convolutional Neural Networks (CNNs) in local feature perception with the strengths of Transformers in global modeling, making it key to achieving high-precision semantic segmentation tasks.

[0051] like Figure 4 As shown, the MobileViT Block module mainly includes three stages: (1) Local representation stage: extracting local spatial structure information from the input features through convolution operation, and preserving fine-grained details;

[0052] (2) Global Representation Stage: The feature map is flattened into a sequence form using the unfold operation and input into the system.

[0053] The Transformer module effectively models long-distance contextual dependencies through a multi-head self-attention mechanism; subsequently, it uses the fold operation to reconstruct the sequence back to the original spatial structure and restore positional information; (3) Feature fusion stage: The residual connection method is used to fuse local and global features, further improving the integrity and semantic richness of feature expression. The MobileViTBlock module can achieve efficient collaborative modeling of local details and global semantics under a lightweight architecture.

[0054] In this embodiment, the MobileViT backbone feature extraction network first processes the input image through convolutional layers and stem layers, then further extracts deep features through runk layers, finally outputting two feature layers:

[0055] (1) low_level_features: shallow features, larger in size, retain more detailed information, with a shape of [batch, 48, 128, 128];

[0056] (2)x: Deep features, smaller in size, richer in semantic information, with shape [batch,96,8,8].

[0057] Atrous Spatial Pyramid Pooling (ASPP) is commonly used for multi-scale feature extraction. Its core idea is to expand the receptive field without reducing the feature map resolution by setting multiple dilated convolutions with different dilation rates in parallel. This allows the model to cover a wider image region, enhances the ability to perceive contextual information, and thus improves the detection and recognition capabilities of multi-scale targets. Figure 5 The comparison results of the receptive fields of standard convolution and dilated convolution are shown, such as... Figure 5 As shown, compared to the standard 3×3 convolution, its receptive field is 3×3, while when the dilation rate is 2, the dilated 3×3 convolution can obtain a receptive field of 5×5.

[0058] Therefore, to further reduce the computational complexity of the model, this application optimizes the traditional ASPP module by replacing dilated convolutions with different dilation rates with depthwise separable atrous convolutions. This approach decomposes the dilated convolution into two stages: depthwise dilated convolution and pointwise convolution. The former performs dilated convolution operations independently on each channel to extract features, while the latter integrates cross-channel information through 1×1 convolutions. Compared to standard dilated convolutions, this structure reduces the number of parameters and computational overhead.

[0059] However, due to the independent processing of channels, depthwise dilated convolutions suffer from insufficient information exchange between channels. Although pointwise convolutions compensate for this issue to some extent, their overall modeling ability is still slightly inferior to standard dilated convolutions. Therefore, to enhance the expressive power of features, this application introduces a CBAM (Convolutional Block Attention Module) in series after depthwise separable dilated convolutions. Figure 6 As shown, CBAM is a lightweight attention mechanism module that guides the model to focus on more discriminative regions in both the channel dimension and the spatial dimension by connecting the two sub-modules of channel attention and spatial attention, thereby improving the discriminative ability and perception effect of feature representation.

[0060] In this embodiment, the ASPP module employs a multi-branch parallel processing strategy, comprising five feature extraction branches. Branch 1 uses a 1×1 convolution to capture local context information, outputting features of [batch, 256, 8, 8]. Branches 2 through 4 use depthwise separable convolutions with different dilation rates to capture information from different receptive fields. For example, branch 2 uses a 3×3 depthwise separable convolution with a dilation rate of 6 and a 1×1 pointwise convolution; branch 3 uses a 3×3 depthwise separable convolution with a dilation rate of 12 and a 1×1 pointwise convolution; and branch 4 uses a 3×3 depthwise separable convolution with a dilation rate of 18 and a 1×1 pointwise convolution. The output of each of branches 1 through 4 is processed by the CBAM module. Branch 5 uses global average pooling to capture global context information and bilinear interpolation upsampling to the original feature size. The ASPP module concatenates the outputs of the five branches along the channel dimension and integrates multi-scale feature information through 1×1 convolution. The output feature map size is [batch, 256, 8, 8], maintaining the same size as the input image.

[0061] To significantly improve the effects of upsampling and feature fusion, enhance the granularity of semantic expression and the accuracy of feature reconstruction, and improve the model's ability to recover detailed information, the embodiments of this application have made the following optimizations to the decoder's upsampling and feature fusion:

[0062] Upsampling is the process of restoring low-resolution feature maps to higher resolution, and it is an important step in semantic segmentation to reconstruct boundary and spatial structure information. Commonly used upsampling methods, such as nearest neighbor interpolation and bilinear interpolation, are simple to implement and have low computational cost, but they are essentially interpolation operations based on fixed rules, lacking the ability to model semantic information, and are prone to problems such as blurred edges and unclear target structure.

[0063] To enhance the expressive power of upsampling, embodiments of this application introduce the DySample module. For example... Figure 7As shown, DySample is a lightweight and efficient dynamic upsampling module that guides sampling points to focus on semantically key regions by adaptively predicting sampling offsets based on input features. This enhances the expression of semantic information while preserving spatial structure, enabling more discriminative upsampling operations and improving reconstruction quality.

[0064] In this embodiment, the DySample module performs the following upsampling operation to gradually restore the image resolution:

[0065] (1) x1 = self.dysample2(x): Upsample the features from 8×8 to 16×16;

[0066] (2) x2 = self.dysample2(x1): Upsample the features from 16×16 to 32×32;

[0067] (3) x3 = self.dysample2(x2): Upsample the features from 32×32 to 64×64;

[0068] (4) Finally, the final result is upsampled from 64×64 to 256×256 using self.dysample4(x).

[0069] The goal of feature fusion is to integrate feature information from different levels or scales to improve the model's ability to express semantics and restore edge details. The DeepLabv3+ decoder directly concatenates shallow and deep features, which to some extent achieves the combination of multi-scale information, but it fails to fully leverage the complementary advantages of the two in terms of spatial details and semantic information.

[0070] To further optimize the fusion effect of multi-scale features, this application introduces the H-RAMi (Hierarchical Residual Attention Mixer Improved) module, the structure of which is as follows: Figure 8 As shown, H-RAMi first performs spatial resolution alignment and channel concatenation on the four scale feature maps of the input, and then performs blending and enhancement through an efficient convolutional module. This module integrates hierarchical residual mechanism and attention mechanism, which can enhance the collaborative expression of local pixel details and global semantics while maintaining computational efficiency, thereby improving the overall performance of semantic segmentation. The feature map list input to the H-RAMi module in this embodiment includes:

[0071] (1) low_level_features: shallow high-resolution features [batch,256,128,128];

[0072] (2)x2: Medium resolution features [batch,256,32,32];

[0073] (3)x1: Higher resolution features [batch,256,16,16];

[0074] (4)x3: Higher resolution features [batch,256,64,64];

[0075] Finally, multi-scale features are fused through an attention mechanism to enhance feature representation ability and output the fused feature map with size [batch, 256, 128, 128].

[0076] Finally, the DySample module performs the following processing:

[0077] self.cls_conv(x): Adjusts the number of channels from 256 to the number of classes using a 1×1 convolution (2).

[0078] self.dysample4(x): The final upsampling operation restores the feature map from 64×64 to the original input size of 256×256.

[0079] Output segmentation result: [batch, 2, 256, 256], where each pixel corresponds to the probability of two classes.

[0080] Based on the alluvial island instantaneous waterline segmentation method integrating MobileViT and dynamic resampling provided in the above embodiments, this application also provides a specific implementation of the alluvial island instantaneous waterline segmentation system. Please refer to the following embodiments:

[0081] The instantaneous waterline segmentation system for alluvial islands provided in this application includes:

[0082] The image acquisition module uses remote sensing images of alluvial island areas acquired and photographed by drones.

[0083] The image processing module is used to preprocess the acquired remote sensing images and then input them into the trained semantic segmentation model for further processing to obtain the segmentation results of the instantaneous waterline in the remote sensing images.

[0084] The waterline output module is used to output the segmentation results of instantaneous waterlines in remote sensing images.

[0085] The semantic segmentation model is built on DeepLabv3+. Its encoder includes the MobileViT structure and the DSC-ASPP module which introduces the DBAM attention mechanism. The decoder includes the DySample upsampling module and the H-RAMi multi-feature fusion module.

[0086] In other embodiments of this application, an intelligent monitoring platform integrating high-precision sensing and real-time processing can be built based on the aforementioned alluvial island instantaneous waterline segmentation system. This platform, by deploying a trained ALISeg model, analyzes and processes the received multi-source remote sensing image data to achieve real-time monitoring and intelligent early warning of alluvial islands. This intelligent monitoring platform can be used for quantitative assessment of erosion and sedimentation processes, emergency monitoring of disasters after extreme weather events (such as storm surges), and long-term evolution research of coastal ecological environment, providing scientific and reliable decision support and technical assurance for coastal zone management and disaster prevention.

[0087] To comprehensively evaluate the performance of the ALISeg model, this application employs a validation strategy combining ablation experiments and comparative experiments, based on a validation set from a standardized dataset. Ablation experiments analyze the independent contributions and synergistic effects of each improved module within the model. Comparative experiments compare ALISeg's performance with mainstream semantic segmentation models such as SegFormer, PSPNet, HRNet, and UNet to verify the overall superiority and generalization ability of the proposed model. Commonly used metrics for evaluation include mean intersection-over-union (mIoU) and mean pixel accuracy (mPA) to measure the model's segmentation accuracy, and the number of model parameters (params) and the number of floating-point operations (FLOPs) to measure the model's complexity. Figure 9 This paper presents a typical example of instantaneous waterline segmentation of alluvial islands from UAV remote sensing imagery. Table 1 below shows basic information about some alluvial islands from UAV remote sensing imagery.

[0088] Table 1

[0089]

[0090] The hardware and software configuration used in this experiment is as follows: Intel(R) Core(TM) i5-14600 CPU processor and NVIDIA GeForce RTX 5070 graphics card. The operating system is 64-bit Windows 11. For the deep learning task, the PyTorch framework was used, and the Matplotlib library was used for visualization. The model parameters were set as follows: stochastic gradient descent (SGD) with a momentum of 0.9 was selected as the optimization algorithm. The initial learning rate was set to 0.007, the batch size was configured to 4, and the training was performed for 200 epochs. The specific experimental process and results are as follows:

[0091] (1) Ablation experiment:

[0092] An ablation experiment was designed using the controlled variable method. By progressively introducing each improved module, the contribution of each part of the model was systematically evaluated. First, for the backbone network, MobileViT was used to replace the traditional Xception backbone, achieving significant weight reduction and performance improvement. Specifically, the number of model parameters was reduced by approximately 50.41M, the computational complexity was reduced by approximately 29 GFLOPs, while mIoU and mPA were improved by 5.66% and 2.83%, respectively.

[0093] Secondly, regarding module optimization, the ASPP module was replaced with a depthwise separable dilated convolutional structure and a CBAM attention mechanism was introduced in series, effectively improving feature representation capabilities. This improvement reduced the number of model parameters by 0.58M, with only minor fluctuations in segmentation accuracy, indicating that performance was not sacrificed while maintaining lightweight design.

[0094] Further, the independent and synergistic effects of the DySample and H-RAMi modules were evaluated for the decoder. Introducing the DySample dynamic upsampling module improved mIoU and mPA by 0.50% and 0.25%, respectively, while maintaining a relatively stable model complexity. Adding the H-RAMi module improved mIoU by 0.70% and mPA by 0.47%, while significantly reducing computation by approximately 9.2 GFLOPs. When used synergistically, both modules produced a positive gain effect, improving mIoU and mPA by 1.22% and 0.69%, respectively, while still maintaining low model complexity.

[0095] Ultimately, the proposed ALISeg model achieved optimal segmentation accuracy (92.62% mIoU, 96.12% mPA) and low model complexity (3.90M parameters, 3.40 GFLOPs computation) on the test set. Experimental results show that the lightweight improvement strategy of this application has good performance in the task of extracting the instantaneous waterline of complex alluvial islands. The ablation experiment results are shown in Table 2 below.

[0096] Table 2

[0097]

[0098] (2) Comparative Experiment

[0099] To further verify the performance of the proposed ALISeg model, under the premise of strictly ensuring the consistency of experimental conditions, this application adopts a comparative experiment method to comprehensively compare ALISeg with several mainstream semantic segmentation models, including SegFormer, PSPNet, HRNet, and UNet. The results are shown in Table 3 below:

[0100] Table 3

[0101]

[0102] As shown in Table 3 above, in terms of key segmentation metrics, ALISeg exhibits better performance in both mIoU (92.62%) and mPA (96.12%), compared to SegFormer, PSPNet, HRNet, and...

[0103] UNet improved mIoU by 3.81%, 3.45%, 2.88%, and 2.58%, respectively, while mPA surpassed by 2.05%, 1.76%, 1.42%, and 1.45%, respectively. In terms of model complexity, ALISeg has only [a smaller number of parameters].

[0104] It achieves 15.67% of the computational cost of UNet, while its computational complexity is comparable to SegFormer. (And a lightweight model)

[0105] Compared to PSPNet, ALISeg achieves a 3.81% improvement in mIoU at the cost of only 1.49M additional parameters, fully demonstrating its ability to balance accuracy and efficiency.

[0106] Figure 10 The segmentation visualization results of the alluvial island image further validated the practical advantages of ALISeg. By comparing the recognition performance of various models on land areas (red marked areas), the ALISeg model can accurately depict the waterline contours in complex terrains and demonstrates superior detail preservation capabilities in instantaneous waterline extraction tasks.

[0107] Combined with Table 3 Figure 10 The results show that the ALISeg model has outstanding advantages in balancing computational resource utilization and segmentation accuracy, demonstrating its good adaptability in resource-constrained environments such as UAVs and mobile devices, promoting real-time and efficient processing of remote sensing images, and showing broad and far-reaching potential for engineering applications.

[0108] In summary, the instantaneous waterline segmentation method for alluvial islands proposed in this application, which integrates MobileViT and dynamic resampling, effectively improves the accuracy and efficiency of instantaneous waterline extraction for alluvial islands. Experimental results show that, firstly, by replacing Xception with MobileViT as the backbone network in the ALISeg model of this application, the model complexity is significantly reduced while the segmentation accuracy is improved, achieving the dual goals of lightweight design and practicality. Secondly, the proposed DSC-ASPP module combines depthwise separable dilated convolution and CBAM attention mechanism, maintaining feature representation capabilities while maintaining lightweight design, effectively mitigating the accuracy loss caused by lightweight design. Simultaneously, the decoder section further optimizes edge detail recovery and multi-scale feature fusion capabilities by introducing the synergistic effect of DySample and H-RAMi, resulting in excellent performance in waterline contour extraction in blurred transition zones. Compared with mainstream models such as SegFormer, PSPNet, HRNet and UNet, the ALISeg model has shown great advantages in segmentation accuracy and model complexity, demonstrating a high-efficiency balance and is suitable for resource-constrained scenarios such as real-time monitoring by UAVs, providing reliable technical support for dynamic monitoring of alluvial islands.

[0109] Furthermore, in another embodiment of this application, the ALISeg model structure can be adjusted based on the input fused satellite remote sensing, SAR radar and other multi-source data to construct a multi-dimensional information fusion framework, making it adaptable to the multi-information-dimensional alluvial island monitoring scenario and enhancing the comprehensiveness of monitoring.

[0110] In other embodiments of this application, long-term series analysis methods may be introduced to construct a dynamic evolution model of the instantaneous waterline of alluvial islands, thereby revealing the long-term patterns of erosion and deposition of alluvial islands.

[0111] Based on any of the above embodiments, another embodiment of the present invention provides an electronic device, which may include: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor may invoke logical instructions in the memory to execute the above method.

[0112] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] On the other hand, embodiments of the present invention also provide a storage medium storing a plurality of instructions adapted for loading by a processor to execute the alluvial island instantaneous waterline segmentation method that integrates MobileViT and dynamic resampling as provided in the above embodiments.

[0114] On the other hand, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0117] In summary, although the present invention has been disclosed above with reference to preferred embodiments, the above preferred embodiments are not intended to limit the present invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the scope defined in the claims.

Claims

1. An alluvial island instantaneous waterline segmentation method fusing MobileViT with dynamic resampling, characterized in that, The application relates to an instant waterline extraction method based on a semantic segmentation model. The application comprises the following steps: acquiring a remote sensing image of an alluvial island region shot by a UAV and performing pretreatment; inputting the pretreated remote sensing image into a trained semantic segmentation model for processing to acquire a segmentation result of an instant waterline in the remote sensing image; 2. The method of claim 1, wherein, wherein the semantic segmentation model is constructed based on DeepLabv3+, and the encoder of the model comprises a MobileViT structure and a DSC-ASPP module.

3. The method of claim 1, wherein, The decoder of the semantic segmentation model comprises a DySample up-sampling module and an H-RAMi feature fusion module. The MobileViT structure comprises a shallow convolution, an MV2 module and a MobileViT Block module; wherein the MV2 module adopts an inverted residual structure, realizes down-sampling and feature transformation through a depth separable convolution and a jump connection, is used for constructing a lightweight network basis and enhancing the expression capability of features, and adopts an inverted residual structure.

4. The method of claim 3, wherein, The MobileViT Block module fuses a convolutional neural network and a Transformer, is used for capturing global semantic features and fusing the global semantic features with local features, and realizes high-precision semantic segmentation. The MobileViT Block module sequentially executes three stages of local representation, global representation and feature fusion, so as to realize the fusion of local details and global features, and the specific process is as follows: in the local representation stage, local spatial structure information in input features is extracted through a convolution operation, and fine-grained details are reserved; in the global representation stage, an unfold operation is adopted to flatten a feature map into a sequence form, the sequence form is input into a Transformer module, long-distance context dependency relationships are effectively modeled through a multi-head self-attention mechanism, and then a fold operation is adopted to reconstruct the sequence back to an original spatial structure, and position information is recovered; 5. The method of claim 1, wherein, in the feature fusion stage, local and global features are fused through a residual connection mode, and the integrity and semantic richness of feature expression are further improved. The DSC-ASPP module is used for extracting and fusing multi-scale context information around the waterline of the alluvial island in the remote sensing image.

6. The method of claim 5, wherein, The DSC-ASPP module extracts multi-scale features through parallel multi-branch depth separable hole convolutions, the depth separable hole convolutions are composed of a depth hole convolution and a point convolution in series, the depth hole convolution performs a hole convolution on each input channel to extract features, and the point convolution is used for realizing cross-channel information integration.

7. The method of claim 2, wherein the MobileViT is fused with the dynamic resampling of the alluvial island instantaneous waterline segmentation. The DSC-ASPP module further introduces a channel attention module and a spatial attention module in a CBAM attention mechanism in series after each depth separable hole convolution branch, so that the extracted multi-scale feature maps are weighted and reconstructed in the channel and the space. The DySample up-sampling module adaptively predicts a sampling offset of input features, guides sampling points to focus on semantic key areas, and improves the up-sampling precision of the waterline edge.

8. The method of claim 2, wherein the MobileViT is fused with the dynamic resampling of the alluvial island instantaneous waterline segmentation. The H-RAMi feature fusion module is used for spatial resolution alignment and channel splicing of input multi-scale features, and the spliced features are mixed and enhanced through an efficient convolution module fused with a hierarchical residual mechanism and an attention mechanism, so as to strengthen the collaborative expression of local details and global semantics. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor implements the steps of the method of any one of claims 1 to 8 when executing the computer program.

10. A storage medium, characterized by The storage medium has a plurality of instructions stored therein, and the instructions are adapted to be loaded by the processor to execute the steps of the method of any one of claims 1 to 8.

Images