Double-branch water body small target image segmentation method based on feature efficient interaction

Abstract

The invention belongs to the technical field of remote sensing image segmentation, and particularly relates to a double-branch water body small target image segmentation method based on feature efficient interaction, which comprises the following steps: preparing a data set, constructing a network model, training the network model, selecting a proper loss function and evaluation index, and determining a segmentation model. According to the multi-scale detail feature interactive aggregation encoder, efficient fusion of multi-scale features and effective supplement of implicit relative position encoding information are achieved; the long-distance feature efficient capture encoder increases the edge segmentation effect on the small target image of the water body by processing features along a specific space direction; the cavity space convolution pyramid module based on the large convolution kernel is used for improving the ability of the model to learn a large-scale effective receptive field; the non-significant feature extraction module is used for filling up the deficiency of non-significant features. The whole network adopts a learning strategy of parallel connection of global and local features and series connection from a large scale to a small scale, and effectively extracts different scale information of the water body image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image segmentation technology, specifically to a method for segmenting small targets in a two-branch water body image based on efficient feature interaction. Background Technology

[0002] Water body segmentation aims to accurately identify the extent of water bodies in remote sensing images, which is crucial for flood monitoring, water resource management, and ecological environment assessment. However, water bodies have complex and varied morphologies, and the edges of shorelines and water-land boundaries are often blurred or fragmented. These factors combined lead to existing segmentation methods facing challenges in water body applications, including missed or misclassified small targets, insufficient edge localization accuracy, weak anti-interference capabilities in complex scenes, high spectral confusion and misjudgment rates, and difficulty in effectively utilizing the unique temporal and contextual information of water bodies, such as river continuity and lake morphology. With the development of deep learning, more and more universal image segmentation methods have been proposed and widely applied in the field of small target segmentation in water bodies. However, because remote sensing images show small individual objects and a high density of objects, the accuracy requirements for remote sensing segmentation methods are higher than for ordinary images. Therefore, current remote sensing segmentation methods still suffer from problems such as low edge target segmentation accuracy, insufficient utilization of contextual information, class imbalance, slow training speed, and missing edges of segmented targets.

[0003] Chinese patent publication number "CN118608786A" is titled "A Water-Res-Swin Remote Sensing Image Water Body Segmentation Method Based on Improved Attention." This method constructs a Water-Res-Swin network model composed of dual feature extractor fusion and an optimized attention mechanism. The core of the model uses a ResNet50 network as the backbone feature extractor and a Swing Transformer network as the auxiliary feature extractor to extract multi-scale feature maps. A feature map fusion mechanism is used to fuse the feature maps of the same dimension output by the two extractors to enhance the model's feature representation capability. Simultaneously, an optimized surrogate attention (OAG) module is embedded in the network to reduce computational complexity and generate enhanced feature representations. Finally, by decoding and post-processing the fused features, errors or uncertainties in the initial segmentation results are corrected to obtain the final water body segmentation result.

[0004] The auxiliary encoder used in this method, acting as a Transformer, relies heavily on a global self-attention mechanism. While it can capture long-range dependencies, its ability to model local features is weak, especially high-frequency details (such as edges). However, in image segmentation tasks, edge information requires the model to be sensitive to local pixel changes. Due to the limited hierarchical modeling ability of the detail preservation module, the insufficient context awareness of the spatial feature enhancement module, and the simplistic cross-dimensional interaction mechanism of the multi-scale information fusion module, feature extraction is insufficient, resulting in inadequate segmented edge details and low efficiency. Furthermore, its hierarchical downsampling, attention bias, and same-scale fusion mechanism lead to insufficient detail preservation for small water bodies. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a dual-branch water body small target image segmentation method based on efficient feature interaction, which solves the problems mentioned in the background technology.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention specifically adopts the following technical solution:

[0009] A method for segmenting small targets in a two-branch water body image based on efficient feature interaction includes the following steps:

[0010] S1. Dataset Preparation: Acquire image data for small target scenes in water bodies. After classifying and filtering to remove invalid images, preprocess the images to adjust their size to meet the input requirements of the network model. Label the dataset and finally divide it into training and test sets.

[0011] S2, Constructing the network model: The dual-branch water body small target image segmentation network based on large kernel convolution consists of dual encoders, BASP (Big Kernel-based Atrous Spatial Pyramid) module, non-salient feature extraction module, and decoder. The dual encoders consist of a multi-scale detail feature interaction aggregation encoder and a long-distance feature efficient capture encoder.

[0012] S3, Training the network model: Train the segmentation network model for small target images in water bodies. Input the preprocessed dataset in S1 into the network model built in S2 for training and obtain the training weights.

[0013] S4. Select a suitable loss function and evaluation metric: Select a suitable loss function to minimize the loss between the weights of the output image and the true label values ​​of manual segmentation. Set a training loss threshold and continuously iterate to optimize the model until the value of the loss function reaches the set threshold range or the number of training iterations reaches the set threshold. Then the model parameters can be considered to have been pre-trained and saved.

[0014] S5. Determine the segmentation model: After the network is trained, the network model parameters need to be fixed and the final segmentation model needs to be determined. For example, when segmenting small water objects, the trained network model can be directly input into the segmentation model to obtain the final two-branch multi-scale water object segmentation image.

[0015] Furthermore, the multi-scale detail feature interactive aggregation encoder in S2 consists of a detail feature extraction module, an edge multi-dimensional information interaction module, and a global feature fusion module;

[0016] The detailed feature extraction module is used to extract feature information from shallow to deep progressively.

[0017] The edge multidimensional information interaction module processes features along a specific spatial direction to enhance the segmentation effect of the target edge;

[0018] The global feature fusion module is used to further learn and compress feature information.

[0019] Furthermore, the long-range feature efficient capture encoder in S2 consists of a detail feature extraction module and a long-range feature extraction module;

[0020] The detailed feature extraction module is used to extract feature information from shallow to deep progressively.

[0021] The long-range feature extraction module comprises five parts: a normalization layer, a linear layer, a depthwise separable convolutional layer, a two-dimensional selective scanning module, and a spatial cooperative perception module. After feature input, the data is normalized by the normalization layer, followed by a linear transformation to a higher dimension, enabling the model to capture higher-level features. Then, a depthwise separable convolutional layer performs efficient local feature extraction. The further extracted features are then input into the two-dimensional selective scanning module for further learning and capturing long-term dependencies in the sequence data. Next, a weight fusion is performed, fusing the data from the spatial cooperative perception module with the normalized data from the two-dimensional selective scanning module. The spatial cooperative perception module applies soft pooling to the input features to generate more multi-scale features, which are then concatenated through convolution to aggregate local and global relationships, supplementing implicitly encoded feature information. Finally, a residual connection is performed, adding the fused data to the input features through a linear layer to obtain the output of the long-range feature extraction module.

[0022] Furthermore, in S2, the BASP module is used to improve the model's ability to learn large-scale effective receptive fields;

[0023] Furthermore, in S2, the non-salient feature extraction module is used to fill in the missing non-salient features, thereby further improving the segmentation accuracy;

[0024] Furthermore, the decoder in S2 consists of a subpixel upsampling module, an attention mechanism module, a detail feature extraction module, and a classification convolutional block;

[0025] The subpixel upsampling module improves the resolution of the feature map and enhances the ability to reconstruct details by optimizing the feature distribution through channel recombination and spatial dimension optimization.

[0026] The attention mechanism module focuses on key regions by dynamically adjusting feature weights, thereby enhancing the model's ability to perceive and utilize important information.

[0027] The detailed feature extraction module is used to gradually extract feature information from shallow to deep.

[0028] The classification convolutional block is used to output a binary classification image, resulting in a final high-precision segmentation image.

[0029] Furthermore, in step S4, a suitable loss function and evaluation metric are selected. The DiceLoss loss function used is a loss function that measures the intersection-union ratio (IU). It can effectively alleviate the sample imbalance problem in binary classification tasks in the field of remote sensing segmentation, thereby improving the accuracy of the model output. Pixel accuracy, average IU, recall, and frames per second are selected as metrics to evaluate the performance of the segmentation model. Pixel accuracy refers to the proportion of correctly predicted pixels out of the total number of pixels. Average IU is the ratio of the intersection and union of two sets, which are the true values ​​and the predicted values. Recall is the proportion of pixels correctly predicted as targets out of the total number of true target pixels. Frames per second refers to the number of image frames that the model can process per second.

[0030] (III) Beneficial Effects

[0031] Compared with existing technologies, this invention provides a method for segmenting small targets in a two-branch water body image based on efficient feature interaction, which has the following advantages:

[0032] 1. This invention designs a BASP module, which utilizes a standard large convolutional kernel performing depthwise separable convolutions combined with asymmetric small convolutional kernels possessing multi-scale receptive fields to achieve efficient acquisition of powerful semantic information. Through structural reparameterization, two network models are constructed that are completely equivalent in performance for training and inference, but with different structures. Compared with existing ASPP modules, this method further improves the ability to extract detailed features from small target images of water bodies, increases the actual receptive field, and solves the problem of slow feature extraction speed caused by multi-branch parallelism.

[0033] 2. This invention designs a long-distance feature extraction module. This module assists the two-dimensional selective scanning process by designing a spatial collaborative perception module, thereby achieving efficient fusion of multi-scale features and accurate modeling that has been relied upon for a long time; aggregating local and global relationships to improve the detection rate of small targets in water bodies; and effectively supplementing implicit relative position encoding information, thereby improving the performance and adaptability of the model in complex tasks.

[0034] 3. This invention utilizes a non-salient feature extraction module to mitigate the loss of non-salient feature information caused by downsampling; the edge multi-dimensional information interaction module enhances the segmentation effect of small water target images by processing features along specific spatial directions. Compared with existing edge multi-dimensional information interaction modules and global feature fusion modules, this method further improves the model's multi-scale feature learning capability, avoids the loss of detailed information in small water target images, and significantly reduces computational load, achieving high model efficiency. Attached Figure Description

[0035] Figure 1The flowchart shows a two-branch water body small target image segmentation method based on efficient feature interaction.

[0036] Figure 2 This is a schematic diagram of the network structure of a two-branch water body small target image segmentation method based on efficient feature interaction;

[0037] Figure 3 This is a schematic diagram of the detailed feature extraction module described in this invention;

[0038] Figure 4 This is a schematic diagram of the global feature fusion module described in this invention;

[0039] Figure 5 This is a schematic diagram of the edge multidimensional information interaction module described in this invention;

[0040] Figure 6 This is a schematic diagram of the long-distance feature extraction module described in this invention;

[0041] Figure 7 This is a schematic diagram comparing the structural angles and parameter angles of the dilated spatial convolution pyramid module based on large convolution kernels described in this invention.

[0042] Figure 8 This is a schematic diagram of the non-salient feature extraction module of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example 1

[0045] like Figure 1 As shown in the figure, this embodiment of the invention provides a method for segmenting small targets in a two-branch water body image based on efficient feature interaction, which specifically includes the following steps:

[0046] S1. Dataset Preparation: The selected dataset is the open-source water body small object image segmentation dataset from the Kaggle data competition platform. This dataset consists of a set of water body small object images taken by the Sentinel-2 satellite, containing 2841 images. After classifying and filtering to remove invalid images, data augmentation techniques were used to randomly level and flip the images, increasing the original dataset to 5000 images, thereby increasing the diversity of the training data. Preprocessing yielded image patches of size 512×512×3, of which 3500 were used as the training set, 1000 images as the validation set, and 500 images as the test set.

[0047] S2, Constructing the Network Model: The dual-branch water body small target image segmentation network structure based on large kernel convolution constructed in this invention is as follows: Figure 2 As shown, the entire dual-branch water body small target image segmentation network based on large kernel convolution consists of dual encoders, a BASP module, a non-salient feature extraction module, and a decoder. The multi-scale detail feature interaction aggregation encoder and the long-range feature efficient capture encoder are implemented in parallel. The output of each stage of the dual encoder is fused to the decoder after being weighted according to the corresponding features. The multi-scale detail feature interaction aggregation encoder includes four detail feature extraction modules, four global feature fusion modules, and four edge multi-dimensional information interaction modules. The input features are progressively extracted from shallow to deep features by the detail feature extraction module. The four edge multi-dimensional information interaction modules improve the segmentation effect of small water body edges by processing features along specific spatial directions. The four global feature fusion modules are used to further learn and compress feature information.

[0048] Combination Figure 3 Further description of the detail feature extraction module in S2: The 1×1 convolution first increases the number of channels in the input image, amplifying its features and preparing for better feature learning in subsequent images. Parallel deep-separable 3×3, 1×3, and 3×1 convolutions, along with residual connections, are used to extract multi-scale, high-dimensional features from the image. Then, a single 1×1 convolution restores the output channel count back to the input channel count. Finally, a channel rearrangement process promotes the interaction of feature information between different convolution kernels in the deep-separable convolution. For example... Figure 4 As shown, the global feature fusion module introduces a branch consisting of depthwise separable asymmetric convolutions and 1×1 convolutions to perform channel splicing based on the detailed feature extraction module. The asymmetric convolutions enhance the skeleton of the square convolutions, making the model more robust to distortions such as data rotation, thereby improving model performance. Finally, the 1×1 convolutions control the branch to be consistent with the dimension of the backbone, enabling information collaboration between different dimensions of the network.

[0049] The 1×1 convolutions in the output of the detail feature extraction module and the global feature fusion module use a linear activation function, while the 1×1 convolutions in the input use an h-swish activation function to avoid the loss of low-dimensional information.

[0050] like Figure 5 As shown, the edge multidimensional information interaction module first embeds positional and morphological information into each channel of the input feature along the horizontal and vertical directions using soft pooling, resulting in a pair of one-dimensional feature codes with two-dimensional spatial information. Then, the one-dimensional feature codes with spatial information in both directions are concatenated and sub-feature decomposition is performed along the channel dimension. These sub-features are then convolved with one-dimensional depthwise convolutional kernels of different sizes to capture multi-level contextual information. Finally, the independent tensors decoupled from the results in the two dimensions are used to interact with the original input image, aggregating multi-level contextual information to obtain a feature map with target morphological information. The edge multidimensional information interaction module will be described in detail below:

[0051] First, the input features are processed by global soft pooling kernels in two spatial dimensions, along the horizontal and vertical directions, respectively, to optimize the input feature map. Each channel embeds axial morphological information, resulting in a pair of one-dimensional feature codes with two-dimensional spatial information; the information embedding along the horizontal direction is represented as follows:

[0052]

[0053] in, This represents the output encoded at height h and channel c; the information embedding along the vertical direction is represented as:

[0054]

[0055] in, This represents the output encoded at height w and channel c.

[0056] Next, the one-dimensional encoded features obtained along the horizontal and vertical directions are concatenated along the channel direction to obtain:

[0057] f = [z h ,z w ]

[0058] Where [·,·] represents a connection operation along a spatial dimension. It is an intermediate feature map that encodes morphological information in the horizontal and vertical directions; then, f is decomposed into K independent sub-features of the same size along the channel dimension, and the number of channels corresponding to each sub-feature is... The process of decomposing into sub-features is as follows:

[0059]

[0060] Among them, f i Let i represent the i-th sub-feature, where i takes values ​​in the range [1, K].

[0061] Next, we use one-dimensional deep convolutional kernels of different sizes to perform multi-scale feature extraction on the K sub-features to capture multi-level contextual information. The process is as follows:

[0062]

[0063] Among them, g i This represents a feature that has contextual information at the i-th level. Let k represent a one-dimensional depthwise convolution kernel applied to the i-th sub-feature. i This represents the size of the convolution kernel, and k i = i × 2 + 1;

[0064] To aggregate multi-level context features without introducing batch processing statistical noise, the extracted features are first concatenated along the channel dimension to obtain:

[0065] g = [g1, g2, ..., g K ]

[0066] Next, g is decoupled along the spatial dimension into two independent tensors. and Use group normalization layer GN to g h and g w Group normalization is performed separately to reduce semantic interference between sub-features; then, the Sigmoid function is used to normalize g. h and g w By applying nonlinear excitation, we obtain:

[0067]

[0068] Where σ(·) represents the Sigmoid function; finally, and input feature map Element-wise multiplication yields a feature map Y containing target morphology information; the calculation method for the c-th channel and (i,j)-th spatial coordinate features of feature map Y is as follows:

[0069]

[0070] The long-range feature efficient capture encoder includes a detail feature extraction module and a long-range feature extraction module, such as... Figure 6As shown, the long-range feature extraction module comprises five parts: a normalization layer, a linear layer, a depthwise separable convolutional layer, a two-dimensional selective scanning module, and a spatial cooperative perception module. After feature input, the data is normalized by the normalization layer, then linearly transformed to a higher dimension by the linear layer, enabling the model to capture higher-level features. Next, a depthwise separable convolutional layer performs efficient local feature extraction. Finally, the further extracted features are input into the two-dimensional selective scanning module to further learn and capture long-term dependencies in the sequence data.

[0071] The spatial collaborative sensing module decomposes f along the channel dimension into N independent sub-features of the same size, and the number of channels corresponding to each sub-feature is... The process of decomposing into sub-features is as follows:

[0072]

[0073] Among them, f i Let i represent the i-th sub-feature, where i takes values ​​in the range [1, N].

[0074] Next, soft pooling at N different scales is performed on the N sub-features to dynamically allocate spatial information of different dimensions. The process is as follows:

[0075]

[0076] Among them, h i This represents a feature that has been soft-pooled with context information from the i-th level. d represents the soft pooling process applied to the i-th sub-feature. i This represents the size of the convolution kernel, and d i = (i-1)×2;

[0077] Next, feature extraction at different scales is performed on the N soft-pooled features to capture multi-level contextual information. The process is as follows:

[0078]

[0079] Among them, g i This represents a feature that has contextual information at the i-th level. Let k represent a one-dimensional depthwise convolution kernel applied to the i-th sub-feature. i This represents the size of the convolution kernel, and k i = i × 2 + 1;

[0080] Then, the features g of the context information in each level i Upsampling at different magnifications is performed to restore each layer to the same scale information. The process is as follows:

[0081]

[0082] Where, m i This indicates that the features are obtained after upsampling at level i. R represents the process applied to the i-th sub-feature. i This represents the upsampling factor, and r i = (i-1)×2;

[0083] To aggregate multi-level context features without introducing batch processing statistical noise, context features from different levels are concatenated along the channel dimension and spatially modulated using a 1×1 method, resulting in:

[0084] m = [m1, m2, ..., m N ]

[0085] R = Conv1d 1×1 (m)

[0086] Secondly, a group normalization layer (GN) is used to normalize R to reduce semantic interference between sub-features. Then, the sigmoid function is used to apply nonlinear activation to the normalized R, and the result is compared with the input feature map. Element-wise multiplication yields a feature map Y containing target morphology information; the calculation method for the c-th channel and (i,j)-th spatial coordinate features of feature map Y is as follows:

[0087] y c (i,j)=x c (i,j)×σ(GN(R c (i,j)))

[0088] Where σ(·) represents the Sigmoid function.

[0089] Next, a weighted fusion is performed, which merges the data from the spatial collaborative perception module with the data from the two-dimensional selective scanning module and after normalization. Finally, a residual connection is performed, and the fused data is added to the input features through a linear layer to obtain the output of the long-distance feature extraction module.

[0090] Combination Figure 7Further description of the BASP module in S2 reveals that it consists of a 1×1 standard convolutional block with depthwise separability, 3×3 asymmetric convolutions with dilation of 6, 3×3 asymmetric convolutions with dilation of 12, 3×3 asymmetric convolutions with dilation of 18, and a 37×37 depthwise separable convolution. This allows for the acquisition of the largest possible receptive field and multi-scale information extraction capability with only a very low number of parameters. Finally, utilizing the concept of structure reparameterization, the entire convolution is equivalently transformed into a large kernel convolution during network inference. This is because a small convolutional block + dilated convolution is equivalent to a large convolutional block + non-dilated convolution.

[0091] From a mathematical perspective, the specific process of structure reparameterization is as follows: Assume... The parameters represent the convolution kernel of the BASP module, where C is the number of input channels, C is the number of output channels, and k is the kernel size. The model input and output are respectively... and N represents the number of samples, W and H represent the width and height of the input feature map, and W′ and H′ represent the width and height of the output feature map. The convolution operation can be represented as:

[0092] O = I*K + B(b)

[0093] in, Let B represent the bias, and B be the broadcast function that maps the bias b to N×C²×H′×W′; * represents the convolution operation. For a convolutional layer connected to a BN layer, assuming the BN layer's parameters include the mean μ, variance σ, scale factor γ, and bias β, then the output O′ of the i-th channel of the Conv-BN structure... :,i,:,: Represented as:

[0094]

[0095] Assume K′ and b′ i Let the kernel parameters and bias parameters be equivalent to those in the above formula. Then:

[0096]

[0097] The final output of the convolutional layer after conversion is:

[0098] O :,i,:,: =(I*K′) :,i,:,: +b i i∈[1,C2]

[0099] Through the above transformation, dilated convolutions of various small convolutional blocks can be converted into non-dilated convolutions of large convolutional blocks. For parts where the convolutional kernels between branches are inconsistent, zeros are padded to form a 37×37 convolution.

[0100] Based on the additive property of convolution, convolution kernels with consistent shapes can satisfy additivity. Adding four padded convolutions to the original 37×37 convolution is equivalent to creating a new 37×37 depthwise separable convolution, thus achieving the largest possible receptive field and multi-scale information extraction capability with only a very low number of parameters. The structure of the training and inference networks is independently designed. High-precision multi-branch networks are used to learn weights during training, while low-latency single-branch networks are used during inference. Then, structural parameterization is used to transfer the weights of the multi-branch networks to the single-branch networks, thereby achieving high network efficiency.

[0101] Combination Figure 8 The non-salient feature extraction module in S2 is further described. Specifically, the salient feature information obtained by the original image and the original image are subjected to three parallel max pooling and average pooling operations and transposed multiple times. The non-salient feature information obtained from the three operations is then weighted and fused, and added to the original image, thereby fusing multi-scale non-salient features into the small target image of the water body.

[0102] The decoding path includes four decoders, consisting of four detail feature extraction modules, four attention modules, four subpixel convolutional modules, and one classification convolutional block. The composition of the detail feature extraction modules is the same as that of the detail feature extraction modules in the encoding path. The four decoders in the decoding path are added to the weighted fusion outputs of the four parts of the dual encoder. The last layer of the decoding path is a classification convolutional block with a size of 1×1 and a channel size of 2, which is used to output a binary classification image to obtain the final high-precision segmentation image.

[0103] S3, Training the network model: Train the water body small target image segmentation network model. Input the water body small target image dataset to be segmented in S1 into the network model built in S2 for training, obtain training weights, and further segment the water body small target images to obtain the segmentation results.

[0104] S4. Select a suitable loss function and evaluation metrics: Select a suitable loss function that minimizes the loss between the weights of the output image and the true label values ​​of manual segmentation. Set a training loss threshold and iteratively optimize the model until the value of the loss function reaches the set threshold range or the number of training iterations reaches the set threshold. The model parameters can then be considered to have been pre-trained and saved. Select pixel accuracy, average intersection-over-union ratio, recall, and frames per second as metrics to evaluate the performance of the segmentation model.

[0105] The loss function in S4 is designed to measure the similarity between the network's predicted values ​​and the labels. A better loss function leads to better network performance. DiceLoss is used as the loss function for this network. It is a loss function that measures the intersection-union ratio (IU). In binary classification tasks involving small water object segmentation, it effectively alleviates the imbalance problem, thereby improving the model's output accuracy. The formula for calculating the DiceLoss loss function is shown below:

[0106]

[0107] Among them, y i and These are the label value and predicted value for image i, respectively. N is the total number of pixels, which is the number of pixels in a single image multiplied by the batch size.

[0108] In S4, pixel accuracy, mean intersection-over-union ratio (MIR), recall, and frames per second (fps) are selected as metrics to evaluate the performance of the segmentation model. Pixel accuracy refers to the proportion of correctly predicted pixels out of the total number of pixels. MIR is the ratio of the intersection to the union of two sets, which are the true values ​​and the predicted values. Recall is the proportion of correctly predicted pixels out of the total number of true target pixels. fps refers to the number of image frames the model can process per second. The formulas for calculating pixel accuracy, MIR, and recall are shown below:

[0109]

[0110]

[0111] Where k represents the number of pixel classes, P ii P represents the correct number of pixels. ij and P ji This indicates the number of false positives and false negatives in a pixel.

[0112] Optimize the network model by setting the number of training iterations to 200, using the Adam optimizer, a learning rate of 0.001, and a learning rate decay of 0.1 every 10 training iterations. Set the loss threshold to 0.0002 and iterate through the training iterations. When the training loss is infinitely close to the loss threshold, the network is considered to have been basically trained.

[0113] S5. Determine the segmentation model: After the network is trained, the network model parameters need to be fixed and the final segmentation model needs to be determined. For example, when segmenting small water objects, the trained network model can be directly input into the segmentation model to obtain the final two-branch multi-scale water object segmentation image.

[0114] The implementation of convolution operations, activation functions, batch normalization, and multilayer perceptrons are algorithms well-known to those skilled in the art, and the specific processes and methods can be found in relevant textbooks or technical documents.

[0115] This invention designs a bi-branch water body small target image segmentation method based on efficient feature interaction and applies it to remote sensing image segmentation tasks. It effectively solves the segmentation problem caused by category confusion in remote sensing images due to large scale variations, similarity between different classes, and large differences within the same class, thereby improving the accuracy of image segmentation.

[0116] Table 1 shows a comparison of relevant indicators between existing technologies and the method proposed in this invention.

[0117] Table 1 Comparison of relevant indicators between existing technologies and the method proposed in this invention.

[0118]

[0119]

[0120] As can be seen from the table, the method proposed in this invention has higher average state accuracy, success rate, and kernel accuracy than existing methods, further demonstrating that the method proposed in this invention has better segmentation quality and achieves the expected results.

[0121] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for segmenting small targets in a two-branch water body image based on efficient feature interaction, characterized in that: Includes the following steps: S1, Prepare the dataset: Acquire image data for small target scenes in water bodies, remove invalid images after classification and filtering, preprocess the images to adjust their size to meet the input requirements of the network model, label the dataset, and finally divide it into training set, validation set and test set. S2, Constructing the network model: The water body small target image segmentation network based on large kernel convolution dual branch consists of dual encoders, BASP module, non-salient feature extraction module and decoder. The dual encoders are composed of multi-scale detail feature interaction aggregation encoder and long-distance feature efficient capture encoder. S3, Training the network model: Train the water body small target image segmentation network model. Input the preprocessed dataset in S1 into the network model built in S2 for training and obtain the training weights. S4. Select a suitable loss function and evaluation metric: Select a suitable loss function to minimize the loss between the weights of the output image and the true label values ​​of manual segmentation. Set a training loss threshold and continuously iterate to optimize the model until the value of the loss function reaches the set threshold range or the number of training iterations reaches the set threshold. Then the model parameters can be considered to have been pre-trained and saved. S5. Determine the segmentation model: After the network is trained, the network model parameters need to be fixed and the final segmentation model needs to be determined. For example, when segmenting small water objects, the trained network model can be directly input into the segmentation model to obtain the final two-branch multi-scale water object segmentation image.

2. The method for segmenting small targets in a two-branch water body image based on efficient feature interaction according to claim 1, characterized in that: The multi-scale detail feature interactive aggregation encoder in S2 consists of a detail feature extraction module, an edge multi-dimensional information interaction module, and a global feature fusion module. The detailed feature extraction module is used to extract feature information from shallow to deep progressively. The edge multidimensional information interaction module processes features along a specific spatial direction to enhance the segmentation effect of small target images in water bodies. The global feature fusion module is used to further learn and compress feature information.

3. The method for segmenting small targets in a two-branch water body image based on efficient feature interaction according to claim 1, characterized in that: The long-range feature high-efficiency capture encoder in S2 consists of a detail feature extraction module and a long-range feature extraction module; The detailed feature extraction module is used to extract feature information from shallow to deep progressively. The long-range feature extraction module consists of five parts: a normalization layer, a linear layer, a depthwise separable convolutional layer, a two-dimensional selective scanning module, and a spatial collaborative perception module. It is used to further learn features to capture long-term dependencies in sequence data, aggregate local and global relationships, improve the detection rate of small targets in water bodies, and supplement implicitly encoded feature information.

4. The method for segmenting small targets in a two-branch water body image based on efficient feature interaction according to claim 1, characterized in that: In S2, the BASP module is used to improve the model's ability to learn large-scale effective receptive fields.

5. The method for segmenting small targets in a two-branch water body image based on efficient feature interaction according to claim 1, characterized in that: In step S2, the non-salient feature extraction module is used to fill in the lack of non-salient features in the small target image of the water body, thereby further improving the segmentation accuracy.

6. The method for segmenting small targets in a two-branch water body image based on efficient feature interaction according to claim 1, characterized in that: The decoder in S2 consists of a subpixel upsampling module, an attention mechanism module, a detail feature extraction module, and a classification convolutional block; The subpixel upsampling module improves the resolution of the feature map and enhances the ability to reconstruct details by optimizing the feature distribution through channel recombination and spatial dimension optimization. The attention mechanism module focuses on key water areas by dynamically adjusting feature weights, thereby enhancing the model's ability to perceive and utilize important information. The detailed feature extraction module is used to extract feature information from shallow to deep progressively. The classification convolutional block is used to output a binary classification image, resulting in a final high-precision segmentation image.

7. The method for segmenting small targets in a two-branch water body image based on efficient feature interaction according to claim 1, characterized in that: The loss function in S4 is the DiceLoss loss function. The specific evaluation metrics are pixel accuracy, mean intersection-union ratio (MIRR), recall, and frames per second (fps) as indicators to evaluate the performance of the segmentation model. Pixel accuracy refers to the proportion of correctly predicted pixels out of the total number of pixels. MIRR is the ratio of the intersection and union of two sets, which are the true values ​​and the predicted values. Recall is the proportion of the number of pixels correctly predicted as the target out of the total number of true target pixels. fps refers to the number of image frames that the model can process per second.

Images