River and lake floating object semantic segmentation method based on feature fusion

Abstract

The invention discloses a river and lake floating object semantic segmentation method based on feature fusion. According to the method, an encoder adopts a double-backbone network combining a convolutional network and an attention network; a feature aggregation layer is introduced, and the layer aggregates features extracted by two backbone networks to ensure that information of different backbone network codes can work cooperatively; in addition, a detail label graph is generated on the low-layer feature graph through a Laplacian convolution pyramid for detail guidance; and dense jump connection is introduced on the feature map passing through the aggregation layer, all features of different scales can be fused after sampling, and finally a prediction segmentation result is output through a divider. According to the method, semantic segmentation can be effectively carried out on the river and lake floating object monitoring image, excellent performance is shown in multi-scale target segmentation, and meanwhile certain robustness is shown in the aspect of coping with unfixity of a water hyacinth shape and a boundary area which is difficult to accurately recognize.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image processing, and in particular relates to a semantic segmentation method for floating objects in rivers and lakes. Background Art

[0002] Increased human activities and production have led to a significant increase in floating debris on water surfaces. With the development of urbanization and industrialization, people's lifestyles and production patterns are constantly changing, leading to the introduction of more plastic, garbage, and invasive alien species into water bodies. Water hyacinth is a particularly representative invasive alien species. Its strong reproductive capacity and adaptability to its environment allow it to become a dominant species in slow-flowing waterways, such as rivers, canals, and reservoirs. Its rapid reproduction can lead to insufficient oxygen in the water, posing a threat to aquatic life, especially fish. Therefore, the use of intelligent technology to identify floating debris in rivers and lakes provides essential assistance in addressing water ecological issues.

[0003] With the continuous advancement of computer vision technology and convolutional neural networks, semantic segmentation has found widespread application in aquatic ecological and environmental protection operations. However, floating objects such as water hyacinths in natural water bodies exhibit a wide range of scales, from small to large, and their boundaries exhibit intricate details. This multi-scale information and complex boundary details are crucial for water hyacinth identification. However, current semantic segmentation algorithms perform poorly in this regard, and further research is needed on these models. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a semantic segmentation method for floating objects in rivers and lakes. The present invention collects images of floating objects in rivers and lakes. For floating objects in rivers and lakes with complex scenes, this segmentation model is used to segment the floating objects and perform performance evaluation, which has good results and provides technical support for solving water ecological problems.

[0005] Technical solution: A semantic segmentation method for floating objects in rivers and lakes based on feature fusion, including the following steps:

[0006] Step 1: Collect floating object images through river and lake section monitoring cameras, organize and annotate the images, and divide them into training and test sets;

[0007] Step 2: Design an encoder dual-backbone network based on the image characteristics of floating objects in rivers and lakes. The convolutional backbone is mainly constructed using a residual structure. The skip connection allows the model to selectively retain input information during the learning process, and gradients can be more easily propagated in the network, allowing the construction of deep models to extract features. The attention mechanism backbone uses multi-scale convolutional attention to learn contextual information, improving segmentation accuracy and detail expression capabilities.

[0008] Step 3: To address the problem of different feature encoding methods in the dual-branch network and the inability to directly fuse them, a feature aggregation layer is designed. A dual-branch guided aggregation layer is used to fuse the information of the two branches. The smaller feature map of the attention network passes through the first depthwise separable convolution branch and the second upsampling branch. The feature map of the convolution network passes through the depthwise separable convolution branch and the downsampling branch. After unifying the scale, the two feature maps are added together to obtain the guided aggregate feature.

[0009] Step 4: Because the floating objects have complex shapes and semantic segmentation methods have difficulty accurately identifying the boundaries of water hyacinths, we designed edge detail detection. We applied the image's true label map to a Laplacian convolutional pyramid to generate detail labels. Then, we added a segmentation head to the low-level feature map, outputting detail features and using the detail labels for guidance.

[0010] Step 5: In the feature fusion stage, feature maps from different stages are continuously upsampled to their original size. Feature maps of the same size but from different depths are fused together, effectively utilizing multi-scale information and providing the model with more powerful context-aware capabilities.

[0011] Step 6: Initialize the model parameters and optimize them using the backpropagation algorithm. Iterate continuously to minimize the loss function and save the optimal weights. Load the weights and perform inference on the test set to evaluate the model performance.

[0012] Furthermore, the dual-backbone network described in step 2 is constructed using a residual architecture (ResNet) and multi-scale convolutional attention (MSCA), which performs feature extraction through a parallel structure. The residual architecture introduces skip connections, which bypass some layers in the network and add them to the original input. This design helps gradients flow better, making it easier to train deep networks. Multi-scale convolutional attention builds attention at multiple scales through depthwise separable convolutions.

[0013] The specific steps of the residual structure network are as follows:

[0014] Step 2.1.1: Define a basic convolutional block consisting of two convolutional layers, each followed by a batch normalization layer and an activation function (ReLU);

[0015] Step 2.1.2: Skip connection, introduce skip connection in the residual block, add the input directly to the output of the convolution. Make sure the input and output have the same shape;

[0016] Step 2.1.3: Integrate multiple residual blocks. In the backbone network, multiple residual blocks are stacked together to build a deep structure.

[0017] The residual structure formula is as follows:

[0018] H(X)=F(X)+X

[0019] X represents the residual block input, H(X) is the expected output, both the input and output are identity mappings of X, and F(X) represents the residual between the expected output and the input X.

[0020] The specific steps of the multi-scale convolutional attention network are as follows:

[0021] Step 2.2.1: The convolution kernel size of the input is 5×5, the input channels and output channels are equal, and the number of grouped convolutions is equal to the number of two channels;

[0022] Step 2.2.2: The feature maps enter different depth-wise separable convolution branches. The input and output channels are divided into a single group. Independent convolution operations are performed between each group. The convolution operation is 1 pixel in the horizontal direction and 7 pixels in the vertical direction. The padding is 3 pixels in the vertical direction.

[0023] Step 2.2.3: The feature map is then divided into a single group by the input and output channels, and a convolution operation of 7 pixels is performed in the horizontal direction and 1 pixel in the vertical direction, and 3 pixels are padded in the horizontal direction and no padded in the vertical direction. The convolution kernel sizes of the other two branches are a combination of 1 and 11 and 1 and 21, so that the horizontal and vertical information can be captured on feature maps of different scales.

[0024] Step 2.2.4: Add the feature maps obtained by depthwise separable convolution kernels of different sizes element-wise to form a feature map that integrates information at different scales. Then, perform linear combination between channels through a 1×1 convolution layer. The value of each channel of the input feature map is weighted summed with a learned weight to form the output feature map.

[0025] Step 2.2.5: Perform element-wise multiplication of the original input and the weighted sum feature map to obtain the attention feature vector;

[0026] Step 2.2.6: Build BlockMSCA, and build BlockMSCA together with modules such as normalization layer, convolution projection and activation layer

[0027] Step 2.2.6: Integrate BlockMSCA and a feedforward neural network module (BlockFFN) to stack multiple building blocks together to build a deep structure.

[0028] The multi-scale convolutional attention formula is as follows:

[0029]

[0030]

[0031] Where F represents the input feature, Att and Out represent the attention map and output, DW-Conv represents the depth-wise separable convolution, Scale i , i∈{0, 1, 2, 3} represents each branch of MSCA.

[0032] Furthermore, the designed feature aggregation layer described in step 3 fuses two features of different sizes by downsampling and interpolation upsampling through depthwise separable convolution to fully utilize the features extracted by the two backbone networks.

[0033] The specific steps of the feature aggregation layer are as follows:

[0034] Step 3.1: Pass the feature map from the convolutional backbone network with shape H×W×C through two branches. The first branch is a depth-wise separable convolution with a convolution kernel size of 3×3, a batch normalization layer and a convolution with a convolution kernel size of 1×1. The output feature map is denoted as CB1.

[0035] Step 3.2: The second branch is a 3×3 convolution block and batch normalization to achieve downsampling. The output feature map is denoted as CB2 and has a shape of H / 2×W / 2×C.

[0036] Step 3.3: The feature map from the attention backbone network with shape H / 2×W / 2×C is also passed through two branches. The first branch is a depth-wise separable convolution with a convolution kernel size of 3×3, a batch normalization layer, and a Sigmoid activation layer. The output feature map is denoted as AB1. The second branch is a bilinear interpolation upsampling and batch normalization layer. The output feature map is denoted as AB2.

[0037] Step 3.4: Multiply the feature map CB1 and AB2 element-wise to obtain the features of the guided convolution backbone, and then multiply the feature map CB2 and AB1 element-wise to obtain the features of the guided attention backbone;

[0038] Step 3.5: Upsample the features of the guided attention backbone and add them element-wise to the features of the guided convolution backbone to obtain the guided aggregate features. Perform 3×3 convolution blocks, batch normalization and ReLU activation on the aggregate features, and finally output the guided aggregate features.

[0039] Furthermore, the edge detail detection designed in step 4 generates detail labels through the Laplacian convolution pyramid and adds a segmenter on the low-level feature map. Finally, the binary cross entropy loss and dice loss jointly optimize detail learning.

[0040] The specific steps of edge detail detection are as follows:

[0041] Step 4.1: Design a Laplacian convolution pyramid and use three Laplacian convolutions with different step lengths on the real label map to generate feature maps of different sizes. Then merge them and form a pyramid-shaped detail target through threshold processing.

[0042] Step 4.2: Based on the features aggregated from the second stage of the encoder, design a segmenter consisting of a 3x3 convolution, batch normalization, and a ReLU activation layer, followed by a 1x1 convolution to obtain the output prediction details.

[0043] Step 4.3: Use binary cross entropy loss and dice loss to jointly optimize detail learning.

[0044] Furthermore, the decoder designed in step 5 is densely connected on the feature maps of different resolutions to fuse different features. i，j The calculation formula is as follows:

[0045]

[0046] When j = 0, x i，0 is the output of the aggregation layer from the encoder, from x 0，0 to x 4，0 It is not obtained by downsampling. H(·) represents the convolution layer followed by the activation function, U(·) represents the upsampling layer, and [·] represents the connection layer. The nodes in the jth layer (j=0) only receive the self-aggregation layer; the nodes in the jth layer (j=1) receive two inputs, both from the encoder subnetwork but in two consecutive layers; and the nodes in the jth layer (j>1) receive j+1 inputs, where the jth input is the output from the previous j nodes in the same jump connection, and the j+1th input is the upsampled output from the lower jump connection. Finally, in X 0，1 , X 0，2 , X 0，3 , X 0，4 After the output of , a 1×1 convolution and a Sigmoid activation function segmenter are set. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the present invention;

[0048] Figure 2 This is a model network structure diagram of a semantic segmentation method for floating objects in rivers and lakes based on feature fusion in a specific embodiment;

[0049] Figure 3 This is a structural diagram of the multi-scale convolutional attention mechanism in a specific embodiment;

[0050] Figure 4 It is a structural diagram of a characteristic polymerization layer in a specific embodiment;

[0051] Figure 5 Schematic diagram of the Laplacian convolution pyramid in a specific embodiment;

[0052] Figure 6 It is a structural diagram of a decoder in a specific embodiment. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0054] This application provides Figures 1-6 A semantic segmentation method for floating objects in rivers and lakes based on feature fusion is shown in Figure 1 The semantic segmentation method of river and lake floating objects based on feature fusion includes the following steps:

[0055] Step 1: Create a dataset. Using the Hikvision monitoring platform built for the Taihu Lake Basin, we regularly collected river and lake cross-sectional images of Shangta, Liulitang, Taipu Gate, and Yuhuitang. Images containing floating objects on the water surface were used as data sources, annotated, and divided into datasets.

[0056] Step 2: If Figure 2 This is the network structure diagram of the semantic segmentation method model. The backbone network uses a convolutional network and an attention mechanism-based network to work in parallel. The steps to implement the convolutional network backbone are as follows:

[0057] Step 2.1.1: Define a basic convolution module, using a 3x3 convolution function, to construct the convolution layer in ResNet, Conv2d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False);

[0058] Step 2.1.2: Define the basic residual block (BasicBlock), which is the basic building block in ResNet, including two convolutional layers, Batch Normalization and ReLU activation function.

[0059] Step 2.1.3: Define the main structure of ResNet, which includes initializing the convolutional layer, pooling layer and four stages of residual blocks. The feature map with shape [1, 3, 224, 224] passes through the pooling layer and the four residual blocks, and the shapes are [1, 64, 56, 56], [1, 64, 56, 56], [1, 128, 28, 28], [1, 256, 14, 14], and [1, 512, 7, 7] respectively.

[0060] The multi-scale convolutional attention network is built using multi-scale convolutional attention, and its structure is as follows Figure 3 As shown, the implementation steps are as follows:

[0061] Step 2.2.1: Define a multi-scale convolutional attention mechanism (MSCA) and use depth-wise separable convolution to extract features at three scales. Taking the first scale as an example, nn.Conv2d(dim, dim, (1, 7), padding = (0, 3), groups = dim), nn.Conv2d(dim, dim, (7, 1), padding = (3, 0), groups = dim), that is, first perform a 1-pixel convolution operation in the horizontal direction and a 7-pixel convolution operation in the vertical direction; then perform a 7-pixel convolution operation in the horizontal direction and a 1-pixel convolution operation in the vertical direction. The subsequent scales are 7 and 11. These convolution results are then added together for a 1x1 convolution and channel mixing. Finally, perform element-by-element multiplication with the original input to obtain the attention vector;

[0062] Step 2.2.2: Define a multi-scale convolutional attention mechanism module (BlockMSCA), which applies normalization, 1x1 convolution, GELU activation, MSCA module, 1x1 convolution again, channel rescaling, and random depth normalization to the input, and finally adds the output of the operation to the original input;

[0063] Step 2.2.3: Construct the multi-scale convolutional attention mechanism stage (StageMSCA), which is constructed by the multi-scale convolutional attention mechanism module and a feed-forward neural network module;

[0064] Step 2.2.4: Define the multi-scale convolutional attention mechanism backbone network (MSCANet), which is constructed by four multi-scale convolutional attention mechanism stages and performs five downsampling to extract features.

[0065] The encoder backbone network is designed based on these two networks, which work in parallel to extract features.

[0066] Step 3: If Figure 4This is the structure diagram of the feature aggregation layer. The feature maps on the two backbone networks are aligned by sampling operations, and then element-wise multiplied with each other's feature maps before merging. The specific steps are as follows:

[0067] Step 3.1: Pass the feature map of the 56×56×64 shape from the convolutional backbone network through two branches (taking the size of 56*56 as an example). The first branch is a depth-wise separable convolution with a convolution kernel size of 3×3, a batch normalization layer and a convolution with a convolution kernel size of 1×1. The output feature map is denoted as CB1.

[0068] Step 3.2: The second branch is a 3×3 convolution block and batch normalization to achieve downsampling. The output feature map is denoted as CB2 and has a shape of 28×28×64.

[0069] Step 3.3: The feature map of the shape 28×28×64 from the attention backbone network is also passed through two branches. The first branch is a depth-wise separable convolution with a convolution kernel size of 3×3 and a batch normalization layer and a sigmoid activation layer. The output feature map is denoted as AB1. The second branch is a bilinear interpolation upsampling and batch normalization layer. The output feature map is denoted as AB2.

[0070] Step 3.4: Multiply the feature map CB1 and AB2 element-wise to obtain the features of the guided convolution backbone, and then multiply the feature map CB2 and AB1 element-wise to obtain the features of the guided attention backbone;

[0071] Step 3.5: Upsample the features of the guided attention backbone and add them element-wise to the features of the guided convolution backbone to obtain the guided aggregate features. Perform 3×3 convolution blocks, batch normalization and ReLU activation on the aggregate features, and finally output the guided aggregate features.

[0072] Step 4: Edge detail detection generates detail labels by passing the true label map of the image through the Laplacian convolution pyramid, which is shown in the figure. Figure 5 As shown, a segmentation head is then added to the low-level feature map to output detail features and guided by detail labels. The specific steps are as follows:

[0073] Step 4.1: Design a Laplacian convolution pyramid and use three Laplacian convolutions with different step lengths on the real label map to generate feature maps of different sizes. Then merge them and form a pyramid-shaped detail target through threshold processing.

[0074] Step 4.2: Based on the features aggregated from the second stage of the encoder, design a segmenter consisting of a 3×3 convolution, batch normalization, and a ReLU activation layer, followed by a 1×1 convolution to obtain the output prediction details;

[0075] Step 4.3: Use binary cross entropy loss and dice loss to jointly optimize detail learning.

[0076] Step 5: Figure 6 The decoder structure shown is to perform dense connections on feature maps of different resolutions and fuse different features. From a horizontal perspective, the multi-scale features of all previous nodes are combined at the same resolution; from a vertical perspective, the multi-scale features of different resolutions are integrated from its previous nodes. i，j represents the neural network layer, x i，j Represents the output of this layer, where the leftmost column represents the aggregation layer connected to the encoder. 1，0 Start by upsampling and summing with x 0，0 Merge to get x 0，1 ;x 2，0 By upsampling and combining with x 1，0 Merge to get x 1，1 , change x 1，1 Upsampling with x 0，1 and x 0，0 Merge to get x 0，2 ;x 3，0 By upsampling and combining with x 2，0 Merge to get x 2，1 , x 2，1 Then upsample, and x 1，0 and x 1，1 Merge to get x 1，2 , x 1，2 Then upsample, and 0，0 、x 0，1 and x 0，2 Merge to get x 0，3 , and so on, x 0，4 Also originally created by x 4，0 Obtained by step-by-step upsampling and fusion.

[0077] x i，j The calculation formula is as follows.

[0078]

[0079] When j = 0, x i，0 is the output of the aggregation layer from the encoder, from x 0，0 to x 4，0 It is not obtained by downsampling, H(·) represents the convolution layer followed by the activation function, U(·) represents the upsampling layer, and [·] represents the connection layer. Figure 6As shown, the nodes in layer j (j=0) only receive the self-aggregation layer; the nodes in layer j (j=1) receive two inputs, both from the encoder sub-network but in two consecutive layers; and the nodes in layer j (j>1) receive j+1 inputs, where the jth input is the output from the previous j nodes in the same jump connection, and the j+1th input is the upsampled output from the lower jump connection. 0，1 , X 0，2 , X 0 ，3 , X 0，4 After the output of , a 1x1 convolution and a Sigmoid activation function splitter are set respectively.

[0080] Step 6: The model first needs to be initialized. Next, the backpropagation algorithm is used to optimize the model parameters, minimizing the loss function through continuous iteration and saving the optimal weights. These optimal weights are loaded on the test set, and the model is inferred to evaluate its performance.

[0081] The current algorithm of this invention addresses the problems of large size differences and unclear boundary information recognition of floating objects in rivers and lakes, and proposes a semantic segmentation method for floating objects in rivers and lakes based on feature fusion. It uses a dual-backbone network based on convolution and attention to extract features; a feature aggregation layer is designed to ensure the coordinated operation of the two different types of information; edge detail detection is added to improve the ability to recognize boundaries; and finally, dense connection and fusion of features are performed on all features of different sizes. This method not only performs well in handling target size differences and multi-scale feature extraction, but also shows a certain degree of robustness in dealing with the unstable morphology of water hyacinths and the difficult-to-accurate identification of boundary area information.

Claims

1. A semantic segmentation method for floating objects in rivers and lakes based on feature fusion, characterized in that: The following steps are involved: Step 1: Collect floating object images through river and lake section monitoring cameras, organize and annotate the images, and divide them into training and test sets; Step 2: Design an encoder dual-backbone network based on the image characteristics of floating objects in rivers and lakes. The convolutional backbone is mainly constructed using a residual structure (ResNet). Skip connections allow the model to retain input information during the learning process, avoiding the problem of gradient vanishing in deep networks and allowing the construction of deep models to extract features. The attention mechanism backbone is built using multi-scale convolutional attention (MSCA), which can dynamically focus on areas that have an important impact on the segmentation task during the learning process, learn contextual information, and improve the accuracy of segmentation and the ability to express details. Step 3: To address the problem of different feature encoding methods in the dual-backbone network and the inability to directly fuse features, a feature aggregation layer is designed. A dual-branch guided aggregation layer is used to fuse information from the two branches. The smaller feature maps from the attention network pass through a depthwise separable convolution branch and an upsampling branch, while the larger feature maps from the convolution feature map branch pass through a depthwise separable convolution branch and a downsampling branch. The two feature maps are then resized and added together to generate the guided aggregated features. Step 4: Because floating objects have complex shapes, semantic segmentation methods struggle to accurately identify the boundaries of water hyacinths. Therefore, edge detail detection is designed. This method generates detail labels by applying a Laplacian convolutional pyramid to the image's true label map. A segmenter is then added to the low-level feature map to output detail features, which are then used for guidance. Step 5: In the feature fusion stage, the feature maps of different stages are continuously upsampled to their original size, and the feature maps of the same size but from different depths are fused to efficiently utilize multi-scale information, providing the model with more powerful context perception capabilities, which is conducive to improving the details of the segmentation and the reconstruction of the shape.

2. The semantic segmentation method for floating objects in rivers and lakes based on feature fusion according to claim 1, characterized in that: The dual-backbone network described in Step 2 is constructed using a residual architecture (ResNet) and multi-scale convolutional attention (MSCA), using a parallel structure for feature extraction. The residual architecture introduces skip connections, which bypass some layers in the network and add the original input. This design helps gradients flow better, making it easier to train deep networks. Multi-scale convolutional attention constructs an attention mechanism network at multiple scales through depthwise separable convolutions. The specific steps of the residual structure network are as follows: Step 2.1.1: Define a basic convolutional block consisting of two convolutional layers, each followed by a batch normalization layer and an activation function (ReLU); Step 2.1.2: Skip connection, introduce skip connection in the residual block, add the input directly to the output of the convolution. Make sure the input and output have the same shape. If they are different, use additional linear transformation to make them match. Step 2.1.3: Integrate multiple residual blocks. In the backbone network, multiple residual blocks are stacked together to build a deep network structure. The residual structure formula is as follows: H(X)=F(X)+X X represents the residual block input, H(X) is the expected output, both the input and output are identity mappings of X, and F(X) represents the residual between the expected output and the input X. The specific steps of the multi-scale convolutional attention network are as follows: Step 2.2.1: The convolution kernel size of the input is 5×5, the input channels and output channels are equal, and the number of grouped convolutions is equal to the number of two channels; Step 2.2.2: The feature maps enter different depth-wise separable convolution branches. The input and output channels are divided into a single group. Independent convolution operations are performed between each group. The convolution operation is 1 pixel in the horizontal direction and 7 pixels in the vertical direction. The padding is 3 pixels in the vertical direction. Step 2.2.3: The feature map is then divided into a single group by the input and output channels, and a convolution operation of 7 pixels is performed in the horizontal direction and 1 pixel in the vertical direction, and 3 pixels are padded in the horizontal direction and no padded in the vertical direction. The convolution kernel sizes of the other two branches are a combination of 1 and 11 and 1 and 21, so that the horizontal and vertical information can be captured on feature maps of different scales. Step 2.2.4: Add the feature maps obtained by depthwise separable convolution kernels of different sizes element-wise to form a feature map that integrates information at different scales. Then, perform linear combination between channels through a 1×1 convolution layer. The value of each channel of the input feature map is weighted summed with a learned weight to form the output feature map. Step 2.2.5: Perform element-wise multiplication of the original input and the weighted sum feature map to obtain the attention feature vector; Step 2.2.6: Construct BlockMSCA. Combine MSCA with normalization layer, convolutional projection layer, and activation layer to build BlockMSCA. Step 2.2.7: Integrate BlockMSCA and feedforward neural network module (BlockFFN) to construct StageMSCA. In the backbone network, multiple structural blocks are stacked together to build a deep structure. The multi-scale convolutional attention formula is as follows: Where F represents the input feature, Att and Out represent the attention map and output, DW-Conv represents the depth-wise separable convolution, Scale i , i∈{0, 1, 2, 3} represents each branch of MSCA.

3. The semantic segmentation method for floating objects in rivers and lakes based on feature fusion according to claim 1, characterized in that: The designed feature aggregation layer described in step 3 performs downsampling and interpolation upsampling through depthwise separable convolution to fuse two features of different sizes, so as to make full use of the features extracted by the two backbone networks. The specific steps of the feature aggregation layer are as follows: Step 3.1: Pass the feature map from the convolutional backbone network with shape H×W×C through two branches. The first branch is a depth-wise separable convolution with a convolution kernel size of 3x3, a batch normalization layer and a convolution with a convolution kernel size of 1×1. The output feature map is denoted as CB1. Step 3.2: The second branch is a 3×3 convolution block and batch normalization to achieve downsampling. The output feature map is denoted as CB2 and has a shape of H / 2×W / 2×C. Step 3.3: The feature map from the attention backbone network with shape H / 2×W / 2×C is also passed through two branches. The first branch is a depth-wise separable convolution with a convolution kernel size of 3×3, a batch normalization layer, and a Sigmoid activation layer. The output feature map is denoted as AB1. The second branch is a bilinear interpolation upsampling and batch normalization layer. The output feature map is denoted as AB2. Step 3.4: Multiply the feature map CB1 and AB2 element-wise to obtain the features of the guided convolution backbone, and then multiply the feature map CB2 and AB1 element-wise to obtain the features of the guided attention backbone; Step 3.5: Upsample the features of the guided attention backbone and add them element-wise to the features of the guided convolution backbone to obtain the guided aggregate features. Perform 3×3 convolution blocks, batch normalization and ReLU activation on the aggregate features, and finally output the guided aggregate features.

4. The semantic segmentation method for floating objects in rivers and lakes based on feature fusion according to claim 1, characterized in that: The edge detail detection designed in step 4 generates detail labels through the Laplacian convolution pyramid and adds a segmenter on the low-level feature map. Finally, the binary cross entropy loss and dice loss jointly optimize the detail learning. The specific steps of edge detail detection are as follows: Step 4.1: Design a Laplacian convolution pyramid and use three Laplacian convolutions with different step lengths on the real label map to generate feature maps of different sizes. Then merge them and form a pyramid-shaped detail label map through threshold processing. Step 4.2: Based on the features aggregated from the second stage of the encoder, design a segmenter consisting of a 3×3 convolution, batch normalization, and a ReLU activation layer, followed by a 1×1 convolution to obtain the output prediction details; Step 4.3: Use binary cross entropy loss and dice loss to jointly optimize detail learning.

5. The semantic segmentation method for floating objects in rivers and lakes based on feature fusion according to claim 1, characterized in that: The decoder designed in step 5 is densely connected on the feature maps of different resolutions to fuse different features. i，j The calculation formula is as follows: When j = 0, x i，0 is the output of the aggregation layer from the encoder, from x 0，0 to x 4，0 It is not obtained by downsampling. H(·) represents the convolution layer followed by the activation function, U(·) represents the upsampling layer, and [·] represents the connection layer. The nodes in the jth layer (j=0) only receive the self-aggregation layer; the nodes in the jth layer (j=1) receive two inputs, both from the encoder subnetwork but in two consecutive layers; and the nodes in the jth layer (j>1) receive j+1 inputs, where the jth input is the output from the previous j nodes in the same jump connection, and the j+1th input is the upsampled output from the lower jump connection. Finally, in X 0，1 , X 0，2 , X 0，3 , X 0，4 After the output of , a 1x1 convolution and a Sigmoid activation function splitter are set respectively.

Images