Unmanned ship water surface anti-interference real-time obstacle segmentation method based on context information and attention mechanism

Abstract

The invention discloses an unmanned ship water surface anti-interference real-time obstacle segmentation method based on context information and an attention mechanism. The method comprises the following steps: obtaining a data set formed by water surface environment images captured by an unmanned ship from a visual angle; dividing the data set into a training set and a test set according to a proportion; a real-time water surface obstacle segmentation network model is constructed and is used for obstacle detection based on visual information under the condition that water surface interference exists; based on the training set data, training the real-time water surface obstacle segmentation network to obtain a trained real-time water surface obstacle segmentation network model; according to the method, the FPPM captures local and global context information through cascaded average pooling operation and global pooling operation, the AFFM uses a channel-space attention mechanism, the real-time water surface obstacle segmentation network model is trained, the real-time water surface obstacle segmentation network model is trained, and unmanned surface anti-interference real-time obstacle segmentation is realized. Reflection and obstacles are distinguished by capturing differences in detailed features and spatial positions.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to an unmanned boat surface anti-interference real-time obstacle segmentation method based on context information and attention mechanism. Background Art

[0002] As a small intelligent robot, unmanned boats have shown great potential in many fields such as marine scientific research, environmental monitoring, search and rescue operations. However, regardless of the nature of the mission, reliable autonomous navigation is always the key to the effective completion of the mission by unmanned boats. The safety of autonomous navigation systems directly depends on their environmental perception capabilities, especially the ability to detect obstacles, which is crucial for timely response and collision avoidance. Due to the limitations of sensor cost and power consumption, combining cameras and visual algorithms has become the mainstream method for unmanned boats to detect surface obstacles.

[0003] There are two main challenges in vision-based obstacle detection. The first challenge is surface target detection, which focuses on dynamic obstacle detection on the water surface, such as ships and buoys. However, static obstacle detection such as docks and coastlines requires positioning based on coastline maps to ensure navigation safety. The second challenge is obstacle detection without a coastline map, which is the obstacle segmentation task. This task involves dividing the input image into drivable and non-drivable areas and assigning a specific label to each pixel to represent obstacles, water, and sky.

[0004] Obstacle detection methods for marine environments can be roughly divided into two categories, namely classical methods and deep learning-based methods. Specifically, some classical methods divide target detection in marine environments into saliency detection problems. These methods usually assume that objects can be effectively distinguished from the surrounding environment. Classic background subtraction [1] This is not applicable to unmanned boats because the marine environment is dynamically changing and the unmanned boats will shake due to waves, violating the assumption of a static camera and resulting in a high false detection rate. [2] The binocular camera measures the position difference of the object, generates a depth map, and maps it to three-dimensional space, effectively detecting and marking obstacles on the water surface. Research method based on Markov random field framework [3] , [4] Satisfactory results have been achieved in the unsupervised water surface obstacle segmentation task. These studies have used prior information to roughly divide the image into three semantic regions: the bottom region representing water, the top region representing the sky, and the middle region that may represent land or moored ships. By using this prior information, a structured constrained semantic segmentation graph model is constructed using Markov conditional random fields for obstacle detection.

[0005] In recent years, semantic segmentation technology based on deep learning has shown great potential in the field of marine obstacle detection. Ferryman et al. [5] Through methods such as SegNet and ESPNet, the marine environment images in the ADE20k dataset were used for training, and their effectiveness was verified on the MODD and SMD datasets. Bovcon et al. [6] The three classic semantic segmentation networks PSPNet, DeeplabV2 and UNet were compared, and experiments were conducted on the MaSTr1325 dataset, and finally their research results were evaluated on the MODD2 test set. These studies have solved the problem of water surface obstacle detection to a certain extent.

[0006] Recently, the WaSR semantic segmentation network has achieved remarkable results in the segmentation of surface obstacles. The network adopts an encoder-decoder structure and integrates IMU data to more accurately estimate the sea-sky boundary, thereby improving the segmentation accuracy in ambiguous scenes. In addition, WaSR also introduces an innovative feature fusion method, which uses a channel attention module to refine the fused features, further improving the accuracy and effect of segmentation. When deploying a deep learning-based semantic segmentation network on an unmanned boat edge computing device, real-time performance becomes a key consideration. To this end, ShorelineNet [7] It adopts a structure similar to UNet and selects the lightweight MobileNetv2 as the encoder. It reduces the computational cost and improves the inference speed by adjusting the input image size. Chen et al. [8] A novel WODIS is proposed, which aims to solve the obstacle detection problem in marine environment by combining feature enhancement and feature fusion modules, while balancing accuracy and real-time performance.

[0007] Recent Research [9] The modules in the WaSR network were analyzed for computational cost, and the modules with high cost but limited performance improvement were replaced to achieve the best balance between performance and speed. Wang et al. proposed LWDNet, a lightweight segmentation network that uses separable convolutions and lightweight encoders to improve inference speed, but does not solve the common problems of water surface segmentation.

[0008] Another noteworthy study is that of Zhou et al.

[10] The proposed CPWNet is a lightweight semantic segmentation model that combines three effective loss functions to achieve a good balance between accuracy and real-time performance. [6] The best performance was achieved on all datasets.

[0009] (1) Classic obstacle detection methods face many challenges in the maritime application of unmanned boats. Background subtraction is limited by the assumption of a static camera and is not applicable to dynamic maritime environments. Although stereo vision methods are adaptable to dynamic environments, the detection distance is limited by the camera baseline distance and is dependent on scenes with rich textures. Calm water surfaces may cause detection failures. Markov random field methods have difficulty dealing with visual ambiguity, noise, and ocean-specific wakes, fog, and sun glare.

[0010] (2) Object detection methods in deep learning, such as YOLO and Faster RCNN, achieve good results in dynamic obstacle detection, but have deficiencies in static obstacle recognition. In contrast, semantic segmentation methods based on deep learning can detect static and dynamic obstacles in parallel, but the research of Ferryman and Bovcon et al. still has water surface segmentation defects when facing specific scenes such as water surface reflections, weak light or water surface wake, resulting in erroneous detection results and misleading the path planning of the unmanned boat.

[0011] (3) Although WaSR has made a breakthrough in the segmentation of water obstacles, it has neglected real-time performance and faces challenges in practical applications. Existing real-time semantic segmentation networks such as ShorelineNet, LWDNet, and CPWNet often sacrifice some accuracy when improving inference speed. These lightweight models still need to be strengthened in terms of balancing accuracy and speed.

[0012] Technical issues to be resolved:

[0013] The main challenges in the field of surface obstacle segmentation come from water surface reflections, water surface glare, obstacle size variations and category imbalance. In this context, water surface reflections are the most critical issue. The features of reflections are highly similar to those of real obstacles, and their proximity in spatial positions often causes the algorithm to be confused when distinguishing the two, ultimately leading to false alarms. In addition, the significant features of water surface glare on the water surface can cause the algorithm to mistakenly identify them as obstacles. At the same time, the size variations between obstacles on the water surface are significant, often resulting in smaller obstacles being ignored by the algorithm, resulting in missed detections. It is worth noting that in typical ocean scenes, most pixels belong to the water surface or the sky, resulting in relatively few obstacle pixels, which poses a challenge to the accuracy of the algorithm. It should be noted that false alarms often cause the UWV to re-plan its route, thus affecting efficiency, while missed detections directly threaten its navigation safety. Summary of the invention

[0014] The technical solution adopted by the present invention is: a method for anti-interference real-time obstacle segmentation of an unmanned watercraft surface based on context information and attention mechanism, characterized in that it includes the following steps:

[0015] Obtain a data set consisting of water surface environment images captured from the perspective of an unmanned boat;

[0016] Divide the data set into training set and test set according to the proportion;

[0017] Construct a real-time water surface obstacle segmentation network model, which can be used to segment obstacles based on visual information in the presence of water surface interference;

[0018] Based on the training set data, the real-time water surface obstacle segmentation network is trained to obtain a trained real-time water surface obstacle segmentation network model;

[0019] The test set data is input into the trained real-time surface obstacle segmentation network model to realize the anti-interference real-time obstacle segmentation of the unmanned boat surface.

[0020] Furthermore: the real-time water surface obstacle segmentation network model adopts an encoder-decoder architecture. The encoder part uses the ResNet18 lightweight backbone network. A fast pyramid pooling module is introduced in the last stage of the encoder to distinguish water surface glare from real obstacles through local and global context features, thereby reducing false alarms caused by water surface glare. An improved lightweight feature pyramid network is used in the decoder part to solve the significant multi-scale imbalance problem of water surface obstacles. Finally, a feature fusion module based on the channel-spatial attention mechanism is used to effectively identify the subtle differences between real obstacles and water surface reflections in terms of detail features and spatial positions, thereby reducing false alarms caused by water surface reflections.

[0021] Further: the encoder includes four convolutional layers of the ResNet18 backbone network and a fast pyramid pooling module,

[0022] The four convolutional layers and the fast pyramid pooling module of the ResNet18 backbone network are sequentially connected;

[0023] The fast pyramid pooling module uses cascade average pooling operations to expand the receptive field, capture multi-scale local information, and obtain global context information through global average pooling, so that obstacles and water surface glare can be accurately distinguished. The pooling layer with a smaller receptive field can focus on the local area of ​​the image and capture detailed information about obstacles and water surface glare. At the same time, the pooling layer with a larger receptive field can use global context information to identify the overall shape, brightness and relationship between obstacles and the surrounding environment. Comparing and analyzing this information at different scales helps the model distinguish water surface glare from obstacles.

[0024] Further: the fast pyramid pooling module uses cascaded average pooling operations to expand the receptive field, capture multi-scale local information, and obtain global context information through global average pooling, and then splices along the channel dimension to perform feature fusion. The process is as follows:

[0025] The input is represented as Where c, h, and w represent the number of channels, height, and width of the image, respectively;

[0026] first, Through a 1×1 convolution operation, denoted as F c , this operation is used to reduce the number of channels and generate a new feature map, denoted as This conversion is formally expressed as:

[0027]

[0028] in, Afterwards, a feature map containing global context information is obtained through global average pooling and upsampling operations. It is expressed as:

[0029]

[0030] in, U and Represent upsampling and global average pooling operations respectively;

[0031] also, It also passes through three average pooling operations with a pooling kernel size of 5, and the receptive fields of these three average pooling operations are 5, 9, and 13 respectively; thus generating and Three feature maps containing local information are specifically expressed as:

[0032]

[0033]

[0034] in, F avg represents the average pooling operation;

[0035] Finally, the global features And local features and Through the superposition operation, the fused feature map is obtained. Then apply a 1×1 convolution operation to restore the original number of channels to obtain the fused feature map Specifically expressed as:

[0036]

[0037] in, Indicates an overlay operation.

[0038] Further: the decoder includes an attention-based lightweight feature fusion module; the attention-based lightweight feature fusion module uses a channel attention mechanism and a spatial attention mechanism to refine features and model the spatial relationship of input features, which can capture the long-range dependency of enhanced features in spatial positions, thereby effectively enhancing feature representation;

[0039] Among them, the channel attention mechanism calculates the importance of each channel and uses the importance as a weight to weight the feature map of each channel. It can select the most discriminative feature channel in the image, thereby enhancing the feature expression of the real obstacle.

[0040] The spatial attention mechanism calculates the importance of each pixel and uses the importance as a weight to weight the feature map. It can pay more attention to the areas with important information in the image when extracting features, especially the spatial positions of real obstacles, and distinguish between real obstacles and water surface disturbances in complex environments.

[0041] Furthermore: the channel attention mechanism, by calculating the importance of each channel and using the importance as a weight to weight the feature map of each channel, can select the most distinguishing feature channel in the image, thereby strengthening the feature expression of the real obstacle; the spatial attention mechanism, by calculating the importance of each pixel point and using the importance as a weight to weight the feature map, can pay more attention to the area with important information in the image when extracting features, especially the spatial position of the real obstacle, and realize the process of distinguishing the real obstacle from the water surface disturbance in a complex environment as follows:

[0042] The input is represented as

[0043] The input is stacked along the channel dimension to generate a feature map

[0044] after, The input of the channel attention is generated through a 1×1 split convolution operation This conversion is expressed mathematically as:

[0045]

[0046] in, F S Represents a 1×1 depthwise separable convolution operation;

[0047] Then, Through global maximum pooling and global average pooling Feature compression is performed to capture global spatial information in two different forms: maximum and average. These compressed features are represented as and Then through the shared multi-layer perceptron T mlp Processed and normalized using the sigmoid function σ; a channel attention weight matrix α is generated to highlight the importance of different channels; in order to obtain channel enhancement features Output of the convolutional block Multiplying element-by-element with the channel attention weight α, the calculation of the channel enhancement feature is expressed as:

[0048]

[0049] in, is an intermediate feature, The dimensions of α are and ⊙ represent element-by-element addition and element-by-element multiplication operations, respectively.

[0050] feature Take the maximum and average values ​​along the channel dimension to get the features and Then concatenate along the channel dimension to get the feature map The process is expressed as:

[0051]

[0052] in, is an intermediate feature;

[0053] After 3×3 convolution operation T c And the sigmoid loss function σ, the weight matrix β is obtained;

[0054] By combining the spatial attention weight matrix β with the channel enhancement feature Perform element-by-element multiplication to obtain spatial enhancement features

[0055] Afterwards, the spatial enhancement features With the original features Add element by element to get the final feature

[0056] The whole process can be expressed mathematically as follows:

[0057]

[0058]

[0059] in,

[0060] Further: The loss function of the real-time water surface obstacle segmentation network model is expressed as follows:

[0061]

[0062] Where i is the index of the sample, ranging from 1 to the total number of samples, c represents the index of the category, and C is the total number of categories; represents the predicted probability that the i-th pixel belongs to category c, and represents the true label of the i-th pixel. In formula (25), γ is an adjustable parameter used to adjust the impact of difficult and easy samples on the loss, which is set to 0.25 here;

[0063] Therefore, the total loss function of the real-time water surface obstacle segmentation network model training is expressed as:

[0064]

[0065] Among them, λ1, λ2 and λ3 are adjustable weight coefficients.

[0066] Inspired by the encoder-decoder architecture, the present invention designs a real-time surface obstacle segmentation network model (RSOS-Net) to realize online surface obstacle detection of unmanned boats. Mainly, the present invention adopts an improved lightweight feature pyramid network structure to flexibly adapt to significant scale changes and enhance attention to small obstacles. In order to solve the visual ambiguity problem caused by water surface disturbances, the present invention integrally designs a fast pyramid pooling module (FPPM) and an attention-based feature fusion module (AFFM) in the lightweight encoder and decoder, respectively. In this context, FPPM captures local and global contextual information through cascaded average pooling operations and global pooling operations, thereby distinguishing water surface glare from real obstacles. At the same time, AFFM uses the channel-space attention mechanism to distinguish reflections from obstacles by capturing detailed features and differences in spatial positions.

[0067] The present invention provides an unmanned boat surface anti-interference real-time obstacle segmentation method based on context information and attention mechanism, which has the following advantages:

[0068] 1) To address the problem of missed detection of small obstacles caused by multi-scale imbalance of obstacles, a lightweight feature pyramid network is constructed, which utilizes multi-scale feature maps to adapt to significant scale changes while reducing the computational cost.

[0069] 2) In order to reduce the false positives caused by water surface glare, a fast pyramid pooling module (FPPM) is proposed. This module uses cascaded average pooling operations to expand the receptive field and capture multi-scale local information. It also obtains global context information through global average pooling, so that it can accurately distinguish obstacles from water surface glare.

[0070] 3) In order to accurately distinguish water surface reflections from real obstacles, an attention-based feature fusion module (AFFM) is proposed, which integrates multi-scale features and uses the channel-spatial attention mechanism to effectively identify the subtle differences in detail features and spatial positions between real obstacles and water surface reflections. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0072] Figure 1 RSOS-Net overall architecture diagram;

[0073] Figure 2 Fast pyramid pooling module diagram;

[0074] Figure 3 It is a diagram of a lightweight feature fusion module based on attention;

[0075] Figure 4 RSOS-Net baseline model structure diagram;

[0076] Figure 5 Qualitative comparison between the baseline model and RSOS-Net;

[0077] Figure 6 Qualitative comparison between RSOS-Net and different methods;

[0078] Figure 7 Comparison of recall rate (Re) and false positive number (FP) of five different baseline models on different obstacle sizes, including (a) recall rate of different obstacle sizes, (b) false positive number of different obstacle sizes.

[0079] Figure 8 Unmanned boat test platform;

[0080] Fig. 9 Obstacle segmentation results based on the unmanned boat embedded platform;

[0081] Fig.10Comparison of F1 scores and false positive numbers (FP) of different methods for different obstacle sizes. (a) F1 scores for different obstacle sizes, (b) false positive numbers for different obstacle sizes. DETAILED DESCRIPTION

[0082] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] A method for anti-interference real-time obstacle segmentation of an unmanned watercraft surface based on context information and attention mechanism, comprising the following steps:

[0085] S1: Obtain a data set of water surface environment images captured from the perspective of an unmanned boat;

[0086] S2: Divide the data set into training set, validation set and test set according to the proportion;

[0087] S3 builds a real-time surface obstacle segmentation network for use in conditions where there are surface disturbances, such as reflections and glare, so that the unmanned boat can segment obstacles during autonomous navigation based on visual information.

[0088] S4: Based on the training set data and the validation set data, the real-time water surface obstacle segmentation network model (RSOS-Net network model) is trained to obtain a trained real-time water surface obstacle segmentation network model;

[0089] S5: Input the test set data into the trained real-time surface obstacle segmentation network model to achieve anti-interference real-time obstacle segmentation on the unmanned boat surface.

[0090] Steps S1 / S2 / S3 / S4 / S5 are executed sequentially;

[0091] RSOS-Net adopts Figure 1The encoder-decoder architecture shown in Figure 1 is used to efficiently handle semantic segmentation tasks. This architecture uses encoder and decoder structures to extract features hierarchically and perform effective fusion. In the encoder stage, features gradually evolve from low-level to high-level, the number of channels gradually increases, and the spatial dimension gradually decreases. This approach balances the computational costs of different stages and ultimately ensures the efficiency of the encoder.

[11] Inspired by the CNN network, RSOS-Net utilizes a standard encoder network to generate multiple resolution features. These features are then seamlessly integrated through a top-down path and lateral connections to ensure that spatial and semantic information is preserved during the fusion process.

[0092] In order to strike a balance between performance and computational cost, RSOS-Net tested three lightweight encoder networks.

[12] It is the first choice because of its high efficiency.

[11] Different from RSOS-Net, it uses separable convolution in the lateral connection part.

[13] It replaces the original standard 3×3 convolution, thereby reducing the computational cost. In addition, RSOS-Net also cancels the lateral connection during the 1 / 4 downsampling process to further optimize the computational efficiency.

[0093] On the decoder side, RSOS-Net follows the pattern of most real-time segmentation networks. Specifically, the spatial size of features is gradually increased while keeping the consistency of feature channels in different layers of the decoder. To mitigate the impact of upsampling on the output of each feature layer, separable convolutions are used instead of standard 3×3 convolutions, thereby improving computational efficiency.

[0094] The performance and speed of RSOS-Net are mainly affected by the number of feature channels in the decoder. Therefore, experiments were conducted using 512 and 256 channels. The experimental results show that when the number of feature channels is set to 256, RSOS-Net achieves the best balance between performance and speed.

[0095] The real-time water surface obstacle segmentation network model adopts an encoder-decoder architecture. The encoder part uses the ResNet18 lightweight backbone network. In the last stage of the encoder, the Fast Pyramid Pooling Module (FPPM) is introduced to distinguish water surface glare from real obstacles through local and global context features, thereby reducing false alarms caused by water surface glare. In the decoder part, an improved lightweight Feature Pyramid Network (FPN) is used to solve the problem of significant multi-scale imbalance of water surface obstacles. Finally, the Attention-Based Feature Fusion Module (AFFM) based on the channel-spatial attention mechanism is used to effectively identify the subtle differences between real obstacles and water surface reflections in terms of detail features and spatial positions, thereby reducing false alarms caused by water surface reflections.

[0096] The encoder includes four convolutional layers of the ResNet18 backbone network and a fast pyramid pooling module.

[0097] The four convolutional layers and the fast pyramid pooling module of the ResNet18 backbone network are sequentially connected;

[0098] Figure 2 Fast pyramid pooling module diagram;

[0099] The fast pyramid pooling module uses cascaded average pooling operations to expand the receptive field, capture multi-scale local information, and obtain global context information through global average pooling, so that obstacles and water glare can be accurately distinguished. Pooling layers with smaller receptive fields can focus on local areas of the image and capture detailed information about obstacles and water glare. At the same time, pooling layers with larger receptive fields can use global context information to identify the overall shape, brightness, and relationship of obstacles with the surrounding environment. Comparing and analyzing this information at different scales helps the model distinguish water glare from obstacles.

[0100] The fast pyramid pooling module uses cascaded average pooling operations to expand the receptive field, capture multi-scale local information, and obtain global context information through global average pooling. The process of splicing along the channel dimension for feature fusion is as follows:

[0101] like Figure 4 As shown, the input of the fast pyramid pooling module is expressed as Where c, h, and w represent the number of channels, height, and width of the image, respectively.

[0102] first, Through a 1×1 convolution operation, denoted as Fc This operation is used to reduce the number of channels and generate a new feature map, denoted as This conversion can be expressed formally as:

[0103]

[0104] in, Afterwards, a feature map containing global context information is obtained through global average pooling and upsampling operations. It can be expressed as:

[0105]

[0106] in, U and Represent upsampling and global average pooling operations respectively.

[0107] also, It also passes through three average pooling operations with a pooling kernel size of 5, and the receptive fields of these three average pooling operations are 5, 9, and 13 respectively. This generates and The three contain local information, which can be expressed as:

[0108]

[0109] in, F avg represents the average pooling operation.

[0110] Finally, the global features And local features and Through the superposition operation, the fused feature map is obtained. Then apply a 1×1 convolution operation to restore the original number of channels to obtain the fused feature map Specifically, it can be expressed as:

[0111]

[0112] in, Indicates an overlay operation.

[0113] In addition to contextual information, enhancing feature representation has become another key method to improve segmentation accuracy. Usually, this operation is achieved by fusing low-level and high-level features in the decoder. Based on the observation of water surface interference, the reflection of the water surface is similar to the real obstacle in features and spatial position, but there are still slight differences, that is, the real obstacle is far away from the water surface while the reflection is close to the water surface.

[0114] Figure 3 It is a diagram of a lightweight feature fusion module based on attention;

[0115] The decoder includes an attention-based lightweight feature fusion module; the attention-based lightweight feature fusion module uses a channel attention mechanism and a spatial attention mechanism to refine features and model the spatial relationship of input features, and can capture the long-range dependency of enhanced features in spatial positions, thereby effectively enhancing feature representation;

[0116] Among them, the channel attention mechanism calculates the importance of each channel and uses the importance as a weight to weight the feature map of each channel. It can select the most discriminative feature channel in the image, thereby enhancing the feature expression of the real obstacle.

[0117] The spatial attention mechanism calculates the importance of each pixel and uses the importance as a weight to weight the feature map. It can pay more attention to the areas with important information in the image when extracting features, especially the spatial positions of real obstacles, and distinguish between real obstacles and water surface disturbances in complex environments.

[0118] In this context, channel attention is mainly responsible for identifying and enhancing the feature information of real obstacles in a specific channel. Meanwhile, the spatial attention mechanism is committed to capturing the feature differences at different locations in the image, paying special attention to the spatial regions that can effectively distinguish obstacles and reflections, and giving higher weights to these key regions.

[0119] The input can be expressed as Initially, these inputs are stacked along the channel dimension to generate feature maps

[0120] after, The input of the channel attention is generated through a 1×1 split convolution operation This conversion can be expressed mathematically as:

[0121]

[0122] in, Represents a 1×1 depthwise separable convolution operation.

[0123] Then, Through global maximum pooling and global average pooling Feature compression is performed to capture two different forms of global spatial information: maximum and average. These compressed features are represented as and Then through the shared multi-layer perceptron T mlpProcessed and normalized using the sigmoid function σ. This normalization step produces a channel attention weight matrix α to highlight the importance of different channels. In order to obtain channel enhanced features Convolutional block, the output of the convolutional block Multiply element-wise with the channel attention weight α. The calculation of channel enhancement features can be expressed as:

[0124]

[0125]

[0126] in, is the feature map obtained in the intermediate process, The dimensions of α are and ⊙ represent element-by-element addition and element-by-element multiplication operations, respectively.

[0127] feature Take the maximum and average values ​​along the channel dimension to get the features and Then concatenate along the channel dimension to get the features The process can be expressed as:

[0128]

[0129] in, Generate feature maps for the intermediate process;

[0130] After 3×3 convolution operation T c And the sigmoid loss function σ, the weight matrix β is obtained.

[0131] By combining the spatial attention weight matrix β with the channel enhancement feature Perform element-by-element multiplication to obtain spatial enhancement features

[0132] Afterwards, the spatial enhancement features With the original features Add element by element to get the final feature The whole process can be expressed mathematically as follows:

[0133]

[0134] in,

[0135] The images captured from the perspective of the unmanned boat mainly contain the sky and the water surface, which leads to a significant class imbalance problem. This imbalance, coupled with the different difficulties in segmenting different ocean scenes, poses a challenge to traditional segmentation methods. The commonly used cross entropy loss function not only cannot effectively solve the class imbalance problem, but is also easily interfered by noise. In order to solve these problems, the present invention adopts two loss functions that are particularly effective in alleviating class imbalance: Dice loss and Focal Loss loss function. Then, these loss functions are combined with the cross entropy loss in a weighted manner to create a new joint loss function.

[0136] To ensure stability when using the joint loss function for training, auxiliary segmentation heads are introduced, inspired by BiSeNetv2. These auxiliary heads are only used during training, and they use the early stage features of RSOS-Net for prediction and provide additional supervision information. By leveraging these early stage features, the auxiliary heads are able to capture finer detail features and alleviate the gradient vanishing problem, thereby stabilizing network training. These auxiliary heads use cross entropy loss as the loss function and are not used during inference.

[0137] The loss function involved above can be expressed as follows:

[0138]

[0139] In the water surface semantic segmentation task, there are three main categories: sky, water and obstacles. Among them, i is the index of the sample, ranging from 1 to the total number of samples, c represents the index of the category, and C is the total number of categories. represents the predicted probability that the i-th pixel belongs to category c, and and N represent the true label of the i-th pixel and the total number of pixels belonging to category c, respectively. In the Focal loss function, γ is an adjustable parameter used to adjust the impact of difficult and easy samples on the loss, which is set to 0.25 here.

[0140] Therefore, the total loss function of model training can be expressed as:

[0141]

[0142] Among them, λ1, λ2 and λ3 are adjustable weight coefficients.

[0143] Example 1: Experimental Evaluation

[0144] A. Datasets and Evaluation Metrics

[0145] All methods are trained and tested using the LaRS dataset, which is currently the most challenging dataset for marine obstacle detection. To ensure fair comparison of different methods, all test results come from a standardized test and evaluator. The LaRS dataset uses the same evaluation method as MODS and includes the mean intersection over union (mIoU) metric as a quantitative benchmark for segmentation accuracy. The main performance metrics include: (i) the accuracy of water edge estimation, calculated by determining the boundary between the water surface and static obstacles. (ii) the accuracy of dynamic obstacle detection. The measure of water edge estimation accuracy is defined as the pixel-by-pixel classification accuracy evaluation within a region of d pixels width around the ground truth waterline, denoted by G d The specific expression is:

[0146]

[0147] Among them, p and g represent the predicted value and the true value respectively.

[0148] The accuracy of dynamic obstacle detection is determined by precision (Pr), recall (Re), and F1 score. The method traverses all dynamic obstacles and regards dynamic obstacles with an overlap ratio between predicted obstacle pixels and real obstacle pixels exceeding θ = 0.7 as true positive examples (TP), otherwise they are regarded as false negative examples (FN). In addition, in order to comprehensively evaluate the inference speed, computational efficiency, and model size of the test method, the present invention uses the number of floating-point operations per second (GFLOPs), the number of frames per second (FPS), and the number of trainable parameters (N Param ) as the key indicator for evaluation.

[0149] All experiments adopted the MMSegmentation framework and utilized the pre-trained backbone network weights. It should be noted that WaSR, WODIS, and eWaSR used their official configuration files, while the remaining methods were trained using the COCOStuff-164k configuration in MMSegmentation and adjusted for the LaRS dataset. The corresponding configurations and hyperparameter settings during training are detailed in Table 1.

[0150] Table 1. Training configuration and hyperparameter settings

[0151] Configuration parameter CPU Intel(R)Xeon(R)CPUE5-2680v4 GPU NVIDIA GeForce RTX 3060 PyTorch 2.0.1 CUDA 11.7 BatchSize 4 Iterations 240000 Optimizer SGD Momentum 0.9 DecayRate 0.0005

[0152] B. Encoder selection and decoder channel number determination

[0153] By testing different baseline models, we can select the best baseline model for RSOS-Net. Figure 4Three lightweight backbone networks are evaluated in the network architecture shown. RSOS-Net adopts an encoder-decoder structure, where the encoder gradually extracts features by downsampling, and finally obtains a feature map with a size of 1 / 32 of the original image. Subsequently, the decoder integrates the features with the encoder output through continuous upsampling and skip connections. It is worth noting that the feature map size of the decoder output is 1 / 8 of the original image, and the number of channels is determined by the output of the 4th stage of the encoder network. Regarding skip connections, Table 2 lists the number of channels used in detail. The analysis shows that using ResNet18 as the encoder network can achieve excellent performance and speed. Specifically, the model shows a high recall rate (59.2) and F1 score (60.9), which is significantly better than the other two encoder networks. In addition, ResNet-18 shows a higher inference speed compared to other encoder networks, making it an ideal choice for real-time applications.

[0154] Table 2 Performance of different feature channels under three different encoders and the same encoder

[0155]

[0156] Figure 7 Comparison of recall rate (Re) and false positive number (FP) of five different baseline models on different obstacle sizes, including (a) recall rate of different obstacle sizes, (b) false positive number of different obstacle sizes.

[0157] Figure 7 The detection rate (i.e., Re) and the number of false positive predictions obtained using the methods in Table 2 for different obstacle sizes are shown. It is worth noting that ResNet18 as an encoder shows significant ability in detecting small obstacles. In addition, it significantly outperforms the MobileNetv3 series in detecting medium and large obstacles.

[0158] It is worth noting that although MobileNetv3 has a higher number of channels than ResNet18 in the final stage, it does not bring significant advantages, but increases the computational cost. In encoder design, the choice of the number of channels is crucial because it directly affects the overall representation and feature extraction capabilities of the model. Although increasing the number of channels may enhance feature representation, it also brings challenges. Specifically, a higher number of channels leads to an increase in the number of parameters and increased computational cost, which hinders optimization and may lead to overfitting.

[0159] In order to optimize the feature channels in the decoder and improve the inference speed of the model, a comparative experiment was conducted to compare two different numbers of channels, 256 and 512. Figure 6The standard convolution is replaced by a depth-wise separable convolution. The results show that using 256 channels in the decoder can achieve the best balance between performance and speed. As shown in Table 2, the F1 score of ResNet18(3rd) only drops by 2.3% compared to ResNet18(1st), but the FPS is significantly improved by 51.7%. Therefore, ResNet18(3rd) is set as the final baseline model.

[0160] C. Ablation Experiment

[0161] Ablation Experiments,Aiming to evaluate the effectiveness of the two proposed modules (i.e., FPPM and AFFM) through,ablation experiments. To ensure a fair comparison, the ResNet18 (3rd) described in the previous section,is selected as the baseline model, keeping the same training and inference,configuration throughout the whole process.

[0162] The comparison results are shown in Table 3. It can be found that integrating FPPM into the baseline model significantly improves the performance, with the recall rate increased by 2.3% and the F1 score improved by 0.7%. However, the FPS dropped slightly by 1.67%, and the μ value remained almost unchanged, only dropping by 0.69%. Overall, FPPM effectively improves model performance without affecting efficiency.

[0163] Table 3 Ablation experiments of FPPM and AFFM modules

[0164]

[0165] In addition, integrating AFFM alone into the baseline model also achieves significant performance improvements, reaching a μ value of 72.9, an F1 score of 58.8, and improved precision and recall. These results clearly demonstrate the effectiveness of AFFM in improving the overall performance of the model. However, it is worth noting that this integration leads to a significant decrease in inference speed. Preliminary analysis shows that while the spatial attention mechanism in AFFM is very effective in modeling spatial relationships and enhancing feature importance, it incurs higher computational costs compared to the channel attention mechanism.

[0166] By combining FPPM and AFFM, RSOS-3 improves precision (Pr) by 0.6%, recall (Re) by a significant 4.7%, and F1 score by 2.8%. Although the inference speed drops by 16.99% compared to the baseline model, RSOS-3 still maintains a real-time detection speed of 79.5FPS, ensuring real-time detection of obstacles in practical scenarios. Qualitative comparison (such as Figure 5 The results of the previous study further highlight the effectiveness of FPPM and AFFM in mitigating the problem of misidentification, especially in cases caused by water surface reflections.

[0167] However, in certain scenarios, such as low light conditions or when the obstacle reflection is spatially close to the actual obstacle, FPPM or AFFM alone cannot solve this problem. This limitation stems from the fact that FPPM only combines global and local semantic information. Although it helps to understand the overall scene, it cannot distinguish between visually similar and spatially close objects. On the basis of FPPM, combined with AFFM, it is possible to model the spatial relationship of features, enabling it to dynamically focus on the importance and uniqueness of the feature spatial position. Therefore, real obstacles, reflections, and wakes left by ships can be better distinguished. In summary, the FPPM and AFFM modules proposed in the present invention show high efficiency in obstacle detection.

[0168] D. Comparative Experiments with State-of-the-Art Segmentation Networks

[0169] In this section, RSOS-Net is compared and analyzed with ten state-of-the-art segmentation networks. These networks include ICNet, Fast-SCNN, Semantic FPN, BiSeNet series, STDC, and DDRNet. These networks were selected because they can maintain advanced detection performance while maintaining real-time capabilities, and are not only suitable for segmentation tasks of autonomous vehicles, but also for general segmentation scenarios. When selecting the encoders for these networks, the official configuration files were used to achieve the best balance between performance and speed. In addition, in order to expand the scope of comparison, the present invention also selected WaSR, which is specifically used in the maritime field, its lightweight version eWaSR, and WODIS. To ensure the fairness of the comparison, versions of WaSR and eWaSR that do not contain the IMU feature encoder were selected, and the encoder selection strictly followed the official configuration files.

[0170] The test results are shown in Table 4. To evaluate the model performance, the fastest version RSOS-1 and the most accurate version RSOS-3 were selected from RSOS-Net. Among all the methods, RSOS-3 performed best in terms of precision and F1 score. Among the general semantic segmentation methods, Semantic FPN and BiSeNetv1 performed well, with water edge estimation accuracy (μ) reaching 72.9 and 72.8 respectively, and also performed well in other indicators. DDRNet and STDC performed relatively well in terms of recall and precision, with recall and precision values ​​of 60.1 and 63.0 respectively. In the maritime field, WaSR and eWaSR also performed well. It is worth noting that WaSR ranked third in F1 score and fourth in recall, while achieving the best mIoU. WODIS performed well in recall, but weak in precision.

[0171] Table 4 Performance comparison between different methods and RSOS-Net

[0172]

[0173]

[0174] Furthermore, to fully evaluate the performance of these methods, their model size, computational efficiency, and inference speed on CPU and GPU architectures were analyzed in detail, and the results are summarized in Table 5. Among the tested methods, Fast-SCNN, although slightly lagging behind in performance, requires the least computational resources and achieves the fastest inference speed on the GPU architecture. However, despite its slightly larger model size, eWaSR performs well in terms of computational efficiency and demonstrates the fastest inference speed on the CPU architecture. In addition, despite the fact that the WODIS model is the largest, it is still able to achieve real-time inference speed. The remaining general real-time segmentation methods as well as RSOS-Net are in the middle, with relatively small model sizes while maintaining satisfactory processing speed. However, when considering both speed and performance, RSOS-Net achieves the best balance by introducing the least additional trainable parameters.

[0175] Table 5 Comparison of different methods and RSOS-Net in computational efficiency

[0176] method Input resolution <![CDATA[NN param ]]> GFLOPs FPS(GPU) FPS(CPU) ICNet 1024×512 47.5M 30.8 57.2 1.91 Fast-SCNN 1024×512 1.4M 1.9 118.9 3.83 SemanticFPN 1024×512 28.5M 90.9 35.5 1.34 BiSeNetv1 1024×512 13.3M 29.7 95.3 2.48 BiSeNetv2 1024×512 3.3M 24.6 81.2 2.28 STDC 1024×512 8.3M 16.9 92.4 3.04 DDRNet 1024×512 20.3M 36.1 73.3 2.22 Wlq 512×384 71.5M 399.3 9.8 0.51 WODIS 512×384 89.5M 123.5 38.0 2.87 wxya 512×384 60.3M 12.1 69.2 4.1 RSOS-1 1024×512 12.3M 26.2 94.2 2.45 RSOS-2 1024×512 12.1M 26.0 80.1 2.32 RSOS-3 1024×512 12.4M 26.2 79.5 2.31

[0177] E. Robustness Analysis

[0178] As shown in Table 6, in order to further evaluate the robustness of various methods in complex scenes, the top four real-time segmentation methods ranked by F1 score were selected for quantitative comparison with RSOS-3. In addition, among the methods designed for water scenes, eWaSR with the best overall performance was selected for quantitative comparison with RSOS-3. Fig. 9 In the , the performance related to scene attributes is analyzed, which reflects the adaptability and robustness of different methods in different environments. It is worth noting that RSOS-3 and STDC perform well in processing water reflections, indicating the effectiveness of the proposed module. All methods perform relatively well in dusk scenes, while RSOS-3 and STDC outperform other methods in night scenes. In addition, RSOS-3 performs well in low-light and ship wake scenes, and performs particularly well in sun flash conditions. Figure 6 The above results were qualitatively analyzed.

[0179] Table 6 Comparison of anti-interference capabilities between different methods and RSOS-Net

[0180]

[0181] In order to compare the performance of different methods in detecting small obstacles, the top four real-time segmentation methods based on F1 ranking were selected for comparison with RSOS-3. In addition, among the dedicated water surface obstacle segmentation methods, the real-time eWaSR and WODIS were selected for comparison with RSOS-3.

[0182] Fig.10 Comparison of F1 scores and false positive numbers (FP) of different methods for different obstacle sizes. (a) F1 scores for different obstacle sizes. (b) False positive numbers for different obstacle sizes.

[0183] Fig.10 The specific effects of different obstacle sizes on the F1 score and the number of false positives (FP) of each method are shown. Fig.10 As shown in Figure 2, RSOS-3, DDRNet, and Semantic FPN significantly outperform other methods in detecting small obstacles. Figure 6 The qualitative analysis in shows that only RSOS-3 and Semantic FPN successfully detect multiple small obstacles. In contrast, the other methods perform poorly in detecting small obstacles.

[0184] In summary, RSOS-Net performs well in solving the common segmentation defect problems in water surface obstacle detection.

[0185] F. Deploy RSOS-Net in a port environment and conduct testing

[0186] In this section, the model performance is tested using an unmanned boat in a port environment. The unmanned boat platform adopts a catamaran configuration and is equipped with a 1080P network camera for environmental perception. The network camera communicates with the host computer through a local area network. Figure 8 The unmanned boat platform and experimental environment are demonstrated. RSOS-Net is converted into ONNX format and deployed on a laptop equipped with an NVIDIA GeForce RTX 3060 GPU. The video stream obtained through the webcam communicates with the laptop to test RSOS-Net. Fig. 9 As shown, RSOS-Net performs well in practical scenarios.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0188] [1]D.K.Prasad,C.K.Prasath,D.Rajan,L.Rachmawati,E.Rajabally,andC.Quek,"Object detection in a maritime environment:Performance evaluationofbackground subtraction methods,"IEEE Trans.Intell.Transp.Syst.,vol.20,no.5,pp.1787–1802,May.2019.

[0189] [2]J.Muhoviˇc,R.Mandeljc,B.Bovcon,M.Kristan,and J. "Obstacletracking for unmanned surface vessels using 3-d point cloud,"IEEEJ.Ocean.Eng.,vol.45,no.3,pp.786–798,Jul.2020.

[0190] [3]M.Kristan,V.Suli cKenk,S.Kovaˇciˇc,and J. "Fast image-basedobstacle detection from unmanned surface vehicles,"IEEE Trans.Cybern.,vol.46,no.3,pp.641–654,Mar.2016.

[0191] [4]B.Bovcon and M.Kristan,"Obstacledetection for usvs by jointstereoview semantic segmentation,"in IEEE / RSJ,Int.Conf.Intell.Robots Syst.(IROS),Madrid,Spain,Oct.2018,pp.5807–5812.

[0192] [5]T.Cane and J.Ferryman,"Evaluating deep semantic segmentationnetworks for object detection in maritime surveillance,"in Proc.15th IEEEInt.Conf.Adv.Video Signal Based Surveill.(AVSS),Auckland,New Zealand,Nov.2018,pp.1–6.

[0193] [6]B.Bovcon,J.Muhoviˇc,J. and M.Kristan,"The Mastr1325dataset fortraining deep usv obstacledetection models,"in Proc.IEEE / RSJInt.Conf.Intell.Robots Syst.(IROS),Macau,China,Nov.2019,pp.3431–3438.

[0194] [7]L.Yao,D.Kanoulas,Z.Ji,and Y.Liu,"ShorelineNet:An efficient deeplearning approach for shoreline semantic segmentation for unmanned surfacevehicles,"in IEEE / RSJ,Int.Conf.Intell.Robots Syst.(IROS),Sep.2021,pp.5403–5409.

[0195] [8]X.Chen,Y.Liu,and K.Achuthan,"WODIS:Water obstacledetection networkbased on image segmentation for autonomous surface vehicles in maritimeenvironments,"IEEE Trans.Instrum.Meas.,vol.70,pp.1–13,Jun.2021.

[0196] [9]T.Matija,v.Lojze,and M.Kristan,"eWaSR-An embedded-computereadymaritime obstacledetection network,"vol.23,no.12,Jun.2023.[Online].Available:https: / / doi.org / 10.48550 / arXiv.2304.11249.

[0197]

[10] R.Zhou,Y.Gao,Y.Wang,X.Xie,and X.Zhao,"A Real-time scene parsingnetwork for autonomous maritime transportation,"IEEE Trans.Instrum.Meas.,vol.72,pp.1–14,May.2023.

[0198]

[11] T.-Y.Lin,P.Dollar,R.Girshick,K.He,B.Hariharan,and S.Belongie,"Feature pyramid networks for object detection,"in Proc.IEEEConf.Comput.Vis.Pattern Recog.(CVPR),Honolulu,HI,USA,Jul.2017,pp.2117–2125.

[0199]

[12] K.He,X.Zhang,S.Ren,and S.Jian,"Deep residual learning for imagerecognition,"in Proc.IEEE Conf.Comput.Vis.Pattern Recog.(CVPR),Seattle,WA,USA,Jun.2016,pp.770–778.

[0200]

[13] F.Chollet,"Xception:Deep learning with depthwise separableconvolutions,"in Proc.IEEE Conf.Comput.Vis.Pattern Recog.(CVPR),Honolulu,HI,USA,Jul.2017,pp.1800–1807。

Claims

1. A method for anti-interference real-time obstacle segmentation on the surface of an unmanned watercraft based on context information and attention mechanism, characterized by: The following steps are involved: Obtain a data set consisting of water surface environment images captured from the perspective of an unmanned boat; Divide the data set into training set and test set according to the proportion; Construct a real-time water surface obstacle segmentation network model, which can be used to segment obstacles based on visual information in the presence of water surface interference; Based on the training set data, the real-time water surface obstacle segmentation network is trained to obtain a trained real-time water surface obstacle segmentation network model; The test set data is input into the trained real-time surface obstacle segmentation network model to realize the anti-interference real-time obstacle segmentation of the unmanned boat surface.

2. According to claim 1, a method for anti-interference real-time obstacle segmentation of an unmanned watercraft based on context information and attention mechanism is characterized by: The real-time water surface obstacle segmentation network model adopts an encoder-decoder architecture. The encoder part uses the ResNet18 lightweight backbone network. A fast pyramid pooling module is introduced in the last stage of the encoder to distinguish water surface glare from real obstacles through local and global context features, thereby reducing false alarms caused by water surface glare. An improved lightweight feature pyramid network is used in the decoder part to solve the significant multi-scale imbalance problem of water surface obstacles. Finally, a feature fusion module based on the channel-spatial attention mechanism is used to effectively identify the subtle differences between real obstacles and water surface reflections in terms of detail features and spatial positions, thereby reducing false alarms caused by water surface reflections.

3. According to claim 2, a method for anti-interference real-time obstacle segmentation of an unmanned watercraft based on context information and attention mechanism is characterized by: The encoder includes four convolutional layers of the ResNet18 backbone network and a fast pyramid pooling module. The four convolutional layers and the fast pyramid pooling module of the ResNet18 backbone network are sequentially connected; The fast pyramid pooling module uses cascade average pooling operations to expand the receptive field, capture multi-scale local information, and obtain global context information through global average pooling, so that obstacles and water surface glare can be accurately distinguished. The pooling layer with a smaller receptive field can focus on the local area of ​​the image and capture detailed information about obstacles and water surface glare. At the same time, the pooling layer with a larger receptive field can use global context information to identify the overall shape, brightness and relationship between obstacles and the surrounding environment. Comparing and analyzing this information at different scales helps the model distinguish water surface glare from obstacles.

4. According to claim 3, a method for anti-interference real-time obstacle segmentation of an unmanned watercraft based on context information and attention mechanism is characterized in that: The fast pyramid pooling module uses cascaded average pooling operations to expand the receptive field, capture multi-scale local information, and obtain global context information through global average pooling. The process of splicing along the channel dimension for feature fusion is as follows: The input is represented as Where c, h, and w represent the number of channels, height, and width of the image, respectively; first, Through a 1×1 convolution operation, denoted as F c , this operation is used to reduce the number of channels and generate a new feature map, denoted as This conversion is formally expressed as: in, Afterwards, a feature map containing global context information is obtained through global average pooling and upsampling operations. It is expressed as: in, U and Represent upsampling and global average pooling operations respectively; also, It also passes through three average pooling operations with a pooling kernel size of 5, and the receptive fields of these three average pooling operations are 5, 9, and 13 respectively; thus generating and Three feature maps containing local information are specifically expressed as: in, F avg represents the average pooling operation; Finally, the global features And local features and Through the superposition operation, the fused feature map is obtained. Then apply a 1×1 convolution operation to restore the original number of channels to obtain the fused feature map Specifically expressed as: in, Indicates an overlay operation.

5. According to claim 2, a method for anti-interference real-time obstacle segmentation of an unmanned watercraft based on context information and attention mechanism is characterized in that: The decoder includes an attention-based lightweight feature fusion module; the attention-based lightweight feature fusion module uses a channel attention mechanism and a spatial attention mechanism to refine features and model the spatial relationship of input features, and can capture the long-range dependency of enhanced features in spatial positions, thereby effectively enhancing feature representation; Among them, the channel attention mechanism calculates the importance of each channel and uses the importance as a weight to weight the feature map of each channel. It can select the most discriminative feature channel in the image, thereby enhancing the feature expression of the real obstacle. The spatial attention mechanism calculates the importance of each pixel and uses the importance as a weight to weight the feature map. It can pay more attention to areas with important information in the image when extracting features, especially the spatial position of real obstacles, and distinguish between real obstacles and water surface disturbances in complex environments.

6. The method for anti-interference real-time obstacle segmentation of an unmanned watercraft surface based on context information and attention mechanism according to claim 2 is characterized by: The channel attention mechanism calculates the importance of each channel and uses the importance as a weight to weight the feature map of each channel, so as to select the most discriminative feature channel in the image, thereby enhancing the feature expression of the real obstacle; The spatial attention mechanism calculates the importance of each pixel and uses the importance as a weight to weight the feature map. It can pay more attention to the areas with important information in the image when extracting features, especially the spatial position of real obstacles. The process of distinguishing real obstacles from water surface disturbances in complex environments is as follows: The input is represented as The input is stacked along the channel dimension to generate a feature map after, The input of the channel attention is generated through a 1×1 split convolution operation This conversion is expressed mathematically as: in, F S Represents a 1×1 depthwise separable convolution operation; Then, Through global maximum pooling and global average pooling Feature compression is performed to capture global spatial information in two different forms: maximum and average. These compressed features are represented as and Then through the shared multi-layer perceptron T mlp Processed and normalized using the sigmoid function σ; a channel attention weight matrix α is generated to highlight the importance of different channels; in order to obtain channel enhancement features Output of the convolutional block Multiplying element-by-element with the channel attention weight α, the calculation of the channel enhancement feature is expressed as: in, is an intermediate feature, The dimensions of α are and ⊙ represent element-by-element addition and element-by-element multiplication operations, respectively. feature Take the maximum and average values ​​along the channel dimension to get the features and Then concatenate along the channel dimension to get the feature map The process is expressed as: in, is an intermediate feature; After 3×3 convolution operation T c And the sigmoid loss function σ, the weight matrix β is obtained; By combining the spatial attention weight matrix β with the channel enhancement feature Perform element-by-element multiplication to obtain spatial enhancement features Afterwards, the spatial enhancement features With the original features Add element by element to get the final feature The whole process can be expressed mathematically as follows: in, 7. The method for anti-interference real-time obstacle segmentation of an unmanned watercraft based on context information and attention mechanism according to claim 1 is characterized by: The loss function of the real-time water surface obstacle segmentation network model is expressed as follows: Where i is the index of the sample, ranging from 1 to the total number of samples, c represents the index of the category, and C is the total number of categories; represents the predicted probability that the i-th pixel belongs to category c, and represents the true label of the i-th pixel. In formula (25), γ is an adjustable parameter used to adjust the impact of difficult and easy samples on the loss, which is set to 0.25 here; Therefore, the total loss function of the real-time water surface obstacle segmentation network model training is expressed as: Among them, λ1, λ2 and λ3 are adjustable weight coefficients.

Images