Complex sea area unmanned aerial vehicle image multi-scale feature modeling adaptive segmentation method

Abstract

The invention discloses a complex sea area unmanned aerial vehicle image multi-scale feature modeling adaptive segmentation method. The method comprises the following steps: A, obtaining an unmanned aerial vehicle image data set, and constructing a training set, a verification set and a test set; b, establishing an adaptive segmentation neural network model for unmanned aerial vehicle image multi-scale feature modeling in a complex sea area scene; c, training the adaptive segmentation neural network model by using the training set; d, optimizing the trained neural network model by using the test set; and E, obtaining a to-be-detected unmanned aerial vehicle image, and performing semantic segmentation on the to-be-detected unmanned aerial vehicle image by using the optimized adaptive segmentation neural network model to obtain a corresponding segmentation result. According to the method, the unmanned aerial vehicle image semantic segmentation performance in a complex sea area scene can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of semantic segmentation, and particularly to an adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas. Background Art

[0002] With the rapid development of UAV technology, the application scope of UAVs in fields such as urban planning, agricultural monitoring, environmental protection, and disaster assessment has been expanding day by day, and it has become an important means to obtain high-resolution surface information. Among them, UAV image semantic segmentation, as one of the important tasks of the UAV vision system, performs pixel-level classification on high-resolution images captured by UAVs, extracts ground object features, and provides data support for subsequent ground object detection, target tracking, and intelligent decision-making. However, the characteristics of UAV images make the semantic segmentation task full of challenges. On the one hand, during the flight of the UAV, due to the changing shooting perspectives, the shape and scale of the target objects in the image will change significantly; on the other hand, there are a large number of complex background interferences in complex sea area scenes, such as water body reflection, wave texture, cloud shadow superposition, etc., and these factors further exacerbate the difficulty of the segmentation task. In addition, the scale distribution of objects in UAV images is uneven, usually including large-scale background areas and small-scale key targets, which puts higher requirements on the multi-scale feature modeling ability of the model. In recent years, deep learning technology has made remarkable breakthroughs in the field of semantic segmentation. Segmentation models based on convolutional neural networks (CNNs) (such as UNet, DeepLab series) perform excellently in feature extraction and pixel-level classification due to their encoder-decoder structure. However, these methods still have limitations when facing complex sea area scenes: the traditional convolutional operation has insufficient ability to model global context information and is difficult to effectively capture long-range dependencies between channels.

[0003] The invention patent with the application number 202410119623.4 and the name "A SAR Marine Oil Spill Image Segmentation Method Based on Improved U-Net" discloses the following technical solution: First, obtain the original SAR marine oil spill image and the SAR marine oil spill image with the oil spill area marked, form a SAR marine oil spill dataset, and divide it into a training set and a test set after preprocessing. Subsequently, improve the U-Net semantic segmentation model based on the CoordAttention module, use the VGG16 structure as the encoder, and transfer the ImageNet pre-trained weights to the encoding part to enhance the feature extraction ability. In the decoder part, introduce the CoordAttention module (CA module) before each feature fusion to improve the attention to spatial information and channel information. Finally, train to obtain the weight file of the improved U-Net model and apply it to the automatic segmentation of SAR marine oil spill images, significantly improving the detection accuracy of the oil spill area.

[0004] Although the above-mentioned U-Net-based image segmentation method has achieved good performance in many fields, it still has certain limitations due to its traditional convolutional neural network (CNN) architecture. Traditional U-Net mainly relies on convolutional operations, with a limited receptive field, making it difficult to fully model global context information. In addition, images in complex sea area scenes have the characteristics of high noise and blurred boundaries, and ordinary convolutional operations are difficult to effectively distinguish the real oil spill area from other low-wind speed or natural dark spot areas, thus affecting the segmentation accuracy. Although the improved U-Net introduces the CoordAttention module to enhance feature extraction, its ability to capture long-range dependencies is still limited, and there may still be problems of false detection or missed detection in more complex marine environments. Summary of the Invention

[0005] The purpose of the present invention is to provide an adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas, which can significantly improve the semantic segmentation performance of UAV images in complex sea area scenes, has higher accuracy, stronger robustness and better computational efficiency, is applicable to a variety of complex application scenarios, and has broad prospects for promotion.

[0006] The present invention adopts the following technical solutions:

[0007] An adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas, comprising the following steps:

[0008] A: Obtain a UAV image data set, and construct a training set, a validation set and a test set after preprocessing;

[0009] B: Establish an adaptive segmentation neural network model for multi-scale feature modeling of UAV images in complex sea area scenes; the adaptive segmentation neural network model consists of an encoder and a decoder, uses ResNet-50 as the backbone network, and combines an adaptive channel attention module and a multi-scale moving window attention module;

[0010] C: Use the training set to train the adaptive segmentation neural network model;

[0011] D: Use the test set to optimize the trained neural network model;

[0012] E: Obtain the UAV image to be detected, and use the optimized adaptive segmentation neural network model to perform semantic segmentation on the UAV image to be detected to obtain the corresponding segmentation result.

[0013] In step B, the encoder includes several groups of stacked ResNet blocks, an adaptive channel attention module, and a multi-scale moving window attention module; the ResNet blocks are used to perform preliminary feature extraction on the input image; the adaptive channel attention module is used to dynamically adjust the weights of each channel, enhance the response of key information channels, and suppress the interference of noise and irrelevant features; the multi-scale moving window attention module uses moving window mechanisms of different scales to capture long-range dependencies in the image, retain local details while integrating global context information.

[0014] In step B, the decoder is used to gradually fuse and upsample the features extracted by the encoder, restore the image to the original input size, and finally obtain the segmentation result of the image after being processed by the segmentation head.

[0015] The adaptive channel attention module generates channel-level attention weights through global average pooling, one-dimensional adaptive convolution, and the Sigmoid activation function, and applies them to the input feature map to enhance key channels and suppress non-key channels.

[0016] The adaptive channel attention module first receives the preliminary feature map from the ResNet blocks; then performs global average pooling on the input feature map in the spatial dimension, compresses all spatial information of each channel into a single value to obtain a channel descriptor of a set size; subsequently, uses one-dimensional adaptive convolution to process the channel descriptor, models the interdependencies between channels, and dynamically adjusts the size of the convolution kernel according to the complexity of the input image; then expands the channel weights after convolution operations to the same spatial size as the input feature map, and normalizes them through the Sigmoid activation function to obtain the importance coefficient of each channel; finally, multiplies the input feature map element-wise by the channel weights to achieve feature recalibration and enhance the response of key channels, and finally outputs the feature map after weight adjustment.

[0017] The multi-scale moving window attention module first receives the feature map processed by the ResNet blocks and the adaptive channel attention module, and divides it into multiple patches; then uses a preset number of different-sized windows to apply the moving window attention mechanism for feature extraction respectively; at each scale, the feature map is divided into multiple windows, and self-attention is calculated inside the windows; subsequently, after the attention calculation is completed, the window slides by half of its own size and re-divides the window, and the attention inside the window is calculated again; then after the attention calculations at all scales, the weighted feature maps of different scales are stitched and preliminarily convolved to achieve fusion, and then further convolved after layer normalization and adding non-linearity using the ReLU activation function to fuse features, and residual link with the input feature map; finally, it outputs a feature map that fuses local details and global dependencies.

[0018] In step C, the normalized cross - entropy loss function is used during the training of the adaptive segmentation neural network model to normalize the prediction probabilities.

[0019] In step D, when optimizing the trained adaptive segmentation neural network model, the F1 - score, accuracy, and mean intersection over union are used as evaluation metrics for model evaluation.

[0020] Step A includes the following specific steps:

[0021] A1: Obtain the UAV image dataset;

[0022] A2: Preprocess the UAV image dataset, including data augmentation and size adjustment;

[0023] A3: Divide the preprocessed dataset into a training set, a validation set, and a test set according to a ratio.

[0024] The UAV image dataset is constructed based on the OUC - UAV - SEG dataset.

[0025] Through the adaptive channel attention module and the multi - scale moving window attention module, the present invention can effectively capture multi - scale features, channel relationships, and long - range dependence information, thereby enhancing the model's ability to distinguish complex ground object features in UAV images, significantly improving the segmentation accuracy, especially showing better performance for small targets and edge regions. Secondly, using the multi - level feature fusion design, the feature information of different levels is retained through residual connections, and the decoder gradually fuses and upsamples the feature maps, realizing the effective combination of local details and global context information, making the segmentation results more refined. In addition, the present invention combines the normalized cross - entropy loss function to handle the class imbalance problem and uses the Adam optimizer to accelerate the training, enabling the model to significantly reduce the computational complexity while ensuring high accuracy, and being applicable to resource - constrained UAV platforms.

[0026] The present invention also improves the model's adaptability to complex scenarios through diversified enhancement processing of the UAV image dataset, showing good robustness in multiple fields such as urban planning and agricultural monitoring. At the same time, based on comprehensive performance evaluation and optimization, the present invention ensures the high accuracy and consistency of the segmentation results, providing reliable technical support for the subsequent analysis and application of UAV images. Brief Description of the Drawings

[0027] Figure 1 is a flowchart of the present invention;

[0028] Figure 2 is a structural diagram of the adaptive segmentation neural network model of the present invention;

[0029] Figure 3Schematic diagram of the adaptive channel attention module in the present invention;

[0030] Figure 4 Schematic diagram of the UAV image in the present invention. Specific implementation manners

[0031] The present invention will be described in detail below with reference to the accompanying drawings and embodiments:

[0032] As Figures 1 to 4 shown, the multi-scale feature modeling adaptive segmentation method for UAV images in complex sea areas of the present invention includes the following steps:

[0033] A: Obtain a UAV image dataset, and construct a training set, a validation set, and a test set after preprocessing;

[0034] Among them, the UAV image dataset is derived from the OUC-UAV-SEG dataset; during preprocessing: First, perform data augmentation operations such as flipping and rotating the UAV images in the dataset to expand the diversity and complexity of the dataset, enhance the adaptability of the model to images with different perspectives and directions, thereby improving the generalization ability of the model and effectively reducing overfitting. Subsequently, crop all the augmented images to a unified predefined size to ensure that the input images have consistent spatial dimensions and facilitate model training. Finally, divide the dataset into a training set, a validation set, and a test set according to a ratio.

[0035] In this embodiment, step A includes the following specific steps:

[0036] A1: Obtain a UAV image dataset.

[0037] In this embodiment, the OUC-UAV-SEG dataset is selected as the dataset for the multi-scale feature modeling adaptive segmentation method for UAV images in complex sea areas. This dataset is specifically designed for complex sea area scenarios and contains 2,400 UAV images with a resolution of 1920×1080, covering a variety of complex sea area environments, such as sea waves, reefs, ships, buoys, marine vegetation, etc. The dataset provides real masks for 15 categories, including sea water, sky, ships, buoys, reefs, marine vegetation, etc., and can effectively support semantic segmentation tasks in complex sea area scenarios. The diversity and complexity of this dataset provide rich scene information for model training and are especially suitable for the research of multi-scale feature modeling and adaptive segmentation methods.

[0038] A2: Preprocess the UAV image dataset.

[0039] To ensure the consistency of the model's input images, provide standardized inputs for model training, and reduce the number of computational parameters, each drone image in the drone dataset is preprocessed. The preprocessing steps aim to enhance the diversity of the images while retaining the key features of the complex sea area scenes. The specific preprocessing steps are as follows:

[0040] First, to overcome the problems of sparse dataset and single samples, and improve the robustness and generalization ability of the model, this embodiment uses a variety of data augmentation techniques for the drone sea area images in the dataset to simulate various changes in the complex sea area scenes. The augmentation operations include: randomly rotating the image by a certain angle (such as -30° to 30°) to simulate the sea area scenes captured by the drone at different angles; randomly flipping the image horizontally or vertically to increase the data diversity while retaining the spatial structure of the sea area scenes; randomly adjusting the brightness and contrast of the image to simulate the sea area scenes under different lighting conditions, such as cloudy days, sunny days, or dusk; randomly adding Gaussian noise to the image to simulate the image noise problems that the drone may encounter in the complex sea area environment. Through these data augmentation methods, the model can learn image features more comprehensively, avoid overfitting, and thus improve its generalization ability in complex environments.

[0041] Second, through randomly generated scale parameters, the enhanced images are cropped and scaled to meet the requirements of the model input. The specific operations include: randomly cropping multiple regions from the enhanced images. To ensure the training speed and uniformity of the model, all images are cropped into image patches of a preset size of 256×256, and the cropping step size is To prevent the problem of blurred boundaries and ensure that the cropped regions contain the key features of the complex sea area scenes, such as waves, ships, or reefs;

[0042] The enhanced images are enlarged through randomly generated scale parameters to finally obtain the preprocessed images.

[0043] A3: The dataset after data augmentation and size adjustment is divided into three parts: a training set, a validation set, and a test set in a ratio of 8:1:1.

[0044] B: Establish an adaptive segmentation neural network model for multi-scale feature modeling of drone images in complex sea area scenes; among them, the neural network model consists of an encoder and a decoder, uses ResNet-50 as the backbone network, and combines an adaptive channel attention module and a multi-scale moving window attention module to achieve efficient feature modeling and accurate segmentation of complex sea area scenes. Its structural schematic diagram is as Figure 2 shown.

[0045] In the present invention, the specially designed multi-scale feature extraction and adaptive attention mechanism can address challenges such as diverse target scales, complex backgrounds, and significant illumination changes in complex sea area scenes, and achieve efficient feature modeling and precise segmentation of complex sea area scenes.

[0046] In the present invention, the encoder includes 4 groups of stacked ResNet blocks, an adaptive channel attention module, and a multi-scale moving window attention module; the ResNet blocks are used to perform preliminary feature extraction on the input image, extract shallow and middle-level features in the image, provide rich initial feature representations for subsequent processing, and ensure the efficient transmission of information; based on the preliminary features extracted by the ResNet blocks, the adaptive channel attention module is used to dynamically adjust the weights of each channel, enhance the response of key information channels, and at the same time suppress the interference of noise and irrelevant features. This module enables the model to focus more on the feature expression of key targets when facing complex sea area scenes, and improves the overall feature identification ability;

[0047] Subsequently, the feature map is mapped into patches and sent to the multi-scale moving window attention module. The multi-scale moving window attention module uses moving window mechanisms of different scales to capture long-range dependencies in the image, which can not only retain local details but also integrate global context information, so as to better meet the segmentation requirements of multi-scale targets in complex scenes.

[0048] The four groups of ResNet blocks and the adaptive channel attention module can extract feature information of different scale levels and save the features at each stage to cooperate with the residual connection part in the decoder.

[0049] In the present invention, the decoder restores the image to the original input size by gradually fusing and upsampling the features extracted in the encoder, and finally obtains the segmentation result of the image after being processed by the segmentation head.

[0050] Specifically, after receiving the patches processed by the multi-scale moving window attention module, the decoder unfolds the patches into the shape of the original feature map, stitches them with the features of the corresponding scale in the encoder, then uses convolution to adjust the number of channels and performs gradual upsampling, and finally restores to the original size of the input image. After being mapped to 15 categories by the segmentation head, the final segmentation result map is obtained.

[0051] In this embodiment, the adaptive channel attention module is a feature enhancement module specially designed for complex sea area scenes, aiming to dynamically adjust the weights of each channel in the feature map to enhance the ability to focus on key targets and at the same time suppress the interference of background noise. The adaptive channel attention module generates channel-level attention weights through global average pooling, one-dimensional adaptive convolution, and the Sigmoid activation function, and applies them to the input feature map to achieve the enhancement of key channels and the suppression of non-key channels.

[0052] The structural diagram of the adaptive channel attention module is as Figure 3 shown. The module first receives the preliminary feature map from the ResNet block, with a shape of [H, W, C], where C is the number of channels, and H and W are the height and width of the feature map respectively. Then, global average pooling is performed on the input feature map in the spatial dimension to compress all spatial information of each channel into a single value, obtaining a channel descriptor of size [C, 1, 1]. This process can capture the global statistical information of each channel and provide a basis for subsequent channel modeling. On this basis, a one-dimensional adaptive convolution is used to process the channel descriptor to model the interdependence between channels, and the size of the convolution kernel is dynamically adjusted according to the complexity of the input image, so as to achieve a balance between local details (such as the texture features of small targets) and global context (such as the overall shape and background information). Subsequently, the channel weights after the convolution operation are expanded to the same spatial size as the input feature map and normalized to the range of 0 to 1 through the Sigmoid activation function to obtain the importance coefficient of each channel. Finally, the input feature map is multiplied element-wise by the channel weights to achieve feature recalibration, enhance the response of key channels, and reduce the influence of redundant channels at the same time, enabling the network to more effectively distinguish targets and backgrounds in complex sea area scenes, and finally output the feature map after weight adjustment, providing a more discriminative feature representation for subsequent long-range dependence modeling.

[0053] The overall operation process of the adaptive channel attention module can be expressed as:

[0054] F ACA = F res ⊙ Expand{σ[Conv 1D [GAP(F res )]]}; (1)

[0055] In formula (1), F ACA represents the output of the adaptive channel attention module, F res represents the preliminary feature map processed by the ResNet block, ⊙ is element-wise multiplication, F res ∈ R H×W×C , R H×W×C represents the shape of F res , H, W are the height and width of the feature map, C is the number of feature map channels, Expand means expanding the channel weights to H×W, σ is the Sigmoid activation function, Conv 1D represents one-dimensional adaptive convolution for cross-channel interaction, and GAP represents the global average pooling operation.

[0056] In the sea area scenario, background noises such as wave textures and lighting changes will interfere with target recognition. The adaptive channel attention module reduces the impact of background noise on the model by suppressing the weights of non-critical channels, improving the purity of the segmentation results. In complex sea area scenarios, the target scale changes significantly (such as a ship nearby and a buoy in the distance). The adaptive channel attention module dynamically adjusts the convolution kernel size through one-dimensional adaptive convolution to adapt to the feature distribution of targets at different scales, enhancing the multi-scale modeling ability of the model.

[0057] In this embodiment, the multi-scale moving window attention module is a long-range dependence modeling module designed specifically for complex sea area scenarios. It aims to capture local and global features in the image through the multi-scale moving window attention mechanism. At the same time, compared with the traditional self-attention mechanism, it reduces the computational complexity. The multi-scale moving window attention module extracts multi-scale target features and long-range dependence relationships in complex sea area scenarios through the moving window attention mechanism at multiple scales, adapting to the characteristics of diverse target scales, complex backgrounds, and significant lighting changes in sea area scenarios.

[0058] The structural diagram of the multi-scale moving window attention module is as Figure 4As shown, the multi-scale moving window attention module receives the feature maps processed by the stacked ResNet blocks and the adaptive channel attention module, and divides them into multiple small patches for window-level attention calculation. Subsequently, to meet the information modeling requirements at different scales, multiple windows of different sizes (such as window sizes 4, 6, and 8) are preset inside the module, and the moving window attention mechanism is applied respectively for feature extraction. At each scale, the feature map is divided into multiple windows, and self-attention is calculated inside the windows. The specific process includes generating Query, Key, and Value matrices through linear projection, calculating the attention weights inside the windows in combination with relative position biases, and obtaining the feature interaction relationship inside the windows through Softmax normalization. This process can effectively extract the detailed information of local regions, enabling the model to focus on the precise features of smaller-scale targets. However, due to the limitation of the fixed window size, in order to further capture the long-range dependencies across windows, the present invention introduces a moving window mechanism. After the attention calculation is completed, the window slides by half of its size, re-divides the window, and performs attention calculation again, enabling the cross-window information to be interacted and integrated, thereby enhancing the understanding of global semantics. After the attention calculations at all scales, the weighted feature maps of different scales are concatenated and initially convolved and fused together. Subsequently, through layer normalization operations and the application of the ReLU activation function to add non-linearity, and finally further convolution operations are used to fuse the features and perform residual connection with the input feature map to ensure the stability of the learned features and the effective integration of multi-scale information. Finally, the module outputs a feature map that combines local details and global dependencies, providing high-quality input for the subsequent feature reconstruction and upsampling of the decoder, enabling the model to more accurately process multi-scale targets in complex sea area scenarios and improving the segmentation accuracy and robustness.

[0059] The operation process of the multi-scale moving window attention module is expressed as:

[0060] F MSWA =F in +Conv v2 {ReLU[LN(Conv v1 (F fuse ))]};(2)

[0061] In formula (2), F MSWA represents the output of the multi-scale moving window attention module, F in represents the input feature map, Conv v1 and Conv v2 represent the 3×3 convolution operations, ReLU is the activation function; LN represents layer normalization, F fuseIt is the fused feature map after fusing the moving window attention mechanisms of 3 different scales. At this time, the window scale sizes are set to 4, 6, 8, F fuse The operation process can be expressed as:

[0062] F fuse = Concat(F4, F6, F8); (3)

[0063] In formula (3), Concat represents the concatenation operation, and F k represents the weighted feature maps of different scales, k = 4, 6, 8, and the operation process of F k can be expressed as:

[0064]

[0065] In formula (4), F in represents the input feature map, respectively represent the moving window attention mechanism and the window attention mechanism with a window size of k. The moving window attention mechanism is to enable the model to obtain cross-window feature information. Because in complex sea area scenarios, some targets or objects such as waves, sky, large ships, etc. may exceed the set window size, resulting in deviation in attention calculation in a single window. Therefore, the moving window attention mechanism is adopted, that is, all windows are moved half of the window size distance to the lower right direction of the image, and then the attention mechanism calculation is performed in the new window. The specific calculation formula can be expressed as:

[0066]

[0067] In formula (5), Φ represents the Softmax function, Q, K, V represent the Query, Key, Value matrices obtained through linear projection, T represents the matrix transpose, d represents the scaling coefficient, and B represents the relative position bias matrix. Compared with the traditional self-attention mechanism, we adopt a 2×2 patch size to prevent information loss caused by too large patches in pixel-level tasks such as semantic segmentation. At the same time, since we limit the self-attention mechanism to calculate within the window, it greatly improves the calculation efficiency of the model, reduces the calculation overhead, reduces the number of model parameters, and makes it easier to fit in small sample learning tasks such as complex sea area scene segmentation.

[0068] In complex sea area scenarios, the target scale changes significantly (such as a ship nearby and a buoy in the distance). The multi-scale moving window attention module can capture both local details and global context information through attention mechanisms at multiple scales: small-scale windows (e.g., k = 4) capture local detail features (such as the edges of the ship and the texture of the buoy), medium-scale windows (e.g., k = 6) capture target features in a medium range (such as the overall shape of the ship), and large-scale windows (e.g., k = 8) capture global context information (such as the relative position relationship between the target and the background). This multi-scale design enables the model to adapt to the characteristics of target scale diversity in complex sea area scenarios and improves the recognition ability of multi-scale targets. In addition, in sea area scenarios, the relative position relationship between targets (such as the relative position between a ship and a buoy) and the continuity of the background (such as the continuity of sea waves) are crucial for the segmentation result. The multi-scale moving window attention module realizes information interaction between windows through a sliding window mechanism and captures long-range dependence relationships in complex sea area scenarios. The traditional global self-attention mechanism has a high computational complexity and is difficult to meet the processing requirements of high-resolution UAV images. The multi-scale moving window attention module divides the feature map into multiple windows and calculates self-attention within the windows, significantly reducing the computational complexity.

[0069] C: Iteratively train the neural network model with the training set until the loss value of the neural network model converges to the optimal state to obtain a trained neural network model.

[0070] In the step C described above, the training data in the training set is input into the neural network model in batches for iterative training, and an optimizer, the number of iterations, the learning rate, the batch size, and a loss function are set.

[0071] In this embodiment, the optimizer is set to Adam, the number of iterations is set to 200, the learning rate is set to 0.001, the batch size is set to 8, and the loss function is the normalized cross-entropy loss function. The formula is as follows:

[0072]

[0073] In Equation (6), represents the total loss, N is the number of valid pixel points, H and W are the height and width of the image, C is the number of categories, y ijc is the true category label of the pixel at position (i, j), is the natural logarithm of the predicted probability, which is used to convert the probability in product form to sum form to make the loss calculation more stable. At the same time, if the predicted probability of the model for the true category is low, then will produce a large negative value, so that the contribution of this pixel to the total loss is greater, prompting the model to continuously adjust the parameters during training to improve the prediction accuracy. Pijc is the predicted probability that the pixel at position (i, j) belongs to c, and the calculation formula is as follows:

[0074]

[0075] In Equation (7), EXP is the softmax activation function, and Z ijc are the logits output by the model.

[0076] Aiming at the characteristic that the target category distribution in complex sea area scenarios is unbalanced (such as the number of targets such as ships and buoys is much less than that of background categories such as sea water and sky), the normalized cross-entropy loss function in the present invention improves the loss proportion of rare categories by normalizing the predicted probability, and enhances the model's attention ability to small targets and low-frequency categories. The normalized cross-entropy loss function effectively solves the common category imbalance problem in semantic segmentation tasks by normalizing the predicted probability.

[0077] In addition, the normalized cross-entropy loss function can be well combined with the Softmax activation function, has numerical stability, and avoids the problems of gradient disappearance or explosion. At the same time, the normalized cross-entropy performs normalization calculation at the pixel level, does not increase significant computational cost, and is suitable for semantic segmentation tasks of high-resolution UAV images. Therefore, while improving the segmentation accuracy, the normalized cross-entropy maintains the computational efficiency and is particularly effective for multi-category semantic segmentation tasks.

[0078] In this embodiment, the model parameters obtained from the current training are saved at the end of each round of iteration. At the same time, the test set is used to evaluate the adaptive segmentation model. After training is completed, the one with the best performance is selected from the saved models as the trained adaptive segmentation network model for multi-scale feature modeling of UAV images in complex sea area scenarios.

[0079] D: Use the test set to perform performance testing on the trained neural network model to obtain performance test results and optimize the adaptive segmentation network model;

[0080] Input the test data set into the trained neural network model to evaluate the model performance, and optimize the neural network model according to the obtained performance metrics. Finally, an optimized adaptive segmentation network model for multi-scale feature modeling of UAV images in complex sea area scenarios is obtained;

[0081] In the embodiment, in order to evaluate the neural network model, the F1 score (F1), precision (Precision), and mean intersection over union (mIoU) are used as evaluation metrics during model testing. These comprehensive metrics can comprehensively quantify the performance of the neural network model and its performance on actual test data, which helps to more deeply understand the advantages and limitations of the model, thereby providing targeted guidance for improving the model.

[0082]

[0083] In Equation (9), F1 represents the F1 score, Precision represents the accuracy rate, and Recall represents the recall rate. The specific calculation formulas are as follows:

[0084]

[0085]

[0086] TP represents the number of true positives, FP represents the number of false positives, and FN represents the number of false negatives.

[0087] The calculation formula for the overall pixel segmentation accuracy rate OA is:

[0088]

[0089] In Equation (11), TN represents the number of true negatives.

[0090] The calculation formula for the mean intersection over union mIoU is:

[0091]

[0092] In Equation (12), C represents the categories of semantic segmentation, and the intersection over union IoU c has the following calculation formula:

[0093]

[0094] In Equation (13), TP c , FP c and FN c respectively represent the number of true positives, false positives, and false negatives in category c.

[0095] In this embodiment, by analyzing the three evaluation indicators of F1 score (F1), accuracy (Precision), and mean intersection over union (mIoU), the hyperparameters and architecture of the model can be iteratively adjusted to achieve better performance. The F1 score can balance the precision and recall rate, and is particularly suitable for evaluating the recognition ability of the model for rare categories (such as ships and buoys), avoiding evaluation bias caused by category imbalance. OA can reflect the segmentation accuracy of the model at the overall pixel level. mIoU can reflect the segmentation accuracy of the model for each category, and is particularly suitable for evaluating the performance of the model in multi-category semantic segmentation tasks. In complex sea scenes, mIoU can effectively measure the model's segmentation ability for target categories (such as ships and buoys) and background categories (such as sea water and sky). Finally, through repeated experiments and adjustments, according to the performance of performance indicators, the best performing hyperparameter combination is selected as the hyperparameter of the final model, and the neural network model composed of the hyperparameters of each model at this time is used as the optimized neural network model.

[0096] E: Obtain the drone image to be detected, use the optimized adaptive segmentation neural network model to perform semantic segmentation on the drone image to obtain the corresponding segmentation result.

[0097] Aiming at the characteristics of diverse target scales, complex backgrounds, and significant changes in illumination in complex sea scenes, the present invention realizes high-precision semantic segmentation of drone images through innovative encoder and decoder designs. In order to better capture the low-contrast features between targets and backgrounds in complex sea scenes, we designed an adaptive channel attention module to dynamically assign weights to each channel of the input feature map, enhance the feature expression capabilities of key targets (such as ships and buoys), and suppress the interference of background noise (such as wave textures and illumination changes), effectively solving the problem of too many parameters in the traditional channel attention mechanism. In addition, in view of the characteristics of significant changes in target scales and complex long-range dependencies in complex sea scenes, we proposed a multi-scale moving window attention module, which captures local details and global contextual information through a multi-scale window attention mechanism, making up for the shortcomings of ResNet-50 in long-range dependency modeling when used as a backbone network, and significantly improving the model's adaptability to complex sea scenes.

[0098] By conducting performance tests on the trained neural network model and using evaluation metrics such as F1 score (F1), accuracy (OA), and mean intersection over union (mIoU), the segmentation performance of the model in complex sea area scenarios was comprehensively quantified. The test results show that our method can accurately identify and segment multi-scale targets (such as ships, buoys, reefs, etc.) in complex sea area scenarios, while effectively handling challenges such as complex backgrounds and significant illumination changes. This method not only significantly improves the accuracy and robustness of UAV image semantic segmentation in complex sea area scenarios, but also provides targeted guidance for subsequent model optimization, with important practical application value. In summary, the present invention provides an efficient and reliable solution for UAV image semantic segmentation in complex sea area scenarios.

[0099] The above-described embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention and should all be included in the protection scope of the present invention.

Claims

1. An adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas, characterized in that, It includes the following steps: A: Obtain the UAV image dataset, and construct the training set, validation set and test set after preprocessing; B: Establish an adaptive segmentation neural network model for multi-scale feature modeling of UAV images in complex sea area scenarios; The adaptive segmentation neural network model consists of an encoder and a decoder, uses ResNet-50 as the backbone network, and combines an adaptive channel attention module and a multi-scale moving window attention module; C: Use the training set to train the adaptive segmentation neural network model; D: Use the test set to optimize the trained neural network model; E: Obtain the UAV image to be detected, and use the optimized adaptive segmentation neural network model to perform semantic segmentation on the UAV image to be detected to obtain the corresponding segmentation result.

2. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 1, wherein: In step B, the encoder includes several groups of stacked ResNet blocks, an adaptive channel attention module, and a multi-scale moving window attention module; The ResNet block is used to perform preliminary feature extraction on the input image; The adaptive channel attention module is used to dynamically adjust the weights of each channel, enhance the response of the key information channels and suppress the interference of noise and irrelevant features; The multi-scale moving window attention module uses the moving window mechanism of different scales to capture the long-range dependencies in the image, retain local details while integrating global context information.

3. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 1, wherein: In step B, the decoder is used to gradually fuse and upsample the features extracted by the encoder, restore the image to the original input size, and finally obtain the segmentation result of the image after being processed by the segmentation head.

4. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 2, wherein: The adaptive channel attention module generates channel-level attention weights through global average pooling, one-dimensional adaptive convolution and Sigmoid activation function, and applies them to the input feature map to achieve the enhancement of key channels and the suppression of non-key channels.

5. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 4, wherein: The adaptive channel attention module first receives the preliminary feature map from the ResNet block; then performs global average pooling on the input feature map in the spatial dimension, compresses all spatial information of each channel into a single value to obtain a channel descriptor of a set size; subsequently, uses one-dimensional adaptive convolution to process the channel descriptor, models the interdependence between channels, and dynamically adjusts the size of the convolution kernel according to the complexity of the input image; then expands the channel weights after convolution operation to the same spatial size as the input feature map, and normalizes them through the Sigmoid activation function to obtain the importance coefficient of each channel; finally, multiplies the input feature map element by element with the channel weights to achieve feature recalibration and enhance the response of key channels, and finally outputs the feature map after weight adjustment.

6. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 2, wherein: The multi-scale moving window attention module first receives the feature map processed by the ResNet block and the adaptive channel attention module, and divides it into multiple patches; then uses a preset multiple different sizes of windows to apply the moving window attention mechanism for feature extraction respectively; at each scale, the feature map is divided into multiple windows, and self-attention is calculated inside the windows; After the attention calculation is completed, the window slides by half of its own size and re-divides the window, and the attention calculation inside the window is performed again; then after the attention calculations at all scales, the weighted feature maps at different scales are stitched and preliminarily convolved to achieve fusion, and then through layer normalization operation and using the ReLU activation function to add non-linearity, further convolution operation is performed to fuse features, and residual connection is made with the input feature map; finally, a feature map that fuses local details and global dependencies is output.

7. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 1, characterized in that: In step C, the normalized cross-entropy loss function is used during the training of the adaptive segmentation neural network model for normalizing the prediction probability.

8. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 1, characterized in that: In step D, when optimizing the trained adaptive segmentation neural network model, the F1 score, accuracy, and mean intersection over union are used as evaluation metrics for model evaluation.

9. The adaptive segmentation method for multi-scale feature modeling of UAV images in complex sea areas according to claim 1, characterized in that, Step A includes the following specific steps: A1: Obtain the UAV image dataset; A2: Preprocess the UAV image dataset, including data augmentation and size adjustment; A3: Divide the preprocessed dataset into a training set, a validation set, and a test set according to a ratio.

10. The adaptive segmentation method for multi-scale feature modeling of complex sea area UAV images according to claim 9, wherein: The UAV image dataset is constructed based on the OUC-UAV-SEG dataset.

Images