Unmanned aerial vehicle sea target dynamic identification method based on all-around feature fusion

Abstract

The application discloses a kind of unmanned aerial vehicle offshore target dynamic identification method based on all-around feature fusion, comprising the following steps: A: obtaining unmanned aerial vehicle image dataset;B: construct unmanned aerial vehicle offshore target dynamic identification model based on all-around feature fusion;C: benefit unmanned aerial vehicle offshore target dynamic identification model training;D: optimization unmanned aerial vehicle offshore target dynamic identification model;E: using the optimized unmanned aerial vehicle offshore target dynamic identification model, target recognition is carried out to the unmanned aerial vehicle image to be detected, and the result of target recognition is obtained.The application can effectively improve the accuracy and robustness of target recognition, better capture the real boundary of target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a dynamic target recognition method for unmanned aerial vehicles on the sea based on all-around feature fusion in a complex sea scene. BACKGROUND

[0002] In a complex sea scene, the recognition of targets on the sea by unmanned aerial vehicles faces a series of severe challenges, such as sea surface reflection, changes in light, target occlusion, and dynamic changes in sea waves. These factors make it difficult to accurately capture the appearance, position, and contour of targets on the sea, thereby affecting the effect of target recognition.

[0003] At present, the method for recognizing targets on the sea generally adopts target detection technology based on the YOLO series, such as the technical solution disclosed in the patent application No. 202411477253.8, entitled "A rapid recognition method for sea ship targets based on YOLOv8". After collecting image samples of the ship sailing water area in a digital carrier, the image samples are cleaned and labeled, and the image samples are data enhanced to obtain a data set that can be used for training and verification. A target recognition network model is constructed based on the YOLOv8 deep neural network, and the learning rate, batch size, and other hyperparameters are set, and a pre-trained model is used for transfer learning. The non-maximum suppression algorithm is used to remove duplicate detection boxes, and the confidence threshold is used for result filtering to post-process the inference results of the model. The prediction of the ship target is completed. The trained model neural network is pruned and pruned, and the pruned model is calibrated and quantized. The above-mentioned technology reduces the memory consumption of the device during operation while maximizing the retention of the detection accuracy of the model and speeding up the detection speed of the model.

[0004] The above-mentioned target detection technology based on the YOLO series can provide good performance in real-time detection, but these methods usually rely on rectangular boxes to label the position and size of the target, and cannot accurately outline the complex contour of the target. Especially when the target form is complex or partially occluded, the limitations of the rectangular box are more obvious, which reduces the accuracy and robustness of target recognition.

[0005] Existing technologies also include target detection methods based on feature pyramid networks. For example, the technical solution disclosed in patent application number 202311567359.2, entitled "Small Target Detection Algorithm for UAV Images Based on Maritime Rescue," employs a scale-selection pyramid network structure as the baseline network and embeds a context enhancement module and a spatial attention module. Through the synergistic effect of these modules, the algorithm enhances the contextual feature information transmitted by the input image, deepens the feature layer's learning of relevant feature information, and improves detection accuracy. In its specific implementation, the model detector module uses a balanced loss function, and the optimal parameter values ​​are determined experimentally to enhance the stability of the model during detection. Experiments show that the proposed algorithm can effectively improve the detection performance and recognition accuracy of UAV images targeting maritime targets, reduce the time spent searching for targets in maritime rescue, and play a crucial role in the success of rescue missions.

[0006] While the aforementioned target detection techniques based on feature pyramid networks can improve the robustness of target detection to some extent through multi-scale feature fusion and attention mechanisms, they lack the ability to learn pixel-level subtle features in marine scenarios, which may lead to accelerated attenuation of target edge information during transmission through deep networks. Furthermore, the hierarchical feature reconstruction mechanism of the pyramid structure is prone to redundant computation, making it difficult to meet the requirements of low-power real-time detection. Summary of the Invention

[0007] The purpose of this invention is to provide a dynamic identification method for maritime targets by unmanned aerial vehicles (UAVs) based on omnidirectional feature fusion, which can effectively improve the accuracy and robustness of target identification and better capture the true boundaries of the target.

[0008] The present invention adopts the following technical solution:

[0009] A dynamic identification method for maritime targets by unmanned aerial vehicles (UAVs) based on omnidirectional feature fusion includes the following steps:

[0010] A: Obtain the drone image dataset;

[0011] B: Construct a dynamic identification model for UAV maritime targets based on comprehensive feature fusion. The dynamic identification model for UAV maritime targets consists of a frequency domain enhancement and normalization module, a cross-graph coupling module, and an improved U-Net module based on dual-domain attention.

[0012] C: Train a UAV dynamic target recognition model for maritime targets using UAV image datasets;

[0013] D: Optimize the UAV dynamic target recognition model for maritime targets using UAV image datasets;

[0014] E: Using the optimized UAV maritime target dynamic recognition model, target recognition is performed on the UAV image to be detected to obtain the target recognition result.

[0015] In step B, the frequency domain enhancement and normalization module is used to perform frequency domain enhancement and normalization on the preprocessed image, and normalization makes the image data have a uniform scale. The normalized image is used as the input feature map of the cross-graph coupling module. The cross-graph coupling module is used to learn graph aggregation representations of the input feature map in the horizontal and vertical directions respectively, and couple the learning results in the two directions. The improved U-Net module based on dual-domain attention performs the target recognition task through the backbone network U-Net, combined with multi-layer spatial attention mechanism and dual-stream channel attention mechanism.

[0016] In step B, when performing frequency domain enhancement on the image, the frequency domain enhancement and normalization module first performs a Fourier transform on each channel of the sea area image to convert the image from the spatial domain to the frequency domain; then it filters and enhances the image in the frequency domain; finally, it performs an inverse Fourier transform on each channel to convert the image back from the frequency domain to the spatial domain.

[0017] In step B, when normalizing the image, the frequency domain enhancement and normalization module first calculates the mean and standard deviation of each channel, and then subtracts the mean of each channel from each pixel value and divides it by the standard deviation to ensure that the pixel values ​​of the image fall within the set range.

[0018] In step B, the cross-graph coupling module first performs horizontal graph aggregation learning on each channel of the input feature map, then sequentially reassembles the row tensors after graph aggregation representation learning to recover each single-channel feature map, and then reassembles all the single-channel feature maps into a horizontal feature map according to the channel order; the cross-graph coupling module performs the corresponding operation in the vertical direction of the input feature map to obtain the vertical feature map according to the above method; finally, the horizontal and vertical feature maps are coupled.

[0019] In step B, an improved U-Net module based on dual-domain attention is adopted. By using the U-Net network as the backbone network, a multi-layer spatial attention mechanism is used before each downsampling stage to dynamically adjust the weight of each spatial location. At the same time, after splicing the encoder feature map and the decoder feature map in each skip connection, a dual-stream channel attention mechanism is applied to dynamically adjust the weight of each channel in the horizontal and vertical directions, further optimizing the fusion effect of features at different levels.

[0020] In step B, the improved U-Net module based on dual-domain attention first uses parallel dual-pooling to simultaneously perform global average pooling and global max pooling on the input feature map along the channel dimension, and then concatenates the two along the channel dimension. Subsequently, convolution operations are performed using convolution kernels of different sizes, and the resulting spatial weight maps containing information of different granularities are concatenated along the channel dimension to calculate the spatial weight matrix. Finally, the spatial weight matrix is ​​multiplied by the input feature map to obtain the spatial attention feature map.

[0021] In step B, the dual-channel attention mechanism is used to first perform average pooling in the horizontal and vertical directions on the input feature map to extract feature information of the sea area image from two complementary directions. Then, the pooling results in the horizontal and vertical directions are concatenated in the spatial dimension and a hybrid attention map is obtained through convolution. The hybrid attention map is then separated into horizontal and vertical components along the spatial direction, and the horizontal and vertical weight matrices are obtained by performing convolution and activation functions on the two components respectively. Finally, the input feature map is multiplied by the horizontal and vertical weight matrices to obtain the dual-channel attention feature map.

[0022] In step C, the target dynamic recognition model is trained using the Adam optimizer, and the total loss is calculated using the weighted cross-entropy loss function formula.

[0023] The performance of the UAV maritime target dynamic recognition model trained by precision, recall, F1 score, overall pixel accuracy, and cross-union comparison was tested, and the target recognition model was optimized.

[0024] This invention addresses the challenges of UAV-based maritime target recognition, including issues related to sea surface reflection, varying lighting conditions, target occlusion, and dynamic wave changes. By performing frequency domain enhancement and normalization on the image, it effectively enhances image details and contrast, highlighting the semantic regions of targets within the maritime image while ensuring spatial consistency of the image data and avoiding numerical deviations caused by different lighting or sensor variations. Then, by independently learning and coupling graph aggregation representations in the horizontal and vertical directions of the input feature map, it effectively captures richer spatial information and local features. Furthermore, this invention uses an improved U-Net network based on dual-domain attention to classify each pixel, obtaining semantic segmentation results. This more accurately focuses on the features of important regions in the maritime image, improving target recognition capabilities and effectively suppressing background interference, highlighting the feature expression of the target region. Ultimately, this enables UAV-based maritime target recognition, effectively improving the accuracy and robustness of UAV-based maritime target recognition in complex maritime scenarios. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the process of the present invention;

[0026] Figure 2 This is a structural diagram of the UAV-based dynamic target recognition model for maritime targets in this invention;

[0027] Figure 3 This is a structural diagram of the cross-graph coupling module in this invention;

[0028] Figure 4 This is a structural diagram of the multi-layer spatial attention mechanism in this invention;

[0029] Figure 5 This is a structural diagram of the dual-channel attention mechanism in this invention. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0031] like Figures 1 to 5 As shown, the UAV-based dynamic identification method for maritime targets based on omnidirectional feature fusion, as described in this invention, includes the following steps:

[0032] A: Obtain the drone image dataset, and construct the training set, validation set, and test set after preprocessing;

[0033] The drone image dataset originates from the OUC-UAV-SEG dataset. During preprocessing: First, data augmentation operations such as flipping and rotating were performed on the drone images in the dataset to expand its diversity and complexity, enhance the model's adaptability to images from different viewpoints and orientations, thereby improving the model's generalization ability and effectively reducing overfitting. Then, all augmented images were cropped to a uniform predefined size to ensure that the input images have consistent spatial dimensions, facilitating model training. Finally, the dataset was divided into training, validation, and test sets according to proportions.

[0034] In this embodiment, step A includes the following specific steps:

[0035] A1: Obtain the drone image dataset.

[0036] Obtain the OUC-UAV-SEG dataset. The OUC-UAV-SEG dataset uses high-resolution images of the Bohai Sea coastline collected by eight different types of drones, containing 726 original labeled images and 1750 images to be labeled. The training, validation, and test sets in the OUC-UAV-SEG dataset contain 477, 140, and 109 original images, respectively. The OUC-UAV-SEG dataset includes 15 semantic labels: seagrass, seaweed, ocean, river, land, Spartina alterniflora, Suaeda salsa, vegetation, buildings, sky, road, and boat.

[0037] A2: Preprocess the drone image dataset.

[0038] First, to overcome the problems of sparse datasets and limited sample types, and to improve the robustness and generalization ability of the model, this embodiment employs various data augmentation techniques on the UAV-generated marine images in the dataset. Specifically, for each marine image, image flipping, rotation, scaling, and translation operations are performed to expand the diversity of the dataset. These operations simulate shooting scenarios from different perspectives, headings, and flight altitudes of the UAV, enhancing the model's adaptability to images of different directions, scales, and positions. Through these data augmentation methods, the model can learn image features more comprehensively, avoid overfitting, and thus improve its generalization ability in complex environments, enabling the model to better handle target recognition tasks in unfamiliar scenes.

[0039] Secondly, to ensure that the input images have consistent spatial dimensions and meet the model's input requirements, this embodiment crops all enhanced images to a uniform 512x512 size. The cropping operation determines the appropriate cropping position based on the image's width and height, ensuring that targets in the image are not cropped out, especially larger or critical targets. If the image size is small, it is expanded using zero-padding to ensure it conforms to the predefined size. This cropping step ensures the consistency of the input image size, enhances the model's training efficiency, and improves its adaptability to targets of different sizes.

[0040] A3: The dataset after data augmentation and resizing is divided into three parts: training set, validation set, and test set. The training set, validation set, and test set contain 3816, 1120, and 872 images, respectively.

[0041] B: Construct a dynamic identification model for UAV maritime targets based on comprehensive feature fusion;

[0042] In this invention, the UAV maritime target dynamic recognition model based on all-round feature fusion consists of a frequency domain enhancement and normalization module, a cross-graph coupling module, and an improved U-Net module based on dual-domain attention.

[0043] The frequency domain enhancement and normalization module is used to perform frequency domain enhancement and normalization on the preprocessed image to enhance the image details and contrast. At the same time, normalization makes the image data have a uniform scale. The normalized image is used as the input feature map of the cross-graph coupling module.

[0044] The cross-graph coupling module is used to learn graph aggregation representations of the input feature map in the horizontal and vertical directions respectively, and couple the learning results in these two directions to capture richer spatial information and local features;

[0045] The improved U-Net module based on dual-domain attention uses the U-Net backbone network to combine multi-layer spatial attention mechanism and dual-channel attention mechanism for target recognition tasks. This effectively focuses on important regions and strengthens the information interaction between different channels, thereby improving recognition accuracy and robustness.

[0046] In the dynamic identification of maritime targets by UAVs in complex marine environments, the marine images acquired by UAVs are often affected by complex environmental factors, such as sea surface reflection, changes in illumination, and wave dynamics. These factors cause the target outlines in the marine images to be blurred and the details to be unclear. At the same time, due to the influence of sensor differences, changes in illumination, and environmental noise, the pixel values ​​of marine images often fluctuate greatly, which may lead to significant numerical deviations.

[0047] To address the unique challenges in dynamic target identification using unmanned aerial vehicles (UAVs) in complex maritime scenarios, this invention employs a special approach: frequency domain enhancement and normalization. First, a Fourier transform is performed on each channel of the image. Then, the image contrast is enhanced by adjusting the spectrum, and finally, an inverse Fourier transform is used to convert the image back to the spatial domain. After frequency domain enhancement, the image is normalized using the mean and standard deviation. This involves subtracting the mean from the pixel value of each channel of the UAV image and dividing by the standard deviation. The normalized image then serves as the input feature map for the cross-graph coupling module. This specialized processing effectively enhances image details and contrast, highlighting the semantic regions of targets in maritime images while ensuring spatial consistency of image data and avoiding numerical deviations caused by different lighting conditions or sensor variations.

[0048] In this invention, the specific operation steps of the frequency domain enhancement and normalization module are as follows:

[0049] a1: The frequency domain enhancement and normalization module performs frequency domain enhancement processing on the sea area image.

[0050] First, a Fourier transform is performed on each channel of the sea area image to convert the image from the spatial domain to the frequency domain. Then, the image is filtered and enhanced in the frequency domain to suppress low-frequency noise and enhance high-frequency information in the image, thereby making the contrast between the sea target and the background more obvious and improving the model's target resolution and recognition ability. Finally, an inverse Fourier transform is performed on each channel to convert the image back from the frequency domain to the spatial domain.

[0051] a2: The frequency domain enhancement and normalization module normalizes the image's data space.

[0052] First, calculate the mean and standard deviation for each channel. Then, subtract the mean of each channel from the pixel value of that channel and divide by the standard deviation to ensure that the pixel values ​​of the image fall within the range of [-1, 1]. This normalization operation ensures a more uniform distribution of image features, accelerates model convergence, and improves the model's robustness under different environmental conditions.

[0053] In the dynamic identification of maritime targets by UAVs in complex marine environments, target features in marine images often exhibit highly irregular and complex spatial distributions due to factors such as sea surface undulations, light reflection, and perspective changes. These features typically display different characteristic patterns in the horizontal and vertical directions. To effectively extract local features from these images and capture rich spatial information, this invention features a specially designed cross-graph coupling module. This module learns independent graph aggregation representations of the input feature map in both the horizontal and vertical directions and couples the learning results from both directions, ultimately enhancing the multidimensional spatial structure and target characteristic reflection of the marine image.

[0054] In this invention, the specific operation steps of the cross-graph coupling module are as follows:

[0055] b1: The cross-graph coupling module first performs horizontal graph aggregation learning on each channel of the input feature map; that is, the cross-graph coupling module performs row-by-row segmentation on each single-channel feature map, where each row tensor is treated as a node in the graph structure. The node connection rules follow the spatial proximity principle, that is, each node (representing a row) establishes an edge connection with its directly adjacent nodes (i.e., the previous row and the next row), and adds self-loops to each node. Then, graph convolution operations are applied to update the features of these nodes, as shown below:

[0056]

[0057] in, These are the updated features in row p, where ReLU represents the activation function, and N... p Indicates the neighbor of row p. It is the normalization constant, h q W is the initial feature of the q-th row. k These are trainable parameters;

[0058] b2: The cross-graph coupling module sequentially reassembles the row tensors learned through graph aggregation representation to recover each single-channel feature map. Then, all the single-channel feature maps are reassembled into a horizontal feature map according to channel order. The horizontal feature map obtained from the above operations can effectively capture contextual information along the horizontal axis in the marine image, helping the model understand the horizontal distribution characteristics of targets on the sea surface, such as the target's width and positional relationships.

[0059] b3: Following the methods in steps b1 and b2, perform the corresponding operations in the vertical direction of the input feature map to obtain the vertical feature map. Compared to horizontal graph aggregation, vertical graph aggregation focuses on spatial information along the vertical axis, helping to identify the target's height changes and local details, such as the target's depth features and the impact of wave undulations on the target's shape.

[0060] b4: To fully integrate information from different spatial dimensions and capture more comprehensive and accurate target features, it is necessary to couple the feature maps in the horizontal and vertical directions. Specifically, the horizontal and vertical feature maps are first concatenated according to the channel dimension, and then the original number of channels is restored through three 1*1 convolutional kernels to obtain the coupled feature map.

[0061] The specific calculation method is as follows:

[0062] OH=Conv 1*1 (Concat(S x S y ))))

[0063] Where OH represents the coupling feature map, Conv 1*1 It's a 1x1 convolution operation, Concat represents the concatenation operation, S x and S y These represent the horizontal and vertical feature maps, respectively.

[0064] In the dynamic identification of maritime targets by UAVs in complex maritime scenarios, factors such as sea surface reflection, dynamic waves, and target occlusion can cause significant changes in the position and shape of targets in images. This leads to unreasonable weight allocation in existing dynamic target identification models, resulting in poor robustness and accuracy of segmentation. Therefore, this invention employs an improved U-Net module based on dual-domain attention. By using the U-Net network as the backbone network, a multi-layer spatial attention mechanism is applied before each downsampling stage to dynamically adjust the weight of each spatial location. This helps the model automatically focus on more important regions when processing image features at different locations, thereby improving the robustness and accuracy of semantic segmentation. Furthermore, this invention also applies a dual-stream channel attention mechanism after concatenating encoder and decoder feature maps in each skip connection. This dynamically adjusts the weight of each channel in the horizontal and vertical directions, further optimizing the fusion effect of features at different levels. This captures the horizontal extension of maritime targets and the potential vertical changes in target height, enhancing the model's expressive power and recognition accuracy.

[0065] In this invention, the improved U-Net module based on dual-domain attention utilizes a multi-layer spatial attention mechanism to optimize for the unique complexity of marine images. Particularly when the morphology of targets and the spatial distribution of the background are highly uneven, the model can more accurately focus on the features of important regions in the marine image through multi-scale spatial weight mapping, thereby improving target recognition capabilities. Its structural diagram is shown below. Figure 4 As shown.

[0066] d1: Due to the complexity of marine images, different regions often exhibit different textures and structural features. Relying solely on a single pooling method may not be sufficient to fully capture the rich semantic information within the image. Therefore, in this embodiment, a parallel dual-pooling approach is used to simultaneously perform global average pooling and global max pooling along the channel dimension on the input feature map, and then the two are concatenated according to the channel dimension:

[0067] a ave =Global_AvePooling(A)

[0068] a max =Global_MaxPooling(A)

[0069] a out =Concat(a ave ;a max )

[0070] Among them, a ave and a max Let a represent the results of global average pooling and global max pooling, respectively. out This represents the result of concatenating the two. Global_AvePooling and Global_MaxPooling represent global average pooling and global max pooling, respectively. Concat represents the concatenation operation, and A represents the input feature map of the multi-layer spatial attention mechanism.

[0071] d2: Targets in marine images often exhibit varying scales and irregular shapes, especially under conditions of wave undulation, dynamic backgrounds, and partial target occlusion. Local features of the target show significant differences across different scales. To effectively extract and fuse these multi-scale features, this embodiment employs convolution operations with a 7x7 filter size and padding of 3, a 5x5 filter size and padding of 2, and a 3x3 filter size and padding of 1, respectively, to obtain three spatial weight mappings containing information of different granularities. These three are then concatenated along the channel dimension to ensure the model can perceive both large-scale features and detailed local features in the image. Finally, a 1x1 convolution kernel and a sigmoid activation function are used to calculate the spatial weight matrix.

[0072] d3: Multiply the spatial weight matrix and the input feature map to obtain the spatial attention feature map.

[0073] In this invention, the improved U-Net module based on dual-domain attention utilizes a dual-channel attention mechanism to dynamically adjust the saliency of each feature point in both directions. This not only optimizes weights based on the characteristics of targets in different directions within marine images but also effectively suppresses background interference and highlights the feature representation of the target region. Its structural diagram is shown below. Figure 5 As shown.

[0074] The dual-channel attention mechanism includes the following:

[0075] e1: Perform average pooling in both the horizontal and vertical directions on the input feature map. The purpose of these two average pooling methods is to extract feature information from the marine image from two complementary directions, providing a multi-view feature foundation for the model. The horizontal direction focuses on capturing lateral structural features in the image, such as wave textures or horizontally distributed targets in the marine image, while the vertical direction focuses on capturing longitudinal information, such as vertically arranged ships and buoys in an aerial view. The calculation method is as follows:

[0076] c x =AveragePooling x (F)

[0077] c y =AveragePooling y (F)

[0078] Among them, c x and c y These represent the pooling results in the horizontal and vertical directions, respectively. (AveragePooling) x and AveragePooling y represents the horizontal and vertical average pooling of the input feature map, respectively, and F represents the input feature map of the two-stream attention mechanism;

[0079] e2: To comprehensively integrate feature distributions in the horizontal and vertical directions, further fuse and extract hybrid features, and enhance the collaborative expressive ability of the model, in this embodiment, the pooling results in the horizontal and vertical directions are concatenated in the spatial dimension, and a hybrid attention map is obtained through a 1*1 convolution operation:

[0080] Z = RELU(BN(Conv) 1*1 (Concat(c x ;c y ))))

[0081] Where Z represents hybrid attention mapping and BN is batch normalization.

[0082] e3: Separate the hybrid attention map into horizontal and vertical components along the spatial direction, and apply a 1*1 convolution operation and a sigmoid activation function to the two components respectively to obtain the horizontal and vertical weight matrices:

[0083] Z x Z y =Split(Z)

[0084] G x =Sigmoid(Conv 1*1 (Z x ))

[0085] G Y =Sigmoid(Conv 1*1 (Z y ))

[0086] Among them, Z x and Z y Let G represent the horizontal and vertical components separated from the hybrid attention map, respectively. Split represents the separation operation. x and G Y These represent the horizontal weight matrix and the vertical weight matrix, respectively. The horizontal weight matrix can highlight the horizontally extended wavy areas, while the vertical weight matrix can enhance the sparsely distributed target areas in the vertical direction, thereby improving the ability to express image details and directional information.

[0087] e4: Multiply the input feature map with the horizontal and vertical weight matrices to obtain the dual-channel attention feature map:

[0088]

[0089] Among them, F out This represents the dual-channel attention feature map. This indicates a multiplication operation.

[0090] C: Iteratively train the target dynamic recognition model using the training set until the preset number of training iterations is reached to obtain a well-trained target dynamic recognition model;

[0091] The training data is input into the target recognition model in batches for iterative training; and the optimizer, number of iterations, learning rate, dropout rate, batch size, and loss function are set. In this embodiment, the number of iterations is set to 200, the learning rate is set to 0.0005, the batch size is set to 16, the optimizer is set to Adam, and the dropout rate is set to 0.1.

[0092] To learn features from different categories more evenly and improve the model's generalization ability, in this embodiment, the target dynamic recognition model is trained using the Adam optimizer, and the total loss is calculated using a weighted cross-entropy loss function formula:

[0093]

[0094] in, Represents the total loss; M corresponds to the total number of pixels in a single image; T represents the number of target categories; β t β represents the weight of class t. If class t has few samples in the dataset, then β is used. t Set to 2 otherwise set to 1; b m,t It is a binary indicator (0 or 1) that determines the class of the m-th pixel if the true class is t. m,t If b equals 1, otherwise b m,t Equals 0; log is a logarithmic function, v m,t This represents the probability that the model predicts the class of the m-th pixel to be t.

[0095] In this embodiment, the model parameters obtained from each training stage are saved, and the target recognition model during the training process is validated and evaluated using a validation dataset. After training is completed, the best-performing model is selected from all saved models as the trained target recognition model.

[0096] D: Use the test set to perform performance testing on the trained target recognition model, obtain the performance test results, and optimize the target recognition model;

[0097] The test set is input into the trained target recognition model for testing. In this embodiment, the performance is evaluated using five metrics based on the test results: precision (P), recall (R), F1 score, overall pixel accuracy (OA), and intersection-over-union (IoU).

[0098]

[0099] F1 = 2 * P * R / (P + R)

[0100]

[0101] Where TP represents the number of positive samples predicted as positive samples, FP represents the number of negative samples predicted as positive samples, FN represents the number of positive samples predicted as negative samples, TN represents the number of negative samples predicted as negative samples, and P, R, F1, OA and IoU represent precision, recall, F1 score, overall pixel precision and crossover ratio, respectively.

[0102] Based on the performance test results, the trained target recognition model is optimized. Through repeated experiments and adjustments, the best-performing model is selected as the optimized target recognition model based on the performance metrics.

[0103] E: Using the optimized target recognition model, target recognition is performed on the image of the UAV to be detected to obtain the target recognition result.

[0104] This invention proposes a dynamic maritime target recognition method for UAVs based on omnidirectional feature fusion. First, frequency domain enhancement is performed on the preprocessed sea area image to improve target details and contrast, followed by image normalization. Then, graph aggregation representation learning is performed on the feature maps in both the horizontal and vertical directions, and the learning results in these two directions are coupled to capture richer spatial information and local features. Next, an improved U-Net network based on dual-domain attention is used to classify each pixel, obtaining semantic segmentation results, thereby enabling UAV recognition of maritime targets. Finally, the trained neural network model is tested for performance using five evaluation metrics: precision (P), recall (R), F1 score, overall pixel accuracy (OA), and intersection-over-union ratio (IoU). This comprehensive evaluation of the model's performance on real-world test data provides targeted guidance for model improvement. In summary, the dynamic maritime target recognition method for UAVs based on omnidirectional feature fusion effectively improves the accuracy and robustness of UAV maritime target recognition in complex maritime scenarios.

[0105] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for dynamic identification of maritime targets by unmanned aerial vehicles (UAVs) based on omnidirectional feature fusion, characterized in that, Includes the following steps: A: Obtain the drone image dataset; B: Construct a dynamic identification model for UAV maritime targets based on comprehensive feature fusion. The dynamic identification model for UAV maritime targets consists of a frequency domain enhancement and normalization module, a cross-graph coupling module, and an improved U-Net module based on dual-domain attention. The module includes a frequency domain enhancement and normalization module, which enhances and normalizes the preprocessed image in the frequency domain and ensures that the image data has a uniform scale through normalization. The normalized image is then used as the input feature map for the cross-graph coupling module. The cross-graph coupling module performs graph aggregation representation learning on the input feature map in both the horizontal and vertical directions and couples the learning results in both directions. The improved U-Net module based on dual-domain attention performs target recognition tasks by combining a multi-layer spatial attention mechanism and a dual-stream attention mechanism through the backbone network U-Net. The cross-graph coupling module first performs horizontal graph aggregation learning on each channel of the input feature map, then sequentially reassembles the row tensors after graph aggregation representation learning to recover each single-channel feature map, and then reassembles all the single-channel feature maps into a horizontal feature map according to the channel order; the cross-graph coupling module performs the corresponding operation in the vertical direction of the input feature map to obtain the vertical feature map; finally, the horizontal and vertical feature maps are coupled. An improved U-Net module based on dual-domain attention is adopted. By using the U-Net network as the backbone network, a multi-layer spatial attention mechanism is used before each downsampling stage to dynamically adjust the weight of each spatial location. At the same time, after splicing the encoder feature map and the decoder feature map in each skip connection, a dual-stream channel attention mechanism is applied to dynamically adjust the weight of each channel in the horizontal and vertical directions, further optimizing the fusion effect of features at different levels. C: Train a UAV dynamic target recognition model for maritime targets using UAV image datasets; D: Optimize the UAV dynamic target recognition model for maritime targets using UAV image datasets; E: Using the optimized UAV maritime target dynamic recognition model, target recognition is performed on the UAV image to be detected to obtain the target recognition result.

2. The method for dynamic identification of maritime targets by unmanned aerial vehicles according to claim 1, characterized in that: In step B, when performing frequency domain enhancement on the image, the frequency domain enhancement and normalization module first performs a Fourier transform on each channel of the sea area image to convert the image from the spatial domain to the frequency domain; then it filters and enhances the image in the frequency domain; finally, it performs an inverse Fourier transform on each channel to convert the image back from the frequency domain to the spatial domain.

3. The method for dynamic identification of maritime targets by unmanned aerial vehicles according to claim 1, characterized in that: In step B, when normalizing the image, the frequency domain enhancement and normalization module first calculates the mean and standard deviation of each channel, and then subtracts the mean of each channel from each pixel value and divides it by the standard deviation to ensure that the pixel values ​​of the image fall within the set range.

4. The method for dynamic identification of maritime targets by unmanned aerial vehicles according to claim 1, characterized in that: In step B, the improved U-Net module based on dual-domain attention first uses parallel dual-pooling to simultaneously perform global average pooling and global max pooling on the input feature map along the channel dimension, and then concatenates the two along the channel dimension. Subsequently, convolution operations are performed using convolution kernels of different sizes, and the resulting spatial weight maps containing information of different granularities are concatenated along the channel dimension to calculate the spatial weight matrix. Finally, the spatial weight matrix is ​​multiplied by the input feature map to obtain the spatial attention feature map.

5. The method for dynamic identification of maritime targets by unmanned aerial vehicles according to claim 1, characterized in that: In step B, the dual-channel attention mechanism is used to first perform average pooling in the horizontal and vertical directions on the input feature map to extract feature information of the sea area image from two complementary directions. Then, the pooling results in the horizontal and vertical directions are concatenated in the spatial dimension and a hybrid attention map is obtained through convolution. The hybrid attention map is then separated into horizontal and vertical components along the spatial direction, and the horizontal and vertical weight matrices are obtained by performing convolution and activation functions on the two components respectively. Finally, the input feature map is multiplied by the horizontal and vertical weight matrices to obtain the dual-channel attention feature map.

6. The method for dynamic identification of maritime targets by unmanned aerial vehicles according to claim 1, characterized in that: In step C, the target dynamic recognition model is trained using the Adam optimizer, and the total loss is calculated using the weighted cross-entropy loss function formula.

7. The method for dynamic identification of maritime targets by unmanned aerial vehicles according to claim 1, characterized in that: The performance of the UAV maritime target dynamic recognition model trained by precision, recall, F1 score, overall pixel accuracy, and cross-union comparison was tested, and the target recognition model was optimized.

Images