A hybrid attention driven air-sea heterogeneous multi-view fusion ship detection method

Abstract

The application discloses a mixed attention driven air-sea heterogeneous multi-view fusion ship detection method, and belongs to the technical field of machine vision and multi-source perception fusion, and comprises the following steps: S1, acquiring two-way heterogeneous view images of an air observation platform and a water surface observation platform to form a multi-view input source; S2, adopting a light-weight convolutional neural network as a shared backbone to perform feature coding on the two-way images and outputting same-dimension feature maps; S3, dynamically weighting and fusing the double-view features through a channel-space mixed attention mechanism to form a joint feature map; and S4, inputting the joint feature map into a detection head, performing boundary box regression, class classification and confidence prediction, and outputting a ship detection result. Through the joint learning of the air-sea heterogeneous multi-view fusion and the mixed attention driving, the application eliminates a single view blind area, suppresses water surface reflection, waves and other interference, and realizes accurate, stable and real-time detection of a ship target in a complex shore-sea environment.

Description

Technical Field

[0001] This invention relates to the field of machine vision and multi-source perception fusion technology, and more specifically, to a hybrid attention-driven heterogeneous multi-view fusion ship detection method for air and sea. Background Technology

[0002] With the continuous expansion of maritime shipping and the increasing frequency of near-shore operations, scenarios such as shore-based patrols and autonomous navigation of unmanned surface vessels (USVs) place higher demands on the real-time detection and accurate identification of surface targets. Ship target detection is a core component of maritime environmental awareness and collision avoidance, and its performance directly affects navigation safety and the effectiveness of maritime defense monitoring. Among existing technologies, the most widely used approach is wide-area reconnaissance based on a single UAV equipped with visible light or infrared sensors, capable of covering large areas of water in a short time and acquiring global distribution information of surface targets. However, the high-altitude, top-down perspective of a single UAV has the following significant drawbacks in practical applications:

[0003] First, when drones are shooting from high altitudes, factors such as water reflections, wave occlusion, and shoreline shadows can severely interfere with image quality. Specular reflections on the water surface create bright areas in the image, significantly reducing the contrast between the ship target and the background; dynamic textures generated by wave motion can easily be confused with the features of small ship targets; and shadows from buildings and dock facilities near the shore can also obscure near-shore vessels, causing target features to become blurred.

[0004] Second, in images acquired by UAVs from high altitude, distant ship targets occupy only a very small pixel area, and their features are weak. As the detection distance increases, the effective feature information of ship targets in the image decreases sharply, leading to a significant drop in the recall rate of traditional detection algorithms, and highlighting the problem of missing small targets.

[0005] Third, a single high-altitude overhead view cannot simultaneously achieve both wide-area coverage and close-up detail acquisition. For targets such as low-lying vessels near the water surface or targets partially obscured by other vessels or obstacles, it is difficult to obtain sufficient shape and texture information from overhead images alone, resulting in insufficient detection reliability.

[0006] At the algorithm level, existing ship target detection methods are mainly based on convolutional neural networks, including anchor-frame-based detection models and anchor-frame-less detection models. These methods have achieved good detection performance on standard datasets, but their model designs are all geared towards single-view input and do not consider how to effectively integrate heterogeneous image features from different platforms and perspectives. When single-view detection models are directly applied to complex marine environments, the lack of complementary and verification mechanisms for multi-view information makes it difficult to meet practical requirements in terms of detection accuracy and robustness.

[0007] In the research of multi-source data fusion, some works have explored fusion methods for different types of sensor data, such as registration fusion of radar and optical images, and dual-modal fusion of visible light and infrared images. These methods have made some progress at the sensor modal fusion level, but most of them focus on multimodal data fusion on the same platform and do not address the problem of heterogeneous perspective fusion between different motion platforms. UAVs and unmanned surface vessels belong to two completely different observation platforms, one in the air and one on the water. They have fundamental differences in perspective direction, shooting distance, image resolution, and scene content. How to effectively utilize this heterogeneous complementarity for feature fusion and improve detection performance is a key technical problem that has not yet been solved.

[0008] Furthermore, regarding feature fusion mechanisms, existing multi-source feature fusion methods mostly employ simple feature stitching or element-wise addition operations, lacking adaptive evaluation and dynamic adjustment of the importance of features from different channels and spatial locations during the fusion process. In complex marine environments, feature maps acquired from different perspectives contain not only target information valuable for ship detection but also a large amount of water background, wave texture, and environmental interference information. If fusion is performed indiscriminately, useful target features may be overwhelmed by redundant background information, which is detrimental to improving detection accuracy. Summary of the Invention

[0009] To address the problems of large blind spots, weak anti-interference capabilities, and insufficient fusion of heterogeneous multi-view features in existing ship detection methods, this invention proposes a hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method, which can achieve accurate, stable, and real-time detection of ship targets under complex coastal and marine environments. The method includes the following steps:

[0010] S1, simultaneously acquire two heterogeneous visual images, the first of which is a first-view image provided by the first imaging device mounted on the aerial observation platform, and the second of which is a second-view image acquired by the second imaging device mounted on the water surface observation platform. The two images constitute a multi-view input source that complements the air and sea.

[0011] S2 uses a lightweight convolutional neural network as the shared feature extraction backbone to encode features of the first-view image and the second-view image respectively, and outputs two feature maps of the same dimension. The first feature map corresponds to the first-view image and the second feature map corresponds to the second-view image.

[0012] S3. The two feature maps output from step S2 are dynamically weighted and deeply fused using a channel-space hybrid attention mechanism. The channel attention module adaptively learns and strengthens the feature channels useful for ship detection and suppresses redundant background channels. The spatial attention module focuses on the spatial area where the ship is located and weakens the area of ​​environmental interference. The two feature maps after attention weighting are then spliced ​​and fused element by element to form a joint feature map that has both wide-area positioning capability and close-up detail capability.

[0013] S4. Input the joint feature map obtained in step S3 into the detection head. After bounding box regression, category classification and confidence prediction, the ship detection result is output. The ship detection result includes the target location, target category and confidence.

[0014] Furthermore, in step S1, the aerial observation platform uses a drone, and the first imaging device uses a visible light / infrared dual-light pod; the water surface observation platform uses an unmanned surface vessel, and the second imaging device uses a shipborne visible light / infrared camera.

[0015] The simultaneous acquisition of two heterogeneous visual images specifically involves: a UAV equipped with a visible light / infrared dual-light pod taking wide-area photos of the monitored water area from a high-altitude top-down perspective, acquiring a high-altitude wide-area top-down image containing the global distribution of targets in the water area; and an unmanned surface vessel equipped with a shipborne visible light / infrared camera taking close-up photos of water surface targets from a close-up level-down perspective, acquiring a close-up level-down image of the water surface containing high-resolution details of local ships and water surface textures. The UAV image and the unmanned surface vessel image complement each other in terms of viewing direction, shooting distance, and scene content.

[0016] Further, in step S2, a lightweight convolutional neural network is constructed as a shared feature extraction backbone network. The shared feature extraction backbone network consists of multiple depthwise separable convolutional modules stacked sequentially. Each depthwise separable convolutional module includes a depthwise convolutional layer and a pointwise convolutional layer. The depthwise convolutional layer is used to extract local features in the spatial dimension, and the pointwise convolutional layer is used to perform feature fusion and channel mapping in the channel dimension. Each depthwise separable convolutional module is followed by a batch normalization layer and a ReLU activation function layer.

[0017] Furthermore, the shared feature extraction backbone network has 5-8 depthwise separable convolutional modules, and the number of feature map channels output by each depthwise separable convolutional module increases sequentially.

[0018] Furthermore, step S3 specifically includes the following steps:

[0019] S3.1, Perform global average pooling and global max pooling operations on the two feature maps output from step S2 respectively, and feed the pooling results into the shared multilayer perceptron to obtain channel description vectors; after adding the channel description vectors element by element, generate channel attention weight maps through the Sigmoid activation function; multiply the channel attention weight maps element by element with the two original feature maps respectively to obtain channel weighted feature maps;

[0020] S3.2, after channel attention weighting, the two feature maps are subjected to average pooling and max pooling operations along the channel dimension respectively. The results are concatenated along the channel dimension and then passed through a convolutional layer with a kernel size of 7×7 and a sigmoid activation function to generate a spatial attention weight map.

[0021] The spatial attention weight map is multiplied element-wise with the two channel weighted feature maps to obtain the feature map after channel-spatial hybrid attention weighting;

[0022] S3.3 concatenates the two feature maps after hybrid attention weighting along the channel dimension, then performs channel dimensionality reduction through a 1×1 convolutional layer. Simultaneously, the two feature maps are added element-wise. Finally, the concatenated dimensionality reduction result and the element-wise addition result are added element-wise again to obtain the joint feature map.

[0023] Further, in step S3.1, the shared multilayer perceptron is composed of a fully connected dimensionality reduction layer, a ReLU activation function layer, and a fully connected dimensionality increase layer in sequence, with a dimensionality reduction ratio of r=16.

[0024] Further, in step S4, the joint feature map obtained in step S3 is input into the detection head network for ship target detection; the detection head network includes parallel classification and regression branches; the classification branch consists of several convolutional layers and fully connected layers, which performs category probability prediction and target confidence prediction for each preset anchor point position of the joint feature map, and outputs the probability value of the presence of a ship target at each position and the corresponding target category; the regression branch consists of several convolutional layers and fully connected layers, which predicts the target bounding box coordinate offset at each preset anchor point position, including the bounding box center point coordinate offset and width and height offset.

[0025] Further, in step S4, during the training phase, the cross-entropy loss function is used to optimize the classification branch, and the smooth L1 loss function is used to optimize the regression branch. The total loss function is the weighted sum of the classification loss and the regression loss. During the inference phase, non-maximum suppression is performed on all candidate detection results output by the detection head network. A confidence threshold and an intersection-union (IU) threshold are set to remove candidate boxes with confidence below the threshold and redundant detection boxes with overlap above the IU threshold. The final ship detection result is then output, including the target location, target category, and confidence level.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] This invention constructs a heterogeneous dual-view collaborative perception architecture that integrates high-altitude wide-area overhead images from an aerial observation platform with close-up water surface images from a surface observation platform. This eliminates the inherent blind spots of a single viewpoint at the information source level and compensates for the detection blind spots of the aerial observation platform's high-altitude overhead view for low-lying vessels and obscured targets near the water surface, significantly reducing the false negative rate. By designing a channel-space hybrid attention mechanism to dynamically weight and fuse features from both views, the weights of vessel-related feature channels and the spatial region where the target is located are adaptively strengthened. This effectively suppresses environmental interference such as water surface reflection, wave textures, and shoreline shadows, resulting in detection stability far superior to a single aerial observation platform perspective solution. The combination of high-resolution close-up features from the surface observation platform and wide-area positioning features from the aerial observation platform significantly improves the detection accuracy for both distant small targets and close-range targets. Employing a lightweight backbone network with parameter sharing and a hybrid attention fusion architecture, the system boasts high computational efficiency and can run in real-time on an airborne embedded computing platform, meeting the real-time requirements of border and coastal defense patrols. The two perspectives are redundant. When a single perspective is disturbed by the environment, the other perspective can provide effective supplementation, and the overall robustness of the system is significantly enhanced. Attached Figure Description

[0028] Figure 1 This is a flowchart of the overall process of the hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method of the present invention.

[0029] Figure 2 This is a schematic diagram of the overall network architecture of the present invention, which features shared backbone dual-path feature extraction and hybrid attention fusion.

[0030] Figure 3 This is a schematic diagram of the channel-space hybrid attention module of the present invention.

[0031] Figure 4 This is a schematic diagram of the deployment of the air-sea collaborative sensing system of the present invention.

[0032] Figure 5 This is a comparison chart of the overall performance of the method of the present invention with the single UAV perspective, the single unmanned surface vessel perspective, and the simple fusion of dual perspectives under actual shore-sea conditions.

[0033] Figure 6 This is a comparison chart of the recall rates of the method of the present invention with those of a single UAV perspective, a single unmanned surface vessel perspective, and a simple dual-view fusion method under normal lighting conditions, strong reflection / backlighting, wave texture interference, near-shore shadow / occlusion, and distant small target conditions. Detailed Implementation

[0034] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0035] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0036] like Figures 1 to 4 As shown, a hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method includes the following steps:

[0037] S1, simultaneously acquire two heterogeneous visual images, the first of which is a first-view image provided by the first imaging device mounted on the aerial observation platform, and the second of which is a second-view image acquired by the second imaging device mounted on the water surface observation platform. The two images constitute a multi-view input source that complements the air and sea.

[0038] S2 uses a lightweight convolutional neural network as the shared feature extraction backbone to encode features of the first-view image and the second-view image respectively, and outputs two feature maps of the same dimension. The first feature map corresponds to the first-view image and the second feature map corresponds to the second-view image.

[0039] S3. The two feature maps output from step S2 are dynamically weighted and deeply fused using a channel-space hybrid attention mechanism. The channel attention module adaptively learns and strengthens the feature channels useful for ship detection and suppresses redundant background channels. The spatial attention module focuses on the spatial area where the ship is located and weakens the area of ​​environmental interference. The two feature maps after attention weighting are then spliced ​​and fused element by element to form a joint feature map that has both wide-area positioning capability and close-up detail capability.

[0040] S4. Input the joint feature map obtained in step S3 into the detection head. After bounding box regression, category classification and confidence prediction, the ship detection result is output. The ship detection result includes the target location, target category and confidence.

[0041] The following provides further explanation of each step.

[0042] In step S1, the system simultaneously acquires two heterogeneous visual images, constructing a multi-view input source that complements air and sea perspectives. In this embodiment, the aerial observation platform is specifically a drone, and the first imaging device is specifically a visible light / infrared dual-light pod; the water surface observation platform is specifically an unmanned surface vessel, and the second imaging device is specifically a shipborne visible light / infrared camera. As a specific implementation of this embodiment, such as... Figure 4As shown, the air-sea cooperative perception system includes unmanned aerial vehicles (UAVs) that perform patrol missions in the air and unmanned surface vessels (USVs) that perform patrol missions on the water.

[0043] The drone, equipped with a dual-light (visible / infrared) pod, flies at an altitude of 50-200 meters, providing wide-area coverage imaging of the monitored waters from a high-altitude, top-down perspective. The visible light channel is suitable for target detection under good visibility conditions during the day, while the infrared channel is suitable for target detection under low visibility conditions such as at night or in foggy weather. The high-altitude, wide-area top-down images acquired by the drone can cover a large area of ​​water, enabling coarse target localization and global situational awareness. However, due to limitations in shooting altitude, distant and small vessel targets have small pixel areas in the images, limiting their feature representation capabilities.

[0044] The unmanned surface vessel (USV) is equipped with a shipborne visible light / infrared camera, enabling close-up photography of ship targets on the water from a near-field, eye-level perspective. Positioned on the same horizontal plane as the target, the USV can acquire high-resolution local details such as the ship's side profile, deck structure, and name markings, as well as environmental information like water texture and wave patterns. This close-up detail is unavailable from aerial views, and the two images complement each other in terms of perspective, shooting distance, and scene content.

[0045] In practical applications, UAVs and unmanned surface vessels maintain time synchronization and share location information via a communication link, ensuring that the two images are correlated in time and space. Both images undergo unified preprocessing operations, including size normalization and pixel value normalization, to meet the format requirements of subsequent network input.

[0046] In step S2, a lightweight convolutional neural network is constructed as the backbone network for shared feature extraction. Its overall network architecture is as follows: Figure 2 As shown in the diagram. In this embodiment, the shared feature extraction backbone network adopts a lightweight architecture design based on depthwise separable convolutions, consisting of multiple stacked depthwise separable convolutional modules. Six modules are used as an example below. Each depthwise separable convolutional module contains a depthwise convolutional layer and a pointwise convolutional layer. The depthwise convolutional layer performs independent spatial convolution operations on each channel of the input feature map, with a kernel size of 3×3, used to extract local spatial features within each channel. The pointwise convolutional layer performs a 1×1 convolution operation on the output of the depthwise convolutional layer, used for feature fusion and channel mapping along the channel dimension. Each depthwise separable convolutional module is followed by a batch normalization layer and a ReLU activation function layer. The batch normalization layer accelerates network training convergence and improves model generalization ability, while the ReLU activation function layer introduces non-linear expressive power.

[0047] The feature maps output by each depthwise separable convolutional module have 32, 64, 128, 128, 256, and 256 channels, respectively. This increasing channel count allows the network to extract low-level texture and edge features in shallow layers and high-level semantic and target structural features in deeper layers. Compared to standard convolution, depthwise separable convolution decouples spatial convolution from channel convolution, significantly reducing the number of parameters and computational cost. This enables the entire backbone network to achieve real-time inference on airborne embedded computing platforms for UAVs and unmanned surface vessels.

[0048] Preprocessed UAV and unmanned surface vessel (USV) images are input into the shared feature extraction backbone network. Both images share the same network structure and parameter weights, and feature encoding and extraction are performed separately. After layer-by-layer processing through six depthwise separable convolutional modules, two high-level semantic feature maps of the same dimension are output. and ,in Characterizing wide-area information in the upper atmosphere, Representing close-up details, C is the number of feature channels, and H and W are the height and width of the feature map, respectively. The parameter-sharing design ensures that the two feature maps reside in the same feature space, providing a consistent semantic basis for subsequent feature fusion operations.

[0049] Preferably, the number of depthwise separable convolutional modules can be flexibly selected from 5 to 8, and the number of feature map channels output by each module increases sequentially; the specific values ​​of the number of modules and the number of channels are selected by trade-off based on the computing power and detection accuracy requirements of the target embedded platform, so as to ensure that the method can achieve real-time inference on the airborne embedded computing platform of UAVs and unmanned surface vessels.

[0050] In step S3, such as Figure 3 As shown, a channel-spatial hybrid attention mechanism is introduced into the two feature maps output in step S2 for dynamic weighting and deep fusion. By sequentially executing three sub-steps—channel attention weighting, spatial attention weighting, and dual-path feature deep fusion—accurate fusion of features from both perspectives is achieved.

[0051] S3.1, Channel Attention Weighting: The purpose of channel attention is to adaptively learn and strengthen feature channels that are useful for ship detection, while suppressing the weights of redundant channels such as water background and wave texture.

[0052] Specifically, for the two feature maps and Global average pooling and global max pooling operations are performed respectively. Global average pooling averages the feature values ​​at all spatial locations of each channel to obtain the global mean response of that channel, reflecting its overall activation level. Global max pooling takes the maximum value at each spatial location of each channel to obtain its most salient feature response. The four pooling results are then fed into a shared multilayer perceptron (MLP), which consists of a fully connected dimensionality reduction layer, a ReLU activation function layer, and a fully connected dimensionality increase layer, with a dimensionality reduction ratio of r=16. The four MLP outputs are then element-wise summed and mapped to the [0,1] interval using a Sigmoid activation function to generate the channel attention weight map. The calculation formula is:

[0053]

[0054] Where AvgPool represents global average pooling operation, MaxPool represents global max pooling operation, MLP represents multilayer perceptron, and σ represents sigmoid activation function;

[0055] Channel attention weight map The channel-weighted feature map is obtained by performing element-wise multiplication (i.e., channel-dimensional broadcast multiplication) with the two original feature maps respectively. After channel attention weighting, the weights of feature channels related to ship target detection are enhanced, while the weights of background and interference-related channels are suppressed.

[0056] S3.2, Spatial Attention Weighting: The purpose of spatial attention is to focus on the spatial region where the ship target is located in the image and weaken the weight of interference areas such as waves, reflections, and shorelines.

[0057] Specifically, the two feature maps after channel attention weighting are... and Average pooling and max pooling operations are performed along the channel dimension, with each pooling operation compressing C channels into one channel, resulting in four single-channel feature maps that reflect the mean and maximum responses of the channels at each spatial location. These four single-channel feature maps are then concatenated along the channel dimension to form a 4-channel feature description tensor. This tensor is then passed through a convolutional layer with a 7×7 kernel and one output channel for spatial feature aggregation. Finally, the spatial attention weight map is mapped to the [0,1] interval using a sigmoid activation function. The calculation formula is:

[0058]

[0059] Spatial attention weight map Element-wise multiplication (i.e., spatial dimension broadcast multiplication) is performed with the weighted feature maps of the two channels respectively to obtain the feature map after channel-space hybrid attention weighting: .

[0060] S3.3, Dual-path feature deep fusion: Deeply fuse the two feature maps after hybrid attention weighting.

[0061] Specifically, firstly and The two feature maps are concatenated along the channel dimension to obtain a concatenated feature map with 2C channels. This is then passed through a convolutional layer with a 1×1 kernel and C output channels for channel dimensionality reduction and cross-view channel information interaction. Simultaneously, the two feature maps are element-wise summed to preserve the direct superposition information of the two features at the same spatial location. Finally, the concatenated dimensionality reduction result and the element-wise summation result are summed again element-wise to obtain the final joint feature map. The calculation formula is:

[0062]

[0063] This fusion strategy preserves the complete channel information of the two perspective features through splicing operations and enhances the consistency of the two features in spatial location through element-by-element addition operations. The combination of the two enables the joint feature map to simultaneously possess the wide-area positioning capability of UAVs and the close-range detail recognition capability of unmanned surface vessels.

[0064] In step S4, the joint feature map obtained in step S3 is... The input detection head network performs ship target detection. The detection head network includes parallel classification and regression branches. The classification branch consists of several convolutional and fully connected layers, predicting the class probability and target confidence at each preset anchor point position of the joint feature map, outputting the probability value of the presence of a ship target at each position and the corresponding target class. The regression branch also consists of several convolutional and fully connected layers, predicting the target bounding box coordinate offset at each preset anchor point position, including the offset of the bounding box center point and the width and height offset.

[0065] During the training phase, the cross-entropy loss function is used to optimize the classification branch, and the smoothed L1 loss function is used to optimize the regression branch. The total loss function is the weighted sum of the classification loss and the regression loss. During the inference phase, non-maximum suppression (NMS) is applied to all candidate detection results output by the detection head network. A confidence threshold and an intersection-overlap (IoU) threshold are set to remove candidate boxes with confidence scores below the threshold and redundant detection boxes with overlap scores above the IoU threshold. The final ship detection result is then output, including the target location (boundary box coordinates), target category, and confidence score.

[0066] Because the joint feature map integrates the high-altitude wide-area features of UAVs and the close-up details of unmanned surface vessels, and undergoes adaptive weighted optimization through a hybrid attention mechanism, the detection head network can maintain a high detection rate and positioning accuracy in various complex scenarios, including the detection of small targets at long distances, targets occluded near the water surface, and targets under conditions of strong light and water mist interference.

[0067] To further verify the detection effect of the method of the present invention under actual working conditions, aerial top view images and water surface view images acquired simultaneously in actual shore-sea operation scenarios were selected for comparative experiments.

[0068] The comparison methods include: single UAV perspective, which uses only the high-altitude wide-area overhead view image acquired by the UAV for detection; single unmanned surface vessel perspective, which uses only the close-up water surface view image acquired by the unmanned surface vessel for detection; simple dual-view fusion, which fuses the features of the two perspectives by directly stitching or adding them element by element without introducing a hybrid attention mechanism; and the method of this invention, which uses a channel-space hybrid attention mechanism to dynamically weight and deeply fuse the features of the two perspectives.

[0069] Figure 5 The overall performance comparison results of the four methods described above are shown. Precision represents the proportion of real ship targets detected, characterizing false positive control capability; Recall represents the proportion of correctly detected actual ship targets, characterizing detection completeness; F1 represents the harmonic mean of precision and recall, used to comprehensively evaluate detection accuracy and detection completeness; mAP@0.5 represents the average precision when the Intersection over Union (IoU) threshold is 0.5; mAP@0.5:0.95 represents the average precision when the IoU threshold ranges from 0.5 to 0.95, with intervals of 0.05, used to more rigorously evaluate target localization and detection performance; False negative rate represents the proportion of actual ship targets not detected, which in this embodiment is equal to 1 minus Recall; Inference speed represents the number of image frames processed per second. Except for the false negative rate, higher values ​​of the above indicators indicate better performance; lower false negative rates indicate fewer false negatives.

[0070] Figure 5In this study, the method of the present invention achieved the following results: Precision, Recall, F1, mAP@0.5, mAP@0.5:0.95, false negative rate, and inference speed were 95.2%, 93.6%, 94.4%, 96.1%, 70.9%, 6.4%, and 34.2 FPS, respectively. Compared to the simple dual-view fusion method, the method of the present invention improved Precision, Recall, F1, mAP@0.5, and mAP@0.5:0.95 by 3.7, 5.4, 4.6, 4.4, and 6.1 percentage points, respectively, and reduced the false negative rate by 5.4 percentage points. Compared to the single UAV perspective and the single unmanned surface vessel perspective, the method of the present invention also exhibited higher detection accuracy and lower false negative rate. Although the inference speed decreased by 4.4 FPS after introducing the hybrid attention mechanism compared to the simple dual-view fusion method, it still achieved 34.2 FPS, which can meet the real-time detection requirements of actual patrol scenarios.

[0071] Figure 6 The recall rate comparison results under different complex working conditions are further presented. Normal lighting condition refers to a water surface scene with stable lighting conditions and clear contrast between the target and background; strong reflection / backlight condition refers to a scene where the contrast between the target and background is reduced due to specular reflection or backlighting on the water surface; wave texture interference condition refers to a scene where the high-frequency dynamic texture formed by waves is easily confused with ship features; near-shore shadow / occlusion condition refers to a scene where the shoreline, buildings, dock facilities, or other targets produce shadows or partial occlusion; far-distance small target condition refers to a scene where the ship occupies few pixels in the image and has weak effective features. The false detection count / 100 frames represents the number of non-real ship detection results generated per 100 frames processed under the corresponding working condition; the lower this index, the stronger the false detection suppression capability.

[0072] Under five operating conditions—normal lighting, strong reflection / backlight, wave texture interference, near-shore shadow / occlusion, and distant small target—the recall rates from a single UAV perspective were 91.3%, 72.6%, 75.4%, 70.8%, and 65.2%, respectively; from a single unmanned surface vessel perspective, they were 92.0%, 78.4%, 80.2%, 76.1%, and 69.8%, respectively; and from a simple dual-view fusion perspective, they were 95.6%, 86.7%, 87.3%, 84.5%, and 80.4%, respectively. The recall rates of the method of this invention were 97.5%, 92.1%, 91.8%, 90.6%, and 89.4%, respectively. Correspondingly, the false positives per 100 frames for the method of this invention were 4.2, 7.9, 8.6, 9.4, and 10.6, respectively.

[0073] Depend on Figure 6It can be seen that, compared with the simple dual-view fusion method, the recall rate of the method of the present invention is improved by 1.9, 5.4, 4.5, 6.1 and 9.0 percentage points respectively under the above five working conditions; and it still maintains a recall rate of about 90% or above under adverse working conditions such as strong reflection / backlight, wave texture interference, near-shore shadow / occlusion and distant small targets. This result shows that the channel-space hybrid attention mechanism can adaptively highlight the ship-related channels and target areas, suppress interference such as reflection, wave texture, shadow and occlusion, and improve the detection capability of distant small targets through the complementary information of air and sea dual perspectives, thereby improving the stability and robustness of ship detection under complex coastal and sea conditions.

[0074] In summary, this embodiment constructs a heterogeneous air-sea dual-view collaborative perception architecture, fusing a high-altitude wide-area overhead view from a UAV with a close-up water surface view from an unmanned surface vessel (USV). It extracts dual-path, same-dimensional features through a lightweight, deeply separable convolutional backbone network with shared parameters. A channel-space hybrid attention mechanism dynamically weights and fuses the dual-view features, adaptively enhancing target features and suppressing environmental interference. Finally, a detection head network outputs high-precision ship detection results. This embodiment achieves more accurate, stable, and interference-resistant ship detection performance than a single UAV perspective in complex coastal and marine environments, and can be directly applied to engineering scenarios such as intelligent coastal patrol, autonomous obstacle avoidance by USVs, and air-sea collaborative operations.

Claims

1. A hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method, characterized in that, Includes the following steps: S1, simultaneously acquire two heterogeneous visual images, the first of which is a first-view image provided by the first imaging device mounted on the aerial observation platform, and the second of which is a second-view image acquired by the second imaging device mounted on the water surface observation platform. The two images constitute a multi-view input source that complements the air and sea. S2 uses a lightweight convolutional neural network as the shared feature extraction backbone to encode features of the first-view image and the second-view image respectively, and outputs two feature maps of the same dimension. The first feature map corresponds to the first-view image and the second feature map corresponds to the second-view image. S3. The two feature maps output from step S2 are dynamically weighted and deeply fused using a channel-space hybrid attention mechanism. The channel attention module adaptively learns and strengthens the feature channels useful for ship detection and suppresses redundant background channels. The spatial attention module focuses on the spatial area where the ship is located and weakens the area of ​​environmental interference. The two feature maps after attention weighting are then spliced ​​and fused element by element to form a joint feature map that has both wide-area positioning capability and close-up detail capability. S4. Input the joint feature map obtained in step S3 into the detection head. After bounding box regression, category classification and confidence prediction, the ship detection result is output. The ship detection result includes the target location, target category and confidence.

2. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 1, characterized in that, In step S1, the aerial observation platform uses a drone, and the first imaging device uses a visible light / infrared dual-light pod; the water surface observation platform uses an unmanned surface vessel, and the second imaging device uses a shipborne visible light / infrared camera. The simultaneous acquisition of two heterogeneous visual images specifically involves: a UAV equipped with a visible light / infrared dual-light pod taking wide-area photos of the monitored water area from a high-altitude top-down perspective, acquiring a high-altitude wide-area top-down image containing the global distribution of targets in the water area; and an unmanned surface vessel equipped with a shipborne visible light / infrared camera taking close-up photos of water surface targets from a close-up level-down perspective, acquiring a close-up level-down image of the water surface containing high-resolution details of local ships and water surface textures. The UAV image and the unmanned surface vessel image complement each other in terms of viewing direction, shooting distance, and scene content.

3. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 1, characterized in that, In step S2, a lightweight convolutional neural network is constructed as a shared feature extraction backbone network. The shared feature extraction backbone network consists of multiple depthwise separable convolutional modules stacked sequentially. Each depthwise separable convolutional module includes a depthwise convolutional layer and a pointwise convolutional layer. The depthwise convolutional layer is used to extract local features in the spatial dimension, and the pointwise convolutional layer is used to perform feature fusion and channel mapping in the channel dimension. Each depthwise separable convolutional module is followed by a batch normalization layer and a ReLU activation function layer.

4. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 3, characterized in that, The shared feature extraction backbone network has 5-8 depthwise separable convolutional modules, and the number of feature map channels output by each depthwise separable convolutional module increases sequentially.

5. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 1, characterized in that, Step S3 specifically includes the following steps: S3.1, Perform global average pooling and global max pooling operations on the two feature maps output from step S2 respectively, and feed the pooling results into the shared multilayer perceptron to obtain channel description vectors; after adding the channel description vectors element by element, generate channel attention weight maps through the Sigmoid activation function; multiply the channel attention weight maps element by element with the two original feature maps respectively to obtain channel weighted feature maps; S3.2, after channel attention weighting, the two feature maps are subjected to average pooling and max pooling operations along the channel dimension respectively. The results are concatenated along the channel dimension and then passed through a convolutional layer with a kernel size of 7×7 and a sigmoid activation function to generate a spatial attention weight map. The spatial attention weight map is multiplied element-wise with the two channel weighted feature maps to obtain the feature map after channel-spatial hybrid attention weighting; S3.3 concatenates the two feature maps after hybrid attention weighting along the channel dimension, then performs channel dimensionality reduction through a 1×1 convolutional layer. Simultaneously, the two feature maps are added element-wise. Finally, the concatenated dimensionality reduction result and the element-wise addition result are added element-wise again to obtain the joint feature map.

6. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 5, characterized in that, In step S3.1, the shared multilayer perceptron consists of a fully connected dimensionality reduction layer, a ReLU activation function layer, and a fully connected dimensionality increase layer in sequence, with a dimensionality reduction ratio of r=16.

7. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 1, characterized in that, In step S4, the joint feature map obtained in step S3 is input into the detection head network for ship target detection; the detection head network includes parallel classification and regression branches; The classification branch consists of several convolutional and fully connected layers, which predict the class probability and target confidence at each preset anchor point position of the joint feature map, and output the probability value of the presence of the ship target at each position and the corresponding target class; the regression branch consists of several convolutional and fully connected layers, which predict the target bounding box coordinate offset at each preset anchor point position, including the bounding box center point coordinate offset and width and height offset.

8. The hybrid attention-driven air-sea heterogeneous multi-view fusion ship detection method according to claim 7, characterized in that, In step S4, during the training phase, the cross-entropy loss function is used to optimize the classification branch, and the smooth L1 loss function is used to optimize the regression branch. The total loss function is the weighted sum of the classification loss and the regression loss. During the inference phase, non-maximum suppression is performed on all candidate detection results output by the detection head network. A confidence threshold and an intersection-union (IU) threshold are set to remove candidate boxes with confidence below the threshold and redundant detection boxes with overlap above the IU threshold. The final ship detection result is then output, including the target location, target category, and confidence level.

Images