Image ship detection method and system based on cross-modal wavelet attention

Abstract

The application discloses an image ship detection method based on cross-modal wavelet attention, comprising the following steps: extracting a visible light feature map and an infrared feature map; splicing to form a low-frequency fusion component; weighted fusion to form a high-frequency fusion component; component fusion to obtain a dual-modal feature; feature fusion to obtain a multi-scale dual-modal feature; and obtaining a ship detection result. The application discloses an image ship detection system based on cross-modal wavelet attention, comprising a dual-branch feature extraction module, a cross-modal wavelet attention fusion module, a multi-scale feature fusion module and a ship detection head module connected in sequence. The image ship detection method and system based on cross-modal wavelet attention fuse high-frequency and low-frequency features of different modalities, and then further guide and enhance the high-frequency features, fully utilize the high-frequency and low-frequency features of different modalities, and thus improve the accuracy and reliability of target detection.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and relates to an image ship detection method based on cross-modal wavelet attention, as well as an image ship detection system based on cross-modal wavelet attention. Background Technology

[0002] Ship inspection plays a crucial role in both civilian and military fields, serving as a key link in ensuring maritime safety, promoting maritime trade, and protecting the environment. Furthermore, target detection in remote sensing images is one of the core research directions in the field of remote sensing image interpretation. Against this backdrop, ship target detection technology based on remote sensing images enables large-scale monitoring of distant sea areas, greatly enriching the monitoring capabilities of military and maritime departments.

[0003] Visible light and infrared images are two widely used remote sensing image sources for ship detection. Visible light remote sensing images possess rich color and texture information and significant structural features, which helps extract clear target features, crucial for identifying the details and shape of ship targets. However, they are susceptible to weather interference; in complex backgrounds or with cloud cover, false positives and false negatives may occur. Compared to visible light remote sensing images, infrared remote sensing images have lower resolution and less clear edge textures of ship targets, but they possess irreplaceable advantages such as strong environmental adaptability, strong anti-interference ability, and strong cloud and fog penetration, making them an important supplement to visible light images. Currently, deep learning-based ship target detection methods are mainly divided into two-stage and single-stage methods. Two-stage target detection methods divide the detection problem into two stages: region proposal and candidate region classification. However, while achieving high accuracy, two-stage target detection methods reduce computational speed. Single-stage target detection methods, on the other hand, do not require the region proposal stage in a two-stage method and can directly provide the target classification and bounding box. Compared to two-stage target detection methods, single-stage target detection methods offer faster detection speeds, meeting the demands of real-time ship detection. Furthermore, by fully integrating and utilizing the visible light and infrared characteristics of the ship, comprehensive ship information can be obtained to address the complex and ever-changing marine environment. Therefore, multimodal single-stage ship target detection methods are more suitable for ship target detection tasks.

[0004] While multimodal single-stage target detection methods are suitable for ship target detection tasks, they still present challenges. Different modalities contain different information in their high and low frequency features. Current research on frequency domain feature fusion methods is limited to frequency band decomposition and simple superposition, failing to fully utilize the low and high frequency features of different modalities. Therefore, it is necessary to consider how to fully leverage the low and high frequency features of different modalities to obtain more comprehensive ship information. Summary of the Invention

[0005] The purpose of this invention is to provide an image ship detection method based on cross-modal wavelet attention, which solves the problem of insufficient utilization of frequency domain features in ship detection in the prior art.

[0006] Another objective of this invention is to provide an image-based ship detection system based on cross-modal wavelet attention.

[0007] The technical solution adopted in this invention is an image ship detection method based on cross-modal wavelet attention, comprising: Step 1: Obtain the visible light-infrared dataset; Step 2: Extract visible light feature maps and infrared feature maps; Step 3: Calculate the bidirectional fusion features and splice them to form a low-frequency fusion component; Step 4: Calculate cross-modal attention weights and perform weighted fusion to form high-frequency fusion components; Step 5: Fuse the low-frequency fusion component and the high-frequency fusion component to obtain dual-modal features; Step 6: Fuse the dual-modal features to obtain multi-scale dual-modal features; Step 7: After processing the multi-scale dual-modal features, the ship inspection results are obtained.

[0008] The invention is further characterized by: Step 1 includes: Step 1.1: Establish a visible light remote sensing image dataset and a long-wave infrared dataset for ships; Step 1.2: Using the visible light remote sensing image dataset as the content image and the long-wave infrared dataset as the style image, generate a pseudo-infrared dataset with a long-wave infrared style. Step 1.3: Pair the visible light remote sensing image dataset and the pseudo-infrared dataset to form a visible light-infrared dataset; Step 2 involves inputting the paired visible-infrared dataset into the dual-branch feature extraction module to extract feature maps at multiple different scales; the feature maps include visible light feature maps and infrared feature maps.

[0009] Step 3 includes: Step 3.1: For visible light feature maps and infrared feature maps at the same scale, generate the corresponding queries, keys, and values ​​respectively; Step 3.2: Convert the format using tensor rearrangement operations; The initial shape is transformed into the format required by the multi-head attention mechanism using tensor rearrangement operations; Step 3.3: Calculate the fusion features after querying the infrared mode from the visible light mode and the fusion features after querying the visible light mode from the infrared mode, and then splice the two fusion features to form a low-frequency fusion component; Step 3.3.1: Calculate the fusion features after querying the infrared mode from the visible light mode; Step 3.3.2: Calculate the fusion features after querying the visible light mode from the infrared mode; Step 3.3.3: Combine the two fusion features to form a low-frequency fusion component.

[0010] Step 4 includes: Step 4.1: For the visible light feature map and infrared feature map at the same scale, separate the components in the horizontal, vertical and diagonal directions respectively; Step 4.2: Calculate the cross-modal attention weight for each component, then introduce a learnable bias weight parameter, and perform weighted fusion on the components in the corresponding direction based on the cross-modal attention weight and bias weight parameter. Stack the multimodal adaptive fusion features obtained in the three directions to form a high-frequency fusion component. Step 5 includes: Step 5.1: After concatenating the low-frequency fusion component and the high-frequency fusion component along the channel dimension, input them into the cross-band interaction module to generate cross-band attention weights; Step 5.1.1: The low-frequency guided high-frequency enhancement module first bilinearly upsamples the low-frequency fusion component to the spatial size of the current high-frequency sub-band, and then splices it with the high-frequency fusion component along the three channel dimensions respectively. Step 5.1.2: Input the spliced ​​low-frequency fusion component and high-frequency fusion component along the channel dimension into the cross-band interaction module to generate cross-band attention weights; Step 5.2: Based on cross-band attention weights, fuse the low-frequency fusion component and the high-frequency fusion component to obtain dual-modal features.

[0011] Step 6 includes: Step 6.1: Input the dual-modal features at different scales into the multi-scale feature fusion module; Step 6.2: Starting from the deepest features, perform upsampling operations on them to expand the spatial size of the feature map and make it consistent with the scale of the middle-layer features; Step 6.3: Concatenate the upsampled deepest features and mid-level features along the channel dimension, and complete feature extraction and channel adjustment through convolution operation to obtain the mesoscale fused features; Step 6.4: Continue to upsample the mesoscale fused features to make them consistent with the scale of the shallow features; Step 6.5: Concatenate the upsampled mesoscale fusion features with the shallow features along the channel dimension, and obtain the shallow scale fusion features through convolution operation; Step 6.6: Start with shallow-scale fusion features and perform downsampling to reduce the feature map spatial size and increase the number of channels; Step 6.7: Concatenate the downsampled shallow-scale fusion features with the corresponding mesoscale fusion features along the channel dimension, and obtain new mesoscale fusion features through convolution operation; Step 6.8: Continue to downsample the new mesoscale fusion feature and concatenate and convolve it with the deep-scale fusion feature to obtain the new deep-scale fusion feature; Step 6.9: Output the dual-modal fusion feature maps at three scales: shallow, medium, and deep.

[0012] Step 7 includes: Step 7.1: Perform feature enhancement and channel adjustment on the multi-scale bimodal features at each scale; Step 7.2: For each scale feature, generate a series of anchor frames according to the preset anchor frame size and proportion, match the anchor frames with the real target, and determine the positive and negative samples. Step 7.3: For each matching positive sample anchor box, the detection head predicts the probability of its class and the offset of the bounding box; Step 7.4: Normalize the category predictions to obtain the confidence score for each category; Bounding box regression predicts the center point offset, width offset, and height offset of the anchor box relative to the true target box; Step 7.5: The predicted bounding boxes and class confidence scores will undergo post-processing to remove redundant bounding boxes and retain the most accurate ship detection results.

[0013] Another technical solution adopted in this invention is an image ship detection system based on cross-modal wavelet attention, which includes a dual-branch feature extraction module, a cross-modal wavelet attention fusion module, a multi-scale feature fusion module, and a ship detection head module connected in sequence. The dual-branch feature extraction module includes a visible light feature extraction branch and an infrared feature extraction branch. The visible light feature extraction branch is connected to the visible light image input module, the infrared feature extraction branch is connected to the infrared image input module, and the ship detection head module is connected to the result output module.

[0014] Another feature of the present invention is that: The cross-modal wavelet attention fusion module includes a discrete wavelet transform unit, which is connected to the visible light convolution preprocessing unit, the infrared convolution preprocessing unit, the bidirectional low-frequency attention fusion unit, and the direction-specific multimodal attention fusion unit, respectively. The bidirectional low-frequency attention fusion unit and the direction-specific multimodal attention fusion unit are connected to the low-frequency guided high-frequency enhancement unit, which is connected to the inverse wavelet transform unit. The inverse wavelet transform and residual output unit are connected to the bidirectional low-frequency attention fusion unit, the visible light convolution preprocessing unit, and the infrared convolution preprocessing unit.

[0015] The bidirectional low-frequency attention fusion unit includes a multi-head tensor rearrangement unit, which is connected to the bidirectional attention computation unit, query convolution unit, key convolution unit, and value convolution unit respectively. The bidirectional attention computation unit is connected to the Dropout unit, the Dropout unit is connected to the channel splicing unit, and the channel splicing unit is connected to the output projection convolution unit. The direction-specific multimodal attention fusion unit includes a learnable bias weight unit, a high-frequency direction fusion unit, and a direction dimension stacked output unit connected in sequence. The high-frequency direction fusion unit is connected to the horizontal direction attention branch, the vertical direction attention branch, and the diagonal direction attention branch, respectively. The horizontal direction attention branch, the vertical direction attention branch, and the diagonal direction attention branch are connected to the infrared SE channel attention unit and the visible light SE channel attention unit, respectively. The infrared SE channel attention unit is connected to the infrared direction feature flattening unit, and the visible light SE channel attention unit is connected to the visible light direction feature flattening unit. The low-frequency guided high-frequency enhancement unit includes a low-frequency upsampling unit, which is connected to a cross-band splicing unit and a low-frequency guided convolution unit. The low-frequency guided convolution unit is connected to a high-frequency enhancement fusion unit, the cross-band splicing unit is connected to a cross-band interaction unit, the cross-band interaction unit is connected to a high-frequency enhancement fusion unit, and the high-frequency enhancement fusion unit is connected to a high-frequency sub-band stacking output unit. The inverse wavelet transform and residual output unit includes an inverse wavelet transform unit, a size alignment unit, a residual connection unit, an activation function unit, and an output convolution unit connected in sequence.

[0016] The visible light image input module receives visible light images and outputs visible light input image information; The infrared image input module receives infrared images and outputs infrared input image information. Input visible light image information into the visible light feature extraction branch to obtain multi-scale visible light features; input infrared image information into the infrared feature extraction branch to obtain multi-scale infrared features; The dual-branch feature extraction module outputs visible light and infrared features at three scales respectively. Visible light and infrared features at the same scale are then input into the cross-modal wavelet attention fusion module. In the cross-modal wavelet attention fusion module: first, convolutional preprocessing is performed separately to obtain visible light enhancement features and infrared enhancement features; then, these features are input into a discrete wavelet transform unit to obtain visible light low-frequency subbands, visible light high-frequency subbands, infrared low-frequency subbands, and infrared high-frequency subbands; finally, the visible light low-frequency subbands and infrared low-frequency subbands are input into a bidirectional low-frequency attention fusion unit to obtain low-frequency fused features. The visible light high-frequency subband and the infrared high-frequency subband are input into a direction-specific multimodal attention fusion unit to obtain high-frequency fusion features. The low-frequency fusion features and the high-frequency fusion features are input into a low-frequency guided high-frequency enhancement unit, which uses low-frequency global semantic information to guide the selection and enhancement of high-frequency edge texture information, resulting in enhanced high-frequency fusion features. The low-frequency fusion features and the enhanced high-frequency fusion features are input into an inverse wavelet transform unit to reconstruct spatial domain fusion features. The spatial domain fusion features are then residually fused with the convolutionally preprocessed visible light enhancement features and infrared enhancement features to obtain dual-modal fusion features at the corresponding scale. The three-scale dual-modal fusion features are input into the multi-scale feature fusion module to obtain multi-scale enhanced fusion features; The multi-scale enhanced fusion features are input into the ship detection head module, which outputs the ship target's category prediction results, position prediction results, and confidence prediction results; the result output module generates the final ship detection result based on the detection head output results.

[0017] The beneficial effects of this invention are: This invention decomposes visible light and infrared features into low-frequency subbands containing global structure and contour information and high-frequency subbands containing edge texture information in horizontal, vertical, and diagonal directions through discrete wavelet transform. It breaks through the limitation of existing technologies that simply decompose and superimpose frequency bands, and realizes differentiated and refined utilization of high and low frequency features of different modes, fundamentally solving the problem of insufficient utilization of frequency domain features in multimodal ship detection.

[0018] The bidirectional low-frequency attention fusion unit constructs a bidirectional attention flow between visible light and infrared modes and models global spatial dependencies in parallel based on a multi-head attention mechanism. This enables the low-frequency features of the two modes to actively acquire complementary global structural information from each other. While preserving the global semantics of low-frequency features, it adaptively enhances key structural features and suppresses redundant noise between modes through dynamic weight allocation, thereby improving the effectiveness and robustness of bimodal low-frequency feature fusion.

[0019] The direction-specific multimodal attention fusion unit is designed with three independent attention branches for the directional characteristics of high-frequency features. It combines SE (Squeeze-and-Excitation) channel attention to enhance important directional features and introduces learnable bias weight parameters. It prioritizes the use of clear edge texture information of the visible light mode in the horizontal and vertical directions, and prioritizes the use of significant thermal radiation edge information of the infrared mode in the diagonal direction. This achieves adaptive cross-modal fusion of high-frequency features in different directions and accurately enhances the multi-directional edge and detail representation of ship targets.

[0020] The low-frequency guided high-frequency enhancement unit uses global semantic information in the low-frequency fusion features to generate cross-band attention weights and low-frequency guidance signals to dynamically guide and enhance the high-frequency fusion features. When the cross-band attention weight is large, the high-frequency edge texture details are preserved. When the weight is small, low-frequency semantic information is introduced to suppress background clutter and false edges, which effectively solves the problem that high-frequency features are easily interfered with by noise and further improves the recognition of target edges and details.

[0021] The multi-scale feature fusion module adopts a bidirectional fusion path of top-down and bottom-up, realizing the bidirectional transmission and complementarity of deep semantic features and shallow detail features. The output multi-scale fusion features have both rich global semantic information and fine spatial detail information. The residual connection introduced after inverse wavelet transform fuses the reconstructed fusion features with the original modality enhancement features, effectively reducing information loss in the wavelet domain processing and ensuring the preservation of key information of the original modality. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the network structure of the image ship detection method based on cross-modal wavelet attention provided in an embodiment of the present invention; Figure 2 A schematic diagram of the network structure of BLFAF (Bidirectional Low-Frequency Attention Fusion) provided in an embodiment of the present invention; Figure 3 A schematic diagram of the network structure of DSMAF (Direction-Specific Multimodal Attention Fusion) provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the network structure of LGHE (Low-frequency Guided High-frequency Enhancement) provided in an embodiment of the present invention; Figure 5This is a graph showing the results of a comparative experiment conducted on the HRSC2016 dataset according to an embodiment of the present invention; Figure 6 This is the first result image of a comparative experiment conducted on the DOTAv1.0 dataset according to an embodiment of the present invention; Figure 7 This is the second result figure of the comparative experiment conducted on the DOTAv1.0 dataset in this embodiment of the invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0024] Example 1 This invention proposes an image-based ship detection method based on cross-modal wavelet attention, including: Step 1: Obtain the visible light-infrared dataset; Step 2: Extract visible light feature maps and infrared feature maps; Step 3: Calculate the bidirectional fusion features and splice them to form a low-frequency fusion component; Step 4: Calculate cross-modal attention weights and perform weighted fusion to form high-frequency fusion components; Step 5: Fuse the low-frequency fusion component and the high-frequency fusion component to obtain dual-modal features; Step 6: Fuse the dual-modal features to obtain multi-scale dual-modal features; Step 7: After processing the multi-scale dual-modal features, the ship inspection results are obtained.

[0025] Example 2 Based on Example 1, this embodiment of the invention proposes step 1.1: establishing a visible light remote sensing image dataset and a long-wave infrared dataset for ships; Step 1.2: Using the visible light remote sensing image dataset as the content image and the long-wave infrared dataset as the style image, generate a pseudo-infrared dataset with a long-wave infrared style. Step 1.3: Pair the visible light remote sensing image dataset and the pseudo infrared dataset to form a visible light-infrared dataset.

[0026] Example 3 Based on Example 1, this embodiment of the invention proposes step 2, which includes: inputting the paired visible light-infrared dataset into the dual-branch feature extraction module to extract feature maps at multiple different scales; the feature maps include a visible light feature map Fr and an infrared feature map Ft, the visible light feature map Fr is divided into three scales of depth features Fr1, Fr2 and Fr3, and the infrared feature map Ft is divided into three scales of depth features Ft1, Ft2 and Ft3.

[0027] Example 4 Based on Example 1, this embodiment of the invention proposes step 3.1: For visible light feature maps and infrared feature maps at the same scale, generate corresponding queries, keys and values ​​respectively; For visible light feature map F VI and infrared feature map F IR This generates the corresponding query (Q), key (K), and value (V). That is, F VI The query Q generates visible light feature maps through Q_Conv convolutional layers. VI F IR The key K is used to generate the infrared feature map through K_Conv and V_Conv convolutional layers respectively. IR Sum V IR Meanwhile, to ensure two-way interaction, F IR The query Q generates infrared feature maps through the Q_Conv convolutional layer. IR , and F VI The key K is used to generate visible light feature maps through K_Conv and V_Conv convolutional layers respectively. VI Sum V VI .

[0028] Step 3.2: Convert the format using tensor rearrangement operations; After generating the query, key, and value, a tensor rearrange operation is used to transform the initial shape (b, C, h, w) into the format (b, num_heads, h) required by the multi-head attention mechanism. w, c_per_head), where b represents the batch size, C represents the number of channels, h represents the height, w represents the width, num_heads represents the number of heads in the attention mechanism, and c_per_head represents the dimension of each head, c_per_head = And the spatial dimension (h, w) is flattened to the sequence length h. w; then calculate the two fused features.

[0029] Step 3.3: Calculate the fusion features after querying the infrared mode from the visible light mode and the fusion features after querying the visible light mode from the infrared mode, and then splice the two fusion features to form a low-frequency fusion component; The calculation formulas for the fusion features after querying the infrared mode from the visible light mode and the fusion features after querying the visible light mode from the infrared mode are as follows: , , in, and These represent the fusion features after querying the infrared mode from the visible light mode and the fusion features after querying the visible light mode from the infrared mode, respectively. "Trans" indicates transpose. Represents the dot product. This represents the softmax function. Indicates regularization, Indicates multiplication; Step 3.3.1: Calculate the fusion features after querying the infrared mode from the visible light mode; Bond K of visible light feature map VI Transpose the last two dimensions of K, and then transpose the resulting K. VI Query Q with infrared feature map IR Perform a dot product operation to obtain the first attention matrix. Scale the first attention matrix using a scaling factor. Then, normalize and dropout the scaled first attention matrix sequentially. Finally, apply the obtained attention weights to the value V of the visible light feature map. VI Perform weighted aggregation to generate fused features after querying infrared modes from visible light modalities; Step 3.3.2: Calculate the fusion features after querying the visible light mode from the infrared mode; Key K of infrared feature map IR The last two dimensions are transposed to obtain a shape of (b, num_heads, c_per_head, h). The characteristics of w) will lead to the transposed K IR Query Q with visible light feature map VI Performing a dot product operation yields the second attention matrix, which has the shape (b, num_heads, h). w, h w), the matrix represents Each position in Attention weights at all locations; to stabilize the training process and avoid the vanishing gradient problem, the second attention matrix is ​​scaled using a scaling factor (*scale). The scaled second attention matrix is ​​then normalized using the softmax function and subjected to Dropout to prevent overfitting. The resulting attention weights are then used to adjust the value V of the infrared feature map. IR Weighted aggregation is performed to generate fused features after querying visible light modes from infrared modalities.

[0030] Step 3.3.3: Combine the two fusion features to form a low-frequency fusion component; The two fused features are recombine from the multi-head format into a spatial feature map through tensor rearrangement operations, and then concatenated along the channel dimension to obtain a shape of (b, 2). The concatenated features (C, h, w) are projected back to the original channel dimensions through a convolutional layer to obtain the low-frequency fused component. The calculation formula is as follows: , in, For low-frequency fusion components, This indicates tensor rearrangement. Indicates splicing, This represents convolution.

[0031] The entire module processes information from different subspaces in parallel through a multi-head attention mechanism, achieving bidirectional and efficient feature interaction between the low-frequency features of the visible light mode and the infrared mode.

[0032] The BLFAF module achieves adaptive enhancement of cross-modal complementary information and noise suppression by constructing a bidirectional attention flow between visible and infrared modes. Based on a multi-head attention mechanism, this module models global spatial dependencies in parallel, automatically capturing key inter-modal information while preserving low-frequency global structural information through a dynamic weight allocation mechanism.

[0033] Example 5 Based on Example 1, this embodiment of the invention proposes step 4.1: for visible light feature maps and infrared feature maps at the same scale, separate the components in the horizontal, vertical and diagonal directions respectively; Before the DSMAF module separates the horizontal, vertical, and diagonal components from the visible light feature map and the infrared feature map, the visible light feature map... With infrared feature map The original shape is (b, C, 3, h, w). A reshape operation integrates orientation-related features into the channel dimension, generating a shape of (b, 3). The flattened feature map (C, h, w) is then processed by the SE module to enhance channel attention. The SE-enhanced flattened feature map is then reshaped to restore the original structured multi-directional features. and Finally, the horizontal, vertical and diagonal components are separated from the multi-directional features, and the directional gradient information is transformed into a non-directional edge intensity representation by taking the absolute value.

[0034] The SE module's workflow is as follows: First, through compression, global average pooling is used to compress the spatial dimension (h, w) of the feature map to 1×1, thereby capturing the global context information of each channel. Next, in the activation phase, a bottleneck structure built from two fully connected layers is used to learn the non-linear dependencies between channels, and the Sigmoid function is used to generate weight values ​​(between 0 and 1) for each channel. Finally, through scaling, these weights are applied channel-by-channel to the original feature map, achieving adaptive recalibration of the feature channels to enhance useful features and suppress irrelevant noise. By dynamically adjusting the feature weights of the channel dimensions, the SE module can not only enhance important features and suppress redundant information, but also simultaneously enhance the correlation between features in the horizontal, vertical, and diagonal directions.

[0035] This invention employs three independent attention branches, each consisting of two layers of 1 The algorithm consists of one convolutional layer and one sigmoid activation function. These branches take the concatenated features from the corresponding directions of the infrared and visible light modes as input, and dynamically calculate the cross-modal attention weights, as shown in the following formula: , , , in, , and These represent the components of the visible light channel in the horizontal, vertical, and diagonal directions, respectively. , and These represent the components of the infrared channel in the horizontal, vertical, and diagonal directions, respectively. , and These represent three independent attention branches in the horizontal, vertical, and diagonal directions, respectively. , and These represent the cross-modal attention weights in the horizontal, vertical, and diagonal directions, respectively.

[0036] Step 4.2: Calculate the cross-modal attention weight for each component, then introduce a learnable bias weight parameter, and perform weighted fusion on the components in the corresponding direction based on the cross-modal attention weight and bias weight parameter. Stack the multimodal adaptive fusion features obtained in the three directions to form a high-frequency fusion component. This invention introduces learnable bias weight parameters: a horizontal bias weight parameter beta_LH, a vertical bias weight parameter beta_HL, and a diagonal bias weight parameter beta_HH, with initial values ​​of 0.9, 0.9, and 0.4, respectively. In the horizontal and vertical directions, since visible light typically possesses clearer edge details, a weighted fusion strategy biased towards visible light is adopted. In the diagonal direction, given the significant advantage of infrared modes in rendering thermal radiation edges, a weighted fusion strategy biased towards infrared is employed. Through an explicitly designed bias mechanism combined with dynamic attention weights, adaptive fusion of multimodal features in the three directions is achieved. Finally, the processed features are stacked along the directional dimensions, outputting a high-frequency fusion component Fused-yh with a shape of (b, C, 3, h, w).

[0037] The calculation formula for multimodal adaptive fusion features is shown below: , , , in, , and These represent the multimodal adaptive fusion features in the horizontal, vertical, and diagonal directions, respectively. Indicates addition. , and These represent the components of the visible light channel in the horizontal, vertical, and diagonal directions, respectively. , and These represent the components of the infrared channel in the horizontal, vertical, and diagonal directions, respectively. , and These represent the cross-modal attention weights in the horizontal, vertical, and diagonal directions, respectively. , , These represent the learnable bias weight parameters in the horizontal, vertical, and diagonal directions, respectively. The DSMAF module employs an explicit bias design, prioritizing high-resolution details from the visible light modality in the horizontal / vertical direction and high-contrast edges from the infrared modality in the diagonal direction. Combined with a dynamic attention mechanism, this significantly enhances the flexibility and effectiveness of multimodal high-frequency feature fusion.

[0038] Example 6 Based on Example 1, this embodiment of the invention proposes step 5.1: concatenating the low-frequency fusion component and the high-frequency fusion component along the channel dimension and inputting them into the cross-band interaction module to generate cross-band attention weights; Step 5.1.1: The LGHE module first bilinearly upsamples the low-frequency fusion component to the spatial size of the current high-frequency sub-band, and then splices it with the high-frequency fusion component along the three channel dimensions respectively.

[0039] Step 5.1.2: Input the spliced ​​low-frequency fusion component and high-frequency fusion component along the channel dimension into the cross-band interaction module to generate cross-band attention weights; The calculation formula is as follows: , , , in, , and These represent the high-frequency sub-bands of the high-frequency fusion components in the horizontal, vertical, and diagonal directions, respectively. This represents the low-frequency subband after upsampling of the low-frequency fused components; CIB represents the cross-band interaction module. , and These represent the cross-band attention weights in the horizontal, vertical, and diagonal directions, respectively. In one embodiment of the invention, the cross-band interaction module consists of two 1×1 convolutional layers and a sigmoid activation function.

[0040] Step 5.2: Based on cross-band attention weights, fuse the low-frequency fusion component and the high-frequency fusion component to obtain dual-modal features; The upsampled low-frequency subband passes through 1 1. Convolution is converted into a low-frequency pilot signal High-frequency features retain their edge texture details under attention weighting while incorporating global contextual information guided by low frequencies. Finally, the fused high-frequency subbands are stacked along the directional dimension to output the fused result, i.e., the bimodal features.

[0041] High-frequency subbands are dynamically fused using the following formula:

[0042]

[0043]

[0044] in, , and These represent the high-frequency subbands after low-frequency guidance enhancement and fusion in the horizontal, vertical, and diagonal directions, respectively.

[0045] The bimodal features at different scales are denoted as F1, F2 and F3. F1 is obtained by modal fusion of Fr1 and Ft1, F2 is obtained by modal fusion of Fr2 and Ft2, and F3 is obtained by modal fusion of Fr3 and Ft3.

[0046] Example 7 Based on Example 1, this embodiment of the invention proposes step 6.1, which involves inputting the dual-modal features F1, F2 and F3 of different scales into the multi-scale feature fusion module as the basic feature input for multi-scale fusion.

[0047] Step 6.2: Starting from the deepest feature F3, perform an upsampling operation on it to expand the spatial size of the feature map so that it is consistent with the scale of the middle feature F2.

[0048] Step 6.3: Concatenate the upsampled F3 with the mid-level feature F2 along the channel dimension, and complete feature extraction and channel adjustment through convolution operation to obtain the mid-scale fused feature.

[0049] Step 6.4: Continue to upsample the mesoscale fused features to make them consistent with the scale of the shallow features F1.

[0050] Step 6.5: Concatenate the upsampled mesoscale fusion features with the shallow features F1 along the channel dimension, and obtain the shallow scale fusion features through convolution operation, thereby enhancing the spatial detail representation.

[0051] Step 6.6: Start with shallow-scale fusion features and perform downsampling to reduce the feature map space size and increase the number of channels, so as to realize the transfer of shallow detailed information to deep semantic features.

[0052] Step 6.7: Concatenate the downsampled shallow-scale fusion features with the corresponding mesoscale fusion features along the channel dimension, and obtain new mesoscale fusion features through convolution operation.

[0053] Step 6.8: Continue to downsample the new mesoscale fusion feature and concatenate and convolve it with the deep-scale fusion feature to obtain the new deep-scale fusion feature.

[0054] Step 6.9: Finally, output the dual-modal fusion feature map with three scales: shallow, medium and deep, to provide multi-level and multi-scale feature information for the subsequent detection head.

[0055] Example 8 Based on Example 1, this embodiment of the invention proposes step 7.1: passing the multi-scale dual-modal features of each scale through one or more convolutional layers for feature enhancement and channel adjustment, so as to better adapt to the detection task; One embodiment of the present invention employs a detection head based on YOLOv7. Step 7.2: For each scale feature, generate a series of anchor boxes according to the preset anchor box size and scale. By calculating the IoU (Intersection over Union) and other indicators between the anchor boxes and the real target boxes, match the anchor boxes with the real targets and determine the positive and negative samples.

[0056] Step 7.3: For each matching positive sample anchor box, the detection head predicts its class probability and the offset of the bounding box.

[0057] Step 7.4: Class prediction typically uses a sigmoid or softmax function for normalization to obtain the confidence score for each class. Bounding box regression predicts the center offset, width offset, and height offset of the anchor box relative to the true target box.

[0058] Step 7.5: The predicted bounding boxes and class confidence scores will undergo post-processing operations such as non-maximum suppression (NMS) to remove redundant bounding boxes and retain the most accurate detection results. Example 9 This invention proposes an image ship detection system based on cross-modal wavelet attention, comprising a dual-branch feature extraction module, a cross-modal wavelet attention fusion module, a multi-scale feature fusion module, and a ship detection head module connected in sequence. The dual-branch feature extraction module includes a visible light feature extraction branch and an infrared feature extraction branch. The visible light feature extraction branch is connected to the visible light image input module, and the infrared feature extraction branch is connected to the infrared image input module. The ship detection head module is connected to the result output module.

[0059] In this invention, the visible light image input module receives visible light remote sensing images to obtain visible light input information; the infrared image input module receives infrared remote sensing images spatially registered with the visible light image to obtain infrared input information; the dual-branch feature extraction module extracts multi-scale features from the visible light and infrared images respectively, outputting visible light and infrared features at different scales; the cross-modal wavelet attention fusion module performs frequency domain decomposition, low-frequency fusion, high-frequency fusion, cross-band enhancement, and wavelet reconstruction on visible light and infrared features at the same scale to obtain dual-modal fusion features at the corresponding scale; the multi-scale feature fusion module performs top-down and bottom-up multi-scale fusion on dual-modal fusion features at different scales to obtain multi-scale fusion features that combine semantic and detail information; and the ship detection head module performs ship target classification, position regression, and confidence prediction based on the multi-scale fusion features, outputting ship target detection results.

[0060] Example 10 Building upon Example 9, this embodiment of the invention proposes a dual-branch feature extraction module: a visible light feature extraction branch performs convolution, normalization, activation, and downsampling processing on the visible light image to extract texture, edge, structural, and semantic information, outputting multi-scale visible light features. An infrared feature extraction branch performs convolution, normalization, activation, and downsampling processing on the infrared image to extract thermal radiation distribution, target contours, and salient region information, outputting multi-scale infrared features.

[0061] In one embodiment of the present invention, the dual-branch feature extraction module outputs feature maps at three scales, with dimensions of 80×80×512, 40×40×1024, and 20×20×1024, respectively. The cross-modal wavelet attention fusion module includes a discrete wavelet transform unit, which is connected to the visible light convolution preprocessing unit, the infrared convolution preprocessing unit, the bidirectional low-frequency attention fusion unit, and the direction-specific multimodal attention fusion unit, respectively. The bidirectional low-frequency attention fusion unit and the direction-specific multimodal attention fusion unit are connected to the low-frequency guided high-frequency enhancement unit, which is connected to the inverse wavelet transform unit. The inverse wavelet transform and residual output unit are connected to the bidirectional low-frequency attention fusion unit, the visible light convolution preprocessing unit, and the infrared convolution preprocessing unit. The internal processing of the cross-modal wavelet attention fusion module is as follows: Visible light characteristic F VI After processing by the visible light convolution preprocessing unit, the visible light enhanced feature F is obtained. VI ′;Infrared signature F IR After processing by the infrared convolution preprocessing unit, the infrared enhanced feature F is obtained. IR ′; Discrete wavelet transform units respectively for F VI ′ and F IR Wavelet decomposition is performed to obtain low-frequency subbands and high-frequency subbands; the low-frequency subbands are used to represent the global structure of the image and the overall contour of the target, while the high-frequency subbands are used to represent edge texture information in the horizontal, vertical and diagonal directions. The visible light low-frequency sub-band and the infrared low-frequency sub-band are input into the bidirectional low-frequency attention fusion unit, which outputs low-frequency fusion features. Visible light high-frequency subband and infrared high-frequency subband input direction-specific multimodal attention fusion unit, output high-frequency fusion features; Low-frequency fusion features and high-frequency fusion features are input into the low-frequency guided high-frequency enhancement unit. Cross-band attention is used to guide and enhance high-frequency details through low-frequency semantics, and the enhanced high-frequency fusion features are output. The inverse wavelet transform and residual output unit reconstruct the spatial domain based on the low-frequency fusion features and the enhanced high-frequency fusion features to obtain the reconstructed fusion features. The reconstructed fusion features are then residually fused with the visible light enhancement features and the infrared enhancement features, and after processing, the dual-mode fusion features are obtained.

[0062] The bidirectional low-frequency attention fusion unit includes a multi-head tensor rearrangement unit, which is connected to the bidirectional attention computation unit, query convolution unit (Q_Conv), key convolution unit (K_Conv), and value convolution unit (V_Conv). The bidirectional attention computation unit is connected to a Dropout unit, which is connected to a channel splicing unit, and the channel splicing unit is connected to an output projection convolution unit.

[0063] The internal processing of the bidirectional low-frequency attention fusion unit is as follows: Input visible light low-frequency features and infrared low-frequency features; query convolutional units, key convolutional units, and value convolutional units perform 1×1 convolutional projection on the low-frequency features of the two modes respectively to obtain the corresponding query matrix, key matrix, and value matrix; for the visible light to infrared direction, the query matrix is ​​generated using the visible light low-frequency features, and the key matrix and value matrix are generated using the infrared low-frequency features. The cross-modal attention weight of visible light to infrared is calculated, and the infrared value features are weighted and aggregated to obtain the first low-frequency interaction feature; for the infrared to visible light direction, the query matrix is ​​generated using the infrared low-frequency features, and the key matrix and value matrix are generated using the visible light low-frequency features. The cross-modal attention weight of infrared to visible light is calculated, and the visible light value features are weighted and aggregated to obtain the second low-frequency interaction feature; the first low-frequency interaction feature and the second low-frequency interaction feature are concatenated along the channel dimension to obtain the bidirectional low-frequency concatenated feature; the output projection convolutional unit performs 1×1 convolutional dimensionality reduction on the bidirectional low-frequency concatenated feature to output the low-frequency fusion feature Fusedyl.

[0064] The direction-specific multimodal attention fusion unit includes a learnable bias weight unit, a high-frequency direction fusion unit, and a direction dimension stacked output unit connected in sequence. The high-frequency direction fusion unit is connected to the horizontal direction attention branch, the vertical direction attention branch, and the diagonal direction attention branch, respectively. The horizontal direction attention branch, the vertical direction attention branch, and the diagonal direction attention branch are connected to the infrared SE channel attention unit and the visible light SE channel attention unit, respectively. The infrared SE channel attention unit is connected to the infrared direction feature flattening unit, and the visible light SE channel attention unit is connected to the visible light direction feature flattening unit.

[0065] The internal processing of the direction-specific multimodal attention fusion unit is as follows: The input infrared high-frequency features and visible light high-frequency features both include three directional sub-bands, namely horizontal, vertical and diagonal directional sub-bands; First, the directional dimension of the infrared high-frequency features and the visible light high-frequency features is expanded to the channel dimension to obtain the infrared flattened features and the visible light flattened features. Then, infrared flattening features and visible light flattening features are enhanced by channel attention through infrared SE channel attention units and visible light SE channel attention units, respectively, to highlight important directional features and suppress redundant directional responses; Subsequently, the enhanced flattened features were restored to high-frequency features containing directional dimensions, and infrared high-frequency features and visible light high-frequency features in the horizontal, vertical and diagonal directions were extracted respectively. For the horizontal direction, the infrared horizontal features and visible light horizontal features are concatenated and input into the horizontal direction attention branch to obtain the horizontal cross-modal attention weights; For the vertical direction, the infrared vertical features and visible light vertical features are concatenated and input into the vertical direction attention branch to obtain the vertical direction cross-modal attention weights. For the diagonal direction, the infrared diagonal features and the visible light diagonal features are concatenated and input into the diagonal direction attention branch to obtain the diagonal direction cross-modal attention weights; Furthermore, modal biases in different directions are set through learnable bias weighting units; wherein, the horizontal and vertical directions are biased towards the visible light mode, and the diagonal direction is biased towards the infrared mode; Finally, the high-frequency directional fusion unit adaptively fuses the infrared high-frequency features and the visible light high-frequency features according to the directional attention weight and the learnable bias weight, and stacks them along the directional dimension to output the high-frequency fused feature Fusedyh.

[0066] The low-frequency guided high-frequency enhancement unit includes a low-frequency upsampling unit, which is connected to a cross-band stitching unit and a low-frequency guided convolution unit. The low-frequency guided convolution unit is connected to a high-frequency enhancement fusion unit, the cross-band stitching unit is connected to a cross-band interaction unit, the cross-band interaction unit is connected to a high-frequency enhancement fusion unit, and the high-frequency enhancement fusion unit is connected to a high-frequency sub-band stacking output unit.

[0067] The internal processing of the low-frequency guided high-frequency enhancement unit is as follows: Input the low-frequency fusion feature Fusedyl and the high-frequency fusion feature Fusedyh; First, the low-frequency upsampling unit upsamples the low-frequency fused features to the spatial size of the current high-frequency sub-band to obtain the upsampled low-frequency features. Then, the cross-band stitching unit stitches the upsampled low-frequency features with the current high-frequency sub-band in the channel dimension to obtain the cross-band stitched features. Next, the cross-band interaction unit performs convolution, non-linear activation, and sigmoid processing on the cross-band stitched features to generate cross-band attention weights. At the same time, the low-frequency guiding convolution unit performs a 1×1 convolution transformation on the upsampled low-frequency features to obtain the low-frequency guiding signal; The high-frequency enhancement fusion unit dynamically fuses the current high-frequency sub-band and low-frequency guiding signal according to the cross-band attention weight. Specifically, when the cross-band attention weight is large, more edge texture information in the high-frequency sub-band is retained; when the cross-band attention weight is small, more low-frequency semantic guiding signal is introduced to suppress high-frequency noise and background clutter. Finally, the enhanced horizontal, vertical, and diagonal high-frequency subbands are stacked to output enhanced high-frequency fusion features.

[0068] The inverse wavelet transform and residual output unit includes an inverse wavelet transform unit, a size alignment unit, a residual connection unit, an activation function unit, and an output convolution unit connected in sequence.

[0069] The inverse wavelet transform and the internal processing of the residual output unit are as follows: The inverse wavelet transform unit receives the low-frequency fusion feature and the enhanced high-frequency fusion feature, and reconstructs them into spatial domain fusion features. If the size of the reconstructed spatial domain fused feature is inconsistent with the size of the input feature, the size alignment unit uses bilinear interpolation to adjust the size. Subsequently, the residual connection unit adds the reconstructed fusion features to the visible light enhancement features and the infrared enhancement features element by element to supplement the original modal information and reduce information loss during wavelet domain processing. The activation function unit performs a nonlinear mapping on the residual fusion result, and the output convolution unit further adjusts the channel dimension to obtain the final dual-modal fusion feature.

[0070] The internal processing of the multi-scale feature fusion module is as follows: It receives dual-modal fusion features across three scales; First, starting with the deep bimodal fusion features, the spatial size of the feature map is expanded through upsampling, and then channel splicing and convolution processing are performed with the mid-level bimodal fusion features to obtain the mid-scale enhanced fusion features; Then, the mesoscale enhanced fusion features are further upsampled and channel-joined and convolutionally processed with the shallow dual-modal fusion features to obtain the shallow scale enhanced fusion features; Subsequently, downsampling is performed starting from the shallow-scale enhanced fusion features, and then concatenation and convolution are performed with the medium-scale enhanced fusion features to achieve the transfer of detailed information to the semantic layer. Finally, the mesoscale enhanced fusion features are downsampled and then concatenated and convolved with the deep-scale dual-modal fusion features to obtain the deep-scale enhanced fusion features. The multi-scale feature fusion module ultimately outputs fused features at three scales: shallow, medium, and deep.

[0071] The ship detection head module includes a classification prediction branch, a position regression branch, and a confidence prediction branch. The classification prediction branch predicts whether the target belongs to a ship category; the position regression branch predicts the position parameters of the ship target detection box; the confidence prediction branch predicts the probability that the detection box contains a ship target; and the final output is the ship target detection result, including the ship category, detection box coordinates, and target confidence.

[0072] The internal processing procedures of each module in this invention are as follows: The visible light image input module receives visible light images and outputs visible light input image information; the infrared image input module receives infrared images and outputs infrared input image information. Input visible light image information into the visible light feature extraction branch to obtain multi-scale visible light features; input infrared image information into the infrared feature extraction branch to obtain multi-scale infrared features; The dual-branch feature extraction module outputs visible light and infrared features at three scales, denoted as follows: , , , , , Among them, F 1 F represents shallow high-resolution features. 2 F represents the mid-level features. 3 Represents deep, low-resolution semantic features; Visible light features and infrared features at the same scale are input into the cross-modal wavelet attention fusion module to obtain dual-modal fused features at the corresponding scale; In the cross-modal wavelet attention fusion module, convolutional preprocessing is first performed separately to obtain visible light enhancement features and infrared enhancement features; The visible light enhancement features and infrared enhancement features are respectively input into the discrete wavelet transform unit to obtain the visible light low-frequency sub-band, visible light high-frequency sub-band, infrared low-frequency sub-band and infrared high-frequency sub-band; The visible light low-frequency subband and the infrared low-frequency subband are input into the bidirectional low-frequency attention fusion unit to obtain the low-frequency fusion features. The visible light high-frequency subband and the infrared high-frequency subband are input into a direction-specific multimodal attention fusion unit to obtain high-frequency fusion features; The low-frequency fusion features and high-frequency fusion features are input into the low-frequency guided high-frequency enhancement unit, so that the low-frequency global semantic information guides the selection and enhancement of high-frequency edge texture information, and the enhanced high-frequency fusion features are obtained. The low-frequency fusion features and the enhanced high-frequency fusion features are input into the inverse wavelet transform unit to reconstruct the spatial domain fusion features. The spatial domain fusion features are residually fused with the visible light enhancement features and infrared enhancement features after convolutional preprocessing to obtain the dual-modal fusion features at the corresponding scale. The three-scale dual-modal fusion features are input into the multi-scale feature fusion module to obtain multi-scale enhanced fusion features; The multi-scale enhanced fusion features are input into the ship detection head module, which outputs the ship target's category prediction results, position prediction results, and confidence prediction results. The results output module generates the final ship inspection results based on the output of the inspection head.

[0073] In one embodiment of the present invention, the system input is a set of spatially registered visible light images and infrared images. The visible light image is denoted as I. VI Infrared image denoted as I IR .

[0074] The input size settings for visible light and infrared images are shown in Table 1. Table 1

[0075] After normalization, the input image is fed into the visible light feature extraction branch and the infrared feature extraction branch, respectively.

[0076] In one embodiment of the present invention, the dual-branch feature extraction module includes a visible light feature extraction branch and an infrared feature extraction branch. The two branches extract features from the visible light image and the infrared image respectively, obtaining three depth features at different scales. The three scale features output by the visible light feature extraction branch are denoted as follows: , , The three scale features output by the infrared feature extraction branch are denoted as follows: , , .

[0077] In one specific implementation, the feature parameters for the three scales are shown in Table 2: Table 2

[0078] The first-scale features primarily preserve ship edges, textures, and local details; the second-scale features combine local structural and semantic information; and the third-scale features mainly contain global semantic information about the ship target. After training, the model parameters stored in the dual-branch feature extraction module include the weights of the visible light branch convolutional layers, the weights of the infrared branch convolutional layers, normalized layer parameters, activation layer parameters, and downsampling structure parameters for each stage.

[0079] In one embodiment of the present invention, a cross-modal wavelet attention fusion module is used to fuse the visible light features and infrared features at each scale. This module is used to decompose the visible light features and infrared features into low-frequency sub-bands and high-frequency sub-bands, and to perform low-frequency structure fusion, high-frequency direction fusion and low-frequency guided high-frequency enhancement, respectively.

[0080] The main parameters of the cross-modal wavelet attention fusion module are shown in Table 3. Table 3

[0081] For visible light characteristics at any scale and infrared features In the model (i=1,2,3), initial feature extraction is performed by passing the data through convolutional layers with a kernel size of 3×3 and a padding size of 1, yielding visible light enhancement features and infrared enhancement features. Subsequently, discrete wavelet transforms are applied to both to obtain visible light low-frequency sub-bands, visible light high-frequency sub-bands, infrared low-frequency sub-bands, and infrared high-frequency sub-bands. The low-frequency sub-bands represent the overall structure, spatial layout, and contour information of the ship target; the high-frequency sub-bands represent the edge details and texture abrupt changes of the ship target in the horizontal, vertical, and diagonal directions.

[0082] After training, the parameters stored in the cross-modal wavelet attention fusion module include convolutional preprocessing layer parameters, low-frequency attention parameters, high-frequency directional attention parameters, cross-band interaction parameters, low-frequency guided convolution parameters, and output convolution parameters.

[0083] In one embodiment of the present invention, a bidirectional low-frequency attention fusion unit is used to perform bidirectional interactive fusion of visible light low-frequency features and infrared low-frequency features. This module models the global spatial dependency between visible light modes and infrared modes through a multi-head attention mechanism.

[0084] The parameters of the bidirectional low-frequency attention fusion unit are shown in Table 4.

[0085] Table 4

[0086] Where C represents the number of channels of the input feature at the current scale. When the number of input feature channels is 512, C=512; when the number of input feature channels is 1024, C=1024. During processing, visible light low-frequency features generate query features through query convolution, and infrared low-frequency features generate key and value features through key and value convolution, thus obtaining cross-modal attention results from visible light to infrared. Simultaneously, infrared low-frequency features generate query features through query convolution, and visible light low-frequency features generate key and value features through key and value convolution, thus obtaining cross-modal attention results from infrared to visible light. Then, the low-frequency interaction features from the two directions are concatenated along the channel dimension and projected back to the original number of channels through a 1×1 convolution to obtain low-frequency fused features. After training, the model parameters saved by this module include Q-projection convolution parameters, K-projection convolution parameters, V-projection convolution parameters, output projection convolution parameters, and attention dropout parameters.

[0087] In one embodiment of the present invention, a direction-specific multimodal attention fusion unit is used to perform direction-aware fusion of visible light high-frequency features and infrared high-frequency features. The high-frequency features include sub-band features in three directions: horizontal, vertical, and diagonal. The parameters of the direction-specific multimodal attention fusion unit are shown in Table 5. Table 5

[0088] In the processing, infrared and visible light high-frequency features are first expanded from the directional dimension to the channel dimension, forming a feature map with 3C channels. Then, feature enhancement is performed using the SE channel attention module to highlight important directional features and suppress redundant information. Next, the enhanced high-frequency features are restored to structured features containing the directional dimension, and high-frequency components in the horizontal, vertical, and diagonal directions are extracted. For each direction, the infrared and visible light directional features are concatenated along the channel dimension and input into the corresponding directional attention branch to generate cross-modal attention weights for that direction. In the horizontal and vertical directions, the fusion strategy favors the visible light modality to fully utilize the relatively clear edge texture information in the visible light image; in the diagonal direction, the fusion strategy favors the infrared modality to utilize the more significant thermal radiation edge information in the infrared image. After training, the module saves model parameters including infrared SE module parameters, visible light SE module parameters, horizontal direction attention branch parameters, vertical direction attention branch parameters, diagonal direction attention branch parameters, and learnable bias weight parameters for the three directions.

[0089] In one embodiment of the present invention, the low-frequency guided high-frequency enhancement unit is used to guide and enhance the high-frequency fusion features by utilizing the global semantic information in the low-frequency fusion features, so as to suppress background noise and enhance the edge texture information related to the ship target.

[0090] The parameters of the low-frequency guided high-frequency enhancement unit are shown in Table 6. Table 6

[0091] During processing, the low-frequency fusion features are first upsampled to the spatial size of the current high-frequency sub-band using bilinear interpolation. Then, the upsampled low-frequency fusion features are concatenated with the current high-frequency sub-band along the channel dimension and input into the cross-band interaction unit to generate cross-band attention weights. Simultaneously, the upsampled low-frequency fusion features undergo a 1×1 convolution to generate a low-frequency guiding signal. Subsequently, the high-frequency sub-band is dynamically fused with the low-frequency guiding signal based on the cross-band attention weights. When the cross-band attention weights are large, more edge texture information from the high-frequency sub-band is preserved; when the cross-band attention weights are small, more low-frequency semantic guiding signals are introduced to reduce the interference of high-frequency noise on the detection results. After training, the module saves model parameters including the 1×1 convolution parameters, 3×3 convolution parameters, low-frequency guiding 1×1 convolution parameters, and corresponding bias parameters of the cross-band interaction unit.

[0092] In one embodiment of the present invention, the low-frequency fusion feature and the enhanced high-frequency fusion feature are input into an inverse wavelet transform unit and reconstructed into spatial domain fusion features through inverse discrete wavelet transform. The parameters of the inverse wavelet reconstruction and fusion output module are shown in Table 7. Table 7

[0093] If the spatial domain fusion feature size obtained after inverse wavelet transform is inconsistent with the input feature size, bilinear interpolation is used for size alignment. Subsequently, the reconstructed spatial domain fusion feature is added to the visible light enhancement feature and infrared enhancement feature using residuals to supplement the original modal information and reduce information loss caused by wavelet domain processing. Finally, the dual-modal fusion feature at the current scale is obtained through the GELU activation function and a 1×1 output convolutional layer. In one embodiment of the invention, if the number of input feature channels is 512, the number of output dual-modal fusion feature channels is 1024; if the number of input feature channels is 1024, the number of output dual-modal fusion feature channels is 2048.

[0094] In one embodiment of the present invention, the multi-scale feature fusion module receives bimodal fusion features from three scales and performs multi-scale fusion through top-down and bottom-up paths. The input features of the multi-scale feature fusion module include: , , ,in, This represents a shallow fusion feature with high spatial resolution. This is a mid-level fusion feature. This involves creating deep fusion features with strong semantic information. The process begins by upsampling the deep fusion features and concatenating and convolving them with the mid-level fusion features to obtain enhanced mid-level fusion features. Then, the enhanced mid-level fusion features are upsampled and concatenated and convolved with the shallow fusion features to obtain enhanced shallow fusion features. Subsequently, downsampling is performed on the enhanced shallow fusion features, and they are fused again with the mid-level fusion features. This downsampling is then repeated before fusion with the deep fusion features, resulting in multi-scale fusion features that simultaneously contain both detailed and semantic information. After training, the multi-scale feature fusion module saves model parameters including upsampling connection structure parameters, downsampling convolution parameters, lateral connection convolution parameters, and convolutional fusion parameters after feature concatenation.

[0095] In one embodiment of the present invention, the detection head module is used for ship target detection based on multi-scale fused features. The detection head module includes a classification prediction branch, a position regression branch, and a confidence prediction branch. The parameters of the detection head module are shown in Table 8. Table 8

[0096] After training, the model parameters saved by the detection head module include the convolution weights of the classification branch, the convolution weights of the location regression branch, the convolution weights of the confidence branch, and the corresponding bias parameters.

[0097] In one embodiment of the present invention, the trained image ship detection system based on cross-modal wavelet attention is saved in the form of a model weight file. This model weight file can be named best.pt, and it includes at least the following parameters: convolutional layer weights, normalization parameters, and bias parameters for the visible light feature extraction branch; convolutional layer weights, normalization parameters, and bias parameters for the infrared feature extraction branch; convolutional weights and bias parameters for the visible light convolutional preprocessing unit and the infrared convolutional preprocessing unit in the cross-modal wavelet attention fusion module; Q-projection convolution parameters, K-projection convolution parameters, V-projection convolution parameters, and output projection convolution parameters in the bidirectional low-frequency attention fusion unit; and infrared SE channel attention parameters and visible light SE channel attention parameters in the direction-specific multimodal attention fusion unit. Channel attention parameters, horizontal attention branch parameters, vertical attention branch parameters, and diagonal attention branch parameters; horizontal, vertical, and diagonal learnable bias weights in the direction-specific multimodal attention fusion unit; cross-band interactive convolution parameters and low-frequency guided convolution parameters in the low-frequency guided high-frequency enhancement unit; convolution weights and bias parameters of the output convolution module after inverse wavelet reconstruction; upsampling fusion parameters, downsampling fusion parameters, and lateral connection convolution parameters in the multi-scale feature fusion module; classification branch parameters, location regression branch parameters, and confidence prediction branch parameters in the detection head module.

[0098] Figure 1 This paper illustrates the overall structure of a cross-modal wavelet hierarchical attention feature fusion network based on a certain embodiment of the present invention. The network employs a dual-branch structure with visible light and infrared branches, extracting features from visible light and infrared images respectively. Cross-modal wavelet attention fusion modules are introduced at different scales to achieve deep fusion of visible light and infrared modal information. Specifically, the upper branch is for visible light image feature extraction, and the lower branch is for infrared image feature extraction. The two input images are respectively processed through a multi-layer convolutional feature extraction network to obtain visible light and infrared features at different scales. Subsequently, at the corresponding scales, the visible light features F... VI and infrared signature F IR Input the cross-modal wavelet attention fusion module CWHAFF.

[0099] In the CWHAFF module, the visible light and infrared features are first decomposed into low-frequency and high-frequency subbands by performing Discrete Wavelet Transform (DWT) on each. The low-frequency subband primarily preserves the overall structure, contour, and global semantic information of the target; the high-frequency subband primarily preserves the edge, texture, and directional details of the target. For the low-frequency subband, a BLFAF module is used for bidirectional low-frequency attention fusion; for the high-frequency subband, a DSMAF module is used for direction-specific multimodal attention fusion. Subsequently, an LGHE module is used to guide and enhance the high-frequency detail information using low-frequency semantic information. Finally, the fused low-frequency and high-frequency subbands are restored to the spatial domain by IDWT inverse wavelet transform, and bimodal fused features are obtained through convolution and activation operations.

[0100] The fused features at different scales are further input into the feature fusion network on the right for multi-scale feature fusion. Finally, the detection head outputs the detection results of the ship target, including the target category, confidence level, and detection box position. Figure 1 The meanings of the Chinese and English names and their abbreviations are shown in Table 9.

[0101] Table 9

[0102] Figure 2 The structure of a bidirectional low-frequency attention fusion unit (BLFAF) in one embodiment of the present invention is shown. This module is mainly used to fuse visible light low-frequency features and infrared low-frequency features, and achieves complementary enhancement of low-frequency information from the two modes by constructing bidirectional cross-modal attention. The lower input in the figure is the visible light low-frequency feature F. VI and infrared low-frequency characteristics F IR For the left branch, the module generates a query vector Q using infrared features. IR Generate the key vector K using visible light features. VI Sum vector V VI This allows the calculation of the attention relationship from infrared to visible light. This branch enables infrared low-frequency structural information to actively extract complementary information from visible low-frequency features. For the right branch, the module generates a query vector Q using visible light features. VI Generate key vector K using infrared features IR Sum vector V IRThis allows for the calculation of the attention relationship from visible light to infrared. This branch enables the visible light low-frequency structural information to actively acquire thermal radiation structure supplementation from infrared low-frequency features. In both directions, the input features are first processed by Q_Conv, K_Conv, and V_Conv to obtain query, key, and value features; then, the tensor dimensions are adjusted through a rearrange operation to satisfy the multi-head attention calculation format; next, the dot product between the query and key is calculated and multiplied by a scale factor; then, Softmax is applied to obtain normalized attention weights, followed by Dropout to prevent overfitting; finally, the attention weights and value features are weighted and aggregated to obtain the cross-modal low-frequency fusion result for both directions. The output results from both directions are restored to spatial feature maps through rearrange, then concatenated along the channel dimension, and finally passed through a Conv convolutional layer to obtain the final low-frequency fusion feature. Figure 2 The meanings of the Chinese and English names and their abbreviations are shown in Table 10.

[0103] Table 10

[0104] Figure 3 The structure of a Direction-Specific Multimodal Attention Fusion Unit (DSMAF) according to one embodiment of the present invention is shown. This module is mainly used to fuse high-frequency features of visible light and infrared modes, with particular attention to edge and texture information in the horizontal, vertical, and diagonal directions. The input in the figure is the visible light high-frequency feature F. VI and infrared high-frequency characteristics F IR Since the high-frequency features after wavelet transform typically contain three directional sub-bands, namely LH, HL, and HH, the input features can be represented as high-frequency features containing three directional components.

[0105] First, the module performs a reshape operation on both the visible light and infrared high-frequency features, incorporating the orientation dimension into the channel dimension to obtain oriented unfolded features of size H×W×C×3. Then, the SE module is used to perform channel attention enhancement on the unfolded high-frequency features to highlight effective orientation features and suppress redundant information. Finally, a reshape operation is performed to restore the orientation dimension, resulting in the enhanced visible light high-frequency features. and enhanced infrared high-frequency characteristics Next, the module extracts the high-frequency components of the visible light and infrared modes in the LH, HL, and HH directions, respectively, and inputs them into the three directional attention branches: Attention_LH, Attention_HL, and Attention_HH. Each directional attention branch generates cross-modal attention weights for that direction by concatenating the visible light and infrared features of that direction. ) represents the attention weight in the horizontal direction, ( ) represents the attention weight in the vertical direction, ( ) represents the attention weights in the diagonal direction. To enhance the directional adaptability of the fusion, this module further introduces learnable bias weights ( ), ( ) and( In the horizontal and vertical directions, the fusion process is biased towards the visible light mode to take advantage of the clearer textures and boundaries in the visible light image; in the diagonal direction, the fusion process is biased towards the infrared mode to take advantage of the more significant thermal radiation edge information in the infrared image. Finally, the high-frequency features fused from the three directions are combined through a stacking operation to obtain multi-directional high-frequency fused features. Figure 3 The meanings of the Chinese and English names and abbreviations are shown in Table 11.

[0106] Table 11

[0107] Figure 4 The structure of a low-frequency guided high-frequency enhancement unit (LGHE) in one embodiment of the present invention is shown. The core function of this module is to utilize the global semantic information in the low-frequency fusion features to guide and enhance the edge texture information in the high-frequency fusion features, thereby reducing the interference of background noise and invalid textures on the detection results.

[0108] The input in the figure consists of two parts: a high-frequency fusion feature Fused-yh and a low-frequency fusion feature Fused-yl. Fused-yh contains high-frequency fusion sub-bands in three directions: LH, HL, and HH, primarily reflecting changes in ship edges, details, and textures; Fused-yl mainly contains the overall structure, outline, and global semantic information of the ship target. First, the low-frequency fusion feature Fused-yl is upsampled to the same spatial size as the high-frequency sub-bands. Then, the upsampled low-frequency feature is concatenated with the high-frequency sub-bands in the three directions and input into the CIB module. CIB generates attention weights in the three directions: attn_LH, attn_HL, and attn_HH. Simultaneously, the upsampled low-frequency feature is also processed by a Conv convolutional layer to generate low-frequency guiding features. For each high-frequency sub-band, the module dynamically fuses the high-frequency feature and the low-frequency guiding feature according to the corresponding attention weights. When the attention weight is large, the module retains more edge texture information of the high-frequency subband itself; when the attention weight is small, the module introduces more low-frequency semantic information, thereby suppressing background noise and false edges in the high-frequency subband. Finally, the high-frequency features enhanced in the three directions are combined through a stacking operation to output the low-frequency guided enhanced high-frequency fusion features. Figure 4The meanings of the Chinese and English names and abbreviations are shown in Table 12.

[0109] Table 12

[0110] In the ship detection results of a certain embodiment of the present invention, if a target object does not actually exist in the image, but the detection model detects it (i.e., incorrectly determines that the target object exists), this situation is considered (False Positive, FP). If a target object actually exists in the image, and the detection model correctly detects it (i.e., correctly determines that the target object exists), this situation is considered (True Positive, TP). If a target object does not actually exist in the image, and the detection model correctly determines that the target object does not exist (i.e., correctly determines that the target object does not exist), this situation is considered (True Negative, TN). If a target object actually exists in the image, but the detection model incorrectly determines that the target object does not exist (i.e., misses the target object), this situation is recorded as (False Negative, FN). To evaluate the effectiveness of the network, mean precision (AP) and mean precision average (mAP) are used as evaluation metrics. AP and mAP are calculated based on precision (P) and recall (R). Precision represents the ratio of correctly detected targets to the total number of detected targets, while recall represents the ratio of correctly detected targets to the total number of detected targets.

[0111] In one embodiment of the present invention, a single NVIDIA GeForce GTX 3080Ti GPU with 12GB of memory is used for training and testing. The dataset uses, as follows: Figure 5 The HRSC2016 dataset shown and as follows Figure 6 , Figure 7 The dataset shown is DOTAv1.0. In the diagram, "Baseline" represents the baseline model, "ours" refers to this invention, and "Ground truth" represents the true labels. Simulation content: To demonstrate the effectiveness of the method proposed in this invention, the following eight popular methods proposed in recent years were compared in the experiment. The quantitative average precision (mAP) of the method proposed in this invention and the other eight comparison algorithms on two datasets are shown in Tables 13, 14, and 15.

[0112] Table 13 Algorithm metrics for the HRSC2016 dataset

[0113] Table 14 Eight algorithm metrics for the DOTAv1.0 dataset

[0114] Table 15 shows eight other algorithmic metrics for the DOTA v1.0 dataset.

[0115] In Tables 14 and 15, PL represents airplane, BD represents baseball field, BR represents bridge, GTF represents ground athletic field, SV represents small vehicle, LV represents large vehicle, SH represents ship, TC represents tennis court, BC represents basketball court, ST represents storage tank, SBF represents football field, RA represents roundabout, HA represents port, SP represents swimming pool, and HC represents helicopter.

[0116] As shown in Tables 13-15, the effectiveness of the method proposed in this invention was verified on two datasets. Experimental results demonstrate that, compared to other existing methods, the method of this invention achieves optimal detection results in most cases.

[0117] The network structure parameters in one embodiment of the present invention are as follows: # parameters nc: 1 # number of classes depth_multiple: 1.0# model depth multiple width_multiple: 1.0# layer channel multiple # anchors anchors: - [12,16, 19,36, 40,28]# P3 / 8 - [36,75, 76,55, 72,146]# P4 / 16 - [142,110, 192,243, 459,401]# P5 / 32 # yolov7 backbone backbone1: # [from, number, module, args] [[-1, 1, Conv, [32, 3, 1]],# 0 [-1, 1, Conv, [64, 3, 2]],# 1-P1 / 2 [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [128, 3, 2]],# 3-P2 / 4 [-1, 1, Conv, [64, 1, 1]], [-2, 1, Conv, [64, 1, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [256, 1, 1]],# 11 [-1, 1, MP, []], [-1, 1, Conv, [128, 1, 1]], [-3, 1, Conv, [128, 1, 1]], [-1, 1, Conv, [128, 3, 2]], [[-1, -3], 1, Concat, [1]],# 16-P3 / 8 [-1, 1, Conv, [128, 1, 1]], [-2, 1, Conv, [128, 1, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [512, 1, 1]],# 24 [-1, 1, MP, []], [-1, 1, Conv, [256, 1, 1]], [-3, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 2]], [[-1, -3], 1, Concat, [1]],# 29-P4 / 16 [-1, 1, Conv, [256, 1, 1]], [-2, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [1024, 1, 1]],# 37 [-1, 1, MP, []], [-1, 1, Conv, [512, 1, 1]], [-3, 1, Conv, [512, 1, 1]], [-1, 1, Conv, [512, 3, 2]], [[-1, -3], 1, Concat, [1]],# 42-P5 / 32 [-1, 1, Conv, [256, 1, 1]], [-2, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [1024, 1, 1]],# 50 ] backbone2: # [from, number, module, args] [[-1, 1, Conv, [32, 3, 1]],# 51 [-1, 1, Conv, [64, 3, 2]],# 52 [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [128, 3, 2]],# 54 [-1, 1, Conv, [64, 1, 1]], [-2, 1, Conv, [64, 1, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [256, 1, 1]],# 62 [-1, 1, MP, []], [-1, 1, Conv, [128, 1, 1]], [-3, 1, Conv, [128, 1, 1]], [-1, 1, Conv, [128, 3, 2]], [[-1, -3], 1, Concat, [1]],# 67 [-1, 1, Conv, [128, 1, 1]], [-2, 1, Conv, [128, 1, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [512, 1, 1]],# 75 [-1, 1, MP, []], [-1, 1, Conv, [256, 1, 1]], [-3, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 2]], [[-1, -3], 1, Concat, [1]],# 80 [-1, 1, Conv, [256, 1, 1]], [-2, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [1024, 1, 1]],# 88 [-1, 1, MP, []], [-1, 1, Conv, [512, 1, 1]], [-3, 1, Conv, [512, 1, 1]], [-1, 1, Conv, [512, 3, 2]], [[-1, -3], 1, Concat, [1]],# 93 [-1, 1, Conv, [256, 1, 1]], [-2, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [[-1, -3, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [1024, 1, 1]],# 101 # yolov7 head head: [[[-1,50], 1, BimodalFusion,

[1024] ], # 102 [-1, 1, SPPCSPC,

[512] ], # 103 [-1, 1, Conv, [256, 1, 1]],# 104 [-1, 1, nn.Upsample, [None, 2,'nearest']], # 105 [[37,88], 1, BimodalFusion,

[1024] ],# 106 [-1, 1, Conv, [256, 1, 1]], # route backbone P4 107 [[-1, -3], 1, Concat, [1]], # 108 [-1, 1, Conv, [256, 1, 1]], [-2, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [256, 1, 1]], ## ELAN' module 116 [-1, 1, Conv, [128, 1, 1]], ​[-1, 1, nn.Upsample, [None, 2, 'nearest']], [[24,75], 1, BimodalFusion,

[512] ], [-1, 1, Conv, [128, 1, 1]], # route backbone P3 [[-1, -3], 1, Concat, [1]], # 121 [-1, 1, Conv, [128, 1, 1]], [-2, 1, Conv, [128, 1, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [-1, 1, Conv, [64, 3, 1]], [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [128, 1, 1]], # 129 ELAN' module [-1, 1, MP, []], [-1, 1, Conv, [128, 1, 1]], [-3, 1, Conv, [128, 1, 1]], [-1, 1, Conv, [128, 3, 2]], [[-1, -3, 116], 1, Concat, [1]], # 134 MP-2 module [-1, 1, Conv, [256, 1, 1]], [-2, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]], [-1, 1, Conv, [128, 3, 1]],​​​[[-1, -2, -3, -4, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [256, 1, 1]], # 142 ELAN' module [-1, 1, MP, []], [-1, 1, Conv, [256, 1, 1]], [-3, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 2]], [[-1, -3, 103], 1, Concat, [1]], # MP-2 module 147 [-1, 1, Conv, [512, 1, 1]], [-2, 1, Conv, [512, 1, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [-1, 1, Conv, [256, 3, 1]], [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]], [-1, 1, Conv, [512, 1, 1]], # 155 ELAN' module [129, 1, RepConv, [256, 3, 1]], # 156 [142, 1, RepConv, [512, 3, 1]], # 157 [155, 1, RepConv, [1024, 3, 1]], # 158 [[156,157,158], 1, IDetect, [nc, anchors]],# Detect(P3, P4, P5) ​

Claims

1. An image ship detection method based on cross-modal wavelet attention, characterized in that, include: Step 1: Obtain the visible light-infrared dataset; Step 2: Extract visible light feature maps and infrared feature maps; Step 3: Calculate the bidirectional fusion features and splice them to form a low-frequency fusion component; Step 4: Calculate cross-modal attention weights and perform weighted fusion to form high-frequency fusion components; Step 5: Fuse the low-frequency fusion component and the high-frequency fusion component to obtain dual-modal features; Step 6: Fuse the dual-modal features to obtain multi-scale dual-modal features; Step 7: After processing the multi-scale dual-modal features, the ship inspection results are obtained.

2. The image ship detection method based on cross-modal wavelet attention according to claim 1, characterized in that, Step 1 includes: Step 1.1: Establish a visible light remote sensing image dataset and a long-wave infrared dataset for ships; Step 1.2: Using the visible light remote sensing image dataset as the content image and the long-wave infrared dataset as the style image, generate a pseudo-infrared dataset with a long-wave infrared style. Step 1.3: Pair the visible light remote sensing image dataset and the pseudo-infrared dataset to form a visible light-infrared dataset; Step 2 includes inputting the paired visible-infrared dataset into the dual-branch feature extraction module to extract feature maps at multiple different scales; the feature maps include visible light feature maps and infrared feature maps.

3. The image ship detection method based on cross-modal wavelet attention according to claim 1, characterized in that, Step 3 includes: Step 3.1: For visible light feature maps and infrared feature maps at the same scale, generate the corresponding queries, keys, and values ​​respectively; Step 3.2: Convert the format using tensor rearrangement operations; The initial shape is transformed into the format required by the multi-head attention mechanism using tensor rearrangement operations; Step 3.3: Calculate the fusion features after querying the infrared mode from the visible light mode and the fusion features after querying the visible light mode from the infrared mode, and then splice the two fusion features to form a low-frequency fusion component; Step 3.3.1: Calculate the fusion features after querying the infrared mode from the visible light mode; Step 3.3.2: Calculate the fusion features after querying the visible light mode from the infrared mode; Step 3.3.3: Combine the two fusion features to form a low-frequency fusion component.

4. The image ship detection method based on cross-modal wavelet attention according to claim 1, characterized in that, Step 4 includes: Step 4.1: For the visible light feature map and infrared feature map at the same scale, separate the components in the horizontal, vertical and diagonal directions respectively; Step 4.2: Calculate the cross-modal attention weight for each component, then introduce a learnable bias weight parameter, and perform weighted fusion on the components in the corresponding direction based on the cross-modal attention weight and bias weight parameter. Stack the multimodal adaptive fusion features obtained in the three directions to form a high-frequency fusion component. Step 5 includes: Step 5.1: After concatenating the low-frequency fusion component and the high-frequency fusion component along the channel dimension, input them into the cross-band interaction module to generate cross-band attention weights; Step 5.1.1: The low-frequency guided high-frequency enhancement module first bilinearly upsamples the low-frequency fusion component to the spatial size of the current high-frequency sub-band, and then splices it with the high-frequency fusion component along the three channel dimensions respectively. Step 5.1.2: Input the spliced ​​low-frequency fusion component and high-frequency fusion component along the channel dimension into the cross-band interaction module to generate cross-band attention weights; Step 5.2: Based on cross-band attention weights, fuse the low-frequency fusion component and the high-frequency fusion component to obtain dual-modal features.

5. The image ship detection method based on cross-modal wavelet attention according to claim 1, characterized in that, Step 6 includes: Step 6.1: Input the dual-modal features at different scales into the multi-scale feature fusion module; Step 6.2: Starting from the deepest features, perform upsampling operations on them to expand the spatial size of the feature map and make it consistent with the scale of the middle-layer features; Step 6.3: Concatenate the upsampled deepest features and mid-level features along the channel dimension, and complete feature extraction and channel adjustment through convolution operation to obtain the mesoscale fused features; Step 6.4: Continue to upsample the mesoscale fused features to make them consistent with the scale of the shallow features; Step 6.5: Concatenate the upsampled mesoscale fusion features with the shallow features along the channel dimension, and obtain the shallow scale fusion features through convolution operation; Step 6.6: Start with shallow-scale fusion features and perform downsampling to reduce the feature map spatial size and increase the number of channels; Step 6.7: Concatenate the downsampled shallow-scale fusion features with the corresponding mesoscale fusion features along the channel dimension, and obtain new mesoscale fusion features through convolution operation; Step 6.8: Continue to downsample the new mesoscale fusion feature and concatenate and convolve it with the deep-scale fusion feature to obtain the new deep-scale fusion feature; Step 6.9: Output the dual-modal fusion feature maps at three scales: shallow, medium, and deep.

6. The image ship detection method based on cross-modal wavelet attention according to claim 1, characterized in that, Step 7 includes: Step 7.1: Perform feature enhancement and channel adjustment on the multi-scale bimodal features at each scale; Step 7.2: For each scale feature, generate a series of anchor frames according to the preset anchor frame size and proportion, match the anchor frames with the real target, and determine the positive and negative samples. Step 7.3: For each matching positive sample anchor box, the detection head predicts the probability of its class and the offset of the bounding box; Step 7.4: Normalize the category predictions to obtain the confidence score for each category; Bounding box regression predicts the center point offset, width offset, and height offset of the anchor box relative to the true target box; Step 7.5: The predicted bounding boxes and class confidence scores will undergo post-processing to remove redundant bounding boxes and retain the most accurate ship detection results.

7. An image-based ship detection system based on cross-modal wavelet attention, characterized in that, The image ship detection method based on cross-modal wavelet attention as described in any one of claims 1-6 includes a dual-branch feature extraction module, a cross-modal wavelet attention fusion module, a multi-scale feature fusion module, and a ship detection head module connected in sequence. The dual-branch feature extraction module includes a visible light feature extraction branch and an infrared feature extraction branch. The visible light feature extraction branch is connected to the visible light image input module, the infrared feature extraction branch is connected to the infrared image input module, and the ship detection head module is connected to the result output module.

8. The image ship detection system based on cross-modal wavelet attention according to claim 7, characterized in that, The cross-modal wavelet attention fusion module includes a discrete wavelet transform unit, which is connected to a visible light convolution preprocessing unit, an infrared convolution preprocessing unit, a bidirectional low-frequency attention fusion unit, and a direction-specific multimodal attention fusion unit, respectively. The bidirectional low-frequency attention fusion unit and the direction-specific multimodal attention fusion unit are connected to a low-frequency guided high-frequency enhancement unit, which is connected to an inverse wavelet transform unit. The inverse wavelet transform and residual output unit are connected to the bidirectional low-frequency attention fusion unit, the visible light convolution preprocessing unit, and the infrared convolution preprocessing unit.

9. The image ship detection system based on cross-modal wavelet attention according to claim 8, characterized in that, The bidirectional low-frequency attention fusion unit includes a multi-head tensor rearrangement unit, which is connected to the bidirectional attention calculation unit, query convolution unit, key convolution unit, and value convolution unit respectively. The bidirectional attention calculation unit is connected to a Dropout unit, the Dropout unit is connected to a channel splicing unit, and the channel splicing unit is connected to an output projection convolution unit. The direction-specific multimodal attention fusion unit includes a learnable bias weight unit, a high-frequency direction fusion unit, and a direction dimension stacked output unit connected in sequence. The high-frequency direction fusion unit is connected to the horizontal direction attention branch, the vertical direction attention branch, and the diagonal direction attention branch, respectively. The horizontal direction attention branch, the vertical direction attention branch, and the diagonal direction attention branch are connected to the infrared SE channel attention unit and the visible light SE channel attention unit, respectively. The infrared SE channel attention unit is connected to the infrared direction feature flattening unit, and the visible light SE channel attention unit is connected to the visible light direction feature flattening unit. The low-frequency guided high-frequency enhancement unit includes a low-frequency upsampling unit, which is connected to a cross-band splicing unit and a low-frequency guided convolution unit. The low-frequency guided convolution unit is connected to a high-frequency enhancement fusion unit, the cross-band splicing unit is connected to a cross-band interaction unit, the cross-band interaction unit is connected to a high-frequency enhancement fusion unit, and the high-frequency enhancement fusion unit is connected to a high-frequency sub-band stacking output unit. The inverse wavelet transform and residual output unit includes an inverse wavelet transform unit, a size alignment unit, a residual connection unit, an activation function unit, and an output convolution unit connected in sequence.

10. The image ship detection system based on cross-modal wavelet attention according to claim 7, characterized in that, The visible light image input module receives visible light images and outputs visible light input image information; The infrared image input module receives infrared images and outputs infrared input image information. Input the visible light input image information into the visible light feature extraction branch to obtain multi-scale visible light features; The infrared input image information is input into the infrared feature extraction branch to obtain multi-scale infrared features; The dual-branch feature extraction module outputs visible light and infrared features at three scales respectively. Visible light and infrared features at the same scale are then input into the cross-modal wavelet attention fusion module. In the cross-modal wavelet attention fusion module: first, convolutional preprocessing is performed separately to obtain visible light enhancement features and infrared enhancement features; then, these features are input into a discrete wavelet transform unit to obtain visible light low-frequency subbands, visible light high-frequency subbands, infrared low-frequency subbands, and infrared high-frequency subbands; finally, the visible light low-frequency subbands and infrared low-frequency subbands are input into a bidirectional low-frequency attention fusion unit to obtain low-frequency fused features. The visible light high-frequency subband and the infrared high-frequency subband are input into a direction-specific multimodal attention fusion unit to obtain high-frequency fusion features. The low-frequency fusion features and the high-frequency fusion features are input into a low-frequency guided high-frequency enhancement unit, which uses low-frequency global semantic information to guide the selection and enhancement of high-frequency edge texture information, resulting in enhanced high-frequency fusion features. The low-frequency fusion features and the enhanced high-frequency fusion features are input into an inverse wavelet transform unit to reconstruct spatial domain fusion features. The spatial domain fusion features are then residually fused with the convolutionally preprocessed visible light enhancement features and infrared enhancement features to obtain dual-modal fusion features at the corresponding scale. The three-scale dual-modal fusion features are input into the multi-scale feature fusion module to obtain multi-scale enhanced fusion features; The multi-scale enhanced fusion features are input into the ship detection head module, which outputs the ship target's category prediction results, position prediction results, and confidence prediction results; the result output module generates the final ship detection result based on the detection head output results.

Images