Ship wake detection method and system based on multi-dimensional feature collaborative fusion

Abstract

The application discloses a ship wake detection method and system based on a feature balance fusion technology, and belongs to the technical field of ocean detection. In the feature fusion part in the deep neural network, multi-dimensional feature cooperative fusion is adopted for feature fusion. Specifically, the first feature map output by the multi-dimensional feature embedded Gaussian non-local attention part is fused with the second feature output by the multi-dimensional cooperative attention, to obtain a final feature map, and feature positioning and classification are performed. The application utilizes multi-dimensional feature cooperative fusion to enable the network to simultaneously fuse low-dimensional features and high-dimensional features, which is helpful to solve the feature layer inconsistency problem existing in the single-stage target detection feature pyramid in the prior art, and better feature fusion effect is obtained.

Description

Technical Field

[0001] This invention relates to the technical field of marine detection, specifically to a method and system for detecting ship wakes based on multi-dimensional feature collaborative fusion. Background Technology

[0002] Over the past few decades, with the continuous development and advancement of remote sensing technology, the resolution and coverage of remote sensing image data acquired by satellite, aerial, and other remote sensing platforms have been continuously improving, making remote sensing data one of the main means of acquiring information about the Earth's surface. Remote sensing images refer to films or photographs that record the electromagnetic wave magnitudes of various ground features. The main advantages of optical remote sensing images are as follows:

[0003] (1) Obtaining large-scale geographic information: including land, ocean and atmosphere. It can provide global data to help us better understand the changes and characteristics of the Earth's surface.

[0004] (2) Provide high-resolution spatial information: Through remote sensing technology, we can obtain high-resolution images so as to observe and analyze surface features in more detail, such as land use, vegetation cover, buildings, etc.

[0005] (3) Providing multispectral information: Remote sensing sensors can capture electromagnetic radiation in different bands, from visible light to infrared light. This information in different bands can be used for different applications, such as agriculture, environmental monitoring and resource management.

[0006] (4) Provide time series information: By acquiring remote sensing images at multiple time points, we can observe and analyze changes on the Earth's surface, such as glacier retreat, urban expansion, and natural disasters.

[0007] (5) Unmanned data collection: Remote sensing technology can collect data through platforms such as satellites, airplanes or drones, without requiring personnel to enter dangerous or hard-to-reach areas, thus providing a safer and more efficient way to acquire data.

[0008] In summary, remote sensing technology has wide applications in scientific research, natural resource management, environmental protection, and urban planning. Its advantages include broad data coverage, rich information, low cost, diversity, and ease of processing. This makes remote sensing an indispensable tool in modern science and engineering, providing us with the ability to gain a deeper understanding of the Earth and make intelligent decisions.

[0009] Traditional ship wake detection algorithms typically require manual feature extraction, which is very time-consuming and labor-intensive. Furthermore, these methods suffer from poor robustness and generalization ability, making it impossible for the system to automatically identify targets. However, with the advancement and improvement of deep learning technology, it has been gradually applied to the field of remote sensing image target detection and recognition, achieving very good results.

[0010] The biggest advantage of deep learning is its ability to adaptively extract useful features, which greatly reduces or eliminates the need for human feature extraction. At the same time, deep neural networks can effectively learn from massive amounts of data, thereby improving the accuracy and robustness of the algorithm.

[0011] In recent years, with the development of deep learning technology, deep convolutional neural networks (CNNs) have played an increasingly important role in object detection, and many CNN models have been successfully applied to ship wake detection. CNN-based methods have achieved excellent results in ship wake detection, such as YOLO and Faster R-CNN. Therefore, the application of deep learning technology will further promote the progress of ship wake detection and lay the foundation for broader applications such as ship identification and tracking. However, most existing deep learning methods employ a single-stage object detection feature pyramid, leading to inconsistent and unbalanced feature layers, increasing the actual computational load and detection efficiency. Summary of the Invention

[0012] To address the aforementioned technical problems, this invention proposes a method and system for detecting ship wakes based on multi-dimensional feature collaborative fusion.

[0013] To achieve the above objectives, a first aspect of the present invention discloses a method for detecting ship wakes based on multi-dimensional feature collaborative fusion, the method comprising:

[0014] Step S1: Input the remote sensing image to be detected, perform image preprocessing, and form a preprocessed image set;

[0015] Step S2: Extract features from each remote sensing image in the preprocessed image set using the feature extraction part of a deep convolutional neural network, and map the image data to the feature space to obtain a feature map;

[0016] Step S3: The feature map is balanced and fused by the multi-dimensional feature collaborative fusion part of the deep convolutional neural network. The multi-dimensional feature collaborative fusion part balances and fuses the feature map by inputting the feature map into the multi-dimensional feature embedded Gaussian non-local attention part and the multi-dimensional collaborative attention part to obtain the first feature map and the second feature map respectively, and then performing an addition operation on the first feature map and the second feature map to obtain the final feature map.

[0017] Step S4: The detection and regression part of the deep convolutional neural network obtains the feature localization and classification results of the remote sensing image data based on the final feature map.

[0018] Furthermore, in step S3, inputting the feature map into a multi-dimensional feature embedded Gaussian nonlocal attention part to obtain the first feature map includes:

[0019] Step S311: Perform feature scaling on the feature map to obtain multiple feature maps of the same size, wherein the size of the multiple feature maps of the same size is the same as the size of the middle layer of the feature map;

[0020] Step S312: Perform semantic feature balancing on multiple feature maps of the same size, and calculate the average of the multiple feature maps of the same size to obtain a single feature map. , will the The characteristics of a layer are denoted as The number of multi-layer features is denoted as , The minimum and maximum values ​​are denoted as and Feature map The calculation formula is:

[0021] ;

[0022] Step S313: The feature map is processed by embedded Gaussian nonlocal attention fusion and feature scaling to obtain the first feature map output by the multidimensional feature embedded Gaussian nonlocal attention part.

[0023] Furthermore, the embedded Gaussian nonlocal attention processing formula in step S313 includes:

[0024]

[0025] in It is the input feature map. It is the output feature map; It is the index of the output position. It is an index of all possible positions; It is the correlation coefficient between two points. The calculation is the position. The representation of the input feature map; the final Through impact factor The result obtained after standardization.

[0026] in The formula is:

[0027]

[0028] in In practice, it is implemented using a 1×1 convolution. In practice, it is implemented using a 1×1 convolution.

[0029] in It is a weight matrix that needs to be learned, and in practice it is implemented using a 1×1 convolution.

[0030] This represents the normalization operation, which is implemented using the Sigmoid function in embedded Gaussian implementations. The formula is:

[0031] .

[0032] Furthermore, in step S3, inputting the feature map into the multidimensional collaborative attention part to obtain the second feature map includes: performing global max pooling, global standard deviation pooling, and depthwise separable convolution on the feature map to obtain the second feature map output by the multidimensional collaborative attention part.

[0033] Furthermore, step S3, which inputs the feature map into the multidimensional collaborative attention part to obtain the second feature map, further includes: using global average pooling and global standard deviation pooling to process the input feature map respectively. Perform feature aggregation to obtain average pooled features along the channel dimension. and standard deviation pooling features The two pooling operations for the m-th channel can be described as follows:

[0034]

[0035]

[0036] in This represents the feature of the m-th channel in the input feature map. and It performs feature aggregation on the features of the m-th channel in the input feature map F to obtain the average pooling feature and standard deviation pooling feature in the channel dimension;

[0037] Then, an adaptive combination mechanism is used to fuse the two pooling features to obtain feature maps after global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations. :

[0038]

[0039] in, and These are two learnable coefficients used to adaptively adjust the relative importance of average pooling and standard deviation pooling features.

[0040] The multidimensional collaborative attention part in step S3 includes three branches. The first two branches are responsible for capturing the interdependencies of features in spatial dimensions W and H, respectively, while the last branch is used to capture the interactions between channels. In the first two branches, a permutation operation is used to capture the long-range dependencies between the channel dimension and any spatial dimension. Finally, in the integration stage, the outputs of all three branches are aggregated by simple averaging.

[0041] Deep convolutional neural networks with R 3 The Det network is the basic model, in which the feature extraction network is a ResNet50 structure.

[0042] In a second aspect, the present invention provides a ship wake detection system based on multi-dimensional feature collaborative fusion, characterized in that it includes a preprocessing module, a deep learning neural network based on multi-dimensional feature collaborative fusion, and a control output module.

[0043] The preprocessing module is used to perform general data augmentation or standardization on the input remote sensing images to be detected, forming a preprocessed image set.

[0044] The deep learning neural network based on multidimensional feature collaborative fusion includes a feature extraction part, a multidimensional feature collaborative fusion part, and a detection and regression part; the feature extraction part is used to extract feature maps of each image data in the preprocessed image set, the multidimensional feature collaborative fusion part is used to fuse the feature maps, and the detection and regression part is used to output feature localization and classification results based on the feature maps;

[0045] The control output module is used to output the feature localization and classification results of the deep learning neural network based on multi-dimensional feature collaborative fusion to the user.

[0046] Furthermore, the multi-dimensional feature collaborative fusion part includes a multi-dimensional feature embedded Gaussian non-local attention part and a multi-dimensional collaborative attention part. The multi-dimensional feature embedded Gaussian non-local attention part is used to perform feature scaling, semantic feature balancing, embedded Gaussian non-local attention fusion, and feature scaling processing on the input features to obtain a first feature map. The multi-dimensional collaborative attention part is used to perform global max pooling, global standard deviation pooling, and depthwise separable convolution processing on the feature map to obtain a second feature map output by the multi-dimensional feature collaborative attention part.

[0047] In a third aspect, the present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method described in any one of the first aspects.

[0048] In a fourth aspect, the present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method described in any one of the first aspects.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] (1) This invention proposes to apply multi-dimensional feature collaboration fusion to the field of remote sensing target detection, and solve the problem of inconsistent feature layers in the single-stage target detection feature pyramid by multi-dimensional feature collaboration fusion. Multi-dimensional feature collaboration fusion allows the network to fuse features from low dimension to high dimension at the same time.

[0051] (2) The multi-dimensional feature collaborative fusion proposed in this invention has a small increase in computation and a small impact on detection efficiency in practical applications.

[0052] (3) This invention optimizes the problem of feature layer imbalance and the problem of the model's potential not being fully utilized during the training process. Attached Figure Description

[0053] Figure 1 This is a flowchart of multi-dimensional feature collaborative fusion based on an attention mechanism;

[0054] Figure 2 A rendering of the detection effect of a real ship wake image;

[0055] Figure 3 A rendering of the detection effect of a real ship wake image;

[0056] Figure 4 This is a rendering of the detection effect of a real ship wake image. Detailed Implementation

[0057] To enable those skilled in the art to better understand the solutions of this invention, the technical solutions of the embodiments of this application will be further described clearly and completely below with reference to the accompanying drawings. It should be noted that the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] To make the inventive objectives, technical solutions, and advantages of this application clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of this invention, and these all fall within the protection scope of this application.

[0059] In existing object detection methods, while high-level features contain rich semantic information, their low resolution makes it difficult to accurately preserve object location information. Conversely, while low-level features contain less semantic information, their high resolution allows them to accurately capture object location information. This invention enhances the original features by fusing semantic features through multi-dimensional feature collaboration. In this way, each resolution in the feature pyramid receives equal information from other resolutions, thus balancing the information, making the features more discriminative, and achieving better trail detection results.

[0060] To address the aforementioned technical problems, this invention provides a method and system for detecting ship wakes based on multi-dimensional feature collaborative fusion. The overall network structure of this invention employs a deep convolutional neural network, including a feature extraction part, a multi-dimensional feature collaborative fusion part, and a detection and regression part. The feature extraction part extracts features from the preprocessed image, downsamples the image data through convolution, extracts features at different scales, and maps the image data to a feature space to obtain a feature map. The multi-dimensional feature collaborative fusion part performs feature fusion processing on the feature map, adjusting, integrating, and fusing the feature map size, adding an attention mechanism, and outputting the final fused feature map. The detection and regression part obtains the feature localization and classification results of the remote sensing image data based on the final feature map.

[0061] like Figure 1 As shown, in Embodiment 1 of the present invention, the present invention proposes a ship wake detection method based on multi-dimensional feature collaborative fusion, comprising the following steps:

[0062] Step S1: Input the remote sensing image to be detected, perform image preprocessing, and form a preprocessed image set;

[0063] Step S2: Extract features from each remote sensing image in the preprocessed image set using the feature extraction part of a deep convolutional neural network, and map the image data to the feature space to obtain a feature map;

[0064] Step S3: The feature map is balanced and fused by the multi-dimensional feature collaborative fusion part of the deep convolutional neural network. The multi-dimensional feature collaborative fusion part balances and fuses the feature map by inputting the feature map into the multi-dimensional feature embedded Gaussian non-local attention part and the multi-dimensional collaborative attention part to obtain the first feature map and the second feature map respectively, and then performing an addition operation on the first feature map and the second feature map to obtain the final feature map.

[0065] Step S4: The detection and regression part of the deep convolutional neural network obtains the feature localization and classification results of the remote sensing image data based on the final feature map.

[0066] In Embodiment 1 of this invention, the remote sensing image to be detected can be a remote sensing image acquired by a UAV or a remote sensing satellite. Optical remote sensing images, as a technology that uses electromagnetic waves (light) to acquire information about the Earth's surface, have advantages such as high resolution, rich spectral data, efficient and rapid acquisition, and wide scanning coverage. These advantages enable optical remote sensing images to be widely used in environmental protection, resource management, land use, urban planning, disaster management, and other fields. In this embodiment, optical remote sensing images are selected to detect changes in the coastal zone. By acquiring optical remote sensing images of the area to be detected at different times and performing image preprocessing, a preprocessed image dataset is obtained.

[0067] It should be noted that after acquiring the remote sensing image to be detected, basic image preprocessing must be performed first. Image preprocessing can help improve the accuracy and reliability of feature detection. The image preprocessing in step S1 includes one or more of radiometric correction, geometric correction, and image enhancement. The optical remote sensing image after image preprocessing has higher geometric and pixel accuracy.

[0068] In this embodiment, the feature extraction part of the deep convolutional neural network is used to extract features from the preprocessed remote sensing image. The purpose of feature extraction is to improve the key feature information in the remote sensing image, eliminate redundant information unrelated to feature detection, and improve the accuracy and efficiency of detection calculations. Existing feature extraction methods include raw feature extraction and neural network feature extraction. This embodiment selects the neural network feature extraction method, using a ResNet50 structure. Its main function is to downsample the image data through convolution and extract features at different scales. In this embodiment, multiple feature maps include four feature maps of different sizes, C2-C5.

[0069] In this embodiment, the multi-dimensional feature collaborative fusion part includes a multi-dimensional feature embedded Gaussian non-local attention part and a multi-dimensional collaborative attention part. After feature extraction is completed, the extracted feature maps are input into the multi-dimensional feature embedded Gaussian non-local attention part and the multi-dimensional collaborative attention part of the multi-dimensional feature collaborative fusion part for further feature processing.

[0070] The feature map is processed in the multi-dimensional feature embedded Gaussian non-local attention part, including feature scaling, semantic feature balancing, embedded Gaussian non-local attention fusion, and feature scaling to obtain feature maps of different scales. Step S3, inputting the feature map into the multi-dimensional feature embedded Gaussian non-local attention part to obtain the first feature map, includes:

[0071] Step S311: Perform feature scaling on the feature map to create multiple feature maps of the same size, wherein the size of the multiple feature maps of the same size is the same as the size of the middle layer of the feature map;

[0072] Step S312 performs semantic feature balancing on multiple feature maps of the same size, and calculates the average of the multiple feature maps of the same size to obtain a single feature map. , will the The characteristics of a layer are denoted as The number of multi-layer features is denoted as , The minimum and maximum values ​​are denoted as and Feature map The calculation formula is:

[0073] ;

[0074] Step S313 processes the feature map through embedded Gaussian nonlocal attention fusion and feature scaling to obtain the first feature map output by the multidimensional feature embedded Gaussian nonlocal attention part.

[0075] After feature extraction, the multidimensional feature embedded Gaussian nonlocal attention part performs feature scaling on the feature maps, adjusting their size to a fixed size. The fixed size is the median size of the multiple feature maps. In this embodiment, the feature extraction part obtains four feature maps of different scales, C2-C5, with the resolution gradually increasing from C2 to C5. When scaling the features, the median-sized feature map C4 is selected as the reference standard for feature map size adjustment, allowing other feature maps to be adjusted to match the size of feature map C4. Specifically, layers C2 and C3 are resized to the size of layer C4 using max pooling, and layer C5 is resized to the size of layer C4 using bilinear interpolation.

[0076] After scaling the features, semantic balancing is performed on the feature maps within the Gaussian nonlocal attention region embedded in the multidimensional features. Specifically, multiple feature maps of the same size are averaged to obtain a single feature map. , will the The characteristics of a layer are denoted as The number of multi-layer features is denoted as , The minimum and maximum values ​​are denoted as and Feature map The calculation formula is:

[0077] .

[0078] Furthermore, the embedded Gaussian nonlocal attention processing formula in step S313 includes:

[0079]

[0080] in It is the input feature map. It is the output feature map; It is the index of the output position. It is an index of all possible positions; It is the correlation coefficient between two points. The calculation is the position. The representation of the input feature map; the final Through impact factor The result obtained after standardization.

[0081] in The formula is:

[0082]

[0083] in In practice, one is used It is implemented using convolution; In practice, one is used It is implemented using convolution;

[0084] in , It is a weight matrix that needs to be learned; in practice, it is used with a... It is implemented using convolution;

[0085] This represents the normalization operation, which is implemented using the Sigmoid function in embedded Gaussian implementations. The formula is:

[0086] .

[0087] The feature maps obtained by embedded Gaussian nonlocal attention processing are scaled to obtain multiple feature maps at different scales. Specifically, the fused feature maps are scaled back to the original sizes of layers C2, C3, C4, and C5. Bilinear interpolation is used to adjust them to the sizes of C2 and C3 respectively, and max pooling is used to adjust the fused feature map C5 to the size of C5 after the multidimensional collaborative attention part, resulting in the first output feature map.

[0088] Step S3, inputting the feature map into the multidimensional collaborative attention part to obtain the second feature map, includes: performing global max pooling, global standard deviation pooling, and depthwise separable convolution on the feature map to obtain the second feature map output by the multidimensional collaborative attention part. Step S3, inputting the feature map into the multidimensional collaborative attention part to obtain the second feature map, further includes: applying global average pooling and global standard deviation pooling to the input feature map respectively. Perform feature aggregation to obtain average pooled features along the channel dimension. and standard deviation pooling features The two pooling operations for the m-th channel can be described as follows:

[0089]

[0090]

[0091] in This represents the feature of the m-th channel in the input feature map. and It performs feature aggregation on the features of the m-th channel in the input feature map F to obtain the average pooling feature and standard deviation pooling feature in the channel dimension;

[0092] Then, an adaptive combination mechanism is used to fuse these two pooling features to obtain feature maps after global average pooling and global standard deviation pooling operations. :

[0093]

[0094] in, and These are two learnable coefficients used to adaptively adjust the relative importance of average pooling and standard deviation pooling features.

[0095] Furthermore, the multidimensional collaborative attention mechanism in step S3 comprises three branches. The first two branches are responsible for capturing the interdependencies of features in spatial dimensions W and H, respectively, while the last branch is mainly used to capture inter-channel interactions. In the first two branches, permutation operations are used to capture the long-range dependencies between the channel dimension and any spatial dimension. Finally, in the integration stage, the outputs of all three branches are aggregated by simple averaging. The multidimensional collaborative attention mechanism consists of three branches. The first two branches are responsible for capturing the interdependencies of features in spatial dimensions W and H, respectively, while the last branch is used to capture inter-channel interactions. In the first two branches, permutation operations are used to capture the long-range dependencies between the channel dimension and any spatial dimension. Finally, in the integration stage, the outputs of all three branches are aggregated by simple averaging. The first two branches are the top and middle branches, and the last branch is the bottom branch. Then, the input feature map is first subjected to independent channel-wise convolution in the spatial dimension, without inter-channel interactions, using... Each independent 2D convolutional kernel has a size of 3×3, resulting in an intermediate output feature map. Then... Perform standard 1x1 convolution operations to establish interactions between channels.

[0096] Specifically, in the top branch, the feature map is input to the MCA module. First, rotate the feature map 90° counterclockwise along the H-axis. The resulting feature map is shown below: To model the long-distance dependency between the channel dimension C and the spatial dimension H, then... Performing global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, the resulting aggregated feature map is described as follows: Then Perform depthwise separable convolution to capture the interactions between features in the spatial dimension W. The resulting feature weights along the width direction are represented as follows: Next, a sigmoid activation function is passed in. Generate input-specific attention weights in W dimensions Then, element-wise multiplication is used to... Applied to The above yields an enhanced feature map. Finally, along Rotate the axis 90 degrees clockwise 。 This yields a feature map with the same shape as the original input. Mathematically, this process can be summarized by the following equation:

[0097]

[0098]

[0099] in This indicates a 90° counterclockwise rotation along the H-axis. This represents the opposite operation, both of which can be easily achieved using the permute function in the PyTorch framework. This represents the sigmoid activation function, where This represents the global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations. This indicates a depthwise separable convolution operation.

[0100] In the intermediate branch, the input feature map F is first rotated 90° counterclockwise along the W-axis to obtain the rotated feature map. To model the interdependence between channel dimension C and spatial dimension H, and further capture the interaction between heights, Global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, along with depthwise separable convolution operations, are performed to sequentially derive the aggregated feature map. and height direction feature weights Then it is passed through the sigmoid activation function. It is possible to generate input-specific attention weights in the H dimension. Then, by Recalibrate Generate an enhanced feature map .final Rotate 90° clockwise along the W-axis to obtain a feature map with the same shape as the original input. In essence, this process can be summarized as follows:

[0101]

[0102]

[0103] in This indicates a 90° counterclockwise rotation along the W-axis. This represents the opposite operation. Both can be easily achieved using the `permute` function in the PyTorch framework. This represents the sigmoid activation function, where This represents the global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations. This indicates a depthwise separable convolution operation.

[0104] The lower-level branches are primarily responsible for modeling the interdependencies between spaces (H and W) and capturing the interactions between channels. In short, it first generates feature maps that share the same identity. Then The aggregated feature maps can be inferred sequentially by feeding the data into global average pooling (AvgPool), global standard deviation pooling (StdPool), and depthwise separable convolution. and channel feature weights .after By using the sigmoid activation function, input-specific channel attention weights are derived. Subsequently, pass The scaled-up version generates an enhanced feature map. Finally, the identity mapping function is used to... Remapping Similarly, this process can be summarized as follows:

[0105]

[0106]

[0107] in This represents the sigmoid activation function, where This represents the global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations. This indicates a depthwise separable convolution operation.

[0108] The second feature map is obtained after processing by the multi-dimensional collaborative attention part. This second feature map includes feature maps of multiple scales. Finally, the first and second feature maps are added together to obtain the final feature map. Specifically, C2-C5 in the first feature map is added to C2-C5 in the second feature map at corresponding sizes to obtain the final feature map.

[0109] In this embodiment, the deep convolutional neural network uses R... 3 The Det network is the basic model. The multi-dimensional feature collaborative fusion component within this neural network is primarily a feature pyramid-like structure. Typical feature fusion tends to focus more on adjacent feature layers and less on other feature layers. In the information flow, each fusion dilutes the semantic information contained in non-adjacent layers. Multi-dimensional feature collaborative fusion addresses the feature layer inconsistency problem inherent in single-stage object detection feature pyramids, allowing the network to simultaneously fuse features from low to high dimensions.

[0110] In a second embodiment of the present invention, the present invention provides a ship wake detection system based on multi-dimensional feature collaborative fusion, which is characterized by including a preprocessing module, a deep learning neural network based on multi-dimensional feature collaborative fusion, and a control output module.

[0111] The preprocessing module is used to perform general data augmentation or standardization on the input remote sensing images to be detected, forming a preprocessed image set.

[0112] The deep learning neural network based on multidimensional feature collaborative fusion includes a feature extraction part, a multidimensional feature collaborative fusion part, and a detection and regression part; the feature extraction part is used to extract feature maps of each image data in the preprocessed image set, the multidimensional feature collaborative fusion part is used to fuse the feature maps, and the detection and regression part is used to output feature localization and classification results based on the feature maps;

[0113] The control output module is used to output the feature localization and classification results of the deep learning neural network based on multi-dimensional feature collaborative fusion to the user.

[0114] Furthermore, the multi-dimensional feature collaborative fusion part includes a multi-dimensional feature embedded Gaussian non-local attention part and a multi-dimensional collaborative attention part. The multi-dimensional feature embedded Gaussian non-local attention part is used to perform feature scaling, semantic feature balancing, embedded Gaussian non-local attention fusion, and feature scaling processing on the input features to obtain a first feature map. The multi-dimensional collaborative attention part is used to perform global max pooling, global standard deviation pooling, and depthwise separable convolution processing on the feature map to obtain a second feature map output by the multi-dimensional feature collaborative attention part.

[0115] In a third embodiment of the present invention, the present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method described in any one of the first aspects.

[0116] In a fourth embodiment of the present invention, the present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method described in any one of the first aspects.

[0117] Figure 2 A ship's wake was identified, with a detection probability of 83%. Figure 3 Two ship wakes were identified, with probabilities of being ship wakes of 78% and 77%, respectively. Figure 4 Three ship wakes were identified, with probabilities of being ship wakes of 75%, 73%, and 75%, respectively. The basic model R in this embodiment... 3The average detection result of Det on all images in the SWIM dataset is 75.2%, while the average detection result of our method on all images in the SWIM dataset is 77.2%.

[0118] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0119] Finally, it should be noted that the above specific embodiments are only used to illustrate the technical solution of this patent and not to limit it. Although this patent has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solution of this patent without departing from the spirit and scope of the technical solution of this patent, and all such modifications and substitutions should be covered within the scope of the claims of this patent.

Claims

1. A method for detecting ship wakes based on multi-dimensional feature collaborative fusion, characterized in that: The method includes: Step S1: Input the remote sensing image to be detected, perform image preprocessing, and form a preprocessed image set; Step S2: Extract features from each remote sensing image in the preprocessed image set using the feature extraction part of a deep convolutional neural network, and map the image data to the feature space to obtain a feature map; Step S3: The feature map is balanced and fused using the multi-dimensional feature collaborative fusion part of the deep convolutional neural network. This fusion includes inputting the feature map into a multi-dimensional feature embedded Gaussian non-local attention part and a multi-dimensional collaborative attention part to obtain a first feature map and a second feature map, respectively. The first and second feature maps are then added together to obtain the final feature map. Step S3, in which the feature map is input into the multi-dimensional feature embedded Gaussian non-local attention part to obtain the first feature map, includes: Step S311: Perform feature scaling on the feature map to obtain multiple feature maps of the same size, wherein the size of the multiple feature maps of the same size is the same as the size of the middle layer of the feature map; Step S312: Perform semantic feature balancing on multiple feature maps of the same size, and calculate the average of the multiple feature maps of the same size to obtain a single feature map C. The characteristics of a layer are denoted as The number of multi-layer features is denoted as , The minimum and maximum values ​​are denoted as and The formula for calculating feature map C is: ; Step S313: The feature map is processed by embedded Gaussian nonlocal attention and feature scaling to obtain the first feature map output by the multidimensional feature embedded Gaussian nonlocal attention part; the step S3 of inputting the feature map into the multidimensional collaborative attention part to obtain the second feature map includes: The feature map is processed by global max pooling, global standard deviation pooling and depthwise separable convolution to obtain the second feature map output by the multi-dimensional feature collaborative attention part. Step S4: The detection and regression part of the deep convolutional neural network obtains the feature localization and classification results of the remote sensing image data based on the final feature map.

2. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1, characterized in that, The embedded Gaussian nonlocal attention processing formula in step S313 includes: ; in It is the input position Feature map, It is the input position Feature map, Output position Feature map, Output position Feature map; It is the correlation coefficient between two points. The calculation is the position. The representation of the input feature map; the final Through impact factor The result obtained after standardization. in The formula is: ； in In practice, it is implemented using a 1×1 convolution. This is the weight matrix; , In practice, it is implemented using a 1×1 convolution. This is the weight matrix; in , It is a weight matrix that needs to be learned. In practice, it is implemented using a 1×1 convolution; This represents the normalization operation, which is implemented using the Sigmoid function in embedded Gaussian implementations. The formula is: 。 3. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 2, characterized in that, Step S3, in which the feature map is input into the multidimensional collaborative attention part to obtain the second feature map, further includes: applying global average pooling and global standard deviation pooling to the input feature map respectively. Perform feature aggregation to obtain average pooled features along the channel dimension. and standard deviation pooling features The two pooling operations for the m-th channel are described below: ; ； in These represent the characteristics of channels 1, 2...C respectively. These represent the average pooling characteristics of channels 1, 2, ..., C, respectively. These represent the standard deviation pooling characteristics of channels 1, 2, ..., C, respectively. This represents the feature of the m-th channel in the input feature map. and It performs feature aggregation on the features of the m-th channel in the input feature map F, and obtains the average pooling feature and standard deviation pooling feature in the channel dimension respectively; Then, an adaptive combination mechanism is used to fuse these two pooling features to obtain feature maps after global average pooling and global standard deviation pooling operations. : ； in, and These are two learnable coefficients used to adaptively adjust the relative importance of average pooling and standard deviation pooling features.

4. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1, characterized in that, The multidimensional collaborative attention part in step S3 includes three branches. The first two branches are responsible for capturing the interdependencies of features on spatial dimensions W and H, respectively, while the last branch is used to capture the interactions between channels. In the first two branches, a permutation operation is used to capture the long-range dependencies between the channel dimension and any spatial dimension. Finally, in the integration stage, the outputs of all three branches are aggregated by simple averaging.

5. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1, characterized in that, The deep convolutional neural network uses R 3 The Det network is the basic model, in which the feature extraction network is a ResNet50 structure.

6. A ship wake detection system based on multi-dimensional feature collaborative fusion, characterized in that, It includes a preprocessing module, a deep learning neural network based on multi-dimensional feature collaborative fusion, and a control output module; The preprocessing module is used to perform general data augmentation or standardization on the input remote sensing images to be detected, forming a preprocessed image set. The deep learning neural network based on multidimensional feature collaborative fusion includes a feature extraction part, a multidimensional feature collaborative fusion part, and a detection and regression part; the feature extraction part is used to extract feature maps of each image data in the preprocessed image set, the multidimensional feature collaborative fusion part is used to fuse the feature maps, and the detection and regression part is used to output feature localization and classification results based on the feature maps; The multi-dimensional feature collaborative fusion part for fusing feature maps includes: inputting the feature maps into a multi-dimensional feature embedded Gaussian non-local attention part and a multi-dimensional collaborative attention part respectively to obtain a first feature map and a second feature map; and performing an addition operation on the first feature map and the second feature map to obtain the final feature map; the step of inputting the feature map into the multi-dimensional feature embedded Gaussian non-local attention part to obtain the first feature map includes: The feature map is scaled to obtain multiple feature maps of the same size, and the size of the multiple feature maps of the same size is the same as the size of the middle layer of the feature map. Multiple feature maps of the same size undergo semantic feature balancing. The average of these feature maps is then calculated to obtain a single feature map C. The th feature map is then... The characteristics of a layer are denoted as The number of multi-layer features is denoted as , The minimum and maximum values ​​are denoted as and The formula for calculating feature map C is: ; The feature map is processed by embedded Gaussian nonlocal attention and feature scaling to obtain the first feature map output by the multidimensional feature embedded Gaussian nonlocal attention part; the step of inputting the feature map into the multidimensional collaborative attention part to obtain the second feature map includes: performing global max pooling, global standard deviation pooling and depthwise separable convolution on the feature map to obtain the second feature map output by the multidimensional feature collaborative attention part. The control output module is used to output the feature localization and classification results of the deep learning neural network based on multi-dimensional feature collaborative fusion to the user.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images