Ship wake detection method and system based on multi-dimensional feature collaborative fusion
Through the deep convolutional neural network of multi-dimensional feature collaboration and fusion, the problem of inconsistency in feature layers in ship stern detection is solved, the detection efficiency and accuracy are improved, and the ship stern detection effect is achieved.
Patent Information
- Application Number
- CN202411406975.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-10-10
AI Technical Summary
The existing ship stern detection algorithm requires manual extraction of features, which is time-consuming and labor-intensive, and has poor robustness and generalization capabilities. In deep learning methods, the single-stage target detection feature pyramid leads to inconsistent feature layers, increasing the computational volume and low detection efficiency.
A deep convolutional neural network based on multi-dimensional feature collaboration fusion is adopted to balance the feature map through a multi-dimensional feature embedded Gaussian non-layout attention and multi-dimensional cooperative attention mechanism to generate the final feature map for ship stern detection.
It improves the balance and detection efficiency of the feature layer, enhances the discriminant characteristics, improves the accuracy and robustness of ship stern detection, and reduces the calculation amount.
Smart Images

Figure CN118941965A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of ocean detection, and in particular to a ship wake detection method and system based on multi-dimensional feature collaborative fusion. Background Art
[0002] In the past few decades, with the continuous development and progress of remote sensing technology, the resolution and coverage of remote sensing image data collected by satellite, aviation and other remote sensing platforms have been continuously improved, making remote sensing data one of the main means of obtaining information on the earth's surface. Remote sensing images refer to films or photos that record the size of electromagnetic waves of various ground objects. The main advantages of optical remote sensing images are as follows: (1) Obtaining large-scale geographic information: including land, ocean, and atmosphere. It can provide global data to help us better understand the changes and characteristics of the earth's surface.
[0003] (2) Providing high-resolution spatial information: Through remote sensing technology, we can obtain high-resolution images to observe and analyze surface features in more detail, such as land use, vegetation cover, buildings, etc.
[0004] (3) Providing multispectral information: Remote sensing sensors can capture electromagnetic radiation in different bands, from visible light to infrared. Information from these different bands can be used for different applications, such as agriculture, environmental monitoring, and resource management.
[0005] (4) Providing time series information: By acquiring remote sensing images at multiple time points, we can observe and analyze changes in the earth’s surface, such as glacier retreat, urban expansion, and natural disasters.
[0006] (5) Data collection without human intervention: Remote sensing technology can collect data through platforms such as satellites, aircraft or drones, without the need for personnel to enter dangerous or hard-to-reach areas, thus providing a safer and more efficient way to obtain data.
[0007] In general, remote sensing technology has a wide range of applications in scientific research, natural resource management, environmental protection, urban planning and other fields. Its advantages include wide data coverage, rich information, low cost, diversity and easy processing. This makes remote sensing an indispensable tool in modern science and engineering, providing us with the ability to gain a deeper understanding of the earth and make intelligent decisions.
[0008] Traditional ship wake detection algorithms usually require manual feature extraction, which is very time-consuming and labor-intensive. In addition, such methods have poor robustness and generalization capabilities and cannot automatically identify targets. However, with the advancement and improvement of deep learning technology, it has been gradually used in the field of remote sensing image target detection and recognition, and has achieved very good results.
[0009] The biggest advantage of deep learning is that it can extract useful features adaptively, which will greatly reduce or avoid human feature extraction operations. At the same time, deep neural networks can effectively learn from massive samples, thereby improving the accuracy and robustness of the algorithm.
[0010] In recent years, with the development of deep learning technology, deep convolutional neural networks have played an increasingly important role in the field of target detection, and many deep convolutional neural network models have been successfully applied to ship wake detection. Methods based on convolutional neural networks (CNNs) have achieved very good results in ship wake detection, such as YOLO and FasterRCNN. Therefore, the application of deep learning technology will further promote the progress of ship wake detection and lay the foundation for a wider range of applications such as ship identification and tracking. However, most of the deep learning methods in the existing technology use a single-stage target detection feature pyramid, which leads to inconsistent and unbalanced extracted feature layers, increasing the actual amount of calculation and detection efficiency. Summary of the invention
[0011] In order to solve the above technical problems, the present invention proposes a ship wake detection method and system based on multi-dimensional feature collaborative fusion.
[0012] To achieve the above object, the first aspect of the present invention discloses a ship wake detection method based on multi-dimensional feature collaborative fusion, the method comprising: Step S1: input the remote sensing image to be detected, perform image preprocessing, and form a preprocessed image set; Step S2: extract features from each remote sensing image in the preprocessed image set through the feature extraction part of the deep convolutional neural network, map the image data to the feature space, and obtain a feature map; Step S3: performing balanced fusion of features on the feature map through the multidimensional feature collaborative fusion part of the deep convolutional neural network, wherein the balanced fusion of features on the feature map by the multidimensional feature collaborative fusion part includes inputting the feature map into the multidimensional feature embedded Gaussian non-layout attention part and the multidimensional collaborative attention part to obtain a first feature map and a second feature map, and performing an addition operation on the first feature map and the second feature map to obtain a final feature map; Step S4: The detection regression part of the deep convolutional neural network is used to obtain the feature location and classification results of the remote sensing image data according to the final feature map.
[0013] Furthermore, in step S3, inputting the feature map into the multi-dimensional feature embedded Gaussian non-layout attention part to obtain the first feature map includes: Step S311: performing feature scaling processing on the feature map to obtain a plurality of feature maps of a consistent size, wherein the sizes of the plurality of feature maps of the consistent size are consistent with the size of the middle layer of the feature map; Step S312: Perform semantic feature balancing on multiple feature maps of the same size, average multiple feature maps of the same size to obtain a feature map C, and The characteristics of the layer are recorded as , the number of multi-layer features is recorded as , The minimum and maximum values of and , the calculation formula of the feature map C is: ;
[0014] Step S313: The feature map is subjected to embedded Gaussian non-layout attention fusion and feature scaling processing to obtain the first feature map output by the multi-dimensional feature embedded Gaussian non-layout attention part.
[0015] Furthermore, the embedded Gaussian non-local attention processing formula in step S313 includes:
[0016] is the input feature map, is the output feature map; i is the index of the output position, and j is the index of all possible positions; is the correlation coefficient between two points, The calculation is the representation of the input feature map at position j; the final Through the impact factor After standardization, in The formula is:
[0017] in , the actual operation is implemented using a 1×1 convolution; , the actual operation is implemented using a 1×1 convolution; in It is a weight matrix that needs to be learned. The actual operation is implemented by a 1×1 convolution. represents the normalization operation, which is implemented using the Sigmoid function in the embedded Gaussian. The formula is: .
[0018] Furthermore, in step S3, inputting the feature map into the multi-dimensional collaborative attention part to obtain a second feature map includes: performing global maximum pooling, global standard deviation pooling and depth-wise separable convolution processing on the feature map to obtain a second feature map output by the multi-dimensional feature collaborative attention part.
[0019] Furthermore, in step S3, inputting the feature map into the multi-dimensional collaborative attention part to obtain the second feature map further includes: using global average pooling and global standard deviation pooling to respectively input the feature map Perform feature aggregation to obtain the average pooling feature in the channel dimension And standard error pooling features , where the two pooling operations of the mth channel can be expressed as follows:
[0020]
[0021] in represents the feature of the mth channel in the input feature map, and It aggregates the features of the mth channel in the input feature map F to obtain the average pooling features and standard deviation pooling features in the channel dimension; Then, an adaptive combination mechanism is used to fuse the two pooling features to obtain the feature map after global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations. :
[0022] in, and are two learnable coefficients used to adaptively adjust the relative importance of average pooling and standard deviation pooling features.
[0023] The multidimensional collaborative attention part in step S3 consists of three branches, where the first two branches are responsible for capturing the interdependence of features on the spatial dimensions W and H, respectively, and the last branch is used to capture the interaction between channels; in the first two branches, permutation operations are used to capture the long-range dependency between the channel dimension and any spatial dimension; finally, the outputs of all three branches are aggregated by simple averaging in the integration stage.
[0024] Deep Convolutional Neural Networks in R 3 The Det network is the basic model, in which the feature extraction network is the Resnet50 structure.
[0025] In a second aspect of the present invention, the present invention provides a ship wake detection system based on multi-dimensional feature collaborative fusion, which is special in that it includes a preprocessing module, a deep learning neural network based on multi-dimensional feature collaborative fusion, and a control output module; The preprocessing module is used to perform general data enhancement or standardization processing on the input remote sensing image to be detected to form a preprocessed image set; The deep learning neural network based on multi-dimensional feature collaborative fusion includes a feature extraction part, a multi-dimensional feature collaborative fusion part and a detection regression part; the feature extraction part is used to extract the feature map of each image data in the pre-processed image set, the multi-dimensional feature collaborative fusion part is used to fuse the feature map, and the detection regression part is used to output feature positioning and classification results according to the feature map; The control output module is used to output the feature positioning and classification results of the deep learning neural network based on multi-dimensional feature collaborative fusion to the user.
[0026] Furthermore, the multidimensional feature collaborative fusion part includes a multidimensional feature embedded Gaussian non-layout attention part and a multidimensional collaborative attention part. The multidimensional feature embedded Gaussian non-layout attention part is used to perform feature scaling, semantic feature balancing, embedded Gaussian non-local attention fusion and feature scaling processing on the input features to obtain a first feature map; the multidimensional collaborative attention part is used to perform global maximum pooling, global standard deviation pooling and depth-wise separable convolution processing on the feature map to obtain a second feature map output by the multidimensional feature collaborative attention part.
[0027] In a third aspect of the present invention, the present invention discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.
[0028] In a fourth aspect of the present invention, the present invention discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.
[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention proposes to apply multi-dimensional feature collaborative fusion to the field of remote sensing target detection, and solves the problem of inconsistent feature layers in the feature pyramid of single-stage target detection by means of multi-dimensional feature collaborative fusion. The network can simultaneously fuse low-dimensional to high-dimensional features by using multi-dimensional feature collaborative fusion.
[0030] (2) The multi-dimensional feature collaborative fusion proposed in this invention only increases the amount of computation and has little impact on detection efficiency in practical applications.
[0031] (3) The present invention optimizes the problem of feature layer imbalance in the training process and the problem that the potential of the model is not fully utilized. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a flowchart of collaborative fusion of multi-dimensional features based on the attention mechanism; Figure 2 This is the effect diagram of the real ship wake image detection; Figure 3 This is the effect diagram of the real ship wake image detection; Figure 4 This is the effect diagram of real ship wake image detection. DETAILED DESCRIPTION
[0033] In order to enable those skilled in the art to better understand the solution of the present invention, the following will be combined with the drawings in the embodiments of the present application to further clearly and completely describe the technical solutions in the embodiments of the present application. It should be noted that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.
[0034] In order to make the invention purpose, technical scheme and advantages of this application clearer, the technical scheme of the present invention is described clearly and completely in conjunction with the accompanying drawings and specific implementation schemes. It should be pointed out that for ordinary technicians in this field, several changes and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of this application.
[0035] In existing target detection methods, although high-level features contain rich semantic information, it is difficult to accurately preserve the location information of objects due to low resolution. In contrast, although low-level features have less semantic information, they can accurately contain object location information due to high resolution. The present invention relies on semantic features of multi-dimensional feature collaborative fusion to enhance original features. In this way, each resolution in the feature pyramid obtains equal information from other resolutions, thereby balancing the information, making the features more discriminative, and obtaining better trail detection effects.
[0036] In order to solve the above technical problems, the present invention provides a ship wake detection method and system based on multi-dimensional feature collaborative fusion. The overall network structure of the present invention adopts a deep convolutional neural network, including a feature extraction part, a multi-dimensional feature collaborative fusion part and a detection regression part. The feature extraction part is used to extract features from the preprocessed image, downsample the image data through convolution, extract features of different scales, map the image data to the feature space, and obtain a feature map; the multi-dimensional feature collaborative fusion part is used to perform feature fusion processing on the feature map, adjust, integrate, fuse the size of the feature map, add an attention mechanism, and output the final fused feature map; the detection regression part is used to obtain the feature positioning and classification results of the remote sensing image data according to the final feature map.
[0037] like Figure 1 As shown, in the first embodiment of the present invention, the present invention proposes a ship wake detection method based on multi-dimensional feature collaborative fusion, comprising the following steps: Step S1: input the remote sensing image to be detected, perform image preprocessing, and form a preprocessed image set; Step S2: extract features from each remote sensing image in the preprocessed image set through the feature extraction part of the deep convolutional neural network, map the image data to the feature space, and obtain a feature map; Step S3: performing balanced fusion of features on the feature map through the multidimensional feature collaborative fusion part of the deep convolutional neural network, wherein the balanced fusion of features on the feature map by the multidimensional feature collaborative fusion part includes inputting the feature map into the multidimensional feature embedded Gaussian non-layout attention part and the multidimensional collaborative attention part to obtain a first feature map and a second feature map, and performing an addition operation on the first feature map and the second feature map to obtain a final feature map; Step S4: The detection regression part of the deep convolutional neural network is used to obtain the feature location and classification results of the remote sensing image data according to the final feature map.
[0038] In the first embodiment of the present invention, the remote sensing image to be detected can be a remote sensing image acquired by a drone or a remote sensing image acquired by a remote sensing satellite. Optical remote sensing images, as a technology that uses electromagnetic waves (light) to obtain information on the earth's surface, have the advantages of high resolution, rich band data, efficient and fast acquisition, and wide scanning coverage. These advantages make optical remote sensing images widely used in environmental protection, resource management, land use, urban planning, disaster management and other fields. In this embodiment, optical remote sensing images are selected to detect changes in the coastal zone. Image preprocessing is performed by obtaining optical remote sensing images of different phases of the area to be detected to obtain a preprocessed image data set.
[0039] It should be noted that after acquiring the remote sensing image to be detected, the remote sensing image must first be subjected to basic image preprocessing, which can help improve the accuracy and reliability of feature detection. The image preprocessing of the remote sensing image in step S1 includes one or more of radiation correction, geometric correction, and image enhancement. The optical remote sensing image after image preprocessing has higher geometric and pixel accuracy.
[0040] In this embodiment, the feature extraction part of the deep convolutional neural network is used to extract features from the preprocessed input remote sensing image. The purpose of extracting features from the remote sensing image is to improve the key feature information in the remote sensing image, eliminate redundant information irrelevant to feature detection, and improve the accuracy and efficiency of the detection calculation. The feature extraction methods in the prior art include original feature extraction, neural network feature extraction, etc. In this embodiment, the neural network feature extraction method is selected, and the feature extraction network adopts the Resnet50 structure. Its main function is to downsample the image of the data through convolution and extract features of different scales. In this embodiment, the multiple feature maps include feature maps C2-C5 of different sizes, a total of four feature maps.
[0041] In this embodiment, the multidimensional feature collaborative fusion part includes a multidimensional feature embedded Gaussian non-layout attention part and a multidimensional collaborative attention part. After completing the feature extraction, the extracted feature maps are respectively input into the multidimensional feature embedded Gaussian non-layout attention part and the multidimensional collaborative attention part of the multidimensional feature collaborative fusion part for further feature processing.
[0042] In the multi-dimensional feature embedded Gaussian non-layout attention part, the feature map is processed including feature scaling, semantic feature balancing, embedded Gaussian non-layout attention fusion and feature scaling processing to obtain feature maps of different scales. In step S3, the feature map is input into the multi-dimensional feature embedded Gaussian non-layout attention part to obtain the first feature map, including: Step S311: performing feature scaling processing on the feature map to obtain multiple feature maps of the same size, wherein the sizes of the multiple feature maps of the same size are consistent with the size of the middle layer of the feature map; Step S312 performs semantic feature balancing on multiple feature maps of the same size, averages multiple feature maps of the same size to obtain a feature map C, and The characteristics of the layer are recorded as , the number of multi-layer features is recorded as , The minimum and maximum values of and , the calculation formula of the feature map C is: ;
[0043] In step S313, the feature map is subjected to embedded Gaussian non-layout attention fusion and feature scaling processing to obtain the first feature map output by the multi-dimensional feature embedded Gaussian non-layout attention part.
[0044] After completing the feature extraction, the multi-dimensional feature embedded Gaussian non-layout attention part performs feature scaling on the feature map and adjusts the size of the feature map to a fixed size. The fixed size is selected from the middle size of multiple feature maps. In this embodiment, the feature maps of different scales obtained by the feature extraction part include 4 feature maps C2-C5, and the resolution gradually increases from C2-C5. When scaling the features, the middle-sized feature map C4 is selected as the reference standard for feature map size adjustment, so that other feature maps are adjusted to the same size as the feature map C4. The specific adjustment method includes resizing the C2 and C3 layers to the size of the C4 layer using the maximum pooling operation, and resizing the C5 layer to the size of the C4 layer using the bilinear interpolation operation.
[0045] After completing the feature scaling, the feature map is semantically balanced in the multi-dimensional feature embedded Gaussian non-layout attention part. Specifically, multiple feature maps with the same size are averaged to obtain a feature map C. The characteristics of the layer are recorded as , the number of multi-layer features is recorded as , The minimum and maximum values of and , the calculation formula of the feature map C is: .
[0046] Further, the embedded Gaussian non-local attention processing formula in step S313 includes:
[0047] is the input feature map, is the output feature map; i is the index of the output position, and j is the index of all possible positions; is the correlation coefficient between two points, The calculation is the representation of the input feature map at position j; the final Through the impact factor After standardization, in The formula is:
[0048] in , the actual operation is implemented using a 1×1 convolution; , the actual operation is implemented using a 1×1 convolution; in , It is a weight matrix that needs to be learned. The actual operation is implemented by a 1×1 convolution. represents the normalization operation, which is implemented using the Sigmoid function in the embedded Gaussian. The formula is: .
[0049] The feature maps obtained by the embedded Gaussian non-layout attention processing are feature scaled to obtain multiple feature maps of different scales. Specifically, the fused feature maps are finally scaled to the size of the original C2, C3, C4, and C5 layers, and the bilinear interpolation operation is used to adjust them to the size of C2 and C3 respectively. The maximum pooling operation is used to adjust the fused feature map C5 to the size of C5 after the multi-dimensional collaborative attention part is processed to obtain the first feature map of the output.
[0050] Inputting the feature map into the multi-dimensional collaborative attention part to obtain the second feature map in step S3 includes: performing global maximum pooling, global standard deviation pooling and depth-separable convolution on the feature map to obtain the second feature map output by the multi-dimensional feature collaborative attention part. Inputting the feature map into the multi-dimensional collaborative attention part to obtain the second feature map in step S3 further includes: using global average pooling and global standard deviation pooling to respectively process the input feature map Perform feature aggregation to obtain the average pooling feature in the channel dimension And standard error pooling features , where the two pooling operations of the mth channel can be expressed as follows:
[0051]
[0052] in represents the feature of the mth channel in the input feature map, and It aggregates the features of the mth channel in the input feature map F to obtain the average pooling features and standard deviation pooling features in the channel dimension; Then, an adaptive combination mechanism is used to fuse the two pooling features to obtain the feature map after global average pooling and global standard deviation pooling operations. :
[0053] in, and are two learnable coefficients used to adaptively adjust the relative importance of average pooling and standard deviation pooling features.
[0054] Furthermore, the multi-dimensional collaborative attention part in step S3 includes three branches, of which the first two branches are responsible for capturing the interdependence of features on the spatial dimensions W and H, respectively, while the last branch is mainly used to capture the interaction between channels; in the first two branches, a permutation operation is used to capture the long-range dependency between the channel dimension and any spatial dimension; finally, the outputs of all three branches are aggregated by simple averaging in the integration stage. The multi-dimensional collaborative attention mechanism consists of three branches, of which the first two branches are responsible for capturing the interdependence of features on the spatial dimensions W and H, respectively, while the last branch is used to capture the interaction between channels. In the first two branches, a permutation operation is used to capture the long-range dependency between the channel dimension and any spatial dimension. Finally, the outputs of all three branches are aggregated by simple averaging in the integration stage. Among them, the first two branches are the top branch and the middle branch, and the last branch is the bottom branch. Then, the input feature map is firstly subjected to independent channel-by-channel convolution in the spatial dimension without interaction between channels. C independent two-dimensional convolution kernels are used, each with a size of ̂3×3, to obtain the intermediate output feature map. . Then A standard 1x1 convolution operation is performed to establish the interaction between channels.
[0055] Specifically, in the top branch, the feature map input to the MCA module First, rotate the feature map 90° counterclockwise along the H axis. The resulting feature map is represented as follows: In order to model the long-range dependencies between the channel dimension C and the spatial dimension H, we then Perform global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, and the resulting aggregate feature map is described as: Then A depth-wise separable convolution operation is performed to capture the interaction between features in the spatial dimension W, and the feature weights along the width direction are expressed as Next, we pass a , generating input-specific attention weights in W dimensions Then, we multiply the element by element. Apply to On the top, we get the enhanced feature map Finally, along The axis is rotated 90° clockwise to obtain a feature map with the same shape as the original input Mathematically, this process can be summarized in the following equation:
[0056]
[0057] in It means rotating 90° counterclockwise along the H axis. represents the opposite operation, both of which can be easily implemented through the permute function in the PyTorch framework. represents the sigmoid activation function, where Represents the global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, Represents a depthwise separable convolution operation.
[0058] In the middle branch, the input feature map F is first rotated 90° counterclockwise along the W axis to obtain the rotated feature map In order to model the interdependence between the channel dimension C and the spatial dimension H and further capture the interaction between the heights, Perform global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, and deep separable convolution operations, thereby deriving aggregate feature maps in turn and height direction feature weight . Then pass it through the sigmoid activation function , which can generate input-specific attention weights in the H dimension Then, by Recalibration , generating an enhanced feature map .final Rotate 90° clockwise along the W axis to obtain a feature map with the same shape as the original input Formally, the process can be summarized as follows:
[0059]
[0060] in It means rotating 90° counterclockwise along the W axis. represents the opposite operation. Both of these can be easily implemented through the permute function in the PyTorch framework. represents the sigmoid activation function, where Represents the global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, Represents a depthwise separable convolution operation.
[0061] The bottom-level branch is mainly responsible for modeling the spatial (H and W) interdependencies and capturing the interactions between channels. In short, first an identity map is generated to generate the same feature map Then The aggregated feature maps can be inferred by sequentially inputting into global average pooling (AvgPool), global standard deviation pooling (StdPool), and depth-separable convolution. and channel feature weights .after Through the sigmoid activation function, the input-specific channel attention weights are derived . Then, pass Rescaled to generate an enhanced feature map Finally, the identity mapping function is used to transform Remapping . Again, the process can be summarized as follows:
[0062]
[0063] in represents the sigmoid activation function, where Represents the global average pooling (AvgPool) and global standard deviation pooling (StdPool) operations, Represents a depthwise separable convolution operation.
[0064] After the multi-dimensional collaborative attention part is processed and output, the second feature map is obtained, which includes feature maps of multiple scales. Finally, the first feature map and the second feature map are added to obtain the final feature map. Specifically, C2-C5 in the first feature map is added to C2-C5 in the second feature map in corresponding sizes to obtain the final feature map.
[0065] The deep convolutional neural network in this embodiment is based on R 3 The Det network is the basic model. The part of the neural network based on the collaborative fusion of multi-dimensional features is mainly a structure similar to a feature pyramid. For general feature fusion, the fused features will pay more attention to adjacent feature layers and less attention to other feature layers. In the information flow, the semantic information contained in non-adjacent layers will be diluted once each fusion. The collaborative fusion of multi-dimensional features can solve the problem of inconsistent feature layers in the feature pyramid of single-stage target detection. The collaborative fusion of multi-dimensional features allows the network to simultaneously fuse low-dimensional to high-dimensional features.
[0066] In the second embodiment of the present invention, the present invention provides a ship wake detection system based on multi-dimensional feature collaborative fusion, which is special in that it includes a preprocessing module, a deep learning neural network based on multi-dimensional feature collaborative fusion, and a control output module; The preprocessing module is used to perform general data enhancement or standardization processing on the input remote sensing image to be detected to form a preprocessed image set; The deep learning neural network based on multi-dimensional feature collaborative fusion includes a feature extraction part, a multi-dimensional feature collaborative fusion part and a detection regression part; the feature extraction part is used to extract the feature map of each image data in the pre-processed image set, the multi-dimensional feature collaborative fusion part is used to fuse the feature map, and the detection regression part is used to output feature positioning and classification results according to the feature map; The control output module is used to output the feature positioning and classification results of the deep learning neural network based on multi-dimensional feature collaborative fusion to the user.
[0067] Furthermore, the multidimensional feature collaborative fusion part includes a multidimensional feature embedded Gaussian non-layout attention part and a multidimensional collaborative attention part. The multidimensional feature embedded Gaussian non-layout attention part is used to perform feature scaling, semantic feature balancing, embedded Gaussian non-local attention fusion and feature scaling processing on the input features to obtain a first feature map; the multidimensional collaborative attention part is used to perform global maximum pooling, global standard deviation pooling and depth-wise separable convolution processing on the feature map to obtain a second feature map output by the multidimensional feature collaborative attention part.
[0068] In a third embodiment of the present invention, the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program implements any one of the methods described in the first aspect when executed by a processor.
[0069] In a fourth embodiment of the present invention, the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program implements any one of the methods described in the first aspect when executed by a processor.
[0070] Figure 2 A ship wake is identified, and the detection result shows that the probability of a ship wake is 83%; Figure 3 Two ship wakes were identified, with the probability of detecting ship wakes being 78% and 77% respectively; Figure 4 Three ship wakes were identified, and the detection results showed that the probability of ship wakes was 75%, 73%, and 75% respectively. 3The average detection result of all images of Det on the SWIM dataset is 75.2%, and the average detection result of all images of this detection method on the SWIM dataset is 77.2%.
[0071] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.
[0072] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solution of this patent rather than to limit it. Although this patent has been described in detail with reference to the preferred embodiments, ordinary technicians in this field should understand that the technical solution of this patent can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of this patent, which should be included in the scope of the claims of this patent.
Claims
1. A ship wake detection method based on multi-dimensional feature collaborative fusion, characterized by: The method comprises: Step S1: input the remote sensing image to be detected, perform image preprocessing, and form a preprocessed image set; Step S2: extract features from each remote sensing image in the preprocessed image set through the feature extraction part of the deep convolutional neural network, map the image data to the feature space, and obtain a feature map; Step S3: performing balanced fusion of features on the feature map through the multidimensional feature collaborative fusion part of the deep convolutional neural network, wherein the balanced fusion of features on the feature map by the multidimensional feature collaborative fusion part includes inputting the feature map into the multidimensional feature embedded Gaussian non-layout attention part and the multidimensional collaborative attention part to obtain a first feature map and a second feature map, and performing an addition operation on the first feature map and the second feature map to obtain a final feature map; Step S4: The detection regression part of the deep convolutional neural network is used to obtain the feature location and classification results of the remote sensing image data according to the final feature map.
2. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1 is characterized in that: In step S3, inputting the feature map into the multi-dimensional feature embedded Gaussian non-layout attention part to obtain the first feature map includes: Step S311: performing feature scaling processing on the feature map to obtain feature maps of chairs of multiple sizes, wherein the sizes of the feature maps of the multiple sizes are consistent with the size of the middle layer of the feature map; Step S312: Perform semantic feature balancing on multiple feature maps of the same size, average multiple feature maps of the same size to obtain a feature map C, and The characteristics of the layer are recorded as , the number of multi-layer features is recorded as , The minimum and maximum values of and , the calculation formula of the feature map C is: ; Step S313: The feature map is processed by embedded Gaussian non-layout attention and feature scaling to obtain the first feature map output by the embedded Gaussian non-layout attention part of the multi-dimensional feature.
3. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 2 is characterized in that: The embedded Gaussian non-local attention processing formula in step S313 includes: ; is the input feature map, is the output feature map; i is the index of the output position, and j is the index of all possible positions; is the correlation coefficient between two points, The calculation is the representation of the input feature map at position j; the final Through the impact factor After standardization, in The formula is: ; in , the actual operation is implemented using a 1×1 convolution; , the actual operation is implemented using a 1×1 convolution; in It is a weight matrix that needs to be learned. The actual operation is implemented by a 1×1 convolution. represents the normalization operation, which is implemented using the Sigmoid function in the embedded Gaussian. The formula is: 。 4. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1 is characterized in that: In step S3, inputting the feature map into the multi-dimensional collaborative attention part to obtain the second feature map includes: The feature map is processed with global maximum pooling, global standard deviation pooling and depthwise separable convolution to obtain the second feature map output by the multi-dimensional feature collaborative attention part.
5. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 4 is characterized in that: The step S3 of inputting the feature map into the multi-dimensional collaborative attention part to obtain the second feature map further includes: using global average pooling and global standard deviation pooling to respectively input the feature map Perform feature aggregation to obtain the average pooling feature in the channel dimension And standard error pooling features , where the two pooling operations of the mth channel can be expressed as follows: ; ; in represents the feature of the mth channel in the input feature map, and It aggregates the features of the mth channel in the input feature map F to obtain the average pooling features and standard deviation pooling features in the channel dimension; Then, an adaptive combination mechanism is used to fuse the two pooling features to obtain the feature map after global average pooling and global standard deviation pooling operations. : ; in, and are two learnable coefficients used to adaptively adjust the relative importance of average pooling and standard deviation pooling features.
6. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1 is characterized in that: The multi-dimensional collaborative attention part in step S3 includes three branches, wherein the first two branches are responsible for capturing the interdependence of features on the spatial dimensions W and H, respectively, and the last branch is used to capture the interaction between channels; in the first two branches, a permutation operation is used to capture the long-range dependency between the channel dimension and any spatial dimension; finally, the outputs of all three branches are aggregated by simple averaging in the integration stage.
7. The ship wake detection method based on multi-dimensional feature collaborative fusion according to claim 1 is characterized in that: The deep convolutional neural network is based on R 3 The Det network is the basic model, in which the feature extraction network is the Resnet50 structure.
8. A ship wake detection system based on multi-dimensional feature collaborative fusion, characterized in that: It includes a preprocessing module, a deep learning neural network based on multi-dimensional feature collaborative fusion, and a control output module; The preprocessing module is used to perform general data enhancement or standardization processing on the input remote sensing image to be detected to form a preprocessed image set; The deep learning neural network based on multi-dimensional feature collaborative fusion includes a feature extraction part, a multi-dimensional feature collaborative fusion part and a detection regression part; the feature extraction part is used to extract the feature map of each image data in the pre-processed image set, the multi-dimensional feature collaborative fusion part is used to fuse the feature map, and the detection regression part is used to output feature positioning and classification results according to the feature map; The control output module is used to output the feature positioning and classification results of the deep learning neural network based on multi-dimensional feature collaborative fusion to the user.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Remote sensing ship target detection method based on deformation attention pyramid
CN115115601A
Lightweight infrared small target segmentation system and method
CN116416430A
Convolutional neural network crop disease image recognition method based on attention mechanism
CN116612425A
Ship wake detection method and device
CN116824389A
System and method for enhancing target tracking via detector and tracker fusion for unmanned aerial vehicles
US20180182109A1
Cited By
Remote sensing satellite image ship target association method and system
CN120877089A
