Ocean internal wave detection method and device based on visual attention lightweight convolutional neural network

Abstract

The invention discloses an ocean internal wave detection method based on a visual attention lightweight convolutional neural network, and belongs to the technical field of image processing. Carrying out image downsampling; sliding window processing; extracting features; and result clustering discrimination. The algorithm model constructed by the method is high in generalization, high in robustness, excellent in performance and easy to carry out engineering deployment, and high-precision detection of the internal waves can be realized under a strong constraint condition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an ocean internal wave detection method based on a visual attention lightweight convolutional neural network. Background Art

[0002] Ocean internal waves are waves that are generated, propagated, and dissipated within the ocean. They are widely distributed in the global ocean. The detection of ocean internal waves is crucial for the study of ocean phenomena and maritime safety. The generation and propagation of internal waves are highly random, so the detection of ocean internal waves in SAR images has become a difficult and hot research topic.

[0003] At present, the detection methods for ocean internal wave phenomena mainly include traditional detection methods based on image processing and artificial intelligence detection methods based on deep learning. Traditional detection mainly extracts edge texture information of images through image processing to achieve internal wave detection. Xu Suqin et al. (2019) used the Canny operator to extract edge features of SAR images, and then used hierarchical clustering to extract internal wave features. Zheng Yinggang et al. (2020) used the column separation field method to denoise the image before detecting internal waves based on the Canny operator, which improved the detection effect to a certain extent. Although the commonly used edge feature detection operator has high detection accuracy for specific internal wave images, it also has problems such as inability to effectively remove texture features other than internal waves and susceptibility to noise interference. The artificial intelligence detection method based on deep learning extracts feature information of images through deep learning networks to achieve internal wave detection. Bao et al. (2020) detected ocean internal waves in SAR images by using the Faster R-CNN framework with convolutional neural network features. In addition, the detection results of internal wave samples under different conditions were analyzed. This paper lays the foundation for using convolutional neural networks to detect ocean internal waves. Sun Hongliang et al. (2021) combined the Faster R-CNN framework and used the transfer learning method to realize the automatic detection of ocean internal waves on SAR images. The establishment of this algorithm makes it possible to detect ocean internal waves based on massive satellite SAR data, thus providing a technical and data basis for targeted inversion of internal wave dynamic parameters and process research. The above methods all provide a theoretical and research basis for internal wave detection. However, the existing internal wave detection methods often require a lot of computing resources, and the accuracy needs to be improved. Therefore, how to achieve efficient and rapid detection of ocean internal wave phenomena under the constraints of limited storage and computing resources is a difficult problem at present. Summary of the invention

[0004] In order to address the shortcomings of the prior art, the present invention provides an ocean internal wave detection method based on a visual attention lightweight convolutional neural network. Through a deep learning network, the characteristic information of the internal wave phenomenon is extracted, and high-precision detection of ocean internal waves in SAR images can be achieved under the constraints of limited resource storage and computational constraints.

[0005] The ocean internal wave detection method based on visual attention lightweight convolutional neural network provided in the present disclosure mainly includes the following steps:

[0006] S1, image quantization: convert the original SAR image into an 8-bit depth image;

[0007] S2, image sampling: downsampling the image processed in step S1 by sampling at alternate points to reduce the number of pixels;

[0008] S3, sliding window processing: Use a sliding window on the downsampled image to perform uniform sliding window processing on the entire image to obtain 256*256 slices;

[0009] S4, network feature extraction, including:

[0010] S41, feature extraction of the input 256x256 slice;

[0011] S42, the feature map extracted in step S41 is subjected to spatial attention extraction and channel attention extraction respectively, and then feature fusion is performed;

[0012] S43, classifying the fused feature map to obtain a slice category;

[0013] S5, result clustering and discrimination: cluster the obtained results by position, delete isolated individuals, and obtain the results and confidence of the internal wave area.

[0014] Furthermore, in step S1, the contrast between the strong scattered points and the sea background is enhanced by data stretching enhancement processing.

[0015] Furthermore, the specific method of step S2 includes:

[0016] Only one pixel is retained in each 2*2 pixel block, and the pixel takes the maximum value in the pixel block, so that the width and height of the image are reduced to 1 / 2 of the original.

[0017] Further, in step S41, the residual network module and the maximum reduction sampling The feature extraction network composed of modules performs feature extraction, including:

[0018] 5 residual network modules and maximum downsampling modules, where each residual network module includes 2 depth-separable convolutional layers and 1 1*1 convolutional layer. The depth-separable convolutional layer is used to realize lightweight network feature extraction, and the 1*1 convolutional layer is used to perform channel dimensionality reduction and dimensionality increase processing on the input feature map;

[0019] The maximum downsampling module is used to perform downsampling processing on the feature map to obtain feature maps of different scales.

[0020] Furthermore, the step S42 specifically includes:

[0021] The feature map extracted in step S41 is processed by the spatial attention module and the channel attention module respectively;

[0022] Then, the obtained feature map is multiplied to obtain the fused feature map;

[0023] Among them, the spatial attention module first passes the input feature map through two convolution modules, and then processes it through the Softmax activation function to filter and enhance the feature information of the feature map in space;

[0024] The channel attention module includes: a maximum mean pooling layer, connected to 2 convolutional layers, and then connected to a softmax layer, and the output is a C-dimensional weight coefficient.

[0025] Furthermore, the step S43 specifically includes:

[0026] The feature map obtained after feature fusion is flattened and then passed through two fully connected layers to obtain the final category.

[0027] The ocean internal wave detection device based on visual attention lightweight convolutional neural network using the above method mainly includes:

[0028] Image quantization unit: used to convert the original SAR image into an 8-bit depth image;

[0029] Image sampling unit: used to downsample the quantized image;

[0030] Sliding window processing unit: used to perform uniform sliding window processing on the downsampled image to obtain 256*256 slices;

[0031] Network feature extraction unit: including: feature extraction network, used to extract features from the input 256x256 slice; feature fusion network, used to extract spatial attention and channel attention from the feature map extracted by the feature extraction network, and then perform feature fusion; classification module, used to classify the feature map obtained after feature fusion to obtain the slice category;

[0032] Result clustering and discrimination unit: used to cluster the results by location, delete isolated individuals, and obtain the results and confidence of the internal wave area.

[0033] Furthermore, the feature extraction network includes:

[0034] 5 residual network modules and maximum downsampling modules, where each residual network module includes 2 depth-separable convolutional layers and 1 1*1 convolutional layer. The depth-separable convolutional layer is used to realize lightweight network feature extraction, and the 1*1 convolutional layer is used to perform channel dimensionality reduction and dimensionality increase processing on the input feature map;

[0035] The maximum downsampling module is used to perform downsampling processing on the feature map to obtain feature maps of different scales.

[0036] Furthermore, the feature fusion network includes:

[0037] The spatial attention module is used to filter and enhance the feature information of the input feature map by first passing it through two convolution modules and then through the Softmax activation function;

[0038] The channel attention module includes: a maximum mean pooling layer, connected to two convolutional layers, and then connected to a softmax layer, and the output is a C-dimensional weight coefficient;

[0039] The fusion module is used to perform a product operation on the feature maps obtained separately to obtain a fused feature map.

[0040] Compared with the prior art, the beneficial effects of the present invention are: (1) the original image data is quantized from 32 bits to 8 bits to improve processing efficiency, and a data stretching enhancement module is added to enhance the contrast between strong scattered points and the sea background; (2) the image is downsampled to reduce the amount of data and speed up the processing speed; (3) the internal wave features under complex backgrounds are effectively extracted through the channel attention mechanism, the spatial texture attention mechanism and the lightweight feature extraction module, and the feature pyramid fusion mechanism is introduced to identify the internal wave textures such as waves and oil spills, effectively solving the problem of difficult detection of internal wave phenomena and high false alarms under complex backgrounds; (4) high-precision detection of ocean internal waves in SAR images can be achieved under the constraints of limited resource storage and computing power; (5) the constructed lightweight network model has a simple and easy-to-implement process, a small number of model parameters and low computational complexity, and is easy to deploy in engineering, so that high-precision and rapid detection of internal waves in SAR images can be achieved on the terminal processing platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0042] Figure 1 is an overall flow chart of the ocean internal wave detection method based on visual attention lightweight convolutional neural network according to the present disclosure, wherein (a) is the overall flow chart, (b) and (c) are flow charts of feature extraction and feature fusion classification, respectively;

[0043] Figure 2 The following are typical internal wave phenomenon detection results. The solid-line frame is the true value annotated by experts, the thin dotted-line frame is the detection result of the sliding window algorithm, and the thick dotted-line frame is the result after the algorithm detection results are merged and processed. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] The present disclosure provides a method and device for detecting ocean internal waves based on a visual attention lightweight convolutional neural network. In an exemplary embodiment, the main processing flow is as follows: Figure 1 As shown:

[0046] First, the image is quantized and downsampled, and then the image is processed by uniform sliding window to obtain 256*256 small blocks. Then, the deep learning network is used to extract features, including feature extraction, feature fusion and classification modules. Features are extracted through convolutional networks and downsampling layers, and classified through feature fusion layers and fully connected layers. Finally, the results are clustered and judged. The results are clustered by position, isolated individuals are deleted, and the results and confidence of the internal wave area are obtained.

[0047] Further details are as follows:

[0048] Step (1), image quantization: convert the original image into an 8-bit image. The original imaging data is 32-bit floating point type, and the effective data width of image processing is 8 bits. In order to improve the processing efficiency, the data is quantized from 32 bits to 8 bits, and a data stretching enhancement module is added to enhance the contrast between the strong scattered points and the sea background;

[0049] Step (2), image sampling: downsample the image, that is, reduce the number of pixels in the image. This is achieved by sampling at alternate points, retaining only one pixel (maximum value) in each 2*2 pixel block, so that the width and height of the image are reduced to 1 / 2 of the original. This method can further reduce the amount of data and speed up processing;

[0050] Step (3), sliding window: Apply a 256x256 sliding window to the downsampled image. This window moves from left to right and from top to bottom on the image, one block at a time, so that the entire image is processed evenly;

[0051] Step (4), network feature extraction: First, the feature extraction module is used to extract features from the input 256x256 slice, and then the feature fusion layer is used to fuse the features mentioned in deep learning. Finally, the classification module is used to classify the slice category.

[0052] Step (4.1), construct a feature extraction network to extract features from the input image to form a feature map:

[0053] Preferably, the network is composed of 5 residual network modules and a maximum downsampling module. Each residual network module is composed of 2 depth-separable convolutional layers and 1 1*1 convolution. The depth-separable convolutional layer can realize lightweight network feature extraction, and the 1*1 convolutional layer can perform channel dimension reduction and dimension increase processing on the input feature map. In view of the multi-scale problem of internal wave phenomenon, the maximum downsampling module is used to downsample the feature map to obtain feature maps of different scales.

[0054] Step (4.2), construct the spatial attention module and the channel attention module to fuse the feature map. The feature map extracted in step (4.1) passes through the spatial attention module and the channel attention module respectively, and then the obtained feature map is multiplied to obtain the fused feature map.

[0055] Among them, as a preferred embodiment: the spatial attention module first passes the feature map through two convolution modules, and then processes it through the Softmax activation function, filtering and enhancing the feature information of the feature map in space. The channel attention module consists of a maximum mean pooling layer, then connected to two convolution layers, and then connected to a softmax layer, and the output is a C-dimensional weight coefficient. Through the above operations, the interference information of the feature map can be effectively filtered out from the channel and space, and the feature information of the internal wave can be enhanced.

[0056] Step (4.3), category determination. Flatten the feature map obtained in step (4.2), and then pass it through two fully connected layers to obtain the final category.

[0057] Step (5), result clustering and discrimination: cluster the obtained results by position, delete isolated individuals, and obtain the results and confidence of the internal wave area.

[0058] Application Examples

[0059] 1. Experimental conditions:

[0060] The hardware environment is: the processor is Intel(R) Xeon Gold 5218cpu, the main frequency is 2.3GHz, the memory is 256GB, and the graphics card is NVIDIA V100.

[0061] The software environment is: Ubuntu 18.04, Python 3.8.13, PyTorch 1.7.1, cuda 11.1.

[0062] 2. Experimental content and results analysis:

[0063] The internal wave detection database is constructed using 600 scenes of satellite imaging data, and 500 scenes are randomly selected as training sets and 100 scenes are selected as test sets.

[0064] In the training set, there are 300 scenes containing internal wave phenomena and 200 scenes without internal wave phenomena. The data are downsampled and windowed to obtain 1832 internal wave samples and 16493 samples without internal wave. For the internal wave sample data, the positive samples are augmented by automatic contrast, histogram equalization, image rotation, image sharpness adjustment, image translation transformation and other data augmentation methods. At the same time, during the training process, the weights of positive and negative samples are set to 1:1.6, the initial learning rate is 0.01, and the momentum is 0.009. After 300 epochs of training, the optimal model is obtained.

[0065] The optimal model obtained through training was used to test 100 test data scenes, and finally a detection rate of 90.18% and a false alarm rate of 9.28% were obtained. Compared with the traditional residual neural network resnet18, the detection rate was increased by 9% and the false alarm rate was reduced by 5%, which greatly improved the performance of internal wave detection.

[0066] The indicators are defined as follows:

[0067]

[0068] in

[0069] TP: The area is judged to have quantity; FN: The area is judged to have no quantity;

[0070] TN: No area is judged as no quantity; FP: No area is judged as having quantity;

[0071] P: the number of regions with (TP+FN); N: the number of regions without (TN+FP).

[0072] The above technical scheme is only an exemplary embodiment of the present invention. For those skilled in the art, it is easy to make various types of improvements or modifications based on the application methods and principles disclosed in the present invention, and it is not limited to the method described in the above specific embodiment of the present invention. Therefore, the method described above is only preferred and does not have a restrictive meaning.

Claims

1. A method for detecting ocean internal waves based on a visual attention lightweight convolutional neural network, comprising the following steps: S1, image quantization: convert the original SAR image into an 8-bit image and enhance the contrast between the strong scattered points and the sea background; S2, image sampling: downsampling the image processed in step S1 by sampling at alternate points to reduce the number of pixels; S3, sliding window processing: Use a sliding window on the downsampled image to perform uniform sliding window processing on the entire image to obtain 256*256 slices; S4, network feature extraction, including: S41, feature extraction of the input 256x256 slice; S42, the feature map extracted in step S41 is subjected to spatial attention extraction and channel attention extraction respectively, and then feature fusion is performed; S43, classifying the fused feature map to obtain a slice category; S5, result clustering and discrimination: cluster the obtained results by location, delete isolated individuals, and obtain the recognition results and confidence of the ocean internal wave area.

2. The method according to claim 1, characterized in that In step S1: By adding data stretching enhancement processing, the contrast between strong scattered points and the sea background is enhanced.

3. The method according to claim 1 or 2, characterized in that: The specific method of step S2 includes: Only one pixel is retained in each 2*2 pixel block, and the pixel takes the maximum value in the pixel block, so that the width and height of the image are reduced to 1 / 2 of the original.

4. The method according to claim 1 or 2, characterized in that: In step S41, the residual network module and the maximum reduction sampling The feature extraction network composed of modules performs feature extraction, including: 5 residual network modules and maximum downsampling modules, where each residual network module includes 2 depth-separable convolutional layers and 1 1*1 convolutional layer. The depth-separable convolutional layer is used to realize lightweight network feature extraction, and the 1*1 convolutional layer is used to perform channel dimensionality reduction and dimensionality increase processing on the input feature map; The maximum downsampling module is used to perform downsampling processing on the feature map to obtain feature maps of different scales.

5. The method according to claim 4, characterized in that The step S42 specifically includes: The feature map extracted in step S41 is processed by the spatial attention module and the channel attention module respectively; Then, the obtained feature map is multiplied to obtain the fused feature map; Among them, the spatial attention module first passes the input feature map through two convolution modules, and then processes it through the Softmax activation function to filter and enhance the feature information of the feature map in space; The channel attention module includes: a maximum mean pooling layer, connected to 2 convolutional layers, and then connected to a softmax layer, and the output is a C-dimensional weight coefficient.

6. The method according to claim 4, characterized in that The step S43 specifically includes: The feature map obtained after feature fusion is flattened and then passed through two fully connected layers to obtain the final category.

7. An ocean internal wave detection device based on a visual attention lightweight convolutional neural network using the method described in any one of claims 1 to 6, characterized in that: include: Image quantization unit: used to convert the original SAR image into an 8-bit depth image; Image sampling unit: used to downsample the quantized image; Sliding window processing unit: used to perform uniform sliding window processing on the downsampled image to obtain 256*256 slices; Network feature extraction unit: including: feature extraction network, used to extract features from the input 256x256 slice; feature fusion network, used to extract spatial attention and channel attention from the feature map extracted by the feature extraction network, and then perform feature fusion; classification module, used to classify the feature map obtained after feature fusion to obtain the slice category; Result clustering and discrimination unit: used to cluster the results by location, delete isolated individuals, and obtain the results and confidence of the ocean internal wave area.

8. The device according to claim 7, characterized in that The feature extraction network includes: 5 residual network modules and maximum downsampling modules, where each residual network module includes 2 depth-separable convolutional layers and 1 1*1 convolutional layer. The depth-separable convolutional layer is used to realize lightweight network feature extraction, and the 1*1 convolutional layer is used to perform channel dimensionality reduction and dimensionality increase processing on the input feature map; The maximum downsampling module is used to perform downsampling processing on the feature map to obtain feature maps of different scales.

9. The device according to claim 7, characterized in that The feature fusion network includes: The spatial attention module is used to filter and enhance the feature information of the input feature map by first passing it through two convolution modules and then through the Softmax activation function; The channel attention module includes: a maximum mean pooling layer, connected to two convolutional layers, and then connected to a softmax layer, and the output is a C-dimensional weight coefficient; The fusion module is used to perform a product operation on the feature maps obtained separately to obtain a fused feature map.

Images