A method for detecting marine ship targets in sea fog based on deep learning

Abstract

The invention discloses a method for detecting marine ship targets based on deep learning in a sea fog environment, comprising the following steps: obtaining a to-be-processed image containing rain and fog; performing rain and fog removal preprocessing on the to-be-processed image containing rain and fog based on a GCANet deep learning network to obtain the to-be-processed image after the rain and fog are removed; using a SEMSSD target detection model on the to-be-processed image after the rain and fog are removed to identify the type and position of the marine ship targets in the to-be-processed image. The invention adds an image rain and fog removal preprocessing process before target detection, and highlights the significant features of the target by improving the contrast between the target and the background. Compared with the traditional defogging model, the GCANet deep learning rain and fog removal network can not only defog the foggy image, but also keep the fog-free image clear, which is beneficial to improving the detection and recognition accuracy of ship targets in the sea fog environment.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a method for detecting marine ship targets based on deep learning in a sea fog environment. Background Art

[0002] my country has vast sea areas and rich marine resources, and the development and utilization of the ocean has always been a key issue. Target detection for ships at sea has very important research significance.

[0003] The marine environment is complex and changeable. In view of the difficulty of target detection in the current sea fog, deep learning methods are used to first defog the original image, and then detect the target in the defogged image, so as to improve the accuracy of ship target detection in the sea fog environment, so that it can achieve good recognition effect in complex sea fog conditions. In terms of defogging and target detection algorithms, traditional image defogging algorithms are mainly divided into two categories: defogging algorithms based on image enhancement [1] And dehazing algorithm based on image restoration [2] The starting point of the defogging algorithm based on image enhancement is to remove image noise as much as possible, improve image contrast, and thus restore a clear image. The disadvantage is that it is not targeted and is not a true defogging algorithm. The defogging algorithm based on image restoration is basically based on the atmospheric scattering model. [3] , by analyzing the factors affecting image degradation, compensating for the image loss caused by fog and restoring the original image. This method is generally better than the defogging algorithm based on image enhancement, but it relies on prior knowledge. These traditional methods are characterized by large computational complexity and poor robustness. With the rapid development of big data and deep learning, defogging algorithms based on deep learning are becoming more and more popular. [4-9] It came into being. It does not rely on prior knowledge. It uses an end-to-end network to directly regress the residual between the blurry image and the clear image and establish a connection between the two. It can thus realize the direct output of a clear and fog-free image by inputting a foggy image into the network. It has the advantages of very fast running speed and good robustness.

[0004] Object detection is one of the main research directions in the field of computer vision. The task of marine ship target detection is to identify the location and category of ship targets in the acquired sea surface images. Today, target detection is mainly divided into two mainstream research directions: based on digital image processing technology and combined with machine learning

[10] Traditional target detection algorithms and deep convolutional neural networks

[11] The traditional ship target detection algorithm first generates a feature extraction candidate area on the original image, then extracts appropriate target features from the candidate area, and finally generates a recognition and positioning model through Adaboost, support vector machine and other classifiers to perform image recognition detection on the image to be recognized.

[12] The traditional target detection algorithm focuses on improving the recognition accuracy and positioning accuracy of ship target detection, but ignores the time complexity of the algorithm and cannot guarantee the timeliness of the system, making the practicality of the system low. Deep Convolutional Neural Network [13-19] There is no need to generate specific candidate regions for feature extraction. Through multiple convolutions, the system can autonomously extract the overall features of the target and continuously train and improve its own model to finally select the optimal model. Compared with traditional algorithms, the target detection algorithm can detect targets quickly while maintaining high robustness under the premise of high recognition accuracy, and the practicality of the system is greatly improved.

[0005] The current dehazing algorithm is not targeted and cannot cope with the marine environment with a lot of sea fog, resulting in the loss of important information of the target after dehazing. It does not consider compensating for the loss of important information that is beneficial to detection due to image preprocessing, resulting in insufficient feature extraction capabilities of the network.

[0006] The existing target detection model does not fully consider the impact of the shape statistical characteristics of marine ship targets on detection and recognition, resulting in a high false positive rate in marine ship target detection based on deep learning. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a technical solution adopted by the present invention: a method for detecting marine ship targets based on deep learning in a sea fog environment, characterized in that it comprises the following steps:

[0008] Obtain an image to be processed containing rain and fog;

[0009] Based on the GCANet deep learning network, the image to be processed containing rain and fog is preprocessed to remove rain and fog, and the image to be processed after removing rain and fog is obtained;

[0010] The SEMSSD target detection model is used in the image to be processed after rain and fog are removed to identify the type and location of the marine ship targets in the image to be processed.

[0011] Further: the process of performing rain and fog removal preprocessing on the image to be processed containing rain and fog based on the GCANet deep learning network to obtain the image to be processed after removing the rain and fog is as follows:

[0012] For the image to be processed with rain and fog, two convolution kernels with a size of 3×3 and a step size of 1 are used in sequence, followed by a first convolution normalization module with a convolution kernel size of 3×3 and a step size of 2 to encode the image into a feature map.

[0013] The feature map is passed through smooth hole residual blocks with different hole rates in sequence to obtain image features at different levels;

[0014] Features at different levels are fused through a gated fusion sub-network to learn the weights of low, medium, and high-level features and obtain feature maps;

[0015] The feature map is passed through a deconvolution layer and upsampled to the original resolution, and then converted back to the image space through two convolution layers to obtain the fog residual of the marine ship target;

[0016] The fog residual is added to the original foggy input image to obtain the image after rain and fog removal.

[0017] Further: the SEMSSD target detection model uses a VGG network as a main network, and the VGG network includes the third convolutional layer of the fourth convolutional block, a channel attention module, a fully connected layer, three groups of convolutional layers and a group of average pooling layers;

[0018] The third convolutional layer, the channel attention module, and the fully connected layer of the fourth convolutional block are connected in series in sequence;

[0019] Adding three groups of convolutional layers in sequence after the fully connected layer; the three groups of convolutional layers include a first convolutional layer, a second convolutional layer and a third convolutional layer;

[0020] A group of average pooling layers are connected in series after the third convolutional layer.

[0021] Further: the SEMSSD target detection model uses a priori box to predict the target position, and the priori box is determined as follows:

[0022] Counting the shape features of the marine ship targets in the image to be processed;

[0023] The statistical features of the shape of the prior frame are adjusted based on the target features of the marine ship target; the statistical features include aspect ratio information of the prior frame.

[0024] Furthermore: the aspect ratios of the six output layers of the SEMSSD target detection model are: [1, 2], [1, 2, 4], [1, 2, 3], [1, 2, 3], [1, 2], [1, 2].

[0025] Furthermore: it also includes screening a large number of detection frames of different sizes predicted by the feature layer through a non-maximum suppression method to determine the detection frame and the corresponding category corresponding to each target in the image.

[0026] Furthermore: the input feature map of the channel attention module SE is denoted as P1, and the P1 feature map is divided into two paths, one of which is not processed at all and the output is denoted as P2, and the other path passes through global pooling, fully connected layer, ReLU function, fully connected layer, Sigmoid function, and the output is denoted as P3. P2 and P3 are added to obtain the output P4 of the channel attention module SE, which is consistent with the dimension of the input layer P1.

[0027] A deep learning-based marine ship target detection device in a sea fog environment comprises the following steps:

[0028] Acquisition module: used to acquire the image to be processed containing rain and fog;

[0029] Rain and fog preprocessing module: used to perform rain and fog removal preprocessing on the image to be processed containing rain and fog based on the GCANet deep learning network to obtain the image to be processed after removing the rain and fog;

[0030] Detection and recognition module: It is used to use the SEMSSD target detection model to identify the type and location of the marine ship target in the image to be processed after removing the rain and fog.

[0031] The present invention provides a method for detecting marine ship targets based on deep learning in a sea fog environment, which has the following advantages:

[0032] 1. There is a lot of rain and fog at sea. The features of the targets photographed in the rain and fog environment are blurred, which inevitably makes it difficult to extract the features of the targets at sea. The extraction of effective features is closely related to the subsequent detection and recognition accuracy. Based on this, the present invention adds an image de-raining and de-fogging preprocessing process before target detection. By improving the contrast between the target and the background, the significant features of the target are highlighted. Compared with the traditional de-fogging model, the GCANet deep learning de-raining and de-fogging network can not only de-fog the foggy image, but also keep the fog-free image clear, which is beneficial to improve the detection and recognition accuracy of ship targets in the sea fog environment.

[0033] 2. The image dehazing preprocessing operation will also cause the loss of important information that is beneficial to detection. The channel attention module SE is introduced in the basic convolutional layer with rich feature information. This module learns the importance of each feature channel through the attention mechanism, thereby increasing the network's ability to extract features, which is beneficial to improving the subsequent target detection and recognition accuracy.

[0034] 3. In the detection of marine targets based on convolutional neural networks, full use of statistical information such as the shape of the target can make the network training converge faster. Based on this, the present invention abandons the general prior frame, and determines a reasonable prior frame through statistical analysis of the shape characteristics of the ship target data set, effectively reducing the false detection rate of marine ship targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 is a flow chart of the method;

[0037] Figure 2 This is the GCANet network structure diagram;

[0038] Figure 3 It is the structure diagram of G-SEMSSD model;

[0039] Figure 4 (a) 2D distribution of aspect ratios of the MVDD13 dataset (b) 3D distribution of aspect ratios of the MVDD13 dataset;

[0040] Figure 5 (a) is the test result diagram of various models for fishing boats, (b) is the test result diagram of various models for sailboats; (c) is the test result diagram of various models for fireboats; (d) is the test result diagram of various models for cargo ships; (e) is the test result diagram of various models for container ships; (f) is the test result diagram of various models for warships; (g) is the test result diagram of various models for passenger ships; (h) is the test result diagram of various models for submarines; (i) is the test result diagram of various models for bulk carriers; (j) is the test result diagram of various models for tugboats; (k) is the test result diagram of various models for drilling ships; (l) is the test result diagram of various models for tankers; (m) is the test result diagram of various models for cruise ships. DETAILED DESCRIPTION

[0041] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, the numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be clear that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The technology, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be regarded as part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0045] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the device or element referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention: the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0046] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below their position devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0047] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0048] Figure 1 is a flow chart of the method;

[0049] A method for detecting marine ship targets based on deep learning in a sea fog environment comprises the following steps:

[0050] S1: Obtain the image to be processed containing rain and fog;

[0051] S2: Based on the GCANet deep learning network, the image to be processed containing rain and fog is preprocessed to remove rain and fog, and the image to be processed after removing rain and fog is obtained;

[0052] S3: The image to be processed after rain and fog is removed uses the trained SEMSSD target detection model to identify the type and location of the marine ship target in the image to be processed.

[0053] Steps S1 / S2 / S3 are executed sequentially;

[0054] Figure 2 This is the GCANet network structure diagram;

[0055] Further: the process of performing rain and fog removal preprocessing on the image to be processed containing rain and fog based on the GCANet deep learning network to obtain the image to be processed after removing the rain and fog is as follows:

[0056] S11: for the image to be processed containing rain and fog, two first convolution normalization modules with a convolution kernel size of 3×3 and a step size of 1 and a second convolution normalization module with a convolution kernel size of 3×3 and a step size of 2 are used to encode the image into a feature map; the width and height of the feature map are reduced to half of the original respectively;

[0057] S12: The feature map is passed through smooth hole residual blocks with different hole rates in sequence to obtain image features at different levels;

[0058] The output feature map can be sequentially passed through 7 smooth hole residual blocks with hole rates of 2, 2, 2, 4, 4, 4, and 1 to obtain image features at different levels;

[0059] S13: Features at different levels are fused through a gated fusion sub-network to learn the weights of low, medium and high-level features; and a feature map is obtained;

[0060] S14: passing the feature map through a deconvolution layer, upsampling the feature map to the original resolution, and then converting the feature map back to the image space through two convolution layers to obtain the fog residual of the marine ship target;

[0061] The deconvolution layer step size may be 1 / 2;

[0062] The fog residual is added to the original foggy input image to obtain the image after rain and fog removal.

[0063] Figure 3 It is the structure diagram of G-SEMSSD model;

[0064] Furthermore, the SEMSSD target detection model uses a VGG network as a backbone network, and the VGG network includes the third convolutional layer of the fourth convolutional block, a channel attention module, a fully connected layer, three groups of convolutional layers and a group of average pooling layers;

[0065] The third convolutional layer, the channel attention module, and the fully connected layer of the fourth convolutional block are connected in series in sequence;

[0066] Adding three groups of convolutional layers in sequence after the fully connected layer; the three groups of convolutional layers include a first convolutional layer, a second convolutional layer and a third convolutional layer;

[0067] A group of average pooling layers are connected in series after the third convolutional layer.

[0068] Furthermore, regarding the channel attention module, due to defects such as decreased contrast and brightness of foggy images, the edges of targets in the image are often blurred and the texture features are not obvious. In addition, the image defogging preprocessing operation will also cause the loss of important information that is conducive to detection. The channel attention module SE is introduced in the VGG basic convolution layer, and its input feature map is denoted as P1. The P1 feature map is divided into two paths, one of which is not processed in any way and the output is denoted as P2. The other path is successively processed by global pooling, fully connected layer, ReLU function, fully connected layer, Sigmoid function, and the output is denoted as P3. P2 and P3 are added to obtain the output P4 of the SE module, which is consistent with the dimension of the input layer P1.

[0069] The training process of the SEMSSD target detection model is as follows:

[0070] At the front end of the ship target detection network, namely the SEMSSD target detection model, GCANet is added as the image dehazing preprocessing process. The input image size remains unchanged before and after dehazing. The dehazed image is scaled to 300×300 and then input into the backbone network VGG of the SSD model. The Conv4_3 layer (the third convolutional layer of the fourth convolutional block) is connected to the SE module to obtain a feature map F1 with a shape of 38×38×512.

[0071] F1 has two branches, one of which is normalized and output, denoted as M1; the other passes through the FC7 layer of VGG to obtain a 19×19×1024 feature map M2; M2 passes through the convolution layer to obtain a 10×10×512 feature map M3; M3 passes through the convolution layer to obtain a 5×5×256 feature map M4; M4 passes through the convolution layer to obtain a 3×3×256 feature map M5;

[0072] After average pooling, M5 obtains a 1×1×256 feature map M6.

[0073] At this time, the network has a total of six output feature maps M1, M2, M3, M4, M5, and M6. The SSD model uses the prior box to determine the target position. It sets the prior boxes of different sizes and values ​​for the above feature maps, obtains the prior box confidence and position offset, and then performs non-maximum suppression to obtain the predicted box closest to the actual position. Assuming that the network uses feature maps for prediction, for any feature map,

[0074] Further: the SEMSSD target detection model uses a priori box to determine the target position, and the priori box is determined as follows:

[0075] Counting target features of marine ship targets in the image to be processed;

[0076] Adjusting the statistical features of the prior frame shape based on the target features of the marine ship target, wherein the statistical features may be aspect ratio information;

[0077] Because the prior box aspect ratio AR in the original SSD model is obtained based on a general dataset, it is not suitable for ship target detection tasks in marine environments. It is necessary to obtain the statistical features of the target from the ship dataset. Taking the MVDD13 dataset as an example, the AR of the real target box is counted.

[0078] Figure 4 (a) 2D distribution of aspect ratio of MVDD13 dataset (b) 3D distribution of aspect ratio of MVDD13 dataset; the target AR distribution feature is high in the middle and low on both sides, similar to the normal distribution, with mean μ=2.33 and variance σ 2 =1.77. At this time, about 68.2% of the targets are distributed in the interval [μ-σ, μ+σ], that is,

[0079] According to the comparative experimental results, the AR distribution of the 6 output layers is: [1, 2], [1, 2, 4], [1, 2, 3], [1, 2, 3], [1, 2], [1, 2].

[0080] The formula for calculating the prior frame ratio is:

[0081] where s min =0.2,s max =0.9, which represent the minimum and maximum proportions of the prior box to the feature map area. Since the detected targets have different ARs, different ARs are set in the six different output feature maps as described above.

[0082] The width w and height h of each prior box are calculated using the following two formulas:

[0083]

[0084]

[0085] Where: s is the size of the original image;

[0086] The center coordinates of each prior box are Among them, |f k | represents the size of the kth feature map, i and j are the position coordinates on the feature map. For each position coordinate of the six output feature maps, according to the corresponding AR given in Table 1, 4, 6, 6, 6, 4, and 4 prior boxes of different sizes and ARs are applied respectively, and the total number of prior boxes is 8732.

[0087] Table 1 Comparison of detection results before and after changing the prior frame AR

[0088]

[0089] It can be seen from Table 1 that the mAP@.5 index of MSSD is 3.7% higher than that of SSD. This shows that the prior box obtained from data statistics in the present invention can improve the detection performance of SSD to a certain extent, and the operation of fine-tuning AR has a positive effect on the model.

[0090] Among all 8732 prior frames, a single detection frame corresponding to each target is determined, so non-maximum suppression is used to screen it. The steps are as follows:

[0091] (1) Arrange all prior boxes according to their confidence scores and find the prior box with the highest score.

[0092] (2) Set the IoU threshold to 0.5, traverse other prior boxes, and delete them if their IoU with the prior box with the highest score determined in (1) is greater than 0.5.

[0093] (3) Select the box with the highest score from the remaining boxes and repeat steps (1) and (2) until there are no boxes left to delete.

[0094] Furthermore, the settings for the SEMSSD target detection model training process are as follows: the optimizer uses the stochastic gradient descent method, and trains for a total of 300 generations. In the first 50 generations, some weights of the backbone network are frozen and only some weights are updated. After 50 generations, all weights are unfrozen and updated. The learning rate decreases linearly with the increase of training generations. The initial learning rate is 2×10-3 and the final learning rate is 1×10-4. During the training process, 8 images are input in each batch. When all training set images have completed one training, the current generation training is completed. It is stipulated that after each training, the network effect is tested using the validation set images.

[0095] In order to objectively reflect the improvement of the ship target detection effect in the sea fog environment, mAP is used as an indicator to evaluate the detection accuracy, and its calculation process is as follows:

[0096] S41: IoU (intersection over union), which is the ratio of the intersection and union of the area of ​​the box predicted by the SSD network and the area of ​​the real box. The larger the IoU, the closer the predicted box is to the actual box. An IoU greater than the threshold of 0.5 indicates that the prediction is correct. The specific formula is as follows:

[0097]

[0098] Among them: B p With B gt They are the predicted box and the true box respectively.

[0099] S42:AP (Average Precision).

[0100] Under a given IoU threshold, the recall rate R (Recall) and the precision P (Precision) can be determined accordingly. For each ship type, the corresponding PR curve can be drawn. In this case, AP is defined as the area enclosed by the PR curve:

[0101]

[0102] 4.3. mAP (mean Average Precision). mAP reflects the average AP of all target categories, and is expressed in the formula as follows:

[0103]

[0104] Where is the AP value of the ith category, and C is the total number of categories. The evaluation index selected by the present invention is the mAP score when the IoU threshold is 0.5, denoted as mAP@.5.

[0105] A deep learning-based marine ship target detection device in a sea fog environment, comprising:

[0106] Acquisition module: used to acquire the image to be processed containing rain and fog;

[0107] Rain and fog preprocessing module: used to perform rain and fog removal preprocessing on the image to be processed containing rain and fog based on the GCANet deep learning network to obtain the image to be processed after removing the rain and fog;

[0108] Recognition module: It is used to use the trained SEMSSD target detection model to identify the type and location of the maritime ship targets in the image to be processed after removing rain and fog.

[0109] In order to illustrate the effectiveness and superiority of the model proposed in the present invention, the following 7 groups of comparative experiments were carried out, and the experimental model is described as follows:

[0110] (1) G-SSD: The original SSD network trained on images dehazed using the GCANet module;

[0111] (2) MSSD: SSD network with prior box adjusted according to MVDD13 dataset;

[0112] (3) SESSD: embedding SE module in the convolutional layer of SSD network;

[0113] (4) SE-MSSD: embedding the SE module in the MSSD convolutional layer;

[0114] (5) G-MSSD: Use the GCANet module to train the MSSD network after dehazing the image;

[0115] (6) G-SESSD: Use the GCANet module to train the SESSD network after dehazing the images;

[0116] (7) G-SEMSSD: Use the GCANet module to train the SE-MSSD network after defogging images;

[0117] The experiment also uses mAP@.5 as the evaluation indicator. The results are shown in the following table:

[0118] Table 2 Comparison of detection accuracy of different models

[0119]

[0120] Compared with the original SSD, the mAP of the G-SSD, MSSD and SESSD models increased by 1.00%, 3.70% and 3.96% respectively. It can be seen that these three technologies can improve the detection performance of the SSD model to a certain extent. Specifically, the detection performance of the MSSD model has been significantly improved due to the application of the statistical characteristics of the data to determine a reasonable default box. In addition, G-MSSD is 3.1% higher than the mAP of G-SSD, which further illustrates that adjusting the aspect ratio of the prior box is beneficial to detection. It can be seen from Table 2 that G-SESSD has a more significant advantage over the G-SSD model, and the SE module can effectively improve the detection performance. Although G-SESSD is slightly inferior to SESSD and SE-MSSD is slightly inferior to SE-MSSD, the algorithm G-SEMSSD of the present invention has achieved a more obvious advantage.

[0121] For each ship category in the MVDD13 dataset, the effects of different models are shown in Figure 5. Figure 5 (a) is the test result diagram of various models for fishing boats, (b) is the test result diagram of various models for sailboats; (c) is the test result diagram of various models for fireboats; (d) is the test result diagram of various models for cargo ships; (e) is the test result diagram of various models for container ships; (f) is the test result diagram of various models for warships; (g) is the test result diagram of various models for passenger ships; (h) is the test result diagram of various models for submarines; (i) is the test result diagram of various models for bulk carriers; (j) is the test result diagram of various models for tugboats; (k) is the test result diagram of various models for drilling ships; (l) is the test result diagram of various models for tankers; (m) is the test result diagram of various models for cruise ships.

[0122] In addition to container ships and warships, the G-SEMSSD model can balance precision and recall well on other ship types, especially on fireboats, sailboats and fishing boats, which far outperforms other models. The ship types in the dataset have significant shape characteristics, and most ships are wider than they are tall. The aspect ratio reset by the present invention is more conducive to accurate detection of the algorithm.

[0123] In summary, the G-SEMSSD model proposed in this invention can achieve higher detection accuracy in sea fog environment and can be applied to ship detection tasks in sea fog environment.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0125] 1. Wu D, Zhu Q S. The latest research progress of image dehazing. ActaAutomatica Sinica, 2015, 41(2): 221-239.

[0126] 2.Gan JJ, Xiao C X. Fast image dehazing based on accurate scatteringmap. Journal of Image and Graphics, 2013, 18(5):583-590.

[0127] 3.He K, Sun J, Tang X, et al. Single image haze removal using darkchannel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.

[0128] 4.Chen D,He M,Fan Q,Liao J,Zhang L,Hou D,Yuan L,Hua G.Gated contextaggregation network for image dehazing and deraining.2019IEEE WinterConference on Applications of Computer Vision,2019,1375-1383.

[0129] 5.Cai B,Xu X,Jia K,Qing C,Tao D.Dehazenet:An end-to-end system forsingle image haze removal.IEEE Transactions on Image Processing,2016,25(11):5187-5198.

[0130] 6.Li B,Peng X,Wang Z,Xu J,Feng D.An all-in-one network for dehazingand beyond.arXiv preprint arXiv,2017,1707.06543.

[0131] 7.Zhang H,Patel V M.Densely connected pyramid dehazing network.In:Proceedings of the IEEE Conference on Computer Vision and PatternRecognition,2018,3194-3203.

[0132] 8.Li C,Guo C,Guo J,Han P,Fu H,Cong R.PDR-Net:Perception-inspiredsingle image dehazing network with refinement.IEEE Transactions onMultimedia,2020,22(3):704-716.

[0133] 9.Chen X,Lu Y,Wu Z,Yu J,Wen L.Reveal of domain effect:How visualrestoration contributes to object detection in aquatic scenes.arXiv preprintarXiv,2020,2003.01913.

[0134] 10.Isaza C,Anaya K,Fuentes-Silva C,et al.Dynamic set point model fordriver alert state using digital image processing.Multimedia Tools andApplications,2019,78(14):19543-19563.

[0135] 11.Sun W F,Dai Y S,Ji Y G,et al.New ideas for HFSWR maritime targettracking.Marine Sciences,2017,41(6):144-149.

[0136] 12.Zhang H,Mao H,Qiu D.Feature extraction for the stored-grain insectdetection system based on image recognition technology.Transactions of theChinese Society of Agricultural Engineering,2009,25(2):126-130.

[0137] 13.Huang H,Zhou H,Yang X,Zhang L,Qi L,Zang A Y.Faster R-CNN formarine organisms detection and recognition using dataaugmentation.Neurocomputing,2019,337:372-384.

[0138] 14.He K,Zhang X,Ren S,Sun J.Spatial pyramid pooling in deepconvolutional networks for visual recognition.IEEE Transactions on PatternAnalysis and Machine Intelligence,2015,37(9):1904-1916.

[0139] 15.Ren S,He K,Girshick R,Sun J.Faster R-CNN:Towards real-time objectdetection with region proposal networks.IEEE Transactions on Pattern Analysisand Machine Intelligence,2017,39(6):1137-1149.

[0140] 16.Redmon J,Farhadi A.Yolo9000:Better,faster,stronger.In:2017 IEEEConference on Computer Vision and Pattern Recognition,2017,6517-6525.

[0141] 17.Liu W,Anguelov D,Erhan D,Szegedy C,Berg A C.SSD:Single shotmultibox detector,in European Conference on Computer Vision,2015,21-37.

[0142] 18.Redmon J,Farhadi A.Yolov3.An incremental improvement,arXivpreprint arXiv,2018,1804.02767.

[0143] 19.Bochkovskiy A,Wang C Y,Liao H Y.Yolov4:Optimal speed and accuracyof object detection.arXiv preprint,2020,2004.10934.

Claims

1. A method for detecting marine ship targets based on deep learning in a sea fog environment, characterized by: The following steps are involved: Obtain an image to be processed containing rain and fog; Based on the GCANet deep learning network, the image to be processed containing rain and fog is preprocessed to remove rain and fog, and the image to be processed after removing rain and fog is obtained; The SE-MSSD target detection model is used to identify the types and locations of marine ship targets in the images to be processed after rain and fog are removed. The process of performing rain and fog removal preprocessing on the image to be processed containing rain and fog based on the GCANet deep learning network to obtain the image to be processed after removing the rain and fog is as follows: For the image to be processed containing rain and fog, two convolution kernels with sizes of , a first convolutional normalization module with a stride of 1 and a convolutional kernel size of , the second convolutional normalization module with a stride of 2 encodes the image into a feature map; The feature map is passed through smooth hole residual blocks with different hole rates in sequence to obtain image features at different levels; The output feature map can be sequentially passed through 7 smooth hole residual blocks with hole rates of 2, 2, 2, 4, 4, 4, and 1 to obtain image features at different levels; Features at different levels are fused through a gated fusion sub-network to learn the weights of low, medium, and high-level features; Get the feature map; The feature map is passed through a deconvolution layer and upsampled to the original resolution, and then converted back to the image space through two convolution layers to obtain the fog residual of the marine ship target; The step size of the deconvolution layer is 1 / 2; Add the fog residual to the original foggy input image to get the image after rain and fog removal; The SE-MSSD target detection model uses the VGG network as the backbone network, and a channel attention module is connected after the third convolution layer of the fourth convolution block of the VGG network. The VGG network also includes a fully connected layer, three groups of convolutional layers and a group of average pooling layers; The channel attention module, the fully connected layer, three groups of convolutional layers and a group of average pooling layers are sequentially connected; three groups of convolutional layers are sequentially added after the fully connected layer; the three groups of convolutional layers include a first convolutional layer, a second convolutional layer and a third convolutional layer; A set of average pooling layers are connected in series after the third convolutional layer; The channel attention module SE is introduced in the VGG basic convolutional layer. Its input feature map is denoted as P1. The P1 feature map is divided into two paths, one of which is not processed at all and the output is denoted as P2. The other path passes through global pooling, fully connected layer, ReLU function, fully connected layer, Sigmoid function, and the output is denoted as P3. P2 and P3 are added to get the output P4 of the SE module, which is consistent with the dimension of the input layer P1.

2. According to the method for detecting marine ship targets based on deep learning in a sea fog environment as described in claim 1, it is characterized by: It also includes adding GCANet as the image defogging preprocessing process in the front end of the ship target detection network, namely the SE-MSSD target detection model. The input image size remains unchanged before and after defogging. The defogging image is scaled to 300×300 and then input into the backbone network VGG of the SSD model. The SE module is connected after the Conv4_3 layer to obtain a feature map F1 with a shape of 38×38×512; F1 has two branches, one of which is normalized and output, denoted as M1; the other passes through the FC7 layer of VGG to obtain a 19×19×1024 feature map M2; M2 passes through the convolution layer to obtain a 10×10×512 feature map M3; M3 passes through the convolution layer to obtain a 5×5×256 feature map M4; M4 passes through the convolution layer to obtain a 3×3×256 feature map M5; After average pooling, M5 obtains a 1×1×256 feature map M6; At this time, the network has a total of six output feature maps M1, M2, M3, M4, M5, and M6. The SSD model uses the prior box to determine the target position, sets prior boxes of different sizes and values ​​for the above feature maps, obtains the prior box confidence and position offset, and then performs non-maximum suppression to obtain the predicted box closest to the actual position.

3. The method for detecting marine ship targets based on deep learning in a sea fog environment according to claim 1, characterized in that: The SE-MSSD target detection model uses a priori boxes to predict the target position. The priori boxes are determined as follows: Counting the shape features of the marine ship targets in the image to be processed; The statistical features of the shape of the prior frame are adjusted based on the target features of the marine ship target; the statistical features include aspect ratio information of the prior frame.

4. The method for detecting marine ship targets based on deep learning in a sea fog environment according to claim 1, characterized in that: The aspect ratios of the 6 output layers of the SE-MSSD target detection model are: , , , , , ; The formula for calculating the prior frame ratio is: in , , respectively represent the minimum and maximum proportions of the prior frame to the feature map area. Since the detected targets have different , according to the different settings in the six different output feature maps ; The width of each prior box With height Use the following two formulas to calculate: in: is the size of the original image; The center coordinates of each prior box are ,in, Indicates The size of the feature map, is the position coordinate on the feature map. For each position coordinate of the six output feature maps, , respectively apply 4, 6, 6, 6, 4, 4 different sizes The total number of prior boxes is 8732.

Images