An Improved Efficientdet-Based Water Surface Target Detection Method and System

By improving the backbone feature extraction network and feature fusion network of the Efficientdet model, combining the attention mechanism and the context Transformer module, the activation function is optimized, and the problems of large calculation volume and low accuracy in surface object detection are solved, and efficient and accurate surface object detection is achieved.

CN116311092BActive Publication Date: 2025-08-01JIANGSU UNIV OF SCI & TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310071394.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-02-07
Publication Date
2025-08-01
Estimated Expiration
2043-02-07

AI Technical Summary

Technical Problem

Existing deep learning object detection models such as the YOLO series are large in water surface object detection and have high memory requirements, making them difficult to effectively apply on water surface mobile platforms, and have high detection accuracy and leakage detection rate, which cannot meet the needs of complex time-varying water surface environments.

Method used

Based on the Efficientdet model, the backbone feature extraction network is improved by introducing the CA module, and it is improved to CA-MBConv, combined with the Cot module, the feature fusion network is improved to Cot-BIFPN, and the activation functions of ClassNet and BoxNet are replaced as H-Swish, which optimizes the model calculation amount and detection accuracy.

Benefits of technology

In complex water surface environments, the detection accuracy is significantly improved and the leakage detection rate is reduced. The improved model has increased mAP by nearly 17% on challenging water surface datasets, and the leakage detection rate has decreased significantly.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116311092B_ABST
    Figure CN116311092B_ABST
Patent Text Reader

Abstract

The present invention discloses a water surface target detection method and system based on improved Efficientdet, including: introducing an attention mechanism CA into the main module MBConv of the backbone feature extraction network Efficientnet, and improving MBConv to CA-MBConv to improve the accuracy of the network model in detecting water surface targets; introducing a Cot module into the main convolution operation of the feature fusion network BIFPN, and improving the feature fusion network to Cot-BIFPN, so that the feature fusion network can more fully learn the context information of the feature map and enhance the expression ability of the output features; changing the activation function in the head Box and Class networks to H-Swish to reduce the model calculation amount and optimize the model training. The present invention greatly improves the object detection accuracy and effectively reduces the model missed detection rate for target detection in complex time-varying water surface environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of surface target detection, and specifically relates to a surface target detection method and system based on an improved EfficientDet. Background Art

[0002] In the current world context, marine security is a key direction of national security and a top priority for national defense and people's livelihood security. Detecting important information in maritime target images belongs to the technical field of surface target detection. The development of surface target detection technology has far-reaching significance for the management of military bays and ports and the supervision of civilian fishing boats, etc., and is an important way to achieve marine security. Surface target detection systems can be divided into long-distance systems and short-distance systems. The long-distance systems mainly include radar and infrared target detection, and the short-distance systems mainly include visual acquisition and detection. Since the long-distance systems have a relatively long operating range and the amount of image information obtained is small, they can no longer meet the needs of national defense security. Therefore, the short-distance systems have become a popular choice for current surface target detection technology.

[0003] In recent years, with the rapid development of deep learning in the context of artificial intelligence, it is urgent to apply it to short-distance systems to improve the accuracy of surface target detection. The ocean has more severe conditions than land, with various factors such as water surface refraction, changeable weather, and unstable equipment. Therefore, the information processing ability and information detection accuracy of the surface target detection model determine whether surface targets can be accurately detected in a complex and time-varying water surface environment.

[0004] Currently, deep learning target detection models emerge in an endless stream. Among the single-stage detection models, the most classic is the YOLO series. Chinese Patent Application (CN109492561A) discloses an optical remote sensing image ship detection method based on an improved YOLOV2 model, which improves the detection ability of YOLOV2 for ships by modifying the loss function. "Residual YOLOX-based Ship Object Detection Method" published by Liu et al. in "2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE)" improves the ability to detect ships by introducing a residual structure and a loss function to improve YOLOX. The YOLO series models used by these methods have a large amount of calculation and high requirements for device memory, and it is very challenging to apply them to surface mobile platforms. Therefore, it is necessary to explore excellent models with less calculation for target detection on surface mobile platforms. Summary of the Invention

[0005] The object of the present invention is to overcome the defects of the prior art and provide an improved Efficientdet water surface target detection method and system thereof. An Efficientdet model with a simple structure and small computational complexity is adopted, and Efficientdet-D0 is used as the initial model. The accuracy of the Efficientdet-D0 model in water surface target detection and the missed detection rate of the model are improved, so that higher detection accuracy and lower missed detection rate can be obtained when detecting targets in a complex and time-varying water surface environment.

[0006] To achieve the above technical objectives, the present invention adopts the following technical solutions.

[0007] An improved Efficientdet-based water surface target detection method of the present invention includes the following steps:

[0008] Step 1: Store the challenging water surface dataset WSODD in VOC format in the algorithm directory, and divide the training set, test set, and validation set according to the ratio of 8:1:1. The divided image information is stored in.txt format.

[0009] Step 2: Input the training set images into the backbone feature extraction network Efficientnet after introducing the CA module for training, and extract the P3-P5 level features therefrom; among them, an attention mechanism CA is introduced into each layer of MBConv in the 2-8 layers of the backbone feature extraction network Efficientnet to obtain CA-MBConv.

[0010] Step 3: Generate P3_in to P5_in from the P3-P5 level features output by the backbone feature extraction network through 1×1 convolution, and then obtain P6_in and P7_in by downsampling P5_in twice; then input P3_in to P7_in into the feature fusion network Cot-BIFPN to enable the features to undergo three top-down and bottom-up feature fusions; among them, a Cot module is introduced into the convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network BIFPN to obtain Cot-BIFPN.

[0011] Step 4: Transmit the five effective feature layers P3_out to P7_out output by the feature fusion network Cot-BIFPN to ClassNet+BoxNet to obtain the final detection result; among them, the activation functions in ClassNet and BoxNet are replaced with H-Swish.

[0012] Step 5: Train the improved Efficientdet model using the WSODD training set. The epoch parameter takes values at intervals of 50 from 300 to 650. The weight files of each training result are stored in the.pth format. Calculate the mAP for all weight files using the WSODD test set. Compare the relationship between the epoch value and the mAP to obtain the optimal epoch value. Among them, when epoch = 500, the mAP is the highest, reaching 48.00%. Therefore, 500 is selected as the epoch value of the improved model.

[0013] Step 6: Select two pictures taken in a real water surface environment from the WSODD validation set, input them into the original Efficientdet and the improved Efficientdet models respectively for detection, and compare the detection results.

[0014] Specifically, in Step 2, input the training set pictures into the backbone feature extraction network Efficientnet after introducing the CA module for training, and extract P3 - P5 level features from it. Introduce the attention mechanism CA into each layer of MBConv from the 2nd to the 8th layer of the backbone feature extraction network to obtain CA - MBConv. The implementation steps are as follows:

[0015] S2.1 Upscale the input feature map through a 1×1 convolution to increase the number of input channels.

[0016] S2.2 Input the upscaled feature map into the depthwise separable convolution DepthwiseConv.

[0017] S2.2.1 Group the feature map, and perform 3×3 convolution operations on each group to collect the spatial features of each channel.

[0018] S2.2.2 Input the feature maps of each group after convolution operations into a 1×1 convolution to construct a linear combination of the 3×3 convolutions of each group and collect the features of each point.

[0019] S2.3 Input the feature map after depthwise separable convolution into the CA module to obtain the correlation information between channels and the position information of spatial feature points:

[0020] S2.3.1 First, perform coordinate information embedding on the input feature map.

[0021] S2.3.1.1 Input the feature map x of C×H×W after depthwise separable convolution, where C, H, and W represent the number of channels, the height of the feature map, and the width of the feature map respectively.

[0022] S2.3.1.2 Pass the feature map x through a pooling kernel of size (H, 1) or (1, W) to encode the information of each channel separately along the horizontal and vertical directions. The output of the c-th channel with height h is:

[0023]

[0024] The output of the c-th channel with width w is:

[0025]

[0026] where x c represents the feature map of the c-th channel; i, j represent loop variables;

[0027] S2.3.1.3 Aggregate the encoded information along the horizontal and vertical directions to obtain a pair of feature maps with direction awareness, acquire the global receptive field, and encode the precise position information to improve the accuracy of the network in detecting the target of interest;

[0028] S2.3.2 Then input the feature map encoded with coordinate information into the Coordinate attention generation;

[0029] S2.3.2.1 Concatenate and encode the feature maps y h and y w output in the horizontal and vertical directions, perform operations using 1×1 convolution, denoted as F1, and the output f represents the intermediate feature map information in the horizontal and vertical directions. The expression is as follows:

[0030] f = δ(F1([y h , y w )) (3)

[0031] where δ represents the activation function Relu;

[0032] S2.3.2.2 Divide the output f into f h and f w along the horizontal and vertical directions, and then pass through 1×1 convolution, denoted as F h and F w , and output the feature map information g h and g w with the same number of channels as the input x. The expression is:

[0033] g h = σ(F h (f h )) (4)

[0034] g w = σ(F w (fw )) (5)

[0035] Among them, σ represents the Sigmoid activation function;

[0036] S2.3.2.3 Expand g h and g w and use them as the weights of attention respectively;

[0037] S2.3.3 Output a feature map of C×H×W, and its expression:

[0038]

[0039] S2.4 Input the feature map output by the CA module into a 1×1 convolution for dimensionality reduction, maintaining the consistency of the input and output channels.

[0040] Specifically, in the third step, in the convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network, introducing the Cot module, the implementation steps of the Cot-BIFPN include:

[0041] S3.1 Pass the P3-P5 level features extracted by the improved backbone feature extraction network through a 1×1 convolution to generate P3_in to P5_in, and then pass P5_in through two downsamplings to obtain P6_in and P7_in;

[0042] S3.2 Pass P7_in through the Cot module, combine it with P6_in to obtain P6_td, pass P6_td through the Cot module, combine it with P5_in to obtain P5_td, and so on until combined with P3_in to obtain P3_out;

[0043] S3.3 Pass P3_out through the Cot module, combine it with P4_td and P4_in2 to obtain P4_out, and so on until combined with P6_in2 and P6_td to obtain P6_out, and pass P6_out through the Cot module, combine it with P7_in to obtain P7_out.

[0044] Furthermore, the process of passing through the Cot module is as follows:

[0045] S3.2.1. Input a feature map X of size C×H×W, where C, H, and W represent the number of channels, the height of the feature map, and the width of the feature map respectively;

[0046] S3.2.2. Perform context encoding on all adjacent keys within all k×k grids in the feature map X in space to obtain the context key K 1 , K 1 contains the static context information between local adjacent keys, where K = X,

[0047] S3.2.3. Concatenate the context key K 1 and Q to obtain a feature map of H×W×2C, where Q = X;

[0048] S3.2.4. Input the feature map of H×W×2C into two consecutive 1×1 convolutions with a Relu activation function and without an activation function, denoted by W θ and W δ respectively, to obtain a local attention matrix A for learning the combination of query features and context key features:

[0049] A = [K 1 ,Q]W θ W δ (7)

[0050] S3.2.5. Aggregate the local attention matrix A with V to obtain a feature map K containing dynamic feature interactions between inputs after weighting, where V = XW 2 , v , indicating the matrix multiplication operation:

[0051]

[0052] S3.2.6. Fuse the static context K 1 with the dynamic context K 2 and output a feature map Y of size H×W×C:

[0053] Y = K 1 +K 2 (9)

[0054] Specifically, in the above step four, replace the activation functions in ClassNet and BoxNet with H-Swish, and the expression of this function is:

[0055] H-Swish(x) = x×Relu6(x + 3)÷6 (10)

[0056] where,

[0057] Relu6 = min(Relu,6) (11)

[0058] A water surface target detection system based on the improved Efficientdet of the present invention is used to execute the above-mentioned water surface target detection method based on the improved Efficientdet during implementation, and includes an improved Efficientdet model unit.

[0059] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0060] 1. In each MBConv of the 2 - 8 layers of the backbone feature extraction network Efficientnet of the present invention, the attention mechanism CA (Coordinate Attention) is introduced, and the MBConv is improved to CA - MBConv. The CA mechanism takes into account both the relationship between channels and the spatial position information, and the output position information is beneficial to the generation of spatial - selective attention maps.

[0061] 2. By introducing the Cot (Contextual Transformer) module into the main convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network BIFPN, the feature fusion network is improved to Cot - BIFPN. Cot - BIFPN can learn the context information of the feature map more fully, thereby achieving the purpose of improving the output feature expression ability.

[0062] 3. In the present invention, the activation function in the head Box and Class networks is changed to H - Swish, which greatly reduces the problems of large computational amount, complex calculation and derivative time of the Swish activation function.

[0063] 4. Considering the complex situation of water surface object detection, the present invention is trained and verified on the challenging water surface dataset WSODD (WaterSurface Object Detection Dataset). Compared with the original model, the mAP of the improved model is increased by nearly 17%, and the missed detection rate is also significantly decreased. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a water surface object detection method based on the improved Efficientdet of the present invention.

[0065] Figure 2 It is a schematic diagram of the network structure of CA - MBConv in an embodiment of the present invention.

[0066] Figure 3 It is a schematic diagram of the network structure of Cot - BIFPN in an embodiment of the present invention.

[0067] Figure 4 It is a flowchart of the introduced Cot module in an embodiment of the present invention.

[0068] Figure 5 It is a relationship diagram between the epoch parameter and mAP in an embodiment of the present invention.

[0069] Figure 6 is a comparison chart of the mAP of the original model of the prior art and the improved model mAP of an embodiment of the present invention. Among them, Figure 6a is the mAP chart of the original model; Figure 6b is the mAP chart of the improved model.

[0070] Figure 7 is a comparison chart of the miss detection rate of the original model of the prior art and the improved model of an embodiment of the present invention. Among them, Figure 7a is the miss detection rate chart of the original model; Figure 7b is the miss detection rate chart of the improved model.

[0071] Figure 8 is a comparison chart of the detection results of the prior art and the present invention. Among them, Figure 8a 、 Figure 8c are the detection result charts of the original model of the prior art; Figure 8b 、 Figure 8d are the detection result charts of the improved model of an embodiment of the present invention. Specific Embodiments

[0072] The present invention provides a water surface target detection method and system based on improved Efficientdet, which is applicable to water surface target detection, including: introducing an attention mechanism CA (Coordinate Attention) into the main module MBConv of the backbone feature extraction network Efficientnet, and improving MBConv to CA-MBConv to improve the accuracy of the network model in detecting water surface targets; introducing a Cot (Contextual Transformer) module into the main convolution operation of the feature fusion network BIFPN, and improving the feature fusion network to Cot-BIFPN, so that the feature fusion network can more fully learn the context information of the feature map and improve the expression ability of the output features; changing the activation function in the head Box and Class networks to H-Swish to reduce the model calculation amount and optimize model training; compared with the original model, the object detection accuracy of the improved model has been greatly improved and the model miss detection rate has decreased significantly.

[0073] The following further describes the present invention in detail with reference to the drawings.

[0074] Figure 1 is a flowchart of a water surface target detection method based on improved Efficientdet of the present invention. As Figure 1 shown, the method of this embodiment includes the following steps:

[0075] Step 1: Store the challenging water surface dataset WSODD in the algorithm directory in VOC format, and divide the training set, test set, and validation set according to the ratio of 8:1:1. The divided picture information is stored in.txt format;

[0076] Step 2: Input the training set images into the backbone feature extraction network Efficientnet after introducing the CA module for training, and extract P3-P5 level features from it. Among them, the attention mechanism CA is introduced into each layer of MBConv in layers 2-8 of the backbone feature extraction network Efficientnet to obtain CA-MBConv, thereby improving the network's ability to detect water surface targets. The network structure of CA-MBConv is as Figure 2 shown;

[0077] Step 3: Generate P3_in to P5_in from the P3-P5 level features output by the backbone feature extraction network through 1×1 convolution, and then obtain P6_in and P7_in by downsampling P5_in twice. Next, input P3_in to P7_in into the feature fusion network Cot-BIFPN to enable the features to undergo three top-down and bottom-up feature fusions. Among them, the Cot module is introduced into the main convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network BIFPN to obtain Cot-BIFPN. Cot-BIFPN can learn the context information of the feature map more fully, thereby enhancing the expression ability of the output features. The network structure of Cot-BIFPN is as Figure 3 shown;

[0078] Step 4: Transmit the five effective feature layers P3_out to P7_out output by the feature fusion network Cot-BIFPN to ClassNet+BoxNet to obtain the final detection result. Among them, the activation functions in ClassNet and BoxNet are replaced with H-Swish, and H-Swish has the advantages of small computational complexity and simple derivative calculation;

[0079] Step 5: Train the improved Efficientdet model with the WSODD training set, and the epoch parameter takes values at intervals of 50 from 300 to 650. The weight files of each training result are stored in the.pth format. Use the WSODD test set to calculate the mAP for all weight files, and compare the relationship between the epoch value and the mAP (as Figure 5 shown), to obtain the optimal epoch value. Among them, when epoch = 500, the mAP is the highest, reaching 48.00%. Therefore, 500 is selected as the epoch value of the improved model;

[0080] Step 6: Select two pictures taken in a real water surface environment from the WSODD validation set, input them into the original Efficientdet and the improved Efficientdet models respectively for detection, and compare the detection results. The detection results are shown in Figure 8. Among them, it can be seen from the comparison of the detection results that the improved model can identify more objects in the water surface, and the detection accuracy has also been improved to a certain extent.

[0081] In step 2, the training set pictures are input into the backbone feature extraction network Efficientnet after introducing the CA module for training, and the P3 - P5 level features are extracted from it. The attention mechanism CA is introduced into each layer of MBConv in the 2 - 8 layers of the backbone feature extraction network to obtain CA - MBConv. The implementation steps are as follows:

[0082] S2.1 Upscale the input feature map through a 1×1 convolution to increase the number of input channels.

[0083] S2.2 Input the upscaled feature map into the depthwise separable convolution (DepthwiseConv).

[0084] S2.2.1 First, group the feature map, and perform 3×3 convolution operations on each group to collect the spatial features of each channel.

[0085] S2.2.2 Then input the feature maps of each group after convolution operation into a 1×1 convolution to construct a linear combination of the 3×3 convolutions of each group to collect the features of each point.

[0086] S2.3 Input the feature map after depthwise separable convolution into the CA module to obtain the correlation information between channels and the position information of spatial feature points:

[0087] S2.3.1 First, perform coordinate information embedding on the input feature map.

[0088] S2.3.1.1 Input the feature map x of C×H×W after depthwise separable convolution, where C, H, and W represent the number of channels, the height of the feature map, and the width of the feature map respectively.

[0089] S2.3.1.2 Pass the feature map x through a pooling kernel of size (H,1) or (1,W) to encode the information of each channel along the horizontal and vertical directions respectively. The output of the c - th channel with height h is:

[0090]

[0091] The output of the c - th channel with width w is:

[0092]

[0093] Among them, x c represents the feature map of the c-th channel; i and j represent loop variables;

[0094] S2.3.1.3 Aggregate features of the encoded information in the horizontal and vertical directions, output a pair of feature maps with direction perception, obtain the global receptive field and encode the accurate position information to improve the accuracy of the network in detecting the target of interest;

[0095] S2.3.2 Then input the feature map encoded with coordinate information into the Coordinate attention generation;

[0096] S2.3.2.1 Concatenate and encode the feature maps y h and y w output in the horizontal and vertical directions, perform operations using 1×1 convolution, denoted by F1, and output f representing the intermediate feature map information in the horizontal and vertical directions. The specific expression is as follows:

[0097] f = δ(F1([y h , y w )) (3)

[0098] Among them, δ represents the activation function Relu.

[0099] S2.3.2.2 Divide the output f into f h and f w along the horizontal and vertical directions, and then pass through 1×1 convolution, denoted by F h and F w , and output the feature map information g h and g w with the same number of channels as the input x. The specific expression is as follows:

[0100] g h = σ(F h (f h )) (4)

[0101] g w = σ(F w (f w )) (5)

[0102] Among them, σ represents the activation function Sigmoid.

[0103] S2.3.2.3 Expand g h and g w respectively as the weights of the attention;

[0104] S2.3.3 Outputs a feature map of C×H×W, and the specific expression is as follows:

[0105]

[0106] S2.4 Inputs the feature map output by the CA module into a 1×1 convolution for dimensionality reduction, maintaining the consistency of the input and output channels.

[0107] In the third step described above, the Cot module is introduced in the main convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network. The specific implementation steps of the Cot-BIFPN are as follows:

[0108] S3.1 First, the P3-P5 level features extracted by the improved backbone feature extraction network are convolved through a 1×1 convolution to generate P3_in to P5_in, and then P5_in is downsampled twice to obtain P6_in and P7_in;

[0109] S3.2 Pass P7_in through the Cot module, combine it with P6_in to obtain P6_td, pass P6_td through the Cot module, combine it with P5_in to obtain P5_td, and so on until combined with P3_in to obtain P3_out;

[0110] S3.3 Pass P3_out through the Cot module, combine it with P4_td and P4_in2 to obtain P4_out, and so on until combined with P6_in2 and P6_td to obtain P6_out. Pass P6_out through the Cot module, combine it with P7_in to obtain P7_out.

[0111] Further, the implementation steps of the Cot module passed through in S3.2 and S3.3 are as follows (as Figure 4 shown):

[0112] S1. Input a feature map X of size C×H×W, where C, H, and W represent the number of channels, the height of the feature map, and the width of the feature map respectively;

[0113] S2. Contextually encode all adjacent keys within all k×k grids in the feature map X in space to obtain the context key K 1 , K 1 contains the static context information between local adjacent keys, where K = X,

[0114] S3. Concatenate (Concat) the context key K 1 and Q to obtain a feature map of H×W×2C, where Q = X;

[0115] S4. Input the feature map of H×W×2C into two consecutive 1×1 convolutions (one with the Relu activation function and the other without the activation function), denoted by W θ and W δ respectively, to obtain the local attention matrix A for combined learning of the query feature and the context key feature:

[0116] A = [K 1 , Q]W θ W δ (7)

[0117] S5. Aggregate the local attention matrix A with V to obtain the feature map K 2 that contains dynamic feature interactions between the inputs after weighting, where V = XW v , representing the matrix multiplication operation:

[0118]

[0119] S6. Fuse the static context K 1 with the dynamic context K 2 , and output the feature map Y of size H×W×C.

[0120] Y = K 1 + K 2 (9)

[0121] In the above-mentioned step four, replace it with the computationally less expensive H-Swish, and the expression of this function is as follows:

[0122] H-Swish(x) = x × Relu6(x + 3) ÷ 6 (10)

[0123] where

[0124] Relu6 = min(Relu, 6) (11)

[0125] A water surface target detection system based on the improved Efficientdet of the present invention is used to execute the above-mentioned water surface target detection method based on the improved Efficientdet during implementation, and includes an improved Efficientdet model unit.

[0126] In summary, the present invention introduces multiple methods for optimizing the network model to improve the model and achieves an ideal effect. Compared with the model before improvement, when the epoch is set to 500, the mAP value of the improved model reaches 48.00%, which is about 17 percentage points higher than the original model (as shown in Figure 6), and the overall false detection rate of water surface targets decreases significantly (as shown in Figure 7).

[0127] In summary, considering the complex time-varying environment of the water surface, in order to improve the detection accuracy of the detection model for small targets and difficult-to-detect targets on the water surface, the attention mechanism CA (Coordinate Attention) is introduced into each layer of MBConv in the 2-8 layers of the backbone feature extraction network Efficientnet, and MBConv is improved to CA-MBConv; in order to enable the feature fusion network BIFPN to learn more context information of the output feature map of the backbone feature extraction network and enhance the expression ability of the output features, the Cot (Contextual Transformer) module is introduced in the main convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network, and the feature fusion network is improved to Cot-BIFPN; in order to reduce the model calculation amount and optimize the model training, the activation function in the head Box and Class networks is replaced with H-Swish; considering the complex situation of water surface target detection, the improved model is trained and verified on the challenging water surface dataset WSODD (Water Surface Object Detection Dataset), and the improved model has a significant improvement in target detection accuracy, and the miss detection rate of the model also drops significantly.

Claims

1. An improved Efficientdet-based water surface target detection method, characterized in that, It includes the following steps: Step 1: Store the challenging water surface dataset WSODD in the algorithm directory in VOC format, and divide the training set, test set, and validation set according to the ratio of 8:1:

1. The divided image information is stored in.txt format. Step 2: Input the training set images into the backbone feature extraction network Efficientnet after introducing the CA module for training, and extract P3 - P5 level features from it. Among them, introduce the attention mechanism CA into each MBConv in layers 2 - 8 of the backbone feature extraction network Efficientnet to obtain CA - MBConv. Step 3: Generate P3_in to P5_in from the P3 - P5 level features output by the backbone feature extraction network through 1×1 convolution, and then obtain P6_in and P7_in by performing two downsamplings on P5_in. Then input P3_in to P7_in into the feature fusion network Cot - BIFPN to enable the features to undergo three top - down and bottom - up feature fusions. Among them, introduce the Cot module into the convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network BIFPN to obtain Cot - BIFPN. Step 4: Transmit the five effective feature layers P3_out to P7_out output by the feature fusion network Cot - BIFPN to ClassNet + BoxNet to obtain the final detection result. Among them, replace the activation functions in ClassNet and BoxNet with H - Swish. Step 5: Train the improved Efficientdet model with the WSODD training set, and the epoch parameter takes values at intervals of 50 from 300 - 650. The weight files of each training result are stored in.pth format, and calculate the mAP for all weight files using the WSODD test set. Compare the relationship between the epoch value and mAP to obtain the optimal epoch value. Among them, when epoch = 500, the mAP is the highest, reaching 48.00%, so 500 is selected as the epoch value of the improved model. Step 6: Select two pictures taken in a real water surface environment from the WSODD validation set, input them into the original Efficientdet and the improved Efficientdet models respectively for detection, and compare the detection results.

2. The water surface target detection method based on the improved Efficientdet according to claim 1, characterized in that, In the said Step 2, input the training set images into the backbone feature extraction network Efficientnet after introducing the CA module for training, and extract P3 - P5 level features from it. Introduce the attention mechanism CA into each MBConv in layers 2 - 8 of the backbone feature extraction network to obtain CA - MBConv, and its implementation steps are as follows: S2.1: Upscale the input feature map through 1×1 convolution to increase the number of input channels. S2.2: Input the upscaled feature map into the depth - separable convolution DepthwiseConv. S2.2.1 Group the feature maps, perform 3×3 convolution operations on each group, and collect the spatial features of each channel; S2.2.2 Input the feature maps of each group after the convolution operation into a 1×1 convolution to construct a linear combination of the 3×3 convolutions of each group, and collect the features of each point; S2.3 Input the feature maps after depthwise separable convolution into the CA module to obtain the correlation information between channels and the position information of spatial feature points: S2.3.1 First, perform coordinate information embedding on the input feature maps; S2.3.1.1 Input the feature map x of size C×H×W after depthwise separable convolution, where C, H, and W represent the number of channels, the height of the feature map, and the width of the feature map respectively; S2.3.1.2 Pass the feature map x through a pooling kernel of size (H,1) or (1,W) to encode the information of each channel along the horizontal and vertical directions respectively. The output of the c-th channel with height h is: The output of the c-th channel with width w is: where x c represents the feature map of the c-th channel; i, j represent loop variables; S2.3.1.3 Aggregate the features of the encoded information in the horizontal and vertical directions, output a pair of feature maps with direction awareness, obtain the global receptive field, and encode the precise position information to improve the accuracy of the network for detecting targets of interest; S2.3.2 Then input the feature maps after coordinate information encoding into coordinate attention generation; S2.3.2.1 Output feature maps y in horizontal and vertical directions h and y w Perform tandem encoding and use 1×1 convolution to operate, represented by F1, and the output f represents the intermediate feature map information in the horizontal and vertical directions. The expression is as follows: f = δ(F1([y h , y w )) (3) Among them, δ represents the activation function Relu; S2.3.2.2 Divide the output f into f h and f w along the horizontal and vertical directions, then perform 1×1 convolution, and use F h and F w to represent, and output the feature map information g h and g w with the same number of channels as the input x. The expression is: g h = σ(F h (f h )) (4) g w = σ(F w (f w )) (5) Among them, σ represents the activation function Sigmoid; S2.3.2.3 Expand g h and g w respectively as the weights of attention; S2.3.3 Output a feature map of size C×H×W, and its expression: S2.4 Input the feature maps output by the CA module into a 1×1 convolution for dimensionality reduction to maintain the consistency of the input and output channels.

3. The method for detecting water surface targets based on improved Efficientdet according to claim 1, wherein In the third step described above, the Cot module is introduced in the convolution operations Conv7_down to Conv4_down and Conv3_up to Conv6_up of the feature fusion network. The implementation steps of Cot-BIFPN include: S3.1 Pass the P3 - P5 level features extracted by the improved backbone feature extraction network through a 1×1 convolution to generate P3_in to P5_in, and then downsample P5_in twice to obtain P6_in and P7_in; S3.2 Pass P7_in through the Cot module, combine it with P6_in to obtain P6_td, pass P6_td through the Cot module, combine it with P5_in to obtain P5_td, and so on until combined with P3_in to obtain P3_out; S3.3 Pass P3_out through the Cot module, combine it with P4_td and P4_in2 to obtain P4_out, and so on until combined with P6_in2 and P6_td to obtain P6_out. Pass P6_out through the Cot module, combine it with P7_in to obtain P7_out.

4. The method for detecting water surface targets based on improved Efficientdet according to claim 3, wherein The process passing through the Cot module is as follows: S3.2.

1. Input the feature map X of size C×H×W, where C, H, and W represent the number of channels, the height of the feature map, and the width of the feature map respectively; S3.2.

2. Spatially contextually encode all adjacent keys within all k×k grids in the feature map X to obtain the contextual key K 1 , K 1 contains static contextual information between locally adjacent keys, where K = X, S3.2.

3. Concatenate context keys K 1 and Q to obtain a feature map of H×W×2C, where Q = X; S3.2.

4. Input the feature map of H×W×2C into two consecutive 1×1 convolutions with and without the Relu activation function, denoted by W θ and W δ respectively, to obtain the local attention matrix A for combined learning of the query feature and the context key feature: A = [K 1 , Q]W θ W δ (7) S3.2.

5. Aggregate the local attention matrix A with V to obtain the feature map K after weighting, which contains the dynamic feature interaction between the inputs 2 , where V = XW v , represents the matrix multiplication operation S3.2.

6. Fusing Static Context K 1 with Dynamic Context K 2 to output a feature map Y of size H×W×C: Y = K 1 + K 2 (9).

5. The method for detecting water surface targets based on improved Efficientdet according to claim 1, characterized in that, In the fourth step described above, replace the activation functions in ClassNet and BoxNet with H-Swish, and the expression of this function is: H-Swish(x) = x × Relu6(x + 3) ÷ 6 (10) where Relu6 = min(Relu, 6) (11).

6. An improved Efficientdet-based water surface target detection system, characterized in that, A method for detecting water surface targets based on the improved Efficientdet, which is used to execute any one of claims 1 to 5 during implementation, includes an improved Efficientdet model unit.

Citation Information

Patent Citations

  • Optical remote sensing image ship detection method based on an improved YOLO V2 model

    CN109492561A

  • Article detection method based on convolutional neural network

    CN114612770A

  • River surface sand dredger facing edge calculation and passing ship identification method

    CN115082872A