Ship target detection method based on horizontal direction perception and bidirectional feature fusion

Abstract

The invention discloses a ship target detection method based on horizontal direction perception and bidirectional feature fusion, and the method comprises the steps: constructing a horizontal direction perception layer, and replacing an ELAN-Tiny layer of the shallowest layer in a feature extraction network of a YOLOv7-tiny network with the horizontal direction perception layer; the detection head with the largest size in the YOLOv7-tiny network is removed; constructing a bidirectional feature fusion layer, embedding the bidirectional feature fusion layer into output ends of a shallow layer and a middle layer ELAN-Tiny layer of a feature fusion path from top to bottom in a feature fusion network of the YOLOv7-tiny network, and performing mutual feature fusion; constructing a loss function with length and width decoupling weighting as a positioning loss function; and the optimized YOLOv7-tiny network is trained, and a ship target is detected. The ship is high in adaptability in the horizontal direction, the model load degree is low, and the recognition and positioning accuracy and the detection efficiency of the ship target are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application design relates to the technical field of ship target detection, and in particular to a ship target detection method based on horizontal direction perception and bidirectional feature fusion. BACKGROUND

[0002] In the key fields of offshore surveillance, channel management, emergency rescue, etc., ship target detection technology is particularly important as an important pillar of maritime safety. At present, ship target detection technology based on visible light imaging continues to dominate in maritime target perception systems due to its low equipment cost, excellent spatial resolution, and close proximity to human visual cognition. Although emerging sensing technologies such as synthetic aperture radar and infrared thermal imaging have made significant progress in recent years, visible light imaging technology still plays an irreplaceable role due to its unique advantages in target detail presentation, system compatibility, and environmental adaptability. However, for ship target detection with horizontal features, existing technologies have limitations. Traditional standard convolution operations cannot effectively capture the horizontal features of ships, resulting in limited detection accuracy. In addition, when the aspect ratio of a ship target is extremely large, the positioning accuracy is insufficient. Furthermore, the complexity of the network structure not only increases the demand for computing resources but also poses challenges to real-time performance. Therefore, constructing an efficient and accurate ship target detection technology is crucial for improving all-weather and intelligent maritime monitoring networks and enhancing maritime safety and security capabilities.

[0003] The paper "Maritime Ship Target Detection Algorithm in Complex Sea Conditions" (Modern Defense Technology, Vol. 52, No. 06, June 2024, Xing Huiyuan et al.) proposes an improved maritime target detection algorithm based on YOLOv7-Tiny. The algorithm is based on YOLOv7-Tiny, which designs a feature extraction module RepELAN without information loss through RepVGG to enhance feature extraction capability, improves the feature sharing fusion network to enhance positioning and detection performance, and clips low-resolution feature maps to reduce computational complexity. However, the model lacks optimization in bounding box regression, and the accuracy is limited when dealing with ship targets with extreme aspect ratios.

[0004] The paper "Ship Visual Detection Method in Complex Environment Based on Deep Learning" (Ship Engineering, Vol. 46, No. 02, February 2024, Cui Yuxiao et al.) proposes a ship visual detection method based on the YOLOv7 algorithm. The algorithm replaces the standard convolution layer with RepVGG to reduce computational complexity, fuses the SimAM attention mechanism to enhance ship feature extraction, and uses dark channel dehazing preprocessing to improve accuracy. However, the model has the problem of insufficient fusion of shallow and deep features, and in complex scenes, the model cannot effectively integrate feature information at different levels, leading to missed or false detections. SUMMARY

[0005] In view of the deficiencies in the prior art, the present application provides a ship target detection method based on horizontal direction perception and bidirectional feature fusion to solve the technical problems of insufficient adaptability of standard convolution and ship horizontal direction, high model complexity and insufficient boundary box regression accuracy in the prior art.

[0006] The present application provides a ship target detection method based on horizontal direction perception and bidirectional feature fusion, comprising the following steps:

[0007] Step 1: constructing a basic YOLOv7-tiny network;

[0008] Step 2: constructing a horizontal direction perception layer, replacing the ELAN-Tiny layer in the shallowest layer of the feature extraction network of the YOLOv7-tiny network with the horizontal direction perception layer, wherein the image input into the horizontal direction perception layer is processed by standard convolution and direction deformable convolution enhanced by horizontal perception, then the feature maps are spliced, and the channel number is adjusted by standard convolution to serve as the output of the horizontal direction perception layer;

[0009] Step 3: removing the largest detection head in the YOLOv7-tiny network;

[0010] Step 4: constructing a bidirectional feature fusion layer, embedding the bidirectional feature fusion layer into the output ends of the shallow and middle ELAN-Tiny layers of the feature fusion network of the YOLOv7-tiny network from top to bottom, and performing mutual feature fusion on the output feature maps of the two;

[0011] Step 5: constructing a loss function with long and wide decoupling weighting as the positioning loss function in the loss function of the YOLOv7-tiny network;

[0012] Step 6: training the YOLOv7-tiny network optimized by steps 2-5 through a training set;

[0013] Step 7: using the trained and optimized YOLOv7-tiny network to detect ship targets.

[0014] Further, in step 2, the direction deformable convolution enhanced by horizontal perception is specifically:

[0015] The output of the auxiliary convolution in the direction deformable convolution is enhanced by horizontal perception through the constructed 1x5 convolution and 3x3 convolution, and the specific formula is as follows:

[0016]

[0017] wherein, D hor =σ(Conv 3×3 (Conv1×5 (X)))

[0018] wherein, is the final dynamic offset of the horizontal deformable convolution; X is the input image; a is an adjustment factor; · is point multiplication; Δp k is the output of the auxiliary convolution; D hor is the horizontal direction confidence matrix, D hor ∈[0,1]; σ(·) is a Sigmoid function.

[0019] Further, in the step 3, the upper layer feature map output in the YOLOv7-tiny network feature fusion network is removed, and the middle layer and shallow layer feature map outputs are retained.

[0020] Further, in the step 4, the specific process of the bidirectional feature fusion layer fusing two feature maps includes: taking the feature map P1 output by the middle layer ELAN-Tiny layer as the basis, interacting the feature map P1 output by the middle layer ELAN-Tiny layer with the feature map P2 output by the shallow layer ELAN-Tiny layer, and generating the feature map P1 after interaction,

[0021] and taking the feature map P2 output by the shallow layer ELAN-Tiny layer as the basis, interacting the feature map P2 output by the shallow layer ELAN-Tiny layer with the feature map P1 output by the middle layer ELAN-Tiny layer, and generating the feature map P2 after interaction;

[0022] The process of generating the feature map P1 after interaction includes:

[0023] Step A41: sequentially passing the feature map P2 through the CBS and convolution processing to obtain a feature map P2 with the same size as the feature map P1;

[0024] Step A42: performing maximum pooling and average pooling operations on the feature map P1 respectively, and dimensionally splicing the pooling results;

[0025] performing maximum pooling and average pooling operations on the feature map P2 processed in step A41 respectively, and dimensionally splicing the pooling results;

[0026] Step A43: dimensionally splicing the two dimensionally spliced feature maps, and obtaining the weighted symbol of the two feature maps through a fully connected layer with shared parameters;

[0027] Step A44: point multiplying the two feature maps with the corresponding weighted symbol and adding them to generate the feature map P1 after interaction;

[0028] The process of generating the feature map P2 after interaction includes:

[0029] Step B41: the feature map P1 is sequentially subjected to upsampling and convolution processing to obtain a feature map P1 with the same size as the feature map P2;

[0030] Step B42: the feature map P2 is subjected to maximum pooling and average pooling operations respectively, and the pooling results are subjected to dimension splicing;

[0031] The feature map P1 processed in step B41 is subjected to maximum pooling and average pooling operations respectively, and the pooling results are subjected to dimension splicing;

[0032] Step B43: the two dimension-spliced feature maps are subjected to dimension splicing, and the weighted symbol of the two feature maps is obtained through a fully connected layer with shared parameters.

[0033] Step B44: the two feature maps are multiplied by the corresponding weighted symbols and then added to generate the interactive feature map P2.

[0034] Further, in step 5, the positioning loss function with long-width decoupling weighting constructed is specifically as follows:

[0035] L loc =ω s ·L IoU +β·L dim

[0036] Wherein, L IoU =1-IoU

[0037]

[0038] L dim =λ1|ω-ω * |+λ2|h-h * |

[0039] λ1=p·γ+(1-p)

[0040] λ2=(1-p)·γ+p

[0041]

[0042] In the formula, β is a balance coefficient; λ1 and λ2 are weight coefficients; ω * is the width of the real box; h * is the height of the real box; r * is the ratio of the width of the real box to the height of the real box; p is a discrimination coefficient; τ is a direction determination threshold; and γ is a long-side strengthening coefficient.

[0043] The beneficial effects of the present application are as follows:

[0044] The application improves the direction deformable convolution to build a horizontal direction perception layer to improve the ship horizontal feature extraction capability, simplifies the feature pyramid to reduce the parameter quantity and the calculation complexity to speed up the reasoning, the bidirectional feature fusion layer promotes the information complementary compensation loss, the length-width decoupling weighted loss function optimizes the positioning accuracy for the extreme ship ratio, and the whole significantly enhances the recognition, positioning accuracy and detection efficiency of the ship target.

[0045] The application improves the direction deformable convolution and the standard convolution to build a horizontal direction perception layer, improves the ship horizontal direction feature perception capability, can more accurately adapt and capture the ship horizontal main shaft direction feature, effectively solves the problem that the existing method does not extract the ship horizontal main shaft direction feature sufficiently, thereby directly improves the recognition and positioning accuracy of the ship target, especially the long strip-shaped ship.

[0046] The application simplifies the feature pyramid structure, removes the redundant feature fusion layer and the low resolution detection head, significantly reduces the parameter quantity and the calculation complexity of the model, and improves the reasoning speed.

[0047] The bidirectional feature fusion module proposed in the application effectively promotes the information complementary and interaction between the middle layer and the shallow layer feature map, strengthens the fusion and utilization of the key details (such as edges and textures) and the context semantic information of the ship target, and compensates the information loss caused by the structure simplification.

[0048] The length-width decoupling weighted loss function proposed in the application decouples the length and width regression of the bounding box into independent branches, and according to the characteristics that the horizontal direction ship generally has a large length-width ratio, dynamically allocates higher weight to the width prediction through a nonlinear weight function, effectively alleviates the positioning deviation problem caused by the unsensitive width prediction of the long strip-shaped ship in the traditional coupled regression mode, and further improves the positioning accuracy of the horizontal feature ship bounding box. BRIEF DESCRIPTION OF DRAWINGS

[0049] The features and advantages of the present application will be more clearly understood through reference to the accompanying drawings, which are schematic and should not be understood as limiting the present application, in which:

[0050] Figure 1 The flowchart of the specific embodiment of the present application is shown in the figure;

[0051] Figure 2 The network model overall structure diagram in the specific embodiment of the present application is shown in the figure;

[0052] Figure 3 The bidirectional feature fusion layer structure diagram in the specific embodiment of the present application is shown in the figure;

[0053] Figure 4 The structure schematic diagram of the horizontal direction perception layer in the specific embodiment of the present application is shown in the figure;

[0054] Figure 5 For the purpose of the embodiment of the present application, the target detection result and the comparison chart are shown. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0056] The present application will be further illustrated below in combination with specific embodiments. Those skilled in the art should understand that these embodiments are only used to illustrate the present application and not to limit the scope of the present application. Various modifications of the equivalent forms of the present application all fall within the scope of the appended claims of the present application.

[0057] As shown in Figure 1 , 2 , the present application provides a ship target detection method based on horizontal direction perception and bidirectional feature fusion, comprising the following steps:

[0058] Step 1: constructing a basic YOLOv7-tiny network;

[0059] The public maritime ship Singapore Maritime Dataset is selected, which includes high-resolution videos, including shoreline videos, covering visible light and infrared, and airborne videos obtained from shore-based cameras on fixed platforms and cameras installed on moving ships. The dataset contains six types of ships of different sizes, including steamers, passenger ships, speedboats, ferries, sailboats and kayaks. The video images in the dataset are processed by frame division to obtain a picture dataset containing ship targets, which are integrated and classified to produce training set, test set and validation set. The ratio of training set: test set: validation set is set to 7:2:1. 4470 images are selected for training, 1270 images are selected for validation, and 610 images are selected for testing. The image resolution is 1920x1080;

[0060] Step 2: constructing a horizontal direction perception layer, as shown in Figure 3 , the ELAN-Tiny layer in the shallowest layer of the feature extraction network of the YOLOv7-tiny network is replaced by the horizontal direction perception layer, wherein the images input into the horizontal direction perception layer are processed by standard convolution and direction deformable convolution with horizontal perception enhancement, then the feature maps are spliced, and then the channel number is adjusted by standard convolution to serve as the output of the horizontal direction perception layer;

[0061] The horizontal perception enhanced direction deformable convolution is specifically as follows:

[0062] The output of the auxiliary convolution in the direction deformable convolution is horizontally perception enhanced by constructing a 1x5 convolution and a 3x3 convolution, and the specific formula is as follows:

[0063]

[0064] wherein, D hor = σ(Conv 3×3 (Conv 1×5 (X)))

[0065] In the formula, D is the final dynamic offset of the horizontal direction deformable convolution; X is an input image; a is an adjustment factor, which is set to 0.6; · is dot multiplication; Δp k is the output of the auxiliary convolution; D hor is a horizontal direction confidence matrix, D hor ∈ [0, 1]; and σ(·) is a Sigmoid function.

[0066] An asymmetric horizontal convolution with a kernel size of 1x5 is used to extract long-distance horizontal features, and then a 3x3 standard convolution is used to generate a horizontal direction confidence, as shown by the green dashed box in Figure 4 , to quantify the horizontal direction saliency of the target to be detected;

[0067] Step 3: Remove the detection head with the largest size in the YOLOv7-tiny network and the upper layer feature map output in the YOLOv7-tiny network feature fusion network, and retain the middle layer and shallow layer feature map outputs;

[0068] There are three detection heads in the YOLOv7-tiny network: a low-resolution detection head, a medium-resolution detection head, and a high-resolution detection head, with sizes of 20x20x512, 40x40x256, and 80x80x128, respectively. In this embodiment, the detection head with the largest size, i.e., the low-resolution detection head, is removed. Meanwhile, the feature fusion network of the YOLOv7-tiny network has three output feature maps, i.e., upper layer, middle layer, and shallow layer feature maps. In this embodiment, the upper layer output feature map is removed, and only the middle layer and shallow layer output feature maps are retained, which are the input sources of the bidirectional feature fusion layer;

[0069] Step 4: Construct a bidirectional feature fusion layer, as shown in Figure 4 , which is embedded into the output ends of the shallow and middle ELAN-Tiny layers of the feature fusion network of the YOLOv7-tiny network from top to bottom, to perform mutual feature fusion on the two output feature maps;

[0070] The feature fusion network of YOLOv7-tiny adopts a path aggregation-feature pyramid for feature fusion. It first uses a top-down structure for feature fusion, and outputs a middle layer feature map P1 after the first feature fusion layer, with a feature map size of 40x40x128; and outputs a shallow layer output feature map P2 after the second feature fusion layer, with a feature map size of 80x80x64. Bidirectional information exchange is needed for P1 and P2, which requires pre-processing of the two to adjust the size to be consistent;

[0071] The specific process of the bidirectional feature fusion layer to fuse the two feature maps includes:

[0072] a process of generating the feature map P1 after interaction, based on the feature map P1 output by the middle layer ELAN-Tiny layer, interacting the feature map P1 output by the middle layer ELAN-Tiny layer with the feature map P2 output by the shallow layer ELAN-Tiny layer, and

[0073] a process of generating the feature map P2 after interaction, based on the feature map P2 output by the shallow layer ELAN-Tiny layer, interacting the feature map P2 output by the shallow layer ELAN-Tiny layer with the feature map P1 output by the middle layer ELAN-Tiny layer;

[0074] Step A41: sequentially passing the feature map P2 through CBS and convolution processing to obtain a feature map P2 with the same size as the feature map P1;

[0075] The feature map P2 needs to be adjusted in size to be consistent with the feature map P1 for information exchange. For the feature map P2, a 3x3 convolution operation is performed, which is set to a step size of 2 to ensure that the adjusted size is consistent with the feature map P1, and then aligned with the channel dimension through a 1x1 convolution. At this point, the feature map P2 is completely consistent with the feature map P1 in size, both being 40x40x128.

[0076] Step A42: performing maximum pooling and average pooling operations on the feature map P1 respectively, and performing dimension splicing on the pooling results;

[0077] Step A43: performing maximum pooling and average pooling operations on the feature map P2 processed in step A41 respectively, and performing dimension splicing on the pooling results;

[0078] The bidirectional feature fusion layer respectively performs double-modal pooling on the middle layer feature map P1 and the processed shallow layer feature map P2, and simultaneously performs maximum pooling and average pooling operations on the feature map P1 and the feature map P2 in the spatial dimension. The maximum pooling is used to extract the saliency information in the feature map, highlighting the important features; the average pooling is used to extract the global context information, capturing the overall distribution of the features. Then, the two kinds of pooling results are spliced along the channel dimension into a feature map with a size of 40x40x2; then, a 1x1 convolution is used to construct a low-cost feature interaction channel, realizing cross-channel information fusion, and generating an interaction feature symbol. At this point, the feature map P2 and the feature map P1 have completed the information strengthening in the channel and spatial dimension, and the output feature map size is 40x40x1.

[0079] Step A43: The feature map spliced in two dimensions is dimensionally spliced, and the weighted symbols of the two feature maps are obtained through a fully connected layer with shared parameters;

[0080] Step A44: The two feature maps are multiplied by the corresponding weighted symbols and added to generate the feature map P1 after interaction.

[0081] The two output feature maps obtained by processing are spliced to interact information. Then, a fully connected layer with shared parameters is used for processing, and the feature maps are separated into two parts, corresponding to the feature map P1 and the feature map P2, which are denoted as P1' and P2', respectively. P1' and P2' are the weighted symbols of the feature map P1 and the feature map P2 after information interaction. At this point, the middle layer feature map P1 has communicated information to the shallow layer feature map P2 once. Then, P1' and P2' are multiplied by the feature map P1 and the feature map P2 at the same position, respectively, to obtain the output of the weighted feature map P1 and the feature map P2. Finally, the two are added to obtain the output corresponding to the middle layer feature map P1, and the output feature map size remains 40x40x128.

[0082] The process of generating the feature map P2 after interaction is as follows:

[0083] Step B41: The feature map P1 is sequentially processed by upsampling and convolution to obtain a feature map P1 with the same size as the feature map P2;

[0084] The feature map P1 needs to be adjusted to the same size as the feature map P2 for information interaction. For the feature map P1, an upsampling operation is performed to adjust its size to the same size as the shallow layer feature map P2, i.e. 80x80, to ensure alignment in the spatial dimension; then, the channel dimension is aligned through a 1x1 convolution. At this point, the feature map P1 processed above is completely consistent in size with the feature map P2, both being 80x80x64.

[0085] Step B42: Perform maximum pooling and average pooling operations on the feature map P2 respectively, and perform dimension splicing on the pooled results;

[0086] Perform maximum pooling and average pooling operations on the feature map P1 processed in step B41 respectively, and perform dimension splicing on the pooled results;

[0087] The bidirectional feature fusion layer performs bidirectional modal pooling on the processed middle-level feature map P1 and shallow-level feature map P2 respectively, and simultaneously performs maximum pooling and average pooling operations on the feature map P1 and feature map P2 in the spatial dimension. Then, the two pooled results are spliced in the channel dimension to form a feature map with a size of 80x80x2; then, a 1x1 convolution is used to construct a low-cost feature interaction channel to realize cross-channel information fusion and generate an interaction feature symbol. At this point, both the feature map P2 and the feature map P1 have completed information strengthening in the channel and spatial dimensions, and the output feature map size is 80x80x1.

[0088] Step B43: Perform dimension splicing on the two dimension-spliced feature maps, and obtain the weighted symbols of the two feature maps through a fully connected layer with shared parameters;

[0089] Step B44: Multiply the two feature maps by the corresponding weighted symbols and add them to generate the feature map P2 after interaction.

[0090] Perform splicing operation on the two output feature maps obtained by processing to facilitate information interaction. Then, use a fully connected layer with shared parameters to process and separate into two feature maps corresponding to the feature map P1 and the feature map P2, denoted as P1" and P2", respectively. P1" and P2" are the weighted symbols of the feature map P1 and the feature map P2 after information interaction. At this point, the shallow-level feature map P2 has already communicated information to the middle-level feature map P1 once. Then, perform dot multiplication operation on the feature map P1 and the feature map P2 with P1" and P2" at the same position to obtain the output of the weighted feature map P1 and the feature map P2. Finally, add the two to obtain the output corresponding to the shallow-level feature map P2, and the output feature map size remains 80x80x64.

[0091] Step 5: Construct a loss function with long-width decoupling weighting as the positioning loss function in the YOLOv7-tiny network loss function;

[0092] The specific formula of the constructed loss function with long-width decoupling weighting is:

[0093] L loc =ω s ·L IoU +β·L dim

[0094] Wherein, L IoU= 1 - IoU

[0095]

[0096] L dim = λ1|ω - ω * | + λ2|h - h * |

[0097] λ1= p · γ + (1 - p)

[0098] λ2= (1 - p) · γ + p

[0099]

[0100] In the formula, β is the equilibrium coefficient; λ1, λ2 are weight coefficients; ω * is the width of the real box; h * is the height of the real box; r * is the ratio of the width of the real box to the height of the real box; p is the discrimination coefficient; τ is the direction determination threshold; γ is the long side enhancement coefficient.

[0101] The long and wide decoupling design process is as follows:

[0102] Set the parameters of the prediction box in the ship detection task as B pre = (x, y, ω, h), and the real box as B gt = (x * , y * , ω * , h * ).

[0103]

[0104] In the formula, ω * is the width of the real box; h * is the height of the real box; r * represents the ratio of the width of the real box to the height of the real box; p is the discrimination coefficient, indicating whether the target to be detected belongs to a low aspect ratio target, and p is 1 when the ship main shaft is; τ is the direction determination threshold, which is set to 2.

[0105] Next, the decoupling regression loss calculation is performed:

[0106] L dim = λ1|ω - ω * | + λ2|h - h * |

[0107] In the formula, L dim is the decoupling regression loss function;

[0108] λ1, λ2 are weight coefficients, which can be represented as:

[0109] λ1 = p · γ (1 - p)

[0110] λ2 = (1 - p) · γ + p

[0111] In the formula, γ is the long side reinforcement coefficient, which is set to 1.5.

[0112] Assuming that the true width to true height ratio of the target to be detected is greater than 2, through simple calculation, the discrimination coefficient p is 1, λ1 is 1.5, and λ2 is 1, so at this time:

[0113] L dim = 1.5 × |ω - ω * | + |h - h * |

[0114] The width regression gets a higher proportion, and is more suitable for low aspect ratio ship targets.

[0115] Secondly, the adaptive scale weighting of various types of ship targets to be detected is carried out. The nonlinear weight function is defined as:

[0116]

[0117] In the formula, ω s represents the scale of the aspect ratio of the target to be detected. Here, for low aspect ratio ship targets and the like, the smaller the aspect ratio, the larger the ω s , so the weight given to the target to be detected will be larger.

[0118] In summary, the improved positioning loss function can be expressed as:

[0119] L loc = ω s · L IoU + β · L dim

[0120] In the formula, β is the balance coefficient, which is set to 0.6;

[0121] L IoU represents the loss function of IoU:

[0122] L IoU = 1 - IoU

[0123] Step 6: Train the YOLOv7-tiny network optimized through steps 2-5 through the training set;

[0124] The optimized YOLOv7-tiny network does not use a data enhancement strategy, and all training and test images are scaled to 640x640. The experiment is carried out on a computing platform of CUDA 11.3, PyTorch 1.11, and uses a single NVIDIA GeForce RTX 3080Ti 12G GPU for training. The initial learning rate is set to 0.01, the final learning rate is set to 0.1, the momentum is set to 0.937, the weight decay rate is set to 0.0005, the batch size is set to 16, the number of training iterations is set to 200, and the stochastic gradient descent (SGD) algorithm is used. The precision P, the recall R, the average precision mean index mAP@0.5 and mAP@0.5:0.95 are used for comprehensive evaluation. The performance of each class in the data set is evaluated.

[0125] Step 7: The trained and optimized YOLOv7-tiny network is used to detect ship targets.

[0126] The detection results of the baseline model YOLOv7-tiny and the improved model YOLOv7-tiny are visualized as shown in Figure 5 , where the white round frame represents a target with better recognition effect than the original model. In Figure 5 , there are a total of four ship targets in the figure, and the improved model has improved the recognition confidence of three of them, which represents the improvement of the overall recognition accuracy of the model.

[0127] To further verify the effectiveness of the improved model in processing ship target detection, the present application conducts an ablation experiment to verify the effectiveness of each improvement. The experimental indicators are shown in Table 1 below:

[0128]

[0129] Table 1

[0130] Table 1 shows the experimental results of each improvement strategy applied to YOLOv7-tiny and the three improvements applied to YOLOv7-tiny together. Through systematic ablation experiments on the baseline model, the improvement of each improvement module to the ship detection performance is clear. The introduction of the horizontal direction perception module effectively improves the precision, recall rate and mAP@0.5. The bidirectional feature fusion module makes a significant contribution, while further improving the recall rate R to 93.3% and the mAP@0.5 to 94.9%, it significantly reduces the model complexity, the parameter quantity is reduced to 4.15M, the calculation quantity is reduced to 9.9GFLOPs, and the inference speed is significantly improved. The optimized loss function mainly improves mAP@0.5:0.95. The final integrated optimization model integrates the advantages of each module and achieves the best balance in key indicators. The model achieves a high precision of 95.5%, improves the recall rate to 92.3%, the mAP@0.5 to 95.0%, and the mAP@0.5:0.95 to 64.3%. At the same time, its parameter quantity and calculation quantity are slightly lower than the baseline, and the inference speed is greatly increased to 99FPS, which is significantly better than the baseline of 85FPS, fully verifying the synergistic effectiveness of the present application in improving the precision and speed of ship detection.

[0131] The present application selects some YOLO series target detection models for comparison, and also verifies on Singapore Maritime Dataset, and the experimental results are shown in Table 2 as follows:

[0132] Model P(％) R(％) mAP@0.5 mAP@0.5:0.95 Params(M) Gflops FPS YOLOv7-tiny 94.3 91.2 93.7 62.8 6.98 15.6 85 Faster-RCNN 41.2 68.4 86.1 57.7 47.98 301.7 12 SSD 98.2 57.5 91.3 63.7 27.53 36.5 81 YOLOv5n 92.8 77.6 83.1 56.4 1.78 4.2 119 YOLOv5s 95.7 93.8 95.1 67.5 7.04 63.4 90 TPH-YOLOv5 91.9 90.1 95.1 69.3 15.9 14.9 82 YOLOv8n 89.2 85.3 90.2 61.4 3.01 8.2 120 BL-Net 92.6 95.2 94.8 63.5 55.53 501.4 4 YOLOv9 83.3 84.0 84.5 59.3 51.01 241.8 32 YOLOv10n 96.2 60.4 96.0 61.3 4.51 10.2 183 EfficientDet 96.9 40.4 58.2 39.0 4.4 2.8 78 Ours 95.5 92.3 95.0 64.3 4.14 9.8 99

[0133] Table 2

[0134] As shown in Table 2, in the model comparison of real-time ship detection, the optimized model exhibits better comprehensive performance in precision, speed and other indicators. It realizes an inference speed of up to 99FPS while maintaining a high precision of 95.5% in detection precision P, 92.3% in recall rate R, and 95.0% in average precision mAP@0.5, which is significantly ahead of YOLOv5s and TPH-YOLOv5 with similar precision, and far exceeds other large models such as Faster-RCNN or BL-Net. In terms of efficiency, the parameter quantity of 4.14M and the floating point operation quantity of 9.8GFLOPs make it much lower than most models, ensuring the feasibility of lightweight deployment on ship-mounted edge devices. Compared with YOLOv5n, which is extremely lightweight but has lower precision, the optimized model has greatly improved precision.

[0135] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A ship target detection method based on horizontal perception and bidirectional feature fusion, characterized in that, Includes the following steps: Step 1: Construct the basic YOLOv7-tiny network; Step 2: Construct a horizontal orientation sensing layer. Replace the shallowest ELAN-Tiny layer in the feature extraction network of the YOLOv7-tiny network with a horizontal orientation sensing layer. The image input to the horizontal orientation sensing layer is processed by standard convolution and horizontal perception enhancement directional deformable convolution, respectively. The feature maps are then stitched together and then adjusted by standard convolution to get the number of channels as the output of the horizontal orientation sensing layer. Step 3: Remove the largest detector head in the YOLOv7-tiny network; Step 4: Construct a bidirectional feature fusion layer. Embed the bidirectional feature fusion layer into the output of the shallow and middle ELAN-Tiny layers of the feature fusion path from top to bottom in the YOLOv7-tiny network, and perform feature fusion on the output feature maps of the two layers. Step 5: Construct a loss function with decoupled length and width weights as the localization loss function in the YOLOv7-tiny network loss function; Step 6: Train the YOLOv7-tiny network optimized in steps 2-5 using the training set; Step 7: Use the trained and optimized YOLOv7-tiny network to detect ship targets.

2. The ship target detection method based on horizontal perception and bidirectional feature fusion as described in claim 1, characterized in that, In step 2, the directional deformable convolution with enhanced horizontal perception specifically comprises: The horizontal perception enhancement of the output of the auxiliary convolution in the oriented deformable convolution is achieved by constructing 1×5 convolutions and 3×3 convolutions, as shown in the following formula: Among them, D hor =σ(Conv 3×3 (Conv 1×5 (X))) In the formula, Δp represents the final dynamic offset of the deformable convolution in the horizontal direction; X is the input image; α is the adjustment factor; · is the dot product; Δp k To assist the output of the convolution; D hor D is the confidence matrix in the horizontal direction. hor ∈[0,1]; σ(·) is the Sigmoid function.

3. The ship target detection method based on horizontal perception and bidirectional feature fusion as described in claim 1, characterized in that, Step 3 further includes removing the upper-layer feature map output from the YOLOv7-tiny network feature fusion network, while retaining the middle and shallow-layer feature map outputs.

4. The ship target detection method based on horizontal perception and bidirectional feature fusion as described in claim 1, characterized in that, In step 4, the specific process of fusing the two feature maps in the bidirectional feature fusion layer includes: based on the feature map P1 output by the middle ELAN-Tiny layer, performing interaction between the feature map P1 output by the middle ELAN-Tiny layer and the feature map P2 output by the shallow ELAN-Tiny layer to generate the interacted feature map P1. And the process of generating the interactive feature map P2 based on the feature map P2 output by the shallow ELAN-Tiny layer and the feature map P1 output by the middle ELAN-Tiny layer. The process of generating the interactive feature map P1 is as follows: Step A41: Process feature map P2 sequentially through CBS and convolution to obtain feature map P2 with the same size as feature map P1; Step A42: Perform max pooling and average pooling operations on feature map P1 respectively, and concatenate the pooling results in terms of dimensions; Max pooling and average pooling operations are performed on the feature map P2 after step A41, and the pooling results are then concatenated in dimensions. Step A43: Concatenate the two feature maps in two dimensions, and obtain the weights of the two feature maps through a fully connected layer with shared parameters; Step A44: Multiply the two feature maps by their corresponding weighting symbols and then add them together to generate the interactive feature map P1; The process of generating the interactive feature map P2 is as follows: Step B41: The feature map P1 is processed by upsampling and convolution in sequence to obtain a feature map P1 with the same size as the feature map P2; Step B42: Perform max pooling and average pooling operations on feature map P2 respectively, and concatenate the pooling results in terms of dimensions; Max pooling and average pooling operations are performed on the feature map P1 after step B41, and the pooling results are then concatenated in dimensions. Step B43: Concatenate the two feature maps in two dimensions, and obtain the weights of the two feature maps through a fully connected layer with shared parameters; Step B44: Multiply the two feature maps by their corresponding weighting symbols and then add them together to generate the interactive feature map P2.

5. The ship target detection method based on horizontal perception and bidirectional feature fusion as described in claim 1, characterized in that, In step 5, the specific formula for the localization loss function with length and width decoupling weights is as follows: L loc =ω s ·L IoU +β·L dim Among them, L IoU =1-IoU L dim =λ1|ω-ω * |+λ2|hh * | λ1=p·γ+(1-p) λ2=(1-p)·γ+p In the formula, β is the balance coefficient; λ1 and λ2 are the weighting coefficients; ω * h is the width of the actual bounding box. * r is the height of the true bounding box. * γ is the ratio of the width to the height of the ground truth bounding box; p is the discrimination coefficient; τ is the direction determination threshold; γ is the long side enhancement coefficient.

Images