Aerial target detection assisted sea surface search and rescue method based on improved YOLOv7

Abstract

The present invention discloses a method for aerial target detection assisted sea surface search and rescue based on an improved YOLOv7. A dataset of aerial images of the sea surface with targets is obtained and divided into a training set, a validation set, and a test set. An improved YOLOv7 network model is constructed: the improved YOLOv7 network model replaces the Concat module and the ELAN module in the Neck portion of the existing YOLOv7 network with a Transformer-based feature fusion module TTF, and simultaneously introduces the SA attention mechanism in the Neck portion. The trained, validated, and tested improved YOLOv7 network model is used to implement aerial target detection assisted sea surface search and rescue. The present invention utilizes the improved YOLOv7 network model to improve the sea surface search and rescue target detection effect when detecting targets with multi-scale changes and high dynamic background interference on the sea surface.

Description

Technical Field

[0001] The present invention relates to the field of maritime search and rescue technology, and in particular to an aerial target detection-assisted sea surface search and rescue method based on improved YOLOv7. Background Art

[0002] The ocean is a vital venue for human activity. With the increasing frequency of maritime transportation, fishing, tourism, and other activities, the incidence of maritime accidents is also increasing. From ship collisions and groundings to encounters with severe weather at sea, maritime accidents are numerous and often occur suddenly, resulting in significant loss of life and property. In such situations, maritime search and rescue operations are particularly crucial. Only timely, accurate, and efficient search and rescue can minimize the losses caused by accidents and safeguard the safety of life and property.

[0003] Traditional maritime search and rescue methods rely heavily on manpower, particularly human vision, but this method is time-consuming, labor-intensive, and inefficient. In recent years, with the advancement of computer technology and artificial intelligence, deep learning-based algorithms have been gradually applied to target detection. By acquiring real-time aerial images and using deep learning algorithms to automatically detect targets such as people and ships in distress at sea, they can, to a certain extent, replace manual search and better protect the lives of people at sea.

[0004] Due to the particularity of the sea surface environment, existing deep learning-based target detection algorithms have shown shortcomings in this scenario. The strong light, sunlight reflection and refraction, waves, and water splashes that are prevalent on the sea surface will cause people who fall into the water and objects waiting to be detected on ships to face high-dynamic background interference, resulting in significant changes in the size, shape, texture and other features of the targets to be detected. Existing target detection algorithms such as the R-CNN (Rich feature hierarchies for accurate object detection and semantic segmentation) series of algorithms and the YOLO (You Only LookOnce: Unified, Real-Time Object Detection) series of algorithms generally have low detection accuracy and high missed detection and false detection rates when detecting these multi-scale targets with high-dynamic background interference, making it difficult to apply existing algorithms to practical sea surface search and rescue. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem that the existing deep learning algorithm proposed in the above background technology has poor detection effect when detecting targets with multi-scale changes and high dynamic background interference on the sea surface, and is difficult to be applied in actual sea surface search and rescue. A method for aerial target detection assisted sea surface search and rescue based on improved YOLOv7 is provided.

[0006] In a first aspect, the present invention provides an aerial target detection-assisted sea surface search and rescue method based on improved YOLOv7, the method comprising the following steps:

[0007] S1: Obtain a dataset of aerial images of the sea surface with targets and divide the dataset into a training set, a validation set, and a test set.

[0008] S2: Build and improve the YOLOv7 network model:

[0009] The improved YOLOv7 network model replaces the Concat module and the ELAN module in the Neck part of the existing YOLOv7 network with a Transformer-based feature fusion module TTF, and introduces the SA attention mechanism in the Neck part.

[0010] S3: Input the training set and validation set divided in step S1 into the improved YOLOv7 network model for training and validation.

[0011] S4: Input the test set and use the evaluation indicators to test and evaluate the performance of the trained and verified improved YOLOv7 network model.

[0012] S5: Use the trained, verified, and tested improved YOLOv7 network model to realize aerial target detection to assist sea surface search and rescue.

[0013] Furthermore, the loss function of the improved YOLOv7 network model adopts the Inner-CIoU loss function, and the calculation formula is as follows:

[0014] L Inner-CIoU =L CIoU +IoU-IoU inner

[0015] Among them, IoU inner represents the IoU based on the auxiliary bounding box.

[0016] The calculation formula for IoU is as follows:

[0017]

[0018] Where A represents the predicted box and B represents the true annotation box.

[0019] L CIoU The calculation formula is as follows:

[0020]

[0021] where b and b gtare the center points of A and B respectively, ρ(·) represents the Euclidean distance, c is the diagonal length of the minimum bounding rectangle that contains both A and B, and α is a positive trade-off parameter. The calculation formula is as follows:

[0022]

[0023] Where v is a parameter that measures the consistency of aspect ratio, and the calculation formula is as follows:

[0024]

[0025] where w gt and h gt They represent the width and height of the true annotation box, w and h represent the width and height of the predicted box, respectively.

[0026] IoU inner The calculation formula and intermediate calculation process are as follows:

[0027]

[0028]

[0029] union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter

[0030]

[0031]

[0032]

[0033]

[0034] in, and Respectively represent the horizontal and vertical coordinates of the center point of the real annotation box, x c and y c They represent the horizontal and vertical coordinates of the center point of the prediction box respectively. The variable ratio represents a scale factor, and its value range is usually [0.5, 1.5].

[0035] The Inner-CIoU loss function is used to improve the original CIoU loss function of YOLOv7. The Inner-CIoU loss function introduces an auxiliary bounding box calculation loss, which accelerates the convergence of the loss function and improves the accuracy of bounding box matching, thereby improving the algorithm's detection accuracy for sea surface targets.

[0036] Furthermore, the Transformer-based feature fusion module TTF includes a feature enhancement module TFA, a first fusion module, and a second fusion module. Deep feature map F d , shallow feature map F s The enhanced feature map F is obtained by the feature enhancement module TFA d_new , then the deep feature map F d With the enhanced feature map F d_new The intermediate feature map F is obtained by fusion of the first fusion module f0 . Intermediate feature map F f0 , shallow feature map F s The fused feature map F is obtained by weighted fusion using the learnable parameters α and β through the second fusion module. f The calculation formula is as follows:

[0037] F f =α(F d +F d_new )+βF s

[0038] Furthermore, the feature enhancement module TFA specifically operates as follows:

[0039] Deep feature map F d , shallow feature map F s After pooling and reshaping operations, we get the matrix L d , L s , the calculation formula is as follows:

[0040]

[0041] Among them, Maxpooling represents the maximum pooling operation, S d and S s Represents F d , F s Reshape means reshaping the matrix so that its shape changes from C×H×W to HW×C.

[0042] Then, the obtained L d , L s Add position code PE respectively, And it is a learnable weight matrix, and we get L′ d , L s ′, the calculation formula is as follows:

[0043]

[0044] Using the idea of ​​Transformer, the matrix L′ d Projected onto the key matrix K dSum matrix V d , the matrix L s ′ is projected onto the query matrix Q s , the projection process formula is as follows:

[0045]

[0046] Among them, W Q , W K and W V Represents the learnable weight matrix of the Linear layer.

[0047] Then, the multi-head cross attention mechanism of Transformer is used to obtain the multi-head cross attention output. The calculation formula is as follows:

[0048] MHCA=MultiHeadCrossAttention(Q s ,K d ,V d )=Concat(H1,…,H8)W o

[0049] MHCA stands for Multi-Head Cross Attention Output, MultiHeadCrossAttention stands for Transformer Multi-Head Cross Attention Mechanism, which calculates the attention matrix on 8 independent heads. i (i=1,…,8) represents the attention matrix output by each independent head, Concat represents the concatenation operation, W o Represents the learnable weight matrix of the Linear layer.

[0050] Attention matrix H i The calculation formula is as follows:

[0051]

[0052] in, Denote the query matrix, key matrix and value matrix of the i-th head respectively, d k express dimension.

[0053] Next, two residual connections and the feedforward neural network (FFN) in the standard Transformer are introduced, and finally the shape of the input feature map is restored by reshaping and upsampling, and the new feature map F with enhanced features is output. d_new , the calculation formula is as follows:

[0054] F d_new =Upsampling(Reshape(MHCA+L′ d +FFN(MHCA+L′d )))

[0055] Among them, Upsampling represents the upsampling operation, and Reshape represents reshaping the matrix so that its shape changes from HW×C to C×H×W.

[0056] The TFA module leverages the Transformer's multi-head cross-attention mechanism, enabling shallow feature maps to globally assist deep feature maps in learning more feature information while also supplementing the deep feature maps. The TTF module uses shallow feature maps to enhance deep feature maps before performing feature fusion. This achieves more refined and comprehensive feature fusion while effectively improving the algorithm's robustness, making it suitable for maritime search and rescue scenarios.

[0057] Furthermore, in step S2, in the Neck part of the YOLOv7 network model, the SA (Shuffle Attention) attention mechanism with extremely small parameters is added before the SPPCSPC module and after the two CBS modules derived from the Backbone part. This allows the network to pay more attention to the area of ​​interest from both spatial and channel dimensions, enabling the model to capture more detailed features and improving the algorithm's perception and detection effect on sea surface targets.

[0058] In a second aspect, the present invention provides an aerial target detection assisted sea surface search and rescue system, comprising:

[0059] Data acquisition module, acquiring aerial images of the sea surface;

[0060] The detection module inputs the sea surface aerial images into the improved YOLOv7 network model that has been trained, verified, and tested to realize the detection of aerial targets.

[0061] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described above.

[0062] In a fourth aspect, the present invention provides a computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described is implemented.

[0063] Compared with the prior art, the present invention has at least the following beneficial effects:

[0064] (1) The present invention proposes a feature enhancement module TFA, which uses the multi-head cross attention mechanism of Transformer to enable the shallow feature map to assist the deep feature map in learning more feature information from a global perspective while supplementing the features of the deep feature map. This unique design can effectively repair some features of the deep feature map that have been lost or deformed during a series of continuous sampling. At the same time, the enhanced feature map calculated and generated by the Transformer multi-head cross attention mechanism can effectively capture target information of different scales, thereby improving the algorithm's detection effect on multi-scale changing targets, small targets and highly dynamic background interference targets in complex and changing sea environments in sea search and rescue scenarios.

[0065] (2) The present invention proposes a Transformer-based feature fusion module TFF to replace the Concat module and ELAN module of the Neck part in YOLOv7. It uses shallow feature maps to assist deep feature maps in feature enhancement and then performs feature fusion. While achieving more refined and comprehensive feature fusion, it also effectively improves the robustness of the algorithm and better copes with the problems of strong interference and multi-scale changes of targets in sea search and rescue scenarios.

[0066] (3) The present invention introduces the TFF module into YOLOv7, replacing the fusion operation of the Neck part in the original network, reducing the number of parameters and floating-point operations of the algorithm, improving the algorithm operation efficiency and resource utilization, and facilitating the practical application of the detection algorithm in sea search and rescue scenarios.

[0067] (4) The present invention introduces the SA attention mechanism with extremely small parameters into YOLOv7, which enables the model to pay more attention to the area of ​​interest from the two dimensions of space and channel, enhances the perception ability of the model, enables the model to capture more detailed features, and improves the algorithm's perception ability and detection effect for sea surface targets.

[0068] (5) This paper improves the original CIoU loss function in YOLOv7 into the Inner-CIoU loss function, introduces the auxiliary bounding box calculation loss, accelerates the convergence process of the loss function, improves the accuracy of bounding box matching, and thus improves the algorithm's detection accuracy for sea surface targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of the steps of the aerial target detection assisted sea surface search and rescue method based on improved YOLOv7 provided in an embodiment of the present invention.

[0070] Figure 2 A schematic diagram of the structure of a Transformer-based feature fusion module (TTF) provided in an embodiment of the present invention.

[0071] Figure 3 A schematic structural diagram of a feature enhancement module (TFA) in a TTF module provided in an embodiment of the present invention.

[0072] Figure 4 Schematic diagram of the improved YOLOv7 network model provided by an embodiment of the present invention.

[0073] Figure 5 A schematic diagram of the visual detection results provided by an embodiment of the present invention.

[0074] Figure 6 A heat map representation of the visual detection results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0076] like Figures 1-6 As shown, this embodiment provides an aerial target detection-assisted sea surface search and rescue method based on improved YOLOv7, which specifically includes the following steps:

[0077] S1: Obtain a dataset of aerial images of the sea surface with targets and divide the dataset into a training set, a validation set, and a test set.

[0078] S2: Build an improved YOLOv7 network model.

[0079] The improved YOLOv7 network model replaces the Concat module and the ELAN module in the Neck part of the existing YOLOv7 network with a Transformer-based feature fusion module TTF, introduces the SA attention mechanism in the Neck part, and improves the loss function of the network model.

[0080] S3: Input the training set and validation set divided in step S1 into the improved YOLOv7 network model for training and validation.

[0081] S4: Input the test set and use the evaluation indicators to test and evaluate the performance of the trained and verified improved YOLOv7 network model.

[0082] S5: Use the trained, verified, and tested improved YOLOv7 network model to realize aerial target detection to assist sea surface search and rescue.

[0083] Furthermore, in step S1, an AFO (Aerial Dataset of Floating Objects) dataset, obtained from drones, is obtained. This dataset contains six instance categories: humans, surfboards, boats, buoys, sailboats, and kayaks. Buoys and humans are considered small objects, while surfboards, sailboats, kayaks, and boats are considered large objects. This dataset is divided into training, validation, and test sets in a ratio of 0.70:0.15:0.15.

[0084] Further, in step S2, refer to Figure 2-Figure 4 , construct the Transformer-based feature fusion module TFF and replace the Concat module and ELAN module in the Neck part of the original YOLOv7 network with the TFF module. The TTF module includes the feature enhancement module TFA, the first fusion module, and the second fusion module. Deep feature map F d , shallow feature map F s The enhanced feature map F is obtained by the feature enhancement module TFA d_new , then the deep feature map F d With the enhanced feature map F d_new The intermediate feature map F is obtained by fusion of the first fusion module f0 . Intermediate feature map F f0 , shallow feature map F s The fused feature map F is obtained by weighted fusion using the learnable parameters α and β through the second fusion module. f The calculation formula is as follows:

[0085] F f =α(F d +F d_new )+βF s

[0086] The specific calculation details of the TFA module are as follows:

[0087] Let the input feature map be F d and F s , F d Represents the deep feature map of the network, F s Represents the shallow feature map of the network. In the TFA module, F d and F s After pooling and reshaping operations, the matrix L is obtained d , L s The calculation formula is as follows:

[0088]

[0089] Among them, Maxpooling represents the maximum pooling operation, S d and Ss Represents F d , F s Reshape means reshaping the matrix so that its shape changes from C×H×W to HW×C.

[0090] Then, the obtained L d , L s Add position code PE respectively, And it is a learnable weight matrix, and we get L′ d , L s ′, the calculation formula is as follows:

[0091]

[0092] Using the idea of ​​Transformer, the matrix L′ d Projected onto the key matrix K d Sum matrix V d , the matrix L s ′ is projected onto the query matrix Q s The projection process formula is as follows:

[0093]

[0094] Among them, W Q , W K and W V Represents the learnable weight matrix of the Linear layer.

[0095] In order to enable the model to capture different and diverse representations of data from multiple perspectives during training, and further understand and learn different types of information from multiple perspectives, a multi-head cross attention mechanism is introduced in the TFA module. The calculation formula is as follows:

[0096] MHCA=MultiHeadCrossAttention(Q s ,K d ,V d )=Concat(H1,…,H8)W o

[0097] MHCA stands for Multi-Head Cross Attention Output, MultiHeadCrossAttention stands for Transformer Multi-Head Cross Attention Mechanism, which calculates the attention matrix on 8 independent heads. i (i=1,…,8) represents the attention matrix output by each independent head, Concat represents the concatenation operation, W o Represents the learnable weight matrix of the Linear layer.

[0098] Attention matrix Hi The calculation formula is as follows:

[0099]

[0100] in, Denote the query matrix, key matrix and value matrix of the i-th head respectively, d k express dimension.

[0101] Next, we introduce two residual connections and the feedforward neural network (FFN) in the standard Transformer. The FFN contains two fully connected layers. The first layer will increase the dimension of the input vector, and the second layer will reduce the dimension of the vector again, so that the model can learn more abstract features, further improving the robustness and accuracy of the model. Finally, the matrix is ​​reshaped and upsampled using bilinear interpolation to restore it to the shape of the input feature map, and the new feature map with enhanced features is output, which is denoted as F d_new The calculation formula is as follows:

[0102] F d_new =Upsampling(Reshape(MHCA+L′ d +FFN(MHCA+L′ d )))

[0103] Among them, Upsampling represents the upsampling operation, and Reshape represents reshaping the matrix so that its shape changes from HW×C to C×H×W.

[0104] Further, in step S2, refer to Figure 4 The SA attention mechanism is added to the network, placed before the SPPCSPC module in the Neck portion of the YOLOv7 network model and after the two CBS modules derived from the Backbone portion. The SA attention mechanism organically combines spatial attention and channel attention, allowing the model to pay more attention to areas of interest from both spatial and channel dimensions. This enhances the model's perception and enables it to capture more detailed features, improving the algorithm's perception and detection of surface targets.

[0105] The specific calculation details of the SA module are as follows:

[0106] Given the input feature map of the SA module is F S , F along the channel dimension S Divided into g groups, the calculation formula is as follows:

[0107]

[0108] The divided FSi It is further divided into two branches along the channel dimension, and the calculation formula is as follows:

[0109]

[0110] F Si_1 , F Si_2 These two branches are used to generate channel attention maps and spatial attention maps respectively.

[0111] In terms of channel attention, the SA module uses global average pooling (GAP) to embed global information and then generate channel statistics s. The calculation formula is as follows:

[0112]

[0113] Then use the Sigmoid activation function to implement a simple gating mechanism. The calculation formula is as follows:

[0114]

[0115] Among them, W1 is the learnable weight matrix, b1 is the bias term, both used to scale and move parameters, and there is σ(·) represents the Sigmoid activation function.

[0116] In terms of spatial attention, the SA module uses Group Norm (GN) to obtain spatial statistical information, and then uses Operation to F Si_2 To enhance, the calculation formula is as follows:

[0117]

[0118] Among them, W2 is the learnable weight matrix, b2 is the bias term, and there is

[0119] After completing the above operations, the two branches of channel attention and spatial attention output are reorganized, and the reorganized sub-feature is F S ' i , the calculation formula is as follows:

[0120]

[0121] Finally, all sub-features F S ' i Reaggregate together, keep the input and output dimensions of the SA module consistent, and obtain the output feature map F S ′.

[0122] Furthermore, in step S2, the original CIoU loss function in YOLOv7 is improved to the Inner-CIoU loss function. In YOLOv7, the CIoU loss function is used to evaluate the similarity between the predicted bounding box and the ground-truth bounding box, but it has certain shortcomings. For example, small objects or objects with large aspect ratios may be ignored or misclassified by the model due to calculation bias. The Inner-CIoU loss function introduces an auxiliary bounding box calculation loss, which accelerates the convergence of the loss function and improves the accuracy of bounding box matching with almost no increase in computational effort, thus improving the algorithm's detection accuracy for sea surface objects.

[0123] The calculation formula of the Inner-CIoU loss function is as follows:

[0124] L Inner-CIoU =L CIoU +IoU-IoU inner

[0125] Among them, IoU inner represents the IoU based on the auxiliary bounding box.

[0126] The calculation formula for IoU is as follows:

[0127]

[0128] Where A represents the predicted box and B represents the true annotation box.

[0129] L CIoU The calculation formula is as follows:

[0130]

[0131] where b and b gt are the center points of A and B respectively, ρ(·) represents the Euclidean distance, c is the diagonal length of the minimum bounding rectangle that contains both A and B, and α is a positive trade-off parameter. The calculation formula is as follows:

[0132]

[0133] Where v is a parameter that measures the consistency of aspect ratio, and the calculation formula is as follows:

[0134]

[0135] where w gt and h gt They represent the width and height of the true annotation box, w and h represent the width and height of the predicted box, respectively.

[0136] IoU inner The calculation formula and intermediate calculation process are as follows:

[0137]

[0138]

[0139] union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter

[0140]

[0141]

[0142]

[0143]

[0144] in, and Respectively represent the horizontal and vertical coordinates of the center point of the real annotation box, x c and y c They represent the horizontal and vertical coordinates of the center point of the prediction box respectively. The variable ratio represents a scale factor, which usually ranges from [0.5, 1.5] and is 1.2 here.

[0145] Furthermore, in step S3, the experimental platform uses the Windows 11 operating system, based on Pytorch 2.0.0, CUDA 11.7, and Python 3.11.2. The CPU uses an Intel Core i7-13700KF and the GPU uses an NVIDIA GeForce RTX 4090. The initial learning rate is set to 0.01, the batch size is set to 16, the number of epochs is set to 100, and the SGD optimizer is used. The divided training and validation sets are input into the improved YOLOv7 network model described above. The pre-trained model is used for training and validation using the experimental platform described above.

[0146] Furthermore, in step S4, the weight file trained in step S3 is loaded into the network, and the test set is input into the network to test and evaluate the performance of the improved YOLOv7 network model.

[0147] The evaluation indicators are as follows: precision (P), recall (R), average precision (AP), mean average precision (mAP), floating point operations (FLOPs), and model parameters.

[0148] The accuracy rate (P) indicates the proportion of samples that are actually positive among all samples predicted to be positive. The calculation formula is as follows:

[0149]

[0150] Recall (R) indicates the proportion of positive predictions among all actually positive samples. The calculation formula is as follows:

[0151]

[0152] Among them, F p (False positive) indicates the number of false positives, n (false negative) indicates the number of positive cases missed, T p (True Positives) represents the number of positive examples that are correctly identified.

[0153] Average precision (AP) represents the area under the precision-recall curve (PR curve). It is a method to measure the average performance of accuracy at various recall levels. The calculation formula is as follows:

[0154]

[0155] mAP represents the average AP value of all target categories. It can comprehensively evaluate the performance of the model. The calculation formula is as follows:

[0156]

[0157] Where N represents the total number of target categories.

[0158] mAP50 represents the mAP when the IOU threshold is set to 0.5, and mAP50:95 represents the average of a series of mAP calculations when the IOU threshold is set in the range of 0.5 to 0.95.

[0159] Floating-point operations (FLOPs) are used to estimate the computational complexity of the model. The larger the FLOPs, the higher the computational complexity of the model. GFLOPs means that the unit of FLOPs is 1 billion.

[0160] The number of model parameters refers to the number of all learnable parameters in the model. The larger the number of model parameters, the higher the complexity of the model.

[0161] Based on the above evaluation indicators, the performance of the improved YOLOv7 model of the present invention was evaluated, and the following comparative experiments and ablation experiments were designed.

[0162] (1) Comparative experiment

[0163] Ten classic target detection algorithms, including Faster R-CNN, SSD, YOLOv3, YOLOv4, YOLOv5, YOLOX, YOLOv7, RetinaNet, CenterNet, and YOLOv8, were used for comparative experiments on the AFO dataset. All detection algorithms were trained using pre-trained weights, and the experimental results are shown in Table 1. The table shows that the improved YOLOv7 algorithm proposed in the present invention has a significant advantage over other algorithms. Compared with the baseline algorithm YOLOv7, the mAP50 of the improved YOLOv7 algorithm increased by 10.26%, while the number of parameters and floating-point operations decreased by 1.98M and 13.69, respectively. Compared with the new detection algorithm YOLOv8 in the current YOLO series, the mAP50 of the improved YOLOv7 algorithm is 1.52% higher. It can be seen that the improved YOLOv7 algorithm proposed in the present invention is more suitable for sea search and rescue scenarios than other algorithms.

[0164] Table 1 shows the results of comparative experiments on the AFO dataset.

[0165]

[0166]

[0167] (2) Ablation experiment

[0168] To further evaluate the improved YOLOv7 network model, eight sets of ablation experiments were conducted on the AFO dataset. The first set of experiments used YOLOv7 as the baseline model; the second set of experiments improved the Inner-CIoU loss function based on YOLOv7; the third set of experiments added the SA attention mechanism to YOLOv7; the fourth set of experiments simultaneously introduced Inner-CIoU and SA based on YOLOv7; the fifth set of experiments introduced the TFF module into YOLOv7, named TFF-YOLOv7; the sixth set of experiments improved the Inner-CIoU loss function based on TFF-YOLOv7; the seventh set of experiments added the SA attention mechanism to TFF-YOLOv7; and the eighth set of experiments simultaneously introduced Inner-CIoU and SA based on TFF-YOLOv7. The experimental results are shown in Table 2. As can be seen from the experimental results, the improved YOLOv7 network model of the present invention performs well in detecting various targets in sea scenes, and its detection accuracy is superior to that of the original YOLOv7 network model. Compared with the original YOLOv7 network model, the improved YOLOv7 network model improves the mAP50 of boat detection by 23.14% and the mAP50 of buoy detection by 15.14%, which is a significant improvement.

[0169] Table 2 shows the mAP50 results of each target obtained by ablation experiments on the AFO dataset.

[0170]

[0171] Further, in step S4, as Figure 5 and Figure 6 As shown in Figure 3, the visual detection results and heat map representation of the improved YOLOv7 network model on the test set are shown. It can be seen that the improved YOLOv7 network model of the present invention can detect targets on the sea surface more accurately and is suitable for sea surface search and rescue scenarios.

[0172] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for assisting sea surface search and rescue based on aerial target detection using improved YOLOv7, characterized in that The method comprises the following steps: S1: Obtain a dataset of aerial images of the sea surface with targets and divide the dataset into a training set, a validation set, and a test set; S2: Build and improve the YOLOv7 network model: The improved YOLOv7 network model replaces the Concat module and the ELAN module in the Neck part of the existing YOLOv7 network with a Transformer-based feature fusion module TTF, and introduces the SA attention mechanism in the Neck part; S3: Input the training set and validation set divided in step S1 into the improved YOLOv7 network model for training and validation; S4: Input the test set and use the evaluation indicators to test and evaluate the performance of the trained and verified improved YOLOv7 network model; S5: Use the trained, verified, and tested improved YOLOv7 network model to realize aerial target detection to assist sea surface search and rescue; The loss function of the improved YOLOv7 network model adopts the Inner-CIoU loss function, and the calculation formula is as follows: ； in Represents the IoU based on the auxiliary bounding box; The calculation formula for IoU is as follows: ； Where A represents the predicted box and B represents the real annotation box; The calculation formula is as follows: ； where b and are the center points of A and B respectively, represents the Euclidean distance, c is the diagonal length of the minimum circumscribed rectangle that contains both A and B, is a positive trade-off parameter, and v is a parameter that measures the consistency of aspect ratio; The Transformer-based feature fusion module TTF includes a feature enhancement module TFA, a first fusion module, and a second fusion module; a deep feature map , shallow feature map The enhanced feature map is obtained by the feature enhancement module TFA , then the deep feature map With enhanced feature map The intermediate feature map is obtained by fusion of the first fusion module ; Intermediate feature map , shallow feature map Using learnable parameters 、 The fused feature map is obtained by weighted fusion of the second fusion module ; The specific operation of the feature enhancement module TFA is: Deep feature map , shallow feature map After pooling and reshaping operations, we get the matrix , ; Then, the obtained , Add position code PE respectively, and get , ; Using the idea of ​​Transformer, the matrix Projected onto the bond matrix Sum Matrix ; The matrix Projection to query matrix ; Then, the multi-head cross attention mechanism of Transformer is used to obtain the multi-head cross attention output; next, two residual connections and the feedforward neural network FFN in the standard Transformer are introduced, and finally the shape of the input feature map is restored by reshaping and upsampling, and the new feature map with enhanced features is output. .

2. The method according to claim 1, characterized in that In step S2, in the Neck part of the YOLOv7 network model, the SA attention mechanism is added before the SPPCSPC module and after the two CBS modules derived from the Backbone part.

3. An aerial target detection assisted sea surface search and rescue system implementing the method according to any one of claims 1-2, characterized in that include: Data acquisition module, acquiring aerial images of the sea surface; The detection module inputs the sea surface aerial images into the improved YOLOv7 network model that has been trained, verified, and tested to realize the detection of aerial targets.

4. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 2.

5. A computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 2 is implemented.

Images