Infrared ship wake detection method and system combining deformable convolution and attention mechanism

Abstract

The invention discloses an infrared ship wake detection method and system combining deformable convolution and an attention mechanism. According to the method, the wake detection model based on yov8 improvement is provided, the image noise problem and the small target detection problem under the complex sea condition are comprehensively considered, the deformable convolution, the attention mechanism and the multi-scale feature fusion module are provided, the model detection performance is improved, and therefore the infrared ship wake detection precision is improved. The method is mainly characterized in that a DCNv3-based improved feature extraction module is provided for solving the problems that the shape of a ship wake target is irregular, marine image noise is caused and a small target is difficult to detect, and an improved MLCA attention mechanism is used for improving the attention degree of a model on the target; and an improved multi-scale feature fusion module GFPN is used to improve the detection precision of the small target of the model.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, target detection, and in particular to an infrared ship wake detection method and system combining deformable convolution with an attention mechanism. Background Art

[0002] Ship wake detection has become a hot topic because of the small size of marine ship targets and the complex sea conditions. In recent years, artificial intelligence technology has been continuously applied to the field of target detection. Most of the existing studies have been conducted on marine ship wakes based on SAR images and visible light images. However, the research on infrared ship wakes started late, and publicly available data sets are difficult to obtain. Studies have shown that infrared data is more obvious than visible light wake information, and the image acquisition cost is lower than SAR images. However, infrared ship wake detection also faces many challenges. Specifically, since most wake images are irregular, Kelvin wakes present a deep V shape, and turbulent wakes present a dark wake, and the sea conditions are relatively complex with more image noise, it is easy to miss detection and pick up by mistake. Therefore, how to establish an infrared ship wake dataset and how to perform detection tasks in complex sea conditions and facing irregular wake targets is an important scientific issue. Summary of the invention

[0003] In order to solve the problem of difficulty in obtaining infrared ship wake datasets, the public data of Sentinel-2 satellite is used to collect infrared ship wake data of coastal cities and ports and make a complete dataset. In order to solve the problem that the wake image is irregular and difficult to detect in infrared ship wake detection, the deformable convolution DCNv3 module is adopted, which is integrated into the feature extraction module C2f of yolov8, and the deformable convolution is used to expand the receptive field of the special image; in order to solve the image noise problem in the detection task, the improved MLCA attention mechanism is adopted, and the attention mechanism combining space and channel is used to make the model fuse the information of spatial dimension and channel dimension, enhance the model's attention to the detection target, thereby reducing the impact of image noise and ultimately improving the target detection accuracy. In order to solve the problem that small targets are difficult to detect, the GFPN improved by multi-scale feature fusion module is adopted to improve the small target detection ability of the model.

[0004] The present invention will collect data from the public optical satellite Sentinel-2, collect ship wake data from coastlines, sea routes, docks, etc. around the world, and enhance and label the data to produce a complete infrared ship wake target detection data set. The present invention uses the deformable convolution DCNv3 module to fuse the core convolution module C2f module of yolov8 downsampling, and replaces the core convolution operator with our improved C2f_DCNv3 convolution operator. At the same time, an improved MLCA attention mechanism is added to increase the model's attention to the detection target, and the attention mechanism is used for the p3, p4, and p5 detection layers output by the yolov8 model feature extraction module.

[0005] In order to solve the problem that the multi-scale feature fusion module in the yolov8 network has insufficient feature fusion and insufficient information exchange between high-level feature maps and low-level feature maps, resulting in low algorithm detection accuracy, especially the difficulty in detecting small targets, the RepGFPN module is used to improve the FPN module. The above improved modules are integrated to form the final detection algorithm model, and the effectiveness of the model is verified by comparative experiments and ablation experiments on a self-built infrared ship wake dataset.

[0006] The technical solution adopted by the present invention is: an infrared ship wake detection method combining deformable convolution and attention mechanism, comprising the following steps:

[0007] Step 1, collecting ship wake image data;

[0008] Step 2: Use data enhancement to expand the collected image data and label the data to produce a complete infrared ship wake target detection dataset;

[0009] Step 3: construct an improved tail detection model based on yolov8. The improvements include: (1) introducing the DCNv3 algorithm into the C2f module to obtain the improved C2f_DCNv3, and replacing the feature extraction module in the yolov8 model with the improved C2f_DCNv3; (2) adding an improved MLCA attention mechanism to process the yolov8 network feature extraction layers p3, p4, and p5 to improve the model's attention to the detection target; (3) improving the FPN module to the RepGFPN module, which first convolves the input feature maps of different resolutions to the same size through upsampling and downsampling operations, and then fuses the semantic information from different levels through the Fusion fusion operation;

[0010] Step 4: Construct a loss function, train the wake detection model to find the optimal parameters, and use the trained model for the final infrared ship wake target detection task.

[0011] Furthermore, data enhancement includes flipping, rotation, scaling, cropping, adding Gaussian noise, brightness adjustment, Mosaic data enhancement, and Mixup data enhancement.

[0012] Furthermore, DCNv3 converts the original convolution weight w k It is separated into a depth part and a point-by-point part, where the depth part is composed of the original position-aware modulation scalar m k The point-by-point part is the projection weight w shared between the sampling points, and a grouping mechanism is introduced, so the output of DCNv3 is expressed as:

[0013]

[0014] Among them, y(p0) is the convolution output result of the current pixel p0, G represents the total number of aggregation groups, and for the g-th group, w g represents the position of the group, m gk represents the offset of the kth sample point in the gth group, normalized along dimension k by the softmax function, x g represents the slice input feature map, p k represents the kth position of the predefined grid sampling, Δp gk is the grid sampling position p in the gth group k The corresponding offset.

[0015] Furthermore, the process of improving the MLCA attention mechanism is as follows:

[0016] The input features are first locally pooled to convert the input (C, H, W) into a vector of (C, ks, ks) to extract local spatial information, where C, H, W are the channel, height and width of the input feature map, respectively, and ks is the hyperparameter step size; after the initial stage of processing, three branches are used to convert the input to obtain spatial information of different sizes: the first branch converts the feature map (C, ks, ks) into (1, 1, C) through global flat pooling, then extracts features through a 1*1 convolution operation, and finally obtains a feature map of size (C, ks, ks) through a depooling operation; the second branch processes the feature vector into (C, ks / 2, ks / 2) through a local spatial downsampling operation with a step size of 2, then exchanges spatial information through a convolution operation, and finally deconvolves into (C, ks, ks) through a deconvolution operation; the third branch feature (C, ks, ks) is converted into a (C*ks*ks, 1, 1) feature map through a Reshape operation, then extracts features through a 1*1 convolution operation, and finally reshapes to (C, ks, ks);

[0017] After the above steps, three feature maps (C, ks, ks) are obtained. The three feature maps are added and fused into one feature map to achieve the fusion of global context information. Then, the deconvolution operation is performed to obtain a feature map of the size of the original feature map, which is multiplied with the input feature to highlight the useful features. The final feature map is the output result.

[0018] Furthermore, the processing of the RepGFPN module is as follows:

[0019] The input is feature maps of three different levels First, the feature maps of different resolutions are convolved to the same size through upsampling and downsampling operations, and then the semantic information from different levels is fused through the fusion operation. The overall structure still follows the structure of the FPN feature pyramid, that is, the feature map is first upsampled, then downsampled, and finally output The feature map is used as the prediction target of the detection head;

[0020] Specifically, in the top-down feature fusion process, Through the downsampling operation, and then Perform Fusion operation to generate After downsampling Fusion Generation Next step, After upsampling Fusion output p3 layer results After downsampling Fusion output After downsampling Fusion output Final output The specific fusion formula is as follows:

[0021]

[0022] Among them, Resize is an upsampling or downsampling operation. The Fusion Blcok module first concatenates different input features through a fusion operation, and then one of the branches uses a 1*1 convolution operation to obtain the output. The other branch first passes through a 1*1 convolution, and then passes through N Simplify Rep 3*3 convolution operations. The Simplify Rep 3*3 convolution operation is to use two branches for training during training, and only use the main branch during reasoning. In this way, the model can be fully trained in the training stage, and fewer model parameters are used during reasoning to improve the model reasoning speed; after N Simplify Rep 3*3 convolution kernel convolution operations, each convolution result is output and finally fused through the ELAN connection idea, and the number of channels is controlled again through the 1*1 convolution kernel and the result is output.

[0023] Furthermore, the loss function of the wake detection model consists of multiple parts, among which the classification loss function uses VFL Loss, the regression loss is CIOU Loss and DFL Loss, and the final loss function is the weighted sum of the above three loss functions.

[0024] Furthermore, the VFL LOSS function formula is as follows:

[0025]

[0026] q is the intersection over union (IoU) of the predicted box bbox and the true box gt. IoU is the intersection of the predicted box and the true box divided by the union of the two boxes. Then p is the score, that is, the probability. The improvement of this formula is to change the division by N in the basic formula to multiplication by q. Then the two boxes intersect, that is, q>0, which is a positive sample. If the two boxes do not intersect, let q=0, which is a negative sample. α is the loss weight of the foreground and background of the image, and γ is the weight of each sample.

[0027] Furthermore, the CIoU Loss formula is as follows:

[0028]

[0029] Among them, IoU is the intersection over union ratio, b and bgt represent the center points of the two rectangular boxes, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed area of ​​the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient.

[0030] Furthermore, the DFL Loss formula is as follows:

[0031] DFL(S i , S i+1 )=-(y i+1 -y)log(S i )+(yyi )log(S i+1 )) (12)

[0032]

[0033] In the above formula, S i represents the predicted probability of the category, y represents the true value, and y i With y i+1 It is the two predicted values ​​on the left and right of the true value y, that is, y i i+1 , i is the index of the bounding box.

[0034] The present invention also provides an infrared ship wake detection system combining deformable convolution and attention mechanism, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute an infrared ship wake detection method combining deformable convolution and attention mechanism as described in the above technical solution.

[0035] Therefore, the improved wake detection model based on yolov8 proposed in the present invention comprehensively considers the image noise problem and small target detection problem under complex sea conditions, proposes deformable convolution, attention mechanism and multi-scale feature fusion module to improve the model detection performance, thereby improving the infrared ship wake detection accuracy. Its main innovation is that: this method focuses on the irregular shape of ship wake targets, marine image noise, and the difficulty in detecting small targets, proposes a feature extraction module based on DCNv3 improvement, uses the improved MLCA attention mechanism to improve the model's attention to the target, and uses the improved multi-scale feature fusion module GFPN to improve the detection accuracy of small targets in the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the present invention.

[0037] Figure 2 Schematic diagram of the deformable convolution module.

[0038] Figure 3 Schematic diagram of the attention mechanism module.

[0039] Figure 4 Schematic diagram of the multi-feature fusion module.

[0040] Figure 5 Schematic diagram of the fusion module details. DETAILED DESCRIPTION

[0041] ​In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the embodiments described are only part of the embodiments of the present invention, rather than all of the embodiments.

[0042] Combine the following Figures 1 to 5 The present invention is described in detail. Figure 1 It is a flow chart of the present invention. Figure 2 This is the infrared ship wake target detection model diagram of the present invention. The model is divided into three parts. The first part is the backbone feature extraction part, which is mainly responsible for extracting the main features of the input image and generating three feature maps p3, p4, and p5 of different sizes. At this time, the three feature maps already contain the main feature information of the image; the second part is the attention mechanism part. This part of the model will further improve the attention to the target to be detected through the attention mechanism; the third part is the multi-scale feature fusion module, which will fully fuse the information between feature maps of different scales to improve the detection accuracy of small target objects. The following will introduce each component of the present invention and the innovation of the present invention one by one.

[0043] Step S1: Access Sentinel-2 satellite data from the Copernicus Data Center, collect ship wake image data from shipping routes and ports with large traffic volume around the world, and collect infrared spectrum image data from Sentinel-2L2A satellite. Filter the dates with cloud cover of no more than 10% in the past few months, download images in the selected area, and then filter the images containing ships and wakes.

[0044] Step S2: Use data enhancement to expand the data set. The data augmentation methods used include flipping, rotating, scaling, cropping, adding Gaussian noise, brightness adjustment, Mosaic data enhancement, and Mixup data enhancement. Obtain 600 complete infrared ship wake data sets. Then label the target objects in the image, use the data labeling tool LabelImg, use this tool to label the image data and export a txt file. txt is a label file containing the label name and annotation box information of the target. After all the image data are labeled, the labeled data is randomly divided into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0045] Step S3: Improve the downsampling convolution operator to C2f_DCNv3. The present invention improves the core operator of the yolov8 model feature extraction network to a deformable convolution. Add an offset to the regular grid sampling position in the standard convolution. It can make the sampling grid deform freely, and the offset is learned from the previous feature map through the additional convolution layer. The 2D standard convolution operation is divided into two steps: 1) Sampling using a regular grid R on the input feature map x; 2) Sum the sampled values ​​weighted by w. R can be expressed as:

[0046] R={(-1,-1),,(-1,0),…,(0,1),,(1,1)} (1)

[0047] After the convolution kernel operation, for each position p0 on the output feature map y (where p n Enumerate positions in R):

[0048]

[0049] In deformable convolution, the regular grid R adds an offset Δp n :

[0050]

[0051] In order to further enhance the ability of deformable convolutional networks to manipulate spatial support areas, DCNv2 introduces a modulation mechanism, in which each sample not only undergoes a learned offset, but is also modulated by a learned feature amplitude. Through the modulation mechanism, the deformable convolutional network module can not only adjust the offset of the perceived input features, but also modulate the input feature amplitude from different spatial locations. In extreme cases, the module can decide not to perceive signals from a specific location by setting its feature amplitude to zero. Therefore, the impact of the image content from the corresponding spatial location on the module output will be significantly reduced or have no effect. However, DCNv2 is often used as an extension of regular convolution, loading the pre-trained weights of regular convolution and fine-tuning them to achieve better performance, which is not entirely suitable for large-scale visual base models that need to be trained from scratch. DCNv3 borrows the idea of ​​separable convolution and converts the original convolution weights w k It is separated into a depth part and a point-by-point part, where the depth part is composed of the original position-aware modulation scalar m k Responsible, the point-wise part is the projection weight w shared between sampling points, and a grouping mechanism is introduced.

[0052]

[0053] Where y(p0) is the convolution output of the current pixel p0. G represents the total number of aggregation groups. For the g-th group, w g Indicates the position of the group. mgk represents the offset of the k-th sample point in the g-th group, normalized along dimension k by the soft max function. g Represents a slice input feature map. p k represents the kth position of the predefined grid sampling. Δp gk is the grid sampling position p in the gth group k The corresponding offset.

[0054] Finally, the DCNv3 algorithm is introduced into the C2f module, and the feature extraction module in the model is replaced by the improved C2f_DCNv3.

[0055] Step S4: Add an improved MLCA attention mechanism. In order to improve the model's attention to the detection target and reduce the impact of image noise on the detection task, it is proposed to add an improved MLCA attention mechanism. MLCA is a lightweight local attention mechanism that takes into account channel information, spatial information, local information, and global information. Figure 3 The attention mechanism is described in detail. LAP is a local spatial downsampling operation, GAP is a full spatial downsampling operation, ReShape flattens the feature map into 1*1*C, UNAP is a deconvolution, and Conv1d is a convolution operation that does not change the size of the feature map.

[0056] First, the channel, height and width (C, H, W) of the input feature map are locally pooled and converted to a vector of (C, ks, ks), where ks is the hyperparameter step size with a default value of 5 to extract local spatial information. After the initial stage, the input is transformed using three branches to obtain spatial information of different sizes. The first branch contains global information, the second branch contains local spatial information with a step size of 2ks, and the third branch contains local spatial information with a step size of ks. The first branch transforms the feature map (C, ks, ks) into (1, 1, C) through global average pooling (GAP), and then extracts features through a 1*1 convolution operation to compress the feature channels and keep the spatial dimension unchanged. Finally, a feature map of size (C, ks, ks) is obtained through an unpooling operation. The second branch processes the feature vector into (C, ks / 2, ks / 2) through a LAP operation with a stride of 2, then exchanges spatial information through a convolution operation, and finally deconvolves into (C, ks, ks) through a UNAP operation. This branch can further exchange local spatial information, allowing the model to focus on local spatial information with different receptive field sizes. The third branch feature (C, ks, ks) is converted into a (C*ks*ks, 1, 1) feature map through a Reshape operation, and then undergoes a 1*1 convolution (Conv1d) for feature conversion to compress the feature channel and keep the spatial dimension unchanged, and finally reshapes into (C, ks, ks). So far, the local spatial information and the global spatial and channel information have been fused through the convolution operations of the three branches.

[0057] After the above steps, three feature maps (C, ks, ks) are obtained. The three feature maps are added and fused into one feature map to achieve the fusion of global context information. Then the deconvolution operation is performed to obtain the original feature map size (C, H, W). The feature map is multiplied with the input feature to highlight the useful features. The final feature map is the output result of this module.

[0058] The improved MLCA attention mechanism is integrated with the yolov8 network, and the network feature extraction layers p3, p4, and p5 are processed and then the results are output. To improve the model's attention to the detection target.

[0059] Step S5: Improve the multi-scale feature fusion module. The multi-scale feature fusion module is an important module for realizing information exchange of different feature levels in the yolov8 network. The feature maps of different levels output by the module also provide support for the model to predict targets of different sizes. The present invention improves the FPN module to RepGFPN.

[0060] The module structure diagram is as follows Figure 4 As shown, the input is feature maps of three different levels First, the feature maps of different resolutions are convolved to the same size through upsampling and downsampling operations, and then the semantic information from different levels is fused through the Fusion fusion operation. The overall structure still follows the structure of the FPN feature pyramid, that is, the feature map is first upsampled, then downsampled, and finally output The feature map is used as the target for detection head prediction.

[0061] Specifically, in the top-down feature fusion process, Through the downsampling operation and then Perform Fusion operation to generate Downsampling and Fusion Generation Next step, Upsampling and Fusion output p3 layer results Downsampling and Fusion output Downsampling and Fusion output Final output The specific fusion formula is as follows:

[0062]

[0063] Resize is an upsampling or downsampling operation. The specific structure of the Fusion module is as follows: Figure 5 As shown. The Fusion module first concatenates different input features through the concat operation, and then one branch uses a 1*1 convolution operation to obtain the output. The other branch first passes through a 1*1 convolution, and then passes through N Simplify Re p 3*3 convolution operations. The Rep 3*3 convolution operation is to use two branches for training during training, and only use the main branch during inference. This allows the model to be fully trained during the training phase, and fewer model parameters during inference to improve the model inference speed. After N Simplify Rep 3*3 convolution kernel convolution operations, each convolution result is output and finally concat through the ELAN connection idea, and the number of channels is controlled again through the 1*1 convolution kernel and the result is output.

[0064] Step S6: Calculate the loss function. The loss function of the model is composed of multiple parts. The classification loss function uses VFL Loss, the regression loss is CIOU Loss and DFL loss function, and the final loss function is the weighted sum of the above three loss functions.

[0065] Cross entropy loss, also known as logarithmic loss, is a loss function commonly used in classification problems. It measures the difference between the probability distribution predicted by the model and the actual label, and is used to measure the accuracy of the model's prediction. The VFL LOSS improved cross entropy loss function formula is as follows:

[0066]

[0067] q is the IoU (intersection over union) of bbox (prediction box) and gt (real box). IoU is the intersection of the prediction box and the real box divided by the union of the two boxes. Then p is the score, that is, the probability. The improvement of this formula is to change the basic formula of dividing by N to multiplying by q. Then the two boxes intersect, that is, q>0, which is a positive sample. If the two boxes do not intersect, let q=0, which is a negative sample. α is the loss weight of the image foreground and background, and γ is the weight of each sample.

[0068] The CIoU Loss formula is as follows:

[0069]

[0070] Among them, IoU is the intersection over union ratio, b and bgt represent the center points of the two rectangular boxes, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed area of ​​the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient.

[0071] The DFL Loss formula is as follows:

[0072] DFL(S i , S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 )) (12)

[0073]

[0074] In the above formula, S i represents the predicted probability of the category, y represents the true value, and y i With y i+1 It is the two predicted values ​​on the left and right of the true value y, that is, y i <y i+1 , i is the index of the bounding box.

[0075] Step S7: Train the model to find the best parameters. After completing the above steps, the infrared ship wake target detection model has been built. The specific structure of the model is as follows: Figure 2As shown. The prepared infrared ship wake dataset is used for model training to find the best parameter configuration. The specific training parameters of the model are 300 epochs, 8 processor cores, 16 batch sizes, 0.01 initial learning rate lr0, and 0.01 final learning rate ratio. The model is trained multiple times to adjust the learning rate and find the learning rate parameter that achieves the best effect in the test set.

[0076] Step S8: Use the trained model for the final infrared ship wake target detection task.

[0077] On the other hand, an embodiment of the present invention also provides an infrared ship wake detection system combining deformable convolution and attention mechanism, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute an infrared ship wake detection method combining deformable convolution and attention mechanism as described in the above technical solution.

[0078] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An infrared ship wake detection method combining deformable convolution and attention mechanism, characterized in that: The steps include: Step 1, collecting ship wake image data; Step 2: Use data enhancement to expand the collected image data and label the data to produce a complete infrared ship wake target detection dataset; Step 3: construct an improved tail detection model based on yolov8. The improvements include: (1) introducing the DCNv3 algorithm into the C2f module to obtain the improved C2f_DCNv3, and replacing the feature extraction module in the yolov8 model with the improved C2f_DCNv3; (2) adding an improved MLCA attention mechanism to process the yolov8 network feature extraction layers p3, p4, and p5 to improve the model's attention to the detection target; (3) improving the FPN module to the RepGFPN module, which first convolves the input feature maps of different resolutions to the same size through upsampling and downsampling operations, and then fuses the semantic information from different levels through the Fusion fusion operation; Step 4: Construct a loss function, train the wake detection model to find the optimal parameters, and use the trained model for the final infrared ship wake target detection task.

2. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 1, characterized in that: Data enhancement includes flipping, rotation, scaling, cropping, adding Gaussian noise, brightness adjustment, Mosaic data enhancement, and Mixup data enhancement.

3. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 1, characterized in that: DCNv3 converts the original convolution weight w k It is separated into a depth part and a point-by-point part, where the depth part is composed of the original position-aware modulation scalar m k The point-by-point part is the projection weight w shared between the sampling points, and a grouping mechanism is introduced, so the output of DCNv3 is expressed as: Among them, y(p0) is the convolution output result of the current pixel p0, G represents the total number of aggregation groups, and for the g-th group, w g represents the position of the group, m gk represents the offset of the kth sample point in the gth group, normalized along dimension k by the softmax function, x g represents the slice input feature map, p k represents the kth position of the predefined grid sampling, Δp gk is the grid sampling position p in the gth group k The corresponding offset.

4. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 1, characterized in that: The process of improving the MLCA attention mechanism is as follows: The input features are first locally pooled to convert the input (C, H, W) into a vector of (C, ks, ks) to extract local spatial information, where C, H, W are the channel, height and width of the input feature map, respectively, and ks is the hyperparameter step size; after the initial stage of processing, three branches are used to convert the input to obtain spatial information of different sizes: the first branch converts the feature map (C, ks, ks) into (1, 1, C) through global flat pooling, then extracts features through a 1*1 convolution operation, and finally obtains a feature map of size (C, ks, ks) through a depooling operation; the second branch processes the feature vector into (C, ks / 2, ks / 2) through a local spatial downsampling operation with a step size of 2, then exchanges spatial information through a convolution operation, and finally deconvolves into (C, ks, ks) through a deconvolution operation; the third branch feature (C, ks, ks) is converted into a (C*ks*ks, 1, 1) feature map through a Reshape operation, then extracts features through a 1*1 convolution operation, and finally reshapes to (C, ks, ks); After the above steps, three feature maps (C, ks, ks) are obtained. The three feature maps are added and fused into one feature map to achieve the fusion of global context information. Then, the deconvolution operation is performed to obtain a feature map of the size of the original feature map, which is multiplied with the input feature to highlight the useful features. The final feature map is the output result.

5. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 1, characterized in that: The processing of the RepGFPN module is as follows: The input is feature maps of three different levels First, the feature maps of different resolutions are convolved to the same size through upsampling and downsampling operations, and then the semantic information from different levels is fused through the fusion operation. The overall structure still follows the structure of the FPN feature pyramid, that is, the feature map is first upsampled, then downsampled, and finally output The feature map is used as the prediction target of the detection head; Specifically, in the top-down feature fusion process, Through the downsampling operation, and then Perform Fusion operation to generate After downsampling Fusion Generation Next step, After upsampling Fusion output p3 layer results After downsampling Fusion output After downsampling Fusion output Final output The specific fusion formula is as follows: Among them, Resize is an upsampling or downsampling operation. The Fusion Blcok module first concatenates different input features through a fusion operation, and then one of the branches uses a 1*1 convolution operation to obtain the output. The other branch first passes through a 1*1 convolution, and then passes through N Simplify Rep 3*3 convolution operations. The Simplify Rep3*3 convolution operation is to use two branches for training during training, and only use the main branch during reasoning. In this way, the model can be fully trained in the training stage, and fewer model parameters are used during reasoning to improve the model reasoning speed; after N Simplify Rep 3*3 convolution kernel convolution operations, each convolution result is output and finally fused through the ELAN connection idea, and the number of channels is controlled again through the 1*1 convolution kernel and the result is output.

6. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 1, characterized in that: The loss function of the wake detection model consists of multiple parts, among which the classification loss function uses VFL Loss, the regression loss is CIOU Loss and DFL Loss, and the final loss function is the weighted sum of the above three loss functions.

7. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 6, characterized in that: The VFL LOSS function formula is as follows: q is the intersection over union (IoU) of the predicted box bbox and the true box gt. IoU is the intersection of the predicted box and the true box divided by the union of the two boxes. Then p is the score, that is, the probability. The improvement of this formula is to change the division by N in the basic formula to multiplication by q. Then the two boxes intersect, that is, q>0, which is a positive sample. If the two boxes do not intersect, let q=0, which is a negative sample. α is the loss weight of the foreground and background of the image, and γ is the weight of each sample.

8. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 6, characterized in that: The CIoU Loss formula is as follows: Among them, IoU is the intersection over union ratio, b and bgt represent the center points of the two rectangular boxes, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed area of ​​the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient.

9. The infrared ship wake detection method combining deformable convolution and attention mechanism as claimed in claim 6, characterized in that: The DFL Loss formula is as follows: DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(y-y i )log(S i+1 )) (12) In the above formula, S i represents the predicted probability of the category, y represents the true value, and y i With y i+1 It is the two predicted values ​​on the left and right of the true value y, that is, y i <y i+1 , i is the index of the bounding box.

10. An infrared ship wake detection system combining deformable convolution and attention mechanism, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute an infrared ship wake detection method combining deformable convolution and attention mechanism as described in any one of claims 1 to 9.

Images