Ship orientation detection system and method based on rotation convnext and enhanced feature fusion

Abstract

The invention provides a ship orientation detection system and method based on rotation convnext and enhanced feature fusion, and belongs to the technical field of ship detection. Comprising the steps of performing down-sampling processing on an image; through stride convolution operation, the resolution of the feature map is reduced, and the channel depth is increased; processing the down-sampled feature map through a rotary adaptive module; performing multi-scale feature fusion through a multi-scale feature fusion module; the bounding boxes are converted into Gaussian distribution according to the size and the rotation angle, then the overlapping degree of the distribution is calculated through a Gaussian kernel function, and the overlapping degree is converted into accurate overlapping measurement between the rotation bounding boxes; after all feature extraction, fusion and IoU calculation are completed, the system performs separation processing on different detection tasks through a decoupling output module; the module carries out separation processing on classification, bounding box regression and direction prediction tasks. According to the method, the technical defects in the aspects of rotating object detection, multi-scale feature fusion, IoU calculation and the like in the prior art are overcome.

Description

Technical Field

[0001] The invention provides a ship direction detection system and method based on rotation convnext and enhanced feature fusion, belonging to the technical field of ship detection. Background Art

[0002] Ship detection is widely used in remote sensing images, especially in maritime navigation, traffic monitoring and defense. Traditional ship detection methods mainly focus on the position detection of ships, that is, locating ships through horizontal bounding boxes (HBB). However, simply detecting the position of the ship is often not enough to meet the needs of practical applications. The system also needs to predict the direction of the ship in order to infer its future movement trajectory. Therefore, ship direction detection has gradually become a research focus.

[0003] In recent years, the oriented bounding box (OBB) detection method has gradually replaced the traditional horizontal bounding box (HBB) method, especially in scenes with complex backgrounds, dense ships, and changing directions, the oriented bounding box has a significant effect in improving detection accuracy. In addition, with the rapid development of deep learning, the combination of convolutional neural networks (CNN) and Transformer has greatly improved the overall performance of ship detection.

[0004] One of the existing technologies is H2RBox-v2, which improves the ability to detect the direction of ships in complex backgrounds by using reflection symmetry and self-supervised learning. It takes the rotation invariance of objects as the main issue and reduces the impact of rotation on detection accuracy through reflection symmetry. The self-supervised learning module can extract features from unlabeled data and reduce background interference.

[0005] Although this technology improves the ability of ship direction detection in complex backgrounds by using reflection symmetry and self-supervised learning, it still has the following shortcomings, which are exactly the problems that RACR-ShipDet of the present invention addresses and solves:

[0006] Insufficient rotation invariance: H2RBox-v2 reduces the dependence on object rotation through reflection symmetry, but it is not robust enough for the detection of objects with large angle rotation, especially in multi-rotation scenarios, the accuracy of the model decreases. This paper solves this problem by adding rotation-adaptive ConvNeXt, and improves the detection accuracy of objects with large angle rotation.

[0007] Poor multi-scale feature fusion effect: H2RBox-v2 performs poorly in detecting ships of different scales, especially when large, medium and small ships appear in the same image, its feature fusion ability is limited, resulting in poor detection effect. This paper uses the enhanced version of RepBiFPAN and a weighted feature aggregation mechanism to significantly improve the effect of multi-scale feature fusion.

[0008] High consumption of computing resources: The self-supervised learning module of H2RBox-v2 has high computational cost when processing large-scale data, especially the processing speed of images with complex background is slow. This paper reduces the computational complexity and improves the detection speed by optimizing the calculation process, especially by rotating IoU based on Gaussian distribution.

[0009] The second existing technology is RoI Transformer, which adapts the horizontal bounding box to the oriented bounding box by performing spatial transformation on the region of interest (RoI). The model first determines the object position using traditional object detection methods, and then rotates the horizontal bounding box to the actual orientation of the object through a spatial transformation mechanism.

[0010] Although this technology solves some of the challenges of ship direction detection by using a spatial transformer, it still has the following disadvantages:

[0011] High computational complexity: RoI Transformer needs to perform a large number of spatial transformation operations on the region of interest (RoIs), especially when processing high-resolution images, which consumes a lot of computing resources and leads to slow processing speed. The present invention reduces the computational steps and complexity through a more efficient weighted feature aggregation module.

[0012] Limitations of traditional IoU: RoI Transformer relies on traditional IoU calculation, but when detecting rotated objects, traditional IoU cannot accurately reflect the overlap of rotated bounding boxes, especially when the rotation angle difference is large. This paper solves the limitations of IoU in rotating object detection by adopting a rotation IoU based on Gaussian distribution, and improves the detection accuracy of rotating objects. Summary of the invention

[0013] The purpose of the present invention is to provide a ship heading detection system and method based on rotation convnext and enhanced feature fusion to address the deficiencies in the prior art, and to solve the technical defects of the prior art in rotating object detection, multi-scale feature fusion, and IoU calculation. Specifically, the technical problems to be solved by the present invention include the following:

[0014] 1. Insufficient rotation invariance

[0015] The existing technology (such as H2RBox-v2) is not robust enough when processing objects with large angle rotation, and it is difficult to maintain detection accuracy, especially in the scene where the object rotates at multiple angles, the performance decreases. The present invention improves the ConvNeXt network and introduces a rotation adaptive module (RotConvNeXt), which realizes efficient detection of objects with large angle rotation, enhances the rotation invariance of the model, and enables it to adapt to complex rotation scenes.

[0016] 2. Poor fusion of multi-scale features

[0017] The existing technology performs poorly when processing ships of different scales, especially when there are objects of different sizes in the same image, it is unable to effectively fuse multi-scale features, resulting in reduced detection accuracy. The present invention improves the ability to fuse features at different scales through an enhanced version of the RepBiFPAN module combined with a weighted feature aggregation mechanism (WFA), and can maintain high detection accuracy in complex images where small boats and large ships coexist.

[0018] 3. Limitations of traditional IoU calculation for rotation object detection

[0019] The existing technology generally relies on the traditional IoU calculation method, but in the detection of rotated objects, the traditional IoU cannot accurately measure the overlap of the rotated bounding box, especially when the angle difference is large, it will cause a large error. The present invention introduces the Gaussian distribution-based rotation IoU (GDRIoU), which solves the calculation error problem of the traditional IoU in the detection of rotated objects by modeling the bounding box as a Gaussian distribution, and improves the detection accuracy of rotated objects.

[0020] 4. The computational complexity is too high

[0021] Some existing technologies (such as RoI Transformer) consume huge computing resources when performing spatial transformation, which leads to a decrease in the processing speed of the model and makes it difficult to meet the needs of real-time detection. The present invention simplifies the calculation process, optimizes feature extraction and IoU calculation, effectively reduces the computational complexity, improves the detection efficiency, and is suitable for large-scale, real-time ship direction detection tasks.

[0022] The ship direction detection system and method based on rotation convnext and enhanced feature fusion provided by the present invention, especially for the problems of rotation invariance, multi-scale feature fusion and traditional IoU calculation limitations, proposes a ship direction detection method based on rotation adaptive ConvNeXt (RotConvNeXt) and enhanced version RepBiFPAN. By introducing a new weighted feature aggregation module (WFA) and Gaussian distribution based rotation IoU (GDRIoU), the accuracy and efficiency of ship direction detection are effectively improved, which is suitable for complex remote sensing image processing tasks.

[0023] The specific technical solution is:

[0024] The ship heading detection system based on rotation convnext and enhanced feature fusion includes:

[0025] Rotation Adaptive Module RotConvNeXt: Uses adaptive rotation convolution to enhance the ConvNeXt network, enabling it to handle large-angle and irregular object rotations and enhance the rotation invariance of the model;

[0026] Multi-scale feature fusion module RepBiFPAN: enhances the traditional path aggregation network PAN through a bidirectional pyramid structure to improve the fusion of fine-grained and global features;

[0027] Weighted Feature Aggregation Module (WFA): It fuses multi-scale features by dynamically assigning weights to different feature maps. When processing ships of different sizes in an image, it highlights important features and suppresses unimportant features, thereby improving detection accuracy.

[0028] Rotation bounding box IoU calculation module GDRIoU: Model the bounding box as a Gaussian distribution and calculate the overlapping area of ​​the rotated object;

[0029] The decoupled output module DecoupleHead separates the classification, bounding box regression and direction prediction tasks.

[0030] The ship direction detection method based on rotation convnext and enhanced feature fusion includes the following steps:

[0031] Step 1: Feature extraction;

[0032] After the input remote sensing image enters the detection system, the image is first downsampled. This process gradually reduces the resolution of the feature map and increases the channel depth through a convolution operation with a stride of 2. The formula for the downsampling operation is:

[0033] X ds =D(X; k, s, p)

[0034] Among them, X is the input feature map, D represents the downsampling operation, k, s, p are the size, stride and padding of the convolution kernel, and the output X is ds is the downsampled feature map.

[0035] Next, the downsampled feature map is further processed by the rotation adaptation module (RotConvNeXt). The rotation angle and weight of the convolution kernel are dynamically generated;

[0036] The rotation convolution process is described by the following formula:

[0037]

[0038] λ i and θ i Represents the rotation angle of the i-th convolution kernel generated by the parameter module and the weight of the i-th rotation convolution kernel, represents the convolution operation, R(θ i ) is based on the rotation angle θ i The rotation matrix of Conv(X) is the input image feature.

[0039] Step 2: Multi-scale feature fusion;

[0040] After completing the rotation feature extraction, multi-scale feature fusion is performed through the multi-scale feature fusion module (RepBiFPAN) to cope with the detection task of ships of different scales in remote sensing images.

[0041] The multi-scale feature fusion module is based on the bidirectional feature pyramid network (BiFPAN) and optimizes feature fusion through the weighted feature aggregation mechanism (WFA). The process first fuses low-scale features and high-scale features through the weighted feature aggregation mechanism using the following formula:

[0042] X1′=X1⊙σ(ReLU(W0*X0′+W1*X1))

[0043] Among them, X0′ and X1 are the feature map after upsampling and the feature map of the current scale respectively, W0 and W1 are convolution weights, σ ​​is the Sigmoid function, ReLU is used to introduce nonlinear characteristics, and finally outputs the fused feature X1′;

[0044] Next, the fused feature map is further processed to generate multi-scale feature output:

[0045] X output =W f *C(X0′, X1′, X2′)

[0046] Among them, X0′, X1′ and X2′ are the aggregations of feature maps of different scales, and W f is the convolution weight, and C(·) represents the fusion operation.

[0047] Step 3: Rotation IoU loss function based on Gaussian distribution;

[0048] First, the bounding boxes are converted to Gaussian distributions according to their size and rotation angle, and then the overlap of these distributions is calculated through the Gaussian kernel function and converted into an accurate overlap measurement between the rotated bounding boxes. The loss function consists of two parts: L smooth-L1 and L match , the total loss function L is the center point position error L smooth-L1 and matching loss L match The sum.

[0049] L=L smooth-L1 +L match

[0050] Lsmooth-L1 It is used to calculate the error between the center point of the prediction box and the target box, using the smooth-L1 loss form. It is based on the difference in the center point position |p i -t i The size of | determines the form of loss:

[0051] (i) When the error is less than the threshold β, the loss function is approximated as a quadratic loss (i.e., square error), with the goal of making the impact of small errors on the loss smoother.

[0052] (ii) When the error is greater than the threshold β, the loss becomes linear, thus avoiding the excessive impact of large errors on the overall training process.

[0053]

[0054] Where β is a threshold parameter used to distinguish between two different calculation methods of the loss function, p i represents the i-th element in the prediction vector, t i represents the i-th element in the target vector or true vector, L xy is the total loss of the difference between the entire vector p and t.

[0055] After extracting and fusing features, the IoU calculation is performed on the rotated bounding box of the ship target. The rotated bounding box IoU calculation module (GDRIoU) adopts the Gaussian distribution-based rotation IoU (GDRIoU) to calculate the overlap between bounding boxes by converting the rotated bounding box into a Gaussian distribution.

[0056] The specific process is as follows:

[0057] 1) Covariance matrix modeling:

[0058] The geometric information (position, size, rotation angle) of the rotated object is represented as a covariance matrix. The covariance matrix of the rotated bounding box ∑ p and∑ t Represent the prediction box and target box respectively, and their definitions are:

[0059]

[0060] This formula is used to calculate the covariance matrix ∑ p and∑ t , these two matrices define the 2D Gaussian distribution of the box. By replacing the size S of the rectangular box with p and S t Converted into a diagonal matrix and combined with the rotation matrix R of the rotation angle p and R t , representing each rotated rectangle as a Gaussian distribution. S p and St is the width and height of the box, R p and R t are rotation matrices constructed from the rotation angles, which integrate the orientation information of the box into the covariance matrix.

[0061] R p and R t is a rotation matrix, which is calculated based on the rotation angle θ of the prediction box and the target box. Specifically:

[0062]

[0063] This formula converts the size of the rotated bounding box and the rotation angle into covariance matrix form.

[0064] 2) Covariance update of Kalman filter

[0065] In order to combine the information of the prediction box and the target box, the idea of ​​Kalman filtering is used to update the covariance matrix of the prediction box. The specific calculation formula of Kalman gain K and the updated covariance matrix ∑ is as follows:

[0066] K=∑ p (∑ p +∑ t ) -1

[0067] ∑=∑ p -K∑ p

[0068] First, the Kalman gain K is calculated to combine the prediction and target box information, ∑ p is the covariance matrix of the prediction box, indicating the size and direction of the prediction box. t is the covariance matrix of the true box. K is the Kalman gain calculated based on these two covariance matrices, indicating the relative importance of the two.

[0069] 3) Rotation bounding box area calculation:

[0070] By calculating the determinant of the covariance matrix, the area of ​​the rotated bounding box is obtained. The areas of the predicted box and the target box are:

[0071]

[0072] det represents the determinant, and V b Represent the volume of the predicted box, the volume of the target box, and the updated volume respectively. The square root of the determinant represents the area covered by the Gaussian distribution.

[0073] 4) Matching loss L match Calculation

[0074] Matching loss L match The overlap error between the predicted box and the target box is calculated as follows:

[0075]

[0076] and are the volumes of the prediction box and the target box respectively. Formula The union of the predicted box and the target box is calculated, V b represents the updated volume, ∈ is used to prevent the denominator from being zero; exp represents the exponential function, f represents the selected function form, and a small value ∈ is added to prevent division by zero.

[0077] Step 4: Decouple the output;

[0078] After completing all feature extraction, fusion and IoU calculation, the system separates the different detection tasks through the decoupled output module (DecoupleHead). This module separates the classification, bounding box regression and direction prediction tasks:

[0079] Category prediction: Generate the category prediction result of the ship through the convolutional layer.

[0080] Bounding box prediction: Combine the results of the rotation IoU module to generate an accurate rotation bounding box.

[0081] Angle prediction: By extracting the directional information in the feature map, the rotation angle of the ship is predicted to ensure that its orientation can be accurately predicted.

[0082] Beneficial effects brought by the technical solution of the present invention:

[0083] 1. Enhanced rotation invariance;

[0084] The rotation adaptive module RotConvNeXt introduced in the present invention significantly enhances the model's detection ability for large angles and irregular rotations. On the DOTA-Ship dataset, after the introduction of RotConvNeXt, the orientation accuracy of the model increased from 95.5% of the baseline model to 96.4%, the orientation accuracy increased by 0.9%, and the mAP of the model increased from 94.2% to 94.7%, the mAP increased by 0.5%, and the orientation loss decreased by 0.56. This significant improvement proves the effectiveness of RotConvNeXt in enhancing the rotation invariance of the model.

[0085] 2. Optimized multi-scale feature fusion capability;

[0086] The present invention improves the fusion efficiency of features of different scales by introducing the weighted feature aggregation mechanism (WFA) and the enhanced RepBiFPAN module. Experimental data show that after adding WFA, the model significantly improves the detection accuracy of small boats and large ships in complex images, with mAP increased from 94.2% to 94.9%, mAP increased by 0.7%, orientation accuracy increased by 4%, and orientation loss reduced by 3.21.

[0087] 3. Improve IoU calculation for rotated objects:

[0088] After introducing the Gaussian distribution-based rotation IoU (GDRIoU), the present invention solves the problem of calculation error of traditional IoU in rotating object detection. The application of GDRIoU improves the mAP of the model to 94.7%, an increase of 0.5%. It also improves the orientation accuracy to 96.8%, an increase of 1.3%, and reduces the direction loss by 1.48. This improvement not only increases the detection accuracy, but also reduces the computational complexity and improves the efficiency.

[0089] 4. Reduce computational complexity;

[0090] The present invention effectively reduces the computational complexity and speeds up the model processing by simplifying the feature extraction and IoU calculation process, which makes the RACR-ShipDet model more suitable for large-scale, real-time ship direction detection tasks.

[0091] After combining all three modules, the model's mAP increased by 1.1%, the directional accuracy increased by 1.4%, and the directional loss decreased by 1.29. The directional loss was low when WFA was used alone, but the training convergence speed slowed down and the directional loss increased slightly after adding RotConvNeXt due to the increase in parameters. These technical improvements not only solve the core problems in the existing methods, but also bring significant performance improvements in rotating object detection, multi-scale feature fusion, and rotation IoU calculation, making the present invention have broad application prospects in the fields of remote sensing image analysis and ship monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is the overall framework diagram of the network of the present invention;

[0093] Figure 2 is a schematic diagram of the architecture of the rotation adaptive module of the present invention;

[0094] Figure 3 This is the RepBiFPAN feature fusion module structure of the present invention. DETAILED DESCRIPTION

[0095] The specific technical solution of the present invention is described in conjunction with the accompanying drawings.

[0096] Figure 1 The overall network framework diagram of the present invention includes a modular structure of a backbone network, a neck network and a detection head.

[0097] 1. Backbone network:

[0098] It consists of multiple RotConvNeXt blocks, each of which is used to extract features at different levels of the input image. After the downsampling module, the features of each layer are processed by global average pooling, and the obtained features are used as input for subsequent network layers.

[0099] 2. Neck network:

[0100] This part enhances the multi-scale features output from the backbone network through feature fusion, including modules such as downsampling, 1x1 simplified convolution, weighted feature aggregation and transposed convolution, aiming to better fuse feature maps of different resolutions.

[0101] 3. Detection head:

[0102] The final output features are further processed through multiple convolution modules to generate tasks such as target classification, bounding box prediction, and angle prediction. They are divided into three parallel paths based on the requirements of different detection tasks.

[0103] This architecture combines the advantages of multi-scale feature fusion and rotational convolution and is suitable for rotated object detection tasks.

[0104] The ship heading detection system based on rotation convnext and enhanced feature fusion of the present invention is composed of the following key modules:

[0105] Rotation Adaptive Module (RotConvNeXt): Use adaptive rotation convolution to enhance the ConvNeXt network, so that it can effectively handle large-angle and irregular object rotations and enhance the rotation invariance of the model.

[0106] Multi-scale feature fusion module RepBiFPAN: enhances the traditional path aggregation network PAN through a bidirectional pyramid structure to improve the fusion of fine-grained and global features;

[0107] Weighted Feature Aggregation Module (WFA): It fuses multi-scale features by dynamically assigning weights to different feature maps. When processing ships of different sizes in an image, it highlights important features and suppresses unimportant features, thereby improving detection accuracy.

[0108] Gaussian Distribution Based Rotation Bounding Box IoU Calculation Module (GDRIoU): The bounding box is modeled as a Gaussian distribution to more accurately calculate the overlapping area of ​​the rotated object. This method not only improves the accuracy of rotated object detection, but also reduces the computational complexity and eliminates the need for hyperparameter adjustment.

[0109] Decoupled output module (DecoupleHead) This module separates the classification, bounding box regression and direction prediction tasks to reduce mutual interference between tasks and improve detection effects.

[0110] Specifically, the ship direction detection method based on rotation convnext and enhanced feature fusion includes the following steps:

[0111] Step 1: Feature extraction;

[0112] Module used: Rotational adaptive module (RotConvNeXt);

[0113] The overall structure of the rotation adaptive module is as follows: Figure 2 As shown in the figure, the input image is processed by the downsampling module, and the convolution kernel rotation module is responsible for adaptive rotation convolution. It is then optimized through the parameter module and normalization layer, and finally through the linear layer and GELU activation function to extract rotation-invariant features.

[0114] Specific:

[0115] After the remote sensing image is input into the detection system, the image needs to be downsampled first. This process gradually reduces the resolution of the feature map and increases the channel depth through a convolution operation with a stride of 2. The purpose of this process is to reduce the spatial size of the image and reduce the amount of subsequent calculations while retaining key feature information.

[0116] The formula for the downsampling operation is:

[0117] X ds =D(X; k, s, p)

[0118] Among them, X is the input feature map, D represents the downsampling operation, k, s, p are the size, stride and padding of the convolution kernel, and the output X is ds is the downsampled feature map.

[0119] Through downsampling, the size of the input image is greatly reduced, but sufficient feature information is retained, providing a good foundation for subsequent rotation adaptive convolution operations.

[0120] Next, the downsampled feature map is further processed by the rotation adaptive convolution module (RotConvNeXt). The core of this module is to dynamically generate the rotation angle and weight of the convolution kernel to ensure strong robustness when processing rotating targets, especially when facing ships rotating in multiple directions, it can maintain high detection accuracy.

[0121] The rotation convolution process is described by the following formula:

[0122]

[0123] λ i and θ i Represents the rotation angle of the i-th convolution kernel generated by the parameter module and the weight of the i-th rotation convolution kernel, represents the convolution operation, R(θ i ) is based on the rotation angle θ i The rotation matrix of Conv(X) is the input image feature.

[0124] The design of this rotation-adaptive convolution process ensures that the model can adapt to target objects in different directions, thereby improving rotation invariance, especially in the face of complex rotation scenes, while still maintaining high detection accuracy.

[0125] Step 2: Multi-scale feature fusion;

[0126] Modules used: Multi-scale feature fusion module Multi-scale feature fusion module (RepBiFPAN);

[0127] RepBiFPAN enhances the traditional path aggregation network (PAN) through a bidirectional pyramid structure, improving the fusion of fine-grained and global features. However, RepBiFPAN does not fully consider the importance differences of features at different levels in the feature fusion part, which may affect the accuracy in more challenging detection tasks. Therefore, the weighted feature aggregation (WFA) module is introduced in this paper to emphasize key features by dynamically assigning weights while suppressing unimportant features.

[0128] Improved feature fusion module of the present invention Figure 3 This figure shows the feature fusion process of the RepBiFPAN module. Through weighted aggregation and convolution operations of multi-scale features, feature maps of different layers are fused in the connection layer after downsampling, 1x1 simplified convolution, transposed convolution and other operations, and finally the fused feature map is output.

[0129] After completing the rotation feature extraction, the system will perform multi-scale feature fusion through the enhanced RepBiFPAN module to cope with the detection task of ships of different scales in remote sensing images.

[0130] The multi-scale feature fusion module is based on the bidirectional feature pyramid network (BiFPAN) and optimizes feature fusion through the weighted feature aggregation mechanism (WFA). The process first fuses low-scale features and high-scale features through the weighted feature aggregation mechanism using the following formula:

[0131] X1′=X1⊙σ(ReLU(W0*X0′+W1*X1))

[0132] Among them, X0′ and X1 are the feature map after upsampling and the feature map of the current scale, respectively, W0 and W1 are convolution weights, σ ​​is the Sigmoid function, ReLU is used to introduce nonlinear characteristics, and finally outputs the fused feature X1′. This weighted feature aggregation mechanism ensures that key features are effectively enhanced and unimportant features are suppressed.

[0133] Next, the fused feature map is further processed to generate multi-scale feature output:

[0134] X output =W f *C(X0′, X1′, X2′)

[0135] Among them, X0′, X1′ and X2′ are the aggregations of feature maps of different scales, and W f is the convolution weight, and C(·) represents the fusion operation.

[0136] Through multi-scale feature fusion, the system can effectively handle ships of different sizes, ensuring that detection accuracy is also excellent in complex images.

[0137] Step 3: Rotation IoU loss function based on Gaussian distribution;

[0138] Module used: Rotated bounding box IoU calculation module (GDRIoU);

[0139] The traditional IoU calculation method processes the bounding box parameters independently, which makes it difficult to accurately capture the overlap of rotated objects. Therefore, the present invention specifically improves a rotation IoU loss function based on Gaussian distribution for ship detection in remote sensing images. The method first converts the bounding box into a Gaussian distribution according to the size and rotation angle, and then calculates the overlap of these distributions through the Gaussian kernel function and converts it into an accurate overlap measurement between the rotated bounding boxes. The loss function mainly consists of two parts: L smooth-L1 and L match , the total loss function L is the center point position error L smooth-L1 and matching loss L match The sum.

[0140] L=L smooth-L1 +L match

[0141] L smooth-L1 It is used to calculate the error between the center point of the prediction box and the target box, using the smooth-L1 loss form. It is based on the difference in the center point position |p i -t i| determines the form of loss: (i) When the error is less than the threshold β, the loss function is approximately quadratic loss (i.e., square error), in order to make the impact of small errors on the loss smoother. (ii) When the error is greater than the threshold β, the loss becomes linear, thereby avoiding excessive impact of large errors on the overall training process.

[0142]

[0143] Where β is a threshold parameter used to distinguish between two different calculation methods of the loss function, p i represents the i-th element in the prediction vector, t i represents the i-th element in the target vector or true vector, L xy is the total loss of the difference between the entire vector p and t.

[0144] After extracting and fusing features, it is necessary to perform IoU calculation on the rotated bounding box of the ship target. Traditional IoU calculation has great limitations when dealing with rotated objects. This paper introduces Gaussian distribution-based rotation IoU (GDRIoU), which converts the rotated bounding box into a Gaussian distribution and calculates the overlap between bounding boxes. The specific process is as follows: β represents, pi represents, ti represents, L XY express

[0145] 1) Covariance matrix modeling:

[0146] The geometric information (position, size, rotation angle) of the rotated object is represented as a covariance matrix. The covariance matrix of the rotated bounding box Σ p and Σ t Represent the prediction box and target box respectively, and their definitions are:

[0147]

[0148] This formula is used to calculate the covariance matrix Σ of the prediction box and the target box p and∑ t , these two matrices define the 2D Gaussian distribution of the box. By replacing the size S of the rectangular box with p and S t Converted into a diagonal matrix and combined with the rotation matrix R of the rotation angle p and R t , representing each rotated rectangle as a Gaussian distribution. S p and S t is the width and height of the box, R p and R t are rotation matrices constructed from the rotation angles, which integrate the orientation information of the box into the covariance matrix.

[0149] R p and Rt is a rotation matrix, which is calculated based on the rotation angle θ of the prediction box and the target box. Specifically:

[0150]

[0151] This formula can convert the size of the rotated bounding box and the rotation angle into the covariance matrix form.

[0152] 2) Covariance update of Kalman filter

[0153] In order to combine the information of the prediction frame and the target frame, the present invention uses the idea of ​​Kalman filtering to update the covariance matrix of the prediction frame. The specific calculation formula of the Kalman gain K and the updated covariance matrix ∑ is as follows:

[0154] K=∑ p (∑ p +∑ t ) -1

[0155] ∑=∑ p -K∑ p

[0156] First, the Kalman gain K is calculated, which is used to combine the prediction and target box information, ∑ p is the covariance matrix of the prediction box, indicating the size and direction of the prediction box. t is the covariance matrix of the true box. K is the Kalman gain calculated based on these two covariance matrices, indicating the relative importance of the two.

[0157] 3) Rotation bounding box area calculation:

[0158] By calculating the determinant of the covariance matrix, we can get the area of ​​the rotated bounding box. The areas of the predicted box and the target box are:

[0159]

[0160] det represents the determinant, and V b They represent the volume of the prediction box, the volume of the target box, and the updated volume (used to represent the overlapping area of ​​the prediction box after combining the target box information). The square root of the determinant represents the area covered by the Gaussian distribution, and multiplying it by 4 is to convert it to the area of ​​a rectangular box. By calculating the volume, the model can better measure the degree of overlap between the rotated boxes.

[0161] 4) Matching loss L match Calculation

[0162] Matching loss L match The overlap error between the predicted box and the target box is calculated as follows:

[0163]

[0164] V b is the updated volume, representing the intersection of the prediction and target box, and are the volumes of the prediction box and the target box respectively. Formula The union of the predicted box and the target box is calculated, V b represents the updated volume, ∈ is used to prevent the denominator from being zero; exp represents the exponential function, f represents the selected function form, and a small value ∈ is added to prevent division by zero. The calculation form of the matching loss can be selected according to different functions: logarithmic form, exponential form or linear form. This loss function pays more attention to the overlapping area of ​​the predicted box and the target box, and adjusts according to the ratio of the volume to help the model capture the overlap of the rotated target more accurately.

[0165] Step 4: Decouple the output;

[0166] Module used: Decouple Head

[0167] After completing all feature extraction, fusion and IoU calculation, the system separates different detection tasks through the decoupled output module. This module separates the classification, bounding box regression and direction prediction tasks to reduce mutual interference between tasks and improve detection results.

[0168] Category prediction: Generate the category prediction result of the ship through the convolutional layer.

[0169] Bounding box prediction: Combine the results of the rotation IoU module to generate an accurate rotation bounding box.

[0170] Angle prediction: By extracting the directional information in the feature map, the rotation angle of the ship is predicted to ensure that its orientation can be accurately predicted.

Claims

1. A ship heading detection system based on rotation convnext and enhanced feature fusion, characterized in that: include: Rotation Adaptive Module RotConvNeXt: Uses adaptive rotation convolution to enhance the ConvNeXt network, enabling it to handle large-angle and irregular object rotations and enhance the rotation invariance of the model; Multi-scale feature fusion module RepBiFPAN: enhances the traditional path aggregation network PAN through a bidirectional pyramid structure to improve the fusion of fine-grained and global features; Weighted Feature Aggregation Module (WFA): It fuses multi-scale features by dynamically assigning weights to different feature maps. When processing ships of different sizes in an image, it highlights important features and suppresses unimportant features, thereby improving detection accuracy. Rotation bounding box IoU calculation module GDRIoU: Model the bounding box as a Gaussian distribution and calculate the overlapping area of ​​the rotated object; The decoupled output module Decouple Head separates the classification, bounding box regression and direction prediction tasks.

2. A ship heading detection method based on rotation convnext and enhanced feature fusion, characterized in that: A ship heading detection system based on rotation convnext and enhanced feature fusion as described in claim 1 is adopted; The method comprises the following steps: Step 1: Feature extraction; After the input remote sensing image enters the detection system, the image is first downsampled; this process gradually reduces the resolution of the feature map and increases the channel depth through a convolution operation with a stride of 2; Next, the downsampled feature map is further processed by the rotation adaptation module RotConvNeXt; Step 2: Multi-scale feature fusion; After completing the rotation feature extraction, multi-scale feature fusion is performed through the multi-scale feature fusion module RepBiFPAN to cope with the detection task of ships of different scales in remote sensing images; Step 3: Rotation IoU loss function based on Gaussian distribution; First, the bounding boxes are converted into Gaussian distributions according to their size and rotation angle. Then, the overlap of these distributions is calculated through the Gaussian kernel function and converted into an accurate overlap measurement between the rotated bounding boxes. After extracting and fusing features, the IoU calculation is performed on the rotated bounding box of the ship target; the overlap between the bounding boxes is calculated by converting the rotated bounding box into a Gaussian distribution; Step 4: Decouple the output; After completing all feature extraction, fusion and IoU calculation, the system separates different detection tasks through the decoupled output module Decouple Head; this module separates the classification, bounding box regression and direction prediction tasks.

3. The ship direction detection method based on rotation convnext and enhanced feature fusion according to claim 2 is characterized in that: In step 1, the formula for the downsampling operation is: X ds =D(X;k,s,p) Among them, X is the input feature map, D represents the downsampling operation, k, s, p are the size, stride and padding of the convolution kernel, and the output X is ds is the downsampled feature map; The rotation convolution process is described by the following formula: λ i and θ i Represents the rotation angle of the i-th convolution kernel generated by the parameter module and the weight of the i-th rotation convolution kernel, represents the convolution operation, R(θ i ) is based on the rotation angle θ i The rotation matrix of Conv(X) is the input image feature.

4. The ship direction detection method based on rotation convnext and enhanced feature fusion according to claim 2 is characterized in that: Step 2: The multi-scale feature fusion module is based on the bidirectional feature pyramid network BiFPAN and optimizes feature fusion through the weighted feature aggregation mechanism WFA. The process first fuses low-scale features and high-scale features through the weighted feature aggregation mechanism, using the following formula: X1′=X1⊙σ(ReLU(W0*X0′+W1*X1)) Among them, X0′ and X1 are the feature map after upsampling and the feature map of the current scale respectively, W0 and W1 are convolution weights, σ ​​is the Sigmoid function, ReLU is used to introduce nonlinear characteristics, and finally outputs the fused feature X1′; Next, the fused feature map is further processed to generate multi-scale feature output: X output =W f *C(X0′,X1′,X2′) Among them, X0′, X1′ and X2′ are the aggregations of feature maps of different scales, and W f is the convolution weight, and C(·) represents the fusion operation.

5. The ship direction detection method based on rotation convnext and enhanced feature fusion according to claim 2, characterized in that: The loss function in step 3 consists of two parts, L smooth-L1 and L match , the total loss function L is the center point position error L smooth-L1 And matching loss L match of and; L=L smooth-L1 +L match L smooth-L1 It is used to calculate the error between the center point of the prediction box and the target box, using the smooth-L1 loss form; it is based on the difference in the position of the center point |p i -t i The size of | determines the form of loss: (i) When the error is less than the threshold β, the loss function is approximately a quadratic loss; (ii) When the error is greater than the threshold β, the loss becomes linear; Where β is a threshold parameter used to distinguish between two different calculation methods of the loss function, p i represents the i-th element in the prediction vector, t i represents the i-th element in the target vector or true vector, L xy is the total loss of the difference between the entire vector p and t.

6. The ship direction detection method based on rotation convnext and enhanced feature fusion according to claim 5 is characterized in that: The specific process of using the rotation IoU based on Gaussian distribution in step 3 is as follows: 1) Covariance matrix modeling: The geometric information of the rotated object is represented as a covariance matrix; the covariance matrix of the rotated bounding box ∑ p and∑ t Represent the prediction box and target box respectively, defined as: This formula is used to calculate the covariance matrix ∑ p and∑ t , these two matrices define the 2D Gaussian distribution of the box; by changing the size S of the rectangular box p and S t Converted into a diagonal matrix and combined with the rotation matrix R of the rotation angle p and R t , each rotated rectangle is represented as a Gaussian distribution; S p and S t is the width and height of the box, R p and R t are rotation matrices constructed from the rotation angles, which integrate the orientation information of the box into the covariance matrix; R p and R t is a rotation matrix, which is calculated based on the rotation angle θ of the prediction box and the target box. Specifically: This formula converts the size of the rotated bounding box and the rotation angle into the covariance matrix form; 2) Covariance update of Kalman filter In order to combine the information of the prediction box and the target box, the idea of ​​Kalman filtering is used to update the covariance matrix of the prediction box. The specific calculation formula of the Kalman gain K and the updated covariance matrix ∑ is as follows: K＝∑ p (∑ p +∑ t ) -1 ∑＝∑ p -K∑ p First, the Kalman gain K is calculated to combine the prediction and target box information, ∑ p is the covariance matrix of the prediction box, indicating the size and direction of the prediction box; ∑ t is the covariance matrix of the true box; K is the Kalman gain calculated based on the two covariance matrices, indicating the relative importance of the two; 3) Rotation bounding box area calculation: By calculating the determinant of the covariance matrix, the area of ​​the rotated bounding box is obtained; the areas of the prediction box and the target box are: det represents the determinant, and V b Respectively represent the volume of the prediction box, the volume of the target box, and the updated volume; the square root of the determinant represents the area covered by the Gaussian distribution; 4) Matching loss L match Calculation Matching loss L match The overlap error between the predicted box and the target box is calculated as follows: and are the volumes of the prediction box and the target box respectively; the formula The union of the predicted box and the target box is calculated, V b represents the updated volume, ∈ is used to prevent the denominator from being zero; exp represents the exponential function, f represents the selected function form, and a value ∈ is added to prevent division by zero.

7. The ship direction detection method based on rotation convnext and enhanced feature fusion according to claim 5, characterized in that: Step 4: Category prediction: Generate the category prediction result of the ship through the convolution layer; Bounding box prediction: Combine the results of the rotation IoU module to generate an accurate rotation bounding box; Angle prediction: By extracting the directional information in the feature map, the rotation angle of the ship is predicted to ensure that its orientation can be accurately predicted.

Images