Marine small target ship detection method and device for wide remote sensing image

Abstract

The embodiment of the application discloses a small target ship detection method and device for wide remote sensing images, and the method comprises the following steps: acquiring a continuously input original gray image shot by a wide remote sensing satellite; preprocessing and cutting the original gray image to obtain an 8-bit RGB image; constructing a re-parameterized large convolution kernel module C2Rep; constructing a loss function PFL based on unbalanced learning; constructing a small target detection model LKPF-YOLO; and inputting the 8-bit RGB image into the small target detection model LKPF-YOLO to obtain a small target ship detection result.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method and apparatus for detecting small targets and ships at sea using wide-span remote sensing images. Background Technology

[0002] Currently, ship target detection based on satellite remote sensing imagery is in a phase of rapid development. With the continuous advancement of satellite technology and image processing algorithms, researchers have achieved significant results in the field of marine monitoring. In terms of remote sensing technology, satellite remote sensing combined with aerospace imaging technologies such as UAV reconnaissance can acquire high-resolution, high-definition ground target information, providing a rich source of image data for target detection and identification research in the field of maritime security. In terms of algorithms, the application of machine learning technologies such as deep learning and convolutional neural networks has provided new possibilities for the automated identification of ship targets. Researchers have trained models using large amounts of labeled datasets, enabling them to learn from complex satellite imagery and identify ship targets, thus improving the accuracy and efficiency of detection.

[0003] However, current research on ship target detection is mostly based on high spatial resolution remote sensing satellites. These satellites typically focus on ports, and since they don't need to monitor large areas, they are often designed with high spatial or hyperspectral resolution. Therefore, the images acquired by these satellites show ships of moderate scale and rich texture details. Besides these high spatial resolution remote sensing satellites, there are also wide-swath remote sensing satellites designed to monitor large ocean areas. Due to the vastness of the ocean, the primary purpose of these satellites is no longer to acquire clearer images, but to cover a larger area. Therefore, they are designed to increase the imaging range at the expense of other performance aspects, resulting in fewer source pixels for ships in the images and a lack of texture information. Furthermore, the sparse distribution of ships in the ocean leads to a severe imbalance between target and background samples in the images, and the complex marine environment makes ships more susceptible to cloud obscuring.

[0004] Therefore, current research struggles to achieve good performance when dealing with small target vessels at sea captured by wide-swath remote sensing satellites. Models need to be more adaptable, capable of effectively extracting and identifying features of small target vessels in wide-swath remote sensing imagery with low spatial and spectral resolution, thus achieving efficient detection. Although research on vessel detection based on machine learning methods and deep learning technologies has emerged, the main challenges in detecting small target vessels at sea remain unresolved. The current challenges in detecting small target vessels at sea based on wide-swath remote sensing imagery are summarized below:

[0005] (1) There is a lack of wide-swath remote sensing data sets for monitoring large areas of sea. Most existing ship data sets are based on high-resolution images of ports. In order to cover as much area as possible, wide-swath remote sensing satellites reduce spatial and spectral resolution, resulting in low resolution and small spectral range of remote sensing images.

[0006] (2) Ship targets in wide-span remote sensing images are small and lack texture information, typically having fewer than 200 pixels. After multiple feature extractions by the model, the network loses feature information of these small targets.

[0007] (3) The distribution of ships at sea is relatively sparse. In the entire image dataset, there is often a serious imbalance in the class frequency of target class and background class samples, which can easily lead to insufficient learning of the target class. Summary of the Invention

[0008] Based on this, it is necessary to propose a method and device for detecting small targets at sea using wide-span remote sensing imagery to address the aforementioned problems.

[0009] This invention provides a method for detecting small targets (ships) at sea using wide-swath remote sensing imagery, the method comprising:

[0010] Acquire raw grayscale images captured by wide-swath remote sensing satellites in continuous input;

[0011] The original grayscale image is preprocessed and cropped to obtain an 8-bit RGB image;

[0012] Construct a heavily parameterized large convolutional kernel module C2Rep;

[0013] Construct a loss function PFL based on imbalanced learning;

[0014] Construct a small object detection model LKPF-YOLO;

[0015] The 8-bit RGB image is input into the small target detection model LKPF-YOLO to obtain the detection results of small targets (ships) at sea.

[0016] In the above scheme, the large convolutional kernel module C2Rep is represented by the following formula:

[0017] y1 = Split[CBS(X,kernel=1)]

[0018] y n =RepConv[y n-1 kernel=(27,5)]

[0019]

[0020] Where X is the input of the C2Rep module, Z is the output of the C2Rep module, n is the number of times RepConv is stacked in the C2Rep module, Split operation is to slice the input channels according to the required number; CBS consists of 3 consecutive modules: Convolution Conv, Batch Normalization (BN), and Activation Function Silu. One CBS operation is a complete feature extraction process; RepConv is a reparameterized convolution consisting of a small kernel and a large kernel; Concat is to concatenate multiple inputs in terms of the number of channels.

[0021] In the above scheme, the construction of the loss function PFL based on imbalanced learning specifically includes: adding PFL as a factor to ObjectnessLoss and ClassificationLoss, adjusting the weights according to the ground truth box area and confidence of the current sample; LocalizationLoss, as the geometric loss, retains its original form; the improved loss function PFL is obtained by combining prior knowledge factors with QFL; and the total loss function Loss is obtained by integrating the loss function PFL into YOLOv7; the total loss function Loss is:

[0022] In the above scheme, the distribution of the true frame area in the statistical data set is used to calculate the center value obtained when the data is most concentrated, that is, when the mean absolute deviation is the smallest. Finally, the loss weight is determined based on the relationship between the current sample true frame area value and the center value.

[0023] In the above scheme, calculating the most concentrated data, which is the center value obtained when the mean absolute deviation is minimized, specifically includes: according to c = argminMAD(u), OK, MAD refers to Mean Absolute Deviation, u refers to the mean, and n is the number of true bounding boxes.

[0024] In the above scheme, the improved loss function PFL is obtained by combining prior knowledge factors with QFL.

[0025] In the above scheme, the prior knowledge factor is based on Sure.

[0026] In the above scheme, the construction of the small target detection model LKPF-YOLO specifically includes: introducing the large convolutional kernel module C2Rep into YOLOv7, replacing the ELAN-H structure in the head part with the C3 module, using PFL as the loss function, and dynamically adjusting the loss weights based on prior knowledge and the confidence of the current sample.

[0027] This invention also provides a device for detecting small targets at sea using wide-swath remote sensing imagery, the device comprising:

[0028] The image processing unit is used to acquire raw grayscale images captured by a wide-swath remote sensing satellite in continuous input; it is also used to preprocess and crop the raw grayscale images to obtain an 8-bit RGB image.

[0029] The model processing unit is used to construct a reparameterized large convolutional kernel module C2Rep; it is also used to construct a loss function PFL based on imbalanced learning; it is also used to construct a small target detection model LKPF-YOLO; and it is also used to input the 8-bit RGB image into the small target detection model LKPF-YOLO to obtain the detection results of small targets and ships at sea.

[0030] The embodiments of the present invention have the following beneficial effects:

[0031] This invention successfully constructs an improved detection model, LKPF-YOLO, based on YOLOv7. By constructing a new, reparameterized large kernel module, C2Rep, the model gains a larger effective receptive field and richer gradient flow information, thereby improving its feature extraction capabilities for small objects and low-contrast targets. Secondly, depthwise separable convolutions and the C3 module are introduced to reduce the high complexity caused by large convolutional kernels. Finally, this invention proposes a loss function, PrioriFocalLoss, based on imbalanced learning and prior knowledge. It dynamically adjusts the loss weights according to the confidence level of the current sample and uses the distribution of true bounding box sizes in the dataset to help determine the weight values, thus enabling the model to focus more on training with fewer and more difficult samples. This invention uses the above strategies to further improve the model's detection capabilities, enabling it to better meet the needs of practical applications. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] in:

[0034] Figure 1 This is a system overall block diagram of the present invention;

[0035] Figure 2 This is a structural diagram of each module of the present invention;

[0036] Figure 3 This is a diagram of the LKPF-YOLO network structure proposed in this invention;

[0037] Figure 4The transformation graphs of the mAP curves for each model;

[0038] Figure 5 This is a diagram showing the detection effect of the present invention;

[0039] Figure 6 This is a comparison chart of the detection performance of each model. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention provides a method for detecting small targets (ships) at sea using wide-swath remote sensing imagery, such as... Figure 1 As shown, the method includes:

[0042] Step 101: Acquire raw grayscale images captured by wide-swath remote sensing satellites in continuous input;

[0043] Step 102: Preprocess and crop the original grayscale image to obtain an 8-bit RGB image;

[0044] Specifically, the models in this invention only receive standard 8-bit RGB images. The color depth of raw remote sensing images is usually higher than 8 bits, and most wide-swath remote sensing satellites capture single-channel grayscale images.

[0045] This invention transforms the original high color depth image to 8 bits using a grayscale linear transformation, and then copies a single channel to three channels to ensure that the input to the model is an 8-bit RGB image.

[0046] Step 103: Construct the reparameterized large convolutional kernel module C2Rep;

[0047] Specifically, YOLOv7 uses the CBS and ELAN modules multiple times for downsampling and feature extraction, and then fuses feature maps of different scales. However, small target samples have fewer source pixels, and after the image is downsampled significantly by the model, the features are difficult to learn by the deep feature maps.

[0048] RepLKBlock is an architecture incorporating large convolutional kernels, which has demonstrated that using large kernel reparameterized convolutions in a model can produce a larger effective receptive field and higher shape bias. Reparameterized large kernel convolutions, such as... Figure 2As shown in Figure a, it mainly consists of a small kernel and a large kernel. To address the issue of larger convolutional kernels leading to more parameters, RepLKNet introduces depthwise separable convolutions, ensuring that each convolutional kernel computes only one corresponding feature map. This design allows the network to effectively avoid a dramatic increase in the number of parameters while introducing large convolutional kernels, thus maintaining the network's timeliness and improving its performance. The structure of RepLKBlock is as follows: Figure 2 As shown in b, it consists of batch normalization, two ordinary convolutions, a RepConv, and residual connections; by reparameterizing the connection between the large kernel and the small kernel, memory consumption and computational cost can be reduced while increasing the size of the convolution kernel.

[0049] C2f module ( Figure 2 d) was proposed in YOLOv8, which references C3 ( Figure 2 c) This module incorporates the principles of ELAN to design a model that obtains richer gradient flow information while maintaining lightweight design. It stacks multiple Bottleneck layers and then concatenates them to obtain even more gradient flow. However, in traditional C2f modules, the limited receptive field of ordinary convolutions can lead to low accuracy in small object detection.

[0050] Therefore, this invention proposes an improved method, namely, introducing RepLKBlock into the C2f module, which effectively captures the features of small targets by using reparameterized large convolutional kernels, thereby improving the detection capability of small targets.

[0051] The C2Rep module proposed in this invention is as follows: Figure 2 As shown in diagram e, specifically, RepLKBlock is introduced into each feature layer of the C2f structure to achieve more refined feature extraction and fusion. The C2Rep module contains multiple stacked RepLKBlocks, which serve as the main branch gradient modules (residual modules). Through reparameterized large convolutions, they obtain a larger effective receptive field, thereby fully extracting features from small objects. The introduction of RepLKBlocks allows the model to focus more on shape features in the image without significantly degrading performance due to a lack of texture information, thus maintaining excellent performance when dealing with small objects. The number of RepLKBlock stacks is controlled by the parameter n, meaning the module size can be adjusted according to the model's scale or its placement. This design, by performing convolution, upsampling, and channel concatenation on feature maps at different levels, allows the model to simultaneously obtain more contextual and high-resolution information, thereby improving the model's ability to detect small objects and low-contrast targets.

[0052] In summary, the C2rep proposed in this invention increases kernel size while reducing memory consumption and computational cost through reparameterization. Compared to traditional CNN convolutions, it has a larger receptive field and more gradient flow information. This allows the network to extract more features from small and low-contrast targets, and focuses more on shape features rather than texture features, helping the network to better locate and classify small targets.

[0053] This invention addresses the characteristics of small-scale objects and missing textures by incorporating heavily parameterized large-kernel convolutions into the original C2f module, proposing the C2Rep structure. The C2Rep structure can be represented by the following formula:

[0054] y1=Split[CBS(X,kernel=1)](1)

[0055] y n =RepConv[y n-1 ,kernel=(27,5)](2)

[0056]

[0057] Where X is the input of the C2Rep module, Z is the output of the C2Rep module, and n is the number of times RepConv is stacked in the C2Rep module. The Split operation divides the input channels into slices according to the required number; CBS consists of three consecutive modules: Conv (convolution), BN (batch normalization), and Silu (activation function). One CBS operation is a complete feature extraction process; RepConv is a reparameterized convolution consisting of a small kernel and a large kernel (…). Figure 2 a) where the size of the large kernel is 27 and the size of the small kernel is 5, which can extract small target features more effectively than the original small kernel convolution; Concat is to concatenate multiple inputs in terms of channel number. Equation 1 is the result of the Split operation after the first convolution of input X, Equation 2 is the output of each RepLKBlock stacked in the module, and Equation 3 is the output Z obtained by fusing the previous gradient flows.

[0058] Step 104: Construct the loss function PFL based on imbalanced learning.

[0059] Specifically, in ship detection tasks, neural networks typically use IoU scores to define positive and negative anchors. However, this often amplifies the imbalance problem of ship samples at sea, leading to a decline in detection performance. This is because ships are often sparsely distributed in maritime remote sensing imagery, resulting in a severe frequency imbalance between the ship target class and the ocean background class. Consequently, only a small number of prior boxes are used to detect and fit positive samples, while most prior boxes can only match negative samples. The QualityFocal Loss (QFL) function improves upon this problem by using confidence to determine the difficulty of the current sample and adjusting the loss weights accordingly.

[0060] QualityFocalLoss is a loss function improved from FocalLoss, enabling the model to achieve better localization and classification performance, and helping the model to better complete imbalanced learning tasks. Equation 4 is the binary cross-entropy loss function:

[0061]

[0062] Here, y is the label value of the sample; when y = 1, it is a positive sample, otherwise it is a negative sample. p is the probability that the model predicts a particular sample to be positive. t This is the predicted probability based on the predicted score after distinguishing between positive and negative samples. If this invention sets the two parameters α and 1-α to balance the weights of positive and negative samples, then Equation 4 can be transformed into the following form:

[0063]

[0064] To dynamically identify easy and difficult samples and adjust the weights accordingly, a coefficient (1-p) needs to be set for the above formula. t ) γ Here, γ is a hyperparameter, typically ranging from 0.5 to 1.

[0065] Equation 5 will then become the following form, which is FocalLoss:

[0066]

[0067] While retaining the advantages of FL in dealing with class imbalance problems, QFL extends FL to handle continuous label values ​​by: (1) increasing the cross-entropy part of FL to log(p t Extend to the full form -((1-y)log(1-σ)+ylog(σ))(2) and scale each sample by (1-p) γ This generalizes to the absolute value (y-σ) between the predicted and actual values. β (β>0).

[0068] This leads to the specific formula for QFL:

[0069] QFL(σ)=-|y-σ| β [(1-y)log(1-σ)+ylog(σ)](7)

[0070] Where y is the quality label from 0 to 1, and σ is the predicted score of the target detected by the model. Note that the global minimum solution of QFL is σ = y, and β is a constant with a value in (0,1), so that the cross-entropy part becomes the complete cross-entropy.

[0071] In small object detection scenarios, difficult samples are typically those with small ground truth bounding boxes. The smaller the ground truth bounding box, the less source pixel and texture information the sample contains, increasing the detection difficulty. The ground truth bounding box area distribution in the dataset is mainly concentrated around a central value. In small object detection tasks, this invention typically considers objects with ground truth bounding boxes smaller than the central value as small objects (difficult samples), those near the central value as normal-sized objects (normal samples), and those much larger than the central value as large objects (simple samples). This invention then introduces this prior knowledge to pre-set a loss weight value for each sample, making the model pay more attention to small object samples. During model training, the loss weight is then adjusted in real-time based on the confidence level of the current sample.

[0072] Based on the above ideas, this invention proposes a prior knowledge factor that focuses on the distribution of the true bounding box area of ​​samples, and uses this factor to assist the model in determining the loss weights. This factor solves the problem of insufficient adaptability and robustness caused by the overly simplistic criterion used in QFL to judge the difficulty of samples. This allows the network to more effectively handle scenarios with sparsely distributed ships at sea in remote sensing imagery.

[0073] Specifically, the factor first statistically analyzes the distribution of the true bounding box area in the dataset, then calculates the center value obtained when the data is most concentrated, i.e., when the mean absolute deviation is minimized, and finally determines a preliminary loss weight based on the relationship between the current sample's true bounding box area value and the center value. The prior knowledge factor is shown in Equation 8.

[0074]

[0075] Where k is a constant used to adjust the boundary point for adding or subtracting weights; in this invention, a value of 0.8 typically yields good results. x is the true bounding box area of ​​the current sample. c is the center value where the true bounding box area distribution of all samples is most concentrated, generally obtained when the mean absolute deviation is minimized. This invention uses all calculated true bounding box area values ​​x n Put it into a set X = {x1, x2, x3, ..., x} nThe present invention aims to obtain the value of the most concentrated distribution of elements in set X. Here, the mean absolute deviation is used to measure the degree of "concentration". That is, when c is the mean, the mean absolute deviation of set X will be minimized, as shown in Equation 9.

[0076]

[0077] Where MAD refers to Mean Absolute Deviation, u refers to Mean, and n is the number of true bounding boxes.

[0078] Furthermore, this invention combines the proposed prior knowledge factor (Equation 8) with QFL (Equation 7) to obtain the improved loss function PFL, as shown in Equation 10.

[0079]

[0080] PFL effectively addresses scenarios with sparse ship distribution and imbalanced target and background classes in remote sensing imagery. Compared to QFL, which relies solely on confidence scores, PFL achieves higher robustness by incorporating prior knowledge factors. This allows for better performance in the early stages of model training, mitigating the impact of unreliable confidence scores caused by sparse ship distribution during initial training.

[0081] YOLOv7 uses the loss function defined by ObjectnessLoss(Loss). obj ), Classification Loss(Loss) cls ) and LocalizationLoss(Loss box The weighted sum of the three losses is shown in Equation 11.

[0082] Loss = a * Loss obj +b*Loss cls +c*Loss box =a*BCE obj +b*BCE cls +c*CIoU box (11)

[0083] Where a, b, and c are the weighted proportions of the three losses, BCE is the Binary Cross-Entropy loss measure, CIoU is the intersection-union ratio evaluation metric in image classification, and both ObjectnessLoss and ClassificationLoss use the Binary Cross-Entropy loss measure. The formula is as follows:

[0084]

[0085] Where N is the number of categories, y is the true label, and p is the predicted value.

[0086] LocalizationLoss is measured by CIoU. CIoU also incorporates the overlap area between the predicted bounding box A and the ground truth bounding box B, the center distance, and the aspect ratio into the calculation, as shown in Equation 13.

[0087]

[0088] Where ρ is the distance between the center points of the predicted box A and the ground truth box B, c is the diagonal length of the minimum bounding rectangle of the predicted box A and the ground truth box B, v is the aspect ratio similarity between the predicted box A and the ground truth box B, and α is the influence factor of v.

[0089] Finally, this invention adds PFL as a factor to ObjectnessLoss and ClassificationLoss, adjusting their weights based on the ground truth box area and confidence level of the current sample. LocalizationLoss, as the geometric loss, retains its original form. Therefore, combining Equations 10 to 13, we obtain the total loss function Loss that incorporates PFL into YOLOv7:

[0090]

[0091] Step 105: Construct a new small object detection model, LKPF-YOLO.

[0092] Specifically, this invention proposes the LKPF-YOLO model, which addresses the lack of source pixels and texture details in small objects by improving the model's sensitivity to shape and enabling it to obtain richer gradient flow information. The network structure diagram of this model is shown below. Figure 3 As shown. First, the C2Rep module proposed in this study is introduced into YOLOv7. This module has a larger effective receptive field than the ELAN structure and pays more attention to shape features in the image, avoiding the significant performance degradation caused by the lack of texture information in the image. Second, the framework uses the more lightweight C3 module ( Figure 2 c) The ELAN-H structure in the head part is replaced. The C3 module has a more lightweight structure than ELAN-H and C2f, which can effectively reduce the complexity of the model. Finally, the model uses the PrioriFocalLoss (PFL) proposed in this invention as the loss function, which dynamically adjusts the loss weights based on prior knowledge and the confidence of the current sample, making the model pay more attention to difficult samples.

[0093] To meet the need for small target detection in wide-swath remote sensing imagery, this invention proposes a novel model, LKPF-YOLO, based on large kernel modules and imbalanced learning. This model solves the detection difficulties caused by the small scale and sparse distribution of ships in wide-swath satellite remote sensing imagery, achieving high-accuracy detection in complex scenarios such as densely packed ships, sparsely distributed ships, small target ships, and ships obscured by clouds.

[0094] Experimental results show that, compared with the original model, the mAP50 and mAP50:95 of this model reached 93.6% and 50.7%, respectively, representing improvements of 5.5% and 12.9%, while the number of parameters and computational cost decreased by 7% and 18.7%, respectively. This model can meet the needs of small target detection under wide-swath remote sensing imagery and has good adaptability.

[0095] Experimental data

[0096] 1. Experimental Environment and Evaluation Indicators

[0097] In this embodiment of the invention, the target detection network is trained on a server with an Intel Xeon Gold 6132 CPU and an NVIDIA Tesla V100 GPU; all other experiments are run on a PC with an Intel Core i5-8750H CPU, an NVIDIA GeForce 1060 GPU, and 16GB of memory. This embodiment first trains the improved model on the self-built "Hainan One Star" remote sensing dataset, which consists of 4027 images of 1024×1024 pixels each, containing 7602 ship instances. Then, the improved model is trained on the SSDD and DIOR public datasets to verify the generalization and adaptability of this embodiment. SSDD is a ship dataset based on SAR images, consisting of single-channel, low-resolution images. DIOR is a dataset of color optical remote sensing images containing 20 categories, offering richer scenes and more categories.

[0098] The evaluation metrics used in this invention are Precision, Recall, mAP50, and mAP50:95.

[0099] For binary classification problems, positive samples are called Positives, and negative samples are called Negatives. A correct prediction by the classifier is denoted as True, and an incorrect prediction as False. These elements combine to form the four basic elements of the confusion matrix: TP (TruePositive), FN (FalseNegative), TN (True Negative), and FP (FalsePositive). Precision refers to the ratio of correctly predicted positive cases to all positive cases, as shown in Equation 15. Recall represents the ratio of correctly predicted positive cases to all positive cases, as shown in Equation 16.

[0100]

[0101]

[0102] When the confidence level is different, this invention will also obtain different Precision and Recall. If a curve is plotted with confidence level as the independent variable and Precision and Recall as the dependent variables, then this invention obtains a Precision-Recall curve (PR curve). AP actually refers to the area under the PR curve, while mAP is the average of the AP values ​​of all classes. Among them, mAP50 specifically refers to the AP of all images in each class when IoU is set to 0.5, and then the average of all classes is calculated. mAP50:95 represents the average mAP when IoU is in a step size of 0.05, ranging from 0.5 to 0.95. The formulas for mAP50 and mAP50:95 are shown in Equations 17 and 18, respectively.

[0103]

[0104]

[0105] 2. Performance Index Analysis

[0106] To verify the effectiveness of the proposed model, the improved network was compared with the original YOLOv7 network, YOLOv7x, YOLOv7 & QFL network, SSD, Faster R-CNN, YOLOv5n, and YOLOv8n. Performance evaluation metrics included Precision, Recall, mAP50, and mAP50:95, as described above. The mAP variation curves for each network are shown below. Figure 4 As shown, the performance comparison is shown in Table 1.

[0107] Table 1 Performance Comparison of Various Networks

[0108]

[0109] from Figure 4 As can be seen, with the increase of the number of epochs, the mAP of the method in this invention gradually stabilizes, and the overall curve is higher than that of other network models. Furthermore, this invention notes that although the curve of the YOLOv7 & QFL network is higher than that of YOLOv7 at the end, its convergence speed at the beginning of training is significantly worse than other networks, and its fluctuations are larger. The high performance of YOLOv7 & QFL at the end of training is because QFL allows the model to focus on difficult samples, thus achieving better performance. However, at the beginning of training, the model has not yet fully learned the features of the samples, and the confidence level is not very reliable at this time, resulting in poor performance of YOLOv7 & QFL in the early stages of training. The LKPF-YOLO network, which incorporates prior knowledge, solves this problem better. It achieves a high convergence speed in the early stages of training, and the curve does not show excessive oscillations.

[0110] As can be seen from the data in Table 1, the model of this invention exhibits better performance compared to several mainstream models. The improved network achieves performance metrics (Precision, Recall, mAP50, mAP50:95) of 90.5%, 89.2%, 93.6%, and 50.7%, respectively, representing improvements of approximately 3.4%, 5.6%, 5.5%, and 12.9% compared to the original network. Furthermore, the Recall metric of this invention demonstrates a greater advantage than the Precision metric compared to other models, indicating that the large convolutional kernel used in this study makes the model less prone to missing small targets or targets lacking texture.

[0111] 3. Analysis of actual testing results

[0112] This invention demonstrates the recognition performance of the model in complex scenes, such as... Figure 5 As shown, the improved network performs well in detecting densely packed ships, small ships, ships of various scales, and ships obscured by clouds.

[0113] Furthermore, this invention also demonstrates and compares the detection performance of this method with other methods on the same image, such as... Figure 6As shown in the diagram, red circles indicate missed detections of ships, while blue circles indicate other objects incorrectly identified as ships. It can be seen that the network proposed in this invention can accurately identify ships in images, whether in dense ship scenes or multi-scale scenes, while other models exhibit some missed or false detections. YOLOv7 and YOLOv7&QFL both missed small ships and ships close together in dense ship scenes; in port scenes, they both misidentified long, narrow docks as ships; and in foggy scenes, YOLOv7&QFL also misidentified blocky clouds as ships. This invention notes that although long, narrow docks and blocky clouds are both elongated in shape, they are still quite different from ships. Ships are rugby ball-shaped, docks are rectangular, and clouds are irregularly shaped. This indicates that the misidentifications of YOLOv7 and YOLOv7&QFL are not entirely due to a lack of texture information in the image; the model's insensitivity to the shape of small objects also plays a role. For YOLOv5n and YOLOv8n, they missed many small ships. This is because the image loses feature information of small targets after being downsampled multiple times by the network.

[0114] 4. Effectiveness analysis (ablation experiment)

[0115] To verify the effectiveness of the C2Rep module and PFL proposed in this invention, ablation experiments were conducted by comparing the effects of different module combinations. The experimental results are shown in Table 2. The ELAN module is the feature extraction module used in the original YOLOv7.

[0116] Table 2 Comparison of Ablation Experiments

[0117]

[0118] As can be seen from the table, the introduction of both the C2Rep module and PFL effectively improved the network performance. After incorporating only the C2Rep module, both Recall and mAP were significantly improved, with only Precision decreasing. This indicates that the large convolutional kernels in the module played an effective role, fully extracting the features of small targets and thus reducing the missed detections of small targets. After further introducing PFL, all metrics improved, ultimately leading the original network in key performance indicators.

[0119] 5. Generalization analysis

[0120] This invention also conducted generalization experiments on the SSDD and DIOR datasets with other mainstream models. Although SSDD is a ship dataset based on SAR images, it is morphologically similar to grayscale wide-swath remote sensing images; both are single-channel, low-resolution images. DIOR is a dataset of color optical remote sensing images containing 20 categories, offering richer scenes and more categories, and most of its target samples are very small. The experimental results are shown in the table below:

[0121] Table 3 shows the experiments of each model on the SSDD dataset.

[0122]

[0123] Table 4 shows the experiments of each model on the DIOR dataset.

[0124]

[0125] Experimental results show that the proposed model outperforms other networks in terms of mAP50, achieving the best overall detection performance. Among all metrics, it only slightly lags behind YOLOv8n in Recall on the SSDD dataset and mAP50:95 on the DIOR dataset. These results demonstrate that the advantages of the proposed model are not limited to single-sample class grayscale remote sensing images; it also achieves competitive performance on images with multiple sample classes and multiple color channels.

[0126] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting small maritime targets in wide-swath remote sensing imagery, characterized in that, The method includes: Acquire raw grayscale images captured by wide-swath remote sensing satellites in continuous input; The original grayscale image is preprocessed and cropped to obtain an 8-bit RGB image; Construct a heavily parameterized large convolutional kernel module C2Rep; Construct a loss function PFL based on imbalanced learning; Construct a small object detection model LKPF-YOLO; The 8-bit RGB image is input into the small target detection model LKPF-YOLO to obtain the detection results of small targets (ships) at sea. The large convolutional kernel module C2Rep is represented by the following formula: Where X is the input of the C2Rep module, Z is the output of the C2Rep module, n is the number of stacks of RepConv in the C2Rep module, Split operation is to slice the input channels according to the required number; CBS consists of 3 consecutive modules: Convolution Conv, Batch Normalization (BN), and Activation Function Silu. One CBS operation is a complete feature extraction process; RepConv is a reparameterized convolution consisting of a small kernel and a large kernel, where the size of the large kernel is 27 and the size of the small kernel is 5; Concat concatenates multiple inputs along the number of channels; The construction of the imbalanced learning-based loss function PFL specifically includes: adding PFL as a factor to Objectness Loss and Classification Loss, adjusting the weights based on the ground truth bounding box area and confidence of the current sample; Localization Loss, as the geometric loss, retains its original form; the improved loss function PFL is obtained by combining prior knowledge factors with QFL; and the total loss function Loss is obtained by integrating the PFL into YOLOv7; the total loss function Loss is: Where a, b, and c are the weighted proportions of the three losses, BCE is the Binary Cross Entropy loss measure, CIoU is the intersection-union ratio evaluation metric in image classification, and both Objectness Loss and Classification Loss use the Binary Cross Entropy loss measure. The distribution of the true bounding box area in the current sample is statistically analyzed, and the center value obtained when the data is most concentrated, that is, when the mean absolute deviation is the smallest, is calculated. Finally, the loss weight is determined based on the relationship between the true bounding box area value and the center value of the current sample. The calculation of the most concentrated data, i.e., the center value obtained when the mean absolute deviation is minimized, specifically includes: according to OK, MAD refers to Mean Absolute Deviation, u refers to the mean, and n is the number of true bounding boxes; The construction of the small target detection model LKPF-YOLO specifically includes: introducing the large convolutional kernel module C2Rep into YOLOv7, replacing the ELAN-H structure in the head part with the C3 module, using PFL as the loss function, and dynamically adjusting the loss weights based on prior knowledge and the confidence of the current sample.

2. The method for detecting small maritime targets in wide-swath remote sensing imagery according to claim 1, characterized in that, Based on prior knowledge factors combined with the specific formula of QFL The improved loss function PFL is obtained. Where y is the quality label from 0 to 1, σ is the predicted score of the target detected by the model, and β is a constant with a value in (0,1).

3. The method for detecting small maritime targets in wide-swath remote sensing imagery according to claim 1, characterized in that, The aforementioned prior knowledge factors are based on Determine; where k is a constant used to adjust the boundary point for adding or subtracting weights, x is the area of ​​the ground truth box of the current sample, and c is the center value of the distribution of ground truth box areas of all samples.

4. A device for detecting small maritime targets in wide-swath remote sensing imagery, characterized in that, The apparatus is used to perform the method of claim 1; The device includes: The image processing unit is used to acquire raw grayscale images captured by a wide-swath remote sensing satellite in continuous input; it is also used to preprocess and crop the raw grayscale images to obtain an 8-bit RGB image. The model processing unit is used to construct a reparameterized large convolutional kernel module C2Rep; it is also used to construct a loss function PFL based on imbalanced learning; it is also used to construct a small target detection model LKPF-YOLO; and it is also used to input the 8-bit RGB image into the small target detection model LKPF-YOLO to obtain the detection results of small targets and ships at sea.

Images