Small-size ship detection method and system based on improved YOLOX model
By improving the backbone network and IoU of the YOLOX model, using S2D convolution and Scale IoU measurements, the problem of poor detection performance of small and medium-sized ships in SAR images is solved, significantly improving the detection accuracy and performance.
Patent Information
- Application Number
- CN202410426279.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-04-10
AI Technical Summary
The existing deep learning anchor frame-based object detection method is difficult to effectively detect small ships in SAR images, mainly due to the limited visual characteristics of small targets, loss of position information and background noise pollution, resulting in poor detection performance.
Improve the backbone network of the YOLOX model uses S2D convolution instead of convolution and pooling layers to relax the invariance of IoU, improve small ship detection performance through Scale IoU measurements, retain channel information of feature maps and reduce slight deviation sensitivity between borders sex.
Significantly improve the average accuracy of small ship detection on SSDD and SAR-Ship-Dataset, from 0.686 and 0.658 to 0.722 and 0.705, enhancing overall detection performance, especially reducing information loss and insufficient feature learning when handling small ships The problem.
Smart Images

Figure CN118537812B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of small target detection in SAR images, and in particular to a small-sized ship detection method and system based on an improved YOLOX model. Background Art
[0002] Synthetic Aperture Radar (SAR) is an all-day, all-weather sensor and an important tool for marine target monitoring. Traditional detection methods, such as Constant False Alarm Rate (CFAR), are essentially based on unsupervised image segmentation methods. Each step is optimized separately and cannot adapt to SAR images with complex backgrounds. With the emergence of deep learning in 2012, SAR ship detection has also entered the era of deep learning, and most researchers have adopted deep learning as the main solution.
[0003] Object detection methods based on deep learning can be divided into anchor-based and anchor-free. Anchor-based detection algorithms densely preset anchor boxes of different sizes on the image, and then classify and regress them to output the final prediction results. Anchor-free detection algorithms predict objects by generating bounding boxes directly from key points or center points. State-of-the-art detection algorithms are usually anchor-based, but anchor-free detection algorithms have lower computational costs.
[0004] The above deep learning detection methods cannot meet the requirements of small target detection. This situation is even worse in SAR ship detection because the proportion of small ships in SAR images is very high. Small-sized targets refer to those where the ratio of the width and height of the bounding box to the image is less than 0.1, or the ratio of the area of the bounding box to the image is less than 0.03. This means that the visual features of small objects are limited. For deep learning-based detection algorithms, the detection performance of small targets is much worse than that of large targets. For example, the AP50 of MaskR-CNN for large, medium and small objects on the MSCOCO dataset is 48.1%, 39.9% and 21.1% respectively.
[0005] Compared with the datasets in computer vision, the proportion of small targets in SAR ship detection datasets is extremely high. For example, in the SAR detection dataset SSDD, the proportions of small, medium and large targets are 62.8%, 34.6% and 2.6% respectively. In order to alleviate this problem, it is necessary to adopt a small target detection strategy to improve the SAR ship detection performance. The reason for the poor target detection performance is that the position information of small-sized targets is lost, with fewer pixels, limited size, and fewer appearance and geometric clues. In the convolutional network, due to downsampling and pooling operations, the spatial position information will be lost, which makes it difficult to locate small targets. And after CNN, the features of small objects are often contaminated by background noise, making it difficult to capture the discriminative information that is crucial for classification. Summary of the invention
[0006] In order to solve the above problems, the purpose of the present invention is to provide a SAR image small-size ship target detection technology based on a new backbone network and intersection over union ratio. On the basis of YOLOX, the backbone network and IoU are improved to improve the detection performance of small ships.
[0007] In order to achieve the above technical objectives, the present application provides a small-sized ship detection method based on an improved YOLOX model, comprising the following steps:
[0008] Based on the YOLOX model, the convolution and pooling layers of the backbone network of the YOLOX model are replaced by S2D convolution, and the invariance of the IoU of the YOLOX model is relaxed to improve the YOLOX model and generate an improved YOLOX model;
[0009] Small-sized ship targets in SAR images are detected using the improved YOLOX model.
[0010] Preferably, in the process of improving the backbone network, S2D convolution is used to downsample the feature map but retain all information in the channel dimension.
[0011] Preferably, in the process of selecting the YOLOX model, the YOLOX model is YOLOX-DarkNet-53.
[0012] Preferably, in the process of relaxing the invariance of the IoU of the YOLOX model, the improved IoU is expressed as:
[0013] Scale IoU(b1,b2)=IoU(b1,b2) p
[0014] Where b1 represents the true bounding box, b2 represents the predicted bounding box, and p represents the function of the ship size.
[0015] Preferably, in the process of obtaining the function of p, it can be expressed as:
[0016]
[0017] Where K1 and K2 are two parameters that control how to rescale, D represents the Euclidean distance between the centers of the two bounding boxes, and S represents the area of the bounding box.
[0018] Preferably, in the process of obtaining the Euclidean distance D, the Euclidean distance D is expressed as:
[0019]
[0020] In the formula, (x A ,y A ) and (x B ,yB ) represent the center coordinates of the bounding boxes of A and B respectively.
[0021] Preferably, in the process of obtaining the area S of the border, the area S of the border is expressed as:
[0022]
[0023] Where w1, h1, w2 and h2 represent the width and height of the border.
[0024] Preferably, in the process of improving the IoU, the improved IoU focuses on the position and size relationship between the center points and is insensitive to small deviations between the two bounding boxes.
[0025] The present invention discloses a small-sized ship detection system based on an improved YOLOX model, comprising:
[0026] Data input module, used for inputting SAR images;
[0027] The target detection module is used to detect small-sized ship targets in SAR images according to the improved YOLOX model. Based on the YOLOX model, the convolution and pooling layers of the backbone network of the YOLOX model are replaced by S2D convolution, and the invariance of the IoU of the YOLOX model is relaxed to improve the YOLOX model and generate an improved YOLOX model.
[0028] Preferably, the YOLOX model is YOLOX-DarkNet-53.
[0029] The present invention discloses the following technical effects:
[0030] According to the characteristics of small ships, the backbone and IoU are redesigned in the present invention. The backbone uses S2D for downsampling with little information loss. The newly designed IoU takes into account the area and center distance of the real bounding box and the predicted bounding box. Through a large number of experiments on SSDD and SAR-Ship-Dataset, it is found that compared with YOLOX, the method is improved from 0.686 and 0.658 to 0.722 and 0.705 respectively. The overall performance is improved by designing a detection strategy for small ships. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 It is a schematic diagram of the overall architecture of the model described in the present invention;
[0033] Figure 2 is a schematic diagram of the S2D module of the present invention;
[0034] Figure 3 is a schematic diagram of a backbone network with step size and pooling according to the present invention;
[0035] Figure 4 is a schematic diagram of the BL module of the present invention;
[0036] Figure 5 is a schematic diagram of the Res1 module of the present invention;
[0037] Figure 6 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION
[0038] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0039] like Figure 1-6 As shown in the figure, although advanced deep learning technology has made great progress in SAR ship detection, it is still extremely challenging to accurately detect small ships due to the limited size of small targets and few appearance and geometric clues. In order to solve this problem, the present invention proposes a small-size ship target detection method for SAR images based on a new backbone network and intersection-over-union. The backbone network uses space-to-depth convolution instead of stride and pooling. It includes a deep space and a convolution without stride, which can reduce information loss during downsampling. The proposed intersection-over-union method specifically considers the center distance and scale of small ships, reduces the sensitivity of the union intersection to the position deviation of small ships, and is conducive to the training of small ships. A series of experiments were carried out on SSDD and SAR-Ship-Dataset. The experimental results show that this method can improve the detection performance of small ships; the specific process of the method disclosed in the present invention is as follows:
[0040] Overall architecture: The overall architecture of the model based on the method proposed in this invention is as follows Figure 1 As shown, CBL represents convolution, batch normalization and Leaky ReLU activation function, S2D represents space-to-depth convolution, Res represents residual module, FPN represents feature pyramid network, Sigmoid is activation function, Conv represents convolution, and Concat represents feature aggregation operation. It is based on YOLOX, because YOLOX is anchor-free and trained from scratch, and has good performance in SAR ship detection. The present invention can design and train the backbone from scratch according to the needs of small ship detection.
[0041] The proposed method is improved in the backbone network and IoU, which are specially designed for detecting small ships in SAR images. For the backbone network, stride and pooling are replaced by S2D convolution. It can maintain the characteristics of small ships when performing downsampling operations. The S2D convolution consists of a space-to-depth layer and a stride-free convolution layer, which avoids the loss of fine-grained information and the learning of inefficient feature representations. For IoU, the metric for small ships specifically considers the center point and scale. It helps to assign labels to small ships and biases the training towards small ships.
[0042] Improvements to the backbone network:
[0043] The reason for the degradation of small ship performance is that the downsampling operations of convolution and pooling cause the features of small ships to disappear. This problem is not obvious in computer vision because most images have large objects and after downsampling, the model can still learn features well. However, when faced with a large number of small ships in SAR images, this situation becomes very serious. Since it causes information loss and insufficient feature learning, it is difficult to detect small ships. Therefore, it is necessary to use other operations instead of downsampling while ensuring that the information of small ships is not lost.
[0044] The present invention uses S2D (space-to-depth) convolutions, which replace the convolution and pooling layers. An S2D convolution is a space-to-depth layer followed by a convolution layer with no stride. The S2D layer downsamples the feature maps but retains all the information in the channel dimension, so there is no information loss. The present invention adds a convolution with no stride after each S2D to reduce (or increase) the number of channels using the learnable parameters in the added convolution layer.
[0045] Consider an arbitrary size 640×640×32 feature map, and divide the feature into 320×320×32×4. These sub-features are mapped and connected along the channel dimension to obtain a 320×320×128 feature map. Its spatial dimension is reduced and the channel dimension is increased. It adjusts the number of channels to 64 through 1×1 convolution and outputs a 320×320×64 feature map, as shown in Figure 2shown.
[0046] Based on YOLOX-DarkNet-53, the present invention replaces the original convolution with S2D convolution building blocks, such as Figure 3 shown.
[0047] CBL represents convolution, batch normalization layer and Reaky ReLU, while Res1, Res2, Res4 and Res8 represent 8 residual units respectively, and SPP stands for spatial pyramid pooling. It is a backbone network specially designed for small target detection and can be successfully trained on SAR detection dataset.
[0048] Improvements on IOU:
[0049] IoU and its improvements (such as GIoU, DIoU, CIoU, etc.) are usually used to measure the similarity between bounding boxes. We observe that when detecting small ships, they are sensitive to small deviations between the predicted bounding boxes and the true bounding boxes. Small relative motions between two small bounding boxes can lead to significant changes in IoU. In order to reduce the sensitivity of IoU for small ships, we propose a new similarity measurement method, called scaled IoU.
[0050] In fact, small ships can be regarded as points, and the importance of width and height is much lower than the position of the center point. At the same time, the IoU of the present invention is scale-invariant (scaling all coordinates of two bounding boxes with the same factor k will result in the same IoU), while the detection model is not scale-invariant. Therefore, the IoU score of small ships will be lower. To alleviate this problem, the present invention proposes scale IoU to relax the invariance of IoU, so that it can support more small ships. The newly designed intersection union ratio is defined as:
[0051] Scale IoU(b1,b2)=IoU(b1,b2) p ;
[0052] Among them, b1 represents the real box, b2 represents the predicted box, p is a function of the size of the ship, so the score is rescaled based on the size of the ship. k1 and k2 are two parameters that control how the rescaling is done. D represents the Euclidean distance between the centers of the two bounding boxes, and S represents the area of the bounding box. The expression is as follows:
[0053]
[0054]
[0055] In the formula, (x A ,y A) and represent the center coordinates of the bounding boxes of A and B respectively, while w1, h1, w2, and h2 represent the width and height of the bounding boxes. Scaled IoU takes into account both the area and center distance of the bounding box (small area means large loss, small distance means small loss). It focuses on the position and size relationship between the center points, making it more suitable for small ships. It is insensitive to small deviations between the two bounding boxes.
[0056] Experimental verification of the improved method mentioned in the present invention:
[0057] Experimental environment: The experiments were conducted on a Linux operating system using the Pytorch deep learning framework. The hardware was an Intel Xeon (Cascade Lake) Platinum 8269CY CPU @ 2.5 GHz and an NVIDIA GeForce TITAN VGPU (12 GB memory). The SGD optimizer was used for optimization, multi-scale testing was not used, and for the NMS post-processing algorithm, a fixed threshold of 0.5 was selected and the target score threshold was set to 0.05. The classification criteria for small, medium and large ships in the SAR ship detection dataset are consistent with the MS COCO dataset.
[0058] Comparison with state-of-the-art object detection algorithms on SSDD:
[0059] In order to verify the detection performance of different detection algorithms for small, medium and large ships in SSDD, the present invention conducted the following experiments. In Table 1, 22 detection algorithms were selected, and AP (average precision), APs (average precision of small-sized targets), APm (average precision of medium-sized targets), AP1 (average precision of large-sized targets) and FPS (frames per second) were used as evaluation indicators.
[0060] Table 1: Speed and accuracy comparison of YOLOX and other detection algorithms
[0061]
[0062]
[0063] As can be seen from Table 1, YOLOX has significant advantages in speed and accuracy compared with other detection algorithms, and it does not require anchor frames and is trained from scratch. Therefore, the method proposed in this paper is based on YOLOX. It can be observed that the proposed detection algorithm increases AP by 0.036. APs, APm, and APl are improved by 0.04, 0.119, and 0.048, respectively. The proposed method improves the performance of large, medium, and small ships, and the improvement for small ships is considerable. For a large number of small ships, improving AP is the best method for SAR ship detection. Intuitively, APs should be much lower than APm or APl, but the present invention finds that this is not the case in the table. It can be found that sometimes the AP of large ships is higher, and sometimes the AP of small ships is also higher. By analyzing the data set, the reason behind this phenomenon can be found: in SSDD, there are fewer large ships (62.8%, 34.6%, and 2.6% for small, medium, and large ships, respectively). Missing or successfully detecting a large-sized target during the test has a significant impact on the final AP, resulting in significant fluctuations. It can be observed from the table that many large ships have the anomaly of low AP. The reason is that there are fewer large ships in the dataset and the training frequency of large ships is low, resulting in poor detection performance.
[0064] Comparison with state-of-the-art object detection algorithms on SAR-Ship-Dataset:
[0065] In order to verify the detection performance of different detection algorithms for small, medium and large ships in the SAR ship dataset, the present invention conducted the following experiments. In Table 2, 14 detection algorithms were selected, and AP, AP, APm, AP1 and FPS were used as evaluation indicators.
[0066] Table 2 Performance of different detection algorithms on the SAR-Ship-Dataset dataset
[0067]
[0068] From Table 2, we can also find that the single-stage detection algorithm has a higher AP but a lower speed. The anchor-free detection algorithm has a better trade-off between accuracy and latency. Compared with YOLOX, the proposed method improves the average AP, AP, APm, and APl by 0.047, 0.081, 0.029, and 0.034, respectively.
[0069] Missing or successfully detecting one during testing has a significant impact on the final AP. It is normal for APm to be higher than AP. However, it is abnormal for AP1 to be lower than AP and APm. The reason is that there are fewer large ships in the dataset and large ships are less frequently trained, resulting in poor detection performance.
[0070] Effects of different components:
[0071] Improvements of each component on SSDD and SAR-Ship-Dataset datasets: YOLOX has an AP of 0.686 and 0.658. The proposed backbone increases the AP to 0.707 and 0.686, demonstrating the effectiveness of S2D in reducing information loss during downsampling. The proposed IoU increases the AP to 0.722 and 0.705, indicating that it is more suitable as a measure for small ships; From the above results, it can be found that the backbone has the most significant improvement, followed by IoU. This is because small ships are very prone to losing information during feature extraction. Taking measures to protect the information of small ships is crucial to improving detection performance. The above experiments also prove that combining the above four strategies is effective.
[0072] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
Claims
1. A small-size ship detection method based on an improved YOLOX model, characterized in that: The following steps are involved: Based on the YOLOX model, the convolution and pooling layers of the backbone network of the YOLOX model are replaced by S2D convolution, and the invariance of the IoU of the YOLOX model is relaxed to improve the YOLOX model and generate an improved YOLOX model; Small-sized ship targets in SAR images are detected using the improved YOLOX model. In the process of relaxing the invariance of the IoU of the YOLOX model, the improved IoU is expressed as: Scale IoU(b1,b2)=IoU(b1,b2) p In the formula, b1 represents the real bounding box, b2 represents the predicted bounding box; k1 and k2 are two parameters that control how to rescale, D represents the Euclidean distance between the centers of the two bounding boxes, (x A ,y A ) and (x B ,y B ) represent the center coordinates of the bounding boxes of A and B respectively, and w1, h1, w2 and h2 represent the width and height of the bounding boxes.
2. According to claim 1, the small-sized ship detection method based on the improved YOLOX model is characterized by: In the process of improving the backbone network, the S2D convolution is used to downsample the feature map but retain all the information in the channel dimension.
3. According to claim 2, the small-sized ship detection method based on the improved YOLOX model is characterized by: In the process of selecting the YOLOX model, the YOLOX model is YOLOX-DarkNet-53.
4. According to claim 1, the small-sized ship detection method based on the improved YOLOX model is characterized by: In the process of improving IoU, the improved IoU focuses on the position and size relationship between the center points and is insensitive to small deviations between the two bounding boxes.
5. Small-size ship detection system based on improved YOLOX model, characterized by: include: Data input module, used for inputting SAR images; A target detection module, used for detecting small-sized ship targets in the SAR image according to an improved YOLOX model, wherein, based on the YOLOX model, the convolution and pooling layers of the backbone network of the YOLOX model are replaced by S2D convolution, and the invariance of the IoU of the YOLOX model is relaxed, the YOLOX model is improved, and an improved YOLOX model is generated; In the process of relaxing the invariance of the IoU of the YOLOX model, the improved IoU is expressed as: Scale IoU(b1,b2)=IoU(b1,b2) p In the formula, b1 represents the real bounding box, b2 represents the predicted bounding box; k1 and k2 are two parameters that control how to rescale, D represents the Euclidean distance between the centers of the two bounding boxes, (x A ,y A ) and (x B ,y B ) represent the center coordinates of the bounding boxes of A and B respectively, and w1, h1, w2 and h2 represent the width and height of the bounding boxes.
6. According to claim 5, the small-sized ship detection system based on the improved YOLOX model is characterized by: The YOLOX model is YOLOX-DarkNet-53.
