Small ship detection method and system based on new neck network and loss function
By improving the neck network and loss function of the YOLOX model, fusing multi-layer feature information and adjusting the localization loss, the accuracy and speed problems of small vessel detection in SAR images are solved, achieving more efficient small vessel detection.
Patent Information
- Application Number
- CN202410426660.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-10
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-04-10
AI Technical Summary
Existing deep learning-based SAR ship detection methods perform poorly in small target detection, especially because small object features are contaminated by background noise and the IoU threshold is difficult to set, making it difficult to train the network and effectively detect small ships in SAR images.
An improved YOLOX model neck network and loss function are adopted, feature information at different levels is fused through the SAM module, and the positioning loss is adjusted through the gain function to enhance the monitoring signal for small vessels.
It improves the accuracy and speed of small vessel detection, effectively avoids information loss, enhances the information fusion capability of the neck network, balances the distribution of the loss function, and improves the overall performance of small vessel detection.
Smart Images

Figure CN118262299B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of small target detection in SAR images, and in particular to a small ship detection method and system based on a novel neck network and a loss function. Background Art
[0002] SAR has the advantage of being available all day and all weather. SAR ship target detection is widely used in both military and civilian fields. The traditional detection method is the constant false alarm rate (CFAR) detection method, which cannot adapt to complex background environments. In recent years, with the development of deep learning, many researchers have adopted detection methods based on deep learning and achieved very good detection results.
[0003] Object detection methods based on deep learning can be divided into single-stage and two-stage. Single-stage methods can directly predict bounding boxes based on positioning boxes. They are fast but not accurate enough. Two-stage methods first generate a large number of region proposal boxes based on the features of CNN (convolutional neural network), and then use another network to identify the proposal boxes. They are very accurate but slightly slower. Although the above methods are very advanced, they cannot adapt to the special scenario of small target detection. This situation is even worse in SAR ship detection because there are a large number of small ships in SAR images.
[0004] Small-sized objects refer to those with fewer pixels. Taking the division criteria of the COCO dataset as an example, the area of a small object is less than 1024, the area of a medium object is between 1024-9216, and the area of a large object is greater than 9216. This means that the visual features of small objects are limited, so for deep learning-based detection algorithms, the detection performance of small objects is far inferior to that of large objects.
[0005] Compared with the datasets in computer vision, the proportion of small objects in the SAR ship detection dataset is extremely high. For example, the proportions of small objects, medium objects, and large objects in the COCO dataset are 41.4%, 34.3%, and 24.2%, respectively. But in the SAR-Ship-Dataset dataset, the proportions of small, medium, and large objects are 56.5%, 43.2%, and 0.3%, respectively. Therefore, due to the large proportion of small ships in the SAR ship detection dataset, the overall performance of the detection algorithm will drop significantly.
[0006] The reason for the poor performance of small object detection is that after CNN, the features of small objects are often contaminated by background noise, making it difficult to capture the discriminative information that is critical for classification, and the small position deviation of the bounding box in the IoU (intersection over union) metric interferes with small objects more than large objects, making it difficult to find a suitable IoU threshold and provide high-quality positive and negative samples to train the network. In order to solve the above problems, commonly used methods include anchor box design, small target data expansion, loss function design, etc. Summary of the invention
[0007] In order to solve the above problems, the purpose of the present invention is to improve the detection performance of small ships by adopting multiple improvements (including neck and loss function) based on YOLOX.
[0008] In order to achieve the above technical objectives, the present application provides a small ship detection method based on a novel neck network and a loss function, comprising the following steps:
[0009] The neck network of the YOLOX model is improved. The SAM module collects and fuses information from different levels and distributes it to different levels to enhance the information fusion capability of the neck network.
[0010] at the same time,
[0011] The positioning loss of the loss function of the YOLOX model is improved, and the supervision signal of small ships will be more sufficient by adding the gain function;
[0012] Based on the improved YOLOX model, small ships in SAR images are detected.
[0013] Preferably, in the process of improving the neck network, local features and global features are obtained through the SAM module, two different convolutions are used for calculation in the global features, average pooling or bilinear interpolation is used to scale the global features, and at the end of each attention fusion, RepConv is added to further extract and fuse information.
[0014] Preferably, when the SAM module is used to improve the neck network, the SAM module consists of Conv convolution, Sigmoid activation function, Up Sampling, Down Sampling and RepConv units.
[0015] Preferably, in the process of acquiring the global features, the global features are acquired by fusing the local features, wherein the fusion of the local features is used to obtain high-resolution features that retain the information of the small ship.
[0016] Preferably, in the process of downsampling the input features, an average pooling operation is used to downsample the input features.
[0017] Preferably, in the process of improving the neck network of the YOLOX model, based on the SAM module, the features at different levels are globally fused to obtain global information, and the global information is injected into the features at different levels, so as to improve the information fusion capability of the neck network without significantly increasing the delay.
[0018] Preferably, in the process of improving the positioning loss, the gain function f(t) is expressed as:
[0019]
[0020] t=w i ×h i
[0021]
[0022] Where t represents the area of the ground truth bounding box, τ represents the ratio of small ship loss to total loss, and w represents the threshold.
[0023] The present invention discloses a small ship detection system based on a novel neck network and a loss function, comprising:
[0024] Data acquisition module, used to acquire SAR images;
[0025] The detection and recognition module is used to detect small ships in SAR images through the improved YOLOX model. The improvement process of the YOLOX model includes: improving the neck network of the YOLOX model, collecting and fusing information from different levels through the SAM module and distributing it to different levels to enhance the information fusion capability of the neck network; improving the positioning loss of the loss function of the YOLOX model, and increasing the gain function so that the supervision signal of small ships will be more sufficient.
[0026] Preferably, the detection and recognition module is also used to obtain local features and global features through a SAM module, use two different convolutions for calculation in the global features, use average pooling or bilinear interpolation to scale the global features, and add RepConv at the end of each attention fusion to further extract and fuse information.
[0027] Preferably, the detection and recognition module is also used to obtain global information by globally fusing features at different levels based on the SAM module, and inject the global information into features at different levels, so as to improve the information fusion capability of the neck network without significantly increasing the delay.
[0028] The present invention discloses the following technical effects:
[0029] The global information of the neck network newly designed in the present invention is obtained by globally integrating features at different levels and injected into features at different levels, achieving efficient information exchange and fusion. It avoids the information loss problem in FPN (FeaturePyramidNetwork), improves the fusion ability of the neck, and helps detect small targets. The newly designed loss function enhances the monitoring signal of small ships by adaptively adjusting the positioning loss gain, thereby improving the optimization effect of the small ship detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0031] Figure 1 Schematic diagram of the FPN and scale attention module of the present invention;
[0032] Figure 2 is a schematic diagram of the SAM module of the present invention;
[0033] Figure 3 is a schematic diagram of the global functional module of the present invention;
[0034] Figure 4 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION
[0035] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0036] like Figure 1-4As shown, the present invention discloses a method for detecting small-sized ship targets in SAR images based on a novel neck network and a loss function. The neck network globally integrates multiple layers of features and injects global and local information into different levels. It avoids the information loss inherent in the traditional feature pyramid network and enhances the information fusion capability without significantly increasing the delay. During the training process, the smaller the positioning loss of small ships, the greater their positioning loss gain. In this way, the supervision of small ships is strengthened in the loss function, which can make the calculation of the loss more biased towards small ships. A series of experiments were carried out on the SSDD and SAR-Ship-Dataset datasets, and the results showed that this method can greatly improve the detection performance of small ships.
[0037] The proportion of small-sized ship targets in SAR images is very large, which poses great difficulties for detection. There are few existing studies on small ship detection, and the solutions are all based on feature fusion (feature map fusion at different levels and FPN, etc.). There is a lack of in-depth discussion on the proportion of small ships in the dataset, the reasons for the detection difficulties and solutions. This leads to limited improvement in the detection of small ships. Therefore, it is necessary to adopt special strategies to improve the detection results of small ships, which is the main motivation of this invention. The proposed method is based on YOLOX, because YOLOX is anchor-free and trained from scratch, and has good performance in SAR ship detection. On this basis, the neck network and loss function are newly designed. For the neck, the scale attention module is used together with FPN to fuse and interact information of different scales. It globally integrates multiple layers of features and injects global information into higher levels, which significantly enhances the information fusion ability of the neck without significantly increasing the delay and improves the performance of different object sizes. For the loss function, the supervision of small ships is strengthened during the training process, which makes the calculation of the loss more biased towards small ships.
[0038] Deep CNN generates feature maps with different spatial resolutions, where low-level features describe finer details and have more localization information, while high-level features capture richer semantic information. Due to downsampling, the response of small ships may disappear in deeper layers. To alleviate this problem, many researchers build feature pyramids by reusing multi-scale feature maps generated by forward propagation, and then use low-resolution feature maps to detect small ships, such as FPN and its variants.
[0039] FPN and its variants can only fully integrate features from adjacent layers and can only indirectly "recursively" obtain information from other layers. It can only exchange information selected by the intermediate layers, and the unselected information is discarded during the transmission process. This will cause the information of one layer to only affect the adjacent layers, and have limited impact on other global layers. Therefore, the overall effectiveness of information fusion is limited.
[0040] In order to avoid the information loss of the traditional FPN structure during transmission, the original recursive method is abandoned and a new fusion mechanism is constructed. By using a unified module to collect and fuse information from different levels and then distribute it to different levels, it not only avoids the inherent information loss of the traditional FPN, but also enhances the information fusion capability of the neck without significantly increasing the delay.
[0041] The input of the neck consists of feature maps B2, B3, B4 and B5 extracted from the backbone, with sizes of 160×160, 80×80, 40×40 and 20×20 respectively. The SAM (Scale Attention Module) module collects and aligns the features of each level and assigns them to different levels C3, C4, C5, as Figure 1 shown.
[0042] The structure of the SAM module is as follows Figure 2 As shown in the figure, Local and Global represent local and global features respectively, Conv represents convolution, Sigmoid is the activation function, Up / Down Sampling represents upsampling and downsampling, and RepConv blocks represent RepConv units. The input is local features B2-B5 and global features. The global feature is generated by fusing the features of B2-B5. B2, B3, B4, and B5 are local features, and two different convolutions are used in the global feature for calculation. Due to the size difference between local features and global features, average pooling or bilinear interpolation is used to scale the global features to ensure proper alignment. At the end of each attention fusion, RepConv is added to further extract and fuse information.
[0043] The B2, B3, B4, and B5 features are selected for fusion to obtain high-resolution features that retain the information of small ships. The input features are downsampled using an average pooling operation to achieve consistent sizes. Aligned features can be obtained by adjusting the feature size to the smallest feature size in the group. It can effectively aggregate information while minimizing the amount of computation.
[0044] Larger feature maps can retain more positioning information, but will increase computational complexity. Smaller feature maps will lose positioning information for smaller vessels. Therefore, the present invention selects B4 as the size of feature alignment to achieve a balance between speed and accuracy.
[0045] After obtaining the fused global information, it is distributed to each level and injected using a simple attention operation to improve the detection ability of the branches.
[0046] By globally fusing features at different levels to obtain global information and injecting global information into features at different levels, efficient information exchange and fusion is achieved. Without significantly increasing latency, the SAM system significantly improves the information fusion capability of the neck and improves the model's ability to detect small ships.
[0047] During training, small ships contribute little to the total loss in most training iterations due to too few positive samples for them compared to large ships. Insufficient supervision signals for small ships lead to poor detection performance. Therefore, it is necessary to pay more attention to the loss of small ships. For object detection, the loss function usually consists of three parts: classification loss, confidence loss, and localization loss. The classification loss and confidence loss are independent of the ship size, while the localization loss is related to the ship size. In order to balance the loss distribution and alleviate the problem of insufficient supervision of small ships, a new localization loss function is proposed, which can supervise small ships more effectively and train the detection algorithm more evenly. The designed loss function is multiplied by the function before the f(t) localization loss:
[0048] Loss total =λ1Loss cls +λ2Loss obj +λ3f(t)Loss loc .
[0049] The left side of the equation is the total loss, and the three items on the right side are classification loss, likelihood loss, and positioning loss and their coefficients. In order to ensure that the loss function is more inclined to small ships, during the training iteration, the smaller the loss of small ships, the greater their loss gain f(t). Therefore, the formula of the gain function f(t) is as follows:
[0050]
[0051] in,
[0052] t=w i ×h i .
[0053]
[0054] In the formula, t represents the area of the ground truth bounding box and τ represents the ratio of small ship losses to total losses. The gain factor is It introduces τ as a feedback signal that is sensitive to the loss distribution. It adopts a nonlinear scale sensitivity coefficient to make it sensitive to scale changes. The smaller the ship size (t) and the smaller the ship loss (τ), the larger the gain coefficient.
[0055] In order to avoid the problem of negative gain, it is designed as a piecewise function. When t is less than the threshold ω, f(t) uses When it is greater than the threshold, the original scaled sensitivity gain factor (2-t) is used.
[0056] Through the above loss function, the supervision signal of small ships will be more sufficient, the training of the detection algorithm will be more balanced, and the accuracy will be steadily improved. This loss function can be easily integrated into any detection algorithm and imposes almost no additional burden during training and inference.
[0057] Verification of the technology proposed by the present invention:
[0058] The experiments were conducted on a Linux operating system using the Pytorch deep learning framework. The hardware was an Intel Xeon (Cascade Lake) Platinum 8269CY CPU @ 2.5 GHz and an NVIDIA GeForce TITAN V GPU (12 GB memory). The first 500 iterations were performed at one-third of the regular learning rate, and then the learning rate was gradually restored to 0.0025. The momentum of all models was set to 0.9 and the weight rate decay was set to 0.0025.
[0059] Comparison with state-of-the-art object detection algorithms on SSDD
[0060] In order to verify the detection performance of different detection algorithms for small, medium and large ships in SSDD, the following experiments are conducted in this section. 22 detection algorithms are selected in Table 1, and AP (average precision), APs (average precision of small-sized targets), APm (average precision of medium-sized targets), APl (average precision of large-sized targets) and FPS (frames per second) are used as evaluation indicators.
[0061] Table 1
[0062]
[0063] YOLOX has significant advantages over other detection algorithms in terms of speed and accuracy, and it does not require anchor boxes and is trained from scratch. Therefore, the method proposed in this paper is based on YOLOX. It can be observed that the proposed detection algorithm increases the AP by 0.031. APs, APm, and APl are improved by 0.04, 0.018, and 0.046, respectively. The proposed method improves the performance of large, medium, and small ships, and the improvement for small ships is considerable. For a large number of small ships, improving AP is the best method for SAR ship detection.
[0064] Two phenomena can be observed from the table, the first is fluctuation, and the second is that many detection algorithms have lower AP for large ships compared to small and medium-sized ships. Intuitively, the AP should be much lower than APm or APl, but we find that sometimes the AP is higher for large ships and sometimes the AP is higher for small ships. By analyzing the dataset, we can find the reason behind this phenomenon: in SSDD, there are fewer large ships (62.8%, 34.6% and 2.6% for small, medium and large ships respectively). Missing or successfully detecting one during the test has a significant impact on the final AP, resulting in significant fluctuations. Many large ships have the abnormal phenomenon of low AP because there are fewer large ships in the dataset and the training frequency of large ships is low, resulting in poor detection performance.
[0065] Comparison with state-of-the-art object detection algorithms on SAR-Ship-Dataset:
[0066] In order to verify the detection performance of different detection algorithms for small, medium and large ships in the SAR ship dataset, the following experiments are conducted in this section. 14 detection algorithms are selected in Table 2, and AP, AP, APm, APl and FPS are used as evaluation indicators:
[0067] Table 2
[0068]
[0069]
[0070] From Table 2, we can also find that the single-stage detection algorithm has a higher AP but a lower speed. The anchor-free detection algorithm has a better trade-off between accuracy and latency. Compared with YOLOX, the proposed method improves the average AP, AP, APm, and APl by 0.037, 0.048, 0.029, and 0.034, respectively. Two phenomena similar to SSDD can be observed on the above data. Losing or successfully detecting one during the test has a significant impact on the final AP. Sometimes it is abnormal that APl is lower than AP and APm. The reason is that there are fewer large ships in the dataset and the large ships are trained less frequently, resulting in poor detection performance.
[0071] Evaluation of different components:
[0072] The improvements of each component on the SSDD and SAR ship datasets are shown in Tables 3 and 4. The proposed bottleneck improves the AP to 0.70 and 0.674, which proves the effectiveness of using global and local information when building FPN. The proposed loss function increases the AP to 0.717 and 0.695, indicating the importance of increasing the loss weight for small ship localization.
[0073] Table 3 Ablation results of the proposed detection algorithm on SSDD
[0074]
[0075]
[0076] Table 4 Ablation results of the proposed detection algorithm on the SAR ship dataset
[0077]
[0078] From the above results, it can be found that the above experiment also proves that combining the above two strategies is effective.
[0079] Small object detection is challenging because small objects do not contain detailed information and may even disappear in deep networks. There are more small ships in the SAR ship detection dataset, so improving AP is a good choice to improve the overall performance. To the best of our knowledge, this paper is the first to systematically apply a small object detection strategy and achieve good performance. The neck effectively fuses features from different levels, and the loss function solves the problem of insufficient supervision information during training for small ships.
[0080] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0081] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0082] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A small ship detection method based on a novel neck network and loss function, characterized in that: The following steps are involved: The neck network of the YOLOX model is improved. The SAM module collects and fuses information from different levels and distributes it to different levels to enhance the information fusion capability of the neck network. The SAM module consists of Conv convolution, Sigmoid activation function, Up Sampling, Down Sampling and RepConv units; at the same time, The positioning loss of the loss function of the YOLOX model is improved, and the supervision signal of the small ship is made more sufficient by adding the gain function; In the process of improving the positioning loss, the gain function f(t) is expressed as: Where t represents the area of the ground truth bounding box, τ represents the ratio of small ship loss to total loss, and w represents the threshold; Based on the improved YOLOX model, small ships in SAR images are detected; In the process of improving the neck network, the local features and global features are obtained through the SAM module, two different convolutions are used for calculation in the global features, average pooling or bilinear interpolation is used to scale the global features, and at the end of each attention fusion, RepConv is added to further extract and fuse information.
2. The small ship detection method based on the novel neck network and loss function according to claim 1 is characterized in that: In the process of acquiring the global feature, the global feature is acquired by fusing the local features, wherein the fusion of the local features is used to obtain a high-resolution feature that retains the information of the small ship.
3. The small ship detection method based on the novel neck network and loss function according to claim 2 is characterized in that: In the process of downsampling the input features, an average pooling operation is used to downsample the input features.
4. The small ship detection method based on the novel neck network and loss function according to claim 3 is characterized by: In the process of improving the neck network of the YOLOX model, based on the SAM module, the features at different levels are globally fused to obtain global information, and the global information is injected into the features at different levels, so as to improve the information fusion capability of the neck network without significantly increasing the delay.
5. A small ship detection system based on a novel neck network and loss function, characterized in that: include: Data acquisition module, used to acquire SAR images; A detection and recognition module is used to detect small ships in the SAR image through an improved YOLOX model, wherein the improvement process of the YOLOX model includes: improving the neck network of the YOLOX model, collecting and fusing information from different levels through a SAM module and distributing it to different levels, thereby enhancing the information fusion capability of the neck network; improving the positioning loss of the loss function of the YOLOX model, and making the supervision signal of the small ship more sufficient by adding a gain function, wherein the SAM module is composed of a Conv convolution, a Sigmoid activation function, an Up Sampling upsampling, a Down Sampling downsampling, and a RepConv unit; In the process of improving the positioning loss, the gain function f(t) is expressed as: Where t represents the area of the ground truth bounding box, τ represents the ratio of small ship loss to total loss, and w represents the threshold; The detection and recognition module is also used to obtain local features and global features through the SAM module, use two different convolutions to calculate the global features, use average pooling or bilinear interpolation to scale the global features, and add RepConv at the end of each attention fusion to further extract and fuse information.
6. The small ship detection system based on the novel neck network and loss function according to claim 5 is characterized in that: The detection and recognition module is also used to obtain global information by globally fusing features at different levels based on the SAM module, and inject the global information into features at different levels, so as to improve the information fusion capability of the neck network without significantly increasing the delay.
Citation Information
Patent Citations
Unmanned ship target detection tracking method and system
CN114596335A
Display screen defect detection method based on deep learning
CN116823775A
Lightweight SAR image ship target detection method and system
CN116824376A
Water surface floating object identification method in complex scene
CN117036656A
Ship detection and tracking method in bridge area monitoring scene
CN117132951A
