SAR (Synthetic Aperture Radar) ship target detection system and method for efficient neural architecture search

Abstract

The invention discloses an SAR ship target detection system and method for efficient neural architecture search, and belongs to the technical field of SAR ship target detection, and the system comprises a super-network architecture module, a multi-path search module, a PANet module and a model construction module. The super-network architecture module is used for constructing an efficient super-network framework based on weight sharing and convolution kernel compounding by reconstructing a convolution kernel; the multi-path search module is used for constructing a multi-path search structure and converting a discrete multi-path selection problem into a continuous differentiable optimization problem through a Gumbel-Softmax function; the PANet module constructs a searchable PANet structure by unifying a backbone network and a PANet architecture search space to a joint optimization framework; the model construction module constructs a target detection model based on an efficient super-network architecture, a multi-path search structure and a searchable PANet structure, and performs SAR ship target detection by using the target detection model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of SAR ship target detection, and particularly relates to a SAR ship target detection system and method based on efficient neural architecture search. BACKGROUND

[0002] Although the performance of SAR image target detection based on deep learning technology has been significantly improved, the fixed model architecture may not be optimal when facing different data interpretation requirements, and cannot adapt to dynamic data interpretation requirements, because the model designed by hand relies on expert knowledge and needs to be continuously tried and optimized.

[0003] At present, most NAS-based methods adopt the strategy of "classification first and then detection", that is, searching for a backbone network on an image classification dataset first, and then migrating the searched backbone network to the target detection task. This strategy has certain limitations, because there are significant differences between the SAR image target detection task and the image classification task. The target detection task not only needs to classify multiple targets in the image, but also needs to accurately locate the position of the target, while the classification task only needs to output the image level label. Due to the difference in task complexity, directly migrating the backbone network searched on the classification dataset to the target detection task may lead to suboptimal performance. Therefore, how to directly design an efficient and lightweight network architecture for the target detection task is still an important research direction.

[0004] In the SAR image target detection task, there are two major challenges in directly applying NAS technology: (1) long training time. SAR target detection models are usually more complex than classification models, including backbone networks, feature pyramid networks and other modules. Since the detection task needs to train the model from scratch, the training time is significantly increased, especially in the case of large amount of SAR image data; (2) high GPU memory requirement. When searching for a target detection model, especially a NAS method based on an over-parameterized super network, a large amount of GPU memory resources are required. This is a huge burden for resource-constrained edge devices. SUMMARY

[0005] The application aims to solve the problems of the prior art, and provides the following scheme:

[0006] A SAR ship target detection system based on efficient neural architecture search, comprising a super network architecture module, a multi-path search module, a PANet module and a model construction module.

[0007] The super-network architecture module is configured to build an efficient super-network framework based on weight sharing and convolution kernel composition by reconstructing convolution kernels.

[0008] The multi-path search module is configured to build a multi-path search structure and convert a discrete multi-path selection problem into a continuous differentiable optimization problem by using a Gumbel-Softmax function.

[0009] The PANet module is configured to build a searchable PANet structure by unifying a backbone network and a PANet architecture search space into a joint optimization framework.

[0010] The model construction module is configured to build a target detection model based on the efficient super-network architecture, the multi-path search structure, and the searchable PANet structure, acquire a ship SAR image, and perform SAR ship target detection by using the target detection model.

[0011] Preferably, the workflow of the super-network architecture module includes:

[0012] A weighted sum of a plurality of convolution operations on a same edge of a traditional super-network is combined into one convolution operation.

[0013] A 1x1 convolution, a 3x3 convolution, a 5x5 convolution, a 7x7 convolution, a 9x9 convolution, an 11x11 convolution, a 13x13 convolution, a 3x3 atrous convolution, a 5x5 atrous convolution, and a 7x7 atrous convolution are composed into a unified kernel by using depthwise separable convolution kernels to obtain the efficient super-network framework.

[0014] Preferably, the workflow of the multi-path search module includes:

[0015] The multi-path search structure is built based on the Gumbel-Softmax function.

[0016] When an input feature map enters a search convolution component of the multi-path search structure, weight coefficients of each candidate convolution constituting a super kernel are normalized by using the Gumbel-Softmax function to generate the weight coefficients of each convolution and dynamically adjust the weights, thereby converting the discrete multi-path selection problem into the continuous differentiable optimization problem.

[0017] Preferably, the workflow of the PANet module includes:

[0018] A traditional PANet network is built, and a BottleNeckCSP structure in the traditional PANet network is replaced with a searchable structure to obtain the searchable PANet structure.

[0019] The application further provides a SAR ship target detection method based on efficient neural architecture search.

[0020] S1. Constructing an efficient super-network framework based on weight sharing and convolution kernel composition by reconstructing convolution kernels;

[0021] S2. Constructing a multi-path search structure, and converting a discrete multi-path selection problem into a continuous differentiable optimization problem through a Gumbel-Softmax function;

[0022] S3. Unifying backbone network and PANet architecture search spaces into a joint optimization framework, and constructing a searchable PANet structure;

[0023] S4. Constructing a target detection model based on the efficient super-network framework, the multi-path search structure and the searchable PANet structure, obtaining a SAR ship image, and performing SAR ship target detection by using the target detection model.

[0024] Preferably, the S1 comprises:

[0025] Constructing a traditional super-network, and merging the weighted sum of a plurality of convolution operations on the same edge of the traditional super-network into one convolution operation;

[0026] Compositing 1*1 convolution, 3*3 convolution, 5*5 convolution, 7*7 convolution, 9*9 convolution, 11*11 convolution, 13*13 convolution, 3*3 hollow convolution, 5*5 hollow convolution and 7*7 hollow convolution into one unified kernel to obtain the efficient super-network framework.

[0027] Preferably, the S2 comprises:

[0028] Constructing the multi-path search structure based on the Gumbel-Softmax function;

[0029] When an input feature map enters a search convolution component of the multi-path search structure, the weight coefficient of each candidate convolution constituting a super kernel is normalized through the Gumbel-Softmax function to generate the weight coefficient of each convolution, and the weight is dynamically adjusted, so that the discrete multi-path selection problem is converted into a continuous differentiable optimization problem.

[0030] Preferably, the S3 comprises:

[0031] Constructing a traditional PANet network, and replacing a BottleNeckCSP structure in the traditional PANet network with a searchable structure to obtain the searchable PANet structure.

[0032] Compared with the prior art, the application has the beneficial effects that:

[0033] Through the nuclear reconstruction technology, the dynamic channel optimization strategy and the architecture of joint search backbone and FPN, the AutoSarDet realizes efficient model training and optimization under a small amount of GPU resources, and lays a solid foundation for adaptive search of lightweight high-performance SAR target detection models. In the future, the method can be further extended to other complex remote sensing image interpretation tasks, and promote the application of deep learning in the field of edge computing. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0035] Figure 1 The system structure schematic diagram of the embodiment of the present application is shown in the figure.

[0036] Figure 2 The traditional One-shot NAS and convolution kernel reconstruction method structure schematic diagram of the embodiment of the present application is shown in the figure.

[0037] Figure 3 The multi-path search structure schematic diagram based on Gumbel-Softmax of the embodiment of the present application is shown in the figure.

[0038] Figure 4 The traditional PANet network structure and the searchable PANet structure schematic diagram based on the embodiment of the present application are shown in the figure, wherein a is the traditional PANet network structure, and b is the searchable PANet structure based on.

[0039] Figure 5 The structure schematic diagram of the target detection model of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0041] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0042] Embodiment one

[0043] In the embodiment, as shown in the figure, a SAR ship target detection system of efficient neural architecture search includes a super-network architecture module, a multi-path search module, a PANet module and a model construction module. Figure 1

[0044] The super-network architecture module is used to construct an efficient super-network framework based on weight sharing and convolution kernel composition by reconstructing convolution kernels.

[0045] The workflow of the super-network architecture module includes: constructing a traditional super-network, and merging the weighted sum of several convolution operations on the same edge of the traditional super-network into one convolution operation; and composing the depth separable convolution kernels of 1x1 convolution, 3x3 convolution, 5x5 convolution, 7x7 convolution, 9x9 convolution, 11x11 convolution, 13x13 convolution, 3x3 atrous convolution, 5x5 atrous convolution and 7x7 atrous convolution into a unified kernel to obtain the efficient super-network framework.

[0046] In the embodiment, the traditional super-network uses continuous relaxation to parameterize the search space, and each edge contains multiple convolution operations with different kernel sizes. As shown in the traditional One-shot NAS in the figure, Figure 2 assuming that X and Z are the input and output of the convolution operation on a certain edge respectively, then wherein, represents the convolution operation, O is the candidate set of the convolution operation, and θ is the weight of the convolution kernel. In order to reduce the parameter amount, as shown in the convolution kernel reconstruction in the figure, the weights of different convolution operations are reused. That is, the weights of all convolution operations can be extracted from the unified weight by a binary mask M, so only one large convolution kernel needs to be trained. In addition, since the convolution is a linear operation, the weighted sum of multiple convolution operations on the same edge can be merged into one convolution operation. Therefore, the output of each edge can be simplified as the following formula: Figure 2

[0047]

[0048] wherein, M o represents the convolution operation, and α o ​​Represents the weight coefficient. The kernel reuse technology proposed in this embodiment uses a stable network structure design to combine multiple depth-separable convolution kernels such as 1×1 convolution, 3×3 convolution, 5×5 convolution, 7×7 convolution, 9×9 convolution, 11×11 convolution, 13×13 convolution, 3×3 dilated convolution, 5×5 dilated convolution, and 7×7 dilated convolution into a unified kernel without interfering with the supernet training, thereby significantly enriching the size and type of convolution kernels in the supernet. This design enables the model to adaptively meet task requirements by dynamically selecting convolution kernels of different sizes when facing different SAR interpretation tasks. Compared with the traditional supernet construction method, if the size and type of convolution kernels need to be increased, a large amount of calculation is required. The method proposed in this embodiment only introduces a slight computational overhead, which greatly reduces GPU memory usage and computational cost.

[0049] The multi-path search module is used to construct a multi-path search structure and transform the discrete multi-path selection problem into a continuous differentiable optimization problem through the Gumbel-Softmax function.

[0050] The workflow of the multi-path search module is as follows: Figure 3 As shown, it includes: building a multi-path search structure based on the Gumbel-Softmax function; when the input feature map enters the search convolution component of the multi-path search structure, the weight coefficient of each candidate convolution that makes up the super kernel is normalized by the Gumbel-Softmax function to generate the weight coefficient of each convolution, and dynamically adjusts the weight to transform the discrete multi-path selection problem into a continuously differentiable optimization problem.

[0051] In this embodiment, the formula of the Gumbel-Softmax function is as follows:

[0052]

[0053] Among them, α i represents the initial weight of the i-th candidate convolution, g i ~Gumbel(0,1) represents random noise sampled from the Gumbel distribution, τ represents the temperature parameter used to adjust the smoothness of the distribution, k represents the total number of candidate convolutions, and j represents a natural number. In this way, the weights of the convolutional modules in the hyperkernel can be dynamically adjusted during training, thereby achieving efficient module selection and combination. Traditional hypernets usually only select The largest convolution operation, although this strategy is simple, has obvious limitations, because it can only select a single convolution operation, while ignoring other convolution modes that may be equally important, the weights of different convolution operations The difference in values ​​is not significant, especially the differences between the top convolution operations are even smaller. This shows that relying only on the maximum weight Selecting a convolution operation based on weights may lead to suboptimal results because it fails to fully utilize the information provided by other convolution operations of similar importance. Some convolution operations perform well in capturing specific SAR features, but are ignored due to slight differences in weight values. Therefore, a single selection strategy may limit the expressiveness and generalization performance of the model. To address this problem, this embodiment proposes a strategy for fusing multiple convolution operations, that is, after the model search is completed, not only the convolution operation with the largest weight is considered, but also multiple convolution operations with high rankings are comprehensively considered and fused into the final model through dynamic selection. This method can better utilize the advantages of different convolution operations and enhance the robustness and adaptability of the model.

[0054] The PANet module constructs a searchable PANet structure by unifying the backbone network and PANet architecture search space into a joint optimization framework.

[0055] The workflow of the PANet module includes: building a traditional PANet network and replacing the BottleNeckCSP structure in the traditional PANet network with a searchable structure to obtain a searchable PANet structure.

[0056] like Figure 4 As shown in Figure 1, the traditional PANet network further extracts and fuses features from the backbone network through the BottleNeckCSP module. Compared to the single-path fusion operation of FPN, PANet's dual-path operation is more powerful in feature fusion, but it also brings higher computational complexity. PANet's computational workload is primarily concentrated in the BottleNeckCSP module, so its lightweight design is particularly important.

[0057] Considering the multi-scale characteristics of SAR image targets, this embodiment replaces the BottleNeckCSP module with a searchable module to reduce the amount of computation while improving the efficiency of feature extraction. By reusing the convolution kernel, the computational complexity can be significantly reduced while maintaining the feature fusion capability, thus better meeting the needs of SAR image processing. Unlike the multi-path search of the backbone network, the search for PANet only retains The convolution operation with the largest value is adaptively selected through NAS search. Compared to the fixed convolution operation in BottleNeckCSP, this search allows the kernel size to be dynamically adjusted based on the varying scales and resolutions of SAR image targets, further improving the accuracy and robustness of feature extraction. This design not only effectively reduces computing resource consumption but also enhances the model's adaptability to complex SAR image scenarios.

[0058] The model construction module constructs a target detection model based on an efficient super network architecture, a multi-path search structure, and a searchable PANet structure, as shown in Figure 5 As shown in the figure, a ship SAR image is obtained, and a SAR ship target detection is performed using the target detection model.

[0059] In this embodiment, the SAR target detection model searched first pre-processes the ship SAR image, then extracts multi-scale features using the efficient super network architecture, then uses the searched feature pyramid network as the Neck part, and fuses features of different scales through the top-down and bottom-up multi-path search structure and the searchable PANet structure, so that the network has the ability to be sensitive to small targets and accurate in positioning large targets. Finally, in the detection head part, candidate regions are generated based on anchor boxes, the classification branch uses Focal Loss to solve the class imbalance problem, the regression branch uses GIoU Loss to improve the positioning accuracy, and finally the detection results are output through the Soft-NMS processing dense target and multi-threshold filtering mechanism.

[0060] Embodiment two

[0061] In this embodiment, a SAR ship target detection method based on efficient neural architecture search includes the following steps:

[0062] S1. Construct an efficient super network framework based on weight sharing and convolution kernel composition by reconstructing the convolution kernel.

[0063] S1 includes: constructing a traditional super network, combining the weighted sum of several convolution operations on the same edge of the traditional super network into one convolution operation; and combining the depth separable convolution kernels of 1x1 convolution, 3x3 convolution, 5x5 convolution, 7x7 convolution, 9x9 convolution, 11x11 convolution, 13x13 convolution, 3x3 atrous convolution, 5x5 atrous convolution, and 7x7 atrous convolution into a unified kernel to obtain the efficient super network framework.

[0064] S2. Construct a multi-path search structure, and convert the discrete multi-path selection problem into a continuous differentiable optimization problem through a Gumbel-Softmax function.

[0065] S2 includes: constructing a multi-path search structure based on a Gumbel-Softmax function; when the input feature map enters the search convolution component of the multi-path search structure, the weight coefficients of each candidate convolution constituting the super kernel are normalized through the Gumbel-Softmax function to generate the weight coefficients of each convolution, and dynamically adjust the weights, converting the discrete multi-path selection problem into a continuous differentiable optimization problem.

[0066] S3. Unify the backbone network and the PANet architecture search space into a joint optimization framework, and construct a searchable PANet structure.

[0067] S3 includes: constructing a traditional PANet network, and replacing the BottleNeckCSP structure in the traditional PANet network with a searchable structure to obtain a searchable PANet structure.

[0068] S4. Construct a target detection model based on the efficient super network architecture, the multi-path search structure and the searchable PANet structure, and use the target detection model for SAR ship target detection.

[0069] Embodiment three

[0070] In this embodiment, as shown in Table 1, the search method constructed in the model search and training process shows significant efficiency, which is specifically reflected in the proximity of the model search time and the training time. In the case of setting the image resolution to 640x640 on the SSDD dataset, the model search time is 1.981 hours, and the training time is only 0.932 hours; in the case of setting the image resolution to 800x800 on the HRISD dataset, the search time is 11.449 hours, and the training time is 11.558 hours. This high consistency of search and training time shows that the search method proposed in the present application can quickly converge to the optimal model structure, avoiding the redundant calculation caused by the large search space or low efficiency strategy in the traditional method. The experimental results show that the search method proposed in the present application not only can efficiently complete the model architecture search, but also can complete the model training in a relatively short time.

[0071] Table 1

[0072]

[0073] As shown in Table 2, the embodiment further shows the performance comparison experiment of different model configurations on SSDD and HRSID. The performance comparison indicators include the parameter amount, the calculation amount, the average precision and the model inference time (the average recommendation time of each picture).

[0074] Table 2

[0075]

[0076] YOLOv5s (baseline model): On the SSDD dataset, the YOLOv5s model has a parameter quantity of 7.02M, FLOPs of 15.9G, mAP of 98.1%, and an inference time of 4.8ms; on the HRSID dataset, the model has a parameter quantity of 7.02M, FLOPs of 16.5G, mAP of 91.6%, and an inference time of 5.5ms. As a baseline model, YOLOv5s performs well in terms of accuracy and inference speed, but has a high parameter quantity and computational load.

[0077] Model (1): Only search the backbone network, the parameter quantity and FLOPs are significantly reduced (SSDD: 3.62M, 5.9G; HRSID: 2.67M, 4.4G), the mAP slightly decreases (SSDD: 97.3%; HRSID: 90.9%), but the inference time is significantly shortened (SSDD: 3.7ms; HRSID: 3.3ms). This indicates that only searching the backbone network has a significant effect on compressing the model size, but the accuracy is lower than the baseline model.

[0078] Model (2): On the basis of searching the backbone network, add multi-path search, the parameter quantity and FLOPs increase slightly (SSDD: 4.19M, 6.7G; HRSID: 3.75M, 5.7G), but the mAP increases (SSDD: 97.9%; HRSID: 91.5%), and the inference time increases slightly (SSDD: 4.1ms; HRSID: 3.9ms). This indicates that multi-path search improves the model accuracy by capturing multi-scale features, but slightly increases the computational burden.

[0079] The present application: Combining backbone network search, multi-path search and PANet optimization, the parameter quantity and FLOPs are further reduced (SSDD: 2.03M, 4.2G; HRSID: 1.45M, 3.8G), the mAP reaches the highest (SSDD: 98.2%; HRSID: 92.3%), and the inference time is the shortest (SSDD: 2.2ms; HRSID: 3.1ms). This indicates that the method of the present application achieves the best balance in model lightweight, accuracy improvement and inference speed optimization.

[0080] In summary, compared with the baseline model YOLOv5s, the method of the present application achieves significant results in model lightweight, accuracy improvement and inference speed optimization through innovative search strategies and optimization techniques, providing an efficient and reliable solution for SAR target detection tasks.

[0081] The above described embodiments are only to illustrate the preferred modes of the present application, and are not intended to limit the scope of the present application. Any modification and improvement made by those skilled in the art to the technical solutions of the present application without departing from the design spirit of the present application shall fall within the protection scope of the present application as defined by the claims.

Claims

1. An efficient neural architecture search SAR ship target detection system, characterized by: include: Supernet architecture module, multi-path search module, PANet module and model building module; The supernet architecture module is used to construct an efficient supernet framework based on weight sharing and convolution kernel compounding by reconstructing the convolution kernel; The multi-path search module is used to construct a multi-path search structure and transform the discrete multi-path selection problem into a continuous differentiable optimization problem through the Gumbel-Softmax function; The PANet module constructs a searchable PANet structure by unifying the backbone network and PANet architecture search space into a joint optimization framework; The model construction module constructs a target detection model based on the efficient supernet architecture, the multi-path search structure and the searchable PANet structure, obtains ship SAR images, and uses the target detection model to perform SAR ship target detection.

2. The SAR ship target detection system based on efficient neural architecture search according to claim 1 is characterized in that: The workflow of the supernet architecture module includes: Constructing a traditional hypernetwork, combining the weighted sums of several convolution operations on the same edge of the traditional hypernetwork into one convolution operation; The depthwise separable convolution kernels of 1×1 convolution, 3×3 convolution, 5×5 convolution, 7×7 convolution, 9×9 convolution, 11×11 convolution, 13×13 convolution, 3×3 dilated convolution, 5×5 dilated convolution, and 7×7 dilated convolution are combined into a unified kernel to obtain the efficient supernet framework.

3. The SAR ship target detection system based on efficient neural architecture search according to claim 1 is characterized in that: The workflow of the multi-path search module includes: Constructing the multi-path search structure based on the Gumbel-Softmax function; When the input feature map enters the search convolution component of the multi-path search structure, the weight coefficient of each candidate convolution constituting the super-kernel is normalized by the Gumbel-Softmax function to generate the weight coefficient of each convolution, and the weight is dynamically adjusted to transform the discrete multi-path selection problem into a continuously differentiable optimization problem.

4. The SAR ship target detection system based on efficient neural architecture search according to claim 1, characterized in that: The workflow of the PANet module includes: A traditional PANet network is constructed, and a BottleNeckCSP structure in the traditional PANet network is replaced with a searchable structure to obtain the searchable PANet structure.

5. A SAR ship target detection method based on efficient neural architecture search, wherein the method is applied to the system according to any one of claims 1 to 4, characterized in that: The following steps are involved: S1. Construct an efficient supernet framework based on weight sharing and convolution kernel compounding by reconstructing convolution kernels; S2. Construct a multi-path search structure and transform the discrete multi-path selection problem into a continuous differentiable optimization problem through the Gumbel-Softmax function. S3. Unify the backbone network and PANet architecture search space into a joint optimization framework to construct a searchable PANet structure; S4. Build a target detection model based on the efficient supernet architecture, the multi-path search structure, and the searchable PANet structure, acquire ship SAR images, and use the target detection model to perform SAR ship target detection.

6. The SAR ship target detection method based on efficient neural architecture search according to claim 5 is characterized in that: Said S1 comprises: Constructing a traditional hypernetwork, combining the weighted sums of several convolution operations on the same edge of the traditional hypernetwork into one convolution operation; The depthwise separable convolution kernels of 1×1 convolution, 3×3 convolution, 5×5 convolution, 7×7 convolution, 9×9 convolution, 11×11 convolution, 13×13 convolution, 3×3 dilated convolution, 5×5 dilated convolution, and 7×7 dilated convolution are combined into a unified kernel to obtain the efficient supernet framework.

7. The SAR ship target detection method based on efficient neural architecture search according to claim 5, characterized in that: The S2 includes: Constructing the multi-path search structure based on the Gumbel-Softmax function; When the input feature map enters the search convolution component of the multi-path search structure, the weight coefficient of each candidate convolution constituting the super-kernel is normalized by the Gumbel-Softmax function to generate the weight coefficient of each convolution, and the weight is dynamically adjusted to transform the discrete multi-path selection problem into a continuously differentiable optimization problem.

8. The SAR ship target detection method based on efficient neural architecture search according to claim 5, characterized in that: The S3 includes: A traditional PANet network is constructed, and a BottleNeckCSP structure in the traditional PANet network is replaced with a searchable structure to obtain the searchable PANet structure.

Images