Polar coordinate system lightweight SAR image rotating ship target detection method based on neural architecture search

Abstract

The invention discloses a neural architecture search-based polar coordinate system lightweight SAR (Synthetic Aperture Radar) image rotating ship target detection method, which comprehensively applies a neural structure search (NAS)-based search strategy to search a lightweight and efficient network structure, and realizes encoding and decoding of a ship target in a polar coordinate system. On one hand, the method takes an advanced DETR model as a search space to construct a network model with a variable structure, and adopts a weight entanglement strategy to realize automatic lightweight design of the DETR, so that the detection speed and the detection precision of the model can be effectively balanced; on the other hand, the rotating ship target is coded and decoded in a polar coordinate system, so that the problem of boundary regression discontinuity is solved. According to the method, DETR light weight and real-time detection are achieved, the detection performance is improved, simple and efficient label coding can be achieved, and a new technical thought and powerful support are provided for SAR image ship target detection in any direction.

Description

Technical Field

[0001] This invention relates to the field of SAR image processing, and more particularly to a lightweight method for detecting rotating ship targets in polar coordinate SAR images based on Neural Architecture Search (NAS). Background Technology

[0002] Synthetic Aperture Radar (SAR), as an active microwave imaging system, possesses powerful imaging capabilities. Regardless of day or night or weather conditions, it can provide high-resolution imaging of ground targets. Leveraging this advantage, SAR imagery plays a crucial role in numerous fields, such as military surveillance, marine resource management, disaster early warning, and environmental monitoring. In practical applications of SAR imagery, ship target detection holds a vital position, possessing significant military and civilian value. In the military field, real-time ship monitoring is a critical task. Accurate detection of ship targets within SAR images enables effective monitoring and tracking of enemy naval activities. This information provides strong intelligence support for military decision-making and plays an indispensable role in enhancing maritime security.

[0003] In recent years, with the continuous updates and iterations of models, there has been an increasing number of technical approaches to improve model detection performance by increasing model depth and adding extra high-performance modules. However, this approach undoubtedly increases the model size, places higher demands on the hardware resources required to run the model, and lacks certain adaptability. Therefore, a method that can balance detection accuracy and detection speed needs to be introduced. Furthermore, Transformer-based models are now emerging in large numbers. Compared with traditional CNN-based models, it employs a self-attention mechanism, enabling parallel processing of all elements in the input sequence, greatly improving training efficiency. When processing long sequence data, it calculates the association between each position in the sequence and all other positions, easily capturing long-distance dependency information. However, the complexity of each layer in the Transformer is high, and because it combines position encoding with word embeddings to represent features, it cannot accurately represent positional information. To better provide target location information, using a CNN-based detection head to assist in target localization is an effective solution.

[0004] In recent years, with the continuous development of deep learning technology, deep learning-based ship target detection methods have become increasingly mature in near-shore environments. However, existing methods still have room for improvement in balancing model detection accuracy and speed, and in adding Transformer positional information. To address this challenge, we propose a lightweight SAR image rotation ship target detection method based on Neural Architecture Search (NAS) in polar coordinates. This method aims to achieve high detection performance using the most concise model. Specifically, inspired by the high-performance DETR model, we construct a dynamic detection model based on DETR. On this basis, a One-Shot NAS search strategy is employed to achieve a balance between model detection speed and accuracy. Simultaneously, this method uses polar coordinates for encoding and decoding, which effectively solves the problem of discontinuous bounding box regression. Furthermore, an auxiliary detection head based on polar coordinates is introduced to help the model more accurately locate the ship bounding box. This method, with its lightweight model, can effectively solve the problem of the imbalance between detection speed and detection accuracy in SAR images, and enhances the position information during the encoding and decoding process, providing an innovative technical solution for ship target detection in SAR images.

[0005] In the field of object detection, the mainstream OBB representation methods suffer from problems such as high encoding and decoding complexity, discontinuous regression, and difficulty in training convergence. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a lightweight polar coordinate system SAR image rotating ship target detection method based on Neural Architecture Search (NAS). On one hand, it significantly improves the imbalance between detection speed and accuracy, enhancing the model's universality. On the other hand, it solves the problem of discontinuous bounding box regression and improves the situation where positional information is insufficient during encoding and decoding. This invention adopts the following technical solution: a lightweight polar coordinate system SAR image rotating ship target detection method based on Neural Architecture Search (NAS), comprising the following steps:

[0007] Acquire the SAR image to be detected;

[0008] A search space based on the DETR model is constructed, and a weight entanglement strategy is adopted to realize the automatic lightweight design of the DETR model, thereby realizing a structured searchable ship target detection network model for detecting the position of ships in images.

[0009] The ship target detection network model is trained to obtain a trained ship target detection network model;

[0010] The SAR image to be detected is input into a trained ship target detection network model to detect the position of the ship in the image.

[0011] Furthermore: the ship target detection network model includes:

[0012] Backbone network: used for multi-scale feature extraction of input SAR images;

[0013] Encoder: Used to extract ship features at different levels based on the output features received from the backbone network and the adjusted embedding dimension;

[0014] Decoder: Based on the different levels of ship features received from the encoder, decodes the spatial position, size, and rotation angle of a ship target in any direction in polar coordinates, outputs predicted ship target features, and uses Hungarian matching and one-to-many bounding box matching modes to predict the position of the ship target based on polar coordinate encoding.

[0015] Furthermore, the backbone network is constructed based on the Swing Transformer for dynamic model building.

[0016] The backbone network includes a multi-layer sliding window attention module for extracting features at different scales;

[0017] The process by which the backbone network extracts features from the input SAR image is as follows:

[0018] To automatically search for multi-layer sliding window attention Swin Transformer frameworks, the embedding dimension, the number of heads in multi-head attention, the MLP ratio of each stage in Swin Transformer, and the embedding dimension of the backbone output features are set as dynamic candidates.

[0019] First, the input SAR image channels are adjusted, and the image resolution is downsampled to one-quarter through a dynamic patch embedding operation. Features at different scales are obtained through a continuous dynamic sliding window attention mechanism and downsampling, so as to extract ship target features at different scales while preserving global information.

[0020] Furthermore, the process of extracting ship features at different levels based on the output features received from the backbone network and the adjusted embedding dimension is as follows:

[0021] The input features are first fed into a multi-scale variable attention mechanism to enhance the multi-scale features, and then normalization is performed.

[0022] A portion of the normalized feature layer is downsampled and then fed into a feedforward neural network to output a lightweight feature vector.

[0023] The lightweight feature vectors are then processed using a normalization layer.

[0024] Furthermore: Based on the different levels of ship features received from the encoder, the spatial position, size, and rotation angle of the ship target in any direction are decoded in polar coordinates to output predicted ship target features. The process of predicting the position of the ship target based on polar coordinate encoding using Hungarian matching and one-to-many bounding box matching modes is as follows:

[0025] The encoder output is fed into a multi-head attention mechanism to extract feature information at different levels, and then fed into a normalization layer to prevent data anomalies.

[0026] Ship bounding boxes are used to assist sampling, and normalization layers are also used to prevent data anomalies.

[0027] Enhanced results are obtained by outputting the ship target area through a directional multi-scale attention mechanism, and then a normalization layer is used to prevent data anomalies.

[0028] Finally, multi-scale feature information is further extracted through a feedforward neural network with downsampling. In the partial downsampling process, the feature with the largest scale is first reduced in scale by bilinear interpolation, and then all scale information is fed into the hidden layer for feature extraction. A normalization layer is used to prevent data anomalies.

[0029] Using Hungarian matching and one-to-many bounding box matching modes, for the directional bounding boxes obtained from regression, when the intersection-union ratio with the true label reaches or exceeds 0.5, the predicted box at the corresponding position is set as a positive sample. Prediction is performed based on the matching of bounding boxes and categories, and the position of ship targets is predicted based on polar coordinate encoding.

[0030] Furthermore, the training of the ship target detection network model is performed as follows:

[0031] A polar coordinate system-based detection head is used to assist in training the encoder, thereby improving the encoder's ability to learn potential ship features.

[0032] A polar coordinate-based auxiliary head predictive oriented bounding box (OBB) is used for auxiliary training of the decoder, with the decoder layer number as a dynamic candidate value.

[0033] Furthermore, the detection head encodes and decodes the ship bounding box based on a five-parameter polar coordinate system. On the one hand, in the last layer of the encoder, the detection head is used to output the predicted bounding box, thereby improving the encoder's ability to learn potential ship features. On the other hand, the bounding box of the encoder further assists the directional multi-scale attention mechanism in each layer of the decoder and assists the regression of the ship target bounding box of the decoder.

[0034] Furthermore, the prediction of the ship target includes the prediction of the rotated bounding box using a five-parameter encoding method based on polar coordinates; this is achieved by dividing the polar radius by the mean of the short side of the bounding box and then mapping the result to logarithmic space.

[0035] Further: The process of obtaining the enhanced result by outputting the ship target region through the directional multi-scale attention mechanism is as follows: The input is first divided into three different features according to the image size of the data source, including the corresponding query vector, position encoding and the input value in the previous layer. Then, three sets of trainable weights and biases required for the dynamic linear layer are generated. Subsequently, the trainable weights and biases are fed into the dynamic linear layer respectively.

[0036] Value = Linear(V, weight) v bias v (11)

[0037] Where V represents the input value, weight v bias v These represent the weights and offsets of the linear layer corresponding to the processed value, respectively, and Value represents the output value.

[0038] Next, ship target attention is obtained from the input query vector and position encoding to assist in the localization of ship targets:

[0039] Attention=Linear(q+query_pos,weight a bias a (12)

[0040] Where q and query_pos represent the query vector and its corresponding position code, respectively, and weight a bias a These represent the weights and offsets of the corresponding linear layer for attention, respectively, while Attention represents the output attention value.

[0041] Obtain bounding box data represented in polar coordinates based on the bounding box information in the encoder:

[0042] (c x ,c y ,r,α1,α2)=reference_points×factors (13)

[0043] Among them, (c x ,c y(r, α1, α2) represent the x-coordinate, y-coordinate, polar radius, and two polar angles within [0, π] of the center point of the bounding box, respectively. `reference_points` represents the result of the reference points in the encoder, which, along with (c... x ,c y The same as r, α1, α2), factors represent the scaling factors of the image, which normalize the data;

[0044] Get the coordinates of the four corner points:

[0045]

[0046] A rotated bounding box is constructed based on the coordinates of the four corner points, and the value vector is sampled within the box. Finally, the sampled features are multiplied by the attention weights to generate an attention-enhanced result.

[0047]

[0048] Finally, the final result is output through the dynamic linear layer:

[0049] output=Linear(output_tmp,weight o bias o (17)

[0050] The final output shows an enhanced target area for the ship.

[0051] Furthermore, the formula for calculating the loss of the predicted ship category in the encoder is as follows:

[0052]

[0053] L BCE =-(T) ESi )log(P ESi )-(1-T ESi log(1-P) ESi (19)

[0054] Where, N s Num represents the number of positive samples. query P represents the number of query vectors. ESi T represents the class score predicted by the encoder. ESi W represents the category of the label. ESi The label represents the sampling point, and β is a hyperparameter of Quality Focal Loss;

[0055] For the regression loss of bounding boxes in the encoder, L1 Loss is used to calculate the difference between the predicted bounding box and the true bounding box. The specific calculation is as follows:

[0056]

[0057] Where N represents the parameters of the positive samples, P EO T represents the predicted bounding box. EO W represents the actual bounding box. EO This represents the sample weight, with 1 for positive samples and 0 for negative samples.

[0058] This invention discloses a lightweight SAR image rotation target detection method based on Neural Architecture Search (NAS) in polar coordinates. It comprehensively utilizes a NAS-based search strategy to find a lightweight and efficient network structure and performs encoding and decoding of ship targets in polar coordinates. On one hand, this method constructs a structurally variable network model using an advanced DETR model as the search space and employs a weight entanglement strategy to achieve automatic lightweight design of DETR, effectively balancing the model's detection speed and accuracy. On the other hand, this method encodes and decodes rotating ship targets in polar coordinates to address the discontinuity problem in boundary regression.

[0059] This method not only achieves lightweight DETR and real-time detection, but also improves detection performance and enables simple and efficient label encoding, providing new technical ideas and strong support for ship target detection in SAR images from any direction.

[0060] Compared with the prior art, the present invention has the following beneficial technical effects:

[0061] 1. This invention innovatively proposes a lightweight polar coordinate system-based SAR image rotation ship target detection method based on Neural Architecture Search (NAS). This method selects DETR as the search space and employs a polar coordinate system-based directional SAR ship target detection method. This not only effectively alleviates the encoding and decoding complexity problems existing in mainstream OBB representation methods, but also enables a lightweight design of the polar coordinate system DETR, achieving accurate and real-time detection of directional ship targets.

[0062] 2. This invention constructs a DETR detector based on a weighted entanglement strategy in NAS, realizes automatic design of DETR, and improves the FFN in the DETR encoder, effectively reducing the number of floating-point operations of the model. By finding the most suitable FFN structure combination through NAS, the module can run efficiently.

[0063] 3. In order to avoid sampling a large number of non-ship region features during the extraction of target features, this invention proposes a multi-scale attention method in the DETR decoder for feature extraction of ship objects. This method locates the target region based on the auxiliary model of the bounding box regressed in the encoder, effectively solving the problem of sampling a large amount of background based on HBB-based multi-scale deformable attention.

[0064] 4. This invention introduces an auxiliary training method for a DETR encoder with a polar coordinate detection head to enhance the learning of potential ship features. It uses the same polar coordinate encoding and decoding method to achieve consistency of network regression on OBB. At the same time, we further filter positive samples in the auxiliary head for auxiliary training of the decoder to improve cross-attention learning.

[0065] 5. This invention achieves a balance between detection speed and accuracy. This ship target detection network model, while ensuring accurate extraction and analysis of directional ship target features, enables accurate and real-time detection of ship targets. Accurate detection ensures precise identification of the ship's position, size, and orientation, while real-time detection meets the need for rapid response in practical applications, providing more efficient and reliable technical support for the application of SAR imagery in fields such as marine monitoring and military reconnaissance. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This forms the overall framework of the method.

[0068] Figure 2 This is a diagram of the structure of a downsampled feedforward neural network (FFN);

[0069] Figure 3 A structural diagram of a directional multi-scale attention mechanism;

[0070] Figure 4 The following is a comparison of sampling methods for the directional multi-scale attention mechanism: (a) the original bounding box, (b) the sampling method based on the horizontal bounding box, (c) the traditional sampling method based on the rotated bounding box, and (d) our sampling method.

[0071] Figure 5The images show the bounding box regions of ship targets visualized after the detector is decoded. (a) is the near-shore label of the SAR image, (b) is the detection result of the detector of the present invention, (c) is the far-shore label of the SAR image, and (d) is the detection result of the detector of the present invention. Detailed Implementation

[0072] It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0075] A lightweight method for detecting rotating ship targets in polar coordinate system SAR images based on Neural Architecture Search (NAS) includes the following steps:

[0076] S1: Take the SAR image to be detected;

[0077] S2: Construct a search space based on the DETR model framework, and use a weight entanglement strategy to realize the automatic lightweight design of the DETR model, thereby realizing a structured searchable ship target detection network model for detecting the position of ships in images.

[0078] The structure of the DETR model is known; the model launched in 2020 is called Detection Transformer. As the first model to fully apply the Transformer to object detection, DETR pioneered a new paradigm of "end-to-end detection." Its core contribution lies in replacing the manual components in traditional detection with "attention mechanism + bipartite graph matching," simplifying model design.

[0079] S3: Train the ship target detection network model to obtain a trained ship target detection network model;

[0080] S4: Input the SAR image to be detected into the trained ship target detection network model to detect the position of the ship in the image.

[0081] The steps S1 / S2 / S3 / S are executed sequentially;

[0082] The ship target detection network model includes:

[0083] Backbone network: used for efficient multi-scale feature extraction of input SAR images;

[0084] Encoder: Used to extract ship features at different levels based on the output features received from the backbone network and the adjusted embedding dimension;

[0085] Decoder: Based on the different levels of ship features received from the encoder, decodes the spatial position, size, and rotation angle of a ship target in any direction in polar coordinates, outputs predicted ship target features, and uses Hungarian matching and one-to-many bounding box matching modes to predict the position of the ship target based on polar coordinate encoding.

[0086] The location of the ship target includes the coordinates of the ship's center point, the length of half the diagonal of the rectangle, and two rotation angles, including the first angle: from the polar radius, rotating counterclockwise, the angle of the first contact.

[0087] The backbone network is built on the Swing Transformer for dynamic model construction. Swing Transformer, short for Shifted Window Transformer, is a hierarchical vision Transformer model proposed by Microsoft Research Asia in 2021 in the paper "Swin Transformer: Hierarchical Vision Transformer using ShiftedWindows". It breaks through the bottleneck of the surge in computational complexity of early Vision Transformers (such as ViT) when processing high-resolution images. Through innovative "window attention mechanism" and "hierarchical feature extraction" design, it takes into account both global context modeling ability and local detail capture ability, and successfully transfers the Transformer architecture from the field of natural language processing (NLP) to the field of computer vision (CV) more efficiently, becoming the core backbone network for tasks such as image classification, object detection, and semantic segmentation.

[0088] To automatically search for the Swing Transformer framework, the embedding dimension, the number of heads in the multi-head attention, the MLP ratio of each stage in the Swing Transformer, and the embedding dimension of the backbone output features are set as dynamic candidates.

[0089] To reduce the number of model parameters, the input SAR image channels are first adjusted, and the image resolution is downsampled to one-quarter through dynamic patch embedding. Then, a sliding window attention module is used to extract feature information to retain global information while reducing the computational cost of the model.

[0090] The process for extracting ship features at different levels based on the output features received from the backbone network and the adjusted embedding dimension is as follows:

[0091] The output features of the first three layers are used to adjust the embedding dimension through a dynamic linear layer and then input into a Transformer-inspired encoder. Similarly, the number of layers in the encoder is used as a dynamic candidate value.

[0092] The input features are first fed into a multi-scale variable attention mechanism to enhance the multi-scale features, and then normalization is performed to prevent data instability.

[0093] Since multi-scale features cause a sharp increase in floating-point operations in the FFN of the encoder, in order to effectively reduce floating-point operations, we downsample a portion of the normalized feature layer to reduce FLOPs; then we feed it into the feedforward neural network to output a lightweight feature vector.

[0094] The lightweight feature vectors are then processed again using a normalization layer;

[0095] To assist in encoder training, we use a polar coordinate-based detection head to improve the encoder's ability to learn potential ship features.

[0096] In the decoder section, the encoder output is first fed into a multi-head attention mechanism to extract feature information at different levels, and then fed into a normalization layer to prevent data anomalies. To avoid sampling too much non-ship target information, ship bounding boxes from the encoder are used for auxiliary sampling, effectively reducing invalid computations; a normalization layer is also used to prevent data anomalies. Finally, information is further extracted using FFN, and a normalization layer is used again to prevent data anomalies. To improve the decoder's cross-attention learning, a polar coordinate-based auxiliary head predictive oriented bounding box (OBB) is further used for auxiliary training of the decoder. Similarly, the decoder layer number is used as a dynamic candidate value.

[0097] Decode the spatial position, size, and rotation angle of a ship target in any direction in polar coordinates;

[0098] The system outputs predicted ship target features. Then, using Hungarian matching and a one-to-many bounding box matching mode, for the regressed oriented bounding boxes, when the Intersection over Union (IoU) with the ground truth label reaches or exceeds 0.5, the predicted box at the corresponding location is set as a positive sample. Subsequently, prediction is performed in polar coordinates based on the matching results between the bounding boxes and the categories.

[0099] The backbone network consists of multiple layers of Swing Transformer blocks. The network first segments the input features into feature blocks that are 1 / 4 the size of the original scale, and then obtains features at different scales through a continuous dynamic sliding window attention mechanism and downsampling, so as to extract ship target features at different scales.

[0100] The encoder consists of multiple dynamic modules. The input is first fed into a multi-scale deformable attention mechanism to extract multi-scale features. Then, a partially downsampled FFN is used to learn features of different modes in the image.

[0101] The encoder is used to encode ship targets in SAR images and extract ship target features at different scales. The entire encoder consists of various dynamic modules. Specifically, the input is first fed into a multi-scale variable attention mechanism to extract multi-scale features. To make the model more lightweight, a partially downsampled FFN is used to learn features of different modes in the image. Specifically, during the partial downsampling process, the largest scale features are first scaled down using bilinear interpolation, and then all scale information is fed into the hidden layer for feature extraction, finally outputting the result.

[0102] The decoder decodes based on the features extracted by the encoder to display ship features more clearly. The entire decoder layer is also composed of multiple dynamic modules. Specifically: the input is first fed into a multi-head attention mechanism to capture features at different levels. To avoid over-sampling of non-ship target information, the directional multi-scale attention mechanism relies on the bounding box regressed by the encoder to assist in locating the ship's bounding box, sampling feature information within the bounding box to reduce unnecessary resource consumption. Finally, FFN is used to further extract target features.

[0103] The bounding box auxiliary head is designed based on a polar coordinate system to assist in the detection and localization of bounding boxes in this model. Specifically, the detection head encodes and decodes ship bounding boxes based on a five-parameter polar coordinate system. On one hand, in the last layer of the encoder, the predicted bounding boxes output by this detection head are used to improve the encoder's ability to learn potential ship features. On the other hand, the bounding boxes from the encoder further assist the directional multi-scale attention mechanism in each layer of the decoder and aid in the regression of the ship target bounding boxes in the decoder.

[0104] The rotational bounding box of a ship target is predicted using a five-parameter encoding method based on polar coordinates. Its unique feature lies in the standardization operation performed on the polar radius of the bounding box. Specifically, the polar radius is divided by the mean of the shorter sides of the bounding box, and then the result is mapped to logarithmic space. This operation aims to constrain the variation range of the ship target bounding box's polar radius, thereby enhancing the stability of network training. Simultaneously, the variation range of the polar angle is limited to the interval [-1, 1] to ensure the standardization of the network training process. This standardization method effectively controls the distribution of features and improves the stability and prediction accuracy of the network model.

[0105] The bounding box auxiliary head is designed by a polar coordinate system-based detection head. It encodes and decodes the ship bounding box based on a five-parameter polar coordinate system to improve the encoder's ability to learn potential ship features.

[0106] The method for predicting ship target bounding boxes primarily employs a one-to-many bounding box matching approach. Predicted boxes at locations where the Intersection over Union (IoU) between the regressed oriented bounding boxes (OBBs) and the ground truth labels is greater than or equal to 0.5 are set as positive samples. Prediction is then performed based on the bounding box matching results and the class matching results. This method effectively improves the accuracy of bounding box detection.

[0107] The five-parameter encoding method based on polar coordinates is designed based on the rotating bounding box labels of ship targets. In the polar coordinate system, the spatial position, size, and rotation angle of a ship target facing any direction can be clearly defined by five parameters: the coordinates of the center point, the distance from the center point to the corner of the rotating bounding box (i.e., the polar radius), and two polar angles within a specific range of [0, π]. When using polar coordinates to encode the rotating bounding box of a ship target, it can effectively avoid the sudden increase in loss and difficulty in convergence that occurs when training with Cartesian coordinates. Therefore, using the five-parameter encoding method in polar coordinates can efficiently characterize the spatial position, size, and rotation angle of a ship target facing any direction, thereby improving the performance of ship target detection.

[0108] The weight entanglement described is a novel weight-sharing method inspired by works such as OFA, BigNAS, and Slimmable networks. It achieves advantages such as low memory usage, fast convergence speed, and high performance by sharing weights among different operators in the same layer, enabling them to influence and update each other.

[0109] Example 1: The following is combined with Figure 1 The network structure and specific examples in the text describe the design of a rotating target detection network.

[0110] like Figure 1 As shown, the input image is first preprocessed to a size of 608×608, and then fed into the backbone network to output features at different scales. and To reduce the number of model parameters, we remove the largest scale feature and feed the three smaller features into the encoder layer for multi-feature information encoding.

[0111] In the encoder, the input first enters a multi-scale attention mechanism to extract multi-scale feature information, and then normalization is used to prevent data anomalies.

[0112] like Figure 2 As shown, in order to reduce the number of model parameters, we improve upon the traditional feedforward neural network (FFN) by sampling the largest scale features in the input through bilinear interpolation, and then feeding them together with other scale features into the subsequent FFN operation. Here, we set the width scaling factor (MLP) of the multilayer perceptron to be dynamically variable in order to find the optimal structure.

[0113] The encoder input is fed into the decoding layer. First, the input undergoes a multi-head attention mechanism to extract feature information at different levels. Then, the data is normalized. To avoid sampling too much non-ship target information, we rely on the ship target bounding boxes regressed from the encoder to assist the encoder in locating the ship target. Information within the ship bounding boxes is sampled. This sampling method can reduce the number of model parameters while maintaining detection performance. The principle of the directional multi-scale attention mechanism will be described in detail below:

[0114] like Figure 3 As shown, the input is first categorized into three levels of features based on the image size of the data source, including the corresponding query vector, positional encoding, and the input value from the previous layer. Then, three sets of trainable weights and biases required for the dynamic linear layer are generated. These are then fed into the dynamic linear layer respectively.

[0115] Value = Linear(V, weight) v biasv ) (twenty one)

[0116] Here, V represents the input value. v bias v These represent the weights and offsets of the linear layer corresponding to the processed value, respectively, while Value represents the output value.

[0117] Next, ship target attention is obtained from the input query vector and position encoding to assist in the localization of ship targets:

[0118] Attention=Linear(q+query_pos,weight a bias a ) (twenty two)

[0119] Where q and query_pos represent the query vector and its corresponding position code, respectively, and weight a bias a These represent the weights and offsets of the corresponding linear layer for the attention layer. Attention represents the output attention value.

[0120] Obtain bounding box data represented in polar coordinates based on the bounding box information in the encoder:

[0121] (c x ,c y ,r,α1,α2)=reference_points×factors (23)

[0122] Among them, (c x ,c y (r, α1, α2) represent the x-coordinate, y-coordinate, polar radius, and two polar angles within [0, π] of the center point of the bounding box, respectively. `reference_points` represents the result of the reference points in the encoder, which, along with (c... x ,c y The same form as r, α1, α2). factors represent the scaling factor of the image, which normalizes the data.

[0123] Get the coordinates of the four corner points:

[0124]

[0125] A rotated bounding box is constructed based on the coordinates of the four corner points, and the value vector is sampled within the box. Finally, the sampled features are multiplied by the attention weights to generate an attention-enhanced result.

[0126]

[0127] Finally, the final result is output through the dynamic linear layer:

[0128] output=Linear(output_tmp,weight o bias o (27)

[0129] Finally, the enhanced result of the ship target area is output. The enhanced result is fed into a normalizer to standardize the data, then into a feedforward neural network to further extract information, and finally into a normalizer.

[0130] The final model regresses ship target information using Hungarian matching and one-to-many matching. Specifically, the model calculates the ship target loss by considering predicted ship target bounding boxes with an intersection-union (IOU) greater than 0.5 as positive samples and others as negative samples, and regresses the ship target using polar coordinate encoding.

[0131] In dynamic detection tasks using the DETR framework, loss is calculated only in the last encoder. The formula for calculating the loss for the predicted ship category in the encoder is as follows:

[0132]

[0133] L BCE =-(T) ESi )log(P ESi )-(1-T ESi log(1-P) ESi (29)

[0134] Where, N s Num represents the number of positive samples. query P represents the number of query vectors. ESi T represents the class score predicted by the encoder. ESi Represents the category of the label. W ESi The labels represent the sampling points (1 represents a positive sample, 0 represents a negative sample). Finally, β is a hyperparameter of Quality Focal Loss, which is set to 2 in this model.

[0135] For the regression loss of bounding boxes in the encoder, L1 Loss is used to calculate the difference between the predicted bounding box and the true bounding box. The specific calculation is as follows:

[0136]

[0137] Where N represents the parameters of the positive samples, P EO T represents the predicted bounding box. EO Represents the actual bounding box. WEO This represents the sample weight, with 1 for positive samples and 0 for negative samples.

[0138] Unlike the encoder, the decoder layer calculates the ship category and ship target bounding box at each layer, and also uses Quality Focal Loss and L1 Loss. The loss for each layer is as follows:

[0139]

[0140] The loss function of the entire network can be expressed as:

[0141] Loss = L E +L D (12)

[0142] Figure 4 The following is a comparison of sampling methods for the directional multi-scale attention mechanism: (a) the original bounding box, (b) the sampling method based on the horizontal bounding box, (c) the traditional sampling method based on the rotated bounding box, and (d) our sampling method.

[0143] This application relates to a lightweight SAR image rotating ship target detection method based on Neural Architecture Search (NAS) in polar coordinates. To verify the processing capability of the proposed detection method in complex scenarios such as nearshore areas, as well as its detection performance and inference time under combined conditions, nearshore data tests and combined data tests were conducted on the RSSDD dataset. The results are compared with those of currently advanced SAR image arbitrary orientation ship target detectors, as shown in the table below:

[0144]

[0145] Experimental results show that this invention achieves a balance between detection accuracy and speed in the DETR-based detection model using the One-Shot NAS method. Furthermore, it reduces model size and inference time by downsampling the maximum-scale features in the FFN through bilinear interpolation. Additionally, it achieves accurate sampling of ship targets by relying on the target bounding boxes in the encoder, further reducing model size and inference time. As the table shows, our invention maintains high detection accuracy while still enabling fast real-time detection with a small model.

[0146] like Figure 5 As shown, in order to better demonstrate the advantages of our model, the effectiveness of this invention in detecting ship targets in any direction in SAR images is demonstrated by visualizing the detection results. (a) is the near-shore label of the SAR image, (b) is the detection result of the detector of this invention, (c) is the far-shore label of the SAR image, and (d) is the detection result of the detector of this invention.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight method for detecting rotating ship targets in polar coordinate system SAR images based on neural architecture search, characterized in that: Includes the following steps: Acquire the SAR image to be detected; A search space based on the DETR model is constructed, and a weight entanglement strategy is adopted to realize the automatic lightweight design of the DETR model, thereby realizing a structured searchable ship target detection network model for detecting the position of ships in images. The ship target detection network model is trained to obtain a trained ship target detection network model; The SAR image to be detected is input into a trained ship target detection network model to detect the position of the ship in the image.

2. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 1, characterized in that: The ship target detection network model includes: Backbone network: used for multi-scale feature extraction of input SAR images; Encoder: Used to extract ship features at different levels based on the output features received from the backbone network and the adjusted embedding dimension; Decoder: Based on the different levels of ship features received from the encoder, decodes the spatial position, size, and rotation angle of a ship target in any direction in polar coordinates, outputs predicted ship target features, and uses Hungarian matching and one-to-many bounding box matching modes to predict the position of the ship target based on polar coordinate encoding.

3. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 2, characterized in that: The backbone network is constructed based on the Swing Transformer for dynamic model building. The backbone network includes a multi-layer sliding window attention module for extracting features at different scales; The process by which the backbone network extracts features from the input SAR image is as follows: To automatically search for multi-layer sliding window attention Swin Transformer frameworks, the embedding dimension, the number of heads in multi-head attention, the MLP ratio of each stage in Swin Transformer, and the embedding dimension of the backbone output features are set as dynamic candidates. First, the input SAR image channels are adjusted, and the image resolution is downsampled to one-quarter through a dynamic patch embedding operation. Features at different scales are obtained through a continuous dynamic sliding window attention mechanism and downsampling, so as to extract ship target features at different scales while preserving global information.

4. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 2, characterized in that: The process of extracting ship features at different levels based on the output features received from the backbone network and the adjusted embedding dimensions is as follows: The input features are first fed into a multi-scale variable attention mechanism to enhance the multi-scale features, and then normalization is performed. A portion of the normalized feature layer is downsampled and then fed into a feedforward neural network to output a lightweight feature vector. The lightweight feature vectors are then processed using a normalization layer.

5. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 2, characterized in that: The process of decoding the spatial position, size, and rotation angle of a ship target in any direction in polar coordinates based on different levels of ship features received from the encoder, and outputting predicted ship target features, using Hungarian matching and one-to-many bounding box matching modes, to predict the position of the ship target based on polar coordinate encoding is as follows: The encoder output is fed into a multi-head attention mechanism to extract feature information at different levels, and then fed into a normalization layer to prevent data anomalies. Ship bounding boxes are used to assist sampling, and normalization layers are also used to prevent data anomalies. Enhanced results are obtained by outputting the ship target area through a directional multi-scale attention mechanism, and then a normalization layer is used to prevent data anomalies. Finally, multi-scale feature information is further extracted through a feedforward neural network with downsampling. In the partial downsampling process, the feature with the largest scale is first reduced in scale by bilinear interpolation, and then all scale information is fed into the hidden layer for feature extraction. A normalization layer is used to prevent data anomalies. Using Hungarian matching and one-to-many bounding box matching modes, for the directional bounding boxes obtained from regression, when the intersection-union ratio with the true label reaches or exceeds 0.5, the predicted box at the corresponding position is set as a positive sample. Prediction is performed based on the matching of bounding boxes and categories, and the position of ship targets is predicted based on polar coordinate encoding.

6. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 1, characterized in that: The training of the ship target detection network model is as follows: A polar coordinate system-based detection head is used to assist in training the encoder, thereby improving the encoder's ability to learn potential ship features. A polar coordinate-based auxiliary head predictive oriented bounding box (OBB) is used for auxiliary training of the decoder, with the decoder layer number as a dynamic candidate value.

7. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 1, characterized in that: The detection head encodes and decodes the ship bounding box based on a five-parameter polar coordinate system. On the one hand, in the last layer of the encoder, the detection head is used to output the predicted bounding box, which improves the encoder's ability to learn potential ship features. On the other hand, the bounding box of the encoder further assists the directional multi-scale attention mechanism in each layer of the decoder and assists the regression of the ship target bounding box of the decoder.

8. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 1, characterized in that: The prediction of the ship target includes the prediction of the rotated bounding box using a five-parameter encoding method based on polar coordinates; this is achieved by dividing the polar radius by the mean of the short side of the bounding box and then mapping the result to logarithmic space.

9. The lightweight SAR image rotation target detection method based on neural architecture search in polar coordinates according to claim 5, characterized in that: The process of obtaining the enhanced result by outputting the ship target region through the directional multi-scale attention mechanism is as follows: The input is first divided into three different features according to the image size of the data source, including the corresponding query vector, position encoding and the input value in the previous layer. Then, three sets of trainable weights and biases required for the dynamic linear layer are generated. Subsequently, the trainable weights and biases are fed into the dynamic linear layer respectively. Value=Linear(V,weight v ,bias v ) (1) Where V represents the input value, weight v bias v These represent the weights and offsets of the linear layer corresponding to the processed value, respectively, and Value represents the output value. Next, ship target attention is obtained from the input query vector and position encoding to assist in the localization of ship targets: Attention=Linear(q+query_pos,weight a ,bias a ) (2) Where q and query_pos represent the query vector and its corresponding position code, respectively, and weight a bias a These represent the weights and offsets of the corresponding linear layer for attention, respectively, while Attention represents the output attention value. Obtain bounding box data represented in polar coordinates based on the bounding box information in the encoder: (c x ,c y ,r,α1,α2)=reference_points×factors (3) where, (c x ,c y (r, α1, α2) represent the x-coordinate, y-coordinate, polar radius, and two polar angles within [0, π] of the center point of the bounding box, respectively. `reference_points` represents the result of the reference points in the encoder, which, along with (c... x ,c y The same as r, α1, α2), factors represent the scaling factors of the image, which normalize the data; Get the coordinates of the four corner points: A rotated bounding box is constructed based on the coordinates of the four corner points, and the value vector is sampled within the box. Finally, the sampled features are multiplied by the attention weights to generate an attention-enhanced result. Finally, the final result is output through the dynamic linear layer: output=Linear(output_tmp,weight o ,bias o ) (7) The final output shows an enhanced target area for the ship.

10. The lightweight SAR image rotation ship target detection method based on neural architecture search in polar coordinates according to claim 2, characterized in that: The formula used in the encoder to calculate the predicted loss for the ship category is as follows: L BCE =-(T ESi )log(P ESi )-(1-T ESi )log(1-P ESi ) (9) Where, N s Num represents the number of positive samples. query P represents the number of query vectors. ESi T represents the class score predicted by the encoder. ESi W represents the category of the label. ESi The label represents the sampling point, and β is a hyperparameter of Quality Focal Loss; For the regression loss of bounding boxes in the encoder, L1 Loss is used to calculate the difference between the predicted bounding box and the true bounding box. The specific calculation is as follows: Where N represents the parameters of the positive samples, P EO T represents the predicted bounding box. EO W represents the actual bounding box. EO This represents the sample weight, with 1 for positive samples and 0 for negative samples.

Images