A synthetic aperture radar image-oriented target detection method, system and storage medium

Abstract

The application provides a target detection method, system and storage medium for a synthetic aperture radar image, and the method comprises the following steps: acquiring a to-be-detected SAR image; inputting the to-be-detected SAR image into a local feature extraction network and a global feature extraction network respectively, and outputting local features and global features corresponding to the to-be-detected SAR image; fusing the local features and the global features, inputting the fused features into a feature detection model, and outputting a target detection result of the to-be-detected SAR image; wherein the sparse embedding module comprises a density perception module and a target focusing module; the density perception module is used for acquiring local features and obtaining a density tensor based on the local features; the target focusing module takes the density tensor and an attention feature tensor output by a specific attention layer adjacent to the target focusing module as inputs, and takes an output of the target focusing module as an input of another attention layer adjacent to the target focusing module. The application can improve the robustness and detection accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a target detection method and system for synthetic aperture radar images. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) is an active microwave remote sensing imaging system, which has the ability to work all day and all weather, and can penetrate through meteorological conditions such as clouds and rain and fog, and plays an irreplaceable role in military reconnaissance, disaster monitoring and resource exploration and other military and civilian application fields. The SAR system can generate a two-dimensional radar image by actively emitting microwave signals and receiving target scattering echoes through a complex signal processing process. Compared with traditional optical images, SAR images have unique imaging mechanisms and characterization characteristics, and their interpretation usually needs to be completed by technical personnel with professional knowledge.

[0003] In recent years, with the rapid development of artificial intelligence deep learning technology, the SAR image intelligent interpretation method based on neural network has made significant progress, especially in the hot application field of SAR image target detection of interest. The current mainstream SAR target detection technology route can be divided into CNN (Convolutional Neural Networks) based target detection method and Transformer based target detection method from the perspective of applicable neural network architecture, and can be divided into single-stage detection method and two-stage detection method from the perspective of the generation method of the bounding box. These technology routes have their own advantages: the CNN based method has inherent advantages in feature extraction; the Transformer based method is good at processing complex scenes; the single-stage method has high detection efficiency; and the two-stage method has excellent detection accuracy. The current mainstream SAR image oriented target detection method is mainly realized by the following ways:

[0004] From the perspective of neural network architecture, the CNN based method fully utilizes the advantages of CNN in feature extraction, and through the design of a specific network architecture to adapt to the scattering characteristics of SAR images, typical representatives include improved Faster R-CNN and YOLO series, etc. These methods can effectively process the speckle noise in SAR images while maintaining high detection accuracy; DETR (DEtection TRansformer) and other Transformer based methods draw on the successful experience in the field of natural language processing, and use the self-attention mechanism to capture the global context information in SAR images, which has a significant advantage in processing target detection tasks in complex scenes.

[0005] From the perspective of the bounding box, the single-stage detection method is represented by SSD (Single Shot Detector), and this kind of method can directly perform target classification and bounding box regression on the feature map, has the advantage of fast detection speed, and is suitable for application scenarios with high real-time requirements. The two-stage detection method is represented by Faster R-CNN, and this kind of method needs to generate candidate regions and classify and accurately position the candidate regions, and generally has higher detection accuracy, but also has relatively high computational complexity.

[0006] However, current target detection researches for SAR images are mostly aimed at aircraft targets and other scattered and arranged targets of interest, and for targets such as ships and vehicles that are densely arranged in SAR images, false alarms and missed detections account for a certain proportion in the image target detection results. That is, the current mainstream SAR image target detection method cannot adaptively adjust and detect the target to be detected in the image with different distribution densities.

[0007] Therefore, there is an urgent need for a brand-new SAR image detection method that can improve the robustness and accuracy of SAR image target detection. SUMMARY

[0008] In view of this, the embodiments of the present application provide a target detection method and system for synthetic aperture radar images, which can reduce the interference of target distribution on target detection and improve the robustness and accuracy of SAR image target detection.

[0009] One aspect of the present application provides a target detection method for a synthetic aperture radar image, the method comprising the following steps:

[0010] Obtaining a to-be-detected SAR image, inputting the to-be-detected SAR image into a pre-trained local feature extraction network and a global feature extraction network respectively, and outputting the local features and global features corresponding to the to-be-detected SAR image; wherein the local feature extraction network is constructed based on a convolutional neural network (CNN), and the global feature extraction network comprises a plurality of serially connected attention layers and a sparse embedding module located between adjacent attention layers;

[0011] Fusing the local features and the global features, and inputting the fused features into a feature detection model to output a target detection result of the to-be-detected SAR image;

[0012] The sparse embedding module comprises a density perception module and a target focusing module; the density perception module is used to obtain local features and obtain a density tensor corresponding to the to-be-detected SAR image based on the local features; the target focusing module takes the density tensor and an attention feature tensor output by a specific attention layer adjacent to the target focusing module as inputs, and takes the output of the target focusing module as the input of another attention layer adjacent to the target focusing module.

[0013] In some embodiments of the present application, the density tensor corresponding to the to-be-tested SAR image is obtained based on the local features, comprising:

[0014] The local features after performing the average pooling operation are taken as the input of the density perception module, and the output is obtained The density tensor corresponding to the to-be-tested SAR image; wherein the density perception module is constructed based on CNN.

[0015] In some embodiments of the present application, the density tensor includes the predicted mask value of each pixel point in the to-be-tested SAR image.

[0016] Before the density tensor is input into the target focusing module, the method further comprises:

[0017] The normalization of each predicted mask value in the density tensor is performed by the clipping operation to obtain the normalized density tensor.

[0018] In some embodiments of the present application, the density perception module is obtained by training in the following way:

[0019] The initial density perception module takes the local features and the real density tensor corresponding to the training SAR image as input, and obtains the density perception module through iterative training; wherein the real density tensor and the density tensor have the same dimension;

[0020] The real density tensor is calculated based on the Gaussian kernel function, comprising the following steps:

[0021] For each real bounding box of the training SAR image, the mask value of each pixel point in the training SAR image relative to the center pixel point of the real bounding box is calculated using the Gaussian kernel function.

[0022] The mask values for each real bounding box are superimposed based on the same pixel points to obtain the total mask value of each pixel point in the training SAR image, thereby obtaining the real density tensor corresponding to the training SAR image.

[0023] In some embodiments of the present application, the target focusing module is obtained by performing the following operations to obtain the attention feature tensor of its output:

[0024] The attention feature tensor output by the specific attention layer is divided into a first feature tensor and a second feature tensor with the same channel dimension;

[0025] The first feature tensor is weighted averaged in the sequence dimension with the density tensor as the weight to obtain an updated first feature tensor;

[0026] The updated first feature tensor and the second feature tensor are fused, and the fused attention feature tensor is input into the convolution model to output a focus probability;

[0027] The fused attention feature tensor and the focus probability are multiplied to obtain an updated attention feature tensor, which is output as the output of the target focus module.

[0028] In some embodiments of the application, the convolution model is constructed using a fully connected layer and a softmax function;

[0029] Before obtaining the updated first feature tensor, the method further comprises:

[0030] The dimension of the density tensor is expanded based on a broadcast mechanism, so that the expanded density tensor has the same dimension as the first feature tensor.

[0031] In some embodiments of the application, the target focus module is obtained by training in the following way:

[0032] The initial target focus module obtains a density tensor for a training SAR image and an attention feature tensor output by a specific attention layer;

[0033] The attention feature tensor for the training SAR image is divided into a first training feature tensor and a second training feature tensor with the same channel dimension;

[0034] The first training feature tensor is weighted and averaged in the sequence dimension with the density tensor for the training SAR image as the weight to obtain an updated first training feature tensor;

[0035] The updated first training feature tensor and the second training feature tensor are fused, and the fused training attention feature tensor is multiplied by the density tensor for the training SAR image to obtain an updated training attention feature tensor;

[0036] The updated training attention feature tensor is input into the initial convolution model to output a training focus probability;

[0037] The updated training attention feature tensor and the training focus probability are multiplied to obtain an updated attention feature tensor;

[0038] The above steps are repeated to obtain the target focus module through iterative training.

[0039] In some embodiments of the application, the global feature extraction network is constructed based on a ViT architecture;

[0040] Before inputting the to-be-tested SAR image into the pre-trained global feature extraction network, the method further comprises:

[0041] The to-be-tested SAR image is divided into multiple image blocks, each image block is flattened and linearly mapped to obtain an image block embedding vector, and each image block is position embedded to obtain a position embedding vector;

[0042] The position embedding vector and the image block embedding vector are combined to obtain a one-dimensional mark embedding sequence corresponding to the to-be-tested SAR image, and the one-dimensional mark embedding sequence is taken as an input of a pre-trained global feature extraction network.

[0043] Another aspect of the present application provides a target detection system for a synthetic aperture radar image, comprising a processor, a memory and a computer program / instruction stored on the memory, the processor being configured to execute the computer program / instruction, and the system implementing the steps of the method according to any one of the above embodiments when the computer program / instruction is executed.

[0044] Another aspect of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, the computer program / instruction being executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0045] The target detection method and system for a synthetic aperture radar image according to the present application can effectively combine the advantages of CNN and Transformer, and through the innovative design of the density perception module and the target module, the target distribution in the SAR image can be fully considered. The method not only has high detection accuracy in complex scenes, but also effectively avoids the problem of feature overlap and confusion in subsequent detection, thereby improving the accuracy and robustness of target detection.

[0046] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0047] It will be understood by those skilled in the art that the objects and advantages of the present application can be realized and attained by the structure specifically described herein, and the above and other objects of the present application will become more apparent to those skilled in the art upon examination of the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application, form a part of the present application, and do not constitute a limitation of the present application. In the drawings:

[0049] Figure 1 A flowchart of a target detection method for a SAR image according to an embodiment of the present application.

[0050] Figure 2 FIG. 1 is a flowchart of a method for target detection of a SAR image according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but are not intended to limit the present application.

[0052] It should be noted that, in order to avoid obscuring the present application due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0053] It should be emphasized that the term “comprises / comprising” as used herein indicates the presence of the stated features, elements, steps or components, but does not exclude one or more other features, elements, steps or components.

[0054] It should be noted that, if not otherwise specified, the term “connected” as used herein can not only mean direct connection, but also indirect connection in the presence of an intermediate.

[0055] In the following, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts or the same or similar steps.

[0056] The current mainstream SAR target detection method relies on artificial intelligence deep learning technology, and the network architecture is mostly based on CNN or Transformer. For example, the existing solutions include SAR image target detection technology based on improved YOLO architecture and DETR, and the specific implementation is as follows:

[0057] The SAR image target detection technology based on the improved YOLO architecture is usually to optimize the traditional YOLO model to adapt to the special imaging characteristics of SAR images, such as adding attention mechanism, introducing deeper feature extraction network or improving loss function, etc. However, although these improvements have improved the effect of SAR image target detection to some extent, there are still many limitations. First of all, due to the strong speckle noise, complex background interference and low target and background contrast of SAR images, the YOLO model is easily disturbed by noise during detection, resulting in low detection accuracy, especially in the detection task of small targets and dense targets. Secondly, this technology is prone to overfitting, mainly because the feature expression of SAR images is unique, and general data enhancement methods cannot effectively improve the generalization ability of the model. In addition, the improved YOLO detection method still relies on the CNN architecture, and its feature extraction ability is limited, which cannot fully exploit and represent the deep features of SAR images, such as scattering characteristics, polarization information and structured patterns, which further limits the robustness and adaptability of detection.

[0058] Compared with traditional CNN methods, the SAR image target detection technology based on DETR can utilize the Transformer architecture to model image features, so it has a significant advantage in modeling global relationships and capturing complex scene information. The end-to-end target detection mechanism of DETR eliminates the dependence on post-processing steps such as NMS (Non-Maximum Suppression), which can effectively improve the detection efficiency and target positioning accuracy in theory. However, in the task of SAR image target detection, this method still has obvious shortcomings: First, the training of the DETR model depends on a large amount of data, and the convergence speed in the training process is slow, making it difficult to quickly reach the ideal performance under limited data conditions, especially in the case of SAR data, which is usually less. This problem is particularly prominent. Secondly, due to the high computational complexity of the Transformer structure itself, its calculation amount is huge when processing large-scale SAR images, which is difficult to meet the demand of efficient detection, for example, in real-time remote sensing monitoring and military target recognition scenes, the calculation resources are limited, making it difficult to deploy and apply this method. In addition, DETR performs poorly in dense target detection tasks, because the feature interaction method based on self-attention mechanism is prone to target confusion when the target is densely distributed, resulting in a high miss rate, which affects the overall detection effect.

[0059] In summary, the existing SAR image target detection methods based on improved YOLO and DETR still have many technical bottlenecks that need to be optimized. It is urgent to propose a more efficient, accurate and robust target detection scheme for the characteristics of SAR images. The present application faces the SAR image target detection task, based on the density perception idea, the Transformer architecture and the CNN target detection technology, a new SAR image target detection method is proposed, which can overcome the following many defects of the existing detection scheme:

[0060] (1) The existing target detection methods based on YOLO and DETR are often affected by noise interference, target overlap and background complexity, etc. in detection accuracy, so they are not suitable for different density distribution targets in SAR images. For example, the existing mainstream methods have poor detection effect for dense target areas, and are prone to target confusion and missed detection. Based on this, the density perception module is introduced, which accurately perceives the distribution density of the target in the target dense area, and dynamically adjusts the attention focus in the feature extraction process according to the density tensor, avoiding the interference of target density and improving the detection accuracy and robustness.

[0061] (2) The existing technology still faces the problem of low computational efficiency in SAR image target detection, especially for large quantities of images. For example, although the DETR architecture has advantages in global information modeling, it is difficult to meet the demand of real-time monitoring due to its high computational complexity. The present application introduces a sparse embedding mechanism to effectively reduce the computational overhead, so that the model can have high computational efficiency while ensuring high accuracy, especially suitable for real-time processing and target detection of large-scale SAR images, such as remote sensing monitoring and military target recognition scenarios with high real-time requirements.

[0062] (3) The existing target detection methods based on CNN and Transformer have deficiencies in feature extraction, which cannot fully exploit and represent the unique scattering characteristics and polarization information of SAR images. To solve the above problems, the present application uses a carefully designed density perception mechanism and multi-source feature guidance strategy to effectively fuse local features extracted based on CNN and global features extracted using the Transformer mechanism, so as to better capture the deep information of SAR images. The way of fusing features enhances its target detection ability in complex background, especially for small targets and dense targets.

[0063] The method proposed in this application achieves high detection accuracy for targets such as ships, vehicles, and aircraft with varying distribution densities. Its core innovation lies in combining the advantages of CNN and Transformer to extract target features. By constructing a density-aware module and a target-focusing module, it solves the problems of target confusion and high computational cost during detection, thereby achieving accurate extraction and effective fusion of target features to improve target detection accuracy. The method proposed in this application can not only detect dispersed targets such as aircraft but also optimize the detection of densely packed targets.

[0064] Figure 1 This is a flowchart illustrating a target detection method for SAR images according to an embodiment of the present invention. Figure 1 As shown, the method proposed in this application includes steps S110 to S120.

[0065] Step S110: Acquire the SAR image to be tested, input the SAR image to be tested into a pre-trained local feature extraction network; output the local features corresponding to the SAR image to be tested; input the SAR image to be tested into a pre-trained global feature extraction network, and output the global features corresponding to the SAR image to be tested.

[0066] The SAR images mentioned in this application can be images to be interpreted generated by a synthetic aperture radar system. Furthermore, to compensate for the shortcomings of a single feature extraction method, this application combines the advantages of CNN and Transformer in feature extraction. It utilizes a local feature extraction network (input data is the SAR image, output data is a feature tensor) and a global feature extraction network to extract local features (also called local target features or local feature tensors) and global features (also called global target features or global feature tensors) of the SAR image, respectively. Because this application combines the feature extraction advantages of CNN and Transformer, it still achieves high detection accuracy for small-scale targets even in cluttered SAR image backgrounds.

[0067] In some embodiments of the present invention, the local feature extraction network can be constructed based on a convolutional neural network (CNN). Therefore, the multi-scale convolutional features extracted by the local feature extraction network can enhance the adaptability of the target detection model to targets of different sizes. For example, models such as Faster R-CNN or YOLO with a CNN architecture can be used to construct the local feature extraction network. This application does not specifically limit the type of CNN architecture model. Moreover, to reduce computational costs, a lightweight CNN architecture can be used to construct the local feature extraction network, for example, a four-level convolutional neural network can be used.

[0068] Further, the global feature extraction network can be a feature extraction network constructed based on a Transformer mechanism, which is used to capture features containing global context information. The global feature extraction network mentioned in the present application can include a plurality of attention layers connected in series and sparse embedding modules located between adjacent attention layers. The structure of the attention layer and the process of extracting features using the attention layer can adopt the prior art, and the present application does not make specific limitations thereto.

[0069] As an example, the global feature extraction network can be constructed based on the classic ViT (Vision Transformer) architecture. Since the dimensions of the input features and the output features of each attention layer in the global feature extraction network are the same, the input SAR image needs to be converted into a feature tensor for preprocessing before the data is input into the attention branch for global feature extraction. For example, the steps of preprocessing can include: dividing the input SAR image into a plurality of image blocks, and updating the features through token conversion and position embedding. That is, before inputting the SAR image to be tested into the pre-trained global feature extraction network, the method further includes: dividing the SAR image to be tested into a plurality of image blocks, flattening and linearly mapping each image block to obtain an image block embedding vector, and performing position embedding on each image block to obtain a position embedding vector; combining the position embedding vector and the image block embedding vector to obtain a token embedding sequence corresponding to the SAR image to be tested, and taking the token embedding sequence as the input of the pre-trained global feature extraction network.

[0070] The above-mentioned preprocessing method is a common feature conversion method in the ViT architecture, which will not be described here.

[0071] The present application innovatively constructs a sparse embedding module. In the process of extracting SAR global features using the Transformer, not only does the sparse embedding module fuse the multi-scale local features extracted by the local feature extraction network and the features containing global information output by the attention layer, but also the sparse embedding module can be based on a density tensor (also known as a density mask or a density distribution tensor) and a multi-source feature guidance mechanism, so that the attention mechanism in the attention layer focuses more on the target area in the process of extracting features, thereby supplementing the defects of the traditional attention mechanism and better extracting the precision of high-difficulty detection areas.

[0072] As an example, the attention layers are connected in series, i.e., the output of a previous attention layer is taken as the input of a current attention layer, and the output of the current attention layer is taken as the input of a next attention layer. If an attention layer A is adjacent to an attention layer B (assuming the flow direction of the feature tensor is from the attention layer A to the attention layer B), the output of the attention layer A can be taken as the input of the attention layer B. The sparse embedding module can be located between the adjacent attention layer A and the attention layer B (assuming the flow direction of the feature tensor is from the attention layer A to the attention layer B), so that the feature tensor output by the attention layer A is taken as the input of the sparse embedding module, and the output of the sparse embedding module is taken as the input of the attention layer B (i.e., the output of a previous attention layer is taken as the input of a current sparse embedding module, and the output of the current sparse embedding module is taken as the input of a next attention layer). That is, if there is no sparse embedding module between adjacent attention layers, the output of a current attention layer is taken as the input of a next attention layer; if there is a sparse embedding module between adjacent attention layers, the output of a previous attention layer is taken as the input of a current sparse embedding module, and the output of the current sparse embedding module is taken as the input of a next attention layer.

[0073] In addition, a sparse embedding module can be arranged between adjacent attention layers (i.e., a sparse embedding module is inserted between adjacent attention layers), or the attention layers can be directly connected (i.e., there is no sparse embedding module between adjacent attention layers), which can be set according to the target detection effect. However, the target detection method mentioned in the present application needs to use at least one sparse embedding module to adjust the target region of attention focus, so the sparse embedding module can be located between part or all of the adjacent attention layers, i.e., the global feature extraction network includes at least one sparse embedding module, and at most one sparse embedding module is arranged between adjacent attention layers. For example, the global feature extraction network includes 12 attention layers, and sparse embedding modules are inserted after the 3rd, 6th and 9th attention layers, so that the feature tensors output by the 3rd, 6th and 9th attention layers are taken as the inputs of the sparse embedding modules.

[0074] In some embodiments of the present application, the sparse embedding module comprises a density perception module and a target focusing module. Specifically, the density perception module is configured to obtain local features corresponding to the SAR image, and obtain a density tensor corresponding to the SAR image to be detected based on the obtained local features, wherein the density tensor output by the density perception module can be used to represent the distribution of the target in the SAR image. The target focusing module takes the density tensor from the density perception module and the attention feature tensor output by a specific attention layer adjacent to the target focusing module as input, and takes the output of the target focusing module as the input of another attention layer adjacent to the target focusing module, wherein the feature tensor output by the target focusing module contains not only global context information but also local information through the density tensor and the sparse embedding mechanism. Through the synergistic effect of the density perception module and the target focusing module in the sparse embedding module, the present application not only maintains high detection accuracy but also effectively reduces the computational cost and helps to improve the convergence efficiency of the model, so that it shows obvious advantages in different target density distributions in the remote sensing application scenario.

[0075] The sparse embedding module focuses on the area where the target is located in the feature extraction process through the sparse embedding mechanism, which can effectively identify and enhance the target area (such as the area where the target is densely distributed in the SAR image) from the background noise, thereby ensuring accurate identification of the target. Specifically, under the joint action of the density tensor generated by the density perception module and the attention feature tensor output by the attention layer, the sparse embedding module can output a sparse attention feature tensor, thereby ensuring the effective focusing of the global feature extraction network in the target survival area by gradually adjusting the focus area of the model.

[0076] As an example, the local features corresponding to the SAR image obtained by the density perception module can be extracted by a local feature extraction network, or can be extracted by another feature extraction network constructed by the present application, and the present application is not limited thereto. In order to reduce the computational cost and training efficiency, the present application takes the local features extracted by the local feature extraction network as the input of the density perception module. Moreover, according to the flow direction of the feature tensor in the global feature extraction network, the specific attention layer adjacent to the target focusing module refers to the attention layer corresponding to the attention feature tensor obtained by the target focusing module; and the other attention layer adjacent to the target focusing module refers to the attention layer obtaining the attention feature tensor output by the target focusing module.

[0077] In some embodiments of the present application, the density perception module can be used to perceive the distribution density of the target in the SAR image, so that in the subsequent feature extraction process, the focus of attention can be dynamically adjusted based on the target distribution density perceived by the density perception module, thereby reducing the influence of the target distribution density on target detection.

[0078] As an example, a kernel function can be used to map data to a higher dimensional mathematical function in order to make it easier to handle non-linear relationships in high dimensional space, and thus the present application adopts a Gaussian kernel to measure the density tensor of the training SAR image in the training data set. In the training phase, the density perception module is trained based on the real density tensor obtained by the Gaussian kernel function, so that it can generate a density mask in the inference phase In addition to the Gaussian kernel, other ways can also be used to measure the real density tensor corresponding to the training SAR image, and the present application is not limited thereto.

[0079] In the training phase, the initial density perception module can be trained using the relevant data of the training SAR image in the training data set to obtain a pre-trained density perception module. The density perception module can be trained in the following way: the initial density perception module obtains the local features and the real density tensor corresponding to the training SAR image as input, and the density perception module is obtained through iterative training. Among them, there can be multiple real bounding boxes on each training SAR image, and the training data set can include the coordinates of each pixel point in the training SAR image, data for the real bounding box (such as the coordinates of each real bounding box, and the coordinates of the center pixel point of each real bounding box, etc.), the local features corresponding to each training SAR image (which can be obtained by a local feature extraction network), and the real density tensor corresponding to each training SAR image, and other data for training the initial density perception module.

[0080] In some embodiments of the present application, the real density tensor corresponding to each training SAR image can be calculated based on a Gaussian kernel function. For a single training SAR image, the calculation steps of the real density tensor can include:

[0081] For each real bounding box of the training SAR image, the mask value of each pixel point in the training SAR image relative to the center pixel point of the real bounding box is calculated using the Gaussian kernel function; the mask values for each real bounding box are superimposed based on the same pixel points to obtain the total mask value of each pixel point in the training SAR image, thereby obtaining the real density tensor corresponding to the training SAR image.

[0082] As an example, the calculation process of the real density tensor corresponding to each training SAR image is as follows:

[0083] For each training SAR image, the present application takes the center (c x ,c y) as the center, the target distribution density around each real bounding box center is simulated by the Gaussian kernel function, so as to obtain the density tensor corresponding to the training SAR image. For example, for a real bounding box n in the training SAR image a, the mask value of each pixel point (x, y) in the training SAR image a can be expressed as:

[0084]

[0085] where (c x ,c y ) represents the center coordinates of the bounding box n in the training SAR image a, (x, y) represents the coordinates of the pixel point in the training SAR image a, and σ is a standard deviation calculated according to the height, weight and angle of the real bounding box, used to control the spatial range of density influence. The determination method of σ can adopt the prior art, which is not described here. In addition, since the mask technology is adopted in the present application to measure the target distribution in the SAR image, the value calculated for each pixel point is called a mask value, that is, the mask value of each pixel point can be used to measure the target distribution of each pixel point.

[0086] Suppose that there are N real bounding boxes in the training SAR image a, then the mask value of each pixel point (x, y) in the image a is calculated for each real bounding box by the above formula and the Gaussian contribution of all real bounding boxes in the training SAR image a is summarized, the real density tensor corresponding to the training image a can be calculated. Therefore, the total mask value of each pixel point (x, y) in the training SAR image a can be expressed as:

[0087]

[0088] The mask values of N real bounding boxes at the same pixel point are integrated to obtain the total mask value of each pixel point (x, y) in the image a Combining the total mask value of each pixel point can obtain the real density tensor corresponding to the training image a (where B represents the data batch, R represents the sequence length, and R = H x W). The above Gaussian mask summation method can ensure that the image region where the to-be-detected target is relatively concentrated shows strong density information, and the region where the to-be-detected target is relatively sparse shows weak density signal. However, the above mask determination method can only provide a relatively rough density perception representation.

[0089] Furthermore, considering that the true bounding boxes of the SAR image under test cannot be directly determined in the subsequent inference stage, and that the calculated true density tensor can only represent a coarse target density distribution, potentially leading to feature overlap and other issues in subsequent detection, this application utilizes the adaptive learning performance of CNNs to acquire features of the SAR image under test and inputs them into an initial density-aware module built on a CNN architecture for training, resulting in a pre-trained density-aware module (built on a CNN architecture). This allows for the acquisition of a more refined density tensor during the inference stage. Besides the CNN architecture, the density-aware module can also be built based on other models or architectures with deep learning capabilities, capable of learning the target density distribution in the image from the acquired SAR image features.

[0090] More specifically, the density-aware module acquires the multi-scale convolutional features F of the SAR image. a Then, the true density tensor can be... With local features F a The initial density sensing module performs feature integration (e.g., feature fusion through stacking of main channels) and refined density sensing, expressed by the formula:

[0091]

[0092] The above process can generate density tensors with multi-level semantic adaptability. Furthermore, the learning capability of convolution can reduce the feature loss problem caused by subsequent sparse target regions.

[0093] During the training phase, the true density tensor With local feature tensor F a Input the initial density sensing module, which can learn F a and The connection between them is used to output a refined density tensor.

[0094] Furthermore, to perform normalization and ensure that the total mask value of each pixel (x,y) in the training image a is within the range of [0,1], a cropping operation can be used to output robust spatial prior values ​​to describe the characteristics of the object distribution, expressed by the formula:

[0095]

[0096] in, Represents density tensor The total mask value of the middle pixel (x,y). express The largest total mask value in the middle, a normalized density tensor a normalized mask value of the pixel point (x, y) in the normalized density tensor.

[0097] In the inference stage, the pre-trained density perception module obtains the local feature F corresponding to the to-be-detected SAR image b extracted by the local feature extraction network b , and outputs the density tensor corresponding to the to-be-detected SAR image b.

[0098] In some embodiments of the present application, the density perception module obtains the density tensor corresponding to the to-be-detected SAR image b based on the obtained local feature, including: taking the local feature after performing the average pooling operation as the input of the density perception module, and outputting the density tensor corresponding to the to-be-detected SAR image b, which can be represented by the following formula:

[0099]

[0100] Before the density tensor is input into the target focusing module, the method further includes: normalizing each predicted mask value in the density tensor through a clipping operation to obtain a normalized density tensor which can be represented by the following formula:

[0101]

[0102] wherein, denotes a predicted mask value of each pixel point (x, y) in the to-be-detected SAR image, denotes a maximum predicted mask value in the to-be-detected SAR image, denotes a normalized density tensor a normalized mask value of the pixel point (x, y) in the normalized density tensor.

[0103] As an example, the real density tensor and the density tensor output by the pre-trained density perception module have the same dimension.

[0104] The density perception module can generate a density distribution tensor of the to-be-detected SAR image according to the local feature of the input to-be-detected SAR image. As a key component of the detection process, the density perception module can generate a density tensor according to the spatial distribution of the to-be-detected target, represent the density of the to-be-detected target in the SAR image, and make the representation of the target region more accurate. This module uses a carefully designed target measurement method to enable the model to dynamically perceive the target density of different regions, avoid feature overlap and confusion caused by too dense targets in traditional methods, and thus improve the robustness and accuracy of detection.

[0105] In some embodiments of the present invention, the target focusing module can combine density tensors and sparse embeddings to focus attention on the target region during the generation of attention feature tensors. During the inference phase, the target focusing module can effectively integrate density tensors. or The focus module uses attention feature tensors containing global information from the attention layers to improve attention to regions of interest. More specifically, the target focusing module obtains its output attention feature tensors by performing the following operations: dividing the attention feature tensors output by a specific attention layer into a first feature tensor and a second feature tensor with the same channel dimension; weighting the first feature tensor along the sequence dimension using the density tensor to obtain an updated first feature tensor; fusing the updated first and second feature tensors and inputting the fused attention feature tensor into a convolutional model to obtain a focusing probability; and multiplying the fused attention feature tensor by the focusing probability to obtain an updated attention feature tensor, which is then used as the output of the target focusing module.

[0106] As an example, such as Figure 2 As shown, assuming the attention feature tensor output by attention layer j-1 can be expressed as... The density tensor can be expressed as Where B represents the batch size, R represents the sequence length (corresponding to the number of image patches input to the attention layer), and C represents the number of channels in the feature tensor. The target focusing module for the SAR image b under test may include the following steps:

[0107] Step S01: Since the target focusing module can combine local and global information during the generation of attention feature tensors and reduce computational costs by utilizing sparse embedding, it can focus specific attention layers (such as...) along the channel dimension. Figure 2 The attention feature tensor Z output by the (j-1)th attention layer j Divide the first feature tensor Z into channels with the same dimension. j,1 Second feature tensor Z j,2 This is to enable the first feature tensor to represent global information and the second feature tensor to represent local information (or the second feature tensor to represent global information and the first feature tensor to represent local information). Since the first feature tensor and the second feature tensor are interchangeable, the following description uses the first feature tensor representing local information as an example.

[0108] As an example, the attention feature tensor Z output by attention layer j-1 (j=0,1,2,…,k) can be generated based on the channel dimension through a fully connected layer. j Divide into two feature tensors with the same dimension, namely the first feature tensor. Second feature tensor The method of partitioning feature tensors through fully connected layers mentioned above is merely an example, and the present invention is not limited thereto.

[0109] Step S02: To facilitate feature multiplication, a broadcast mechanism can be used to expand the density tensor ( or The method further includes expanding the dimension of the density tensor to be the same as that of the first feature tensor (or the second feature tensor) before obtaining the updated first feature tensor. Specifically, before obtaining the updated first feature tensor, the method further includes expanding the dimension of the density tensor based on a broadcast mechanism to make the expanded density tensor have the same dimension as the first feature tensor. Broadcasting can be performed on the channel dimension of the density tensor. This yields the extended density tensor.

[0110] Step S03: In order to highlight the target region and reduce the influence of irrelevant background regions, the first feature tensor and the density tensor can be combined for sparse embedding to obtain sparsed features.

[0111] Specifically, the first feature tensor Z is multiplied by element-wise multiplication. j,1 (It can also be the second characteristic tensor Z) j,2 (The details are omitted here) Modulation is performed, and a weighted sum is calculated along the dimension of the labeled sequence to achieve sparse embedding. Taking the first feature tensor as an example, this step can be expressed by the formula:

[0112]

[0113] Among them, Z j,1 (r) Z represents the first feature tensor j,1 The feature vector at the r-th marker position, P b (r) Represents the extended density tensor P b The feature vector at the r-th marked position is denoted by ⊙, which represents element-wise multiplication. The weighted average used in this application aims to ensure feature normalization in the denominator of the above formula, preventing the feature tensor from being normalized. The scaling ratio is out of balance due to differences in sequence length or target density.

[0114] Step S04: After the above steps, the feature tensor can be... Includes global information, Z j,2 Includes local information. Connects based on channel dimension. and Z j,2 The fused attention feature tensor can be obtained as follows:

[0115]

[0116] Multiplying the second feature tensor and the density tensor yields the feature tensor in step S03. The fused attention feature tensor can be represented as:

[0117]

[0118] Step S05: In order to obtain the focus probability for generating the final attention feature tensor through the features, Z j is input into the pre-trained convolutional model, and the focus probability The focus probability can determine the effectiveness of the labels in the sequence, so that the key information can be extracted in the dense area.

[0119] Step S06: The fused attention feature tensor Z j is multiplied by the focus probability to obtain the updated attention feature tensor which is taken as the output of the target focus module and input into another attention layer (i.e., the jth attention layer) adjacent to the target focus module. The formula is as follows:

[0120]

[0121] In some embodiments of the present application, the target focus module is obtained by the following method:

[0122] The initial target focus module obtains the density tensor for the training SAR image and the attention feature tensor output by the specific attention layer; the dimension of the density tensor for the training SAR image is expanded based on the broadcast mechanism, so that the expanded density tensor has the same dimension as the first feature tensor; the attention feature tensor for the training SAR image is divided into a first training feature tensor and a second training feature tensor with the same channel dimension; the first training feature tensor is weighted and averaged in the sequence dimension with the density tensor for the training SAR image as the weight, to obtain an updated first training feature tensor; the updated first training feature tensor and the second training feature tensor are fused, and the fused training attention feature tensor is multiplied by the density tensor for the training SAR image, to obtain an updated training attention feature tensor; the updated training attention feature tensor is input into the initial convolutional model, and the training focus probability is output; the updated training attention feature tensor and the training focus probability are multiplied, to obtain an updated attention feature tensor. All the above steps are repeated, and the target focus module is obtained by iterative training.

[0123] That is, in the training phase, the fused training attention feature tensor will be multiplied by the density tensor for the training SAR image a or ) is further modulated to suppress the response region irrelevant to the target distribution, so as to better focus on the dense region in the feature extraction process. That is, the training attention feature tensor Z j,a ′ obtained by fusion needs to be further modulated in the training stage, while the attention feature tensor Z j ′ obtained by fusion can be directly used to calculate the focus probability in the inference stage. Therefore, after obtaining Z j,a ′ in step S04 in the training stage, the training attention feature tensor needs to be updated by using the following formula.

[0124]

[0125] Wherein, Z j,a * represents the updated training attention feature tensor, Z j,a ′ represents the training attention feature tensor obtained by fusion.

[0126] As an example, a calculation module can be artificially modulated or additionally set (not run in the inference stage) to modulate Z j,a ′ to obtain Z j,a * .

[0127] In some embodiments of the present application, the target focusing module includes a convolution model, and the convolution model (or the convolution model) is constructed by using a fully connected layer and a softmax function.

[0128] The present application adopts the density perception idea and the strategy of fusing multiple source features in the sparse embedding module, improves the modeling accuracy of global information, and enhances the detection ability of the target detection model to the target to be detected, so as to finally realize higher detection accuracy and more stable performance in the complex scene of the SAR image. Therefore, the global feature extraction network including the sparse embedding module and multiple attention layers can model the global information of the SAR image at multiple levels, so as to provide effective global context information for target detection.

[0129] Step S120: fuse the local feature and the global feature, input the fused feature into the feature detection model, and output the target detection result of the SAR image to be detected.

[0130] The attention feature tensor (i.e. global feature) generated by the global feature extraction network and the multi-scale feature (i.e. local feature) output by the local feature extraction network can be integrated as the input of the feature detection model. In the present application, the feature detection model can include a region proposal network (Region Proposal Network, RPN) and a classifier.

[0131] Specifically, in step S120, the local features and the global features can be fused by means of a main channel stack, and the fused features are input into an RPN, and then the IoU (intersection over union) selection and position regression strategy are used to perform fine prediction of the candidate target region and determination of the target class. The fusion manner in step S120 and the target detection method using the features can adopt the existing scheme, and the present application does not repeat the description here.

[0132] In addition, the dimensions of the feature tensors output by the local feature extraction network and the global feature extraction network are the same (i.e., the dimensions of the local features and the global features are the same). The attention feature tensor output by the last attention layer in the global feature extraction network is the global feature. The local feature extraction network and the global feature extraction network can be trained integrally or separately, and the two feature extraction networks can be adjusted according to the effect of target detection, which is not specifically limited in the present application.

[0133] The target detection method for SAR images proposed in the present application has the following advantages:

[0134] (1) CNN has a natural advantage in local feature extraction and can handle small targets and complex backgrounds in SAR images, while the Transformer encoder is good at capturing global information, which helps to improve the perception ability of complex scenes in SAR images. Therefore, the present application utilizes the feature extraction advantages of CNN and Transformer to fuse attention features and CNN features for target detection. Compared with the single feature extraction of traditional methods, the present application can significantly improve the detection accuracy in complex scenes.

[0135] (2) The density perception module dynamically perceives the distribution of targets in the SAR image to be detected, accurately describes the density of the target region, and effectively avoids the problem of feature overlap and confusion when the targets are too dense, thereby improving the accuracy and robustness of target detection.

[0136] (3) Combining multi-scale CNN feature extraction with the density perception module, the model can adapt to the detection of targets of different sizes, thereby accurately refining the features of the target region to more accurately locate the target, especially in complex backgrounds and large target size variations.

[0137] (4) By introducing the sparse embedding mechanism, not only the waste of computing resources is effectively reduced, making the detection method more efficient in large-scale SAR image data sets, but also the focus on the target region can be dynamically adjusted, reducing background interference and improving the robustness of target detection, so that it can adapt to SAR images with densely arranged and dispersedly arranged targets.

[0138] Corresponding to the above method, the application also provides a target detection system for a synthetic aperture radar image, which comprises a computer device, the computer device comprising a processor and a memory, the memory storing a computer program / instruction, and the processor being configured to execute the computer program / instruction stored in the memory, and the system implements the steps of the method as described above when the computer program / instruction is executed by the processor.

[0139] The target detection system provided in the application comprises a plurality of modules, each of which plays a key role in the entire detection process, and the performance of SAR image target detection can be improved through the cooperative work of the modules. The connection and interaction between the modules ensure high-precision detection of dense targets in the SAR image and low computational overhead. The system mainly comprises a plurality of attention layers, a sparse embedding module composed of a density perception module and a target focusing module, a local feature extraction network constructed based on a CNN, and a feature detection model comprising an RPN for generating a candidate region. Among them, the density perception module provides the density distribution information of the target, the target focusing module performs fine processing on the features containing global information and local information, the plurality of attention layers ensure that the global information is captured, and the RPN can efficiently generate target proposals.

[0140] The embodiment of the application also provides a computer readable storage medium, which stores a computer program / instruction, and the computer program / instruction is executed by a processor to implement the steps of the above-mentioned edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0141] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination thereof. The choice of hardware or software implementation depends on the specific application and design constraints imposed on the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or communication link.

[0142] It is to be expressly understood that the invention is not limited to the specific configurations and process described above and illustrated in the accompanying drawings. For the sake of clarity, detailed descriptions of known methods are omitted. In the above-described embodiments, several specific steps are described and illustrated as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and various changes, modifications and additions can be made thereto by one of ordinary skill in the art without departing from the spirit of the present invention, and the order of the steps can be changed.

[0143] In the present invention, features described and / or illustrated with respect to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0144] The above description is only preferred embodiments of the present invention, and is not intended to limit the present invention. The embodiments of the present invention can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the present invention.

Claims

1. A target detection method for a synthetic aperture radar (SAR) image, characterized by, The method comprises the following steps: Obtaining a to-be-tested SAR image, inputting the to-be-tested SAR image into a pre-trained local feature extraction network and a global feature extraction network respectively, and outputting local features and global features corresponding to the to-be-tested SAR image; wherein the local feature extraction network is constructed based on a convolutional neural network (CNN), and the global feature extraction network comprises a plurality of attention layers connected in series and a sparse embedding module located between adjacent attention layers; Fusing the local features and the global features, inputting the fused features into a feature detection model, and outputting a target detection result of the to-be-tested SAR image; The sparse embedding module comprises a density perception module and a target focusing module; the density perception module is used to obtain the local features and obtain a density tensor corresponding to the to-be-tested SAR image based on the local features; the target focusing module takes the density tensor and an attention feature tensor output by a specific attention layer adjacent to the target focusing module as inputs, and takes an output of the target focusing module as an input of another attention layer adjacent to the target focusing module.

2. The method of claim 1, wherein, The density tensor corresponding to the to-be-tested SAR image is obtained based on the local features, comprising: The local features after performing an average pooling operation are taken as inputs of the density perception module, and a density tensor corresponding to the to-be-tested SAR image is output; wherein the density perception module is constructed based on a CNN.

3. The method of claim 1, wherein, The density tensor comprises predicted mask values of each pixel point in the to-be-tested SAR image. Before the density tensor is input into the target focusing module, the method further comprises: Each predicted mask value in the density tensor is normalized through a clipping operation to obtain a normalized density tensor.

4. The method of claim 1, wherein, The density perception module is obtained through the following training method: An initial density perception module takes local features and a real density tensor corresponding to a training SAR image as inputs, and the density perception module is obtained through iterative training; wherein the real density tensor and the density tensor have the same dimension; The real density tensor is calculated based on a Gaussian kernel function, comprising the following steps: For each real bounding box of the training SAR image, a mask value of each pixel point in the training SAR image relative to a center pixel point of the real bounding box is calculated using a Gaussian kernel function; The mask values for each real bounding box are superimposed based on the same pixel point to obtain a total mask value of each pixel point in the training SAR image, thereby obtaining a real density tensor corresponding to the training SAR image.

5. The method of claim 1, wherein, The target focusing module obtains an attention feature tensor through the following operation: The attention feature tensor output by the specific attention layer is divided into a first feature tensor and a second feature tensor with the same channel dimension; wherein the attention feature tensor and the density tensor are both tensors with a channel dimension and a sequence dimension; the sequence dimension is used to indicate the number of image blocks input into the attention layer, and the image block is obtained by dividing the to-be-tested SAR image; weighting average the first feature tensor in the sequence dimension with the density tensor as weight to obtain an updated first feature tensor; fuse the updated first feature tensor and the second feature tensor, and input the fused attention feature tensor into a convolution model to output a focus probability; multiply the fused attention feature tensor and the focus probability to obtain an updated attention feature tensor as an output of the target focus module.

6. The method of claim 5, wherein, The convolution model is constructed using a fully connected layer and a softmax function; Before obtaining the updated first feature tensor, the method further comprises: extending the dimension of the density tensor based on a broadcast mechanism to make the extended density tensor have the same dimension as the first feature tensor.

7. The method of claim 1, wherein, The target focus module is obtained by training in the following way: An initial target focus module obtains a density tensor for a training SAR image and an attention feature tensor output by the specific attention layer; wherein the attention feature tensor and the density tensor are tensors having a channel dimension and a sequence dimension; the sequence dimension is used to indicate the number of image blocks input into the attention layer, which are obtained by dividing the training SAR image; divide the attention feature tensor for the training SAR image into a first training feature tensor and a second training feature tensor which have the same channel dimension; weighting average the first training feature tensor in the sequence dimension with the density tensor for the training SAR image as weight to obtain an updated first training feature tensor; fuse the updated first training feature tensor and the second training feature tensor, and multiply the fused training attention feature tensor and the density tensor for the training SAR image to obtain an updated training attention feature tensor; input the updated training attention feature tensor into the initial convolution model to output a training focus probability; multiply the updated training attention feature tensor and the training focus probability to obtain an updated attention feature tensor; repeat the above steps to obtain the target focus module through iterative training.

8. The method of claim 1, wherein, The global feature extraction network is constructed based on a ViT architecture; Before inputting the to-be-tested SAR image into the pre-trained global feature extraction network, the method further comprises: divide the to-be-tested SAR image into a plurality of image blocks, flatten and linearly map each image block to obtain an image block embedding vector, and perform position embedding on each image block to obtain a position embedding vector; combine the position embedding vector and the image block embedding vector to obtain a one-dimensional token embedding sequence corresponding to the to-be-tested SAR image, and input the one-dimensional token embedding sequence into the pre-trained global feature extraction network.

9. A synthetic aperture radar image oriented target detection system comprising a processor, a memory and a computer program / instructions stored on the memory, wherein, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method of any one of claims 1 to 8.

10. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the method of any one of claims 1 to 8.

Images