Multi-scale direction sensing SAR target detection method based on global information fusion

Abstract

The invention discloses a multi-scale direction sensing SAR target detection method based on global information fusion, and mainly solves the problems of low detection precision and efficiency caused by insufficient global information and insufficient direction sensing feature extraction of the existing SAR target detection method. According to the implementation scheme, the method comprises the following steps: acquiring a training sample set and a test sample set of SAR images; constructing a multi-scale direction sensing SAR target detection network comprising two feature extractors and a detection head; using the training sample set to train a target detection network model through back propagation iteration; and inputting the test sample set into the trained network model to obtain a target detection result. The first feature extractor designed by the invention utilizes a bidirectional attention mechanism to extract direction and position information of an object, so that the object has relatively strong robustness under different target attitudes and angles; the second feature extractor is designed, organic fusion of global and local features is achieved, it is ensured that the target detection efficiency is improved, and the method can be applied to environment monitoring and disaster assessment.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a target detection method for synthetic aperture radar (SAR) images, which can be applied to environmental monitoring and disaster assessment. Background Art

[0002] With the rapid development of deep learning, target detection technology for synthetic aperture radar (SAR) images has made significant progress. SAR images have important application value in the fields of military, environmental monitoring, disaster assessment, etc., especially in tasks that require all-weather and all-terrain monitoring. SAR technology has demonstrated its superior imaging capabilities and diverse application scenarios. The main challenge of SAR target detection is that objects are usually multi-scale and multi-directional, and the background is complex and has more interference. This makes how to effectively fuse local and global information, especially directional perception features, a core issue in SAR target detection.

[0003] Traditional target detection methods are mostly based on convolutional neural networks (CNNs). Although these methods have achieved good results when processing optical images, due to the unique characteristics of SAR images, existing methods are difficult to fully utilize directional and multi-scale information, resulting in limited detection accuracy. SAR images usually contain rich detail information, especially when detecting small targets, background noise and loss of directional information have an adverse effect on the detection results. For example, in complex environments such as airports and seaports, when detecting small ships or aircraft, traditional methods easily confuse the background with the target, resulting in erroneous detection results.

[0004] In recent years, researchers have proposed some improved network structures to address these issues, such as the paper "Feature Pyramid Networks for Object Detection" published by Tsung-Yi Lin et al. in 2017, which uses the feature pyramid network FPN to handle multi-scale feature fusion. The FPN fuses multiple layers of features in a top-down manner, so that features of different scales can be processed uniformly. However, since FPN and its variants only play a role in the transmission of local information, they ignore the cross-level global information interaction, resulting in information loss between different levels, which limits the improvement of detection performance. Existing research mainly focuses on how to optimize the extraction and fusion of local information, but the integration of global information is still relatively lacking.

[0005] In order to address the challenges of multi-scale object detection, researchers have also proposed some models based on global context information. For example, Bai Lin et al. published a paper titled "A Novel Anchor-Free Detector UsingGlobal Context-Guide Feature Balance Pyramid and United Attention for SARShip Detection" in 2023, which proposed a global context-guided feature balance pyramid GC-FBP model. This model introduces a global context module to establish a connection between distant objects and the background, thereby solving the problem of information loss to a certain extent. However, since this method still uses a feature flow mechanism similar to FPN, that is, simply superimposing context information on the original feature map, the information interaction is not sufficient, which affects the fusion effect.

[0006] In addition to the multi-scale problem, targets in SAR images are usually directional, and traditional detectors mostly use horizontal boxes for target detection, which fails to fully consider the direction and position information of the target. Some methods try to capture direction-aware information through attention mechanisms or by converting features to the frequency domain. For example, the spatial-frequency feature fusion network obtains rotation-invariant features through polar coordinate Fourier transform, but these methods only focus on pixel dependencies within a single image, ignoring the extraction of deep features and abstract semantic information between images. In addition, existing methods usually fail to effectively combine the association between the direction of the object and the background, resulting in difficulty in distinguishing the target from the background, especially in the detection of small targets, where the noise problem is particularly prominent.

[0007] In order to further improve the target detection effect in SAR images, researchers have also proposed some new network structures that integrate local and global features. For example, the Quad-FPN proposed by Zhang Tianwen in the paper "Quad-FPN: A NovelQuad Feature Pyramid Network for SAR Ship Detection" published in 2021 solves the multi-scale representation problem through the innovative design of four feature pyramids, and effectively deals with background noise and subtle features of small targets. However, these methods usually rely on simple interactions between adjacent feature layers, while ignoring the flow of information across levels, resulting in incomplete information integration.

[0008] Most current research focuses on using FPN or its variants for feature fusion to partially solve the multi-scale problem. However, a significant limitation of these methods is that they lack a mechanism for global information integration at each layer. For example, feature fusion based on FPN only flows local information between adjacent layers, resulting in the gradual fading and loss of information from non-adjacent layers across the hierarchy. This information loss can lead to a decrease in detection accuracy, especially when it is necessary to fusion multiple scale and direction perception information.

[0009] In summary, although the existing SAR target detection methods have made some progress in multi-scale feature fusion, there are still problems such as insufficient global information and insufficient extraction of direction-aware features, resulting in low detection accuracy and efficiency. Summary of the invention

[0010] The purpose of the present invention is to solve the deficiencies of the above-mentioned existing technologies and propose a multi-scale direction-aware synthetic aperture radar image target detection network to fully extract the global information and direction-aware features of synthetic aperture radar images and improve the target detection accuracy and efficiency.

[0011] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0012] (1) Obtain training sample set and test sample set:

[0013] Obtain N SAR images including multiple target types, and perform data enhancement on M of the images to mark the targets;

[0014] The M SAR images after data enhancement and their labels are used as the training sample set, and the remaining R = NM images are used as the test sample set, where N ≥ 10000 and M > N / 2;

[0015] (2) Construct a multi-scale direction-aware SAR target detection network W based on global information fusion:

[0016] (2a) establishing a first feature extractor including a convolution module and a direction-aware attention module, which is used to extract multi-scale image features and capture the direction and position information of the target and reduce background interference;

[0017] (2b) establishing a second feature extractor including a multi-scale alignment module, a multi-level fusion module and a multi-feature embedding module, for realizing multi-scale fusion of global information and generating a fused feature map;

[0018] (2c) connecting the first feature extractor, the second feature extractor and the existing detection head in sequence to form a target detection network W;

[0019] (3) Input the training sample set into the target detection network model W, and use the stochastic gradient descent optimization method to iteratively train it to obtain the trained target detection network model W * ;

[0020] (4) Input the test sample set into the trained object detection network model and output the detection bounding boxes and classification results in R images.

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] 1. The present invention realizes the organic fusion of global and local features through the second feature extractor, ensuring efficient performance in multi-scale target detection tasks. The extractor can be used to align features of different scales, extract the detailed features of small targets and the global information of large targets, and deeply embed and fuse these features. This mechanism allows the network to capture the detailed information of the target without losing the global context information, thereby improving the target detection accuracy of small targets and complex scenes.

[0023] 2. The present invention designs a first feature extractor for the directionality of targets in SAR images, which can utilize a bidirectional attention mechanism to extract the direction and position information of the object, that is, by independently aggregating the features of the object in different directions, including the row direction and the column direction, not only can the network model accurately capture the rotation and tilt direction information of the object, but also can accurately detect rotating targets or targets in complex backgrounds, so that it has strong robustness under different target postures and angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is an implementation flow chart of the present invention.

[0025] Figure 2 It is a structural diagram of the network model in the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without creative work should all fall within the scope of protection of the present invention.

[0027] It should be noted that the step numbers in the specification and claims of the present invention are only for a clear description of the implementation scheme of the present invention to facilitate understanding, and the sequence of the numbers is not limited.

[0028] Reference Figure 1 , the present invention comprises the following steps:

[0029] Step 1: preprocess the SAR image and divide it into a training sample set and a test sample set.

[0030] 1.1) Acquire N SAR images including multiple target types:

[0031] The SAR images in this example are collected by Gaofen-3 satellite and other SAR equipment. These images contain a large number of SAR image samples, covering a variety of targets such as aircraft and ships. The targets have various sizes and obvious directional characteristics, which are suitable for multi-scale and multi-directional target detection tasks.

[0032] 1.2) Preprocess the SAR image and then label the target:

[0033] 1.2.1) All samples are standardized to a uniform input size of 800×800×3 to meet the input requirements of the W network;

[0034] 1.2.2) Perform data enhancement on the standardized samples to improve data diversity, so that the model can learn target features under different viewing angles and scales, reduce the risk of overfitting, and improve the generalization and robustness of the target detection model, thereby improving the detection performance. The specific implementation is: first randomly scale and crop each SAR image, and then randomly rotate and flip the randomly scaled and cropped SAR image to complete the preprocessing of the SAR image. The format of the image data after preprocessing ensures that the network model can effectively handle multi-scale and multi-directional target detection tasks, and the standardized processing reduces the differences between different data sets;

[0035] 1.2.3) Draw the bounding box of the target on the preprocessed SAR image and add a category label for each annotated target;

[0036] 1.3) The preprocessed M visible light images and their labels are used as the training sample set, and the remaining R = NM images are used as the test sample set, where N ≥ 10000, M > N / 2, and the mth preprocessed SAR image is X m .

[0037] Step 2: Construct a multi-scale direction-aware SAR target detection network W based on global information fusion.

[0038] Reference Figure 2 The SAR target detection network proposed in the present invention includes three main components: a first feature extractor, a second feature extractor and a detection head. The network can effectively cope with multi-scale and multi-directional target detection tasks in SAR images.

[0039] 2.1) Construct a first feature extractor consisting of a convolution module and a direction-aware attention module to extract features of different scales from SAR images, where:

[0040] The convolution module consists of a convolution kernel and an activation function ReLU. The convolution kernel size is (3, 3), the convolution step is 1, and the padding is (1, 1). The convolution kernel is used for local feature extraction to avoid boundary information loss while keeping the feature size unchanged; the ReLU function is used to calculate the features extracted by the convolution kernel to accelerate model convergence.

[0041] The direction-aware attention module is composed of three identical sub-modules in cascade, each sub-module is composed of two fully connected layers and a normalization layer, and a ReLU activation function between the two fully connected layers. The first convolution kernel is used to extract features in the horizontal direction, and its size is (1, 3), the convolution step is 1, and the padding is (0, 1); the second convolution kernel is used to extract features in the vertical direction, and its size is (3, 1), the convolution step is 1, and the padding is (1, 0); the normalization layer is used to process the features extracted by the convolution kernel and stabilize the training; the ReLU activation function is used to calculate the features processed by the normalization layer to improve the nonlinear expression ability.

[0042] 2.2) Construct a feature extractor consisting of a multi-scale alignment module, a multi-level fusion module and a multi-feature embedding module in cascade to realize multi-scale fusion of global information to generate fusion features, where:

[0043] The multi-scale alignment module is composed of average pooling and linear interpolation connection, the average pooling is used to reduce the dimension and retain the feature smoothness, and the linear interpolation is used for multi-scale feature alignment;

[0044] The multi-level fusion module is composed of a direction-aware attention module and a multi-layer perceptron MLP, wherein the direction-aware attention module is used to enhance the aligned features; the multi-layer perceptron MLP includes two fully connected layers and a normalization layer, wherein the fully connected layer is used for feature conversion and the normalization layer is used for stable training;

[0045] The multi-feature embedding module is composed of three convolution kernels, a Sigmoid function and a direction-aware attention module, wherein the size of each convolution kernel is (1, 1), the convolution step is 1, and the padding is (0, 0), which is used to realize channel feature transformation; the Sigmoid function is used for feature compression; and the direction-aware attention module is used to enhance the directional features output by the multi-level fusion module.

[0046] 2.3) Selecting an existing target detection module as a detection head for outputting target detection results from the fused features generated by the second feature extractor;

[0047] 2.4) The first feature extractor, the second feature extractor, and the detection head are cascaded in sequence to form a multi-scale direction-aware SAR target detection network W based on global information fusion.

[0048] Step 3: Iteratively train the target detection network model W.

[0049] In this example, the training process of W uses the stochastic gradient descent optimization method, and the model parameters are updated in an iterative manner to ensure the generalization ability of the model on multiple data sets. The implementation steps include the following:

[0050] (3.1) At the beginning of training, the network parameters of W are initialized:

[0051] The convolutional layer weights are randomly initialized using a normal distribution, the bias parameters are initialized to 0, the training batch size is set to 32, the initial learning rate used for backpropagation is set to 0.02 and gradually decreased, and the maximum number of iterations is set to 300;

[0052] (3.2) Each batch of images is input into the network model. First, the first feature extractor in the target detection network model W extracts multi-scale features, namely C2, C3, C4 and C5. These features correspond to feature layers of different depths. C2 represents the shallowest high-resolution feature, which is suitable for detecting small targets; C5 represents the deepest low-resolution feature, which contains rich global semantic information. The formula for extracting multi-scale features by the first feature extractor is as follows:

[0053]

[0054] in, is the Sigmoid activation function, Represents element-wise multiplication operation, DConv row and DConv col Denotes the deformable convolution along the row and column directions, respectively, X direction is the extracted multi-scale features;

[0055] (3.3) The features extracted by the first feature extractor are input into the second feature extractor. The features from different levels are first aligned through pooling and interpolation operations. The alignment formula is as follows:

[0056] F global_align =[AvgPool(X i ),X j ,Interpolate(X k )] c

[0057] Among them, X i , X j , X kRepresent feature maps of different scales, respectively. c represents concatenation along the channel dimension, AvgPool represents average pooling, Interpolate represents linear interpolation, and F global_align is the aligned feature;

[0058] (3.4) Aligned feature F global_align Then perform shallow fusion and deep fusion:

[0059] Aligned feature F global_align The input DA Block module, i.e. the direction-aware attention module, passes through the first fully connected layer of DABlock for linear transformation, and then passes through the ReLU activation function to enhance the nonlinearity of the output. The activated features are input to the second fully connected layer to further extract deep feature information, and then pass through the normalization layer to obtain the shallow fusion features:

[0060] F shallow =DA Block(F global_align );

[0061] Feature F after shallow fusion shallow Input into the existing Transformer module to obtain deep fusion features:

[0062] F deep = Transformer(F shallow );

[0063] (3.5) In order to compensate for the information that may be lost during the feature fusion process, it is necessary to embed the deep feature C5 into the shallow features C2, C3, and C4 for compensation to obtain the output fusion feature Y:

[0064]

[0065] Among them, Conv h and Conv w are the convolution operations along the row and column directions respectively, is the Sigmoid activation function, g h and g w is the extracted global feature. This compensation process enables C2, C3, and C4 to embed more global semantic information in shallow features and improve detection accuracy.

[0066] (3.6) The fused feature Y outputs the model prediction value z through the detection head;

[0067] (3.7) According to the predicted value z, classification loss L cls and the bounding box regression loss L regDefine the total loss function L of the target detection network model W total :

[0068] L total =L cls (z,Label)+L reg (z, Label)

[0069] Where Label is the true value of the target in the image; the classification loss L cls and the bounding box regression loss L reg The formulas are expressed as follows:

[0070] L cls = -α(1-p t ) γ log(p t )

[0071] L reg =λ iou L iou +λ dfl L dfl

[0072] Where pt is the probability of correct prediction, α is the weight factor used to balance positive and negative samples, and γ is the focusing parameter; L iou is the intersection over union (IoU) loss, L dfl is the distributed focal loss; λ iou It is used to adjust L iou The loss parameter, λ dfl is a parameter used to adjust the distributed focal loss, λ iou Set to 1, λ dfl Set to 0.5.

[0073] (3.8) Calculate the loss function L through forward propagation total The value of L is then propagated back through total The value of is used to calculate the gradient of the current round t Gradually reduce the learning rate η, the weight parameter ω in the network model W t Update and get the weight parameter ω after the next round of update t+1 :

[0074]

[0075] (3.9) After each round of training, the classification and bounding box prediction results are generated, and the average precision mAP and F1-score indicators are calculated:

[0076]

[0077]

[0078] In the formula, represents precision; FP is a false positive, which means the number of samples that are actually wrong targets but predicted as correct targets; TP is a true positive, which means the number of samples that are actually correct targets and successfully predicted as correct targets; N represents the number of categories, and AP represents the average precision; Represents the recall rate; FN is the false negative example, which means the number of samples that are actually correct targets but predicted as wrong targets;

[0079] (3.10) Repeat steps (3.2) to (3.9) until 300 rounds of training are completed, and select the model weight with the largest value from the saved mAP and F1-score indicators to obtain the trained object detection network model W * .

[0080] Step 4: Input the test sample set into the trained target detection network model W * Detect the images in the test sample and output the detection bounding boxes and classification results of R images.

[0081] The effect of the present invention is further described below in conjunction with simulation experiments.

[0082] 1. Simulation conditions:

[0083] The hardware platform of the simulation experiment is: a Xeon(R) Platinum 8255C CPU with a main frequency of 2.50GHz*96, a 256GB memory, and an NVIDIA A100 SXM4 80GB graphics processor. The software platform is: an Ubuntu 18.04.5 operating system, python 3.8.2, and pytorch 2.0.1.

[0084] 2. Simulation content and result analysis:

[0085] The network of the present invention and the network of the prior art CCDN-DETR are trained with the same training sample set, and then the two trained networks are used to detect the same test sample set, respectively, to obtain their respective detection accuracy and detection efficiency, as shown in Table 1.

[0086] Table 1 Comparison of the test results of the present invention and the prior art

[0087] Method Type mAP F1-score Parameters Params (M) GFLOPs Prior art 91.6％ 87.9％ 23.6 163.7 The present invention 93.4％ 89.8％ 6.0 20.8

[0088] In the table, GFLOPs represents the number of floating-point operations that can be performed per second, in billion floating-point operations;

[0089] Parameter quantity Params refers to the total number of weights or bias parameters that can be learned in the model. The number of parameters directly affects the complexity, computing requirements, and storage requirements of the model. The unit is million M.

[0090] From Table 1, it can be seen that the present invention has the following significant beneficial effects compared with the prior art:

[0091] First, the average precision mAP and F1-score indicators of the present invention are higher than those of the prior art, indicating that the present invention has the advantage of high detection accuracy;

[0092] Second, the number of parameters and the number of operations GFLOPs of the present invention are significantly lower than those of the prior art, that is, the number of parameters is only 1 / 4 of the prior art, and the GFLOPs is only 1 / 8 of the prior art, indicating that the present invention greatly improves the detection efficiency.

Claims

1. A multi-scale direction-aware SAR target detection method based on global information fusion, characterized in that: The steps include: (1) Obtain training sample set and test sample set: Obtain N SAR images including multiple target types, and perform data enhancement on M of the images to mark the targets; The M SAR images after data enhancement and their labels are used as the training sample set, and the remaining R = NM images are used as the test sample set, where N ≥ 10000 and M > N / 2; (2) Construct a multi-scale direction-aware SAR target detection network W based on global information fusion: (2a) establishing a first feature extractor including a convolution module and a direction-aware attention module, which is used to extract multi-scale image features and capture the direction and position information of the target and reduce background interference; (2b) establishing a second feature extractor including a multi-scale alignment module, a multi-level fusion module and a multi-feature embedding module, for realizing multi-scale fusion of global information and generating a fused feature map; (2c) connecting the first feature extractor, the second feature extractor and the existing detection head in sequence to form a target detection network W; (3) Input the training sample set into the target detection network model W, and use the stochastic gradient descent optimization method to iteratively train it to obtain the trained target detection network model W * ; (4) Input the test sample set into the trained object detection network model and output the detection bounding boxes and classification results in R images.

2. The method according to claim 1, characterized in that The data enhancement and target labeling of the M SAR images in step (1) are implemented by the following steps: (1a) Each SAR image is randomly scaled and cropped, and then the randomly scaled and cropped SAR images are randomly rotated and flipped to obtain the data enhanced image; (1b) Draw the bounding box of the target for the data augmented SAR image and add a category label for each annotated target.

3. The method according to claim 1, characterized in that Step (2a) establishes the convolution module and the direction-aware attention module in the first feature extractor, and its structural parameters are as follows: The convolution module is composed of a convolution kernel and an activation function ReLU. The convolution kernel is used for local feature extraction to avoid boundary information loss while keeping the feature size unchanged. Its size is (3, 3), the convolution step is 1, and the padding is (1, 1); the ReLU function is used to calculate the features extracted by the convolution kernel to accelerate model convergence; The direction-aware attention module is composed of three identical sub-modules in cascade, each of which is composed of two convolution kernels, a normalization layer and an activation function ReLU, wherein the first convolution kernel is used to extract horizontal features, and its size is (1, 3), the convolution step is 1, and the padding is (0, 1); the second convolution kernel is used to extract vertical features, and its size is (3, 1), the convolution step is 1, and the padding is (1, 0); the normalization layer is used to process the features extracted by the convolution kernel and stabilize the training; ReLU calculates the features processed by the normalization layer to improve the nonlinear expression ability.

4. The method according to claim 1, characterized in that: Step (2b) establishes a multi-scale alignment module, a multi-level fusion module and a multi-feature embedding module in the second feature extractor, and its structural parameters are as follows: The multi-scale alignment module is composed of average pooling and linear interpolation connection, the average pooling is used to reduce the dimension and retain the feature smoothness, and the linear interpolation is used for multi-scale size alignment; The multi-level fusion module is composed of a direction-aware attention module and a multi-layer perceptron MLP, and the direction-aware attention module is used to enhance the aligned features; The multi-layer perceptron MLP contains two fully connected layers and a normalization layer. The fully connected layer is used for feature conversion, and the normalization layer is used for stable training. The multi-feature embedding module is composed of three convolution kernels, a Sigmoid function and a direction-aware attention module, wherein the size of each convolution kernel is (1, 1), the convolution step is 1, and the padding is (0, 0), which is used to realize channel feature transformation; the Sigmoid function is used for feature compression; and the direction-aware attention module is used to enhance the directional features output by the multi-level fusion module.

5. The method according to claim 1, characterized in that Step (3) uses the stochastic gradient descent optimization method to iteratively train the target detection network model W, and the implementation steps include the following: (3a) The convolutional layer weights are randomly initialized using normal distribution, the initial bias parameter is 0, the training batch size is set to 32, the initial learning rate is set to 0.02, and the maximum number of iterations is set to 300; (3b) The first feature extractor in the target detection network model W outputs the multi-scale features of the input image, namely C2, C3, C4 and C5. These features correspond to feature layers of different depths. C2 represents the shallowest high-resolution feature, which is suitable for detecting small targets; C5 represents the deepest low-resolution feature, which contains rich global semantic information. (3c) The features extracted by the first feature extractor are passed through the second feature extractor to align, shallowly fuse, deeply fuse and compensate the features from different scales in sequence to obtain fused features; (3d) The fused features are output by the detection model to predict the value Z; (3e) According to the classification loss L cls and the bounding box regression loss L reg Define the total loss function of the target detection network model W: L total =L cls (Z,Label)+L reg (Z, Label), where Label is the true value of the target in the image; (3f) Calculate the loss function L through forward propagation total , calculate the gradient at the current time t through back propagation Gradually reduce the learning rate η, the weight parameter ω in the network model W t Update and get the weight parameters after this round of update: Among them, ω t+1 Represents ω t The updated results, represents the partial derivative operation; (3g) At the end of each round of training, the validation set images are input into the model to generate classification and bounding box prediction results, and the mean average precision (mAP) and F1-score indicators are calculated; (3h) Repeat steps (3b) to (3g) until 300 rounds of training are completed, and finally select the model weight with the largest value from the saved mAP and F1-score indicators to obtain the trained object detection network model W * .

6. The method according to claim 5, characterized in that The first feature extractor in step (3b) outputs the multi-scale features of the input image, which is expressed as: in, is the Sigmoid activation function, Represents an element-wise multiplication operation, DConv row and DConv col Denotes the deformable convolution along the row and column directions, respectively, X direction is the extracted multi-scale features.

7. The method according to claim 5, characterized in that Step (3c) aligns features from different levels, and the formula is: F global_align =[AvgPool(X i ),X j ,Interpolate(X k )] c Among them, X i , X j , X k Represents features of different scales, respectively. c represents concatenation along the channel dimension, AvgPool represents average pooling, Interpolate represents linear interpolation, and F global_align is the aligned feature.

8. The method according to claim 5, characterized in that: The shallow fusion and deep fusion described in step (3c) are to align the features F global_align First, it is input into the direction-aware attention module in the multi-level fusion module, which passes through the two fully connected layers and the normalization layer in the module and the ReLu activation function between the two fully connected layers to output the shallow fusion feature F shallow ; Then the shallow fusion feature F shallow Input to the existing Transformer module and output the deep fusion feature F deep .

9. The method according to claim 5, characterized in that The characteristic compensation described in step (3c) is as follows: Among them, Xonv h and Conv w are the convolution operations along the row and column directions respectively, is the Sigmoid activation function, g h and g w is the extracted global feature, and Y is the compensated feature.

10. The method according to claim 5, characterized in that The classification loss L in step (3e) cls and the bounding box regression loss L reg , respectively expressed as follows: L cls =-α(1-p t ) γ log(p t ) L reg =λ iou L iou +λ dfl L dfl Among them, p t To predict the correct probability, α is the weight factor used to balance positive and negative samples, γ is the focusing parameter; L iou is the intersection over union (IoU) loss, L dfl is the distributed focal loss; λ iou It is used to adjust L iou The loss parameter, λ dfl is a parameter used to tune the distributed focal loss.

Images