SAR Target Detection and Recognition Method Based on Multi-Level Enhancement Network

Abstract

This invention discloses a SAR target detection and recognition method based on a multi-level enhancement network. This method primarily addresses the problems of existing technologies in complex environments, such as poor robustness, high false alarm and missed detection rates, and low detection and recognition accuracy. The method involves labeling and partitioning SAR measured data to obtain training and test sets; constructing a multi-level enhancement network consisting of a cascade of data-level enhancement modules, feature-level enhancement modules, region proposal modules, and decision-level enhancement modules; training the multi-level enhancement network using the training set based on a stochastic gradient descent algorithm; and inputting the test set images into the trained multi-level enhancement network to obtain SAR target detection and recognition results. This invention significantly improves SAR target detection and recognition performance in complex environments and can be used for battlefield reconnaissance and situational awareness.

Description

Technical Field

[0001] The present invention belongs to the field of radar remote sensing technology, and further relates to a SAR target detection and recognition method, which can be used for battlefield reconnaissance and situation awareness. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging sensor that transmits a large time-bandwidth product signal and uses aperture synthesis to produce two-dimensional, high-resolution images. Compared to optical and infrared sensors, SAR offers unique advantages, including all-day, all-weather operation, long range, and strong penetration. It has become a vital tool for Earth observation and is widely used in both military and civilian fields. With the continuous improvement of SAR systems and the advancement of SAR imaging capabilities, SAR image interpretation technology has gradually attracted the attention of scholars and researchers in related fields. As a difficult and critical step in this process, the accurate detection and identification of key targets is of great significance and research value.

[0003] Traditional SAR target detection and recognition methods primarily utilize a three-stage process, including target detection, target discrimination, and target recognition. Target detection is primarily based on the Constant False Alarm Rate (CFAR) algorithm. This algorithm performs target detection by sliding a window across the SAR image and comparing the selected value with an adaptive threshold, assuming that background clutter satisfies a certain probability distribution model. However, due to its difficulty in effectively modeling non-uniform, strong clutter backgrounds, the algorithm suffers from poor adaptability and low detection accuracy in complex scenarios. Target discrimination and recognition primarily rely on manual feature design and classifier construction based on image statistics and physical properties. This requires significant expertise and experience, and the algorithm suffers from poor accuracy and flexibility, making it difficult to achieve ideal results in practical applications. Furthermore, the inefficient connections between the various links in the traditional three-stage process significantly reduce the algorithm's computational efficiency, necessitating the development of a new architecture.

[0004] In recent years, with the continuous development of deep learning technology, object detection and recognition methods based on deep neural networks have achieved significant breakthroughs in the field of computer vision. Thanks to the unique structure of deep networks, these algorithms can simultaneously predict both the location and category of an object without requiring multiple processing steps, significantly improving detection and recognition performance and efficiency. Currently, mainstream detection and recognition algorithms can be categorized into two modes: single-stage and two-stage. The former directly decodes features extracted by the network to detect and recognize objects, offering faster inference speeds. Representative algorithms include YOLO, SSD, and RetinaNet. The latter incorporates a candidate region extraction stage, first extracting candidate regions from the image that may contain key objects using a deep network. The positions of these regions are then further refined to produce the recognition results. Representative algorithms include R-CNN, Faster R-CNN, and Cascade R-CNN. Compared to single-stage algorithms, two-stage algorithms achieve higher detection and recognition accuracy.

[0005] Although the above-mentioned deep learning-based methods provide a feasible approach for SAR target detection and recognition, compared with optical images, SAR image scenes are more complex, the similarity of targets of different categories is higher, and the edges of targets are not clear due to the influence of coherent speckle noise. Therefore, there are still problems such as instability in complex environments and difficulty in distinguishing similar categories.

[0006] Patent document No. 201710461303.7 discloses an "integrated method for SAR image target detection and recognition." This method first extracts SAR image features using a convolutional neural network. Based on these features, it generates candidate regions that may contain targets. Finally, it uses a fully connected network to predict the category and location of the region of interest to detect and recognize SAR targets. Because this method lacks optimization for the characteristics of SAR images, it is not robust in complex environments, resulting in inaccurate bounding box predictions, high false alarm and missed detection rates, and difficulty effectively mining fine-grained target features, resulting in low detection and recognition accuracy. Summary of the Invention

[0007] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a SAR target detection and recognition method based on a multi-level enhancement network to improve the robustness of detection and recognition in complex environments, reduce the false alarm rate and missed detection rate, enhance the separability of features, and significantly improve the accuracy of SAR target detection and recognition.

[0008] The technical idea of the present invention is to improve the SAR target detection and recognition performance in complex environments by designing a multi-level enhancement network. The implementation steps include the following:

[0009] (1) Obtain SAR images with multiple target categories, mark the target location and target category in each SAR image, and randomly divide the marked SAR images into training and test sets;

[0010] (2) Constructing a multi-level enhancement network:

[0011] (2a) Establishing a data-level enhancement module that sequentially performs multi-scale transformation, random flipping, random rotation, power transformation, and random noise operations;

[0012] (2b) Establish a feature-level enhancement module consisting of a cascade of backbone network A, feature optimization pyramid network F, recursive backbone network Q, and recursive feature optimization pyramid network E;

[0013] (2c) Select the existing region proposal network to form the region proposal module G, and select cross entropy loss and CIOU loss as its classification and regression losses;

[0014] (2d) Establish a decision-level enhancement module D consisting of three decision makers d1, d2, and d3 in cascade, and select cross entropy loss and CIOU loss as its classification and regression losses;

[0015] (2e) cascading the data-level enhancement module, feature-level enhancement module, region proposal module, and decision-level enhancement module in sequence to form a multi-level enhancement network;

[0016] (3) Training the multi-level enhancement network:

[0017] (3a) Randomly sample a set of SAR images from the training set and input them into the multi-stage enhancement network. Calculate the loss and update the network parameters based on the loss using the stochastic gradient descent algorithm.

[0018] (3b) Repeat the process (3a) until the network converges and a trained multi-level enhanced network is obtained;

[0019] (4) Input the SAR images in the test set into the trained multi-level enhancement network to obtain the detection and recognition results.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] First, the present invention improves the robustness of the algorithm in complex environments and reduces the false alarm rate and missed detection rate by designing a data-level enhancement module to simulate the scale and orientation changes of the target as well as the interference of clutter and noise.

[0022] Second, the present invention fully exploits the fine-grained features of targets in SAR images by designing a feature-level enhancement module, thereby enhancing the separability of targets of similar categories.

[0023] Third, the present invention designs a decision-level enhancement module to perform multiple fine-tuning on the prediction results to gradually reduce the deviation between the predicted target position and the true value, effectively suppressing the impact of SAR target edge blur on detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Flowchart for the implementation of the present invention;

[0025] Figure 2 A diagram of a multi-level enhanced network model constructed in the present invention;

[0026] Figure 3 This is a simulation result diagram of the present invention. DETAILED DESCRIPTION

[0027] The examples and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0028] refer to Figure 1 In this example, the SAR target detection and recognition method based on the multi-level enhancement network includes data labeling and division, building a multi-level enhancement network, training the multi-level enhancement network, and obtaining SAR target detection and recognition results. The specific implementation is as follows:

[0029] Step 1: Data labeling and division.

[0030] SAR images with multiple types of targets are obtained, the target positions and target categories in each SAR image are annotated, and the annotated SAR images are randomly divided into 7:3 ratios to obtain training and test sets.

[0031] In an embodiment of the present invention, the SAR images are from the onboard radar of the Gaofen-3 satellite. The SAR image scales include 600×600, 1024×1024, and 2048×2048, with a total of seven types of aircraft targets. The number of training set images is 1,400, and the number of test set images is 600.

[0032] Step 2: Build a multi-level enhancement network.

[0033] refer to Figure 2 The multi-level enhancement network constructed in this step includes a data-level enhancement module, a feature-level enhancement module, a region proposal module, and a decision-level enhancement module cascaded in sequence. The construction steps are as follows:

[0034] 2.1) Establishing a data-level enhancement module that sequentially performs multi-scale transformation, random flipping, random rotation, power transformation, and random noise operations. In an embodiment of the present invention, all of the above operations are performed during training, and only the multi-scale transformation and random flipping operations are performed during testing. The scales of the multi-scale transformation include 1024×1024, 1088×1088, and 1152×1152. The directions of random flipping include horizontal, vertical, and diagonal. The angles of random rotation include 90°, 180°, and 270°. The coefficients of the power transformation are random values between [0.8 and 1.2].

[0035] 2.2) Establish a feature-level enhancement module consisting of a cascade of backbone network A, feature optimization pyramid network F, recursive backbone network Q, and recursive feature optimization pyramid network E:

[0036] 2.2.1) Create a backbone network A consisting of five cascaded convolutional modules a1, a2, a3, a4, and a5, where:

[0037] The first convolution module a1 consists of a 7×7 standard convolution layer, a batch normalization layer, a ReLU activation layer, and a maximum pooling downsampling layer cascade;

[0038] The second convolutional module a2 consists of three residual blocks cascaded;

[0039] The third convolutional module a3 consists of 4 residual blocks cascaded;

[0040] The fourth convolutional module a4 consists of 6 residual blocks cascaded;

[0041] The fifth convolutional module a5 consists of three residual blocks cascaded;

[0042] In an embodiment of the present invention, the backbone network A is used to extract multi-scale feature maps. For an input SAR image with a width of W and a height of H, respectively, The multi-scale feature map output by the backbone network is where Y i is the i-th convolution module a i The output feature map of .

[0043] 2.2.2) Establish a feature optimization pyramid network F consisting of four parallel branches f1, f2, f3, and f4. Each branch f i By feature optimization module f i s And feature fusion module f i u Cascade composition, each feature optimization module f i s , consisting of two parallel sub-branches And 1×1 standard convolution layer cascade, the first sub-branch The structure includes a global average pooling layer, a 1D convolutional layer, a Sigmoid activation layer, and the second sub-branch is the identity branch; each feature fusion module f i u It consists of 3×3 standard convolutional layers.

[0044] In an embodiment of the present invention, the feature optimization pyramid network F is used to optimize the multi-scale feature map Y output by the backbone network. i (i=2,3,4,5) perform feature optimization and feature fusion, Y i After feature optimization, it is expressed as:

[0045]

[0046] Among them, S i Optimize module f for the i-th feature i s The output feature map of , ⊙ represents channel-by-channel multiplication operation, GAP(·) represents global average pooling, Conv1d(·) represents 1D convolution, σ(·) represents Sigmoid function, Conv1×1(·) represents 1×1 standard convolution; S i After feature fusion, it is expressed as:

[0047]

[0048] Among them, U i is the i-th feature fusion module f i u The output feature map of , Conv3×3(·) represents 3×3 standard convolution, and Up(·) represents the bilinear interpolation upsampling function.

[0049] 2.2.3) Build a recursive backbone network Q consisting of five cascaded convolutional modules q1, q2, q3, q4, and q5, where:

[0050] The first convolution module q1 consists of a 7×7 standard convolution layer, a batch normalization layer, a ReLU activation layer, and a maximum pooling downsampling layer cascade;

[0051] The second convolution module q2 consists of three cascaded residual blocks and a 1×1 standard convolution layer in parallel;

[0052] The third convolution module q3 consists of four cascaded residual blocks and a 1×1 standard convolution layer in parallel;

[0053] The fourth convolutional module q4 consists of 6 cascaded residual blocks and a 1×1 standard convolutional layer in parallel;

[0054] The fifth convolutional module q5 consists of three cascaded residual blocks and a 1×1 standard convolutional layer in parallel;

[0055] In an embodiment of the present invention, the recursive backbone network Q is used to extract multi-scale feature maps. For an input SAR image with a width and height of W and H, respectively, The multi-scale feature map output by the recursive backbone network is:

[0056]

[0057] Among them, Z i is the i-th convolution module q i The output feature map, U i-1 The i-1th feature fusion module of the feature optimization pyramid network The output feature map of , Conv1×1(·) represents 1×1 standard convolution.

[0058] 2.2.4) Establish a recursive feature optimization pyramid network E with the same structure and parameters as the feature optimization pyramid network F.

[0059] In an embodiment of the present invention, the recursive feature optimization pyramid E is used to select the multi-scale feature map Z output by the recursive backbone network Q. i (i=2,3,4,5) perform feature optimization and feature fusion, Z i After feature optimization, it is expressed as:

[0060] W i =f i s (Z i+1 )

[0061] =Conv1×1(σ(Conv1d(GAP(Z i+1 )))⊙Z i+1 ),i=1,2,3,4

[0062] Among them, W i Optimize module f for the i-th feature i s The output feature map of , ⊙ represents the channel-by-channel multiplication operation, GAP(·) represents global average pooling, Conv1d(·) represents 1-dimensional convolution, σ(·) represents Sigmoid function, Conv1×1(·) represents 1×1 standard convolution; W i After feature fusion, it is expressed as:

[0063]

[0064] Among them, P i is the i-th feature fusion module f i uThe output feature map of , Conv3×3(·) represents 3×3 standard convolution, and Up(·) represents the bilinear interpolation upsampling function.

[0065] 2.3) Establish a region proposal module G composed of a region proposal network. The region proposal network includes a candidate region generation module, a classification and regression module, a post-processing module, and a positive and negative sample allocation module, where:

[0066] The candidate region generation module is used to generate the candidate region in the input feature map P i (i=1,2,3,4) The generated area at each point is 2 2i+2 , three rectangular candidate regions with width and height ratios of 1:2, 1:1, and 2:1 respectively;

[0067] The classification and regression module consists of two parallel 3×3 standard convolutional layers g1 and g2. The first convolutional layer g1 is used to adjust the center point position and width and height of the candidate region, and the second convolutional layer g2 is used to predict the target confidence of the candidate region.

[0068] The post-processing module is used to filter redundant candidate regions and output the top N candidate regions with the highest target confidence {(p j ,b j )|j=1,2,...,N}, where p j is the target confidence of the j-th candidate region, b j =(x j ,y j ,w j ,h j ) is the bounding box of the jth candidate region, (x j ,y j ) is the center point coordinate of the bounding box, (w j ,h j ) are the width and height of the bounding box;

[0069] The positive and negative sample allocation module is used to allocate candidate regions as positive and negative samples, that is, to allocate candidate regions with an intersection-over-union ratio greater than 0.7 with the labeled bounding box as positive samples, and to allocate candidate regions with an intersection-over-union ratio less than 0.3 with the labeled bounding box as negative samples;

[0070] In an embodiment of the present invention, the bounding box R output by the region proposal module G is G Expressed as:

[0071] R G ={b j =(x j ,y j ,w j ,h j )|j=1,2,...N}.

[0072] 2.4) Establish a decision-level enhancement module D consisting of three cascaded sub-decision makers d1, d2, and d3. Each sub-decision maker has the same structure, including a region of interest extraction module, a classification and regression module, and a positive and negative sample allocation module, where:

[0073] The region of interest extraction module consists of a cascade of an adaptive average pooling layer and a flattening layer. The adaptive average pooling layer is used to pool the candidate region features into 7×7 features of interest, and the flattening layer is used to flatten the features of interest;

[0074] The classification and regression module consists of two linear layers in parallel. The first linear layer adjusts the position of the bounding box, and the second linear layer predicts the class score of the bounding box.

[0075] The positive and negative sample assignment module is used to assign bounding boxes as positive and negative samples. That is, the bounding boxes whose intersection-over-union ratio with the annotated bounding box is greater than the threshold are assigned as positive samples, and the bounding boxes whose intersection-over-union ratio with the annotated bounding box is less than the threshold are assigned as negative samples. The thresholds of the three sub-decision makers are 0.5, 0.6, and 0.7 respectively.

[0076] The bounding box R output by the decision-level enhancement module D Expressed as:

[0077]

[0078] Where N is the number of bounding boxes, is the position of the j-th bounding box output by the third sub-decision maker, Respectively represent The center point coordinates and width and height, The mean of the j-th bounding box category scores output by each sub-decision maker is expressed as:

[0079]

[0080] where N c is the total number of categories, Represent the category scores of the j-th bounding box output by sub-decision makers d1, d2, and d3 respectively.

[0081] Step three: train the multi-level enhancement network.

[0082] 3.1) Randomly sample a set of SAR images from the training set and input them into the multi-level enhancement network to calculate its loss. Based on this loss, the network parameters are updated using the stochastic gradient descent algorithm:

[0083] 3.1.1) Calculating the loss of the multi-level enhancement network in and are the losses of the region proposal module and the decision-level enhancement module, respectively, expressed as:

[0084]

[0085]

[0086] Among them, N G is the number of candidate regions randomly sampled by the region proposal module, p m and are the target confidence and the corresponding true label of the mth candidate region sampled by the region proposal module, b m and are the bounding box of the mth candidate region sampled by the region proposal module and the corresponding true label, L cls and L reg are cross entropy classification loss and CIOU regression loss, N D is the number of bounding boxes randomly sampled by each sub-decision maker, c l i and are the category scores and corresponding true labels of the lth bounding box sampled by the i-th sub-decision maker, respectively, and b l i and are the lth bounding box sampled by the i-th sub-decision maker and the corresponding true label, λ i is the loss weight of the i-th sub-decision maker, satisfying is the activation function, and its expression is:

[0087]

[0088] 3.1.2) Solve the multi-level enhancement network loss in 3.1.1) The gradient of the multi-level enhanced network parameter θ is expressed as:

[0089]

[0090] in and are the losses of the region proposal module and the decision-level enhancement module, respectively.

[0091] 3.1.3) According to the gradient solved in 3.1.2) Update the multi-level enhanced network parameters, expressed as:

[0092]

[0093] Wherein θ′ is the updated network parameter, θ is the updated network parameter; lr is the learning rate, which is set according to the input image batch size. In the embodiment of the present invention, lr=0.005.

[0094] 3.2) Repeat step 3.1) until the network converges to obtain a trained multi-level enhanced network.

[0095] Step 4: Obtain SAR target detection and recognition results.

[0096] The SAR images in the test set are input into the trained multi-level enhancement network to obtain the detection and recognition results.

[0097] The effect of the present invention can be further illustrated by the following simulation experiments:

[0098] 1. Simulation experiment conditions:

[0099] The software platform of the simulation experiment of the present invention is: Ubuntu 18.04 operating system and Pytorch 1.8.0, and the hardware configuration is: Core i9-10980XE CPU and NVIDIA GeForce RTX 3090 GPU.

[0100] The simulation experiment of the present invention uses the measured data of Gaofen-3 SAR, the scene type is airport, the image resolution is 1m×1m, the number of SAR images is 2000, the image sizes are 600×600, 1024×1024, and 2048×2048, the number of target categories is 7, the total number of targets is 6556, the number of training set images is 1400, and the number of test set images is 600.

[0101] 2. Simulation content and result analysis:

[0102] Under the above simulation conditions, the present invention and the existing "A SAR Image Target Detection and Recognition Integrated Method" were used to complete training on the training set respectively. Then, a test set image was randomly selected and input into the trained network, and the detection and recognition results were visualized on the test set image. The results are shown in Figure 2. Figure 3 As shown. Among them, Figure 3 (a) is the detection and recognition result of the prior art, Figure 3 (b) is the detection and recognition result of the present invention. In the figure, the green rectangle represents the target that is detected and recognized correctly, and the red rectangle represents the target that is detected or recognized incorrectly.

[0103] contrast Figure 3 As can be seen from (a) and 3(b), the detection and identification results obtained by the prior art have more false alarms and missed detections, while the detection and identification results obtained by the present invention have fewer false alarms and missed detections.

[0104] The detection and recognition indicators of the present invention and the prior art on all test set images are compared, including the average precision, average recall, average F1 score, and class average precision of seven target categories. The results are shown in Table 1:

[0105] Table 1

[0106] Evaluation indicators Existing technology The present invention Average precision 81.4％ 96.5％ Average recall 78.8％ 97.1％ Average F1 score 0.80 0.97 Class Average Precision 83.1％ 97.3％

[0107] As can be seen from Table 1, the average precision, average recall, average F1 score and class average accuracy of the present invention are all higher than those of the prior art, indicating that the detection and recognition performance of the present invention is significantly better than that of the prior art.

Claims

1. A SAR target detection and recognition method based on a multi-level enhancement network, characterized in that: The following steps are involved: (1) Obtain SAR images with multiple target categories, annotate the target location and target category in each SAR image, and randomly divide the annotated SAR images into training and test sets; (2) Constructing a multi-level enhancement network: (2a) Establishing a data-level enhancement module that sequentially performs multi-scale transformation, random flipping, random rotation, power transformation, and random noise operations; (2b) Establish a feature-level enhancement module consisting of a cascade of backbone network A, feature optimization pyramid network F, recursive backbone network Q, and recursive feature optimization pyramid network E; (2c) Select the existing region proposal network to form the region proposal module G, and select cross entropy loss and CIOU loss as its classification and regression losses; (2d) Establish a decision-level enhancement module D consisting of three decision makers d1, d2, and d3 in cascade, and select cross entropy loss and CIOU loss as its classification and regression losses; (2e) cascading the data-level enhancement module, feature-level enhancement module, region proposal module, and decision-level enhancement module in sequence to form a multi-level enhancement network; (3) Training the multi-level enhancement network: (3a) Randomly sample a set of SAR images from the training set and input them into the multi-stage enhancement network. Calculate the loss and update the network parameters based on the loss using the stochastic gradient descent algorithm. (3b) Repeat the process (3a) until the network converges and a trained multi-level enhanced network is obtained; (4) Input the SAR images in the test set into the trained multi-level enhancement network to obtain the detection and recognition results.

2. The method according to claim 1, characterized in that The backbone network A in step (2b) consists of five cascaded convolutional modules a1, a2, a3, a4, and a5; The first convolution module a1 consists of a 7×7 standard convolution layer, a batch normalization layer, a ReLU activation layer, and a maximum pooling downsampling layer cascade; The second convolution module a2 is composed of 3 residual blocks in cascade; The third convolution module a3 is composed of 4 residual blocks in cascade; The fourth convolution module a4 is composed of 6 residual blocks cascaded; The fifth convolution module a5 is composed of three cascaded residual blocks; The output feature map of the entire backbone network is expressed as where Y i is the i-th convolution module a i The output feature map of is the input SAR image with width W and height H respectively.

3. The method according to claim 1, characterized in that The feature selection pyramid network F in step (2b) includes four parallel branches f1, f2, f3, and f4. Each branch f i By feature optimization module f i s And feature fusion module f i u Cascade composition; Each feature optimization module f i s , consisting of two parallel sub-branches And 1×1 standard convolution layer cascade, the first sub-branch The structure of the network includes a global average pooling layer, a 1D convolutional layer, and a Sigmoid activation layer. The second sub-branch It is the identity branch; Each feature fusion module f i u It consists of 3×3 standard convolutional layers; The output feature map of the entire feature optimization pyramid network F is expressed as: Among them U i is the i-th feature fusion module f i u The output feature map of Conv3×3(·) represents 3×3 standard convolution, Up(·) represents the bilinear interpolation upsampling function, S i Optimize module f for the i-th feature i s The output feature map is expressed as: where Y i+1 The i+1 convolutional module a of the backbone network i+1 The output feature map of , ⊙ represents channel-by-channel multiplication operation, GAP(·) represents global average pooling, Conv1d(·) represents 1D convolution, σ(·) represents Sigmoid function, and Conv1×1(·) represents 1×1 standard convolution.

4. The method according to claim 1, wherein The recursive backbone network Q in step (2b) consists of five cascaded convolutional modules q1, q2, q3, q4, and q5; The first convolution module q1 consists of a 7×7 standard convolution layer, a batch normalization layer, a ReLU activation layer, and a maximum pooling downsampling layer cascade; The second convolution module q2 consists of three cascaded residual blocks and a 1×1 standard convolution layer in parallel; The third convolution module q3 consists of four cascaded residual blocks and a 1×1 standard convolution layer in parallel; The fourth convolution module q4 consists of 6 cascaded residual blocks and a 1×1 standard convolution layer in parallel; The fifth convolution module q5 consists of three cascaded residual blocks and a 1×1 standard convolution layer in parallel; The output feature map of the entire recursive backbone network A is expressed as: where Z i is the i-th convolution module q i The output feature map of is the input SAR image with width W and height H respectively, U i-1 The i-1th feature fusion module of the feature optimization pyramid network The output feature map of , Conv1×1(·) represents 1×1 standard convolution.

5. The method according to claim 1, wherein The recursive feature optimization pyramid network E in step (2b) has the same structure and parameters as the feature optimization pyramid network F, and its output feature map is represented as follows: Among them, P i is the i-th feature fusion module f i u The output feature map of Conv3×3(·) represents 3×3 standard convolution, Up(·) represents the bilinear interpolation upsampling function, W i Optimize module f for the i-th feature i s The output feature map is expressed as: IN i =f i s (WITH i+1 ) =Conv1×1(σ(Conv1d(GAP(Z i+1 )))⊙Z i+1 ),i=1,2,3,4where Z i+1 is the i+1th convolutional module q of the recursive backbone network i+1 The output feature map of , ⊙ represents channel-by-channel multiplication operation, GAP(·) represents global average pooling, Conv1d(·) represents 1D convolution, σ(·) represents Sigmoid function, and Conv1×1(·) represents 1×1 standard convolution.

6. The method according to claim 1, characterized in that The region proposal network in step (2c) includes a candidate region generation module, a classification and regression module, a post-processing module, and a positive and negative sample allocation module. The candidate region generation module is used to input the feature map P i The generated area at each point is 2 2i+2 , three rectangular candidate regions with width and height ratios of 1:2, 1:1, and 2:1 respectively; The classification and regression module includes two parallel 3×3 standard convolutional layers g1 and g2. The first convolutional layer g1 is used to adjust the center point position and width and height of the candidate region, and the second convolutional layer g2 is used to predict the target confidence of the candidate region. The post-processing module is used to filter redundant candidate regions and output the top N candidate regions with the highest target confidence {(p j ,b j )|j=1,2,...,N}, where p j is the target confidence of the j-th candidate region, b j =(x j ,y j ,w j ,h j ) is the bounding box of the jth candidate region, (x j ,y j ) is the center point coordinate of the bounding box, (w j ,h j ) are the width and height of the bounding box; The positive and negative sample allocation module is used to allocate candidate regions as positive and negative samples, that is, to allocate candidate regions with an intersection-over-union ratio with the labeled bounding box greater than 0.7 as positive samples, and to allocate candidate regions with an intersection-over-union ratio with the labeled bounding box less than 0.3 as negative samples; The bounding box output by the entire region proposal network is represented as: R G ={b j =(x j ,y j ,w j ,h j )|j=1,2,...N}.

7. The method according to claim 1, characterized in that The three cascaded sub-decision makers d1, d2, and d3 in step (2d) have the same structure. Each decision maker includes a region of interest extraction module, a classification and regression module, and a positive and negative sample allocation module in sequence. The region of interest extraction module is composed of a cascade of an adaptive average pooling layer and a flattening layer. The adaptive average pooling layer is used to pool the candidate region features into 7×7 features of interest, and the flattening layer is used to flatten the features of interest. The classification and regression module consists of two linear layers in parallel, the first linear layer adjusts the position of the bounding box, and the second linear layer predicts the category score of the bounding box; The positive and negative sample allocation module is used to allocate bounding boxes as positive and negative samples, that is, to allocate bounding boxes whose intersection-over-union ratio with the labeled bounding box is greater than a threshold as positive samples, and to allocate bounding boxes whose intersection-over-union ratio with the labeled bounding box is less than a threshold as negative samples. The thresholds of the three sub-decision makers are 0.5, 0.6, and 0.7 respectively; The bounding box output by the decision-level enhancement module is represented as: Where N is the number of bounding boxes, is the position of the j-th bounding box output by the third sub-decision maker, Respectively represent The center point coordinates and width and height, The mean of the j-th bounding box category scores output by each sub-decision maker is expressed as: where N c is the total number of categories, Represent the category scores of the j-th bounding box output by sub-decision makers d1, d2, and d3 respectively.

8. The method according to claim 1, characterized in that The loss in step (3a), including the loss of the region proposal module and the loss of the decision-level enhancement module They are represented as follows: Among them, N G is the number of candidate regions randomly sampled by the region proposal module, p m and are the target confidence and the corresponding true label of the mth candidate region sampled by the region proposal module, b m and are the bounding box of the mth candidate region sampled by the region proposal module and the corresponding true label, L cls and L reg are cross entropy classification loss and CIOU regression loss, N D is the number of bounding boxes randomly sampled by each sub-decision maker, and are the category scores and corresponding true labels of the lth bounding box sampled by the i-th sub-decision maker, and are the lth bounding box sampled by the i-th sub-decision maker and the corresponding true label, λ i is the loss weight of the i-th sub-decision maker, satisfying is the activation function, and its expression is: The loss of the entire multi-level enhancement network is expressed as:

9. The method according to claim 1, characterized in that In step (3b), the network parameters are updated using stochastic gradient descent, which is implemented as follows: (3b1) The gradient of the multi-level enhanced network parameters is solved and expressed as: in is the loss of the multi-level enhancement network, and are the losses of the region proposal module and the decision-level enhancement module, respectively, and θ is the learnable parameter of the multi-level enhancement network; (3b2), according to the gradient of the solution Update the parameters of the multi-level enhancement network, expressed as: Where θ′ is the updated network parameter, θ is the updated network parameter; lr is the learning rate, which is set according to the input image batch size.

Images