Radar image detection method based on LBP features and improved Yolov5

Abstract

The present invention discloses a radar image detection method based on LBP features and improved Yolov5. The method extracts LBP features from the original SAR image and fuses them with the original image as training data. Deformable convolution is added to the C3 module of the Yolov5 model feature extraction network, and the feature map is upsampled using the feature-based content-aware reorganization CARAFE operator in the feature fusion pyramid network module, and feature expression is enhanced using adaptive feature fusion ASFF in the three prediction feature layers. The training data is then used to train the network, and the trained network is used to perform radar image target detection. This method supplements the training data with low-level texture information, reduces background interference in complex scenes, and improves the overall target detection performance by fusing low-level local texture features with high-level semantic features of the deep network.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar target detection, and in particular relates to a radar image detection method based on LBP features and improved Yolov5. Background Art

[0002] Synthetic Aperture Radar (SAR) is an advanced active microwave sensor. Compared to other remote sensing tools such as optical, infrared, and hyperspectral sensors, it offers all-day, all-weather Earth observation capabilities, unrestricted by lighting and weather conditions. It is widely used in geological surveys, disaster relief, and ocean surveillance. Detecting vehicles and ships in SAR images is crucial. However, SAR image data is massive, and manual detection is inefficient. SAR target detection methods based on convolutional neural networks are attracting increasing attention due to their powerful feature extraction capabilities and the lack of manual intervention.

[0003] Common target detection methods in the existing technology are categorized into single-stage algorithms such as YOLO, SSD, and RetinaNet, and two-stage algorithms such as Faster R-CNN and Cascade R-CNN. Although the features learned by neural networks are relatively abstract and difficult for humans to understand, they are faster and more accurate than humans in target detection. However, with the deepening of feature extraction, the output results of most neural networks rely on high-level deep abstract features while ignoring low-level texture features, resulting in insufficient feature expression. Furthermore, SAR images inherently contain coherent speckle noise, which makes target edges unclear. In complex scenes, the high clutter intensity generates a lot of interference information, which seriously affects the performance of the detection network and causes a large number of false alarms or missed detections. Summary of the Invention

[0004] To address the shortcomings of the existing technology, the present invention proposes a radar image detection method based on LBP features and improved Yolov5. By fusing LBP (Local Binary Pattern) features into the original image to supplement low-level texture information, the deformable convolution and upsampling operator CARAFE are introduced into the Yolov5 model to improve the detection performance.

[0005] The radar image detection method based on LBP features and improved Yolov5 includes the following steps:

[0006] Step 1: Collect SAR images, grayscale them, and crop them into slices of equal size. For each slice, extract the LBP features from one channel image. Then, merge them with the remaining two channel images along the channel dimension. The resulting three-channel image serves as the training sample. Annotate the target locations within the sample as training labels.

[0007] The LBP feature extraction method is: for each pixel in the image, the local LBP feature value VAR is calculated in turn P,R , get the LBP features of the image:

[0008]

[0009]

[0010] Among them, R represents the neighborhood radius of the pixel point, P represents the number of sampling points in the area, and g p Represents the pixel value of the pth sampling point, p = 0, 2, ..., P-1.

[0011] Step 2. Build a Yolov5, then replace the second convolution of Bottleneck with deformable convolution DCN in the C3 module of the backbone network (backone), use the upsampling operator CARAFE for feature upsampling in the feature pyramid, and finally use the adaptive feature fusion ASFF module in the three prediction feature layers of the head structure (head) to enhance feature expression and obtain an improved Yolov5 model.

[0012] Step 3: Input the training samples obtained in step 1 into the improved Yolov5 model constructed in step 2, calculate the bounding box regression loss based on the model output, and optimize the model parameters.

[0013] Step 4: For the SAR image to be detected, extract and fuse the LBP features according to the method in step 1, and then input them into the model trained in step 3 to obtain the target position in the SAR image.

[0014] The present invention has the following beneficial effects:

[0015] 1. This method introduces the LBP features of SAR images, supplements the low-level texture information, reduces the interference of the background in complex scenes, provides more helpful information for the network input data, reduces the possibility of false detection, and improves the detection performance of the algorithm.

[0016] 2. The present invention improves the feature extraction capability by incorporating deformable convolution, allowing the model to fit the multi-scale deformation of the target, reducing background interference, and making it easier to separate the target and background in complex scenes; using the CARAFE upsampling operator, the feature map is upsampled based on the content-aware reconstruction of the features, so that the features contain richer semantic information; using adaptive feature fusion ASFF to adaptively fuse the predicted feature layer, making the feature expression more complete, and further making full use of the semantic information obtained by the CARAFE upsampling operator, thereby obtaining better SAR target detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flowchart of the radar image detection method based on LBP features and improved Yolov5 in the embodiment;

[0018] Figure 2 Schematic diagram of sampling points for extracting LBP features in the embodiment;

[0019] Figure 3 is the LBP feature map extracted in the embodiment;

[0020] Figure 4 The improved Yolov5 model constructed in the embodiment;

[0021] Figure 5 Schematic diagram of the improved C3 module in the embodiment;

[0022] Figure 6 Schematic diagram of the variable convolution DCN;

[0023] Figure 7 Schematic diagram of the upsampling operator CARAFE;

[0024] Figure 8 Schematic diagram of the adaptive feature fusion ASFF module;

[0025] Figure 9 3. It is a comparison chart of the detection results of various schemes in the embodiment. DETAILED DESCRIPTION

[0026] The present invention will be further explained below with reference to the accompanying drawings;

[0027] Radar image detection method based on LBP features and improved Yolov5, such as Figure 1 As shown, the specific steps include:

[0028] Step 1: Nine images were selected from the MiniSAR dataset and cropped into 300×300 slices using overlapping sliding windows, totaling 1171 images, as the original training set A. Each slice in the original training set A was then grayscaled, and the LBP features of the first channel image were extracted. These were then concatenated and fused with the remaining two channel images along the channel dimension. The resulting three-channel images were used as training samples, and the target locations in the samples were annotated as training labels, resulting in the new training set A'. Three images were selected from the MiniSAR dataset and, following the same method, 293 slices were obtained as the original test set B. After LBP feature extraction and fusion, the target locations were annotated, resulting in the new test set B'.

[0029] like Figure 2 As shown, for each pixel point in the original SAR image slice, 16 sampling points are collected in the area with a radius of 2, and the variance VAR of the pixel values of these 16 sampling points is calculated. P,R , as the LBP eigenvalue of the pixel:

[0030]

[0031]

[0032] Among them, R represents the neighborhood radius of the pixel point, P represents the number of sampling points in the area, and g p Represents the pixel value of the pth sampling point, p = 0, 2, ..., P-1.

[0033] Traverse each pixel point and calculate its LBP eigenvalue to obtain the LBP feature map corresponding to the original SAR image slice, such as Figure 3 As shown in Figure 1, the LBP eigenvalue of each pixel is scaled to the interval [0, 225] and fused with the other two channel images of the original SAR image slice in the channel dimension to form a new three-channel image, that is, to obtain new training samples and test samples.

[0034] Step 2: Build a Yolov5, then replace the second convolution of Bottleneck with deformable convolution DCN in the C3 module of the backbone network (backone), use the upsampling operator CARAFE to upsample the features in the feature pyramid, and finally use the adaptive feature fusion ASFF module to enhance the feature expression in the three prediction feature layers of the head structure (head), and obtain the improved Yolov5 model, as shown in the figure. Figure 4 shown.

[0035] Step 2.1, the C3 module in Yolov5 includes 3 Conv modules and 1 Bottleneck module. The Bottleneck module includes two consecutive convolutions and a residual connection. This method replaces the second convolution of Bottleneck with deformable convolution DCN, as shown in Figure 5 shown.

[0036] The deformable convolution DCN first performs a convolution operation on the input feature map to obtain a set of feature maps with convolution kernel offsets. The size of the offset feature map is the same as the size of the input feature map, and the number of channels is 3N, where N is the number of pixels of the convolution kernel, and 3 represents the offset (x, y) of the convolution kernel in the x-axis and y-axis directions, as well as the weight parameter △m of the offset (x, y). Figure 6 As shown in Figure 2, the output of the pixel point x(p0) with the initial position p0 in the input feature map x after deformable convolution is:

[0037]

[0038] Where w is the weight parameter of the convolution kernel, Z represents the convolution area, Z = {(-1,-1), (-1,0), ...., (0,1), (1,1)}, where (-1,-1) represents the position one unit length to the left and below x(p0), and (1,1) represents the position one unit length to the right and above x(p0). Δp n is the learnable offset parameter, △m∈[0,1].

[0039] Step 2.2, such as Figure 7 As shown in FIG, the upsampling operator CARAFE performs an upsampling operation on the feature map based on content-aware reorganization of features, including an upsampling kernel prediction module and a feature reorganization module.

[0040] The upsampling kernel prediction module includes three parts: channel compression, content encoding and upsampling kernel prediction, and upsampling kernel normalization. For an input feature map of size H×W×C, the upsampling kernel prediction module first compresses its channel number from C to C through 1×1 convolution. m , in order to reduce the amount of subsequent calculations. Then use the input channel number C m , the number of output channels is A convolution of size 3 is used to encode the content, and then a pixel shuffle operation is used to expand the channel dimension in the spatial dimension to obtain a shape of Finally, the upsampling kernel is normalized using SoftMax to complete the upsampling kernel prediction.

[0041] The feature recombining module outputs an output feature map of size σH×σW×C. For each position l'=(i',j') in the output feature map, Kup ×K up The neighborhood of the same size is dot-producted with its corresponding upsampling kernel and mapped back to the original feature map l = (i, j), i = i' / σ, j = j' / σ.

[0042] In this embodiment, set C m =64, upsampling factor σ=2, upsampling kernel size K up =5.

[0043] Step 2.3, such as Figure 8 As shown in , for the feature maps of three different levels in the head structure prediction feature layer, weighted summation is performed on each level and the feature maps of the other two levels in turn, making the feature expression more complete, as shown in Figure 8 As shown:

[0044]

[0045] in Represents the fusion result of the lth level, l = 1, 2, 3. Represents the features from the 1st to 3rd levels respectively, which are After upsampling or downsampling, the feature map obtained by 1×1 convolution has the same size and number of channels as the feature map of the lth level. Represents the corresponding fusion space weight, which is automatically trained by the network to achieve adaptive feature fusion, and

[0046] Step 3: Input the samples in the new training set obtained in step 1 into the improved Yolov5 model constructed in step 2 for network training. The software platform used in this example is the Windows 10 operating system and Pytorch 1.8.1, and the hardware configuration is an i9-13900K CPU and an NVIDIA GeForce RTX 3090 GPU. The number of training rounds is set to 300, the batch size is 64, CIOU Loss is used as the loss function for bounding box regression, and the stochastic gradient descent (SGD) optimizer is used to optimize the model parameters. The learning rate lr is set to 0.001, and the IOU threshold is set to 0.5.

[0047] The CIOU Loss loss function is:

[0048]

[0049]

[0050]

[0051]

[0052] Where A∩B is the intersection area of A’s bounding box and B’s bounding box, and A∪B is the union area. A and h A Represents the width and height of the bounding box A, w B and h B Represent the width and height of the bounding box B, ρ 2 (A, B) is the distance between the centers of the two boxes, and c is the length of the diagonals of the two bounding boxes.

[0053] Step 4: Input the samples in the new test set B' into the model trained in step 3, compare the prediction results output by the model with the labels, and calculate the model performance indicators:

[0054]

[0055]

[0056]

[0057]

[0058] Among them, TP represents the number of actual positive classes predicted as positive classes; FP represents the number of actual negative classes predicted as positive classes; FN represents the number of actual positive classes predicted as negative classes; P(R) represents the Precision-Recall curve.

[0059] To verify the effectiveness of this method, this example selected single-stage object detection algorithms YOLOV3, RetinaNet, and SSD, as well as two-stage object detection algorithms Faster-RCNN, Cascade-RCNN, Grid-RCNN, and Libra-RCNN for comparative experiments. The test parameter confidence threshold was set to 0.5, and the IOU threshold was set to 0.5. The comparison algorithm was implemented based on the MMDetection tool, and the network structure is shown in Table 1:

[0060] Table 1

[0061]

[0062] The detection results are visualized on the test image, such as Figure 9 As shown, the solid rectangle represents the target that is detected correctly, the dotted rectangle represents the false alarm, and the circle represents the missed detection. The specific performance index comparison results are shown in Table 2:

[0063] Table 2

[0064]

[0065] From the data in Table 2, we can see that the Precision, mAP, and F1 values of this method have all reached the best. Figure 9 The detection results also show that the five methods (Cascade-RCNN, Libra-RCNN, Grid-RCNN, Libra-RCNN, and RetinaNet) all produce a large number of false alarms. YOLOV3 has relatively few false alarms, but some missed detections. Our method, on the other hand, has only three false alarms, far fewer than the other methods, and no missed detections, demonstrating that our method effectively improves SAR image detection performance.

Claims

1. Radar image detection method based on LBP features and improved Yolov5, characterized by: The method specifically comprises the following steps: Step 1: Collect SAR images, grayscale them and cut them into slices of the same size; for each pixel in the image, calculate its local LBP eigenvalue VAR in turn P,R , get the LBP features of the entire image: Among them, R represents the neighborhood radius of the pixel point, P represents the number of sampling points in the area, and g p Represents the pixel value of the p-th sampling point, p = 0, 2, ..., P-1; For each slice, the LBP features of one channel image are extracted, and then spliced and fused with the other two channel images in the channel dimension. The resulting new three-channel image is used as a training sample; the target position in the sample is marked as a training label; Step 2: Build a Yolov5, then replace the second convolution of Bottleneck with deformable convolution DCN in the C3 module of the backbone network; use the upsampling operator CARAFE for feature upsampling in the feature pyramid, and finally use the adaptive feature fusion ASFF module in the three prediction feature layers of the head structure to obtain an improved Yolov5 model; The deformable convolution DCN first performs a convolution operation on the input feature map. The output of the pixel point x(p0) with the initial position p0 in the input feature map after deformable convolution is: Among them, w is the weight parameter of the convolution kernel, Z is the convolution area; Δp n is a learnable offset parameter, △m represents the weight parameter of the convolution kernel offset, △m∈[0,1]; The adaptive feature fusion (ASFF) module predicts the feature maps of three different levels in the head structure feature layer and performs weighted summation on the feature maps of the other two levels at each level: in represents the fusion result of the lth level, l = 1, 2, 3; Represents the features from the 1st to 3rd levels respectively, which are After upsampling or downsampling, the feature map obtained by 1×1 convolution has the same size and number of channels as the feature map of the lth level; Represents the corresponding fusion space weight, which is automatically trained by the network. Step 3: Input the training samples obtained in step 1 into the improved Yolov5 model constructed in step 2, calculate the bounding box regression loss based on the model output, and optimize the model parameters; Step 4: For the SAR image to be detected, extract and fuse the LBP features according to the method in step 1, and then input them into the model trained in step 3 to obtain the target position in the SAR image.

2. The radar image detection method based on LBP features and improved Yolov5 as claimed in claim 1, characterized in that: Set the neighborhood radius R=2 and the number of sampling points P=16.

3. The radar image detection method based on LBP features and improved Yolov5 as claimed in claim 1, characterized in that: The upsampling operator CARAFE includes an upsampling kernel prediction module and a feature reconstruction module; The upsampling kernel prediction module first compresses the number of channels of the input feature map of size H×W×C from C to C through 1×1 convolution. m , then use the input channel number C m , the number of output channels is A convolution of size 3 is used to encode the content, and then a pixel shuffle operation is used to expand the channel dimension in the spatial dimension to obtain a shape of Finally, the upsampling kernel is normalized using SoftMax to complete the upsampling kernel prediction; σ represents the upsampling ratio, K up Indicates the upsampling kernel size; The feature reorganization module outputs an output feature map of size σH×σW×C. For each position l'=(i',j') in the output feature map, K up ×K up The neighborhood of the same size is dot-producted with its corresponding upsampling kernel and mapped back to the original feature map l = (i, j), i = i' / σ, j = j' / σ.

4. The radar image detection method based on LBP features and improved Yolov5 as claimed in claim 3, characterized in that: Setting C m =64, upsampling factor σ=2, upsampling kernel size K up =5.

5. The radar image detection method based on LBP features and improved Yolov5 as claimed in claim 1, characterized in that: In step 3, the number of training rounds is set to 300, the batch-size is 64, CIOU Loss is used as the loss function for bounding box regression, the stochastic gradient descent SGD optimizer is used to optimize the model parameters, the learning rate lr is set to 0.001, and the IOU threshold is set to 0.

5.

6. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 5.

Images