Method for detecting ship target in large-scene wide remote sensing image based on satellite-borne SAR (Synthetic Aperture Radar)

Abstract

The invention discloses a ship target detection method based on a satellite-borne SAR large-scene wide remote sensing image. The method comprises the following steps: constructing a ship target detection data set of a synthetic aperture radar SAR; constructing a YOLOv7 deep learning model for improving the attention mechanism; training a target detection model by using a YOLOv7 deep learning method for improving an attention mechanism; preprocessing an image to be detected in the satellite-borne SAR remote sensing image; sending the divided sub-region images into a detection model; and integrating and outputting detection results. According to the method, parallel operation is carried out on the divided sub-regions, and meanwhile, each region adopts an independent quantization operation to improve the regional image quality, so that the timeliness and precision of ship target detection are remarkably improved. In a ship target detection task, features of an interested target area can be better concerned, and the method has higher precision and stronger robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of marine target image recognition, and in particular to a method for detecting ship targets in large-scene and wide-width remote sensing images based on satellite-borne SAR. Background Art

[0002] Synthetic Aperture Radar (SAR) is an advanced remote sensing technology that uses microwaves for long-distance imaging. Compared with optical remote sensing, SAR has unique advantages, such as being insensitive to weather and lighting conditions and being able to observe at night and under clouds. This makes SAR technology widely used in fields such as ocean monitoring, geological exploration, and military reconnaissance. In this context, research on spaceborne SAR images for ship target detection has attracted increasing attention.

[0003] First of all, the working principle of SAR technology is to acquire the image of the target by emitting microwave pulses and recording the echoes returned. Compared with traditional radar, SAR can provide high-resolution images and observe the tiny details of surface targets, which provides a basis for the accurate detection of ship targets. In spaceborne SAR images, the detection of ship targets faces some challenges. First, the ocean surface has complex background interference, such as waves, ripples, and oil films, which makes it more difficult to extract the target. Secondly, the inherent characteristics of the ship, such as irregular shapes and different radar reflection cross sections, increase the complexity of target detection.

[0004] Target detection algorithms based on deep learning have been widely introduced. By training deep neural networks, they can learn and identify ship targets in SAR images. This method has achieved remarkable results in dealing with complex background interference and diverse target features. However, due to the development of spaceborne SAR observation technology, the imaging width has reached a large scale, and conventional intelligent learning networks are often not directly applied to target detection tasks. At the same time, quantization and sliding window operations of the original image will greatly improve the timeliness of target detection tasks. Therefore, efficient ship target detection is a key issue that needs to be urgently solved in the application of wide-width SAR ocean images. Summary of the invention

[0005] The object of the present invention is to provide a method for detecting ship targets in large-scene and wide-width remote sensing images based on spaceborne SAR, comprising the following steps:

[0006] Step 1: Build a ship target detection dataset for synthetic aperture radar SAR. Based on the beam and strip mode data of GF-3 and combined with AIS ship target data, construct a ship target slice with a pixel resolution of 640×640. Combined with AIS information, interpret the target in the slice, and use the VOC format annotation method to standardize the ship target.

[0007] Step 2: Build a YOLOv7 deep learning model with an improved attention mechanism. Build a YOLOv7 deep learning target detection network based on the YOLOv7 backbone network, detection head network, and connection structure. At the same time, add CBAM channels and spatial attention mechanisms to the P4 and P5 layers of the backbone network to guide feature map transmission and improve the backbone network's attention to the target when extracting feature information.

[0008] Step 3: Use the YOLOv7 deep learning method with improved attention mechanism to train the target detection model. Divide the data set constructed in step 1 into training set, validation set, and test set according to the ratio of 7:2:1. Combine the label information to train the improved YOLOv7 deep learning model constructed in step 2. Evaluate the model performance based on the test set and retain the weight with the best performance.

[0009] Step 4, preprocess the image to be detected in the spaceborne SAR remote sensing image, divide it into sub-region images, read the wide-band SAR ocean image to be detected in matrix form, set the overlap distance to 100 to 300 pixels according to the image resolution, and divide the large scene image into regional grids of 800×800 pixels in size on the principle of retaining the overlap distance in adjacent areas, and quantify the sub-matrix in each grid interval based on grayscale histogram statistics;

[0010] Step 5: Send the divided sub-region images to the detection model, perform sub-region image ship target detection, load the improved YOLOv7 deep learning model trained in step 3, send the sub-region images divided in step 4 to the detection model, and obtain the ship target detection results of each sub-region;

[0011] Step 6: Integration and output of detection results. For the target detection results of each sub-region in step 5, coordinate mapping is performed according to the position of the sub-region in the original wide-width SAR ocean image, and non-maximum suppression operation is performed. The image coordinates and longitude and latitude coordinates are converted to obtain the longitude and latitude information of the ship target in the wide-width SAR ocean image, and the information is output to complete the target detection task.

[0012] Further, the step 1 includes the sub-steps of:

[0013] Step 1.1, data collection, select appropriate SAR image data sources, including TerraSAR-X, Sentinel-1 and GF-3, collect SAR image data containing ship targets, the SAR image data is a public data set, including the OpenSARShip data set or the original SAR image acquired by satellite;

[0014] Step 1.2: Data preprocessing: preprocess the collected SAR images, including denoising and correction. Based on the GF-3 spotlight and stripe mode data and combined with the AIS ship target data, construct a ship target slice with a pixel resolution of 640×640. Combined with the AIS information, interpret the target in the slice to ensure that the resolution and quality of the image meet the requirements of target detection.

[0015] Step 1.3: Data annotation: Use the VOC format annotation method to accurately annotate the ship targets in the SAR image, including the location, size and category of the ship;

[0016] Step 1.4: Data enhancement: Apply data enhancement techniques to rotate, scale, and crop to increase the diversity and robustness of the data set, simulate motion on the SAR image, and add a secondary phase error in the range-Doppler domain to simulate the imaging results of moving ship targets at sea.

[0017] Step 1.5: Dataset construction: Integrate the preprocessed, annotated and enhanced datasets to form the final SAR ship target detection dataset;

[0018] Step 1.6: Model training and evaluation: Use the constructed dataset to train the SAR ship target detection model, including the improved MobilenetV3 neural network. During the training process, regularly evaluate the performance of the model to ensure that the model can effectively detect ship targets in SAR images.

[0019] Step 1.7: Iterative optimization: According to the evaluation results of the model, the dataset is iteratively optimized, including adding new data and improving the annotation method.

[0020] Further, the step 2 includes the following sub-steps:

[0021] Step 2.1. Select the YOLOv7 model structure. YOLOv7 is a target detection model based on convolutional neural network. It divides the input image into multiple grids, and each grid is responsible for predicting the target whose center point falls within the grid.

[0022] Step 2.2, select an attention mechanism, which enables the model to focus on the most important part of the image, thereby improving the accuracy of detection. In YOLOv7, attention mechanisms including CBAM, SE, and ECA are used;

[0023] Step 2.3, modifying the model structure, according to the selected attention mechanism, modifying the network structure of YOLOv7, adding an attention module after one or some convolutional layers of YOLOv7, or introducing an attention mechanism in the feature fusion stage;

[0024] Step 2.4: Prepare a dataset containing the categories required for target detection and annotate them. The dataset containing the categories required for target detection contains diverse images to ensure that the model can learn features in various situations.

[0025] Step 2.5: Model training: Use the prepared data set to train the improved YOLOv7 model. During the training process, use the scale-invariant intersection-over-union (SIOU) as the bounding box loss function to improve the accuracy of bounding box regression.

[0026] Step 2.6: Model evaluation and optimization: Evaluate the performance of the model on the validation set, use the mean average precision (mAP) indicator to measure the detection effect of the model, and adjust and optimize the model based on the evaluation results;

[0027] Step 2.7: Iterate and improve. Based on the performance of the model in actual applications, continuously iterate and improve the model structure and training strategy. You may need to adjust the parameters of the attention mechanism or try different attention modules to find the best combination.

[0028] Step 2.8, code implementation;

[0029] Step 2.9, model deployment, after the model training and optimization are completed, it is deployed to practical applications, such as monitoring systems and autonomous driving vehicles, to perform real-time target detection.

[0030] Further, the step 3 includes the following sub-steps:

[0031] Step 3.1, Dataset preparation: Collect and prepare a dataset containing the categories required for target detection, pre-process the dataset by scaling and normalizing the images, and perform data enhancement by random cropping, rotation, and flipping to increase data diversity;

[0032] Step 3.2, model selection and improvement, select YOLOv7 as the basic model, improve the model structure as needed, embed ACmix attention mechanism in the Backbone layer to improve the model's detection efficiency for small targets, and embed ODconv attention mechanism in the Neck layer to enhance the model's ability to capture contextual clues;

[0033] Step 3.3, loss function selection, select the loss function Varifocal loss to train the set on high-quality positive examples;

[0034] Step 3.4, select a training strategy, use optimizer Adam or SGD, set a learning rate strategy, the learning rate strategy learning rate decay or use learning rate warm-up, use multi-scale training to improve the model's ability to detect targets of different sizes;

[0035] Step 3.5: Training process: Perform model training on GPU to accelerate the training process, monitor the loss and accuracy during training to ensure model convergence, and use the validation set to evaluate model performance to avoid overfitting.

[0036] Step 3.6: Model evaluation and optimization: Evaluate the model performance on the test set, use the mean average precision (mAP) indicator, adjust the model structure or training strategy according to the evaluation results, perform iterative optimization, and retain the weight with the best performance;

[0037] Step 3.7: Model deployment: deploy the trained model to actual applications for real-time target detection.

[0038] Further, the step 4 includes the following sub-steps:

[0039] Step 4.1: Check the image quality to ensure that the image is clear without image blur or noise. If the image quality is poor, perform image enhancement such as denoising and sharpening.

[0040] Step 4.2: Image format conversion: Convert the image to the format used for model training, including JPEG or PNG, to ensure that the image resolution is suitable for the input requirements of the model;

[0041] Step 4.3: Image resizing: resize the image to the size used during model training. The size is a fixed aspect ratio. If the image sizes are inconsistent, unify the size by scaling, cropping, or padding.

[0042] Step 4.4: Color space conversion: convert the image from the original color space to the RGB format used in model training. For grayscale images, convert them to single-channel grayscale images.

[0043] Step 4.5: Normalize the pixel values ​​of the image to make them between 0 and 1, or standardize them according to the requirements of the model. Normalization makes the model converge.

[0044] Step 4.6: Data enhancement: Apply data enhancement techniques to rotate, scale, crop, and flip to increase data diversity. Data enhancement can improve the generalization ability of the model.

[0045] Step 4.7. Select preprocessing tools and libraries. Use OpenCV, Pillow, TensorFlow, or PyTorch libraries to implement the above preprocessing steps. These libraries provide rich image processing functions to facilitate preprocessing operations.

[0046] Step 4.8, verification after preprocessing, after the preprocessing is completed, check whether the preprocessed image meets the requirements of the model, and verify the effect of preprocessing by visualizing the preprocessed image.

[0047] Further, the step 5 includes the following sub-steps:

[0048] Step 5.1: Image preprocessing: preprocessing the remote sensing image, including denoising, contrast enhancement, and brightness and contrast adjustment to improve image quality;

[0049] Step 5.2: Sub-region cutting: along the long axis of the ship, the ship image is densely cut into several local sub-regions to ensure that each sub-region is contained in the square annotation box and the optimal effective proportion of the sub-region area in the annotation box is ensured to ensure the generalization ability of the core detection network.

[0050] Step 5.3, feature extraction, using a deep learning model convolutional neural network CNN, to extract features from each sub-region, the extracted features include color, shape, texture, the extracted features are helpful to distinguish different types of ships;

[0051] Step 5.4: Sub-region identification: train a core network, take sub-regions as detection targets, integrate overlapping sub-regions, and use data enhancement technology to improve the generalization ability of the model during training;

[0052] Step 5.5, post-processing, based on sub-image segmentation, the detected sub-regions are merged to estimate the key ship target parameters of the direction angle, and the sub-region merging post-processing is used instead of the non-maximum suppression NMS post-processing to ensure the detection speed;

[0053] Step 5.6: Model evaluation: evaluate the performance of the model on a standard dataset, including the HRSC2016 dataset, and use the mean average precision (mAP) indicator to measure the detection accuracy of the model.

[0054] Step 5.7: Model optimization. Based on the evaluation results, optimize the model by adjusting the network structure and hyperparameters, and further improve the model performance by adding more training data or using transfer learning.

[0055] Furthermore, the sub-region image ship target detection can combine different deep learning models and feature extraction techniques to adapt to different remote sensing images and ship target detection needs, use YOLO, Faster R-CNN, SSD target detection models, and combine attention mechanism and feature fusion to improve detection accuracy.

[0056] Further, the step 6 includes the following sub-steps:

[0057] Step 6.1: Obtain the detection results. Obtain preliminary detection results from each target detection model. These detection results include the target's bounding box BBox, category label, and confidence score.

[0058] Step 6.2: Filter the results. Filter the detection results according to the confidence score and retain the results with confidence higher than a certain threshold to reduce false detections. Map the coordinates of the sub-regions according to their positions in the original wide-width SAR ocean image.

[0059] Step 6.3: Non-maximum suppression (NMS), where the NMS algorithm is used to remove overlapping detection boxes. The NMS compares the overlap and confidence scores of the bounding boxes, retains the bounding box with the highest confidence, and suppresses other bounding boxes with higher overlap;

[0060] Step 6.4: Result fusion: if multiple models are used for detection, weighted average, voting or other fusion strategies are used to integrate the detection results of different models; image coordinates and longitude and latitude coordinates are converted to obtain the longitude and latitude information of the ship target in the wide-width SAR ocean image, and output it to complete the target detection task;

[0061] Step 6.5: Post-processing: Post-process the integrated detection results to adjust the position and size of the bounding box to more accurately match the target;

[0062] Step 6.6, output format, output the final detection results in a standard format, the standard format includes JSON, XML or CSV file, which contains information about the target category, bounding box coordinates, and confidence score;

[0063] Step 6.7: Visualize the detection results and draw bounding boxes and category labels on the original image to facilitate manual review and verification.

[0064] Step 6.8: Performance evaluation: Use standard evaluation indicators mAP and IOU to evaluate the performance of the detection results.

[0065] Furthermore, in the result fusion of step 6.4, a method for fusing target detection results is adopted, and the fusion method specifically includes training to obtain at least two target detection models, obtaining the recognition weights of each target detection model for different classification types and the overlap thresholds corresponding to each classification type, and then inputting the image to be identified into each target detection model respectively, obtaining the target detection results output by each target detection model, and fusing the target detection results according to the recognition weights and the overlap thresholds to obtain the fused detection results corresponding to the image to be identified.

[0066] Furthermore, the integration and output steps of integrating the detection results of different models in the result fusion of step 6.4 are adjusted according to the model characteristics. For specific ship target detection tasks, the aspect ratio, posture, and occlusion factors of the ship are considered, and the detection and integration strategies are adjusted to ensure that the integration and output steps are consistent with the model training and reasoning environment.

[0067] The method of the present invention has the following advantages:

[0068] (1) Compared with the traditional sliding window detection algorithm, the wide-band SAR ocean image region division method based on overlapping cutting proposed in the present invention can reduce the overall task computational complexity while performing parallel computation design on the divided sub-regions. Meanwhile, independent quantization operations are performed on each region to further improve the regional image quality and significantly improve the timeliness and accuracy of ship target detection.

[0069] (2) The attention mechanism proposed in the present invention improves the YOLOv7 ship target detection algorithm. Compared with the traditional YOLO series of deep learning target detection models, it can better focus on the features of the target area of ​​interest in the ship target detection task, and has higher accuracy and stronger robustness.

[0070] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following is a preferred embodiment, which is described in detail with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 The present invention proposes a ship target detection method based on spaceborne SAR large-scene wide-area remote sensing images. DETAILED DESCRIPTION

[0072] The present invention provides a method for detecting ship targets in large-scene and wide-format satellite-borne SAR remote sensing images. Aiming at the problems that the sliding window detection mechanism of the traditional intelligent detection algorithm has low timeliness, occupies a large amount of computing resources, and is not conducive to real-time tasks, a large-scene region segmentation method for wide-format SAR images based on the overlapping cutting algorithm is proposed, and the intelligent detection network of YOLOv7 is improved in combination with the advanced attention mechanism, so as to realize efficient detection and extraction of ship targets in wide-format ocean SAR images.

[0073] The technical scheme of the present invention will be clearly and completely described below in conjunction with specific embodiments, but those skilled in the art should understand that the embodiments described below are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Aiming at the demand for ship target detection in large-scene and wide-format satellite-borne SAR remote sensing images, the present invention proposes a large-scene region segmentation method for wide-format SAR images based on an overlapping cutting algorithm, aiming at the problems that the sliding window detection mechanism of the traditional intelligent detection algorithm has low timeliness, occupies a large amount of computing resources, and is not conducive to real-time tasks. The intelligent detection network of YOLOv7 is improved in combination with the advanced attention mechanism, so as to realize efficient detection and extraction of ship targets in wide-format ocean SAR images.

[0075] The present invention provides a method for detecting ship targets in a spaceborne SAR large-scene wide-width remote sensing image, comprising the following steps:

[0076] Step 1: Construct a SAR ship target detection dataset;

[0077] Based on the beam and strip mode data of GF-3 and combined with AIS ship target data, a ship target slice with a pixel resolution of 640×640 is constructed. The targets in the slice are interpreted in combination with AIS information, and the ship targets are standardizedly annotated using the VOC format annotation method.

[0078] Building a SAR ship target detection dataset is a complex process, involving multiple steps such as data collection, preprocessing, annotation, and enhancement. The following are the specific steps for building a SAR ship target detection dataset:

[0079] Step 1.1: Data collection:

[0080] Select appropriate SAR image data sources, such as TerraSAR-X, Sentinel-1, and GF-3.

[0081] Collect SAR image data containing ship targets. These data can be public datasets, such as the OpenSARShip dataset, or raw SAR images acquired by satellite.

[0082] Step 1.2: Data preprocessing:

[0083] The collected SAR images are preprocessed, including denoising and correction. Based on the GF-3 beamforming and stripping mode data and combined with the AIS ship target data, a ship target slice with a pixel resolution of 640×640 is constructed. The targets in the slice are interpreted in combination with the AIS information to ensure that the resolution and quality of the image meet the requirements of target detection.

[0084] Step 1.3: Data labeling:

[0085] Accurately mark the ship targets in SAR images, including information such as the ship's location, size, and category.

[0086] Use professional annotation tools, such as LabelImg, CVAT, etc., to improve the accuracy and efficiency of annotation.

[0087] Step 1.4, data enhancement:

[0088] Apply data augmentation techniques such as rotation, scaling, cropping, etc. to increase the diversity and robustness of the dataset.

[0089] Perform motion simulation on SAR images, such as adding quadratic phase error in the range-Doppler domain to simulate the imaging results of moving ship targets at sea.

[0090] Step 1.5, data set construction:

[0091] The preprocessed, annotated and enhanced datasets are integrated to form the final SAR ship target detection dataset.

[0092] Ensure that the dataset is diverse and representative, covering different environments, weather conditions, and ship types.

[0093] Step 1.6, model training and evaluation:

[0094] Use the constructed dataset to train SAR ship target detection models, such as the improved MobilenetV3 neural network.

[0095] During the training process, the performance of the model is regularly evaluated to ensure that the model can effectively detect ship targets in SAR images.

[0096] Step 1.7, iterative optimization:

[0097] Based on the evaluation results of the model, the dataset is iteratively optimized, including adding new data, improving annotation methods, etc.

[0098] Building a SAR ship target detection dataset is an ongoing process that requires continuous collection of new data, improvement of annotation methods and enhancement techniques, and optimization of model training strategies. Through these steps, a high-quality SAR ship target detection dataset can be built, and the model structure and training strategy can be optimized to improve the accuracy and efficiency of target detection, providing a solid foundation for subsequent target detection research and applications.

[0099] Step 2: Build a YOLOv7 deep learning model with improved attention mechanism;

[0100] The YOLOv7 deep learning target detection network is constructed based on the YOLOv7 backbone network, detection head network and connection structure. At the same time, CBAM channels and spatial attention mechanisms are added to the P4 and P5 layers of the backbone network to guide the transmission of feature maps and enhance the backbone network's attention to the target when extracting feature information.

[0101] Building an attention-modified YOLOv7 deep learning model consists of the following steps:

[0102] Step 2.1, select the YOLOv7 model structure:

[0103] First of all, you need to have a deep understanding of the basic structure of YOLOv7. YOLOv7 is a target detection model based on convolutional neural network. It divides the input image into multiple grids, and each grid is responsible for predicting the target whose center point falls within the grid.

[0104] Step 2.2, choose the appropriate attention mechanism:

[0105] The attention mechanism can help the model focus on the most important parts of the image, thereby improving the accuracy of detection. In YOLOv7, you can consider using attention mechanisms such as CBAM, SE, and ECA.

[0106] Step 2.3: Modification of model structure:

[0107] According to the selected attention mechanism, the network structure of YOLOv7 is modified. For example, an attention module can be added after one or some convolutional layers of YOLOv7, or an attention mechanism can be introduced in the feature fusion stage.

[0108] Step 2.4, Dataset Preparation:

[0109] Prepare a dataset containing the categories required for object detection and annotate them. The dataset should contain diverse images to ensure that the model can learn features in a variety of situations.

[0110] Step 2.5, model training:

[0111] The improved YOLOv7 model is trained using the prepared dataset. During the training process, SIOU (scale-invariant intersection-over-union) can be used as the bounding box loss function to improve the accuracy of bounding box regression.

[0112] Step 2.6, model evaluation and optimization:

[0113] Evaluate the performance of the model on the validation set, using indicators such as mAP (mean average precision) to measure the detection effect of the model. Make necessary adjustments and optimizations to the model based on the evaluation results.

[0114] Step 2.7, iterative improvement:

[0115] Based on the performance of the model in practical applications, the model structure and training strategy are continuously improved iteratively. It may be necessary to adjust the parameters of the attention mechanism or try different attention modules to find the best combination.

[0116] Step 2.8, code implementation:

[0117] If necessary, you can refer to existing open source code implementations, such as the YOLOv7 improvement series mentioned in the CSDN blog, which contains the implementation of multiple attention mechanisms.

[0118] Step 2.9, model deployment:

[0119] After the model is trained and optimized, it can be deployed in real-world applications such as surveillance systems, autonomous vehicles, etc. for real-time object detection.

[0120] Through the above steps, a YOLOv7 deep learning model with improved attention mechanism is constructed to improve the performance of target detection. According to the evaluation results of the model, the data set is iteratively optimized, including adding new data, improving the annotation method, etc. The model structure and training strategy are optimized to improve the accuracy and efficiency of target detection.

[0121] Step 3: Improve YOLOv7 deep learning target detection model training;

[0122] The dataset constructed in step 1 is divided into training set, validation set, and test set according to the ratio of 7:2:1. The improved YOLOv7 deep learning model constructed in step 2 is trained with the label information, and the model performance is evaluated based on the test set, retaining the weight with the best performance.

[0123] The training of the improved YOLOv7 deep learning target detection model is carried out in the following steps:

[0124] Step 3.1, Dataset preparation:

[0125] Collect and prepare a dataset containing the categories required for object detection.

[0126] Preprocess the dataset, including image scaling and normalization.

[0127] Perform data augmentation, such as random cropping, rotation, flipping, etc., to increase data diversity.

[0128] Step 3.2: Model selection and improvement:

[0129] Choose YOLOv7 as the base model.

[0130] Improve the model structure as needed, such as embedding the ACmix attention mechanism in the Backbone layer to improve the model's detection efficiency for small objects.

[0131] The ODconv attention mechanism is embedded in the Neck layer to enhance the model’s ability to capture contextual clues.

[0132] Step 3.3, loss function selection:

[0133] Choose an appropriate loss function, such as Varifocal loss, to focus training on high-quality positive examples.

[0134] Step 3.4, training strategy:

[0135] Use an appropriate optimizer such as Adam or SGD.

[0136] Set the learning rate strategy, such as learning rate decay or using learning rate warmup.

[0137] Use multi-scale training to improve the model's ability to detect objects of different sizes.

[0138] Step 3.5, training process:

[0139] Model training is performed on GPU to speed up the training process.

[0140] Monitor the loss and accuracy during training to ensure the model converges.

[0141] Use the validation set to evaluate model performance and avoid overfitting.

[0142] Step 3.6, model evaluation and optimization:

[0143] Evaluate model performance on the test set using metrics such as mAP (mean average precision).

[0144] Adjust the model structure or training strategy based on the evaluation results, perform iterative optimization, and retain the weights with the best performance.

[0145] Step 3.7, model deployment:

[0146] Deploy the trained model to actual applications for real-time object detection.

[0147] Through the above steps, an improved YOLOv7 deep learning target detection model is constructed to improve the performance of target detection.

[0148] Step 4: Preprocessing of the image to be detected;

[0149] The wide-band SAR ocean image to be detected is read in matrix form. The overlap distance is set to 100 to 300 pixels according to the image resolution. The overlapping distance is retained in adjacent areas. The large scene image is divided into regional grids of 800×800 pixels in size, and the sub-matrix in each grid interval is quantified based on grayscale histogram statistics.

[0150] The preprocessing of the image to be detected is an important step in the target detection task, which directly affects the detection performance of the model. The following are the specific steps for preprocessing the image to be detected:

[0151] Step 4.1: Image quality check:

[0152] Make sure the image is clear without severe blur or noise.

[0153] If the image quality is poor, image enhancement may be required, such as denoising, sharpening, etc.

[0154] Step 4.2, image format conversion:

[0155] Convert the image to the format used when training the model, such as JPEG or PNG.

[0156] Make sure the image has a resolution that is appropriate for the model's input requirements.

[0157] Step 4.3, image resizing:

[0158] Resize the images to the size used when the model was trained, usually a fixed aspect ratio.

[0159] If the images are not the same size, you can resize them by scaling, cropping, or padding them.

[0160] Step 4.4, color space conversion:

[0161] Convert the image from its original color space to the color space used when the model was trained, such as RGB.

[0162] For grayscale images, they can be converted to single-channel grayscale images.

[0163] Step 4.5, normalization:

[0164] Normalize the pixel values ​​of the image to be between 0 and 1, or as required by the model.

[0165] Normalization helps the model converge faster.

[0166] Step 4.6, data enhancement:

[0167] Apply data augmentation techniques such as rotation, scaling, cropping, flipping, etc. to increase data diversity.

[0168] Data augmentation helps improve the generalization ability of the model.

[0169] Step 4.7, Preprocessing Tools and Libraries:

[0170] Use libraries such as OpenCV, Pillow, TensorFlow, or PyTorch to implement the above preprocessing steps.

[0171] These libraries provide rich image processing functions and can easily perform pre-processing operations.

[0172] Step 4.8: Verification after preprocessing:

[0173] After preprocessing is completed, check whether the preprocessed image meets the requirements of the model.

[0174] The effect of preprocessing can be verified by visualizing the preprocessed image.

[0175] In practice, the preprocessing steps may vary depending on the specific tasks and model requirements. For example, if the model is designed for specific SAR images, the preprocessing steps may need to include specific SAR image enhancement techniques. In addition, if the model needs to process multimodal data, the preprocessing steps may need to include fusion and alignment of different data types.

[0176] When performing preprocessing, ensure that the preprocessing steps are consistent with the model training and inference environments to avoid preprocessing mismatch issues in actual applications.

[0177] Step 5: Ship target detection in sub-region image;

[0178] Load the improved YOLOv7 deep learning model trained in step 3, send the sub-region images divided in step 4 to the detection model, and obtain the ship target detection results for each sub-region.

[0179] Sub-region image ship target detection is a detection method for ship targets in remote sensing images. It divides the ship image into multiple sub-regions, and then extracts and identifies features of these sub-regions to improve the accuracy and efficiency of detection. This method is particularly suitable for ship targets with large aspect ratios, which can reduce the false alarm rate of single-stage target detection and improve detection speed.

[0180] The following are the specific steps for detecting ship targets in sub-region images:

[0181] Step 5.1: Image preprocessing:

[0182] Preprocessing of remote sensing images, including denoising, contrast enhancement, brightness and contrast adjustment, etc., is performed to improve image quality.

[0183] Step 5.2: Sub-region cutting:

[0184] The ship image is densely cut into several local sub-regions along the long axis of the ship, ensuring that each sub-region is contained in the square annotation box.

[0185] Ensure the best effective proportion of sub-region area within the annotation box to ensure the generalization ability of the core detection network.

[0186] Step 5.3, feature extraction:

[0187] Using deep learning models, such as convolutional neural networks (CNN), features are extracted for each sub-region.

[0188] The extracted features include color, shape, texture, etc., which help to distinguish different ship types.

[0189] Step 5.4: Sub-region identification:

[0190] A core network is trained to detect sub-regions and integrate overlapping sub-regions.

[0191] During the training process, data augmentation techniques can be used to improve the generalization ability of the model.

[0192] Step 5.5, post-processing:

[0193] Based on sub-image segmentation, the detected sub-regions are merged to estimate key ship target parameters such as direction angle.

[0194] Sub-region merging post-processing is used instead of non-maximum suppression (NMS) post-processing to ensure detection speed.

[0195] Step 5.6, model evaluation:

[0196] The performance of the model is evaluated on standard datasets, such as the HRSC2016 dataset.

[0197] The detection accuracy of the model is measured using indicators such as mean average precision (mAP).

[0198] Step 5.7, model optimization:

[0199] According to the evaluation results, the model is optimized, such as adjusting the network structure and hyperparameters.

[0200] The model performance can be further improved by adding more training data or using transfer learning.

[0201] In practical applications, the sub-region image ship target detection method can be combined with different deep learning models and feature extraction techniques to adapt to different remote sensing images and ship target detection requirements. For example, target detection models such as YOLO, Faster R-CNN, SSD, etc. can be used, and combined with attention mechanisms, feature fusion and other technologies to improve detection accuracy.

[0202] The performance of the sub-region image ship target detection method is also affected by many factors such as image resolution, size and posture of the ship target, environmental factors (such as lighting and weather conditions), etc. Therefore, in practical applications, it may be necessary to appropriately adjust and optimize the method according to the specific situation.

[0203] Step 6: Integration and output of test results;

[0204] For the target detection results of each sub-region in step 5, coordinate mapping is performed according to the position of the sub-region in the original wide-width SAR ocean image, and a non-maximum suppression operation is performed. The image coordinates and longitude and latitude coordinates are converted to obtain the longitude and latitude information of the ship target in the wide-width SAR ocean image, and the information is output to complete the target detection task.

[0205] The integration and output of detection results is a key step in the object detection task, which involves merging the detection results of multiple models or different stages to generate the final detection results. The following are the specific steps for the integration and output of detection results:

[0206] Step 6.1, get the test results:

[0207] Get preliminary detection results from various target detection models. These results usually include the target's bounding box (BBox), category label, confidence score, etc.

[0208] Step 6.2, result screening:

[0209] The detection results are filtered according to the confidence score, usually only retaining the results with confidence above a certain threshold to reduce false detections.

[0210] Step 6.3, non-maximum suppression (NMS):

[0211] The non-maximum suppression (NMS) algorithm is applied to remove overlapping detection boxes. NMS compares the overlap and confidence scores of bounding boxes, retains the bounding box with the highest confidence, and suppresses other bounding boxes with high overlap.

[0212] Step 6.4: Result fusion:

[0213] If multiple models are used for detection, weighted averaging, voting, or other fusion strategies can be used to integrate the detection results of different models.

[0214] For sub-region image ship target detection, the present invention adopts a target detection result fusion method in result fusion, which specifically includes training to obtain at least two target detection models, obtaining the recognition weights of each target detection model for different classification types, and the overlap thresholds corresponding to each classification type, and then inputting the image to be identified into each target detection model respectively, obtaining the target detection results output by each target detection model, and fusing the target detection results according to the recognition weights and the overlap thresholds to obtain the fused detection results corresponding to the image to be identified.

[0215] Step 6.5, post-processing:

[0216] Post-process the integrated detection results, such as adjusting the position and size of the bounding box to more accurately match the target.

[0217] Step 6.6, output format:

[0218] The final detection results are output in a standard format, such as JSON, XML, or CSV file, which contains information such as the target category, bounding box coordinates, confidence score, etc.

[0219] Step 6.7, Visualization:

[0220] Visualize detection results, usually by drawing bounding boxes and class labels on the original image to facilitate human review and verification.

[0221] Step 6.8, performance evaluation:

[0222] Standard evaluation metrics (such as mAP, IOU, etc.) are used to evaluate the performance of the detection results.

[0223] In actual applications, the method of integrating and outputting detection results may be adjusted according to specific task requirements and model characteristics. For example, for a specific ship target detection task, it may be necessary to consider factors such as the ship's aspect ratio, posture, and occlusion to design a more appropriate detection and integration strategy, ensuring that the integration and output steps are consistent with the model training and reasoning environment to avoid mismatching problems in actual applications.

[0224] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for detecting ship targets in large-scene and wide-width remote sensing images based on spaceborne SAR, characterized in that: The steps include: Step 1: Build a ship target detection dataset for synthetic aperture radar SAR. Based on the beam and strip mode data of GF-3 and combined with AIS ship target data, construct a ship target slice with a pixel resolution of 640×640. Combined with AIS information, interpret the target in the slice, and use the VOC format annotation method to standardize the ship target. Step 2: Build a YOLOv7 deep learning model with an improved attention mechanism. Build a YOLOv7 deep learning target detection network based on the YOLOv7 backbone network, detection head network, and connection structure. At the same time, add CBAM channels and spatial attention mechanisms to the P4 and P5 layers of the backbone network to guide feature map transmission and improve the backbone network's attention to the target when extracting feature information. Step 3: Use the YOLOv7 deep learning method with improved attention mechanism to train the target detection model. Divide the data set constructed in step 1 into training set, validation set, and test set according to the ratio of 7:2:

1. Combine the label information to train the improved YOLOv7 deep learning model constructed in step 2. Evaluate the model performance based on the test set and retain the weight with the best performance. Step 4, preprocess the image to be detected in the spaceborne SAR remote sensing image, divide it into sub-region images, read the wide-band SAR ocean image to be detected in matrix form, set the overlap distance to 100 to 300 pixels according to the image resolution, and divide the large scene image into regional grids of 800×800 pixels in size on the principle of retaining the overlap distance in adjacent areas, and quantify the sub-matrix in each grid interval based on grayscale histogram statistics; Step 5: Send the divided sub-region images to the detection model, perform sub-region image ship target detection, load the improved YOLOv7 deep learning model trained in step 3, send the sub-region images divided in step 4 to the detection model, and obtain the ship target detection results of each sub-region; Step 6: Integration and output of detection results. For the target detection results of each sub-region in step 5, coordinate mapping is performed according to the position of the sub-region in the original wide-width SAR ocean image, and non-maximum suppression operation is performed. The image coordinates and longitude and latitude coordinates are converted to obtain the longitude and latitude information of the ship target in the wide-width SAR ocean image, and the information is output to complete the target detection task.

2. The method according to claim 1, characterized in that The step 1 comprises the following sub-steps: Step 1.1, data collection, select appropriate SAR image data sources, including TerraSAR-X, Sentinel-1 and GF-3, collect SAR image data containing ship targets, the SAR image data is a public data set, including the OpenSARShip data set or the original SAR image acquired by satellite; Step 1.2: Data preprocessing: preprocess the collected SAR images, including denoising and correction. Based on the GF-3 spotlight and stripe mode data and combined with the AIS ship target data, construct a ship target slice with a pixel resolution of 640×640. Combined with the AIS information, interpret the target in the slice to ensure that the resolution and quality of the image meet the requirements of target detection. Step 1.3: Data annotation: Use the VOC format annotation method to accurately annotate the ship targets in the SAR image, including the location, size and category of the ship; Step 1.4: Data enhancement: Apply data enhancement techniques to rotate, scale, and crop to increase the diversity and robustness of the data set, simulate motion on the SAR image, and add a secondary phase error in the range-Doppler domain to simulate the imaging results of moving ship targets at sea. Step 1.5: Dataset construction: Integrate the preprocessed, annotated and enhanced datasets to form the final SAR ship target detection dataset; Step 1.6: Model training and evaluation: Use the constructed dataset to train the SAR ship target detection model, including the improved MobilenetV3 neural network. During the training process, regularly evaluate the performance of the model to ensure that the model can effectively detect ship targets in SAR images. Step 1.7: Iterative optimization: According to the evaluation results of the model, the dataset is iteratively optimized, including adding new data and improving the annotation method.

3. The method according to claim 1, characterized in that The step 2 includes the following sub-steps: Step 2.

1. Select the YOLOv7 model structure. YOLOv7 is a target detection model based on convolutional neural network. It divides the input image into multiple grids, and each grid is responsible for predicting the target whose center point falls within the grid. Step 2.2, select an attention mechanism, which enables the model to focus on the most important part of the image, thereby improving the accuracy of detection. In YOLOv7, attention mechanisms including CBAM, SE, and ECA are used; Step 2.3, modifying the model structure, according to the selected attention mechanism, modifying the network structure of YOLOv7, adding an attention module after one or some convolutional layers of YOLOv7, or introducing an attention mechanism in the feature fusion stage; Step 2.4: Prepare a dataset containing the categories required for target detection and annotate them. The dataset containing the categories required for target detection contains diverse images to ensure that the model can learn features in various situations. Step 2.5: Model training: Use the prepared data set to train the improved YOLOv7 model. During the training process, use the scale-invariant intersection-over-union (SIOU) as the bounding box loss function to improve the accuracy of bounding box regression. Step 2.6: Model evaluation and optimization: Evaluate the performance of the model on the validation set, use the mean average precision (mAP) indicator to measure the detection effect of the model, and adjust and optimize the model based on the evaluation results; Step 2.7: Iterate and improve. Based on the performance of the model in actual applications, continuously iterate and improve the model structure and training strategy. You may need to adjust the parameters of the attention mechanism or try different attention modules to find the best combination. Step 2.8, code implementation; Step 2.9, model deployment, after the model training and optimization are completed, it is deployed to practical applications, such as monitoring systems and autonomous driving vehicles, to perform real-time target detection.

4. The method according to claim 1, characterized in that The step 3 includes the following sub-steps: Step 3.1, Dataset preparation: Collect and prepare a dataset containing the categories required for target detection, pre-process the dataset by scaling and normalizing the images, and perform data enhancement by random cropping, rotation, and flipping to increase data diversity; Step 3.2, model selection and improvement, select YOLOv7 as the basic model, improve the model structure as needed, embed ACmix attention mechanism in the Backbone layer to improve the model's detection efficiency for small targets, and embed ODconv attention mechanism in the Neck layer to enhance the model's ability to capture contextual clues; Step 3.3, loss function selection, select the loss function Varifocal loss to train the set on high-quality positive examples; Step 3.4, select a training strategy, use optimizer Adam or SGD, set a learning rate strategy, the learning rate strategy learning rate decay or use learning rate warm-up, use multi-scale training to improve the model's ability to detect targets of different sizes; Step 3.5: Training process: Perform model training on GPU to accelerate the training process, monitor the loss and accuracy during training to ensure model convergence, and use the validation set to evaluate model performance to avoid overfitting. Step 3.6: Model evaluation and optimization: Evaluate the model performance on the test set, use the mean average precision (mAP) indicator, adjust the model structure or training strategy according to the evaluation results, perform iterative optimization, and retain the weight with the best performance; Step 3.7: Model deployment: deploy the trained model to actual applications for real-time target detection.

5. The method according to claim 1, characterized in that The step 4 includes the following sub-steps: Step 4.1: Check the image quality to ensure that the image is clear without image blur or noise. If the image quality is poor, perform image enhancement such as denoising and sharpening. Step 4.2: Image format conversion: Convert the image to the format used for model training, including JPEG or PNG, to ensure that the image resolution is suitable for the input requirements of the model; Step 4.3: Image resizing: resize the image to the size used during model training. The size is a fixed aspect ratio. If the image sizes are inconsistent, unify the size by scaling, cropping, or padding. Step 4.4: Color space conversion: convert the image from the original color space to the RGB format used in model training. For grayscale images, convert them to single-channel grayscale images. Step 4.5: Normalize the pixel values ​​of the image to make them between 0 and 1, or standardize them according to the requirements of the model. Normalization makes the model converge. Step 4.6: Data enhancement: Apply data enhancement techniques to rotate, scale, crop, and flip to increase data diversity. Data enhancement can improve the generalization ability of the model. Step 4.

7. Select preprocessing tools and libraries. Use OpenCV, Pillow, TensorFlow, or PyTorch libraries to implement the above preprocessing steps. These libraries provide rich image processing functions to facilitate preprocessing operations. Step 4.8, verification after preprocessing. After preprocessing is completed, check whether the preprocessed image meets the requirements of the model and verify the effect of preprocessing by visualizing the preprocessed image.

6. The method according to claim 1, characterized in that The step 5 includes the following sub-steps: Step 5.1: Image preprocessing: preprocessing the remote sensing image, including denoising, contrast enhancement, and brightness and contrast adjustment to improve image quality; Step 5.2: Sub-region cutting: along the long axis of the ship, the ship image is densely cut into several local sub-regions to ensure that each sub-region is contained in the square annotation box and the optimal effective proportion of the sub-region area in the annotation box is ensured to ensure the generalization ability of the core detection network. Step 5.3, feature extraction, using a deep learning model convolutional neural network CNN, to extract features from each sub-region, the extracted features include color, shape, texture, the extracted features are helpful to distinguish different types of ships; Step 5.4: Sub-region identification: train a core network, take sub-regions as detection targets, integrate overlapping sub-regions, and use data enhancement technology to improve the generalization ability of the model during training; Step 5.5, post-processing, based on sub-image segmentation, the detected sub-regions are merged to estimate the key ship target parameters of the direction angle, and the sub-region merging post-processing is used instead of the non-maximum suppression NMS post-processing to ensure the detection speed; Step 5.6: Model evaluation: evaluate the performance of the model on a standard dataset, including the HRSC2016 dataset, and use the mean average precision (mAP) indicator to measure the detection accuracy of the model. Step 5.7: Model optimization. Based on the evaluation results, optimize the model by adjusting the network structure and hyperparameters, and further improve the model performance by adding more training data or using transfer learning.

7. The method according to claim 6, characterized in that The sub-region image ship target detection can combine different deep learning models and feature extraction technologies to adapt to different remote sensing images and ship target detection needs, use YOLO, Faster R-CNN, SSD target detection models, and combine attention mechanism and feature fusion to improve detection accuracy.

8. The method according to claim 1, characterized in that The step 6 includes the following sub-steps: Step 6.1: Obtain the detection results. Obtain preliminary detection results from each target detection model. These detection results include the target's bounding box BBox, category label, and confidence score. Step 6.2: Filter the results. Filter the detection results according to the confidence score and retain the results with confidence higher than a certain threshold to reduce false detections. Map the coordinates of the sub-regions according to their positions in the original wide-width SAR ocean image. Step 6.3: Non-maximum suppression (NMS), where the NMS algorithm is used to remove overlapping detection boxes. The NMS compares the overlap and confidence scores of the bounding boxes, retains the bounding box with the highest confidence, and suppresses other bounding boxes with higher overlap; Step 6.4: Result fusion: if multiple models are used for detection, weighted average, voting or other fusion strategies are used to integrate the detection results of different models; image coordinates and longitude and latitude coordinates are converted to obtain the longitude and latitude information of the ship target in the wide-width SAR ocean image, and output it to complete the target detection task; Step 6.5: Post-processing: Post-process the integrated detection results to adjust the position and size of the bounding box to more accurately match the target; Step 6.6, output format, output the final detection results in a standard format, the standard format includes JSON, XML or CSV file, which contains information about the target category, bounding box coordinates, and confidence score; Step 6.7: Visualize the detection results and draw bounding boxes and category labels on the original image to facilitate manual review and verification. Step 6.8: Performance evaluation: Use standard evaluation indicators mAP and IOU to evaluate the performance of the detection results.

9. The method according to claim 8, characterized in that In the result fusion of step 6.4, a method for fusing target detection results is adopted. The fusion method specifically includes training to obtain at least two target detection models, obtaining the recognition weights of each target detection model for different classification types and the overlap thresholds corresponding to each classification type, and then inputting the image to be identified into each target detection model respectively, obtaining the target detection results output by each target detection model, and fusing the target detection results according to the recognition weights and the overlap thresholds to obtain the fused detection results corresponding to the image to be identified.

10. The method according to claim 9, characterized in that The integration and output steps of integrating the detection results of different models in the result fusion of step 6.4 are adjusted according to the model characteristics. For specific ship target detection tasks, the aspect ratio, posture, and occlusion factors of the ship are considered, and the detection and integration strategies are adjusted to ensure that the integration and output steps are consistent with the model training and reasoning environment.

Images