A dock area detection method based on geostationary satellite remote sensing images

Abstract

This invention discloses an automatic detection method for dock areas based on geostationary satellite remote sensing images, belonging to the field of dock detection technology. The invention employs a data processing flow based on a land-sea database, combining coarse detection with fine judgment. A lookup table is constructed based on a global DEM database to eliminate purely land and purely marine areas that do not contain docks, effectively reducing subsequent computational load and improving the timeliness of the algorithm. A single-stage YOLOv5 network with precise spacing and timeliness is used to perform a one-time preliminary detection of the image, acquiring suspected target areas. A generative adversarial super-resolution model is employed to perform super-resolution processing on slices of suspected target areas and to accurately interpret them, thereby effectively eliminating false alarms. Ultimately, this achieves accurate and rapid detection of docks within a large field of view using low-resolution geostationary satellite images. This invention achieves fully automatic detection of dock areas from geostationary satellite remote sensing images, with a large detection range, good real-time performance, and high accuracy.

Description

Technical Field

[0001] This invention belongs to the field of dock inspection technology, specifically relating to a dock area inspection method based on geostationary satellite remote sensing images. Background Technology

[0002] Currently, remote sensing image detection technology is widely used in practical scenarios. However, most of these applications are for high-resolution low-Earth orbit (LEO) remote sensing satellite imagery. For target detection in geostationary orbit (GEO) remote sensing imagery, the lower resolution, larger image size, and more complex scene information compared to LEO images make target detection extremely difficult. Large-scale, high-timeliness automated inspection of ports is crucial for shipping information management, and the advent of geostationary orbit remote sensing satellites has made this possible.

[0003] Compared to remote sensing images acquired by traditional reconnaissance satellites, geostationary orbit satellite remote sensing images have lower resolution and larger size range, making the scene information more complex and increasing the processing difficulty. Traditional remote sensing target detection algorithms mostly rely on prior knowledge to manually design features, thus they are difficult to adapt to the more complex scenes in geostationary orbit satellite remote sensing images and have poor robustness (Cheng, G; Han, JW; Guo, L; Qian, XL; Zhou, PC; Yao, XW; Hu, XTObject detection in remote sensing imagery using a discriminatively trained mixture model[J].ISPRS J.Photogramm.RemoteSens.2013,85:32-43). As for the currently mainstream deep learning-based remote sensing target detection algorithms, these algorithms have powerful feature extraction capabilities, which have solved many problems that traditional remote sensing target detection algorithms cannot overcome. However, they are still difficult to directly apply to the detection task of geostationary satellite remote sensing images (S.Ren; K.He; R.Girshick; J.Sun.Faster R-CNN:Towards real-time object detection with regionproposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,39(6):1137-1149). This makes the dock detection task based on geostationary satellite remote sensing images a challenging topic. Summary of the Invention

[0004] The purpose of this invention is to provide an automatic detection method for dock areas based on geostationary satellite remote sensing images, which solves the technical problems of low efficiency due to reliance on manual labor and limited coverage due to low temporal resolution of low-orbit remote sensing satellites in the existing technology for dock detection using remote sensing images.

[0005] The automatic detection method for dock areas based on geostationary satellite remote sensing images provided by this invention includes:

[0006] S1. Collect optical remote sensing image data including the dock area, and divide the optical remote sensing image data into a validation dataset and a training dataset;

[0007] S2. Use annotation tools to annotate the optical remote sensing image data in the training dataset, and annotate the location coordinate information of the dock area to obtain a label set;

[0008] S3. Using the label set, the optical remote sensing image data in the training dataset is augmented to obtain an augmented dataset;

[0009] S4. Input the augmented training dataset from the label set into the YOLOv5 network for training to obtain the optimal model;

[0010] S5. Collect high-resolution optical remote sensing image data slices including the dock area and non-dock areas, and provide a label set;

[0011] S6. The high-resolution optical remote sensing image data is reduced by a factor of 4 to generate a low-resolution-high-resolution image pair;

[0012] S7. Use the above image pairs and label set to train the generative adversarial super-resolution network to obtain the optimal model.

[0013] S8. Acquire geostationary satellite images of the area to be tested, including the dock area;

[0014] S9. Construct a land-sea lookup table based on the global DEM database, and use this lookup table to mask the land and sea areas of the above geostationary satellite images, while preserving the land-sea interface area;

[0015] S10. Use the trained YOLOv5 optimal model to detect the above-mentioned land-sea junction area and obtain the location information of the dock area;

[0016] S11. Based on the above geostationary satellite images and the location information of the dock area, obtain the slice data of the dock location, and use the trained generative adversarial super-resolution network optimal model to perform fine discrimination on the dock slice data.

[0017] S12. Provide the final results of the dock area inspection.

[0018] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0019] 1. A rapid land and sea area masking technology based on a lookup table constructed from a global DEM database can achieve rapid and automatic segmentation and identification of marine, land, and land-sea boundaries, effectively reducing the amount of data required for computation and directly reducing the number of false alarms.

[0020] 2. By utilizing the optimal detection model based on YOLOv5, coarse extraction of suspected dock areas in the scene can be achieved, effectively filtering redundant information in the image to be processed and quickly obtaining candidate dock areas.

[0021] 3. Based on the generative adversarial super-resolution network model, candidate dock areas can be identified, which can realize the fast and efficient removal of false alarms in low-resolution images.

[0022] 4. The method of the present invention realizes fully automatic detection of dock areas from geostationary satellite remote sensing images without relying on manual screening, and can quickly and accurately identify docks from a large area. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the implementation process of the automatic detection method of the present invention;

[0024] Figure 2 This is the original remote sensing image of a geostationary satellite in an embodiment of the present invention;

[0025] Figure 3 This is an image patch representing the land-sea boundary region in an embodiment of the present invention;

[0026] Figure 4 This is a flowchart illustrating the preliminary detection process of the dock area based on the YOLOv5 model, according to an embodiment of the present invention.

[0027] Figure 5 A flowchart for accurate interpretation of the dock area based on a generative adversarial super-resolution model;

[0028] Figure 6 This is a diagram of the dock area obtained after processing using the method of this invention. Detailed Implementation

[0029] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0030] The design concept of this invention is based on a data processing flow of land-sea database and coarse detection + fine judgment to detect dock areas in geostationary satellite remote sensing images with a large field of view and low resolution. Considering the huge amount of data processing for remote sensing images and the fact that the dock areas to be detected only exist in the land-sea junction, a lookup table is first constructed based on the global DEM database to remove pure land and pure ocean areas that do not contain docks, effectively reducing the subsequent computation and improving the timeliness of the algorithm. Considering the high difficulty of detecting dock areas in low-resolution images, in order to ensure processing time and high detection rate of the initial detection, this invention uses the single-stage network YOLOv5, which combines accuracy and timeliness, to perform a one-time preliminary detection of the image and obtain suspected target areas. The above detection based on low-resolution data is difficult to accurately distinguish non-dock areas. This invention adopts a generative adversarial super-resolution model to perform super-resolution processing on the suspected target area slices and accurately interpret them, thereby effectively eliminating false alarms and finally achieving accurate and rapid detection of docks within a large field of view of low-resolution geostationary satellites.

[0031] The overall process of implementing this invention is as follows: Figure 1 As shown, it mainly includes 12 specific implementation steps, which can quickly and accurately extract slices of the dock area of ​​interest from low-resolution image data of geostationary satellites.

[0032] Step S1: Collect optical remote sensing image data including the dock area, and divide the optical remote sensing image data into a validation dataset and a training dataset;

[0033] Specifically, in step S1, the number of optical remote sensing image data is N, the pixel resolution of each optical remote sensing image data is greater than or equal to 1k×1k, and the optical remote sensing image data is divided into a validation dataset and a training dataset in a 2:8 ratio.

[0034] Step S2: Use annotation tools to annotate the optical remote sensing image data in the training dataset, and annotate the location coordinate information of the dock area to obtain a label set;

[0035] Specifically, step S2 mainly includes:

[0036] S21. Use the labelme open-source annotation tool to annotate the optical remote sensing image data containing the dock area in step S1. The annotation is a positive rectangle of variable size. The size of the rectangle should be such that it completely contains a single dock area. Generate the corresponding XML annotation file.

[0037] S22. Clean the labeled data, including deleting erroneous labels and correcting irregular labels;

[0038] Step S3: Use the label set to augment the optical remote sensing image data in the training dataset to obtain an augmented dataset;

[0039] S31. The optical remote sensing image data in the training dataset labeled in S22 are augmented to obtain the augmented dataset. The augmentation methods mainly include:

[0040] S311 performs cropping on the entire image: sampling the entire image to obtain sub-images; performing IOU cropping, i.e., the target in the sub-image occupies the GT annotation box with an IOU ratio of 0.1, 0.3, 0.5, 0.7, or 0.9; and randomly sampling the entire image to obtain sub-images.

[0041] S312 flips the image;

[0042] S313 reduces the size of the image and fills the empty space caused by the reduction with white pixels, so that the image reaches the size of the input image of the model.

[0043] S314Mosaic Data Augmentation: Mosaic data augmentation uses four images to stitch together. Each image has its corresponding bounding box. After stitching the four images together, a new image is obtained, and a new bounding box corresponding to this image is also obtained.

[0044] S32. Perform statistical analysis on the size of the bounding boxes in the augmented dataset;

[0045] Using statistical results, anchor parameters for network training input are generated based on the K-means clustering algorithm.

[0046] Step S4: Input the augmented training dataset from the label set into the YOLOv5 network for training to obtain the optimized model;

[0047] S41. Based on the current application, set the hyperparameters of the YOLOv5 network, including the number of target categories, network learning rate, momentum, weight decay coefficient, number of iterations, input image size, and GPU number used.

[0048] S42. Based on the hyperparameter settings in step S41, perform iterative training of the network to obtain the network model after training is completed.

[0049] S43. Based on the test dataset labeled in step S1, test the network model obtained in step S42 and calculate the accuracy and false alarm rate of the dock area detection.

[0050] S44. If the detection results obtained in step S43 are not satisfactory, adjust the hyperparameters, analyze the detection results, and iterate the training multiple times until the optimal model with satisfactory detection effect is obtained.

[0051] It should be noted that, given the large field of view and extensive information coverage of geostationary satellite remote sensing images, and considering the performance, version stability, and ease of secondary development and deployment of deep learning networks, the single-stage target detection network YOLOv5, which combines accuracy and speed, was selected to process the dock data in geostationary satellite remote sensing images.

[0052] Step S5: Collect high-resolution optical remote sensing image data slices including the dock area and non-dock areas, and provide a label set;

[0053] The dock data collected in step S5 has a resolution four times that of geostationary satellite imaging and is used for subsequent network training. The number of optical remote sensing images of the dock area and non-dock area are M and N, respectively, with a total of 500 images. The dock area slices and non-dock area slices are labeled 1 and 2, respectively.

[0054] Step S6: Reduce the high-resolution optical remote sensing image data by a factor of 4 to generate low-resolution-high-resolution image pairs;

[0055] Specifically, for the high-resolution optical remote sensing image data in step S6, the cubic interpolation algorithm in the Python public library is used to reduce the image by 4 times, generating a corresponding number of low-resolution images.

[0056] Step S7: Train the generative adversarial super-resolution network using the above image pairs and label set to obtain the optimal model;

[0057] Specifically, step S7 is basically the same as step S4. The hyperparameters for network training are set, and the low-resolution-high-resolution image pairs and their corresponding label sets are input into the generative adversarial super-resolution network for training. The network is iteratively optimized, and the model parameters with the highest accuracy are saved as the optimal model.

[0058] Step S8: Acquire geostationary satellite images of the area to be tested, including the dock area;

[0059] Specifically, the size of the optical remote sensing image data in step S8 is M1*N1. Typical geostationary satellite image data is as follows: Figure 2 As shown.

[0060] Step S9: Construct a land-sea lookup table based on the global DEM database, and use the lookup table to mask the land and sea areas of the above geostationary satellite images, while preserving the land-sea interface area;

[0061] S91. Collect the global digital elevation model library;

[0062] S92. Based on the data values ​​in the elevation database, construct a land-sea lookup table, where locations with an elevation greater than 0 (i.e., above sea level) are identified as land, and locations with an elevation less than 0 (i.e., below sea level) are identified as ocean.

[0063] S93. The geostationary satellite imagery from step S8 is divided into multiple 1K data blocks of a fixed size of 1024 pixels * 1024 pixels.

[0064] S94. Based on the lookup table constructed in step S92, determine the land-sea condition of the data block in step S93. It can be determined into three cases: pure land, pure ocean, and land-sea boundary; considering that the dock area to be detected only exists in the land-sea boundary area, only the land-sea boundary area is retained for subsequent processing.

[0065] S95. Use the selected land-sea boundary area images as input for subsequent steps. The selected land-sea boundary data blocks are as follows: Figure 3 As shown, Figure 3 Yes Figure 2 Maps after dividing the area into purely land, purely sea, and the sea-land interface;

[0066] Step S10: Use the trained YOLOv5 optimal model to detect the above-mentioned land-sea junction area and obtain the location information of the dock area;

[0067] Specifically, step S10 includes:

[0068] S101. Set the confidence parameters for YOLOv5 network detection. Considering that the detection results in this step will be finely screened in the future, the parameter selection should ensure the recall rate of the detection terminal as much as possible.

[0069] S102. Load the optimal YOLOV5 model from step S4.

[0070] S103. Input the optical remote sensing image data containing the land-sea boundary area to be processed into the YOLOv5 network.

[0071] S104. Output the network detection results.

[0072] The detection result flow and diagram based on YOLOv5 are as follows: Figure 4 As shown.

[0073] Step S11: Based on the above geostationary satellite images and the location information of the dock area, obtain the slice data of the dock location, and use the trained generative adversarial super-resolution network optimal model to perform fine discrimination on the dock slice data;

[0074] Specifically, step S11 includes:

[0075] S111, Set the external parameters for generating adversarial super-resolution network detection.

[0076] S112. Load the optimal model of the generative adversarial super-resolution network from step S7.

[0077] S113. Based on the aforementioned geostationary satellite imagery and the location information of the dock area, obtain a slice of the dock's location. Input the slice into a generative adversarial super-resolution network for processing.

[0078] S114, Output the accurate slice recognition results.

[0079] The process and results of fine-grained discrimination based on a generative adversarial super-resolution model are as follows: Figure 5 As shown.

[0080] Step S12: Provide the final results of the dock area inspection, such as... Figure 6 As shown.

[0081] This invention employs a data processing workflow based on a land-sea database, combining coarse detection with fine judgment, to detect dock areas in low-resolution geostationary satellite remote sensing images with a large field of view. First, a lookup table is constructed based on a global DEM database to eliminate purely land and purely marine areas that do not contain docks, effectively reducing subsequent computational load and improving the algorithm's timeliness. A single-stage YOLOv5 network with precise spacing and timeliness is used to perform a preliminary one-time detection of the image, identifying potential target areas. A generative adversarial super-resolution model is then employed to perform super-resolution processing on slices of potential target areas and to accurately interpret them, effectively eliminating false alarms. Ultimately, this achieves accurate and rapid detection of docks within a large field of view in low-resolution geostationary satellite images.

Claims

1. An automatic detection method for dock areas based on geostationary satellite remote sensing images, characterized in that, Includes the following steps: S1. Collect optical remote sensing image data of the dock area from geostationary satellites, and divide the collected optical remote sensing image data into a validation dataset and a training dataset. S2. Use annotation tools to annotate the optical remote sensing image data in the training dataset, and annotate the location coordinate information of the dock area to obtain a label set; S3. Using the label set, the optical remote sensing image data in the training dataset is augmented to obtain an augmented dataset; S4. Input the augmented training dataset from the label set into the YOLOv5 network for training to obtain the optimal model; S5. Collect high-resolution optical remote sensing image data slices including the dock area and non-dock areas, and provide a label set; S6. The high-resolution optical remote sensing image data is reduced by a factor of 4 to generate a low-resolution-high-resolution image pair; S7. Use the above image pairs and label set to train the generative adversarial super-resolution network to obtain the optimal model. S8. Acquire geostationary satellite images of the area to be tested, including the dock area; S9. Construct a land-sea lookup table based on the global DEM database, and use this land-sea lookup table to mask the land and sea areas of the above geostationary satellite images, while preserving the land-sea interface area; S10. Use the trained YOLOv5 optimal model to detect the above-mentioned land-sea junction area and obtain the location information of the dock area; S11. Based on the above geostationary satellite images and the location information of the dock area, obtain the slice data of the dock location, and use the trained optimal model of the generative adversarial super-resolution network to perform fine discrimination on the dock slice data. S12. Give the final results of the inspection of the dock area.

2. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, The number of optical remote sensing image data in step S1 is N, the pixel resolution of each optical remote sensing image data is greater than or equal to 1k×1k, and the optical remote sensing image data is divided into a validation dataset and a training dataset in a 2:8 ratio.

3. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, The step S2 comprises: S21. Use the annotation tool to annotate the optical remote sensing image data containing the dock area in step S1. The annotation is a positive rectangle of variable size. The size of the rectangle should be large enough to completely contain a single dock area. Generate the corresponding XML annotation file. S22. Clean the labeled data, including deleting erroneous labels and correcting irregular labels.

4. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, The step S3 comprises: S31. The optical remote sensing image data in the training dataset labeled in S22 are augmented to obtain the augmented dataset. S311 performs cropping on the entire image: sampling the entire image to obtain sub-images; performing IOU ratio cropping, i.e., the target in the sub-image occupies the GT annotation box with an IOU ratio of 0.1, 0.3, 0.5, 0.7, or 0.9; and randomly sampling the entire image to obtain sub-images. S312 flips the image; S313 shrinks the image and fills the empty parts of the image caused by the shrinking with white pixels, so that the image reaches the size of the input image of the model. S314Mosaic Data Augmentation: Mosaic data augmentation uses four images to stitch together. Each image has its corresponding bounding box. After stitching the four images together, a new image is obtained, and a new bounding box corresponding to this image is also obtained. S32. Perform statistical analysis on the size of the bounding boxes in the augmented dataset; Using statistical results, anchor parameters for network training input are generated based on the K-means clustering algorithm.

5. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, Step S4 includes; S41. Based on the current application, set the hyperparameters of the YOLOv5 network, including the number of target categories, network learning rate, momentum, weight decay coefficient, number of iterations, input image size, and GPU number used. S42. Based on the hyperparameter settings in step S41, perform iterative training of the network to obtain the network model after training is completed. S43. Based on the test dataset labeled in step S1, test the network model obtained in step S42 and calculate the accuracy and false alarm rate of the dock area detection. S44. If the detection results obtained in step S43 are not satisfactory, adjust the hyperparameters, analyze the detection results, and iterate the training multiple times until the optimal model with satisfactory detection results is obtained.

6. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, The dock data collected in step S5 has a resolution four times that of geostationary satellite imaging and is used for subsequent network training. The number of optical remote sensing image data of the dock area and the non-dock area are M and N, respectively, with a total of 500 images. The dock area slice and the non-dock area slice are labeled as 1 and 2, respectively. Step S7: Use the above image pairs and label set to train the generative adversarial super-resolution network to obtain the optimal model.

7. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, The size of the optical remote sensing image data in step S8 is M1*N1.

8. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, Step S9 includes: S91. Collect the global digital elevation model library; S92. Based on the data values ​​in the elevation database, construct a land-sea lookup table, where locations with an elevation greater than 0 are defined as land, and locations with an elevation less than 0 are defined as ocean. S93. Divide the geostationary satellite image from step S8 into multiple 1K data blocks of a fixed size of 1024 pixels * 1024 pixels. S94. Based on the lookup table constructed in step S92, determine the land and sea conditions of the data block in step S93; determine three conditions: pure land, pure ocean, and land-sea boundary; only retain the land-sea boundary area for subsequent processing; S95. Use the selected land-sea boundary area images as input for subsequent steps to select land-sea boundary data blocks.

9. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, Step S10 includes: S101. Set the confidence parameters for YOLOv5 network detection. Considering that the detection results in this step will be finely screened in the future, the parameter selection should try to ensure the recall rate of the detection terminal. S102. Load the optimal YOLOv5 model from step S4; S103. Input the optical remote sensing image data containing the land-sea boundary area to be processed into the YOLOv5 network; S104. Output the network detection results.

10. The automatic detection method for dock areas based on geostationary satellite remote sensing images according to claim 1, characterized in that, Step S11 includes: S111, Set the external parameters for generative adversarial super-resolution network detection; S112. Load the optimal model of the generative adversarial super-resolution network from step S7; S113. Based on the above geostationary satellite images and the location information of the dock area, obtain slices of the dock location, and input the slices into an adversarial super-resolution network for processing. S114, Output the accurate slice recognition results.

Images