Method, device and product for extracting iceberg information based on SAR wave mode image

Abstract

The invention discloses an iceberg information extraction method, device and product based on an SAR wave mode image, and relates to the field of image recognition. The method comprises the following steps: firstly, acquiring an SAR wave mode SSR image and a corresponding annotation image of a typical iceberg scene, and making a sample data set; constructing an iceberg semantic segmentation model based on an SARIcebergNet network, and training the iceberg semantic segmentation model by adopting the sample data set to obtain a trained iceberg semantic segmentation model; to-be-detected SAR wave mode data are read and preprocessed, and a to-be-detected SSR image is generated; scene classification and screening are carried out on the SSR images to be detected, and SSR images with iceberg scenes are screened out; performing semantic segmentation on the SSR image with the iceberg scene by using the trained iceberg semantic segmentation model to obtain an iceberg recognition result; and the position, area and volume of the iceberg are extracted based on the iceberg identification result, so that the iceberg information extraction efficiency, accuracy and information richness can be improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device and product for extracting iceberg information based on SAR wave pattern images. Background Art

[0002] Icebergs refer to large pieces of freshwater ice that have broken away from polar ice shelves and ice tongues due to melting and stress fractures and are now floating freely in the open ocean. Sea icebergs have a significant impact on changes in ocean water temperature and salinity, rising sea levels, and pose a serious safety threat to ships sailing at sea and offshore drilling platforms. Therefore, the identification, detection, and information extraction of sea icebergs have important scientific and engineering significance. Since icebergs basically appear in polar oceans where it is inconvenient to conduct on-site observations, polar iceberg information extraction mostly relies on satellite remote sensing data. Compared with optical and other satellite remote sensing methods, Synthetic Aperture Radar (SAR), as an active microwave remote sensing imaging radar, has the advantages of all-day, all-weather, and high resolution, especially SAR wave mode images that are often turned on in the open ocean, which have great potential in the application research of iceberg remote sensing monitoring.

[0003] For spaceborne SAR systems, the default shooting is usually performed in the open ocean with small jumping images, which is called wave mode. SAR wave mode images have a huge amount of data. Taking the Sentinel-1 SAR constellation as an example, a single satellite has about 700,000 scenes per year, but only a small part of them are images of polar icebergs. Existing iceberg recognition methods mostly directly identify icebergs on SAR wide-swath images. If the existing methods are directly applied to massive SAR wave mode images for iceberg recognition, it will lead to low efficiency.

[0004] In terms of SAR iceberg identification, existing methods mostly adopt conventional means such as manual visual interpretation, threshold segmentation, constant false alarm rate iteration, or traditional machine learning methods such as random forest. However, due to the complexity of sea conditions and interference from factors such as multi-year fixed ice, in terms of SAR iceberg identification, conventional methods rely on feature selection and threshold setting, and have problems such as sensitivity to noise and poor generalization ability. The random forest method has the defects of complex parameters and slow model training and prediction. In addition, most existing methods can only identify icebergs from SAR remote sensing images, but cannot further extract information such as the area and volume of offshore icebergs, which cannot meet the current needs of iceberg information extraction. Summary of the invention

[0005] The purpose of this application is to provide a method, device and product for extracting iceberg information based on SAR wave pattern images, so as to improve the efficiency, accuracy and information richness of iceberg information extraction.

[0006] To achieve the above objectives, this application provides the following solutions.

[0007] In a first aspect, the present application provides a method for extracting iceberg information based on SAR wave pattern images, comprising: Obtain SAR wave mode SSR images and corresponding annotated images of typical iceberg scenes and create a sample data set; the SAR refers to synthetic aperture radar; the SSR image refers to a standardized sea surface roughness image; An iceberg semantic segmentation model based on the SARIcebergNet network is constructed, and a sample data set is used for training to obtain a trained iceberg semantic segmentation model; the input of the iceberg semantic segmentation model is an SSR image, and the output is a predicted probability image; Read the SAR wave pattern data to be detected and pre-process it to generate the SSR image to be detected; Classify and screen the SSR images to be detected, and screen out the SSR images with iceberg scenes; The trained iceberg semantic segmentation model is used to perform semantic segmentation on the SSR image with iceberg scenes to obtain the iceberg recognition result; Iceberg information is extracted based on the iceberg recognition result; the iceberg information includes the location, area and volume of the iceberg.

[0008] Optionally, the obtaining of a SAR wave mode SSR image of a typical iceberg scene and a corresponding annotated image and preparing a sample data set specifically includes: Read SAR wave mode SSR images of typical iceberg scenes from the TenGeoP-SARwv dataset; The pixel label of the iceberg area in the SAR wave mode SSR image is set to 1, and the pixel label of the background area is set to 0 to form the corresponding annotated image; The SAR wave mode SSR image and the corresponding annotated image are sliced, and the slices containing the iceberg are retained as image samples; Perform image data enhancement processing on the image samples to obtain a sample data set with amplified sample size.

[0009] Optionally, the constructing of an iceberg semantic segmentation model based on the SARIcebergNet network specifically includes: The SARIcebergNet network adopts a U-shaped network structure, including an encoder and a decoder; the encoder adopts a VGG16 convolutional neural network as a skeleton, and includes five encoding modules in total; each encoding module consists of multiple convolution units and a 2×2 maximum pooling layer; the convolution unit consists of a 3×3 convolution layer, a batch normalization layer, and a ReLU activation layer; wherein the first and second encoding modules contain two convolution units, and the third, fourth, and fifth encoding modules contain three convolution units; the number of cores of the convolution layers in the five encoding modules are 64, 128, 256, 512, and 512, respectively; The decoder includes four decoding modules, each of which first performs upsampling through an upward convolution with a convolution step of 2 and a size of 4×4; then, the upsampling result is jump-connected with the feature map of the corresponding encoding module; finally, through a 1×1 convolution layer, the feature vector of each pixel position is mapped to the predicted probability value, and finally a predicted probability image with the same size as the SSR image is output.

[0010] Optionally, the adopting of a sample data set for training to obtain a trained iceberg semantic segmentation model specifically includes: Randomly shuffle the order of image samples in the sample data set and divide them into training set and test set; Download and use the pre-trained weight parameters on the ImageNet dataset as the initial weights of the SARIcebergNet network; The SARIcebergNet network is trained using the training set. The weight parameters of each node in the network are continuously adjusted until the convergence state with the minimum Dice loss function is reached, and a trained iceberg semantic segmentation model is obtained. The test set is used to verify the iceberg recognition accuracy of the trained iceberg semantic segmentation model.

[0011] Optionally, the reading of the SAR wave pattern data to be detected and preprocessing to generate the SSR image to be detected specifically includes: Read the SAR wave pattern data to be detected, including the SAR image of the SAR wave pattern, the original digital value of the SAR image, the longitude and latitude, and the radar incident angle; The SAR image is calibrated based on the original digital value of the SAR image. The calibration formula is used. Calculate the radar backscatter coefficient for each pixel in the SAR image ;in is the original digital value of the SAR image; is the noise equivalent scattering coefficient value; is the calibration factor; According to the radar backscatter coefficient and radar incident angle , using the formula Calculate sea surface roughness value , and then generate the SR image to be detected; is the geophysical model function; is the wind speed at 10m above sea level; is the angle between the sea surface wind direction and the SAR antenna viewing direction; The SR image to be detected is standardized by a standardization method based on percentile statistics, mapped to a 16-bit grayscale range, and the SSR image to be detected is generated.

[0012] Optionally, the scene classification and screening of the SSR images to be detected to screen out SSR images with iceberg scenes specifically includes: The CMwv classifier is used to classify the scene of the SSR image to be detected, and the probability that the SSR image to be detected belongs to the iceberg scene is obtained; The SSR images to be detected with a probability of belonging to the iceberg scene greater than 50% are screened out as SSR images with iceberg scenes.

[0013] Optionally, the method of using the trained iceberg semantic segmentation model to perform semantic segmentation on the SSR image with the iceberg scene to obtain the iceberg recognition result specifically includes: Slice the SSR image with the iceberg scene to obtain multiple slices to be detected; Input a single slice to be detected into the trained iceberg semantic segmentation model to obtain the predicted probability image corresponding to each slice to be detected; According to the position of each slice to be detected in the whole scene SSR image, the predicted probability images of the single slice to be detected are spliced ​​to obtain the iceberg prediction result of the whole scene SSR image; In the iceberg prediction results of the entire SSR image, pixels whose predicted probability values ​​are greater than the set threshold are classified as icebergs, and pixels whose predicted probability values ​​are less than the set threshold are classified as background, and the iceberg binary image of the entire SSR image is obtained as the iceberg recognition result.

[0014] Optionally, extracting iceberg information based on the iceberg identification result specifically includes: Mark the connected regions of the iceberg recognition results and output a series of connected regions marked as iceberg objects; The centroid pixel coordinates of each connected area are extracted, and combined with the longitude and latitude of the entire SSR image, the longitude and latitude corresponding to the centroid pixel coordinates are obtained as the location of the iceberg; The area of ​​the iceberg is calculated based on the number of pixels in each connected region and the spatial resolution of the entire SSR image; Calculate the volume of the iceberg based on its area.

[0015] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the iceberg information extraction method based on SAR wave pattern images.

[0016] In a third aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the iceberg information extraction method based on SAR wave pattern images.

[0017] According to the specific embodiments provided in this application, this application discloses the following technical effects.

[0018] The present application provides a method, device and product for extracting iceberg information based on SAR wave pattern images, which adopts an iceberg semantic segmentation model based on the SARIcebergNet network to perform semantic segmentation of SAR wave pattern SSR images, and has high iceberg recognition accuracy and iceberg object segmentation efficiency; on the other hand, by performing scene classification and screening on the SSR images to be detected, a large number of non-iceberg scene SAR wave pattern images can be eliminated, and there is no need to perform iceberg recognition and information extraction on non-target images, thereby further improving the processing efficiency; on another hand, the present application can not only accurately identify icebergs, but also extract rich information such as the location, area and volume of the icebergs, which can better meet the needs of scientific research and engineering. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of a flow chart of an iceberg information extraction method based on SAR wave mode images in this application; Figure 2 Schematic diagram of the iceberg semantic segmentation model architecture based on the SARIcebergNet network. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] In response to the problems of iceberg monitoring using existing synthetic aperture radar (SAR) satellite wave pattern remote sensing images, this application proposes a method, device and product for extracting iceberg information based on SAR wave pattern images. The deep convolutional neural network of UNet combined with VGG16 is used for semantic segmentation of SAR images, and has the feature of pre-screening iceberg scenes for massive SAR wave pattern images. Therefore, for massive SAR wave pattern images, the method has high efficiency and accuracy in identifying icebergs on the surface of polar oceans. At the same time, the method of this application can extract information such as location, area and volume from SAR wave pattern images, thereby improving the richness of information.

[0023] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0024] In an exemplary embodiment, Figure 1 As shown, a method for extracting iceberg information based on SAR wave pattern images is provided, including the following steps 1 to 6.

[0025] Step 1: Obtain SAR wave mode SSR images of typical iceberg scenes and corresponding annotated images and create a sample dataset.

[0026] Generally speaking, SAR satellites have a variety of data collection methods. Taking the Sentinel-1 satellite as an example, there are four different data collection modes, namely Strip Map Mode (SM), Extra Wide Swath (EW), Interferometric Wide Swath (IW) and Wave Mode (WV). Among them, the wave mode uses jumping small images for default shooting, which can continuously obtain remote sensing images of the global open ocean and has massive data. Therefore, the SAR wave mode image (also called image) in this application refers to the remote sensing image taken by the SAR satellite in wave mode. Most existing methods directly identify icebergs for SAR wide swath images such as ultra-wide swath mode (swath width 500km) and wide swath interferometric mode (swath width 300km), but this application is for SAR wave mode images (swath width 20km) for iceberg identification and information extraction, which is very different from the existing methods. In the specific implementation process, the step 1 specifically includes the following steps 1.1 to 1.4.

[0027] Step 1.1: Read the SAR wave mode SSR image of a typical iceberg scene from the TenGeoP-SARwv dataset.

[0028] First, download the publicly released Sentinel-1 satellite SAR wave pattern image ocean / atmospheric phenomenon expert interpretation dataset TenGeoP-SARwv, the URL is https: / / doi.org / 10.17882 / 56796. The TenGeoP-SARwv dataset contains two types of data, metadata and SAR data. Among them, the metadata gives the file name of each GeoTIFF file in text form, as well as the typical scene category interpreted by experts. There are 10 categories in total, and each type is represented by an uppercase English letter. Among them, the data that has been interpreted by experts and its corresponding GeoTIFF file is identified as a typical iceberg scene is marked as L. SAR data is a satellite remote sensing image saved in GeoTIFF file format. Therefore, 200 scenes marked as L can be directly extracted from the TenGeoP-SARwv metadata as typical iceberg scene SAR data.

[0029] Specifically, the GeoTIFF file corresponding to the SAR data of the typical iceberg scene is directly read from the TenGeoP-SARwv dataset to obtain the Standardized Sea Roughness (SSR) image sample. The SSR image is a standardized sea surface roughness image obtained after the SAR data is processed. In this step, the computer directly reads the GeoTIFF file to obtain the SSR image, because the TenGeoP-SARwv dataset has completed the processing of SAR data to SSR images. The SSR image used in this application is the standardized sea surface roughness image observed by SAR satellite in wave mode remote sensing, which is called SAR wave mode SSR image.

[0030] Step 1.2: Set the pixel label of the iceberg area in the SAR wave mode SSR image to 1, and the pixel label of the background area to 0 to form the corresponding annotated image.

[0031] For the SAR wave mode SSR image of the typical iceberg scene, an annotation method is used to form an "iceberg-background" annotated image. Specifically, Labelme software can be used for annotation, and the label of the pixels in the sea iceberg area is set to 1, and the label of the pixels in the background area is set to 0; the Json file generated after annotation is converted into a PNG format file to form an annotated image.

[0032] Step 1.3: Slice the SAR wave mode SSR image and the corresponding annotated image, and retain the slice containing the iceberg as the image sample.

[0033] The samples are made by slicing, that is, the SAR wave mode SSR image and the annotated image are synchronously sliced ​​into small blocks of fixed size. In the specific implementation process, a sliding window of 256×256 pixels is used to crop and slice the SAR wave mode SSR image and the annotated image. Each slice is judged during processing, and only when the slice contains sea icebergs, the slice is retained as an image sample. Therefore, each slice or image sample itself is also an SSR image of the SAR wave mode. Each slice of the SAR wave mode SSR image and the annotated image is stored one by one.

[0034] Step 1.4: Perform image data enhancement processing on the image samples to obtain a sample data set with amplified sample size.

[0035] In order to effectively increase the diversity of training data and improve the generalization ability of the model, the sample size is expanded by performing random horizontal flipping, vertical flipping and other image data enhancement processing on the slices. The amplified image samples together constitute the sample data set.

[0036] Step 2: Construct an iceberg semantic segmentation model based on the SARIcebergNet network, and use the sample data set for training to obtain a trained iceberg semantic segmentation model; the input of the iceberg semantic segmentation model is the SSR image, and the output is the predicted probability image.

[0037] This application adopts the classic UNet semantic segmentation deep neural network as the basic architecture, and uses the VGG16 convolutional network as its backbone, designs and trains the SARIcebergNet network to perform pixel-level semantic segmentation on SAR wave pattern images of iceberg scenes, so as to extract icebergs from SAR wave pattern images.

[0038] like Figure 2 As shown, the SARIcebergNet network of the present application is a deep learning network, the main body of which adopts a U-shaped network structure, with an encoder and a decoder on the left and right sides respectively. The network input is an SSR image of a SAR wave pattern, and the output is a predicted probability image consisting of the probability that each pixel belongs to an iceberg. In a specific implementation, the design of the SARIcebergNet network model can be implemented through the open source Pytorch framework.

[0039] Among them, the encoder uses the VGG16 convolutional neural network as the skeleton, and by continuously increasing the receptive field, the input 256×256 pixel SSR image is downsampled and compressed 4 times to achieve feature concentration. After 4 times of downsampling and compression, the sizes of the feature maps are 128×128, 64×64, 32×32 and 16×16 respectively. Specifically, the encoder includes five encoding modules; each encoding module consists of several convolution units and a 2×2 maximum pooling layer. Each convolution unit consists of a 3×3 convolution layer, a batch normalization layer (Batch Normalization) and a ReLU activation layer. Among them, the first and second encoding modules contain two convolution units, and the third, fourth and fifth encoding modules contain three convolution units. The number of cores of the convolution layer in the five encoding modules is 64, 128, 256, 512 and 512 respectively.

[0040] The decoder includes four decoding modules, each of which first performs upsampling through an upward convolution with a convolution step of 2 and a size of 4×4. Then, the upsampling result is jump-connected with the feature map output by the corresponding encoding module. The specific operation rule adopted by the jump connection is to perform stacking operations according to the channel dimension. Finally, through a 1×1 convolution layer, the feature vector of each pixel position is mapped to the predicted probability value, and finally the pixel-level semantic segmentation result consistent with the size of the SSR image is output, that is, the predicted probability image of 256×256 pixels is output. The value assigned to each pixel in the predicted probability image is the probability that the position belongs to an iceberg.

[0041] After the SARIcebergNet network is constructed, the sample data set generated in step 1 is used to train the constructed SARIcebergNet network. After the training, a trained iceberg semantic segmentation model is obtained.

[0042] First, the image samples in the sample data set are randomly shuffled and divided into a training set and a test set. In the specific implementation, the first 70% of the randomly shuffled image samples are selected as the training set, and the rest are used as the test set.

[0043] Before model training, you can download and use the pre-trained weight parameters on the ImageNet dataset as the initial weights of the SARIcebergNet network. ImageNet is a large image dataset that contains tens of millions of images and covers thousands of categories. The deep learning model trained on this dataset can learn rich image feature representations. The weight parameters pre-trained on the ImageNet dataset refer to the model parameters obtained after training on the deep learning model. These parameters contain the knowledge and experience learned by the model on the ImageNet dataset.

[0044] Next, the SARIcebergNet network is trained using the training set. Specifically, the SAR wave pattern SSR image samples in the training set are input into the SARIcebergNet network, and the training is performed using the computer GPU. The weight parameters of each node in the network are continuously adjusted until the convergence state with the minimum Dice loss function is reached, and a trained iceberg semantic segmentation model is obtained.

[0045] The expression of Dice loss function is: (1); in, Represents the predicted probability image of the SARIcebergNet network (iceberg semantic segmentation model); Annotated image showing “iceberg-background” as a label. Represents the number of pixels predicted to be icebergs in the predicted probability image; represents the number of pixels marked as icebergs in the corresponding annotated image, express and The number of pixels represented as icebergs in both images. is the Dice loss value.

[0046] In the specific implementation process, NVIDIA RTX A4000 GPU equipment is used for training, and the Adam optimizer is used to update the weight parameters in the model. The initial learning rate is set to 0.001, and a total of 200 rounds of training are performed with early stopping settings. The data size of each batch of training is 64. After the model training is completed, the model structure and the optimal weight parameter set are saved as the trained iceberg semantic segmentation model weight parameters for subsequent use.

[0047] Furthermore, the test set is used to verify the iceberg recognition accuracy of the trained iceberg semantic segmentation model.

[0048] In the specific implementation process, the SAR wave mode SSR image samples in the test set are input into the SARIcebergNet network, and the saved optimal weight parameter set is used to obtain the probability that each pixel of the SSR image belongs to an iceberg. Then, pixels with a probability greater than a threshold (set to 0.5 in this application) are classified as icebergs, and pixels with a probability less than the threshold are classified as background. Finally, the iceberg binary image of the SAR wave mode image is obtained as the iceberg recognition result. The iceberg recognition result is then compared with the corresponding label (annotated image) in the test set, and the evaluation index intersection and union ratio is calculated. , to verify the accuracy of the model. The mathematical expression is: (2); in, is the number of pixels in the image that are correctly identified as icebergs; is the number of pixels of the background that are mistakenly classified as icebergs; is the number of pixels where the iceberg is mistakenly classified as the background; is the number of pixels correctly classified as background (not used in the formula).

[0049] When verifying the accuracy of the model, the annotated images are regarded as "true values" to evaluate the accuracy of the iceberg recognition results given by the model. Generally speaking, the accuracy should be higher than 60%. If the accuracy does not meet the requirements, you should try to change the optimizer during training, change the initial learning rate setting, and try more rounds of training until the model accuracy meets the requirements and a trained iceberg semantic segmentation model is obtained.

[0050] Step 3: Read the SAR wave pattern data to be detected and preprocess it to generate the SSR image to be detected.

[0051] The SSR image of the SAR wave pattern in step 1 can be directly read, but in actual detection, the SAR wave pattern data to be detected needs to be preprocessed after reading to obtain the corresponding SSR image to be detected. The step 3 specifically includes the following steps 3.1 to 3.4.

[0052] Step 3.1: Read the SAR wave pattern data to be detected, including the SAR image of the SAR wave pattern, the original digital value of the SAR image, the longitude and latitude, and the radar incident angle.

[0053] SAR wave pattern data is a type of SAR satellite remote sensing data. In the specific implementation process, the SAR image of the SAR wave pattern, the SAR image original digital value, longitude and latitude, and radar incident angle can be directly read from the SAR wave pattern data of the Sentinel-1 satellite to be detected.

[0054] Step 3.2: Perform radiometric calibration on the SAR image based on the original digital value of the SAR image to obtain the radar backscatter coefficient.

[0055] In the specific implementation process, taking the SAR wave mode of the Sentinel-1 satellite as an example, the radiation calibration can be based on the calibration formula in the European Space Agency user manual. The radar backscatter coefficient of each pixel in the read SAR image is calculated by the following formula (3): : (3); in, is the original digital value of the SAR image; is the noise equivalent scattering coefficient value; is the calibration factor.

[0056] Step 3.3: Based on the radar backscatter coefficient and radar incident angle Generate Sea Roughness (SR) image.

[0057] The influence of SAR observation geometry on radar backscatter coefficient is corrected by normalization of Geophysical Model Function, and the sea surface roughness image is obtained: (4); in, Represents the sea surface roughness value of each pixel in the SR image to be detected. Represents the geophysical model function, and the CMOD5n model function can be selected. The wind speed at 10m above sea level is a fixed value of 10m / s. It is the angle between the sea surface wind direction and the SAR antenna viewing direction, and a fixed value of 45° is selected. is the radar incident angle.

[0058] Step 3.4: Use the standardization method based on percentile statistics to standardize the SR image to be detected, map it to a 16-bit grayscale range, and generate the SSR image to be detected.

[0059] In order to enhance the contrast, the sea surface roughness image is standardized by using a standardization method based on percentile statistics and mapped to a 16-bit grayscale range of [0, 65535] to obtain a standardized sea surface roughness image: (5); in, Indicates the SSR value of each pixel in the SSR image to be detected. The values ​​are arranged in descending order. For the 1% position value, It is 99% of the position value.

[0060] Step 4: Classify and screen the SSR images to be detected, and screen out SSR images with iceberg scenes.

[0061] The CMwv classifier (Wang, C., et al., 2019. Classification of the global Sentinel-1 SAR vignettes for ocean surface process studies. Remote Sensing of Environment 234, 111457.) is used to classify the SSR image to be detected, and the probability of the SSR image to be detected belonging to ten ocean and atmospheric scenes, including pure waves, icebergs, ocean fronts, atmospheric fronts, biofilms, sea ice, low wind speed areas, microconvective cells, rainfall, and wind streaks, is obtained. This application only needs to use one of them, that is, the probability that the SSR image to be detected belongs to the iceberg scene.

[0062] According to the results of scene classification of Sentinel-1 SAR wave pattern images, the SSR images to be detected with a probability of belonging to iceberg scenes greater than 50% were screened out and used as SAR wave pattern SSR images with iceberg scenes to enter the next step of semantic segmentation. The remaining data were no longer processed for iceberg extraction.

[0063] Step 5: Use the trained iceberg semantic segmentation model to perform semantic segmentation on the SSR image with iceberg scenes to obtain the iceberg recognition result.

[0064] Using the trained iceberg semantic segmentation model, semantic segmentation is performed on the Sentinel-1 SAR wave pattern image with iceberg scenes to identify icebergs at sea. The step 5 specifically includes the following steps 5.1 to 5.4.

[0065] Step 5.1: Slice the SSR image with the iceberg scene to obtain multiple slices to be detected.

[0066] Similar to step 1.3, the entire SSR image with the iceberg scene is sliced ​​according to a sliding window of 256×256 pixels.

[0067] Step 5.2: Input a single slice to be detected into the trained iceberg semantic segmentation model to obtain the predicted probability image corresponding to each slice to be detected.

[0068] A single slice to be tested is itself an SSR image. By inputting it into the trained iceberg semantic segmentation model, the corresponding predicted probability image can be output.

[0069] Step 5.3: According to the position of each slice to be detected in the whole scene SSR image, the predicted probability images of the single slice to be detected are spliced ​​to obtain the iceberg prediction result of the whole scene SSR image.

[0070] Step 5.4: Pixels with predicted probability values ​​greater than a set threshold (set to 0.5 in this application) in the iceberg prediction results of the entire SSR image are classified as icebergs, and pixels with predicted probability values ​​less than the set threshold are classified as background. Finally, a binary image of the iceberg of the entire SSR image is obtained as the iceberg recognition result.

[0071] Step 6: Extract iceberg information based on the iceberg identification result; the iceberg information includes the location, area and volume of the iceberg.

[0072] The present application can not only accurately identify icebergs from SAR wave pattern images, but also further extract effective information such as the location, area and volume of the icebergs. The step 6 specifically includes the following steps 6.1 to 6.4.

[0073] Step 6.1: Mark the connected regions of the iceberg recognition results and output a series of connected regions marked as iceberg objects.

[0074] In the specific implementation process, the open source Python image processing library scikit-image is used to label the connected areas using the label function under the measure submodule to label each identified iceberg object. Specifically, the input of the label function is the iceberg binary image, the connectivity parameter is selected as 2 (representing 8 connections), and the output is a series of connected areas marked as iceberg objects.

[0075] Step 6.2: Extract the centroid pixel coordinates of each connected area, and combine the longitude and latitude of the entire SSR image to obtain the longitude and latitude corresponding to the centroid pixel coordinates as the location of the iceberg.

[0076] In the specific implementation process, for each connected area corresponding to an iceberg object, the open source Python image processing library scikit-image is first used to call the regionprops function under the measure submodule to obtain the properties of each connected area. The input of the regionprops function is the connected area corresponding to each iceberg object. The regionprops function will give a return value, which represents multiple properties of this connected area (that is, an iceberg). One of the properties is centroid, which is a coordinate value, which is the pixel coordinate of the geometric center point (center of mass) of this connected area. Therefore, by reading the centroid property returned by the regionprops function, the pixel coordinates of the geometric center point (center of mass of the inner area) of the connected area of ​​the iceberg object can be obtained. Finally, the longitude and latitude of the iceberg position can be obtained through the longitude and latitude of the whole scene SAR wave mode image and the position of the center of mass pixel coordinates in the whole scene SSR image through difference processing.

[0077] Step 6.3: Calculate the area of ​​the iceberg based on the number of pixels in each connected region and the spatial resolution of the entire SSR image.

[0078] In the specific implementation process, for each iceberg object, we first use the open source Python image processing library scikit-image to call the regionprops function under the measure submodule to obtain the properties of each connected region. Then, we read the returned area property to get the number of pixels in the connected region. Finally, combined with the spatial resolution of the SAR wave mode image (i.e. the spatial resolution of the entire SSR image), the area of ​​the iceberg is calculated using formula (6): : (6); in, It is the area of ​​the iceberg that is above sea level. and are the range resolutions in the x and y directions in the entire SSR image, respectively.

[0079] Step 6.4: Calculate the volume of the iceberg based on its area.

[0080] For each iceberg object, according to its exposed sea area , estimate its volume In the specific implementation process, the following empirical formula (7) is used to calculate the volume of the iceberg. Make an estimate: (7); in, is the area of ​​the iceberg; the empirical coefficient is , .

[0081] The final extracted iceberg information includes the location (latitude and longitude) and area of ​​the iceberg. and volume .

[0082] Compared with the existing methods, the advantages of this application are mainly in the following aspects. On the one hand, this application uses UNet combined with VGG16's SARIcebergNet network to perform semantic segmentation of SAR wave pattern images. It is 70.23%, which has high iceberg recognition accuracy and iceberg object segmentation efficiency. On the other hand, the present application adopts the method of pre-screening iceberg scenes in SAR wave pattern images, which can eliminate a large number of non-iceberg scene SAR wave pattern images, and there is no need to perform iceberg recognition and information extraction on non-target images, thereby improving efficiency. For example, taking the SAR wave pattern of the Sentinel-1A satellite as an example, a single satellite has about 700,000 images per year, but only about 15% of them are images of polar iceberg scenes. If no screening is performed, the iceberg monitoring efficiency is low. On the other hand, the present application can not only identify SAR icebergs, but also extract rich information such as iceberg location, area and volume from SAR wave pattern images, which can better meet scientific research and engineering needs.

[0083] In an exemplary embodiment, the present application also provides a computer device, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the iceberg information extraction method based on SAR wave pattern images is implemented.

[0084] In an exemplary embodiment, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the iceberg information extraction method based on SAR wave pattern images.

[0085] It can be understood by a person skilled in the art that all or part of the processes in the above-mentioned embodiment method can be completed by hardware related to computer program instructions, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the process of the embodiment of the above-mentioned method. Among them, any reference to the memory or other medium in each embodiment provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0086] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0087] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for extracting iceberg information based on SAR wave pattern images, characterized in that: include: Obtain SAR wave mode SSR images of typical iceberg scenes and corresponding annotated images and create a sample data set; the SAR refers to synthetic aperture radar; the SSR image refers to a standardized sea surface roughness image; An iceberg semantic segmentation model based on the SARIcebergNet network is constructed, and a sample data set is used for training to obtain a trained iceberg semantic segmentation model; the input of the iceberg semantic segmentation model is an SSR image, and the output is a predicted probability image; Read the SAR wave pattern data to be detected and pre-process it to generate the SSR image to be detected; Classify and screen the SSR images to be detected, and screen out the SSR images with iceberg scenes; The trained iceberg semantic segmentation model is used to perform semantic segmentation on the SSR image with iceberg scenes to obtain the iceberg recognition result; Iceberg information is extracted based on the iceberg recognition result; the iceberg information includes the location, area and volume of the iceberg.

2. The iceberg information extraction method based on SAR wave pattern images according to claim 1 is characterized in that: The method of obtaining a SAR wave mode SSR image of a typical iceberg scene and a corresponding annotated image and preparing a sample data set specifically includes: Read SAR wave mode SSR images of typical iceberg scenes from the TenGeoP-SARwv dataset; The pixel labels of the iceberg area in the SAR wave mode SSR image are set to 1, and the pixel labels of the background area are set to 0 to form the corresponding annotated image; The SAR wave mode SSR image and the corresponding annotated image are sliced, and the slices containing the iceberg are retained as image samples; Perform image data enhancement processing on the image samples to obtain a sample data set with amplified sample size.

3. The iceberg information extraction method based on SAR wave pattern images according to claim 2 is characterized in that: The construction of the iceberg semantic segmentation model based on the SARIcebergNet network specifically includes: The SARIcebergNet network adopts a U-shaped network structure, including an encoder and a decoder; the encoder adopts a VGG16 convolutional neural network as a skeleton, and includes five encoding modules in total; each encoding module consists of multiple convolution units and a 2×2 maximum pooling layer; the convolution unit consists of a 3×3 convolution layer, a batch normalization layer, and a ReLU activation layer; wherein the first and second encoding modules contain two convolution units, and the third, fourth, and fifth encoding modules contain three convolution units; the number of cores of the convolution layers in the five encoding modules are 64, 128, 256, 512, and 512, respectively; The decoder includes four decoding modules, each of which first performs upsampling through an upward convolution with a convolution step of 2 and a size of 4×4; then, the upsampling result is jump-connected with the feature map of the corresponding encoding module; finally, through a 1×1 convolution layer, the feature vector of each pixel position is mapped to the predicted probability value, and finally a predicted probability image with the same size as the SSR image is output.

4. The iceberg information extraction method based on SAR wave pattern images according to claim 3 is characterized in that: The sample data set is used for training to obtain a trained iceberg semantic segmentation model, which specifically includes: Randomly shuffle the order of image samples in the sample data set and divide them into training set and test set; Download and use the pre-trained weight parameters on the ImageNet dataset as the initial weights of the SARIcebergNet network; The SARIcebergNet network is trained using the training set. The weight parameters of each node in the network are continuously adjusted until the convergence state with the minimum Dice loss function is reached, and a trained iceberg semantic segmentation model is obtained. The test set is used to verify the iceberg recognition accuracy of the trained iceberg semantic segmentation model.

5. The iceberg information extraction method based on SAR wave pattern images according to claim 1 is characterized in that: The method of reading the SAR wave pattern data to be detected and preprocessing it to generate the SSR image to be detected specifically includes: Read the SAR wave pattern data to be detected, including the SAR image of the SAR wave pattern, the original digital value of the SAR image, the longitude and latitude, and the radar incident angle; The SAR image is calibrated based on the original digital value of the SAR image. The calibration formula is used. Calculate the radar backscatter coefficient for each pixel in the SAR image ;in is the original digital value of the SAR image; is the noise equivalent scattering coefficient value; is the calibration factor; According to the radar backscatter coefficient and radar incident angle , using the formula Calculate sea surface roughness value , and then generate the SR image to be detected; is the geophysical model function; is the wind speed at 10m above sea level; is the angle between the sea surface wind direction and the SAR antenna viewing direction; The SR image to be detected is standardized by a standardization method based on percentile statistics, mapped to a 16-bit grayscale range, and the SSR image to be detected is generated.

6. The iceberg information extraction method based on SAR wave pattern images according to claim 5 is characterized in that: The scene classification and screening of the SSR images to be detected to screen out the SSR images with iceberg scenes specifically includes: The CMwv classifier is used to classify the scene of the SSR image to be detected, and the probability that the SSR image to be detected belongs to the iceberg scene is obtained; The SSR images to be detected with a probability of belonging to the iceberg scene greater than 50% are screened out as SSR images with iceberg scenes.

7. The iceberg information extraction method based on SAR wave pattern images according to claim 6 is characterized in that: The method of using the trained iceberg semantic segmentation model to perform semantic segmentation on the SSR image with the iceberg scene to obtain the iceberg recognition result specifically includes: Slice the SSR image with the iceberg scene to obtain multiple slices to be detected; Input a single slice to be detected into the trained iceberg semantic segmentation model to obtain the predicted probability image corresponding to each slice to be detected; According to the position of each slice to be detected in the whole scene SSR image, the predicted probability images of the single slice to be detected are spliced ​​to obtain the iceberg prediction result of the whole scene SSR image; In the iceberg prediction results of the entire SSR image, pixels whose predicted probability values ​​are greater than the set threshold are classified as icebergs, and pixels whose predicted probability values ​​are less than the set threshold are classified as background, and the iceberg binary image of the entire SSR image is obtained as the iceberg recognition result.

8. The iceberg information extraction method based on SAR wave pattern images according to claim 7 is characterized in that: The extracting of iceberg information based on the iceberg identification result specifically includes: Mark the connected regions of the iceberg recognition results and output a series of connected regions marked as iceberg objects; The centroid pixel coordinates of each connected area are extracted, and combined with the longitude and latitude of the entire SSR image, the longitude and latitude corresponding to the centroid pixel coordinates are obtained as the location of the iceberg; The area of ​​the iceberg is calculated based on the number of pixels in each connected region and the spatial resolution of the entire SSR image; Calculate the volume of the iceberg based on its area.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for extracting iceberg information based on SAR wave pattern images as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for extracting iceberg information based on SAR wave pattern images as described in any one of claims 1 to 8 is implemented.

Images