Ship recognition method based on fusion of multi-band fully polarimetric SAR and multispectral remote sensing images

Abstract

The present invention relates to a ship recognition method by fusing multi-band full polarization SAR and multi-spectral remote sensing images, comprising: obtaining multi-band full polarization SAR and multi-spectral remote sensing images of the same geographical location of the ship; performing recognition preprocessing on the multi-band full polarization SAR image to obtain a preprocessed image; cropping and retaining the overlapping area between the preprocessed image and the multi-spectral remote sensing image to obtain a multi-band full polarization SAR cropped image and a multi-spectral remote sensing cropped image; fusing the multi-band full polarization SAR cropped image with the multi-spectral remote sensing cropped image for spatial features and spectral features to obtain a multi-band fused image; performing feature stacking on the multi-band fused image to obtain a multi-band image; performing coastline feature extraction on the multi-band image to obtain water area features and performing ship feature extraction on the water area features to obtain ship recognition and positioning information. The ship recognition method improves the visibility of the image, reduces the recognition cost, and improves the recognition efficiency while ensuring the recognition accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a ship recognition method by fusing multi-band full-polarization SAR and multi-spectral remote sensing images. Background Art

[0002] Polarimetric Synthetic Aperture Radar (PolSAR) is developed on the basis of single-channel SAR. It can provide multi-dimensional remote sensing information for the target by using electromagnetic waves with different polarization modes to measure the polarization scattering characteristics of the ground objects. The resolution of polarimetric SAR images is generally high. At present, the resolution of my country's Gaofen-3 satellite can reach 1 meter, which is the C-band, multi-polarization satellite with the highest resolution in the world. However, polarimetric SAR images have always had the problem of poor visibility, which will lead to the loss of some information about the object itself, especially for small targets in large areas. Therefore, how to improve the visibility of polarimetric SAR images while maintaining a high resolution is one of the important directions in the current research field. Information in different bands has different advantages for different ground object information. For example, the L band of fully polarimetric SAR can penetrate forests and surface vegetation coverage to obtain richer information, which is beneficial to the military in discovering weapons and equipment hidden in the forest and hidden targets shallowly buried on the surface. The C-band images of fully polarized SAR have shorter wavelengths, so they are more useful for observing sea ice, land erosion, geological structures, etc., especially for observing strong targets on the ocean, such as ships. At present, many different frequency band data have been made public for free. How to combine data from different frequency bands for analysis to further obtain richer information, thereby improving the reliability of research, still needs to be studied.

[0003] At present, there are many algorithms for the fusion of panchromatic images and multispectral images. The commonly used algorithms for pixel-level fusion include PCA (principal component transformation method), Gram-Schmidt, HIS, Brovey (ratio transformation method) and other methods; the commonly used methods for feature-level fusion are ensemble learning; the weighted algorithm is mainly used for decision-level fusion. There are fewer algorithms for the fusion of multispectral images and polarimetric SAR images. In 1979, Daliy et al. tried to combine Landsat and radar images, which was the first step towards the fusion of SAR images and optical images. After that, many researchers began to conduct image fusion processing research at the three levels of spatial dimension, spectral dimension and time dimension. Some scholars have improved the accuracy of urban water area extraction by using SPOT-5 optical data and three different bands, different resolutions and dual-polarization SAR for feature-level fusion.

[0004] The current methods for fusing multispectral images with polarimetric SAR images have the following disadvantages: low single data recognition accuracy and poor visibility, and high multi-SAR data fusion recognition cost, low accuracy and poor visibility. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a ship recognition method by fusing multi-band fully polarized SAR and multi-spectral remote sensing images. The technical problem to be solved by the present invention is achieved by the following technical solutions:

[0006] The embodiment of the present invention provides a ship recognition method by fusing multi-band fully polarized SAR and multi-spectral remote sensing images, comprising the steps of:

[0007] Acquire multi-band fully polarized SAR images and multi-spectral remote sensing images of the same geographical location of the ship;

[0008] Performing recognition preprocessing on the multi-band fully polarized SAR image to obtain a multi-band fully polarized SAR preprocessed image;

[0009] The overlapping area between the multi-band fully polarized SAR preprocessed image and the multispectral remote sensing image is cropped and retained to obtain a multi-band fully polarized SAR cropped image and a multispectral remote sensing cropped image;

[0010] The multi-band fully polarized SAR cropped image is fused with the multi-spectral remote sensing cropped image in terms of spatial features and spectral features to obtain a multi-band fused image;

[0011] Performing feature stacking on the multi-band fused image to obtain a multi-band image;

[0012] The trained water area model is used to extract coastline features from the multi-band image to obtain water area features, and the trained ship model is used to extract ship features from the water area features to obtain ship identification and positioning information.

[0013] In one embodiment of the present invention, the multi-band fully polarized SAR image includes an L-band fully polarized SAR image and a C-band fully polarized SAR image, wherein the L-band fully polarized SAR image and the C-band fully polarized SAR image both include four polarization modes: HH, HV, VH, and VV.

[0014] In one embodiment of the present invention, the multi-band fully polarimetric SAR image is subjected to recognition preprocessing to obtain a multi-band fully polarimetric SAR preprocessed image, including:

[0015] The multi-band fully polarimetric SAR image is subjected to multi-view processing, image registration processing, filtering processing, geocoding and calibration processing in sequence to obtain the multi-band fully polarimetric SAR preprocessed image.

[0016] In one embodiment of the present invention, cropping and retaining the overlapping area of ​​the multi-band fully polarimetric SAR preprocessed image and the multispectral remote sensing image to obtain the multi-band fully polarimetric SAR cropped image and the multispectral remote sensing cropped image includes:

[0017] The overlapping area of ​​each image is cropped and retained according to the geographic location consistency between the multi-band fully polarized SAR preprocessed image and the multispectral remote sensing image, so as to obtain the multi-band fully polarized SAR cropped image and the multispectral remote sensing cropped image.

[0018] In one embodiment of the present invention, the multi-band fully polarized SAR cropped image is fused with the multi-spectral remote sensing cropped image in terms of spatial features and spectral features to obtain a multi-band fused image, including:

[0019] The Gram Schmidt fusion method is used to fuse each polarization SAR cropped image of each frequency band in the multi-band full polarization SAR cropped image with the remote sensing cropped image of each band in the multispectral remote sensing cropped image to obtain the multi-band fused image.

[0020] In one embodiment of the present invention, the coastline feature extraction of the multi-band image is performed using a trained water area model to obtain water area features, and the ship feature extraction of the water area features is performed using a trained ship model to obtain ship identification and positioning information, including the steps of:

[0021] Using the water feature training data in the multi-band image to train a first deep learning neural network to obtain the trained water model;

[0022] Using the ship feature training data in the multi-band image to train a second deep learning neural network to obtain the trained ship model;

[0023] Extracting coastline features from the multi-band image using the trained water area model to obtain the water area features;

[0024] The trained ship model is used to extract ship features from the water area features to obtain the ship identification and positioning information.

[0025] In one embodiment of the present invention, before the first deep learning neural network is trained using the water feature training data in the multi-band image to obtain the trained water model, the step further includes:

[0026] A deep learning framework is used to perform feature learning on the multi-band image to separate the water area from the land area, and obtain training data for extracting the water area, wherein the training data for extracting the water area includes the water feature training data and the ship feature training data.

[0027] Another embodiment of the present invention provides a ship identification device integrating multi-band fully polarized SAR and multi-spectral remote sensing images, comprising:

[0028] Image acquisition module, used to acquire multi-band full-polarization SAR images and multi-spectral remote sensing images of the same geographical location of the ship;

[0029] An image preprocessing module is used to perform recognition preprocessing on the multi-band fully polarimetric SAR image to obtain a multi-band fully polarimetric SAR preprocessed image;

[0030] An image cropping module is used to crop and retain the overlapping area of ​​the multi-band full-polarization SAR preprocessed image and the multispectral remote sensing image to obtain a multi-band full-polarization SAR cropped image and a multispectral remote sensing cropped image;

[0031] An image fusion module is used to fuse the multi-band fully polarized SAR cropped image with the multi-spectral remote sensing cropped image in terms of spatial features and spectral features to obtain a multi-band fused image;

[0032] A water area feature extraction module, used to extract coastline features from the multi-band image using a trained water area model;

[0033] The ship identification and positioning module is used to extract ship features from the water area features using the trained ship model to obtain ship identification and positioning information.

[0034] Another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0035] Memory, used to store computer programs;

[0036] The processor is used to implement the method steps described in the above embodiment when executing the program stored in the memory.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The ship identification method of the present invention uses a multi-band fully polarized SAR image in combination with a multispectral remote sensing image, which not only retains the true color information of the multispectral image, but also retains the characteristic information of the fully polarized image, enriching the characteristic information of the image itself to a great extent. While ensuring the recognition accuracy, it improves the visibility of the image, reduces the recognition cost, and improves the recognition efficiency.

[0039] 2. The ship identification method of the present invention performs water area ship identification and positioning based on multi-band large-scale remote sensing data, fully utilizes the full polarization feature information and the spatial features of the image, and is conducive to improving the ship identification accuracy of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a ship recognition method by fusing multi-band fully polarized SAR and multi-spectral remote sensing images provided in an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a Sentinel-2B remote sensing image in the R, G, B, and NIR bands provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a preprocessed multi-band fully polarimetric SAR image provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a fused image after fusing a multi-band fully polarized SAR image and a multispectral remote sensing image provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a multi-band image obtained by feature stacking of a multi-band fused image provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0046] Embodiment 1

[0047] See also Figure 1 , Figure 1 A flow chart of a ship recognition method by fusing multi-band full polarization SAR and multi-spectral remote sensing images is provided in an embodiment of the present invention. The ship recognition method by fusing multi-band full polarization SAR and multi-spectral remote sensing images can be applied to accurate positioning of ships by multi-band full polarization SAR and multi-spectral remote sensing images, and comprises the following steps:

[0048] S1. Acquire multi-band fully polarized SAR images and multi-spectral remote sensing images of the same geographical location of the ship.

[0049] Specifically, the frequency bands of the multi-band fully polarized SAR image include at least 2 frequency bands, which may be 2 frequency bands, 3 frequency bands or more frequency bands, and this embodiment does not impose further restrictions; further, the SAR image of each frequency band is a fully polarized image. The multispectral remote sensing image includes at least 2 bands, which may be 2 bands, 3 bands or more bands, and this embodiment does not impose further restrictions.

[0050] In a specific embodiment, the multi-band fully polarimetric SAR image includes an original L-band ALOS PALSAR fully polarimetric SAR image L and a C-band GF-3 fully polarimetric SAR image G of the same geographical location obtained from remote sensing satellite images. Both the ALOS PALSAR fully polarimetric SAR image L and the GF-3 fully polarimetric SAR image G have four polarization modes: HH, HV, VH, and VV. The resolution of the ALOS PALSAR fully polarimetric SAR image L and the GF-3 fully polarimetric SAR image G is 12.5 m.

[0051] In a specific embodiment, the multispectral remote sensing image is a C-band Sentinel-2B remote sensing image S of the same geographical location obtained from remote sensing satellite images. The Sentinel-2B remote sensing image S has 13 bands. This embodiment uses the R band, G band, B band, and NIR band. Please refer to Figure 2 , Figure 2 A schematic diagram of a Sentinel-2B remote sensing image of R, G, B, and NIR bands provided by an embodiment of the present invention, wherein: Figure 2 (a) is the R-band Sentinel-2B remote sensing image. Figure 2 (b) is the Sentinel-2B remote sensing image of the G band. Figure 2 (c) is the B-band Sentinel-2B remote sensing image. Figure 2 (d) is a Sentinel-2B remote sensing image in the NIR band. The resolution of the Sentinel-2B remote sensing image S is 10m, 20m, and 60m. The resolution of the Sentinel-2B remote sensing image S used in this embodiment is 10m.

[0052] S2. Performing recognition preprocessing on the multi-band fully polarimetric SAR image to obtain a multi-band fully polarimetric SAR preprocessed image.

[0053] Specifically, the multi-band fully polarimetric SAR image is subjected to multi-view processing, image registration processing, filtering processing, geocoding and calibration processing in sequence to obtain the multi-band fully polarimetric SAR preprocessed image.

[0054] Specifically, since the fully polarized SAR images obtained in step S1 are all SLC SAR image products, which contain a lot of speckle noise, this embodiment first performs multi-view processing on the multi-band fully polarized SAR images. The specific operation of multi-view processing is to average the image resolution in the range and azimuth directions of the image. After multi-view processing, the speckle noise of the SAR image is suppressed, and at the same time, the radiation resolution of the SAR image is improved and the spatial resolution is reduced. The image registration processing adopts the processing method in the prior art. The filtering processing mainly uses a linear smoothing filtering method, such as a Gaussian filtering method, which can balance the degree of noise suppression and the degree of blurring of the image by adjusting the parameter σ, and the smoothness of the filtered image is better. The geocoding and calibration processing obtains each polarized SAR image with the same coordinate system by adopting elevation geocoding and calibration, so that the data at the same position of different images have a high degree of consistency, avoiding the problem of ghosting after fusion.

[0055] In this embodiment, after the multi-band fully polarized SAR images are subjected to multi-view processing, image registration processing, filtering processing, geocoding and calibration processing in sequence, the problems of poor visual effect, lack of recognition and low usability of the original fully polarized SAR images that have not been preprocessed are improved to a certain extent. Figure 3 , Figure 3 A schematic diagram of a preprocessed multi-band fully polarimetric SAR image provided by an embodiment of the present invention, wherein (a) to (d) correspond to the HH, HV, VH, and VV polarimetric images of ALOS PALSAR, and (e) to (h) correspond to the HH, HV, VH, and VV polarimetric images of GF-3. Figure 3 It can be seen that the visual effect of the preprocessed full-polarization SAR image is improved and has a certain degree of recognition.

[0056] S3. Cropping and retaining the overlapping area between the multi-band full-polarization SAR preprocessed image and the multispectral remote sensing image to obtain a multi-band full-polarization SAR cropped image and a multispectral remote sensing cropped image.

[0057] Specifically, the overlapping area of ​​each image is cropped and retained according to the geographic location consistency between the multi-band fully polarized SAR preprocessed image and the multispectral remote sensing image, so as to obtain the multi-band fully polarized SAR cropped image and the multispectral remote sensing cropped image.

[0058] In a specific embodiment, a cropped image is obtained as data to be fused by cropping overlapping regions of heterogeneous data. The specific steps are as follows: among four ALOS PALSAR fully polarimetric SAR images, four GF-3 fully polarimetric SAR images and four Sentinel-2B remote sensing images (i.e., images of R, G, B and NIR bands), one ALOS PALSAR fully polarimetric SAR image, one GF-3 fully polarimetric SAR image and one Sentinel-2B remote sensing image are selected for simultaneous display, and images of non-overlapping regions are cropped according to the degree of geographical location consistency (i.e., data coincidence), and images of overlapping regions are retained, thereby obtaining multi-band fully polarimetric SAR cropped images and multispectral remote sensing cropped images.

[0059] In another specific embodiment, a cropped image is obtained by stitching multiple homologous and synchronous phase data and then cropping the overlapping area of ​​heterologous data. The specific steps are: stitching multiple ALOS PALSAR full-polarization SAR images into a full-polarization SAR stitched image, then displaying the stitched image simultaneously with one GF-3 full-polarization SAR image and one Sentinel-2B remote sensing image, cropping the image of the non-overlapping area according to the degree of geographical location matching, and retaining the image of the overlapping area, thereby obtaining a multi-band full-polarization SAR cropped image and a multi-spectral remote sensing cropped image. Alternatively, stitching multiple GF-3 full-polarization SAR images into a SAR stitched image, then displaying the stitched image simultaneously with one ALOS PALSAR full-polarization SAR image and one Sentinel-2B remote sensing image, cropping the image of the non-overlapping area according to the degree of geographical location matching, and retaining the image of the overlapping area, thereby obtaining a multi-band full-polarization SAR cropped image and a multi-spectral remote sensing cropped image.

[0060] In this embodiment, different satellite sensors have different remote sensing imaging modes, so the coverage areas of the acquired multi-band heterogeneous raw data cannot be completely consistent. Therefore, the multi-band data of the target area are obtained by trimming the overlapping areas.

[0061] S4, fusing the multi-band fully polarized SAR cropped image with the multi-spectral remote sensing cropped image in terms of spatial features and spectral features to obtain a multi-band fused image.

[0062] Specifically, each polarization SAR cropped image of each frequency band in the multi-band full-polarization SAR cropped image is fused with the remote sensing cropped image of each band in the multispectral remote sensing cropped image by using the Gram-Schmidt fusion method to obtain the multi-band fused image.

[0063] In this embodiment, other existing image fusion technologies may also be used to fuse the multi-band full-polarization SAR cropped image with the multispectral remote sensing cropped image, and this embodiment does not impose any further limitation.

[0064] In a specific embodiment, the HH, HV, VH, and VV polarized images of the L band are respectively fused with the R, G, B, and NIR band images in the Sentinel-2B remote sensing image by Gram-Schmidt, and 16 images are obtained after the L-band full polarized SAR image and the Sentinel-2B remote sensing image are fused; the HH, HV, VH, and VV polarized images of the C band are respectively fused with the R, G, B, and NIR band images in the Sentinel-2B remote sensing image by Gram-Schmidt, and 16 images are obtained after the C-band full polarized SAR image and the Sentinel-2B remote sensing image are fused. For the obtained multi-band fused image, please refer to Figure 4 , Figure 4 A schematic diagram of a fused image after fusing a multi-band fully polarimetric SAR image and a multispectral remote sensing image provided in an embodiment of the present invention, wherein (a) to (d) are diagrams showing the fusion results of ALOS PALSAR images and Sentinel-2B images, and (e) to (h) are diagrams showing the fusion results of GF-3 and Sentinel-2B images. Figure 4 In the figure, the R, G, B, and NIR bands are displayed in the same image.

[0065] In this embodiment, the fully polarized SAR images of different frequency bands contain polarization information of ground object scattering and are rich in spatial texture detail information. The multispectral remote sensing images are rich in spectral information. By fusing SAR images of different polarization modes with multispectral images, the advantages of the two can be superimposed to achieve the effect of "1+1>2".

[0066] S5. Feature stacking is performed on the multi-band fused image to obtain a multi-band image.

[0067] Specifically, the fused image has multiple bands, and the features of the multiple bands are stacked to obtain a multi-band image F, which has the data features of all the above multi-bands.

[0068] In a specific embodiment, 16 images obtained by fusing L-band full polarization SAR images with Sentinel-2B remote sensing images and 16 images obtained by fusing C-band full polarization SAR images with Sentinel-2B remote sensing images are stacked for band features, and the multi-band image F has 32 bands after stacking. Figure 5 , Figure 5A schematic diagram of a multi-band image obtained by feature stacking of a multi-band fused image provided in an embodiment of the present invention. This multi-band image has 32 bands, namely: APHH_R, APHH_G, APHH_B, APHH_NIR, APHV_R, APHV_G, APHV_B, APHV_NIR, APVH_R, APVH_G, APVH_B, APVH_NIR, APVV_R, APVV_G, APVV_B, APVV_NIR, GaofenHH_R, GaofenHH_G, GaofenHH_B, GaofenHH_NIR, GaofenHV_R, GaofenHV_G, GaofenHV_B, GaofenHV_NIR, GaofenVH_R, GaofenVH_G, GaofenVH_B, GaofenVH_NIR, GaofenVV_R, GaofenVV_G, GaofenVV_B, GaofenVV_NIR. Among them, AP refers to ALOS PALSAR image, and Gaofen refers to Gaofen-3 SAR image. This image F has the advantages of L-band, C-band fully polarized SAR images and Sentinel-2B multispectral images, that is, image F has the advantages of L-band fully polarized SAR images for fresh water and underground target observation, C-band fully polarized SAR images for strong target observation on the ocean, and rich spectral information of multispectral images.

[0069] In this embodiment, since the fused image has all the above multi-band data features, it has the advantages of images of all frequency bands.

[0070] S6, using the trained water area model to extract coastline features from the multi-band image to obtain water area features, and using the trained ship model to extract ship features from the water area features to obtain ship identification and positioning information. Specifically including the steps:

[0071] S61. Perform feature learning on the multi-band image using a deep learning framework to separate the water area from the land area, and obtain training data for extracting the water area, wherein the training data for extracting the water area includes the water feature training data and the ship feature training data.

[0072] Specifically, in order to improve the recognition and positioning accuracy, the water area can be separated from the land area (ie, the complex background is filtered out) before the ship recognition and positioning are performed.

[0073] In a specific embodiment, deep learning with powerful feature extraction capability is used to perform feature learning on multi-band images, thereby separating water areas from land areas and obtaining training data for extracting water areas. The training data for extracting water areas includes the water feature training data and the ship feature training data.

[0074] S62. Using the water feature training data in the multi-band image, the first deep learning neural network is trained to obtain the trained water model.

[0075] In a specific embodiment, the water feature training data in the multi-band image is used to train the first deep learning neural network to learn the water features, and the model training batch size is set to 300 and the training rounds are set to 100, thereby obtaining a trained water model M_sea.

[0076] In this embodiment, the water feature training data can be the water feature training data in the multi-band image obtained by fusing the multi-band fully polarized SAR and the multi-spectral remote sensing image using steps S1-S5, or the water feature training data in the multi-band image obtained by fusing the existing identified image using steps S1-S5.

[0077] S63. Train a second deep learning neural network using the ship feature training data in the multi-band image to obtain the trained ship model.

[0078] In a specific embodiment, the ship feature training data in the multi-band image is used to train the second deep learning neural network to learn the ship features, and the model training batch size is set to 300 and the training rounds are set to 100, thereby obtaining a trained ship model M_ship.

[0079] In this embodiment, the ship feature training data can be the ship feature training data in the multi-band image obtained by fusing the multi-band fully polarized SAR and the multi-spectral remote sensing image using steps S1-S5, or the ship feature training data in the multi-band image obtained by fusing the existing identified image using steps S1-S5.

[0080] In this embodiment, the first deep learning neural network and the second deep learning neural network both adopt existing deep learning neural networks, which may be the same or different. Preferably, the first deep learning neural network and the second deep learning neural network are the same.

[0081] In another embodiment, a target detection method may be used to replace the first deep learning neural network and the second deep learning neural network, such as a YOLO series version of the network, Faster RCNN, SSD algorithm, etc.

[0082] S64, using the trained water area model to extract coastline features from the multi-band image to obtain water area features.

[0083] In a specific embodiment, the multi-band image obtained in step S5 is classified using the trained water model M_sea to extract the coastline, thereby obtaining the classification results: water area features and land area features. Further, the multi-band image is cropped according to the classification results to retain the water area features; when cropping, in order to retain the ships on the shore, the connection between the water area and the land area is appropriately retained, that is, when cropping, a part of the land area is retained at the coastline.

[0084] S65. Using the trained ship model, extract ship features from the water area features to obtain ship identification and positioning information.

[0085] In a specific embodiment, the trained ship model M_ship is used to extract the water area features obtained in S64 to obtain ship identification and positioning information.

[0086] The ship recognition method of this embodiment uses a combination of multi-band fully polarized SAR images and multi-spectral remote sensing images, which not only retains the true color information of the image, but also retains the characteristic information of the fully polarized image, enriching the characteristic information of the image itself to a great extent. While ensuring the recognition accuracy, it improves the visibility of the image, reduces the recognition cost, and improves the recognition efficiency.

[0087] The ship identification method of this embodiment performs water area ship identification and positioning based on multi-band large-scale remote sensing data, fully utilizes full polarization feature information and image spatial features, and is conducive to improving the ship identification accuracy of remote sensing images.

[0088] Embodiment 2

[0089] On the basis of the first embodiment, this embodiment further provides a ship recognition device integrating multi-band full-polarization SAR and multi-spectral remote sensing images.

[0090] The ship recognition device integrating multi-band fully polarized SAR and multi-spectral remote sensing images comprises an image acquisition module, an image preprocessing module, an image cropping module, an image fusion module, a water area feature extraction module and a ship recognition and positioning module.

[0091] Among them, the image acquisition module is used to acquire the multi-band full polarization SAR image and the multi-spectral remote sensing image of the same geographical location of the ship. The image preprocessing module is used to perform recognition preprocessing on the multi-band full polarization SAR image to obtain the multi-band full polarization SAR preprocessed image. The image cropping module is used to crop and retain the overlapping area of ​​the multi-band full polarization SAR preprocessed image and the multi-spectral remote sensing image to obtain the multi-band full polarization SAR cropped image and the multi-spectral remote sensing cropped image. The image fusion module is used to fuse the spatial features and spectral features of the multi-band full polarization SAR cropped image and the multi-spectral remote sensing cropped image to obtain a multi-band fused image. The water area feature extraction module is used to extract the coastline features of the multi-band image using the trained water area model. The ship identification and positioning module is used to extract the ship features of the water area features using the trained ship model to obtain ship identification and positioning information.

[0092] The ship identification device for fusing multi-band fully polarized SAR and multi-spectral remote sensing images provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0093] Embodiment 3

[0094] Based on the first embodiment, this embodiment further provides an electronic device.

[0095] The electronic device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the method steps described in the first embodiment when executing the program stored in the memory.

[0096] The electronic device provided by the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0097] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A ship recognition method based on the fusion of multi-band fully polarimetric SAR and multi-spectral remote sensing images. It is characterized in that Includes steps: Acquire multi-band fully polarized SAR images and multi-spectral remote sensing images of the same geographical location of the ship; Performing recognition preprocessing on the multi-band fully polarized SAR image to obtain a multi-band fully polarized SAR preprocessed image; The overlapping area between the multi-band fully polarized SAR preprocessed image and the multispectral remote sensing image is cropped and retained to obtain a multi-band fully polarized SAR cropped image and a multispectral remote sensing cropped image; The multi-band fully polarized SAR cropped image is fused with the multi-spectral remote sensing cropped image in terms of spatial features and spectral features to obtain a multi-band fused image; Performing feature stacking on the multi-band fused image to obtain a multi-band image; The trained water area model is used to extract coastline features from the multi-band image to obtain water area features, and the trained ship model is used to extract ship features from the water area features to obtain ship identification and positioning information.

2. The ship recognition method based on the fusion of multi-band fully polarized SAR and multi-spectral remote sensing images according to claim 1, It is characterized in that The multi-band fully polarized SAR image includes an L-band fully polarized SAR image and a C-band fully polarized SAR image, wherein the L-band fully polarized SAR image and the C-band fully polarized SAR image both include four polarization modes: HH, HV, VH, and VV.

3. The ship recognition method based on the fusion of multi-band fully polarized SAR and multi-spectral remote sensing images according to claim 1, It is characterized in that Performing recognition preprocessing on the multi-band fully polarimetric SAR image to obtain a multi-band fully polarimetric SAR preprocessed image includes: The multi-band fully polarimetric SAR image is subjected to multi-view processing, image registration processing, filtering processing, geocoding and calibration processing in sequence to obtain the multi-band fully polarimetric SAR preprocessed image.

4. The ship recognition method of claim 1 by fusing multi-band fully polarized SAR with multi-spectral remote sensing images, It is characterized in that The method of cropping and retaining the overlapping area of ​​the multi-band fully polarized SAR preprocessed image and the multispectral remote sensing image to obtain a multi-band fully polarized SAR cropped image and a multispectral remote sensing cropped image includes: The overlapping area of ​​each image is cropped and retained according to the geographic location consistency between the multi-band fully polarized SAR preprocessed image and the multispectral remote sensing image, so as to obtain the multi-band fully polarized SAR cropped image and the multispectral remote sensing cropped image.

5. The ship recognition method of claim 1 by fusing multi-band fully polarized SAR with multi-spectral remote sensing images, It is characterized in that The multi-band fully polarized SAR cropped image is fused with the multi-spectral remote sensing cropped image by spatial and spectral features to obtain a multi-band fused image, including: The Gram Schmidt fusion method is used to fuse each polarization SAR cropped image of each frequency band in the multi-band full polarization SAR cropped image with the remote sensing cropped image of each band in the multispectral remote sensing cropped image to obtain the multi-band fused image.

6. The ship recognition method of claim 1 by fusing multi-band fully polarized SAR with multi-spectral remote sensing images, It is characterized in that The method comprises the following steps: extracting coastline features from the multi-band image using a trained water area model to obtain water area features, and extracting ship features from the water area features using a trained ship model to obtain ship identification and positioning information, including the following steps: Using the water feature training data in the multi-band image to train a first deep learning neural network to obtain the trained water model; Using the ship feature training data in the multi-band image to train a second deep learning neural network to obtain the trained ship model; Extracting coastline features from the multi-band image using the trained water area model to obtain the water area features; The trained ship model is used to extract ship features from the water area features to obtain the ship identification and positioning information.

7. The ship recognition method of claim 6 by fusing multi-band fully polarized SAR with multi-spectral remote sensing images, It is characterized in that Before using the water feature training data in the multi-band image to train the first deep learning neural network to obtain the trained water model, the method further includes the following steps: A deep learning framework is used to perform feature learning on the multi-band image to separate the water area from the land area, and obtain training data for extracting the water area, wherein the training data for extracting the water area includes the water feature training data and the ship feature training data.

8. A ship identification device that integrates multi-band fully polarized SAR and multi-spectral remote sensing images. It is characterized in that include: Image acquisition module, used to acquire multi-band full-polarization SAR images and multi-spectral remote sensing images of the same geographical location of the ship; An image preprocessing module is used to perform recognition preprocessing on the multi-band fully polarimetric SAR image to obtain a multi-band fully polarimetric SAR preprocessed image; An image cropping module is used to crop and retain the overlapping area of ​​the multi-band full-polarization SAR preprocessed image and the multispectral remote sensing image to obtain a multi-band full-polarization SAR cropped image and a multispectral remote sensing cropped image; An image fusion module is used to fuse the multi-band fully polarized SAR cropped image with the multi-spectral remote sensing cropped image in terms of spatial features and spectral features to obtain a multi-band fused image; A water area feature extraction module, used to extract coastline features from the multi-band fusion image using a trained water area model; The ship identification and positioning module is used to extract ship features from the water area features using the trained ship model to obtain ship identification and positioning information.

9. An electronic device, It is characterized in that It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

Images