A method, device and medium for repairing holes in SAR images

Abstract

The present application relates to the field of SAR image restoration, and specifically discloses a method, device and medium for repairing holes in SAR images, wherein the method comprises: directly orthorectifying the target data after preprocessing to obtain an initial SAR orthophoto; performing image segmentation and filtering on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto; performing image segmentation and image restoration algorithms on the intermediate orthophoto to segment and repair the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole restoration. First, small holes are repaired to ensure the original information of the SAR image data and to improve the computing speed. Then, large holes are repaired, the image background value is automatically excluded, and the large hole mask is extracted for repair, thereby avoiding manual participation in drawing the mask.

Description

Technical Field

[0001] The present application relates to the field of SAR image restoration, and in particular to a method, device and medium for repairing holes in SAR images. Background Art

[0002] Synthetic Aperture Radar (SAR) satellite is an active microwave remote sensing satellite that can observe all day and all weather conditions by emitting electromagnetic waves to the ground and receiving the target's reflected signal imaging. At the same time, SAR has a certain surface penetration ability and is widely used in ocean, water conservancy, meteorology, surveying and mapping, and military. Due to the side-view imaging characteristics of SAR images, it will produce geometric deformations such as perspective contraction, overlap, and shadows that are different from optical images in uneven areas with large terrain undulations. These geometric deformations have an adverse effect on SAR image target identification and thematic information extraction, greatly reducing its usability. The application of SAR remote sensing images is mostly based on orthophotos. Therefore, in practical applications, orthophoto correction is often used to eliminate these adverse effects, thereby obtaining orthophotos that are not deformed compared to the actual geographical location.

[0003] SAR images can be orthorectified by direct correction and indirect correction. In the direct orthorectification process, due to the overlap and shadow of SAR side-view imaging, the geometric position is compressed, so that some coordinate points in the SAR image coordinate space correspond to the same coordinate point in the orthophoto, resulting in some pixels in the orthophoto not being assigned values, forming "hole" pixels. Holes are manifested as large-scale sporadic noise holes (small holes) and large-area holes in areas with large terrain undulations (large holes).

[0004] At present, the hole repair method of SAR images generally requires manual mask drawing, extracting the hole area, and then filling the hole pixels. Among them, the hole pixel filling mainly adopts filtering processing, interpolation filling, FMM and other technologies.

[0005] For example, the invention patent with application number CN201710445518.X, “Direct orthorectification method and system for SAR images based on DEM”, focuses on the direct orthorectification method for SAR images based on DEM, in which the last step is to repair the holes, which essentially uses the K-nearest neighbor interpolation algorithm.

[0006] The invention has the following shortcomings and deficiencies: the premise of repairing holes is to extract holes. Since holes and the background value of the orthorectified image have the same pixel value, the current technology requires manual drawing of the study area to exclude the background value and then extract the hole area. This obviously cannot meet the requirements of data automation and large-scale processing. Moreover, the single image repair processing method does not have targeted processing for holes of different sizes, resulting in repeated processing and reduced processing efficiency. For complex image repair, the interpolation algorithm is used to repair large holes, such as SAR image mountain repair, which often has limited effect. Summary of the invention

[0007] In order to solve the above problems, the present application proposes a SAR image hole repair method, device and medium, wherein the method comprises:

[0008] Acquire target data and preprocess the target data; the target data includes SAR remote sensing image data and DEM data; perform direct orthorectification processing on the preprocessed target data to obtain an initial SAR orthophoto; perform image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto; perform image segmentation and image repair algorithm on the intermediate orthophoto to segment and repair the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole repair.

[0009] In one example, target data is obtained and preprocessed, specifically including: obtaining 1A-level SAR remote sensing image data of a preset resolution and 30-meter ASTER GDEMV3 DEM data; performing multi-view processing on the SAR remote sensing image data to obtain multi-view intensity data; performing filtering processing on the multi-view intensity data to remove speckle noise in the SAR remote sensing image to obtain denoised multi-view intensity data; performing DEM data mosaicking, cropping and coordinate conversion operations on the multi-view intensity data to obtain digital elevation model data in the WGS84 coordinate system covering the SAR image area.

[0010] In one example, the target data after preprocessing is directly orthorectified to obtain an initial SAR orthophoto, specifically including: based on the header file of the SAR remote sensing image data, using the orbital root description method to fit the satellite orbit and determine the relationship between the satellite position and time; based on the distance condition equation and the Doppler condition equation, establishing an RD model for characterizing the instantaneous image of the SAR remote sensing image; based on known digital elevation model data and the RD model, determining the spatial rectangular coordinates of the ground points of the SAR remote sensing image data; converting the spatial rectangular coordinates of the ground points into geodetic coordinates, and using the interpolation method to confirm the pixel value; traversing all SAR remote sensing images to obtain the initial SAR orthophoto.

[0011] In one example, the image segmentation and filtering processing of the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto specifically includes: generating a first binary image corresponding to the initial SAR orthophoto according to the pixel value of the initial SAR orthophoto; determining a first area to be repaired based on the first binary image; the first area to be repaired includes a first hole area and a first background value area, and the pixel value of the first area to be repaired is set to 255; the first area to be repaired is expanded using a convolution kernel of a preset size to expand a first mask area corresponding to the first area to be repaired; based on the first mask area, the SAR orthophoto is segmented into a foreground image and a background image; the first mask area is a foreground image; and the intermediate orthophoto is obtained by filtering the foreground image.

[0012] In one example, filtering the foreground image to obtain the intermediate orthophoto specifically includes: performing median filtering on the foreground image to remove the first hole noise in the foreground image; and superimposing the foreground image after removing the first hole noise with the background image to obtain the intermediate orthophoto.

[0013] In one example, the image segmentation and image restoration algorithm is performed on the intermediate orthophoto to segment and repair large holes in the intermediate orthophoto, so as to obtain a target SAR orthophoto after hole restoration, specifically including: generating a second binary image corresponding to the intermediate orthophoto according to the pixel value of the intermediate orthophoto; determining a second area to be restored based on the second binary image; the second area to be restored includes a second hole area and a second background value area, and the pixel value of the area to be restored is set to 255; the second area to be restored is expanded using a convolution kernel of a preset size to expand the second mask area corresponding to the second area to be restored; the second mask area is marked with eight connected domains using a Two-Pass algorithm; all connected domains are traversed to determine the connected domain area corresponding to the connected domain; based on and preset area threshold, the connected domain whose area is higher than the preset area threshold is marked as a non-mask area; after the traversal is completed, the target mask area after excluding the background value is obtained based on the non-mask area; and the target mask area is restored using the PatchMatch algorithm to obtain the target SAR orthophoto.

[0014] In one example, the size of the convolution kernel is 5*5.

[0015] In one example, the preset area threshold is proportional to the longitudinal size of the SAR remote sensing image.

[0016] The present application also provides a SAR image hole repair device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: acquiring target data and preprocessing the target data; the target data comprises SAR remote sensing image data and DEM data; performing direct orthorectification processing on the preprocessed target data to obtain an initial SAR orthoimage; performing image segmentation and filtering processing on the initial SAR orthoimage to segment and repair the first hole in the initial SAR orthoimage to obtain an intermediate orthoimage; performing image segmentation and image repair algorithm on the intermediate orthoimage to segment and repair the second hole in the intermediate orthoimage, thereby obtaining a target SAR orthoimage after hole repair.

[0017] The present application also provides a non-volatile computer storage medium storing computer executable instructions, characterized in that the computer executable instructions are configured to: obtain target data and preprocess the target data; the target data includes SAR remote sensing image data and DEM data; perform direct orthorectification processing on the preprocessed target data to obtain an initial SAR orthophoto; perform image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto; perform image segmentation and image restoration algorithm on the intermediate orthophoto to segment and repair the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole restoration.

[0018] The method proposed in this application can bring the following beneficial effects: First, the application repairs small holes and performs local filtering based on small hole masks, which maximizes the original information of SAR image data and improves the operation rate. The second step is to repair large holes. Based on morphology and connected domain screening methods, the image background value is automatically excluded, and the large hole mask is extracted for repair, avoiding manual participation in drawing the mask. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a process of repairing a hole in a SAR image in an embodiment of the present application;

[0021] Figure 2 Schematic diagram of the structure of a hole repair device for SAR images in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] There are many algorithms for image restoration, most of which are general image processing, such as photos. Image restoration based on SAR remote sensing images is less widely used, mainly focusing on filtering, interpolation filling, FMM (Fast Marching Method) algorithm, etc., which all have some limitations. This application will use the Patch Match algorithm when introducing the technical solution. The following is a comparison of common image restoration algorithms:

[0024] (1) Using filtering: It can repair noise within the range of 1 to 2 pixels, but it is not suitable for holes in a larger range and cannot be used alone to repair holes in the entire scene (remote sensing image unit, that is, the entire image to be processed). Different filtering methods, such as median, mean, Gaussian, bilateral filtering, etc., have little difference in the effect of noise removal.

[0025] (2) Interpolation-based filling methods: including moving surface fitting interpolation, Kriging interpolation, K-Nearest Neighbor (KNN) interpolation, etc., which perform well in some simple scenarios, but often have limited effects on complex image restoration, such as SAR image mountain restoration.

[0026] (3) Methods based on fluid dynamics: such as the Navier-Stokes (NS) algorithm. This type of algorithm simulates fluid dynamics equations to smooth images. It is often used to remove image defects, red-eye repair, cable or watermark removal, etc. It has a general effect on SAR remote sensing image repair.

[0027] (4) Fast Marching Method (FMM): This method is sensitive to noise. The inherent speckle noise of SAR images makes this algorithm unsuitable for SAR image hole repair, and the visual effect of the repair is average.

[0028] (5) Partial differential equation method: such as the Total Variation (TV) algorithm. The TV model is often used for image denoising and preservation of structured texture. It is suitable for repairing local damage in images of natural scenery, buildings, etc. It has important applications in document restoration, medical image processing and other fields.

[0029] (6) Block repair algorithms, such as the Patch Match algorithm, also known as the Criminisi algorithm, which is a fast approximate nearest neighbor search algorithm that was originally used for texture synthesis and later applied to image repair. It can efficiently find the best image blocks to fill the damaged area. It has been widely used in large-scale image repair, texture repair, and high-quality image content generation.

[0030] (7) Image restoration methods based on deep learning: They have strong data dependence, weak generalization ability, high computational complexity and resource requirements, and are rarely used in the field of remote sensing image restoration (especially SAR image restoration). They are not suitable for all SAR satellite image restoration.

[0031] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0032] Figure 1 A flowchart of a method for repairing holes in SAR images provided for one or more embodiments of this specification. In this application, data after direct orthorectification of SAR is used for illustration, but this method can also be applied to holes generated by other SAR processing (such as DEM, InSAR, etc.). As long as the data is the entire scene, that is, data containing background values, it is suitable for automatic processing of this method. The process can be executed by a computing device in the corresponding field, and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0033] The analysis method involved in the embodiment of the present application can be implemented as a terminal device or a server, and the present application does not impose any special restrictions on this. It should be noted that the server can be a single device or a system composed of multiple devices, that is, a distributed server, and the present application does not impose any specific restrictions on this.

[0034] For ease of understanding and description, the following embodiments are described in detail using a computing device as an example.

[0035] like Figure 1 As shown, the embodiment of the present application provides a method for repairing holes in a SAR image, comprising:

[0036] S101: Acquire SAR remote sensing image data, and preprocess the SAR remote sensing image data.

[0037] First, the target data that needs to be repaired for image holes is obtained, and the target data here includes SAR remote sensing images and DEM data. When preprocessing SAR remote sensing image data, multi-view processing and filtering processing are required for SAR single-view complex images.

[0038] Specifically, when acquiring target data, SAR remote sensing image 1A data can be purchased, and ASTER GDEMV3 DEM data with a resolution of 30 meters can be downloaded. The purchasing method is usually to find a satellite remote sensing data agent to purchase, and the same data can be reused; the DEM data is free and open data and can be downloaded from the Geospatial Data Cloud (https: / / www.gscloud.cn / ). It should be noted that the SAR remote sensing image 1A data here is a single-view complex data containing amplitude and phase information. When preprocessing it, multi-view processing is required, that is, averaging the azimuth and / or distance directions to obtain multi-view intensity data. Then, after filtering processing, such as Refine-Lee filtering, the SAR image speckle noise is removed to obtain denoised multi-view intensity data. When preprocessing DEM data, it is necessary to mosaic, crop, and coordinate transform multiple DEM data to obtain DEM data with WGS84 coordinates covering the SAR image area. The above preprocessing process is a prior art and will not be repeated here.

[0039] S102: Performing direct orthorectification processing on the pre-processed target data to obtain an initial SAR orthoimage.

[0040] In one embodiment, when performing orthorectification processing, the satellite orbit fitting must first be performed. Specifically, it is necessary to perform satellite orbit fitting based on the SAR image header file and the orbit element description method to solve the relationship between the satellite position and time. Then, based on the distance condition equation and the Doppler condition equation, an RD model (range-Doppler model) is established to characterize the instantaneous image of the SAR image. Specifically, the distance condition equation is as follows:

[0041] R 2 =(X S -X) 2 +(Y S -Y) 2 +(Z S -Z) 2

[0042] Where R represents the distance from the instantaneous position S of the radar antenna to the ground point P, X S , Y S , Z S It represents the instantaneous three-dimensional position of the satellite, and X, Y, and Z are the three-dimensional position of the ground point P.

[0043] The Doppler condition equation is as follows:

[0044]

[0045] Among them, f D is the Doppler frequency shift of the ground point P, λ is the radar wavelength, V X、V Y 、V Z Represents the instantaneous three-dimensional velocity of the satellite. Then, based on the known DEM and RD model, the spatial rectangular coordinates of the ground point are solved, converted into geodetic coordinates, and then the pixel value is confirmed using the interpolation method. By traversing all SAR remote sensing image data and executing the above steps, the orthorectified image can be obtained.

[0046] S103: performing image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto.

[0047] For the convenience of distinction, this application describes the smaller hole as the first hole and the larger hole as the second hole. When segmenting and repairing the first hole, it is necessary to combine image segmentation and morphological methods to extract the mask of the small hole in the orthorectified image and perform filtering processing on the mask area. The processing result is superimposed on the original SAR orthophoto to obtain the image after the first hole is repaired.

[0048] Specifically, when performing segmentation and restoration, it is necessary to generate a first binary image corresponding to the initial SAR orthophoto according to the pixel value of the initial SAR orthophoto; based on the first binary image, determine the first area to be restored; the first area to be restored includes a first hole area and a first background value area, and the pixel value of the first area to be restored is set to 255, and then the first area to be restored is expanded using a convolution kernel of a preset size to expand the first mask area corresponding to the first area to be restored. It should be noted that dilation is a commonly used method in morphological operations in the field of image processing:

[0049]

[0050] b represents all elements belonging to the structural element B, represents the set of structural element B after inversion at the origin. For the structural element B symmetric at the origin, The expansion of B to A is expressed as:

[0051]

[0052] That is, the set of all z whose intersection with set A after the inverted structural element B is translated by z is not empty, which can also be expressed as:

[0053]

[0054] The structural element B is usually called the dilated structural element, and a 5x5 convolution kernel is used in this method. After the first mask area is dilated, the SAR orthophoto can be segmented into a foreground image and a background image based on the first mask area; the part of the first mask area is the foreground image, and the non-mask area is the background image. It should be noted that the foreground image here is the area where the first hole is located. Perform median filtering on the foreground image to remove the first hole noise; then superimpose the background image to obtain the image after removing the first hole.

[0055] S104: performing image segmentation and image restoration algorithms on the intermediate orthophoto to segment and restore the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole restoration.

[0056] Specifically, when segmenting the second hole, it is first necessary to generate a second binary image corresponding to the intermediate orthophoto according to the pixel value of the intermediate orthophoto, and then determine the second area to be repaired based on the second binary image. It should be noted that the second area to be repaired here includes the second hole area and the second background value area, and the pixel value of the area to be repaired is set to 255. Then, the second area to be repaired can be expanded using a 5*5 convolution kernel to expand the second mask area corresponding to the second area to be repaired; then, the two-pass algorithm can be used to mark the second mask area with an eight-connected domain.

[0057] Specifically, the Two-Pass algorithm includes the following steps:

[0058] (1) First scan:

[0059] Visit the current pixel B(x,y), if B(x,y) is equal to 1: If the pixel values ​​in the domain of B(x,y) are all 0, assign B(x,y) a new label: label+=1, B(x,y)=label. If there are pixel neighbors with pixel values ​​> 1 in the domain of B(x,y), assign the minimum value in the neighbors to B(x,y): B(x,y)=min{Neighbors}. Record the equality between the values ​​(label) in the neighbors, that is, these values ​​(label) belong to the same connected area.

[0060] (2) Second scan:

[0061] Visit the current pixel B(x, y). If B(x, y)>1, find a minimum label value that is equal to label=B(x, y) and assign it to B(x, y). After the scan is completed, the pixels with the same label value in the image form the same connected area. Calculate the area of ​​the connected domain, that is, calculate the number of pixels in the connected domain. The preset area threshold T here is an empirical value, usually set to 1 / 20 of the entire scene size (pixel width * pixel height).

[0062] After marking the eight connected domains, the area of ​​the connected domain corresponding to the connected domain can be determined by traversing all connected domains; and based on the preset area threshold, the connected domain with an area higher than the preset area threshold is marked as a non-mask area; after the traversal is completed, the target mask area after excluding the background value is obtained based on the non-mask area; the repair algorithm is now explained: the core of the Patch Match algorithm is to calculate the corresponding relationship of the image blocks. The chief press defines the nearest neighbor (NNF) as a function f: The offset is defined on all possible image block coordinates (image block center positions) in image A and is applicable to a distance function D between two image blocks. The classic Patch Match algorithm mainly includes the following steps:

[0063] Position initialization: n neighboring image blocks of the area to be repaired are randomly initialized at offset positions in the non-repaired area, which are used to iterate position propagation and random search operations on all image blocks.

[0064] Iterative operation: After initialization, the iterative process begins. Each iteration is a full-field scanning process. The scanning unit is a patch. Each scan goes through two sub-processes: position propagation and random search.

[0065] Position propagation: The offset f(x,y) of the current block is determined by the following formula:

[0066] f(x,y)=argmin f [D(f(x,y)),D(f(x-1,y)),D(f(x,y-1))]

[0067] Among them, f(x-1,y) represents the offset of the neighboring block on the left of the current block, f(x,y-1) represents the offset of the neighboring block on the top of the current block, and D(v) represents the matching error between the block (x,y) in Figure A and the block (x,y)+v in Figure B.

[0068] Random search: Search for the local optimum through perturbation, with the matching block of the current block as (x, y) + f(x, y) as the center, and randomly match several times in the radius area with exponential decay until the radius shrinks to less than 1 pixel and the iteration is terminated.

[0069] u i =f(x,y)+wα i R i

[0070] Where i is the number of random searches, u i is the relative position of the current block (x, y) in the i-th random search matching block in Figure B, w is the maximum search radius, α is a fixed exponential decay factor between 0 and 1, R i is a two-dimensional random number in [-1,1]×[-1,1] that follows a uniform distribution. After the target mask area is repaired using the Patch Match algorithm, the target SAR orthophoto can be obtained.

[0071] In the actual production environment, SAR data is produced from level 1A, and the orthorectified products obtained may contain small holes (noise) and holes, which cannot be distinguished from the background value. Usually, manual mask drawing is required, which obviously does not meet the requirements of large-scale automated production. At the same time, for the repair of small holes, conventional filtering methods filter the entire scene image, which will change the pixel value of each pixel of the entire scene image on the one hand, and reduce the processing efficiency on the other hand.

[0072] This method proposes an optimized solution for the above two problems. Through morphology plus connected domain screening method, the hole mask is automatically extracted. The efficiency of small hole processing is improved through filtering processing based on small hole masks. In a large-scale automated processing environment, this method can effectively repair holes in SAR images and improve image quality; at the same time, it can achieve full automation and improve processing efficiency.

[0073] like Figure 2 As shown, an embodiment of the present application further provides a SAR image hole repair device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can:

[0074] Acquire target data and preprocess the target data; the target data includes SAR remote sensing image data and DEM data; perform direct orthorectification processing on the preprocessed target data to obtain an initial SAR orthophoto; perform image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto; perform image segmentation and image repair algorithm on the intermediate orthophoto to segment and repair the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole repair.

[0075] The embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0076] Acquire target data and preprocess the target data; the target data includes SAR remote sensing image data and DEM data; perform direct orthorectification processing on the preprocessed target data to obtain an initial SAR orthophoto; perform image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto; perform image segmentation and image repair algorithm on the intermediate orthophoto to segment and repair the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole repair.

[0077] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0078] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0079] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0080] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0084] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0085] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0086] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0087] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for repairing holes in SAR images, characterized in that: include: Acquire target data and pre-process the target data; the target data includes SAR remote sensing image data and DEM data; The pre-processed target data is directly orthorectified to obtain the initial SAR orthophoto; Performing image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto; Performing image segmentation and image restoration algorithms on the intermediate orthophoto to segment and restore the second hole in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole restoration; The performing image segmentation and filtering processing on the initial SAR orthophoto to segment and repair the first hole in the initial SAR orthophoto to obtain an intermediate orthophoto specifically includes: Generating a first binary image corresponding to the initial SAR orthoimage according to the pixel values ​​of the initial SAR orthoimage; Based on the first binary image, determining a first area to be repaired; the first area to be repaired includes a first hole area and a first background value area, and the pixel value of the first area to be repaired is set to 255; Using a convolution kernel of a preset size to dilate the first area to be repaired, so as to expand a first mask area corresponding to the first area to be repaired; Based on the first mask area, the SAR orthophoto is segmented into a foreground image and a background image; the first mask area is the foreground image; By filtering the foreground image, the intermediate orthophoto is obtained; The filtering process is performed on the foreground image to obtain the intermediate orthophoto, which specifically includes: Performing median filtering on the foreground image to remove first hole noise points in the foreground image; Superimposing the foreground image after removing the first hole noise point with the background image to obtain the intermediate orthophoto; The performing of an image segmentation and image restoration algorithm on the intermediate orthophoto to segment and restore the large holes in the intermediate orthophoto, thereby obtaining a target SAR orthophoto after hole restoration, specifically includes: Generating a second binary image corresponding to the intermediate orthoimage according to the pixel values ​​of the intermediate orthoimage; Based on the second binary image, determining a second area to be repaired; the second area to be repaired includes a second hole area and a second background value area, and the pixel value of the area to be repaired is set to 255; Using a convolution kernel of a preset size to dilate the second area to be repaired, so as to expand a second mask area corresponding to the second area to be repaired; Using the Two-Pass algorithm to mark the eight-connected domain in the second mask area; Traverse all connected domains and determine the connected domain area corresponding to the connected domain; Based on the preset area threshold, marking the connected domain whose area is higher than the preset area threshold as a non-mask area; After the traversal is completed, a target mask area is obtained based on the non-mask area after excluding the background value; The Patch Match algorithm is used to repair the target mask area to obtain the target SAR orthophoto.

2. The method according to claim 1, characterized in that: Acquire target data and preprocess the target data, specifically including: Obtain SAR remote sensing image data of preset resolution level 1A and 30-meter ASTER GDEMV3 DEM data; Performing multi-view processing on the SAR remote sensing image data to obtain multi-view intensity data; Performing filtering on the multi-view intensity data to remove speckle noise in the SAR remote sensing image to obtain denoised multi-view intensity data; The DEM data is mosaicked, cropped and converted to obtain digital elevation model data in the WGS84 coordinate system covering the SAR image area.

3. The method according to claim 2, characterized in that The process of directly orthorectifying the preprocessed target data to obtain an initial SAR orthoimage specifically includes: Based on the header file of SAR remote sensing image data, the orbital element description method is used to fit the satellite orbit and determine the relationship between the satellite position and time. Based on the range condition equation and Doppler condition equation, the RD model for characterizing the instantaneous configuration of SAR remote sensing images is established. Based on the known digital elevation model data and RD model, determine the spatial rectangular coordinates of the ground points of the SAR remote sensing image data; Converting the spatial rectangular coordinates of the ground points into geodetic coordinates and confirming pixel values ​​using an interpolation method; Traverse all SAR remote sensing images to obtain the initial SAR orthophoto.

4. The method according to claim 1, characterized in that: The size of the convolution kernel is 5*5.

5. The method according to claim 1, characterized in that The preset area threshold is proportional to the longitudinal size of the SAR remote sensing image.

6. A SAR image hole repair device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as claimed in any one of claims 1-5.

7. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to perform the method as claimed in any one of claims 1 to 5.

Images