Satellite remote sensing image restoration method and device, electronic equipment and storage medium

Abstract

The invention provides a satellite remote sensing image restoration method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing, the method adopts a cyclic multiple restoration technology (because to-be-restored pixels cyclic multiple times are generated), the restoration effect is good, and during restoration, the restoration efficiency is high. The method is realized according to the difference degree between the current to-be-restored pixel and each restoration source pixel, the data volume of the restoration source pixel is smaller than that of the whole to-be-restored satellite remote sensing image, so that the calculated amount is small, the processing speed is high, in addition, the restoration source pixel comes from the to-be-restored satellite remote sensing image, and additional samples and external images are not needed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, electronic equipment and storage medium for repairing satellite remote sensing images. Background Art

[0002] Image restoration refers to restoring the pixel features of the missing parts of a damaged image. Commonly used image restoration methods include optimal matching, fusion, and deep network methods. However, these methods have certain shortcomings for the restoration of satellite remote sensing images. For example, the optimal matching method involves calculating the matching value pixel by pixel, which requires a large amount of calculation and is difficult to meet the requirements of processing timeliness; the fusion method requires multiple images or images taken by different sensors in the same area, which has high requirements for external data sources; the deep network method requires a large number of data samples for model training and learning.

[0003] In summary, traditional satellite remote sensing image restoration methods have technical problems such as large computational complexity, slow processing speed, and complex data preparation. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for repairing satellite remote sensing images to alleviate the technical problems of existing satellite remote sensing image repair methods, such as large amount of calculation, slow processing speed and complex data preparation.

[0005] In a first aspect, an embodiment of the present invention provides a method for repairing a satellite remote sensing image, comprising:

[0006] Acquire a satellite remote sensing image to be repaired, and determine pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels;

[0007] Randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired that are cycled multiple times based on the randomly sorted pixels to be repaired;

[0008] Traversing each current pixel to be repaired among the pixels to be repaired that have been cycled multiple times, determining a first repair source pixel that is located within a preset radius of the position coordinates and belongs to the valid pixel according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, and randomly selecting a preset number of pixels from the valid pixels as second repair source pixels, and then using the first repair source pixel and the second repair source pixel as repair source pixels of the current pixel to be repaired, wherein the second repair source pixel is different from the first repair source pixel;

[0009] The difference between the current pixel to be repaired and each of the repair source pixels is calculated, and the pixel value of the target repair source pixel with the smallest difference is used as the pixel value of the current pixel to be repaired to fill the position coordinate, and the valid pixels are updated until each current pixel to be repaired in the pixels to be repaired in the multiple cycles is traversed, so as to obtain a repaired image corresponding to the satellite remote sensing image to be repaired.

[0010] Furthermore, generating pixels to be repaired for multiple cycles according to the randomly sorted pixels to be repaired includes:

[0011] Determine the pixels to be repaired in the current cycle according to the randomly sorted pixels to be repaired, wherein the pixels to be repaired in the current cycle are the first (1 / 2) of the pixels to be repaired in the randomly sorted pixels. n The pixels to be repaired after random sorting, n represents the number of cycles;

[0012] The pixels to be repaired in the current cycle are appended to the end of the pixels to be repaired obtained in the previous cycle to obtain the pixels to be repaired in the current cycle, until the number of pixels to be repaired in the current cycle is 1, and the pixels to be repaired obtained in the last cycle are used as the pixels to be repaired in the multiple cycles.

[0013] Furthermore, calculating the difference between the current pixel to be repaired and each of the repair source pixels includes:

[0014] Calculation formula based on difference Calculate the difference between the current pixel to be repaired and each of the repair source pixels, where g represents the difference between the current pixel to be repaired and one of the repair source pixels, Nb represents the set of valid pixels within a preset radius of the current pixel to be repaired, and w i Represents the weight value of the two-dimensional Gaussian distribution, a i represents the pixel value of the valid pixel within the preset radius of the current pixel to be repaired, b i represents the pixel value of the valid pixel within the preset radius of the repair source pixel, dx and dy represent the relative positions of the position coordinates of the effective pixels within the preset radius of the repair source pixel relative to the position coordinates of the current pixel to be repaired, σ=R 2 , R represents the preset radius range.

[0015] Furthermore, the preset radius range and the preset number are related to the size of the satellite remote sensing image to be restored.

[0016] Furthermore, the satellite remote sensing image to be repaired is a satellite remote sensing observation image in an unsigned 16-bit integer pixel format.

[0017] In a second aspect, an embodiment of the present invention further provides a satellite remote sensing image repair device, comprising:

[0018] An acquisition and determination unit, used for acquiring a satellite remote sensing image to be repaired, and determining pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels;

[0019] A random sorting and generating unit, used for randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired for multiple cycles based on the randomly sorted pixels to be repaired;

[0020] a determining unit, configured to traverse each current pixel to be repaired among the pixels to be repaired that have been circulated multiple times, determine, according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, a first repair source pixel that is located within a preset radius of the position coordinates and belongs to the valid pixel, and randomly select a preset number of pixels from the valid pixels as second repair source pixels, and then use the first repair source pixel and the second repair source pixel as repair source pixels of the current pixel to be repaired, wherein the second repair source pixel is different from the first repair source pixel;

[0021] The calculation and filling unit is used to calculate the difference between the current pixel to be repaired and each of the repair source pixels, and fill the position coordinate with the pixel value of the target repair source pixel with the smallest difference as the pixel value of the current pixel to be repaired, and update the valid pixels until each current pixel to be repaired in the pixels to be repaired in the multiple cycles is traversed to obtain a repaired image corresponding to the satellite remote sensing image to be repaired.

[0022] Furthermore, the random sorting and generating unit is also used for:

[0023] Determine the pixels to be repaired in the current cycle according to the randomly sorted pixels to be repaired, wherein the pixels to be repaired in the current cycle are the first (1 / 2) of the pixels to be repaired in the randomly sorted pixels. n The pixels to be repaired after random sorting, n represents the number of cycles;

[0024] The pixels to be repaired in the current cycle are appended to the end of the pixels to be repaired obtained in the previous cycle to obtain the pixels to be repaired in the current cycle, until the number of pixels to be repaired in the current cycle is 1, and the pixels to be repaired obtained in the last cycle are used as the pixels to be repaired in the multiple cycles.

[0025] Furthermore, the calculation and filling unit is also used for:

[0026] Calculation formula based on difference Calculate the difference between the current pixel to be repaired and each of the repair source pixels, where g represents the difference between the current pixel to be repaired and one of the repair source pixels, Nb represents the set of valid pixels within a preset radius of the current pixel to be repaired, and w i Represents the weight value of the two-dimensional Gaussian distribution, a i represents the pixel value of the valid pixel within the preset radius of the current pixel to be repaired, b i represents the pixel value of the valid pixel within the preset radius of the repair source pixel, dx and dy represent the relative positions of the position coordinates of the effective pixels within the preset radius of the repair source pixel relative to the position coordinates of the current pixel to be repaired, σ=R 2 , R represents the preset radius range.

[0027] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0028] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0029] In an embodiment of the present invention, a method for repairing a satellite remote sensing image is provided, comprising: obtaining a satellite remote sensing image to be repaired, and determining pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels; randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired that are circulated multiple times based on the randomly sorted pixels to be repaired; traversing each current pixel to be repaired in the pixels to be repaired that are circulated multiple times, and determining, based on the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, pixels that are within a preset radius of the position coordinates and are valid pixels; The first restoration source pixel of the effective pixel is selected, and a preset number of pixels are randomly selected from the effective pixels as the second restoration source pixels, and then the first restoration source pixel and the second restoration source pixel are used as the restoration source pixels of the current pixel to be restored, wherein the second restoration source pixel is different from the first restoration source pixel; the difference between the current pixel to be restored and each restoration source pixel is calculated, and the pixel value of the target restoration source pixel with the smallest difference is filled into the position coordinate as the pixel value of the current pixel to be restored, and the effective pixels are updated until each current pixel to be restored in the pixels to be restored that have been cycled multiple times is traversed, and a restoration image corresponding to the satellite remote sensing image to be restored is obtained. From the above description, it can be seen that the satellite remote sensing image repair method of the present invention adopts a cyclic multiple repair technology (because the pixels to be repaired are generated in multiple cycles), and the repair effect is good. Moreover, when repairing, it is implemented based on the difference between the current pixel to be repaired and each repair source pixel. The data amount of the repair source pixel is small relative to the data amount of the entire satellite remote sensing image to be repaired, so the calculation amount is small and the processing speed is fast. In addition, the repair source pixels are derived from the satellite remote sensing image to be repaired itself, and no additional samples and external images are required, which alleviates the technical problems of large calculation amount, slow processing speed and complex data preparation in the traditional satellite remote sensing image repair method. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A flowchart of a satellite remote sensing image repair method provided by an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of a satellite remote sensing image to be restored provided by an embodiment of the present invention;

[0033] Figure 3A schematic diagram of a restored image corresponding to a satellite remote sensing image to be restored provided by an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of a satellite remote sensing image restoration device provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Traditional satellite remote sensing image restoration methods require large amounts of computation, slow processing speed, and complex data preparation.

[0038] Based on this, the satellite remote sensing image repair method of the present invention adopts a cyclic multiple repair technology (because the pixels to be repaired are generated in multiple cycles), and the repair effect is good. Moreover, when repairing, it is implemented based on the difference between the current pixel to be repaired and each repair source pixel. The data amount of the repair source pixel is small relative to the data amount of the entire satellite remote sensing image to be repaired. Therefore, the calculation amount is small and the processing speed is fast. In addition, the repair source pixels are derived from the satellite remote sensing image to be repaired itself, and no additional samples and external images are required.

[0039] To facilitate understanding of this embodiment, a method for repairing a satellite remote sensing image disclosed in an embodiment of the present invention is first introduced in detail.

[0040] Embodiment 1:

[0041] According to an embodiment of the present invention, an embodiment of a method for repairing a satellite remote sensing image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Figure 1 is a flow chart of a method for repairing a satellite remote sensing image according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0043] Step S102, obtaining a satellite remote sensing image to be repaired, and determining pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels;

[0044] In an embodiment of the present invention, the satellite remote sensing image to be repaired is a satellite remote sensing observation image in an unsigned 16-bit integer pixel format, and the pixels to be repaired include two types: signal saturated pixels and missing pixels caused by imaging instrument problems. If the satellite remote sensing image to be repaired is not in an unsigned 16-bit integer pixel format, the satellite remote sensing image to be repaired is converted into an unsigned 16-bit integer pixel format for storage. Then, the signal saturated pixels and the missing pixels are uniformly marked as pixels to be repaired using a pixel value of 65535, and the pixels corresponding to other pixel values ​​are valid pixels.

[0045] During implementation, the data provider will provide descriptions of the corresponding pixel values ​​in each range in the satellite remote sensing image to be repaired. For example, the pixels corresponding to the pixel values ​​in the range A1-A2 are signal-saturated pixels, the pixels corresponding to the pixel values ​​in the range A2-A3 are valid pixels, and the pixels corresponding to the pixel values ​​in the range A3-A4 are missing pixels.

[0046] In the above process, the signal saturated pixels and the missing pixels are uniformly marked as pixels to be repaired using the pixel value of 65535, so that each pixel of the image can be judged whether it is a pixel to be repaired (judged whether it is a pixel to be repaired based on whether it is a pixel value of 65535. When implemented, it is scanned row by row from the pixel in the first row and first column until the pixel in the last row and last column is scanned). If it is a pixel to be repaired, the row and column position information (x, y) of the pixel in the satellite remote sensing image to be repaired, that is, the position coordinates, are recorded, and these pixels are the set of pixels to be repaired N. For valid pixels, their row and column position information in the satellite remote sensing image to be repaired is also recorded, and these pixels are the set of valid pixels S.

[0047] Step S104, randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired for multiple cycles based on the randomly sorted pixels to be repaired;

[0048] Specifically, the pixels to be repaired obtained in the above process have certain sequence information because they are obtained by scanning line by line. In order to reduce obvious traces of sequential repair in the repair result, the pixels to be repaired are randomly sorted to obtain randomly sorted pixels to be repaired, that is, the randomly sorted pixel set N0 to be repaired.

[0049] The local texture features of an image are an important basis for image restoration. When performing image restoration, it can be performed by either one pixel or a small piece of the image. The present invention adopts one pixel restoration.

[0050] The inventor considers selecting a pixel A to be repaired from the randomly sorted pixels to be repaired one by one in order, and finding the pixel closest to the surrounding texture features of the position of A from the valid pixel set S as the best matching pixel for filling. In the early stage of the algorithm, since there are few or no surrounding pixels of A, the best pixel selected from the valid pixels cannot produce a good effect. Therefore, the present invention repeatedly adds the initial part of the pixels to be repaired in the randomly sorted pixels to be repaired (that is, the front part of the pixels to be repaired in the randomly sorted pixels to be repaired) to the tail of the randomly sorted pixels to be repaired, and obtains the pixels to be repaired that are cycled multiple times, thereby realizing multiple repairs of these initial part of the pixels to be repaired. The process is described in detail below and will not be repeated here.

[0051] Step S106, traversing each current pixel to be repaired among the pixels to be repaired that have been cycled multiple times, determining a first repair source pixel that is located within a preset radius of the position coordinates and is a valid pixel according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, and randomly selecting a preset number of pixels from the valid pixels as second repair source pixels, and then using the first repair source pixel and the second repair source pixel as repair source pixels of the current pixel to be repaired, wherein the second repair source pixel is different from the first repair source pixel;

[0052] Specifically, traditional image restoration methods have two ways to restore the source pixels: one is the satellite remote sensing image to be restored, and the other is the sample image. The problem with the former is that the global search for the optimal matching point results in a large amount of calculation and low processing efficiency; the problem with the latter is that it requires finding suitable samples or providing a large number of samples for training, which requires more time to prepare samples and train models.

[0053] In order to avoid the shortcomings of the above method, the present invention uses the satellite remote sensing image to be repaired as the source of the repair source pixels, that is, the effective pixel set S mentioned above, so as to avoid the work of constructing samples. At the same time, the single observation of the satellite remote sensing image is closely related to the imaging geometry and the sunshine conditions. Finding the repair source pixels from the image itself (that is, the satellite remote sensing image to be repaired) can maximize the reduction of the differences caused by external samples. On the other hand, in order to avoid the time-consuming operation of global search for the optimal matching point, the present invention adopts the method of neighborhood (that is, the first repair source pixel) and random finite global sample set (that is, the second repair source pixel) to reduce the calculation time of the matching sample point. That is, according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, the first repair source pixel located within the preset radius of the position coordinate and belonging to the effective pixel is determined, and a preset number of pixels (and pixels different from the first repair source pixel) are randomly selected from the effective pixels as the second repair source pixel, and then the first repair source pixel and the second repair source pixel are used as the repair source pixel of the current pixel to be repaired.

[0054] The specific method is as follows: (1) Select pixels within the radius R around the position coordinates (x, y) of the pixel A to be repaired (i.e., within the preset radius) and in the valid pixel set S as the first repair source pixel set SUB1; (2) Randomly select M pixels from the valid pixel set S (here S refers to S except SUB1, the purpose is to ensure that SUB1 and SUB2 do not have repeated pixels) as the second repair source pixel set SUB2. SUB1 and SUB2 constitute the repair source pixel set SUB for the pixel A to be repaired. The specific values ​​of R and M need to be adjusted according to the size of the satellite remote sensing image to be repaired. Taking a 1200x1200 satellite remote sensing image to be repaired as an example, R=6, M=144 can achieve better results.

[0055] Step S108, calculate the difference between the current pixel to be repaired and each repair source pixel, and fill the position coordinate with the pixel value of the target repair source pixel with the smallest difference as the pixel value of the current pixel to be repaired, and update the valid pixels until each current pixel to be repaired in the pixels to be repaired that have been cycled multiple times is traversed to obtain the repaired image corresponding to the satellite remote sensing image to be repaired.

[0056] Specifically, after the current pixel to be repaired is repaired, it is placed in the valid pixel set as a valid pixel (that is, the next repair is performed based on the result of the previous repair), and then the repair source pixel of the next pixel to be repaired is determined, and the subsequent repair process is executed.

[0057] In an embodiment of the present invention, a method for repairing a satellite remote sensing image is provided, comprising: obtaining a satellite remote sensing image to be repaired, and determining pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels; randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired that are circulated multiple times based on the randomly sorted pixels to be repaired; traversing each current pixel to be repaired in the pixels to be repaired that are circulated multiple times, and determining, based on the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, pixels that are within a preset radius of the position coordinates and are valid pixels; The first restoration source pixel of the effective pixel is selected, and a preset number of pixels are randomly selected from the effective pixels as the second restoration source pixels, and then the first restoration source pixel and the second restoration source pixel are used as the restoration source pixels of the current pixel to be restored, wherein the second restoration source pixel is different from the first restoration source pixel; the difference between the current pixel to be restored and each restoration source pixel is calculated, and the pixel value of the target restoration source pixel with the smallest difference is filled into the position coordinate as the pixel value of the current pixel to be restored, and the effective pixels are updated until each current pixel to be restored in the pixels to be restored that have been cycled multiple times is traversed, and a restoration image corresponding to the satellite remote sensing image to be restored is obtained. From the above description, it can be seen that the satellite remote sensing image repair method of the present invention adopts a cyclic multiple repair technology (because the pixels to be repaired are generated in multiple cycles), and the repair effect is good. Moreover, when repairing, it is implemented based on the difference between the current pixel to be repaired and each repair source pixel. The data amount of the repair source pixel is small relative to the data amount of the entire satellite remote sensing image to be repaired, so the calculation amount is small and the processing speed is fast. In addition, the repair source pixels are derived from the satellite remote sensing image to be repaired itself, and no additional samples and external images are required, which alleviates the technical problems of large calculation amount, slow processing speed and complex data preparation in the traditional satellite remote sensing image repair method.

[0058] The above content briefly introduces the satellite remote sensing image repair method of the present invention, and the specific contents involved are described in detail below.

[0059] In an optional embodiment of the present invention, generating pixels to be repaired in multiple cycles according to the randomly sorted pixels to be repaired specifically includes the following steps:

[0060] (1) Determine the pixels to be repaired in the current cycle according to the randomly sorted pixels to be repaired, wherein the pixels to be repaired in the current cycle are the first (1 / 2) of the pixels to be repaired in the randomly sorted pixels. n The pixels to be repaired after random sorting, n represents the number of cycles;

[0061] (2) The pixels to be repaired in the current cycle are appended to the end of the pixels to be repaired obtained in the previous cycle to obtain the pixels to be repaired in the current cycle, until the number of pixels to be repaired in the current cycle is 1, and the pixels to be repaired obtained in the last cycle are used as the pixels to be repaired in multiple cycles.

[0062] Specifically, according to the order of the pixels to be repaired in the randomly sorted pixel set N0, the first half of the pixels to be repaired in N0 are selected, and then they are sequentially appended to the end of N0 to form a pixel set N1 to be repaired that is circulated once, and then the first quarter of the pixels to be repaired in N0 are selected and sequentially appended to the end of N1 to form a pixel set N2 to be repaired that is circulated twice, and so on, each time taking out the first half of the pixels of the previous additional pixels and appending them to the data to be repaired, until there is only one additional pixel. According to the above operations, the final pixel set (NN) to be repaired that is circulated multiple times is formed.

[0063] In an optional embodiment of the present invention, calculating the difference between the current pixel to be repaired and each repair source pixel specifically includes the following steps:

[0064] Calculation formula based on difference Calculate the difference between the current pixel to be repaired and each repair source pixel, where g represents the difference between the current pixel to be repaired and a repair source pixel, Nb represents the set of valid pixels within the preset radius of the current pixel to be repaired, and w i Represents the weight value of the two-dimensional Gaussian distribution, a i Indicates the pixel value of the valid pixel within the preset radius of the current pixel to be repaired, b i Indicates the pixel value of the valid pixel within the preset radius of the repaired source pixel. dx and dy represent the relative positions of the position coordinates of the effective pixels within the preset radius of the repair source pixel relative to the position coordinates of the current pixel to be repaired, σ = R 2 , R represents the preset radius range.

[0065] To better understand the process, let's take an example:

[0066] Assume that the current pixel to be repaired is A, the preset radius range is 1, and there are 8 valid pixels within a pixel radius range of the current pixel A to be repaired, which are located above, below, left, right, upper left, upper right, lower left and lower right of the current pixel A to be repaired. There are 8 valid pixels within a pixel radius range of the current pixel A to be repaired, and the number of valid pixels randomly selected from the valid pixels is 144. Therefore, there are a total of 152 repair source pixels for the current pixel A to be repaired.

[0067] a iThat is, the 8 valid pixels within a pixel radius of the current pixel A to be repaired, b i That is, for the 8 valid pixels (upper, lower, left, right, upper left, upper right, lower left and lower right) within a pixel radius of one of the 152 repair source pixels, calculate the above a i and b i The square of the difference between the pixel values ​​at corresponding positions (for example, the square of the difference between the upper pixel value and the upper pixel value, the square of the difference between the lower pixel value and the lower pixel value, etc., and finally their weighted sum is calculated, and the difference between the current pixel A to be repaired and a repair source pixel is obtained, and then the difference between the current pixel A to be repaired and other repair source pixels is solved. In this way, 152 differences can be solved.

[0068] The present invention proposes random sampling, restoration based on image features, and multiple restoration cycles to achieve relatively good processing efficiency and effect. The above process has a small amount of calculation and a fast processing speed. The restoration source pixels come from the satellite remote sensing image to be restored, and no additional samples or external images are required. Figure 2 Schematic diagram of the satellite remote sensing image to be repaired is shown in FIG. Figure 3 Schematic diagram of the repaired image corresponding to the satellite remote sensing image to be repaired is shown in FIG. 1 . It can be seen that the repair effect is good by adopting the method of the present invention.

[0069] Embodiment 2:

[0070] An embodiment of the present invention also provides a satellite remote sensing image repair device, which is mainly used to execute the satellite remote sensing image repair method provided in the first embodiment of the present invention. The satellite remote sensing image repair device provided in the embodiment of the present invention is specifically introduced below.

[0071] Figure 4 is a schematic diagram of a satellite remote sensing image restoration device according to an embodiment of the present invention. Figure 4 As shown, the device mainly includes: an acquisition and determination unit 10, a random sorting and generation unit 20, a determination unit 30, and a calculation and filling unit 40, wherein:

[0072] The acquisition and determination unit is used to acquire the satellite remote sensing image to be repaired, and determine the pixels to be repaired and the valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels;

[0073] A random sorting and generating unit, used for randomly sorting the pixels to be repaired to obtain the randomly sorted pixels to be repaired, and generating the pixels to be repaired for multiple cycles according to the randomly sorted pixels to be repaired;

[0074] A determination unit is used to traverse each current pixel to be repaired among the pixels to be repaired that have been cycled multiple times, determine a first repair source pixel that is located within a preset radius of the position coordinates and is a valid pixel according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, and randomly select a preset number of pixels from the valid pixels as second repair source pixels, and then use the first repair source pixel and the second repair source pixel as repair source pixels of the current pixel to be repaired, wherein the second repair source pixel is different from the first repair source pixel;

[0075] The calculation and filling unit is used to calculate the difference between the current pixel to be repaired and each repair source pixel, and fill the position coordinate with the pixel value of the target repair source pixel with the smallest difference as the pixel value of the current pixel to be repaired, and update the valid pixels until each current pixel to be repaired in the pixels to be repaired that have been cycled multiple times is traversed to obtain the repaired image corresponding to the satellite remote sensing image to be repaired.

[0076] In an embodiment of the present invention, a satellite remote sensing image repair device is provided, comprising: obtaining a satellite remote sensing image to be repaired, and determining pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels; randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired that are circulated multiple times based on the randomly sorted pixels to be repaired; traversing each current pixel to be repaired in the pixels to be repaired that are circulated multiple times, and determining, based on the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, pixels that are within a preset radius of the position coordinates and are valid pixels; The first restoration source pixel of the effective pixel is selected, and a preset number of pixels are randomly selected from the effective pixels as the second restoration source pixels, and then the first restoration source pixel and the second restoration source pixel are used as the restoration source pixels of the current pixel to be restored, wherein the second restoration source pixel is different from the first restoration source pixel; the difference between the current pixel to be restored and each restoration source pixel is calculated, and the pixel value of the target restoration source pixel with the smallest difference is filled into the position coordinate as the pixel value of the current pixel to be restored, and the effective pixels are updated until each current pixel to be restored in the pixels to be restored that have been cycled multiple times is traversed, and a restoration image corresponding to the satellite remote sensing image to be restored is obtained. From the above description, it can be seen that the satellite remote sensing image repair device of the present invention adopts a cyclic multiple repair technology (because the pixels to be repaired are generated in multiple cycles), and the repair effect is good. Moreover, when repairing, it is implemented based on the difference between the current pixel to be repaired and each repair source pixel. The data amount of the repair source pixel is small relative to the data amount of the entire satellite remote sensing image to be repaired, so the calculation amount is small and the processing speed is fast. In addition, the repair source pixels are derived from the satellite remote sensing image to be repaired itself, and no additional samples and external images are required, which alleviates the technical problems of large calculation amount, slow processing speed and complex data preparation in the traditional satellite remote sensing image repair method.

[0077] Optionally, the random sorting and generating unit is further used to: determine the pixels to be repaired in the current cycle according to the pixels to be repaired after the random sorting, wherein the pixels to be repaired in the current cycle are the first (1 / 2) of the pixels to be repaired after the random sorting. n The pixels to be repaired after random sorting are obtained, and n represents the number of cycles; the pixels to be repaired in the current cycle are appended to the end of the pixels to be repaired obtained in the previous cycle to obtain the pixels to be repaired obtained in the current cycle, until the number of pixels to be repaired in the current cycle is 1, and the pixels to be repaired obtained in the last cycle are used as the pixels to be repaired for multiple cycles.

[0078] Optionally, the calculation and filling unit is further used to: calculate the formula according to the difference Calculate the difference between the current pixel to be repaired and each repair source pixel, where g represents the difference between the current pixel to be repaired and a repair source pixel, Nb represents the set of valid pixels within the preset radius of the current pixel to be repaired, and w i Represents the weight value of the two-dimensional Gaussian distribution, a i Indicates the pixel value of the valid pixel within the preset radius of the current pixel to be repaired, b i Indicates the pixel value of the valid pixel within the preset radius of the repaired source pixel. dx and dy represent the relative positions of the position coordinates of the effective pixels within the preset radius of the repair source pixel relative to the position coordinates of the current pixel to be repaired, σ = R 2 , R represents the preset radius range.

[0079] Optionally, the preset radius range and the preset number are related to the size of the satellite remote sensing image to be restored.

[0080] Optionally, the satellite remote sensing image to be restored is a satellite remote sensing observation image in an unsigned 16-bit integer pixel format.

[0081] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0082] like Figure 5 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the satellite remote sensing image repair method as described above.

[0083] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memories and processors, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the above-mentioned satellite remote sensing image repair method can be executed.

[0084] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0085] Corresponding to the above-mentioned satellite remote sensing image repair method, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned satellite remote sensing image repair method.

[0086] The satellite remote sensing image repair device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present application are the same as those of the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0087] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0088] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0091] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the satellite remote sensing image repair method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0092] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0093] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for repairing satellite remote sensing images, characterized in that: include: Acquire a satellite remote sensing image to be repaired, and determine pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels; Randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired that are cycled multiple times based on the randomly sorted pixels to be repaired; Traversing each current pixel to be repaired among the pixels to be repaired that have been cycled multiple times, determining a first repair source pixel that is located within a preset radius of the position coordinates and belongs to the valid pixel according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, and randomly selecting a preset number of pixels from the valid pixels as second repair source pixels, and then using the first repair source pixel and the second repair source pixel as repair source pixels of the current pixel to be repaired, wherein the second repair source pixel is different from the first repair source pixel; The difference between the current pixel to be repaired and each of the repair source pixels is calculated, and the pixel value of the target repair source pixel with the smallest difference is used as the pixel value of the current pixel to be repaired to fill the position coordinate, and the valid pixels are updated until each current pixel to be repaired in the pixels to be repaired in the multiple cycles is traversed, so as to obtain a repaired image corresponding to the satellite remote sensing image to be repaired.

2. The method according to claim 1, characterized in that Generating pixels to be repaired for multiple cycles according to the randomly sorted pixels to be repaired includes: Determine the pixels to be repaired in the current cycle according to the randomly sorted pixels to be repaired, wherein the pixels to be repaired in the current cycle are the first (1 / 2) of the pixels to be repaired in the randomly sorted pixels. n The pixels to be repaired after random sorting, n represents the number of cycles; The pixels to be repaired in the current cycle are appended to the end of the pixels to be repaired obtained in the previous cycle to obtain the pixels to be repaired in the current cycle, until the number of pixels to be repaired in the current cycle is 1, and the pixels to be repaired obtained in the last cycle are used as the pixels to be repaired in the multiple cycles.

3. The method according to claim 1, characterized in that Calculating the difference between the current pixel to be repaired and each of the repair source pixels includes: Calculation formula based on difference Calculate the difference between the current pixel to be repaired and each of the repair source pixels, where g represents the difference between the current pixel to be repaired and one of the repair source pixels, Nb represents the set of valid pixels within a preset radius of the current pixel to be repaired, and w i Represents the weight value of the two-dimensional Gaussian distribution, a i represents the pixel value of the valid pixel within the preset radius of the current pixel to be repaired, b i represents the pixel value of the valid pixel within the preset radius of the repair source pixel, dx and dy represent the relative positions of the position coordinates of the effective pixels within the preset radius of the repair source pixel relative to the position coordinates of the current pixel to be repaired, σ=R 2 , R represents the preset radius range.

4. The method according to claim 1, characterized in that: The preset radius range and the preset number are related to the size of the satellite remote sensing image to be restored.

5. The method according to claim 1, characterized in that The satellite remote sensing image to be repaired is a satellite remote sensing observation image in an unsigned 16-bit integer pixel format.

6. A satellite remote sensing image restoration device, characterized in that: include: An acquisition and determination unit, used for acquiring a satellite remote sensing image to be repaired, and determining pixels to be repaired and valid pixels in the satellite remote sensing image to be repaired, wherein the pixels to be repaired include: signal saturated pixels and missing pixels; A random sorting and generating unit, used for randomly sorting the pixels to be repaired to obtain randomly sorted pixels to be repaired, and generating pixels to be repaired for multiple cycles based on the randomly sorted pixels to be repaired; a determining unit, configured to traverse each current pixel to be repaired among the pixels to be repaired that have been circulated multiple times, determine, according to the position coordinates of the current pixel to be repaired in the satellite remote sensing image to be repaired, a first repair source pixel that is located within a preset radius of the position coordinates and belongs to the valid pixel, and randomly select a preset number of pixels from the valid pixels as second repair source pixels, and then use the first repair source pixel and the second repair source pixel as repair source pixels of the current pixel to be repaired, wherein the second repair source pixel is different from the first repair source pixel; The calculation and filling unit is used to calculate the difference between the current pixel to be repaired and each of the repair source pixels, and fill the position coordinate with the pixel value of the target repair source pixel with the smallest difference as the pixel value of the current pixel to be repaired, and update the valid pixels until each current pixel to be repaired in the pixels to be repaired in the multiple cycles is traversed to obtain a repaired image corresponding to the satellite remote sensing image to be repaired.

7. The device according to claim 6, characterized in that The random sorting and generating unit is also used for: Determine the pixels to be repaired in the current cycle according to the randomly sorted pixels to be repaired, wherein the pixels to be repaired in the current cycle are the first (1 / 2) of the pixels to be repaired in the randomly sorted pixels. n The pixels to be repaired after random sorting, n represents the number of cycles; The pixels to be repaired in the current cycle are appended to the end of the pixels to be repaired obtained in the previous cycle to obtain the pixels to be repaired in the current cycle, until the number of pixels to be repaired in the current cycle is 1, and the pixels to be repaired obtained in the last cycle are used as the pixels to be repaired in the multiple cycles.

8. The device according to claim 6, characterized in that The calculation and filling unit is also used for: Calculation formula based on difference Calculate the difference between the current pixel to be repaired and each of the repair source pixels, where g represents the difference between the current pixel to be repaired and one of the repair source pixels, Nb represents the set of valid pixels within a preset radius of the current pixel to be repaired, and w i Represents the weight value of the two-dimensional Gaussian distribution, a i represents the pixel value of the valid pixel within the preset radius of the current pixel to be repaired, b i represents the pixel value of the valid pixel within the preset radius of the repair source pixel, dx and dy represent the relative positions of the position coordinates of the effective pixels within the preset radius of the repair source pixel relative to the position coordinates of the current pixel to be repaired, σ=R 2 , R represents the preset radius range.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images