Optical remote sensing image processing method and device

Abstract

The invention discloses an optical remote sensing image processing method and device, and the method comprises the steps: obtaining a remote sensing image, and constructing a remote sensing image set in different use scenes based on the remote sensing image; selecting a target image set from the remote sensing image set, and performing cloud removal processing on each remote sensing image in the target image set; wherein the cloud removal processing comprises time domain cloud removal, space domain cloud removal and edge detection and filling; and in a time domain cloud removal process, determining a selection time limit of time domain information according to a use scene of the target image set, and performing time domain cloud removal on the target image set based on the selection time limit. Therefore, the time domain and space domain information is fully utilized, the computer vision technology is combined, meanwhile, the method has efficient parallel computing capacity, the problems that in the prior art, the remote sensing image cloud removal effect of a long-time-sequence day-by-day remote sensing product is poor, and the processing period is long are solved, the problem of space-time discontinuity caused by cloud pollution is solved, and the remote sensing image cloud removal efficiency is improved. And the usability and the analysis precision of the remote sensing data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an optical remote sensing image processing method and device. Background Art

[0002] At present, remote sensing technology has become the main way to monitor the earth, and has accumulated a large amount of multi-type earth observation data, involving a variety of scenarios, such as environmental monitoring, agricultural management, urbanization monitoring and disaster emergency response, etc., especially remote sensing data with high temporal resolution (daily or even hourly) provides an important data basis for further monitoring of changes in the earth's surface. In this process, a large number of remote sensing products of various topics have been generated. It should be understood that different thematic remote sensing products serve different ground objects, which can also be understood as different application scenarios. Optical images do not contain speckle noise and geographic distortion, and are highly readable by the naked eye. Therefore, using optical remote sensing to image the earth is the most common and intuitive means of remote sensing earth observation. However, optical remote sensing images are inevitably contaminated or covered by clouds and shadows, resulting in the loss of real ground information, which greatly limits the application of optical remote sensing images.

[0003] Therefore, in the prior art, cloud removal methods are often used to reconstruct cloud-covered images. Cloud removal is essentially a process of reconstructing missing data in an image. In the process of reconstructing missing data, it is usually necessary to use the spectral or spatial information of the local spatial neighborhood or local temporal neighborhood image to search for similar pixels to fill in the missing area. Currently, the methods widely used for declouding long-term remote sensing images are mainly time domain analysis, spatial interpolation, spatiotemporal interpolation, and machine learning (deep learning) methods.

[0004] However, when using the time domain analysis method, when encountering long-term continuous data missing (such as cloud cover lasting for weeks or months), the reconstruction effect is significantly reduced; it mainly focuses on the information of the time dimension, and the spatial information is insufficiently utilized; for long time series and large areas, the computational burden of pixel-by-pixel time domain analysis is very heavy; when using the spatial interpolation method, when encountering large-area data missing (such as large-scale cloud cover), the reconstruction effect is significantly reduced; at the boundary of the missing area, the reconstruction result often has obvious discontinuity or blurring effect; some advanced spatial interpolation methods (such as Kriging) have high computational complexity and low efficiency when processing large-scale remote sensing images. The efficiency is low; when using the spatiotemporal interpolation method, it is usually necessary to process a large amount of time and space data at the same time, which has high computational complexity and low efficiency; the spatiotemporal interpolation model usually involves multiple parameters, such as time weight, spatial correlation range, etc. Improper parameter selection will significantly affect the reconstruction quality; when using machine learning and deep learning methods, the performance of machine learning and deep learning models is highly dependent on the quality and quantity of training data; the generalization ability is limited, and it may perform poorly in unseen data or environments, especially when the distribution of test data is very different from that of training data; training complex machine learning or deep learning models usually requires a lot of computing resources.

[0005] In view of this, an optical remote sensing image processing method and device are provided to at least partially solve the problems existing in the prior art of poor cloud removal effect and long processing cycle for remote sensing images of long-time series daily remote sensing products. Summary of the invention

[0006] To this end, the embodiments of the present invention provide an optical remote sensing image processing method and device, in order to make full use of time domain and spatial domain information, combine computer vision technology, and have efficient parallel computing capabilities, so as to solve the problems of poor cloud removal effect and long processing cycle for remote sensing images of long-time series daily remote sensing products in the prior art, and then solve the problem of spatiotemporal discontinuity caused by cloud pollution, and improve the availability and analysis accuracy of remote sensing data.

[0007] In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:

[0008] The present invention provides an optical remote sensing image processing method, the method comprising:

[0009] Acquire remote sensing images and construct remote sensing image sets for different usage scenarios based on remote sensing images;

[0010] Selecting a target image set from the remote sensing image set, and performing cloud removal processing on each remote sensing image in the target image set;

[0011] Among them, the declouding processing includes time domain declouding, spatial domain declouding and edge detection and filling; in the time domain declouding process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the target image set is declouded in the time domain based on the selection time limit.

[0012] In some embodiments, remote sensing image sets for different usage scenarios are constructed based on remote sensing images, which also includes:

[0013] Preprocess the acquired remote sensing images.

[0014] In some embodiments, performing time domain cloud removal on the target image set based on the selected time limit specifically includes:

[0015] S31: Determine any date in the target image set as a target date;

[0016] S32: If the target pixel of the target day is a non-cloud pixel N days before and after, then the average of the N days before and after is assigned to the target pixel;

[0017] S33: If there is only one non-cloud pixel in the N days before and after, the pixel value is directly used to assign the target pixel;

[0018] If the N days before and after are all cloud pixels, the N days before and after are respectively used as the target day and steps S31-S33 are repeated until the target day and the N days before and after are all cloud pixels, and the iteration is stopped;

[0019] Wherein, N is a positive integer greater than or equal to 1.

[0020] In some embodiments, performing spatial cloud removal on the target image set specifically includes:

[0021] The vacant pixels in the target image set are used as the central reference pixels;

[0022] Preliminarily determine similar pixels of the missing pixels from the reference image according to a search window of a preset size, and calculate the spatial weight of each similar pixel;

[0023] The predicted values ​​of the missing pixels are obtained and filled, and multiple iterations are performed until the proportion of cloud pollution in the image decreases to less than the preset value.

[0024] In some embodiments, edge detection and filling are performed on the target image set, specifically including:

[0025] Use edge detection algorithms to identify the edges and texture features of cloud and non-cloud areas;

[0026] Edge recognition is achieved through gradient calculation and dual threshold edge detection;

[0027] The cloud area is filled using the mean of neighboring pixel information, and the edges are filled using the texture features of the cloud-covered area and the non-cloud area.

[0028] In some embodiments, when dual threshold edge detection is used to implement edge recognition, if the edge detection operator returns the horizontal gradient G x and the vertical gradient G y , then the expressions of the gradient amplitude G and direction angle value θ are:

[0029]

[0030] θ=atan2(G x ,G y )

[0031] Here, atan2(·) represents the arctan function with two parameters, and the direction of the gradient is always perpendicular to the edge.

[0032] In some embodiments, a target image set is selected from the remote sensing image set, and cloud removal is performed on the target image set, before which the method further includes:

[0033] Composite multi-temporal images within the same day.

[0034] The present invention also provides an optical remote sensing image processing device, the device comprising:

[0035] A data acquisition unit, used for acquiring remote sensing images and constructing remote sensing image sets in different usage scenarios based on the remote sensing images;

[0036] A cloud removal processing unit is used to select a target image set from the remote sensing image set and perform cloud removal processing on each remote sensing image in the target image set;

[0037] Among them, the declouding processing includes time domain declouding, spatial domain declouding and edge detection and filling; in the time domain declouding process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the target image set is declouded in the time domain based on the selection time limit.

[0038] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0040] The optical remote sensing image processing method provided by the present invention obtains remote sensing images and constructs remote sensing image sets under different usage scenarios based on the remote sensing images; selects a target image set from the remote sensing image set, and performs cloud removal processing on each remote sensing image in the target image set; wherein the cloud removal processing includes time domain cloud removal, spatial domain cloud removal, and edge detection and filling; in the process of time domain cloud removal, the selection time limit of time domain information is determined according to the usage scenario of the target image set, and the time domain cloud removal is performed on the target image set based on the selection time limit.

[0041] In this way, the method and device provided by the present invention combine the time domain-spatial domain two-stage cloud removal and edge detection and filling, use the edge detection algorithm to identify the edge of the cloud and non-cloud area and the image texture features, avoid the fuzziness and discontinuity caused by directly using spatial information to fill the cloud pixels; reconstruct the cloud pollution pixels through the spatial information and texture structure of the image itself, compared with the time domain interpolation method and the time-space interpolation method, greatly reduce the calculation complexity and improve the calculation efficiency; through the parallel calculation of the time domain-spatial domain two-stage algorithm, to a certain extent, fully use the time information and space information, and improve the calculation efficiency. Thus, the time domain and space domain information are fully utilized, combined with computer vision technology, and at the same time have efficient parallel computing capabilities, solve the problems of poor cloud removal effect and long processing cycle for remote sensing images of long-time series daily remote sensing products in the prior art, and then solve the time-space discontinuity problem caused by cloud pollution, and improve the availability and analysis accuracy of remote sensing data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0043] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0044] Figure 1 One of the flow charts of the optical remote sensing image processing method provided by the present invention;

[0045] Figure 2 This is the second flow chart of the optical remote sensing image processing method provided by the present invention;

[0046] Figure 3 A schematic diagram of a time-domain cloud removal algorithm for daily remote sensing products in a specific usage scenario of the present invention;

[0047] Figure 4 The third flowchart of the optical remote sensing image processing method provided by the present invention;

[0048] Figure 5 A schematic diagram of a spatial cloud removal algorithm based on inverse distance weighting in a specific usage scenario of the present invention;

[0049] Figure 6 This is a fourth flow chart of the optical remote sensing image processing method provided by the present invention;

[0050] Figure 7 A schematic diagram of a cloud removal algorithm based on Canny edge detection and filling technology in a specific usage scenario of the present invention;

[0051] Figure 8 This is a structural block diagram of a computer device provided by the present invention. DETAILED DESCRIPTION

[0052] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In view of the limitations of existing technologies and the urgent need for high-quality long-term remote sensing data, there is an urgent need for a method to quickly and efficiently remove clouds from long-term daily remote sensing products. This method should be able to make full use of temporal and spatial information, combined with computer vision technology, and have efficient parallel computing capabilities to solve the problem of spatiotemporal discontinuity caused by cloud pollution and improve the availability and analysis accuracy of remote sensing data. In response to the above needs, the present invention proposes an optical remote sensing image processing method that can quickly remove clouds from long-term daily remote sensing products.

[0054] Please refer to Figure 1 , Figure 1 This is one of the flow charts of the optical remote sensing image processing method provided by the present invention.

[0055] In a specific embodiment, the optical remote sensing image processing method provided by the present invention comprises the following steps:

[0056] S110: Acquire remote sensing images, and construct remote sensing image sets for different usage scenarios based on the remote sensing images; that is, the image processing method is applicable to the remote sensing image processing process of various usage scenarios, and different remote sensing image sets are constructed for different usage scenarios. These remote sensing image sets correspond to different remote sensing thematic products, and the remote sensing thematic products include but are not limited to the National Natural Disaster Relief Index (NDVI), the Snow Cover Index (NDSI) and the Water Body Index (NDWI), etc. The image processing method is applicable to the declouding processing of various long-term daily remote sensing thematic products, that is, to reconstruct the information of cloud-contaminated pixels.

[0057] S120: Select a target image set from the remote sensing image set, and perform cloud removal processing on each remote sensing image in the target image set; that is, perform cloud removal processing on each image in the remote sensing product, and the constituent unit of the long-term daily remote sensing product is the remote sensing image of each day. The cloud removal processing includes time domain cloud removal, spatial domain cloud removal, and edge detection and filling; in the time domain cloud removal process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the time domain cloud removal is performed on the target image set based on the selection time limit. It should be understood that in the actual processing of the above-mentioned time domain cloud removal, spatial domain cloud removal, and edge detection and filling, firstly, the image of the large-scale study area is sliced ​​to divide it into several small areas, and then several small area images are processed at the same time, that is, parallel calculation, to improve calculation efficiency. It should be understood that time domain declouding, spatial domain declouding, and edge detection and filling are performed sequentially, that is, time domain declouding is performed first, spatial domain declouding is performed on the image after time domain declouding, and then edge detection and filling are performed on the image after time-spatial domain second-order declouding. The parallel computing time-spatial domain two-stage algorithm fully utilizes time information and spatial information to a certain extent, and improves computing efficiency.

[0058] In a specific usage scenario, such as Figure 2 As shown, the optical remote sensing image processing method provided by the present invention comprises the following steps:

[0059] S210: Acquisition, preprocessing and preparation of thematic products of remote sensing images;

[0060] S220: Multi-temporal image synthesis within the same day;

[0061] S230: Declouding of long-term daily remote sensing products in the temporal domain. According to different products, temporal information of different time limits is selected to reconstruct the pixel information under the cloud. Iterative calculation is performed according to this strategy to reduce the cloud ratio in the image to a certain threshold (ΔR cloud <2%);

[0062] S240: Spatial cloud removal of long-term daily remote sensing products: Spatial filling of products after temporal interpolation is performed, cloud-contaminated pixels are defined as central reference pixels, and the inverse distance weighted interpolation method is used to reconstruct the pixels under the cloud according to the preset search window size. This strategy is iterated to reduce the cloud ratio in the image to a certain threshold (ΔR cloud <1%);

[0063] S250: Cloud removal based on Canny edge detection and filling technology: Due to the limited available time domain and spatial domain information, it is impossible to completely fill the gaps in the image according to the above steps. Computer vision technology is introduced to perform Canny edge detection and filling on the image, thereby achieving image gap filling.

[0064] The above steps S230-S250 can all be calculated in parallel in the spatial dimension, thereby improving the calculation efficiency and solving the problem of spatiotemporal discontinuity of long-time series daily remote sensing products caused by cloud pollution.

[0065] In the above step S110, a remote sensing image set in different usage scenarios is constructed based on the remote sensing image, which also includes the following steps:

[0066] Preprocess the acquired remote sensing images. That is, after acquiring the remote sensing image data, preprocess the remote sensing image data and prepare remote sensing thematic products; specifically, perform radiometric calibration, atmospheric correction, geometric correction and cropping on the remote sensing images, mark cloud pixels on the remote sensing images, and prepare thematic remote sensing products.

[0067] In a specific usage scenario, long-term remote sensing data is first obtained, that is, MODIS reflectance data, and then the remote sensing image is calibrated by radiation, atmospheric correction, geometric correction and cropping, and used as input data for thematic remote sensing products, such as vegetation index (NDVI), snow cover index (NDSI) and water body index (NDWI).

[0068] In some embodiments, a target image set is selected from the remote sensing image set, and cloud removal is performed on the target image set, before which the method further includes:

[0069] The synthesis of multi-phase images within the same day is to synthesize multi-phase images within the same day. In principle, for specific remote sensing data, it contains multiple phases of data within a day. For long-term daily remote sensing products, cloud pollution pixels can be reconstructed by synthesizing multi-phase images within the same day.

[0070] Still taking the above specific usage scenario as an example, MODIS reflectivity data is divided into Terra satellite data and Aqua satellite data. Therefore, the thematic products are also divided into two phases of data products, namely the morning star Terra and the afternoon star Auqa, within a day. The Aqua product is used to interpolate the cloud pollution pixels in the Terra product.

[0071] In the above step S120, the target image set is subjected to time domain cloud removal based on the selected time limit, specifically comprising the following steps:

[0072] S31: Determine any date in the target image set as a target date;

[0073] S32: If the target pixel of the target day is a non-cloud pixel N days before and after, then the average of the N days before and after is assigned to the target pixel;

[0074] S33: If there is only one non-cloud pixel in the N days before and after, the pixel value is directly used to assign the target pixel;

[0075] If the N days before and after are all cloud pixels, the N days before and after are respectively used as the target day and steps S31-S33 are repeated until the target day and the N days before and after are all cloud pixels, and the iteration is stopped;

[0076] Wherein, N is a positive integer greater than or equal to 1.

[0077] It should be understood that the time domain analysis method is used to reconstruct cloud-contaminated pixels of remote sensing products, that is, to remove clouds. For remote sensing products of different themes, time domain information of different time periods is used to remove clouds. For example, for thematic products with small changes in a short period of time, such as vegetation index and water index, the images of the recent day in a longer period of time (2 days, 3 days, ...) are selected to remove clouds from the current image; for thematic products with large changes in a short period of time, such as snow cover index, the images of the recent day in a shorter period of time (1 day or 2 days) are selected to remove clouds from the current image.

[0078] Specifically, the algorithm rules for time domain cloud removal are as follows, taking the interpolation of the previous and next two days as an example: if the target pixel is a non-cloud pixel on both the previous and next days, the average value of the previous and next days is assigned to the target pixel; if only one of the previous and next days is a non-cloud pixel, the pixel value is directly used to assign the target pixel. If both the previous and next days are cloud pixels, it is necessary to find the pixels of the previous and next two days to continue the judgment, and the algorithm rules are the same as above; if the previous and next two days are also cloud pixels, the target pixel is still a cloud pixel, the calculation is completed, and wait for the next calculation.

[0079] It should be understood that the above explanation is based on the selection of a time limit of one day before and after, or two days before and after. In actual processing, the time limit can be extended to 3 days, 4 days, etc. before and after according to different usage scenarios, but it should not be too long to avoid introducing errors.

[0080] In this embodiment, if Figure 3 As shown in the figure, the above operations are based on daily images, with the help of the powerful data processing function of the NumPy library where() function, and parallel computing to process multiple days of data at the same time to achieve fast processing. According to this strategy, multiple iterations are performed until the image cloud pollution decreases by less than 2%, that is, ΔR cloud <2%.

[0081] In the above step S120, if Figure 4 As shown, the target image set is declouded in the spatial domain, which specifically includes the following steps:

[0082] S410: Using the vacant pixels in the target image set as the central reference pixels;

[0083] S420: Preliminarily determine similar pixels of the missing pixel from the reference image according to a search window of a preset size, and calculate the spatial weight of each similar pixel;

[0084] S430: Obtain predicted values ​​of the missing pixels and fill them, iterating multiple times until the image cloud pollution reduction ratio is less than a preset value.

[0085] Theoretically, inverse distance weighting is a common spatial interpolation method used to predict the value of an unknown point. The basic idea is that the closer the known point is to the predicted point, the greater its influence on the predicted value. The calculation process is:

[0086] 1) Calculate the distance from the unknown point to all points:

[0087]

[0088] 2) Calculate weights:

[0089]

[0090] 3) Calculate the predicted value of the interpolation point:

[0091]

[0092] Using the spatial information contained in the remote sensing image, the cloud pollution pixels are reconstructed by inverse distance weighted interpolation. The closer the known point is to the cloud pollution point, the greater the impact on the predicted value is, so as to generate the predicted value of the cloud pollution point. The vacant pixel is defined as the central reference pixel, and the similar pixels of the missing pixel are preliminarily determined from the reference image according to the preset search window size. The spatial weight of each similar pixel is calculated, and the predicted value of the vacant pixel is obtained and filled. According to this strategy, multiple iterations are performed until the image cloud pollution decreases by less than 1%, that is, ΔR cloud <1%. Specifically, in this embodiment, spatial interpolation is calculated in a slice-parallel manner to improve computational efficiency. Figure 5Schematic diagram of the spatial cloud removal algorithm based on inverse distance weighting.

[0093] In step S120, Figure 6 As shown, edge detection and filling are performed on the target image set, specifically including the following steps:

[0094] S610: using an edge detection algorithm to identify edges and texture features of cloud and non-cloud areas;

[0095] S620: Implement edge recognition through gradient calculation and dual threshold edge detection;

[0096] S630: Fill the cloud area using the mean of the neighboring pixel information, and fill the edge by combining the texture features of the cloud-covered area and the non-cloud area.

[0097] In a specific usage scenario, such as Figure 7 As shown in the figure, the Canny edge detection and cloud pollution pixel interpolation of remote sensing image data specifically include:

[0098] Canny edge detection: Use the Canny algorithm to identify the edges and texture features of cloud and non-cloud areas to ensure edge accuracy. Canny edge detection achieves accurate edge recognition through gradient calculation and dual threshold edge detection. The edge detection operator returns the horizontal gradient G x and the vertical gradient G y , then the magnitude G(x, y) and direction θ (angle value) of the gradient are:

[0099]

[0100] θ=atan2(G x ,G y )

[0101] Here, atan2(·) represents the arctan function with two parameters, and the direction of the gradient is always perpendicular to the edge.

[0102] Image filling: Use the mean of neighboring pixel information to fill the cloud area, and combine the texture features of the cloud-covered area and the non-cloud area to refine the filling edge to avoid the blur and discontinuity problems caused by direct filling. The formula is:

[0103]

[0104] Among them, P fill is the value of the pixel to be filled, P i This method restores the cloud-covered area by integrating the information of surrounding pixels to ensure that the filled pixel value is closer to the true value.

[0105] In this embodiment, the Canny algorithm is used to identify the edges of clouds and non-cloud areas and image texture features, avoiding the blur and discontinuity problems caused by directly using spatial information to fill cloud pixels; cloud-contaminated pixels are reconstructed through the spatial information and texture structure of the image itself, which greatly reduces the computational complexity and improves the computational efficiency compared with the time domain interpolation method and the space-time interpolation method.

[0106] In the above specific implementation manner, the optical remote sensing image processing method provided by the present invention obtains remote sensing images and constructs remote sensing image sets under different usage scenarios based on the remote sensing images; selects a target image set from the remote sensing image set, and performs cloud removal processing on each remote sensing image in the target image set; wherein the cloud removal processing includes time domain cloud removal, spatial domain cloud removal and edge detection and filling; in the time domain cloud removal process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the time domain cloud removal is performed on the target image set based on the selection time limit.

[0107] In this way, the method provided by the present invention combines the time-domain-spatial domain two-stage cloud removal and edge detection and filling, uses the edge detection algorithm to identify the edges of the cloud and non-cloud areas and the image texture features, and avoids the fuzziness and discontinuity problems caused by directly using spatial information to fill cloud pixels; reconstructs cloud-contaminated pixels through the spatial information and texture structure of the image itself, greatly reducing the computational complexity and improving the computational efficiency compared with the time-domain interpolation method and the time-space interpolation method; through the parallel computing time-domain-spatial domain two-stage algorithm, to a certain extent, fully utilizes the time information and space information, and improves the computational efficiency. Thus, the time-domain and space-domain information are fully utilized, combined with computer vision technology, and at the same time, it has efficient parallel computing capabilities, which solves the problems of poor cloud removal effect and long processing cycle for remote sensing images of long-time series daily remote sensing products in the prior art, and further solves the time-space discontinuity problem caused by cloud pollution, and improves the availability and analysis accuracy of remote sensing data.

[0108] In addition to the above method, the present invention also provides an optical remote sensing image processing device, the device comprising:

[0109] A data acquisition unit, used for acquiring remote sensing images and constructing remote sensing image sets in different usage scenarios based on the remote sensing images;

[0110] A cloud removal processing unit is used to select a target image set from the remote sensing image set and perform cloud removal processing on each remote sensing image in the target image set;

[0111] Among them, the declouding processing includes time domain declouding, spatial domain declouding and edge detection and filling; in the time domain declouding process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the target image set is declouded in the time domain based on the selection time limit.

[0112] In some embodiments, remote sensing image sets for different usage scenarios are constructed based on remote sensing images, which also includes:

[0113] Preprocess the acquired remote sensing images.

[0114] In some embodiments, performing time domain cloud removal on the target image set based on the selected time limit specifically includes:

[0115] S31: Determine any date in the target image set as a target date;

[0116] S32: If the days before and after the target pixel of the target day are both non-cloud pixels, the average value of the days before and after is assigned to the target pixel;

[0117] S33: If there is only one non-cloud pixel on the day before or after, the pixel value is directly assigned to the target pixel;

[0118] If both the preceding and following days are cloud pixels, the two preceding and following days are respectively taken as the target day and steps S31-S33 are repeated until the target day and the two preceding and following days are all cloud pixels and the iteration is stopped.

[0119] In some embodiments, performing spatial cloud removal on the target image set specifically includes:

[0120] The vacant pixels in the target image set are used as the central reference pixels;

[0121] Preliminarily determine similar pixels of the missing pixels from the reference image according to a search window of a preset size, and calculate the spatial weight of each similar pixel;

[0122] The predicted values ​​of the missing pixels are obtained and filled, and multiple iterations are performed until the proportion of cloud pollution in the image decreases to less than the preset value.

[0123] In some embodiments, edge detection and filling are performed on the target image set, specifically including:

[0124] Use edge detection algorithms to identify the edges and texture features of cloud and non-cloud areas;

[0125] Edge recognition is achieved through gradient calculation and dual threshold edge detection;

[0126] The cloud area is filled using the mean of neighboring pixel information, and the edges are filled using the texture features of the cloud-covered area and the non-cloud area.

[0127] In some embodiments, when dual threshold edge detection is used to implement edge recognition, if the edge detection operator returns the horizontal gradient G x and the vertical gradient G y , then the expressions of the gradient amplitude G and direction angle value θ are:

[0128]

[0129] θ=atam2(G x ,G y )

[0130] Here, atan2(·) represents the arctan function with two parameters, and the direction of the gradient is always perpendicular to the edge.

[0131] In some embodiments, a target image set is selected from the remote sensing image set, and cloud removal is performed on the target image set, before which the method further includes:

[0132] Composite multi-temporal images within the same day.

[0133] In the above-mentioned specific implementation manner, the optical remote sensing image processing device provided by the present invention acquires remote sensing images and constructs remote sensing image sets under different usage scenarios based on the remote sensing images; selects a target image set from the remote sensing image set, and performs cloud removal processing on each remote sensing image in the target image set; wherein the cloud removal processing includes time domain cloud removal, spatial domain cloud removal, and edge detection and filling; in the process of time domain cloud removal, the selection time limit of time domain information is determined according to the usage scenario of the target image set, and the target image set is subjected to time domain cloud removal based on the selection time limit.

[0134] In this way, the device provided by the present invention combines the time domain-spatial domain two-stage cloud removal and edge detection and filling, uses the edge detection algorithm to identify the edges of the cloud and non-cloud areas and the image texture features, and avoids the fuzziness and discontinuity problems caused by directly using spatial information to fill cloud pixels; reconstructs cloud-contaminated pixels through the spatial information and texture structure of the image itself, greatly reducing the computational complexity and improving the computational efficiency compared with the time domain interpolation method and the time-space interpolation method; through the parallel computing time domain-spatial domain two-stage algorithm, to a certain extent, fully utilizes the time information and space information, and improves the computational efficiency. Thus, the time domain and space domain information are fully utilized, combined with computer vision technology, and at the same time, it has efficient parallel computing capabilities, which solves the problems of poor cloud removal effect and long processing cycle for remote sensing images of long-time series daily remote sensing products in the prior art, and further solves the time-space discontinuity problem caused by cloud pollution, and improves the availability and analysis accuracy of remote sensing data.

[0135] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiment are implemented.

[0136] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0137] Corresponding to the above embodiment, the embodiment of the present invention further provides a computer storage medium, which contains one or more program instructions, wherein the one or more program instructions are used to execute the above method.

[0138] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the above method.

[0139] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0140] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0141] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0142] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0143] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0144] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0145] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0146] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An optical remote sensing image processing method, characterized in that: The method comprises: Acquire remote sensing images and construct remote sensing image sets for different usage scenarios based on remote sensing images; Selecting a target image set from the remote sensing image set, and performing cloud removal processing on each remote sensing image in the target image set; Among them, the declouding processing includes time domain declouding, spatial domain declouding and edge detection and filling; in the time domain declouding process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the target image set is declouded in the time domain based on the selection time limit.

2. The optical remote sensing image processing method according to claim 1, characterized in that: Based on remote sensing images, remote sensing image sets for different usage scenarios were built, including: Preprocess the acquired remote sensing images.

3. The optical remote sensing image processing method according to claim 1, characterized in that: Based on the selected time limit, the target image set is subjected to time-domain cloud removal, specifically comprising: In the target image set, determining any date as a target date; If the target pixel of the target day is a non-cloud pixel N days before and after, the average value of the N days before and after is assigned to the target pixel; If there is only one non-cloud pixel in the N days before and after, the pixel value is directly used to assign the target pixel; If the N days before and after are all cloud pixels, the N days before and after are respectively used as the target day and steps S31-S33 are repeated until the target day and the N days before and after are all cloud pixels, and the iteration is stopped; Wherein, N is a positive integer greater than or equal to 1.

4. The optical remote sensing image processing method according to claim 1, characterized in that: Perform spatial cloud removal on the target image set, including: The vacant pixels in the target image set are used as the central reference pixels; Preliminarily determine similar pixels of the missing pixels from the reference image according to a search window of a preset size, and calculate the spatial weight of each similar pixel; The predicted values ​​of the missing pixels are obtained and filled, and multiple iterations are performed until the proportion of cloud pollution in the image decreases to less than the preset value.

5. The optical remote sensing image processing method according to claim 1, characterized in that: Perform edge detection and filling on the target image set, including: Use edge detection algorithms to identify the edges and texture features of cloud and non-cloud areas; Edge recognition is achieved through gradient calculation and dual threshold edge detection; The cloud area is filled using the mean of neighboring pixel information, and the edges are filled using the texture features of the cloud-covered area and the non-cloud area.

6. The optical remote sensing image processing method according to claim 5, characterized in that: When dual threshold edge detection is used to implement edge recognition, if the edge detection operator returns the horizontal gradient G x and the vertical gradient G y , then the expressions of the gradient amplitude G and direction angle value θ are: θ=atan2(G x ,G y ) Here, atan2(·) represents the arctan function with two parameters, and the direction of the gradient is always perpendicular to the edge.

7. The optical remote sensing image processing method according to claim 1, characterized in that: Selecting a target image set from the remote sensing image set and performing cloud removal processing on the target image set, the method further includes: Composite multi-temporal images within the same day.

8. An optical remote sensing image processing device, characterized in that: The device comprises: A data acquisition unit, used for acquiring remote sensing images and constructing remote sensing image sets in different usage scenarios based on the remote sensing images; A cloud removal processing unit is used to select a target image set from the remote sensing image set and perform cloud removal processing on each remote sensing image in the target image set; Among them, the declouding processing includes time domain declouding, spatial domain declouding and edge detection and filling; in the time domain declouding process, the selection time limit of the time domain information is determined according to the usage scenario of the target image set, and the target image set is declouded in the time domain based on the selection time limit.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images