An automatic monitoring method for blur degradation of high-resolution optical remote sensing images

Abstract

The present invention relates to an automatic monitoring method for blur degradation of high-resolution optical remote sensing images, which belongs to the field of aerospace technology. First, a training sample database is prepared; a classification network is constructed and trained; the remote sensing images to be monitored are cut into blocks to obtain a set of images to be screened; the set of images to be screened is screened to obtain a primary screening result; the primary screening result is secondary screened using the trained classification network; the blur kernel is measured on the image slices after the secondary screening using the ISD algorithm; the energy concentration is calculated; the energy concentration is used as the image clarity, and the energy concentration is saved in the monitoring database for statistical analysis of automatic monitoring of blur degradation of high-resolution optical remote sensing images. The present invention can automatically realize high-precision image blur kernel and energy concentration measurement, greatly improve the reliability of the results, meet the image quality monitoring task requirements of a large number of satellites, have high precision, wide coverage, stable measurement results, and are easy to implement.

Description

Technical Field

[0001] The invention belongs to the field of aerospace technology, and in particular relates to an automatic monitoring method for blur degradation of high-resolution optical remote sensing images. Background Art

[0002] During the operation of optical remote sensing satellites in orbit, the images are blurred and degraded to varying degrees due to factors such as changes in the space environment and decreased camera stability. Therefore, in order to ensure the normal on-orbit status of satellite processing and maintain normal image quality, it is necessary to continuously monitor the image quality of the images taken by the on-orbit satellites.

[0003] The invention patent with the publication number CN101281250A discloses a method for monitoring the modulation transfer function of an on-orbit satellite remote sensor based on image elements. The method monitors the modulation transfer function (MTF) of the on-orbit satellite remote sensor based on image elements, and uses the MTF change of the time series to estimate the image period change and make corresponding compensation. However, the method can only measure the image data of a specific target, so the acquisition period of the monitoring result is relatively long due to the shooting conditions and the transit period. The invention patent with the publication number CN114399688A discloses a method and device for selecting the edge area of ​​a remote sensing image. By establishing a typical target library, the region of interest containing the edge is automatically extracted, and then the MTF value is measured after the edge is extracted. Although the proportion of usable images is improved and the monitoring period is shortened, the accuracy of the results is greatly dependent on the accuracy of regional screening and edge extraction. While the above method realizes the automation of remote sensing image quality monitoring, there are still some problems. First, since the images included in the monitoring are still a small number of images that have been screened or have specific features, they fail to cover all on-orbit shooting tasks, resulting in a relatively long monitoring period. Secondly, when the monitoring indicator is the MTF value at the edge of the blade extracted automatically, the position precision and accuracy of the edge extraction have a greater impact on the result, and a large number of measurement errors lead to large fluctuations in the monitoring results. Summary of the invention

[0004] Aiming at the problems existing in the current automatic monitoring methods of image quality of optical remote sensing satellites on orbit, the present invention proposes an automatic monitoring method of blur degradation of high-resolution optical remote sensing images by combining a regional recommendation algorithm and degradation blur and estimation to meet the needs of automatic monitoring of image quality of optical remote sensing satellites on orbit, thereby realizing fully automatic blur degradation monitoring of all images taken by on-orbit satellites.

[0005] To solve the above problems, the present invention adopts the following technical solutions:

[0006] A method for automatically monitoring blur degradation of high-resolution optical remote sensing images, the method comprising the following steps:

[0007] Step 1: Create a training sample database, where the training sample database is divided into samples suitable for measurement and samples of other categories;

[0008] Step 2: Use the ViT method to build a classification network. First, divide the image into patches and flatten them. Then use the fully connected layer to perform dimension mapping and embed the position information. The main structure of the classification network is the Transformer Encoder encoder. The input of the embedded position information and category information is first processed by LayerNorm. Before entering the Multi-HeadAttention layer, it is transformed to obtain the attention weight and then reorganized into an encoder. Then, the classification category is output through the MLP Head layer.

[0009] Using the training sample database to train the classification network to obtain a trained classification network;

[0010] Step 3: Crop the remote sensing images to be monitored into blocks to obtain a set of images to be screened;

[0011] Step 4: Preliminary screening is performed on the image slices that do not meet the monitoring conditions in the image set to be screened to obtain a preliminary screening result;

[0012] Step 5: Use the trained classification network to perform secondary screening on the initial screening results;

[0013] Step 6: Use the ISD algorithm to measure the blur kernel of the image slices after secondary screening to obtain the blur kernel of each image slice;

[0014] Step 7: Calculate the energy concentration of the remote sensing image to be monitored based on the blur kernel of each image slice. The calculation formula is as follows:

[0015]

[0016] Among them, (p, q) is the coordinate of each pixel in the kernel center n×n, (i, j) is the coordinate of all pixels in the kernel, and D is the grayscale corresponding to each pixel;

[0017] Step 8: Using energy concentration as image clarity, save the energy concentration calculated in step 7 into the monitoring database for statistical analysis of automatic monitoring of blur degradation of high-resolution optical remote sensing images.

[0018] Beneficial effects of the present invention:

[0019] The high-resolution optical remote sensing image blur degradation automatic monitoring method proposed in the present invention can automatically achieve high-precision image blur kernel and energy concentration measurement, greatly improve the reliability of the results, and meet the image quality monitoring task requirements of large quantities of satellites. Compared with the classical image quality monitoring and measurement method, the method of the present invention automatically screens the areas suitable for measurement in the image through two-step screening, and measures the blur kernel and energy concentration. This method has high precision, wide coverage, stable measurement results, is easy to implement, and can be effectively applied to engineering practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the automatic monitoring method of blur degradation of high-resolution optical remote sensing images according to an embodiment of the present invention;

[0021] Figure 2 It is the structural diagram of the classification network;

[0022] Figure 3 The remote sensing image to be monitored and the image slices screened thereon;

[0023] Figure 4 It is the partially screened image slice and the corresponding blur kernel map;

[0024] Figure 5 is the comprehensive fuzzy kernel map;

[0025] Figure 6 This is the automatic measurement area map under different shooting dates;

[0026] Figure 7 The comprehensive blur kernel map under different shooting dates. DETAILED DESCRIPTION

[0027] The embodiment of the present invention provides a method for automatically monitoring blur degradation of high-resolution optical remote sensing images. Figure 1 As shown, the method specifically comprises the following steps:

[0028] Step 1: Create a training sample database.

[0029] The training sample database includes two types of images. One type is suitable for blur kernel measurement, i.e., suitable for measurement samples, whose texture features are dense linear edges and whose ground features are impermeable surfaces such as cities. Images other than this type are classified as unsuitable for blur kernel measurement, i.e., other types of samples. The data source used to make the training sample database in this step can be all images taken by in-orbit satellites.

[0030] Step 2: Build and train the classification network.

[0031] The present invention uses the VisionTransformer (ViT) method to build a classification network. First, the image is divided into patches and flattened. Then, the fully connected layer is used to perform dimensional mapping and embed position information. Figure 2 As shown in the figure, the main structure of the classification network is the Transformer Encoder encoder, and the network achieves better classification results by stacking multiple layers of encoders. The input of the embedded position information and category information is first processed by the LayerNorm (regularization layer), and then transformed before entering the Multi-HeadAttention layer (multi-attention base mechanism layer), and then reorganized into an encoder after obtaining the attention weight, and then outputs the classification category through the MLP Head layer (multi-layer perceptron).

[0032] After training the classification network using the training sample database, a trained classification network is obtained. The network gives an evaluation score of the image category for the input image, and the option with a high evaluation score is the prediction result.

[0033] Step 3: Image segmentation.

[0034] The remote sensing images to be monitored are divided into blocks and cropped to obtain the image set to be screened.

[0035] Step 4: Initial screening.

[0036] Through the initial screening, the image slices that do not meet the monitoring conditions in the image set to be screened are screened out. The images with relatively simple textures, relatively uniform ground objects and less information are mainly screened out.

[0037] The initial screening process includes the following steps:

[0038] Step 4.1: First, perform Gaussian blur on each image slice. Define the Gaussian convolution kernel, make a discrete approximation of the Gaussian function, and calculate the weighted average of the pixels around the pixel by selecting the appropriate convolution kernel size and strength to obtain the final filtering result.

[0039] Step 4.2, then calculate the edge gradient strength. Calculate the Sobel gradient G in the horizontal and vertical directions respectively x , G y , the calculation formula is as follows:

[0040]

[0041] Among them, (x, y) is the pixel position, f(x, y) is the corresponding image value, and A x and A y are the filtering operators in the horizontal and vertical directions, and their values ​​are Then the edge gradient strength G of each pixel is calculated, and the calculation formula is as follows:

[0042]

[0043] Then, an erosion operation is performed on each image slice to obtain the edge gradient intensity map of the corresponding image slice. Specifically, for the input image I, the edge gradient intensity map O(x, y) is:

[0044]

[0045] Among them, S is the structural element, O(x,y) is each pixel point of the output image, and I(x+i,y+j) is the pixel value of the input image centered at (x+i,y+j).

[0046] Step 4.3: Then calculate the sum of all pixel values ​​of the edge gradient intensity map O(x, y) as the edge intensity EI of the image slice.

[0047] Step 4.4: Set the threshold T_EI, screen out the image slices whose edge strength EI is less than the threshold T_EI, and select the image slices whose edge strength EI is greater than the threshold T_EI for secondary screening.

[0048] Step 5: Secondary screening.

[0049] The trained classification network is used to conduct a secondary screening of the initial screening results.

[0050] Step 5.1: Input each image slice after the initial screening in step 4 into the classification network respectively, obtain the image category evaluation score corresponding to each image slice, further screen out the suitable measurement area, and count the number of image slices suitable for measurement N_EI.

[0051] Step 5.2: Set the threshold T_N1. If the number N_EI of image slices suitable for measurement is greater than the threshold T_N1, perform a subsequent secondary screening on the initial screening results. For image slices suitable for measurement, sort them from large to small according to the evaluation scores of their categories, and select the first M image slices for blur kernel measurement in the subsequent step 6. If the number N_EI is less than the threshold T_N1, it is determined that the remote sensing image to be monitored is not suitable for monitoring blur kernel degradation.

[0052] Step 6: Blur kernel measurement.

[0053] For the M image slices after two screenings, the ISD (iterative support detection) algorithm is used to measure the blur kernel. During the measurement, the initial kernel is roughly estimated first, and then the blur kernel of each image slice is finally obtained through iterative optimization estimation.

[0054] Step 7: Calculate the energy concentration.

[0055] The degree of nuclear energy concentration can reflect the clarity of the image. The energy concentration (EC) of the remote sensing image to be monitored is calculated by combining the blur kernel of each image slice. The calculation formula is as follows:

[0056]

[0057] Among them, (p, q) is the coordinate of each pixel in the kernel center n×n (for example, the kernel center size is 3×3), (i, j) is the coordinate of all pixels in the kernel, and D is the grayscale corresponding to each pixel.

[0058] Step 8: Using the calculated energy concentration as the image clarity, the energy concentration calculated in step 7 is saved in the monitoring database for statistical analysis of automatic monitoring of blur degradation of high-resolution optical remote sensing images. For example, time series monitoring of the energy concentration stored in the monitoring database is performed. When the energy concentration is lower than the threshold, it indicates that the image quality of the high-resolution optical remote sensing image taken by the in-orbit satellite is reduced and blur degradation exists.

[0059] The high-resolution optical remote sensing image blur degradation automatic monitoring method proposed in the present invention can automatically achieve high-precision image blur kernel and energy concentration measurement, greatly improve the reliability of the results, and meet the image quality monitoring task requirements of large quantities of satellites. Compared with the classical image quality monitoring and measurement method, the method of the present invention automatically screens the areas suitable for measurement in the image through two-step screening, and measures the blur kernel and energy concentration. This method has high precision, wide coverage, stable measurement results, is easy to implement, and can be effectively applied to engineering practice.

[0060] The technical solution of the present invention is described below in conjunction with specific examples.

[0061] 1. Dataset preparation.

[0062] The data source used to make the dataset is the panchromatic band image of the Jilin-1-GF03D series mass-produced satellite (Jilin1-GF03D) developed by Changguang Satellite Technology Co., Ltd., with a resolution better than 0.75 meters and a width better than 17 kilometers. Through manual identification and screening, samples of the categories of suitable measurement samples and other categories of samples with a size of 512*512 were obtained.

[0063] 2. Train the classification network.

[0064] Use the data set to train the classification network and obtain a trained classification network.

[0065] 3. Image cropping.

[0066] A single-scene image taken by the JL1GF03D14 satellite was selected for testing, and the image was cropped to a certain size to obtain the slices to be screened, forming an image set to be screened.

[0067] 4. Suitable for initial screening of image slices.

[0068] Each image slice is subjected to Gaussian blur, edge gradient intensity calculation, and edge erosion in turn to obtain an edge intensity map, the total image pixel value EI is calculated, and image slices with EI greater than the threshold are extracted for secondary screening.

[0069] 5. Secondary screening.

[0070] The image slices after the initial screening are input into the trained classification network to obtain the category evaluation score of each image slice, and the number of image slices suitable for measurement is counted. If the number of image slices suitable for measurement is greater than the threshold, subsequent measurement is performed, otherwise the scene image is determined to be unsuitable for measurement. The image slices suitable for measurement are sorted according to the category evaluation score, and the image slices with larger scores are selected for subsequent blur kernel and energy concentration measurement. The final screened image slices are as follows Figure 3 shown.

[0071] 6. Fuzzy kernel calculation.

[0072] Calculate the blur kernel of each image slice selected after the initial screening, where some of the screened image slices and the corresponding blur kernels are as follows Figure 4 shown.

[0073] 7. Energy concentration measurement and storage.

[0074] The comprehensive blur kernel of the remote sensing image to be monitored is obtained by integrating the blur kernels of all image slices. Figure 5 As shown in the figure, the 3×3 energy concentration is 0.9418. The energy concentration measurement results are saved in the monitoring database for subsequent statistical analysis of automatic monitoring of blur degradation of high-resolution optical remote sensing images.

[0075] 8. Timing monitoring.

[0076] The comprehensive fuzzy kernel is used to characterize the clarity of the entire image, forming a long-term monitoring record. The time series monitoring results recorded at the beginning of the month are shown in Table 1.

[0077] Table 1 Timing monitoring results

[0078] Date of shooting Automatic measurement area Comprehensive Fuzzy Kernel 3×3 Energy Concentration 2023-11-01 Figure 6 (a) Figure 7 (a) 0.9854 2024-12-01 Figure 6 (b) Figure 7 (b) 0.9835 2024-01-01 Figure 6 (c) Figure 7 (c) 0.9901 2024-02-01 Figure 6 (d) Figure 7 (d) 0.9832 2024-03-01 Figure 6 (e) Figure 7 (e) 0.9843

[0079] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for automatically monitoring blur degradation of high-resolution optical remote sensing images, characterized in that: The following steps are involved: Step 1: Create a training sample database, where the training sample database is divided into samples suitable for measurement and samples of other categories; Step 2: Use the ViT method to build a classification network. First, divide the image into patches and flatten them. Then use the fully connected layer to perform dimension mapping and embed the position information. The main structure of the classification network is the TransformerEncoder encoder. The input of the embedded position information and category information is first processed by LayerNorm. Before entering the Multi-HeadAttention layer, it is transformed to obtain the attention weight and then reorganized into an encoder. Then, the classification category is output through the MLP Head layer. Using the training sample database to train the classification network to obtain a trained classification network; Step 3: Crop the remote sensing images to be monitored into blocks to obtain a set of images to be screened; Step 4: Preliminary screening is performed on the image slices that do not meet the monitoring conditions in the image set to be screened to obtain a preliminary screening result; Step 5: Use the trained classification network to conduct a secondary screening of the initial screening results; Step 6: Use the ISD algorithm to measure the blur kernel of the image slices after secondary screening to obtain the blur kernel of each image slice; Step 7: Calculate the energy concentration of the remote sensing image to be monitored based on the blur kernel of each image slice. The calculation formula is as follows: Among them, (p, q) is the coordinate of each pixel in the kernel center n×n, (i, j) is the coordinate of all pixels in the kernel, and D is the grayscale corresponding to each pixel; Step 8: Using energy concentration as image clarity, save the energy concentration calculated in step 7 into the monitoring database for statistical analysis of automatic monitoring of blur degradation of high-resolution optical remote sensing images.

2. The method for automatically monitoring blur degradation of high-resolution optical remote sensing images according to claim 1, characterized in that: Step 5 includes the following steps: Step 5.1: Input the image slices that have been initially screened in step 4 into the trained classification network to obtain the category evaluation score of each image slice, and count the number of image slices suitable for measurement N_EI; Step 5.2: Set the threshold T_N1. If the number N_EI is greater than the threshold T_N1, perform a secondary screening on the primary screening results obtained in step 4. For image slices suitable for measurement, sort them from large to small according to the evaluation scores of their categories, and select the first M image slices to execute step 6. If the number N_EI is less than the threshold T_N1, it is determined that the remote sensing image to be monitored is not suitable for monitoring blur kernel degradation.

3. The method for automatically monitoring blur degradation of high-resolution optical remote sensing images according to claim 1, characterized in that: The size of the samples in the training sample database is 512*512.

4. The method for automatically monitoring blur degradation of high-resolution optical remote sensing images according to claim 1, characterized in that: When calculating the energy concentration, the kernel center size chosen is 3×3.

Images