Remote sensing image feature enhancement method for extreme scenarios

Abstract

This invention discloses a remote sensing image feature enhancement method for extreme scenarios, belonging to the field of image processing technology. The method includes: acquiring a first image set and a second image set; extracting a first remote sensing image from the first image set and multiple second remote sensing images from the second image set; performing a first resolution enhancement on the first remote sensing image to obtain a third remote sensing image; selecting one second remote sensing image as a base image and performing position registration on the base image to obtain the positional deviation between the base image and different second remote sensing images; performing a second resolution enhancement on the base image to obtain a first enhanced image; correcting the first enhanced image based on the remaining second remote sensing images to obtain a fourth remote sensing image; and analyzing the third and fourth remote sensing images to obtain a final detection image. This invention enables accurate analysis of the impact range of natural disasters in extreme scenarios.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for enhancing remote sensing image features for extreme scenarios. Background Technology

[0002] Remote sensing images, as an important means of acquiring information about the Earth's surface, are widely used in fields such as environmental monitoring, disaster assessment, urban planning, and military reconnaissance. However, in extreme scenarios, such as severe weather conditions (heavy rain, dense fog, sandstorms, etc.), complex terrain (mountainous areas, dense forests, urban areas with high-rise buildings), and low-light or nighttime environments, remote sensing images often exhibit significant quality degradation and difficulties in feature extraction.

[0003] To address the aforementioned issues, existing technologies have proposed methods such as the Chinese patent document CN117423007A, which discloses a multimodal target detection method and apparatus based on feature enhancement and collaborative interaction. This method acquires remote sensing images of at least two modalities, then inputs the remote sensing images into a remote sensing target detection model. The remote sensing target detection model is a deep learning model, which is used to enhance the features of the input remote sensing images of at least two modalities respectively, and to fuse the enhanced features to perform remote sensing target detection based on the fused features and the enhanced features.

[0004] However, deep learning-based feature enhancement methods typically rely on a large amount of labeled data for training. In extreme scenarios, it is difficult to obtain high-quality and diverse training samples, resulting in insufficient model generalization ability, which in turn affects the feature enhancement effect. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a remote sensing image feature enhancement method for extreme scenarios, thereby resolving the issues present in the background art.

[0006] To achieve the aforementioned objectives, this invention proposes a remote sensing image feature enhancement method for extreme scenarios, comprising:

[0007] Acquire a first image set and a second image set, wherein the first image set and the second image set are image sets constructed from remote sensing images of two imaging types, respectively;

[0008] Extract a single first remote sensing image from the first image set, and extract multiple second remote sensing images from the second image set;

[0009] The first remote sensing image is segmented into regions to obtain multiple first sub-images. Each first sub-image is enhanced with a first resolution to obtain a second sub-image. The second sub-images are then fused into a third remote sensing image.

[0010] Select one of the second remote sensing images as the base image, perform position registration on the base image, and obtain the positional deviation between the base image and different second remote sensing images;

[0011] The base image is enhanced by a second resolution to obtain a first enhanced image. The first enhanced image is then corrected based on the remaining second remote sensing images to obtain a fourth remote sensing image.

[0012] Boundary delineation is performed based on the third remote sensing image to obtain a boundary image. Target analysis is performed based on the fourth remote sensing image to obtain a target image. The boundary image and the target image are then fused to obtain the final detection image.

[0013] Further, obtaining the second sub-image includes the following steps:

[0014] The first remote sensing image is a radar image. Historical segmentation records of the first remote sensing image and historical enhancement records of resolution enhancement of the first sub-image are acquired. Based on the historical segmentation records, a first processing time for different segmentation sizes is determined. Based on the historical enhancement records, a second processing time for different resolution enhancement factors at different segmentation sizes is determined. A time distribution matrix is ​​constructed based on the first and second processing times. The time distribution matrix includes the comprehensive processing time of various combinations of segmentation sizes and enhancement factors. Based on the time distribution matrix, an optimal segmentation size and an optimal enhancement factor are determined. The first remote sensing image is segmented based on the optimal segmentation size to obtain the first sub-image. The resolution of each first sub-image is increased by the optimal enhancement factor to obtain the second sub-image.

[0015] Furthermore, fusing the second sub-image into the third remote sensing image includes the following steps:

[0016] There is a first overlapping region among the segmented first sub-images. In each second sub-image, a second overlapping region corresponding to the first overlapping region is located. A first region and a second region are divided in the second overlapping region. The second region is close to the center of the second sub-image. The second sub-images to be merged are defined as the first image and the second image, respectively. The second region of the second image is superimposed on the second region of the first image to complete the merging of the first image and the second image to obtain a merged image. The merged image is then merged with the remaining second sub-images in sequence to obtain the third remote sensing image.

[0017] Further, determining the optimal segmentation size and the optimal enhancement factor based on the time distribution matrix includes the following steps:

[0018] The weighted sum of the integrated processing time, resolution enhancement factor, and segmentation size is used to construct an evaluation function. The element in the time distribution matrix that maximizes the result of the evaluation function is obtained, and the segmentation size and enhancement factor corresponding to this element are taken as the optimal segmentation size and the optimal enhancement factor.

[0019] Furthermore, obtaining the positional deviation between the base image and different second remote sensing images includes the following steps:

[0020] A first cropping region is set in the base image, and the center and corresponding first coordinates of the first cropping region are determined. A second cropping region is generated in other second remote sensing images with the first coordinates as the center. The size of the second cropping region is larger than that of the first cropping region. The first cropping region is compared with each of the second cropping regions to determine a third cropping region in the second cropping region that corresponds to the first cropping region. The second coordinates of the center of the third cropping region are obtained. The position deviation is calculated based on the difference between the first coordinates and the second coordinates.

[0021] Furthermore, obtaining the fourth remote sensing image includes the following steps:

[0022] Determine the resolution upscaling factor, process the base image based on the upscaling factor to obtain the first upscaled image, and based on the upscaling factor and the positional deviation between the base image and each of the second remote sensing images, scale the first upscaled image into multiple first scaled images, and generate a first deviation image based on the first scaled image and the corresponding second remote sensing image.

[0023] The first biased image is magnified based on the magnification factor to obtain a second biased image. The pixel values ​​of the pixels in the first magnified image are corrected based on the second biased image to obtain a second magnified image. The second magnified image is used as the fourth remote sensing image.

[0024] Further, extracting multiple remote sensing images from the second image set includes the following steps:

[0025] The remote sensing image extracted from the second image set is defined as the image to be analyzed. The image to be analyzed is a visible light image. The pixels in the image to be analyzed are clustered based on the pixel value. Based on the clustering results, the pixels are divided into multiple categories. Each category is numbered and the number is assigned to the corresponding pixel in the image to be analyzed to generate a numbered image.

[0026] A sliding window is set up, which traverses the numbered images with a preset step size. After each movement, a first average value including the number and a second average value of the pixel value are obtained within the sliding window. If the first average value of the sliding window is the same at multiple adjacent positions, and the second average value is within a preset range, the area covered by the sliding window at the multiple adjacent positions is taken as the occlusion area. The occlusion rate is calculated based on the ratio of the occlusion area to the area of ​​the image to be analyzed. If the occlusion rate is greater than a critical threshold, the image to be analyzed is removed; otherwise, the image to be analyzed is retained as the second remote sensing image. This step is repeated until a preset number of second remote sensing images are collected.

[0027] The beneficial effects of this invention are as follows: by extracting a first remote sensing image from a first image set and multiple second remote sensing images with low occlusion rates from a second image set, and then using non-deep learning methods to perform resolution enhancement processing on the two images respectively, a third remote sensing image and a fourth remote sensing image are obtained, thereby achieving feature enhancement of the remote sensing images; then, the boundaries of ground features are delineated using the third remote sensing image, and the ground objects are more easily identified using the fourth remote sensing image; the boundary image and the target image obtained based on the two are fused to obtain a detection image; finally, by comparing the detection images before and after the disaster, the changes in the scope of the disaster's impact can be determined. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the steps of a remote sensing image feature enhancement method for extreme scenarios according to the present invention.

[0029] Figure 2 This is a schematic diagram illustrating the principle of fusing the second sub-image in this invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0031] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this application, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script.

[0032] like Figure 1 As shown, a remote sensing image feature enhancement method for extreme scenarios includes:

[0033] S1: Obtain the first image set and the second image set, which are image sets constructed from remote sensing images of two different imaging types.

[0034] The first and second image sets are pre-collected remote sensing images. The types of remote sensing images included in the first and second image sets are different. In this embodiment, the remote sensing images in the first image set are SAR images, which are synthetic aperture radar images. Even in the presence of interference factors such as clouds and fog, synthetic aperture radar imaging technology can still penetrate clouds to obtain ground images. However, because it relies on microwave reflection imaging, there may be situations where ground features are difficult to identify.

[0035] The remote sensing images in the second image set are optical images, that is, images directly formed by optical sensors. The advantage of this method is that the features of ground objects in the images are prominent and easy to identify. However, this method is easily affected by cloud cover and may result in image defects.

[0036] S2: Extract a single first remote sensing image from the first image set, and extract multiple second remote sensing images from the second image set.

[0037] S3: Perform region segmentation on the first remote sensing image to obtain multiple first sub-images, perform first resolution enhancement on each first sub-image to obtain a second sub-image, and fuse the second sub-images into a third remote sensing image.

[0038] Based on the above introduction, this invention combines two types of remote sensing images for analysis to achieve accurate detection of the impact of natural disasters under severe weather conditions. Specifically, the scope of the disaster's impact is determined by comparing the final detection images before and after the disaster. The methods for generating the final detection images before and after the disaster are the same; this embodiment uses the generation of the final detection image before the disaster as an example for explanation.

[0039] During generation, a first remote sensing image is first extracted from a first image set, and multiple second remote sensing images are extracted from a second image set. When extracting the second remote sensing images, an occlusion rate analysis is performed to ensure that the extracted second remote sensing images have a low occlusion rate, meeting the analysis requirements. Since the first and second remote sensing images are based on different imaging methods, this invention uses two different methods to enhance their resolution.

[0040] When enhancing the resolution of the first remote sensing image, the image is first segmented into regions to obtain multiple first sub-images. Then, resolution enhancement is performed on each first sub-image to obtain a second sub-image. Methods for enhancing the resolution of the first sub-images can include non-deep learning methods such as histogram equalization, linear filtering, and multi-scale image contrast enhancement. By segmenting the first remote sensing image before performing resolution enhancement, the computer can process the first sub-images in parallel, thereby improving image processing speed. Finally, the resolution-enhanced second sub-images are stitched together to obtain a third remote sensing image, which is essentially the resolution-enhanced version of the first remote sensing image.

[0041] S4: Select a second remote sensing image as the base image, perform position registration on the base image, and obtain the positional deviation between the base image and different second remote sensing images.

[0042] S5: Perform a second resolution enhancement on the base image to obtain a first enhanced image. Based on the remaining second remote sensing images, correct the first enhanced image to obtain a fourth remote sensing image.

[0043] When enhancing the resolution of the second remote sensing image, one second remote sensing image is first selected as the base image, preferably the second remote sensing image with the lowest occlusion rate. The other second remote sensing images are used to correct the base image after resolution enhancement. Since the extracted second remote sensing images cannot be taken at exactly the same location, it is necessary to determine the corresponding position of each pixel in the base image in the other second remote sensing images. Based on the above description, it is necessary to compare the base image with the remaining second remote sensing images to obtain the positional deviation between the base image and the second remote sensing images.

[0044] The base image is then enhanced to obtain the first enhanced image. Then, based on the calculated positional deviation, the pixel values ​​of the pixels in the first enhanced image are corrected using the remaining second remote sensing images. The corrected image is defined as the fourth remote sensing image. The specific process of enhancing the resolution of the base image will be described in detail later.

[0045] S6: Delineate the boundary based on the third remote sensing image to obtain the boundary image, perform target analysis based on the fourth remote sensing image to obtain the target image, and fuse the boundary image and the target image to obtain the final detection image.

[0046] The third type of remote sensing image is a SAR image. Different terrains produce different reflection intensities, so the third type of remote sensing image can be used to accurately delineate the boundaries of ground features. For example, by interpreting the third type of remote sensing image, areas containing water bodies can be identified, and the boundaries of water bodies can be determined. The fourth type of remote sensing image is an optical image, which makes it easier to identify ground objects.

[0047] For example, if there is rice paddy on the ground and some of the rice paddy is submerged by flood, the third remote sensing image can be used to determine which areas have water accumulation and define them as water accumulation areas. Then, the water accumulation areas are mapped onto the fourth remote sensing image. A sliding window is then set in the water accumulation area, and the distribution characteristics of the pixel values ​​within the sliding window are statistically analyzed to determine whether the sliding window is a rice paddy area or a flood area. The distribution characteristics include, for example, the mean, skewness, and variance.

[0048] Finally, the final detection images before and after the disaster are compared, and the changes in water accumulation in the images are analyzed to determine the changes in the scope of the disaster's impact.

[0049] This invention first establishes a first image set and a second image set of different types. A first remote sensing image is extracted from the first image set, and multiple second remote sensing images with low occlusion rates are extracted from the second image set. Resolution enhancement processing is then performed on the two types of images using non-deep learning methods to obtain a third remote sensing image and a fourth remote sensing image, thereby achieving feature enhancement of the remote sensing images. Then, the boundaries of ground features are delineated using the third remote sensing image, and the fourth remote sensing image makes it easier to identify ground objects. The boundary image and target image obtained based on the two are fused to obtain the detection image. Finally, the changes in the disaster impact range can be determined by comparing the detection images before and after the disaster.

[0050] This embodiment obtains the second sub-image through the following steps:

[0051] The first remote sensing image is a radar image. Historical segmentation records of the first remote sensing image and historical enhancement records of resolution enhancement of the first sub-image are acquired. Based on the historical segmentation records, a first processing time for different segmentation sizes is determined. Based on the historical enhancement records, a second processing time for different resolution enhancement factors under different segmentation sizes is determined. A time distribution matrix is ​​constructed based on the first and second processing times. The time distribution matrix includes the comprehensive processing time of various combinations of segmentation sizes and enhancement factors. Based on the time distribution matrix, the optimal segmentation size and optimal enhancement factor are determined. The first remote sensing image is segmented based on the optimal segmentation size to obtain the first sub-image. The resolution of each first sub-image is increased by the optimal enhancement factor to obtain the second sub-image.

[0052] In the event of an extreme disaster, it is necessary to determine the impact of the disaster in a short period of time. To this end, this invention improves the resolution of the first remote sensing image while ensuring the timeliness of image processing based on the following method.

[0053] First, obtain the historical segmentation records of the first remote sensing image and the historical enhancement records of the first sub-images. Both historical segmentation and enhancement records can be obtained through testing. For example, segment the first remote sensing image into 8×8 sub-images and repeat this process 10 times, recording the segmentation time each time. Take the average segmentation time as the first processing time for segmenting the first remote sensing image into 8×8 sub-images. Similarly, after segmenting the first remote sensing image into 8×8 sub-images, enhance the resolution of each sub-image by a factor of 2, and obtain the enhancement time. Repeat this process ten times and take the average to obtain the second processing time for enhancing the resolution of the first remote sensing image into 8×8 sub-images by a factor of 2. Add the first and second processing times to obtain the comprehensive processing time, which represents the total processing time for enhancing the resolution of each sub-image by a factor of 2 after segmenting the first remote sensing image into 8×8 sub-images.

[0054] Then, a temporal distribution matrix is ​​constructed based on the first and second processing times. For example, the matrix can be constructed using rows as the segmentation size and columns as the enhancement factor. Based on the constructed temporal distribution matrix, the overall processing time for the first remote sensing image at different segmentation sizes and enhancement factors can be determined. Next, the optimal segmentation size and optimal enhancement factor are determined based on the temporal distribution matrix. Finally, the first remote sensing image is segmented according to the optimal segmentation size to obtain the first sub-image. The resolution of each first sub-image is then enhanced according to the enhancement factor to obtain the corresponding second sub-image.

[0055] This embodiment determines the optimal segmentation size and optimal segmentation ratio based on the following steps:

[0056] The evaluation function is constructed by weighted summing of the comprehensive processing time, resolution enhancement factor, and segmentation size. The element that maximizes the evaluation function result in the time distribution matrix is ​​obtained, and the segmentation size and enhancement factor corresponding to this element are taken as the optimal segmentation size and optimal enhancement factor.

[0057] When constructing the evaluation function, a smaller overall processing time is better, while a larger resolution enhancement factor and segmentation size are better. Therefore, a negative sign is added before the overall processing time when constructing the evaluation function. In addition, the overall processing time, resolution enhancement factor, and segmentation size need to be normalized during construction to eliminate the difference in dimensions between different indicators. Then, the weights of the three parameters are set. The larger the parameter weight, the more important the parameter is. The specific weight values ​​can be set according to relevant requirements.

[0058] Specifically, the more sub-images the first remote sensing image is segmented into, the greater the resolution improvement, but the longer the overall processing time. However, the more sub-images there are, the smaller the segmentation size becomes. In addition, as the number of sub-images increases, each sub-image needs to be processed, thus increasing the processing base and making it easier for the image edges to become blurred after processing, which will affect the final resolution improvement quality. Therefore, the three parameters in the evaluation function are interdependent, and a compromise value needs to be determined using the evaluation function.

[0059] In this embodiment, fusing the second sub-image into a third remote sensing image includes the following steps:

[0060] There is a first overlapping region between the segmented first sub-images. In each second sub-image, the second overlapping region corresponding to the first overlapping region is located. The first region and the second region are divided in the second overlapping region. The second region is close to the center of the second sub-image. The second sub-images to be merged are defined as the first image and the second image. The second region of the second image is superimposed on the second region of the first image to complete the merging of the first image and the second image to obtain the merged image. The merged image is then merged with the remaining second sub-images in sequence to obtain the third remote sensing image.

[0061] Since the edges of images are prone to blurring after resolution enhancement, this invention proposes the following method to avoid quality degradation of the stitched third remote sensing image. Firstly, when segmenting the first remote sensing image, there is a first overlapping region between the first sub-images. Then, in the second sub-image generated based on the first sub-image, the second overlapping region generated based on the first overlapping region is located, such as... Figure 2 In the second sub-image A and the second sub-image B, there is a common region, namely the second overlapping region. The second overlapping region in the second sub-image A is the region included by a1 and a2, and the second overlapping region in the second sub-image B is the region included by b1 and b2.

[0062] Then, the first region and the second region were divided. Figure 2 Regions a2 and b1 are the first regions of the second sub-image A and the second sub-image B, respectively, and regions a1 and b2 are the second regions of the second sub-image A and the second sub-image B, respectively.

[0063] When merging the second sub-images A and B, the second sub-image B is shifted to the left until its second region b2 overlaps with the second region a1 of the second sub-image A. Then, regions a2 of the second sub-image A and b1 of the second sub-image B are discarded. One of the two regions, b2 or a1, is retained, or the average of their corresponding pixels is taken, thus completing the stitching of the second sub-images A and B. This stitching method eliminates image edges, which are not involved in the stitching process, thereby improving the quality of the stitched third remote sensing image.

[0064] In this embodiment, obtaining the positional deviation between different second remote sensing images includes the following steps:

[0065] In the base image, a first cropping region is set, and the center of the first cropping region and its corresponding first coordinates are determined. In other second remote sensing images, a second cropping region is generated with the first coordinates as the center. The size of the second cropping region is larger than that of the first cropping region. The first cropping region is compared with each of the second cropping regions to determine the third cropping region in the second cropping region that corresponds to the first cropping region. The second coordinates of the center of the third cropping region are obtained, and the positional deviation is calculated based on the difference between the first coordinates and the second coordinates.

[0066] The size of the first cropping region is, for example, 9×9, meaning the first cropping region includes 81 pixels. Then, the first coordinate of the center of the first cropping region is determined. The first coordinate is the position of the pixel in the first cropping region, such as (5,5). The first coordinate is located in other second remote sensing images besides the base image. Then, a second cropping region is generated with the first coordinate as the center. The second cropping region should be larger than the first cropping region. The size of the second cropping region is, for example, 18×18, so that the deviation of the first cropping region in the second remote sensing image can be determined by comparison.

[0067] To obtain accurate positional deviation, this embodiment uses phase correlation to compare the first and second cropping regions. Phase correlation is an existing technology and will not be described further here. The phase correlation method can identify the part in the second cropping region that is most similar to the first cropping region, which is the third cropping region, and can obtain sub-pixel level offsets. For example, through analysis, the center coordinates of pixel A in the first cropping region are (5,5), while in the third cropping region, the center coordinates are (6.5,4.5). The position of the pixel in the third cropping region is offset to the right by 1.5 pixels and downwards by 0.5 pixels compared to the pixel position in the first cropping region.

[0068] In other embodiments, the base image and the second remote sensing image can be directly compared as a whole to obtain the positional deviation of each pixel.

[0069] In this embodiment, obtaining the fourth remote sensing image includes the following steps:

[0070] Determine the resolution upscaling factor, process the base image based on the upscaling factor to obtain a first upscaled image, and based on the upscaling factor and the positional deviation between the base image and each second remote sensing image, scale the first upscaled image into multiple first scaled images, and generate a first deviation image based on the first scaled image and the corresponding second remote sensing image.

[0071] The first biased image is magnified based on the magnification factor to obtain the second biased image. The pixel values ​​of the pixels in the first magnified image are corrected based on the second biased image to obtain the second magnified image. The second magnified image is used as the fourth remote sensing image.

[0072] For example, a resolution increase factor of 2 means that the resolution of the base image is increased by 2 times. When increasing the resolution of the base image, linear interpolation, bilinear interpolation, or trilinear interpolation can be used for initial resolution increase. For example, by using bilinear interpolation, the size of the first cropped region is changed from 32×32 to 64×64. The calculation method of bilinear interpolation is existing technology and will not be introduced here.

[0073] After obtaining the first uplifted image, the first uplifted image is scaled up to a first scaled image corresponding to each second remote sensing image based on the positional deviation. Specifically, for example, if the coordinates of pixel W in the base image are (17, 20), if the base image is enlarged, the coordinates of pixel W after enlargement are (17×2, 20×2), which is (34, 40).

[0074] For example, through analysis, the offset of the second remote sensing image relative to the base image is (1.2, -0.4), where 1.2 represents an offset of 1.2 pixels to the right and -0.4 represents an offset of 0.4 pixels downward. If the base image is enlarged, the offset will be magnified by a factor of 2. Therefore, the coordinates of pixel W with coordinates (17, 20) in the second remote sensing image are 17×2+1.2×2=36.4, 20×2-0.4×2=39.2. That is, the coordinates of the pixel W in the first image are (36.4, 39.2).

[0075] Based on the above calculation results, a reverse scaling calculation can be performed on the first lifted image to obtain the first scaled image. For example, if the coordinates of pixel C in the first lifted image are (34, 40), and its corresponding pixel in the base image is (17, 20), since both are integers, the pixel value at coordinates (34, 40) in the first lifted image is directly used as the pixel value at coordinates (17, 20) in the base image.

[0076] For example, if pixel D in the first augmented image has coordinates (36.4, 39.2) and corresponds to pixel D in the second remote sensing image with coordinates (17, 20), then the pixel values ​​of the four pixels surrounding (36.4, 39.2) are obtained. These four pixel values ​​are then summed and averaged, and the average value is used as the pixel value at coordinates (17, 20) in the scaled second remote sensing image. The weighted summation can be performed based on the coordinate position, which is a current technique and will not be described in detail here. Using this method, the first augmented image can be scaled, and by incorporating positional deviations during the scaling process, a scaled image corresponding to each second remote sensing image can be obtained.

[0077] Since the first scaled image and the second remote sensing image have the same resolution, a difference operation is performed between the second remote sensing image and the corresponding first scaled image to obtain a first deviation image. The first deviation image represents the difference between the scaled second remote sensing image and the first scaled image. After this, each first scaled image is enlarged to obtain a second deviation image. Finally, the first scaled image is scaled by weighted summation of the second deviation images. For example, if a pixel in the first scaled image has a pixel value of 15, the corresponding pixel in second deviation image 1 has a pixel value of 0.5, and the corresponding pixel in second deviation image 2 has a pixel value of 0.3, and the weights of second deviation image 1 and second deviation image 2 are 0.4 and 0.6 respectively, then the corrected pixel value is 15 + 0.4 × 0.5 + 0.3 × 0.6 = 15.38.

[0078] In particular, the above process can be overlapped multiple times to continuously optimize the first enhanced image.

[0079] The significance of the above scheme is that ordinary interpolation can only smoothly fill pixels and cannot add real details. However, in this embodiment, the second remote sensing images taken at different times may contain different details. By obtaining the deviation image through multiple second remote sensing images, the first uplift image can be used to supplement details, thereby improving the final image presentation quality.

[0080] In this embodiment, extracting multiple second remote sensing images from the second image set includes the following steps:

[0081] The remote sensing images extracted from the second image set are defined as images to be analyzed. The images to be analyzed are visible light images. The pixels in the images to be analyzed are clustered based on the pixel values. Based on the clustering results, the pixels are divided into multiple categories. Each category is numbered, and the number is assigned to the corresponding pixel in the images to be analyzed to generate numbered images.

[0082] A sliding window is set up, which traverses the numbered images with a preset step size. After each movement, the first average value of the number and the second average value of the pixel value are obtained within the sliding window. If the first average value of the sliding window is the same in multiple adjacent positions and the second average value is within a preset range, the area covered by the sliding window in multiple adjacent positions is taken as the occlusion area. The occlusion rate is calculated based on the ratio of the occlusion area to the area of ​​the image to be analyzed. If the occlusion rate is greater than a critical threshold, the image to be analyzed is removed. Otherwise, the image to be analyzed is retained as the second remote sensing image. This step is repeated until a preset number of second remote sensing images are collected.

[0083] To facilitate analysis, the images to be analyzed can be converted to grayscale or HLS images. First, the K-means algorithm is used, with pixel values ​​as clustering features, to cluster the pixels in each image. The number of clusters in the K-means algorithm can be determined using the elbow coefficient method or based on the types of ground features that may be included in the remote sensing image. Through clustering, pixels with similar values ​​are grouped together; for example, pixels representing clouds tend to be white, so their corresponding pixels will be clustered together.

[0084] After clustering, each category is numbered. For example, if the pixels representing clouds are clustered into one category, then all pixels in that category are numbered 1. Then, the pixel values ​​of the pixels in the image to be analyzed are replaced with the numbers to obtain a numbered image.

[0085] The sliding window, for example, comprises 3×3 pixels. After each movement of the sliding window, a first average value including the numbered pixels is calculated. For example, if all 9 pixels within the sliding window are numbered 1, the first average value is 1. If the first average value is the same at multiple adjacent positions, it indicates that these positions correspond to the same type of land cover. If the second average value is within a preset range, the preset range is determined based on the specific occlusion. For example, if the occlusion is a cloud and the image to be analyzed is converted to grayscale, then the second average value is greater than 240, and these positions are considered to be areas where clouds are located, i.e., these areas are considered occluded. This method counts the number of occluded pixels (clouds) in the image to be analyzed, as well as the total number of pixels in the image to be analyzed. The ratio of the number of occluded pixels to the total number of pixels is used as the occlusion rate.

[0086] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

[0088] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote sensing image feature enhancement method for extreme scenarios, characterized in that, include: A first image set and a second image set are acquired, which are image sets constructed from remote sensing images of two different imaging types; wherein, the remote sensing images in the first image set are SAR images, and the remote sensing images in the second image set are optical images. Extract a single first remote sensing image from the first image set, and extract multiple second remote sensing images from the second image set; The first remote sensing image is segmented into regions to obtain multiple first sub-images. Each first sub-image is enhanced with a first resolution to obtain a second sub-image. The second sub-images are then fused into a third remote sensing image. Select one of the second remote sensing images as the base image, perform position registration on the base image, and obtain the positional deviation between the base image and different second remote sensing images; The base image is enhanced by a second resolution to obtain a first enhanced image. The first enhanced image is then corrected based on the remaining second remote sensing images to obtain a fourth remote sensing image. Boundary delineation is performed based on the third remote sensing image to obtain a boundary image; target analysis is performed based on the fourth remote sensing image to obtain a target image; the boundary image and the target image are fused to obtain the final detection image. The process of obtaining the second sub-image includes the following steps: the first remote sensing image is a radar image; historical segmentation records of the first remote sensing image and historical enhancement records of resolution enhancement of the first sub-image are acquired; a first processing time for different segmentation sizes is determined based on the historical segmentation records; a second processing time for different resolution enhancement factors at different segmentation sizes is determined based on the historical enhancement records; a time distribution matrix is ​​constructed based on the first and second processing times, the time distribution matrix including the comprehensive processing time of various combinations of segmentation sizes and enhancement factors; an optimal segmentation size and an optimal enhancement factor are determined based on the time distribution matrix; the first remote sensing image is segmented based on the optimal segmentation size to obtain the first sub-image; and the resolution of each first sub-image is increased by the optimal enhancement factor to obtain the second sub-image. The method for obtaining the positional deviation between the base image and different second remote sensing images includes the following steps: setting a first cropping region in the base image, determining the center of the first cropping region and its corresponding first coordinates, generating a second cropping region centered on the first coordinates in other second remote sensing images, wherein the size of the second cropping region is larger than that of the first cropping region, comparing the first cropping region with each of the second cropping regions respectively, determining a third cropping region in the second cropping region that corresponds to the first cropping region, obtaining the second coordinates of the center of the third cropping region, and calculating the positional deviation based on the difference between the first coordinates and the second coordinates; The process of obtaining the fourth remote sensing image includes the following steps: determining a resolution upscaling factor; processing the base image based on the upscaling factor to obtain a first upscaled image; scaling the first upscaled image into multiple first scaled images based on the upscaling factor and the positional deviations between the base image and each of the second remote sensing images; generating a first deviation image based on the first scaled image and the corresponding second remote sensing image; enlarging the first deviation image based on the upscaling factor to obtain a second deviation image; correcting the pixel values ​​of the pixels in the first upscaled image based on the second deviation image to obtain a second upscaled image; and using the second upscaled image as the fourth remote sensing image.

2. The method according to claim 1, characterized in that, The process of fusing the second sub-image into the third remote sensing image includes the following steps: There is a first overlapping region among the segmented first sub-images. In each second sub-image, a second overlapping region corresponding to the first overlapping region is located. A first region and a second region are divided in the second overlapping region. The second region is close to the center of the second sub-image. The second sub-images to be merged are defined as the first image and the second image, respectively. The second region of the second image is superimposed on the second region of the first image to complete the merging of the first image and the second image to obtain a merged image. The merged image is then merged with the remaining second sub-images in sequence to obtain the third remote sensing image.

3. The method according to claim 1, characterized in that, Determining the optimal segmentation size and the optimal enhancement factor based on the time distribution matrix includes the following steps: The weighted sum of the integrated processing time, resolution enhancement factor, and segmentation size is used to construct an evaluation function. The element in the time distribution matrix that maximizes the result of the evaluation function is obtained, and the segmentation size and enhancement factor corresponding to this element are taken as the optimal segmentation size and the optimal enhancement factor.

4. The method according to claim 1, characterized in that, Extracting multiple remote sensing images from the second image set includes the following steps: The remote sensing image extracted from the second image set is defined as the image to be analyzed. The image to be analyzed is a visible light image. The pixels in the image to be analyzed are clustered based on the pixel value. Based on the clustering results, the pixels are divided into multiple categories. Each category is numbered and the number is assigned to the corresponding pixel in the image to be analyzed to generate a numbered image. A sliding window is set up, which traverses the numbered images with a preset step size. After each movement, a first average value including the number and a second average value of the pixel value are obtained within the sliding window. If the first average value of the sliding window is the same at multiple adjacent positions, and the second average value is within a preset range, the area covered by the sliding window at the multiple adjacent positions is taken as the occlusion area. The occlusion rate is calculated based on the ratio of the occlusion area to the area of ​​the image to be analyzed. If the occlusion rate is greater than a critical threshold, the image to be analyzed is removed; otherwise, the image to be analyzed is retained as the second remote sensing image. This step is repeated until a preset number of second remote sensing images are collected.

Images