Natural disaster satellite remote sensing image short-time lag super-division method based on space shift convolution
Through the method based on spatial shift convolution, the problem of insufficient resolution of satellite remote sensing images is solved, and the generation and clarity of high-resolution images are achieved, which is suitable for real-time application scenarios with resource limitations.
Patent Information
- Application Number
- CN202411097421.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-12
AI Technical Summary
Existing satellite remote sensing images have insufficient resolution, making it difficult to capture disaster features such as ground cracks and hurricanes, and deep learning models are difficult to deploy and operate in resource-constrained environments, resulting in slow inference speed.
Using the short-term delay super-segment method of remote sensing images of natural disaster satellites based on spatial shift convolution, spatial shift operation is realized through 1×1 convolution layer, local feature aggregation, and time delay super-segment model is trained to generate high-resolution images .
It significantly improves the resolution and clarity of the image, meets the needs of real-time applications, can better preserve the details and texture of the original image, and the model structure is lightweight, suitable for system applications.
Smart Images

Figure CN118967446A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of satellite remote sensing image processing, and in particular to a short-time-delay super-resolution method of natural disaster satellite remote sensing images based on spatial shift convolution. Background Art
[0002] Natural disaster monitoring based on satellite remote sensing images can monitor and respond to various disasters in a more comprehensive, timely and efficient manner. Due to the limitations of the hardware facilities for acquiring existing satellite remote sensing images, such as the fidelity of imaging and the bandwidth limitations of data transmission, the resolution of satellite remote sensing images is insufficient, making it difficult to capture detailed disaster features, such as ground cracks, hurricanes, etc., thereby restricting the detection performance of disaster models. In real-time application scenarios such as disaster detection using satellite remote sensing images, the need for short time delays is more important. However, deep learning models usually require a large amount of computing resources and storage space, and larger models may result in slower inference speeds, and may even be difficult to deploy and run in resource-constrained environments. In response to the above problems, the present invention attempts to design a special super-resolution model for satellite remote sensing images in order to improve the resolution of satellite remote sensing images and increase the significance of disaster features. Summary of the invention
[0003] The present invention discloses a short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution. The specific method is as follows:
[0004] Acquire low-resolution satellite remote sensing images with a resolution lower than a preset value, and pre-process the low-resolution satellite remote sensing images;
[0005] The 1×1 convolutional layer is used to implement spatial shift operations, process low-resolution satellite remote sensing images, and complete local feature aggregation;
[0006] Pair low-resolution and high-resolution satellite remote sensing images to train a time-delay super-resolution model;
[0007] The low-resolution remote sensing images are processed with the trained time-delay super-resolution model to complete the super-resolution conversion and generate high-resolution images.
[0008] Furthermore, the low-resolution satellite remote sensing image is preprocessed, including normalization processing. The specific formula is:
[0009]
[0010] Where x is the original pixel value, x min and x max are the minimum and maximum pixel values, respectively, and x normalized is the normalized pixel value.
[0011] Furthermore, local features are aggregated, and the specific method is as follows:
[0012] Given a remote sensing image input feature f, first divide it evenly into n groups of feature maps along the channel dimension, where n = S, and obtain n thinner tensors f i , expressed as
[0013]
[0014] Among them, f i represents the i-th thinner tensor, H is the height, W is the width, C latent represents the channel dimension of the potential feature, and n represents the number of groups after segmentation;
[0015] Each separated feature group is shifted according to the given step size parameter to obtain the shifted feature f shift , f shift Each pixel feature in contains local features along the channel dimension and is padded with a constant 0 by default.
[0016] Furthermore, displacement is performed according to a given step size parameter, and the specific method is as follows:
[0017] Calculate the number of shift groups. According to the number of channels C of the input feature map and the number of shift steps, calculate the number of shift groups shift_groups. The number of shift groups is equal to the number of channels divided by the number of shift steps.
[0018] Calculate the number of channels in each group. The number of channels in each group is group_dim, which is the total number of channels C divided by the number of shift groups shift_groups;
[0019] Fill the input feature map, use the F.pad() function to fill the input feature map f, and the filling size is pad;
[0020] Initialize the output feature map, initialize a zero tensor output with the same shape as the input feature map f
[0021] Circular shift operation, for each shift step in steps, get the index idx, row shift amount s_h and column shift amount s_w corresponding to the current shift step; divide the padded feature map f_pad into shift_groups sub-blocks, and the number of channels of each sub-block is group_dim; shift and crop each sub-block;
[0022] Return the output feature map, return the shifted feature map output;
[0023] The displacement is to move the upper left corner of the sub-block to the lower right corner of the output feature map, and the specific position is determined by (idx+1)*group_dim;
[0024] Cropping is to crop the sub-block to adapt it to the position of the output feature map, and the cropping size is (pad+s_h, pad+s_H) and (pad+s_w, pad+W), ensuring that the size of the output feature map remains unchanged.
[0025] Furthermore, the time-delay super-resolution model is constructed as follows:
[0026] The output of the 1×1 convolutional layer is used as the input layer of the time-delay super-resolution model to map the low-resolution satellite remote sensing image to a latent high-dimensional space. The shallow features f in the latent high-dimensional space are head Described as
[0027]
[0028] Among them, X is the input low-resolution satellite remote sensing image, C latent represents the channel dimension of the latent features, N head Represents the input layer of the time-delay super-resolution model;
[0029] The backbone of the time-delay superresolution model backbone Made of SC-ResBlocks, N backbone With shallow features f head As input, the deep features of the low-resolution satellite remote sensing image extracted are output, which can be expressed as
[0030] f backbone =N backbone (f head )
[0031] Based on the deep feature f backbone , reconstruct the high-resolution satellite remote sensing image Z using the upsampling module;
[0032] Upsampling module N rec Includes SC layer, ReLU, 1×1 convolution and pixel shuffle operations;
[0033] The multi-channel features obtained by upsampling are mapped to a 3-channel high-resolution satellite remote sensing image Z using 1×1 convolution, and bilinear interpolation is introduced to assist the super-resolution process. The whole process can be expressed as
[0034] Z=N rec (f backbone )+Bilinear(X)
[0035] Among them, X is a low-resolution satellite remote sensing image, Z is a high-resolution satellite remote sensing image, and f backbone is the deep features of low-resolution satellite remote sensing images, N rec It is an upsampling module and Bilinear is a bilinear interpolation module.
[0036] Furthermore, the time-delay super-resolution model is trained. The specific method is as follows:
[0037] Divide the original dataset into training set, validation set and test set
[0038] Compress each image to its original resolution and then Figure 1 The images are cut into pieces of 256×256 size and stored in pairs in the final training data set;
[0039] Initialize the parameters of the training model. In the training configuration, the maximum number of iterations is set to 50, the batch size is set to 32, and the Adam optimizer is used to adjust the model parameters. The learning rate is set to 0.001.
[0040] According to the convergence effect of the mean square error loss function.
[0041] Furthermore, the method also includes testing and verification, and the accuracy and generalization of the model are evaluated by testing the performance of the model in actual application scenarios through a test set. The specific method is as follows:
[0042] Peak signal-to-noise ratio and structural similarity index are used as indicators to comprehensively evaluate the performance of the time-delay super-resolution model;
[0043] The peak signal-to-noise ratio calculation formula is as follows:
[0044]
[0045] Among them, MAX I is the maximum pixel value of the image;
[0046] The structural similarity index metric calculation formula is as follows:
[0047] SSIM(x,y)=[lx,y)] α [c(x,y)] β ·[s(x,y)] γ
[0048]
[0049] Among them, x and y represent the output image of the super-resolution model and the real satellite remote sensing super-resolution image, l, c, and s represent the similarity of the three components of brightness, contrast, and structure, respectively, and α, β, and γ are weighting factors, which are usually taken as 1 to ensure equal contribution of each part, but can also be adjusted according to actual applications. x and μ y are the average brightness of images x and y, respectively, x and σ y is the standard deviation of images x and y, σ xyis the covariance of images x and y, and C1, C2, and C3 are constants.
[0050] Furthermore, after generating the high-resolution image, a denormalization process is performed to denormalize each pixel value back to its original range. The specific formula is as follows:
[0051] x original =x normalized *(x max -x min )+x min
[0052] Among them, x original is the original value, x normalized is the normalized value, x max is the maximum value among the original values, x min is the minimum of the original values.
[0053] Due to the adoption of the above technical solution, the present invention has the following beneficial effects:
[0054] 1. From the perspective of visual effects, the present invention can better preserve the details and texture of the original image during the interpolation process. Compared with traditional interpolation methods, the method of the present invention has significantly improved the clarity and detail performance of the interpolated image, has good performance in satellite remote sensing image super-resolution, and can effectively improve the resolution and clarity of the image.
[0055] 2. The structural design of the model of the present invention is based on lightweight modules, which can meet the real-time requirements of the super-resolution model in system applications.
[0056] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] The accompanying drawings of the present invention are as follows.
[0058] Figure 1 This is a schematic diagram of the architecture of the short-time-delay super-resolution algorithm for satellite remote sensing images;
[0059] Figure 2 Schematic diagram of the structural comparison between the traditional 3×3 convolution residual module and the 1×1 shifted convolution residual module;
[0060] Figure 3 Schematic diagram of the overall architecture of the short-delay super-resolution model for satellite remote sensing images. DETAILED DESCRIPTION
[0061] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0062] A short-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution. The algorithm architecture diagram is shown in the figure. Figure 1 As shown, given a low-resolution satellite remote sensing image X, it is represented as a matrix Where H and W are the height and width of the image, respectively, and C is the number of channels. Lightweight satellite remote sensing image super-resolution model F sr Learn a mapping function to generate a high-resolution image Z. sr Using a lightweight deep convolutional neural network for construction, the super-resolution process of satellite remote sensing images can be expressed as:
[0063] Z=F sr (X;θ sr )
[0064] Among them, θ sr is the weight parameter of the lightweight satellite remote sensing image super-resolution model, and the mean square error MSE loss is used to measure the reconstruction error to optimize the weight parameter θ sr , the process can be expressed as:
[0065]
[0066] in, is a real high-resolution image, H represents the height of the image, and W represents the width of the image. and Z ij Represent the pixel values of the real image and the super-resolution model output image at position (i, j). The network parameters are updated through the back propagation algorithm so that the loss function L MSE (θ sr ) minimized.
[0067] After the lightweight super-resolution model, a high-resolution image is obtained Where H' and W' are the height and width of the image Z, respectively, and C' is the number of channels of the image.
[0068] The specific steps are as follows:
[0069] S1. Acquire a low-resolution satellite remote sensing image with a resolution lower than a preset value, and preprocess the low-resolution satellite remote sensing image.
[0070] Satellite remote sensing images can be obtained mainly through the following channels: government and scientific research institutions, such as the United States Geological Survey (USGS) providing Landsat series data, the European Space Agency (ESA) providing Sentinel series data, and the China Resources Satellite Application Center providing Gaofen series satellite data; online map services, such as Google Earth and Microsoft BingMaps; open source data platforms, such as Copernicus OpenAccess Hub and Sentinel Hub, which provide free access to Sentinel satellite data; and data markets and trading platforms, such as Amazon Web Services Marketplace and GBDX, etc., to select the appropriate channel to obtain images according to needs.
[0071] Read satellite remote sensing image data and apply the normalization formula to map the values between 0 and 1.
[0072] The normalization formula is:
[0073]
[0074] Where x is the original pixel value, x min and x max are the minimum and maximum pixel values, respectively, and x normalized is the normalized pixel value.
[0075] S2, uses a 1×1 convolutional layer to implement spatial shift operations, process low-resolution satellite remote sensing images, and complete local feature aggregation.
[0076] Step S2 introduces a parameter-free spatial shift operation based on the traditional convolution operation to take into account the local feature aggregation ability and lightweight requirements of the convolutional neural network. The structure of the 3×3 convolution residual module and the 1×1 shifted convolution residual module is compared as follows: Figure 2 Specifically, based on the 1×1 convolutional layer, a spatial shift operation is designed for local feature aggregation. On this basis, inspired by the residual neural network structure, a shifted convolution residual module is constructed as the backbone network of the lightweight satellite remote sensing image super-resolution model.
[0077] 1×1 convolution is also called point-by-point convolution. The main difference from conventional convolution is that the size of its convolution kernel is 1×1, that is, the convolution kernel matrix has only one element, indicating that the computational complexity of this type of convolution is much lower than that of conventional convolution. At the same time, the nonlinear transformation ReLU is introduced to increase the deep feature extraction capability of the convolutional network.
[0078] Based on the 1×1 convolution, local features are explicitly aggregated through spatial shift. To facilitate the formal description of this process, the displacement direction is first represented as d∈{1,0,-1}, where 1 represents upward or rightward movement, 0 represents no movement, and -1 represents downward or leftward movement. In addition, d h and d w Represent the vertical and horizontal shift directions respectively, and the shift step size is denoted as s accordingly. h and w On this basis, the shift direction and step length form the spatial displacement operation step step = (d h ×s h ,d w ×s w ), the set of spatial displacement steps is expressed as
[0079] S={step i ,i=1,…,n}
[0080] Where n is the number of aggregated features, step i Represents the operation steps of the i-th local pixel feature.
[0081] To achieve the aggregation of 8 local pixel features similar to the standard 3×3 convolution, the set of spatial shift operation steps can be defined as (0,1), (0,-1), (1,0), (1,1), (1,-1), (-1,0), (-1,1), (-1,-1). The steps of the spatial shift operation are described in detail as follows:
[0082] S21. Parameter declaration: f: input feature map, type is torch.Tensor, shape is (B,C,H,W), indicating B batches, C channels, height H and width W. steps: a list of tuples, indicating the spatial shift step in each direction (rows and columns). pad: padding size, used to ensure that the output size is the same as the input.
[0083] S22. Calculate the number of shift groups. According to the number of channels C of the input feature map and the number of shift steps, calculate the number of shift groups shift_groups. The number of shift groups is equal to the number of channels divided by the number of shift steps.
[0084] S23. Calculate the number of channels in each group. The number of channels in each group is group_dim, that is, the total number of channels C divided by the number of shift groups shift_groups.
[0085] S24. Fill the input feature map and use the F.pad() function to fill the input feature map f with a pad size.
[0086] S25. Initialize the output feature map and initialize a zero tensor output with the same shape as the input feature map f.
[0087] S26. For each shift step in steps, which is in the form of a tuple representing the shift amount of rows and columns, the following operations are performed:
[0088] S261. Obtain the index idx, row shift amount s_h, and column shift amount s_w corresponding to the current shift step.
[0089] S262. Divide the padded feature map f_pad into shift_groups sub-blocks, and the number of channels of each sub-block is group_dim.
[0090] S263, shift: move the upper left corner of the sub-block to the lower right corner of the output feature map, the specific position is determined by (idx+1)*group_dim;
[0091] Cropping: Cropping the sub-block to adapt it to the position of the output feature map, the cropping size is (pad+s_h, pad+s_H) and (pad+s_w, pad+W), ensuring that the size of the output feature map remains unchanged.
[0092] S27. Return the output feature map, and return the shifted feature map output.
[0093] In this embodiment, given a satellite remote sensing image input feature f, it is first evenly divided into n groups of feature maps along the channel dimension, where n = S, and n thinner tensors f are obtained. i , which can be expressed as
[0094]
[0095] Then, each separated feature group is shifted according to the given step size parameter to obtain the shifted feature f shift .f shift Each pixel feature in contains local features along the channel dimension and is padded with a constant 0 by default.
[0096] The shifted convolution residual module consists of a 1×1 convolution layer and a spatial shift operation to form a shifted convolution layer (SC). Therefore, the shifted convolution layer expands the ordinary 1×1 convolution and provides local feature aggregation capabilities with fewer parameters and computation. On this basis, the shifted convolution residual module (SC-ResBlock) is designed by combining the shifted convolution layer with the residual neural network structure. SC-ResBlock contains SC layer, ReLU and 1×1 convolution layer. Compared with the residual module based on 3×3 convolution, SC-ResBlock only uses 1×1 convolution, which greatly reduces the number of parameters and computation.
[0097] S3. Pair low-resolution and high-resolution satellite remote sensing images to train the time-delay super-resolution model.
[0098] S31. The construction process of short-time-delay super-resolution network for satellite remote sensing images is as follows:
[0099] First, a standard 1×1 convolution is used as the input layer of the satellite remote sensing image super-resolution network to map the satellite remote sensing image to a latent high-dimensional space. head Can be described as
[0100]
[0101] Among them, X is the input satellite remote sensing image, C latent represents the channel dimension of the latent features, N head Represents the input layer of the satellite remote sensing image super-resolution network.
[0102] Secondly, the backbone part of the satellite remote sensing image super-resolution network N backone Made of stacked SC-ResBlocks. backone With shallow features f head As input, the deep features of the satellite remote sensing image extracted are output, which can be expressed as
[0103] f backbone =N backbone (f head )
[0104] Then, based on the deep feature f backbone , use the upsampling module to reconstruct the high-resolution satellite remote sensing image Z. Upsampling module N rec It consists of SC layer, ReLU, 1×1 convolution and pixel shuffle operation PixelShuffle. Finally, 1×1 convolution is used to map the upsampled multi-channel features to a 3-channel high-resolution satellite remote sensing image Z. At the same time, bilinear interpolation Bilinear assisted super-resolution process is introduced. The whole process can be expressed as
[0105] Z=N rec (f backbone )+Bilinear(X)
[0106] The overall architecture of the short-time-delay super-resolution model for satellite remote sensing images is as follows: Figure 3 shown.
[0107] The super-resolution model is trained. A large number of paired satellite remote sensing images of low resolution LR and high resolution HR are used as training data. The goal of the training process is to minimize the difference between the super-resolution SR images generated by the model and the real high-resolution images. The mean square error is used as the loss function to measure this difference, and the model parameters are adjusted through the back-propagation algorithm to optimize the performance.
[0108] In this embodiment, a satellite remote sensing image disaster monitoring dataset is constructed based on the largest natural disaster image dataset xBD released by MIT to meet the verification requirements of the method. The constructed satellite remote sensing image disaster monitoring dataset contains a total of 7464 satellite remote sensing images, including 7 different natural disasters and normal satellite remote sensing images. For model development, verification and testing, the original dataset is divided into training set, validation set and test set. The number of training set image samples is 4478 for model training, and the number of validation set image samples is 1120 for adjusting the hyperparameters of the model and evaluating the performance of the model to avoid overfitting or underfitting. First compress the resolution of each image, and then compare it with the original image. Figure 1 The training model is cut into 256×256 size fragments and stored in pairs in the final training data set. The parameters for initializing the training model are set. In the training configuration, the maximum number of iterations is set to 50, the batch size is set to 32, and the Adam optimizer is used to adjust the model parameters. The learning rate is set to 0.001 to achieve efficient and stable gradient descent. According to the convergence effect of the mean square error loss function, only the trained model needs to be selected for the test experiment. The 45th generation network model is selected for the experiment.
[0109] S32. After the model training is completed, the generalization ability of the model is evaluated using the test data set. Peak signal-to-noise ratio PSNR and structural similarity index SSIM are selected as quality indicators for calculating super-resolution images.
[0110] The number of test set image samples is 1,866, which is used to evaluate the generalization ability of the model on unknown data, that is, to test the performance of the model in actual application scenarios through the test set to evaluate the accuracy and generalization of the model. The peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) are used as indicators to comprehensively evaluate the performance of the super-resolution model.
[0111] The formula for PSNR is defined as:
[0112]
[0113] Among them, MAX I It is the maximum pixel value of the image. The larger the PSNR is, the better the super-resolution model effect is.
[0114] The structural similarity index metric SSIM takes into account the brightness, contrast, and preservation of image structure information of the image. Its basic formula can be expressed as:
[0115] SSIM(x,y)=[l(x,y)] α [x(x,y)] β ·[s(x,y)] γ
[0116]
[0117] In this embodiment, the average PSNR index is 37.111dB, the standard deviation is 2.417, the average SSIM index is 0.936, and the standard deviation is 0.024. In order to prove that the model algorithm is more superior, the remaining three interpolation methods: based on the bilinear interpolation BILINEAR super-resolution method, based on the cubic interpolation BICUBIC super-resolution method, based on the nearest neighbor interpolation NEAREST super-resolution method, were simulated and the objective evaluation index results were recorded. As shown in Table 1, the PSNR average values based on the BICUBIC, BILINEAR, and NEAREST methods are 33.792, 33.491, and 33.310, respectively, and the SSIM average values are 0.746, 0.722, and 0.700, respectively. In contrast, the PSNR value and SSIM value generated by this embodiment are better than other super-resolution algorithms. The above experimental results show that the method designed by the present invention can not only maintain image details and texture, but also more effectively improve the resolution and clarity of the image.
[0118] Table 1 Statistical results of satellite remote sensing image super-resolution comparison experiment
[0119]
[0120] S4. Use the trained time-delay super-resolution model to process the low-resolution remote sensing image, complete the super-resolution conversion, and generate a high-resolution image. Use the trained model to perform super-resolution conversion on the new low-resolution satellite remote sensing image to generate a high-resolution image, and perform denormalization on the generated super-resolution image.
[0121] Denormalization formula:
[0122] x original =x normalized *(x max -x min )+x min
[0123] Denormalize each pixel value back to its original range.
[0124] S5. Result feedback and model iteration: Use super-resolution images for disaster monitoring and assessment, collect feedback information, and make necessary adjustments and optimizations to the model.
[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution, characterized in that: The specific method is as follows: Acquire low-resolution satellite remote sensing images with a resolution lower than a preset value, and pre-process the low-resolution satellite remote sensing images; The 1×1 convolutional layer is used to implement spatial shift operations, process low-resolution satellite remote sensing images, and complete local feature aggregation; Pair low-resolution and high-resolution satellite remote sensing images to train a time-delay super-resolution model; The low-resolution remote sensing images are processed with the trained time-delay super-resolution model to complete the super-resolution conversion and generate high-resolution images.
2. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 1, characterized in that: Preprocess the low-resolution satellite remote sensing images, including normalization. The specific formula is: Where x is the original pixel value, x min and x max are the minimum and maximum pixel values, respectively, and x normalized is the normalized pixel value.
3. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 1, characterized in that: Local feature aggregation, the specific method is as follows: Given a remote sensing image input feature f, first divide it evenly into n groups of feature maps along the channel dimension, where n = S, and obtain n thinner tensors f i , expressed as Among them, f i represents the i-th thinner tensor, H is the height, W is the width, C latent represents the channel dimension of the potential feature, and n represents the number of groups after segmentation; Each separated feature group is shifted according to the given step size parameter to obtain the shifted feature f shift , f shift Each pixel feature in contains local features along the channel dimension and is padded with a constant 0 by default.
4. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 3, characterized in that: The displacement is performed according to the given step size parameter. The specific method is as follows: Calculate the number of shift groups. According to the number of channels C of the input feature map and the number of shift steps, calculate the number of shift groups shift_groups. The number of shift groups is equal to the number of channels divided by the number of shift steps. Calculate the number of channels in each group. The number of channels in each group is group_dim, which is the total number of channels C divided by the number of shift groups shift_groups; Fill the input feature map, use the F.pad() function to fill the input feature map f, and the filling size is pad; Initialize the output feature map, initialize a zero tensor output with the same shape as the input feature map f Circular shift operation, for each shift step in steps, get the index idx, row shift amount s_h and column shift amount s_w corresponding to the current shift step; divide the padded feature map f_pad into shift_groups sub-blocks, and the number of channels of each sub-block is group_dim; shift and crop each sub-block; Return the output feature map, return the shifted feature map output; The displacement is to move the upper left corner of the sub-block to the lower right corner of the output feature map, and the specific position is determined by (idx+1)*group_dim; Cropping is to crop the sub-block to adapt it to the position of the output feature map, and the cropping size is (pad+s_h, pad+s_H) and (pad+s_w, pad+W), ensuring that the size of the output feature map remains unchanged.
5. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 1, characterized in that: The time-delay super-resolution model is constructed as follows: The output of the 1×1 convolutional layer is used as the input layer of the time-delay super-resolution model to map the low-resolution satellite remote sensing image to a latent high-dimensional space. The shallow features f in the latent high-dimensional space are head Described as Among them, X is the input low-resolution satellite remote sensing image, C latent represents the channel dimension of the latent features, N head Represents the input layer of the time-delay super-resolution model; The backbone of the time-delay superresolution model backbone Made of SC-ResBlocks, N backbone With shallow features f head As input, the deep features of the low-resolution satellite remote sensing image extracted are output, which can be expressed as f backbone =N backbone (f head ) Based on the deep feature f backbone , reconstruct the high-resolution satellite remote sensing image Z using the upsampling module; Upsampling module N rec Includes SC layer, ReLU, 1×1 convolution and pixel shuffle operations; The multi-channel features obtained by upsampling are mapped to a 3-channel high-resolution satellite remote sensing image Z using 1×1 convolution, and bilinear interpolation is introduced to assist the super-resolution process. The whole process can be expressed as Z=N rec (f backbone )+Bilinear(X) Among them, X is a low-resolution satellite remote sensing image, Z is a high-resolution satellite remote sensing image, and f backbone is the deep features of low-resolution satellite remote sensing images, N rec It is an upsampling module and Bilinear is a bilinear interpolation module.
6. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 5, characterized in that: The specific method for training the time-delay super-resolution model is as follows: Divide the original dataset into training set, validation set and test set Each image is compressed and then cropped into 256×256 size fragments together with the original image, and stored in pairs in the final training data set; Initialize the parameters of the training model. In the training configuration, the maximum number of iterations is set to 50, the batch size is set to 32, and the Adam optimizer is used to adjust the model parameters. The learning rate is set to 0.
001. According to the convergence effect of the mean square error loss function.
7. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 6, characterized in that: The method also includes testing and verification, which uses a test set to test the performance of the model in actual application scenarios to evaluate the accuracy and generalization of the model. The specific method is as follows: Peak signal-to-noise ratio and structural similarity index are used as indicators to comprehensively evaluate the performance of the time-delay super-resolution model; The peak signal-to-noise ratio calculation formula is as follows: Among them, MAX I is the maximum pixel value of the image; The structural similarity index metric calculation formula is as follows: SSIM(x,y)=[l(x,y)] α .[c(x,y)] β .[s(x,y)] γ Among them, x and y represent the output image of the super-resolution model and the real satellite remote sensing super-resolution image, l, c, and s represent the similarity of the three components of brightness, contrast, and structure, respectively, and α, β, and γ are weighting factors, which are usually taken as 1 to ensure equal contribution of each part, but can also be adjusted according to actual applications. x and μ y are the average brightness of images x and y, σ x and σ y is the standard deviation of images x and y, σ xy is the covariance of images x and y, and C1, C2, and C3 are constants.
8. The short-time-delay super-resolution method for natural disaster satellite remote sensing images based on spatial shift convolution as claimed in claim 1, characterized in that: After generating the high-resolution image, denormalization is performed to normalize each pixel value back to its original range. The specific formula is as follows: x original =x normalized *(x max -x min )+x min Among them, x original is the original value, x normalized is the normalized value, x max is the maximum value among the original values, x min is the minimum of the original values.
Citation Information
Patent Citations
Disaster monitoring and loss assessment method based on fusion of meteorological and remote sensing data
CN108038566A
Remote sensing image super-resolution reconstruction method and system based on multi-scale enhancement module
CN116342389A
Satellite cloud picture super-resolution reconstruction method based on multipath aggregation Transformer
CN117391958A
Three-dimensional microscopic image super-resolution method and system based on autofluorescence fluctuation
CN118229529A
Satellite rainfall remote sensing image space-time super-resolution reconstruction method and system
CN118333862A
