Rotating perception enhanced variable window attention super-resolution method and system

Abstract

The invention provides a rotation perception enhanced variable window attention super-resolution method, and belongs to the technical field of image processing. Comprising the steps of collecting and obtaining an original remote sensing image of a target object for preprocessing, segmenting the original remote sensing image into a plurality of basic windows, extracting feature vectors, predicting basic parameters of a projective transformation matrix by using quadrangles according to the extracted feature vectors, obtaining a reconstructed projective transformation matrix, extracting a rotation angle difference of any two quadrangles as a rotation sensing item, and obtaining a rotation sensing result. The method comprises the following steps: establishing a variable window attention mechanism, embedding the variable window attention mechanism into a SwinIR network to establish a framework of a high-precision super-resolution model, training to obtain the high-precision super-resolution model, inputting a real-time remote sensing image into the high-precision super-resolution model, and outputting to obtain a high-resolution image containing a target sampling area. Through the combination of the attention mechanism of the variable window and the rotation perception mechanism, the super-resolution reconstruction precision and geometric consistency of the rotating ground feature are significantly improved while the calculation efficiency is maintained.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a rotation-aware enhanced variable window attention super-resolution method and system. Background Art

[0002] Single-image super-resolution (SISR), a fundamental task in computer vision, aims to reconstruct high-quality high-resolution (HR) images from low-resolution (LR) inputs. In recent years, with the rapid development of deep learning technologies, particularly the widespread application of convolutional neural networks (CNNs) and vision transformers (ViTs), the performance of super-resolution techniques has been significantly improved. Early super-resolution (SR) methods, such as the Super-Resolution Convolutional Neural Network (SRCNN), the Enhanced Deep Super-Resolution Network (EDSR), and the Residual Channel Attention Network (RCAN), were primarily based on hierarchical CNN structures, learning texture detail features through local receptive fields. However, this type of method has two major limitations: on the one hand, the fixed-size convolution kernel limits the network's ability to model long-range image dependencies and can only capture local features, resulting in incoherent texture and structure of the image; on the other hand, because the convolution operation only has translation equivariance but not rotation equivariance, its feature extraction ability for rotation patterns is weak, which is particularly evident when processing images containing rotated objects.

[0003] With the successful application of visual transformers in computer vision, ViT-based SR methods, such as the Swin Transformer-based Image Restoration (SwinIR) and EDT, have significantly improved their ability to capture global context by introducing self-attention mechanisms. These methods typically employ a window-based attention mechanism, dividing the image into non-overlapping fixed-size rectangular windows to reduce computational complexity, and implement cross-window information exchange through window offset strategies. However, existing methods still have significant shortcomings when handling rotation and scale-varying structures.

[0004] First, a fixed-shape rectangular window is difficult to align with edges or texture features in any direction in the image, which can lead to problems such as blurring or geometric distortion in the reconstruction results. Second, a fixed window size cannot adaptively process image structures of different scales. For example, it is not effective when processing fine textures and large objects at the same time.

[0005] Although some improved methods such as DAT and Deformable DETR have attempted to introduce deformable convolution into the attention mechanism, these methods still have obvious defects. They mainly focus on modeling translation offsets, but lack explicit processing of rotation and shear transformations. In addition, such methods usually require intensive sampling point calculations, resulting in high memory consumption and computational overhead, making them difficult to apply to high-resolution super-resolution tasks. Another key issue is that current window-based methods such as SwinIR require manual design of window size and offset strategies, which not only limits the generalization ability of the model on different datasets, but also introduces a large amount of unnecessary computational overhead, especially when processing images with irregular textures. This problem is more prominent.

[0006] In addition, in the traditional attention mechanism, the size and position of the fixed window are pre-defined in the model and cannot be dynamically adjusted according to the characteristics of the input data; ordinary variable windows are usually adjusted based on local features or heuristic rules, lacking global context dependencies, which leads to inaccurate window adjustment, causing important information to be ignored or redundant information to be included, failing to capture global dependencies and adapting to multi-scale feature relationships; the attention weight in the existing traditional attention mechanism is usually calculated by calculating the dot product ( ), which mainly reflects the similarity between features. When the object rotates, its feature representation will also change, but the traditional attention mechanism cannot capture this change, resulting in inaccurate feature matching of the rotated target.

[0007] Based on the above background, the present invention proposes a rotation-aware enhanced variable window attention super-resolution network, which innovatively introduces a learnable quadrilateral attention window mechanism. Summary of the Invention

[0008] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a rotation-aware enhanced variable window attention super-resolution method and system.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions: The present invention provides a rotation-aware enhanced variable window attention super-resolution method, comprising the following steps: Collect and obtain the original remote sensing images of the target object, and perform enhancement preprocessing on the collected original remote sensing images; The preprocessed original remote sensing image is divided into several basic windows, and the feature vector is extracted from each basic window; Define a projective transformation matrix, use quadrilaterals to predict proxy parameters of scaling, rotation, shearing, translation, and projection in the projective transformation matrix based on the extracted eigenvectors, and decompose the projective transformation matrix into corresponding transformation matrices based on the predicted proxy parameters; Reconstruct the projective transformation matrix according to the decomposed transformation matrix, and use the reconstructed projective transformation matrix to dynamically transform each basis window into an arbitrary quadrilateral; According to the dynamic transformation of the basic window, the rotation angle difference between the pixels at the same position in any two basic windows after transformation is extracted from the rotation transformation matrix. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. The variable window attention mechanism is embedded into the SwinIR network and applied to the feature extraction layer to establish the architecture of a high-precision super-resolution model; Acquire a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain the trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; The remote sensing image of the target object is acquired in real time, and the acquired real-time remote sensing image is input into the high-precision super-resolution model to output a high-resolution image containing the target sampling area.

[0010] Optionally, the enhancing preprocessing of the acquired original remote sensing image includes: According to the collected original remote sensing image, grayscale processing is performed on the original remote sensing image to obtain a grayscale image; Based on wavelet transform, the grayscale image is decomposed into wavelet coefficients of different frequencies, and the high-frequency coefficients are thresholded. The image is reconstructed using the high-frequency coefficients after thresholding to obtain the denoised original remote sensing image.

[0011] Optionally, the pre-processed original remote sensing image is divided into a plurality of basic windows, and a feature vector is extracted from each basic window, including: According to the predefined window size, the preprocessed original remote sensing image is divided into multiple basic windows of the same window size; Feature extraction is performed on each basic window, and the extracted features are converted into query tokens, key tokens, and value tokens through linear projection. The function expression of linear projection is:

[0012] Where Q represents the query token, K represents the key token, and V represents the value token. represents the linear transformation function, Represents features within the base window; The query token, key token, and value token corresponding to each basis window are concatenated to obtain the feature vector corresponding to the basis window.

[0013] Optionally, the defining of the projective transformation matrix, using a quadrilateral to predict proxy parameters of scaling, rotation, shearing, translation, and projection in the projective transformation matrix according to the extracted eigenvectors, includes: Define the projective transformation matrix. The function expression of the projective transformation matrix is:

[0014] Where, Define scaling, rotation, and shear transformations, Define the translation transformation, is the projection vector, which defines how the observer's perspective changes when the depth dimension changes; Collect existing remote sensing images with known projective transformation matrices, and build a neural network model consisting of an average pooling layer, a LeakyReLU activation layer, and a convolutional layer. Extract features from the existing remote sensing images and input them into the neural network model for training to obtain a quadrilateral prediction model. The feature vector extracted from the basic window is input into the quadrilateral prediction model, and the predicted proxy parameters of scaling, rotation, shearing, translation and projection are output. The function expression of the proxy parameters is:

[0015] Where, is the nth type of proxy parameter, is the convolutional layer, is the activation layer, is the average pooling layer, is the eigenvector after linear projection.

[0016] Optionally, reconstructing a projective transformation matrix according to the decomposed transformation matrix, and dynamically converting each basis window into an arbitrary quadrilateral using the reconstructed projective transformation matrix, includes: Apply the predicted proxy parameters of scaling, rotation, shearing, translation and projection to the defined projection transformation matrix, respectively, to obtain the scaling transformation matrix, rotation transformation matrix, shearing transformation matrix, translation transformation matrix and projection transformation matrix; The function expression of the scaling transformation matrix is:

[0017] Where, is the scaling transformation matrix, Is the scaling factor of the base window in the x-axis direction, Is the scaling factor of the base window in the y-axis direction; The function expression of the rotation transformation matrix is:

[0018] Where, is the rotation transformation matrix, Is the rotation angle of the base window; The function expression of the shear transformation matrix is:

[0019] Where, is the rotation transformation matrix, is the shear factor of the base window in the x-axis direction, is the shear factor of the base window in the y-axis direction; The function expression of the translation transformation matrix is:

[0020] Where, is the translation transformation matrix, is the translation amount of the base window in the x-axis direction, is the translation amount of the base window in the y-axis direction, is the translation factor; The function expression of the projection transformation matrix is:

[0021] Where, is the projection transformation matrix, is the projection of the basic window in the x-axis direction, Is the projection amount of the basic window in the y-axis direction; According to the decomposed scaling transformation matrix, rotation transformation matrix, shear transformation matrix, translation transformation matrix and projection transformation matrix, the reconstructed projection transformation matrix is ​​established. The function expression of the reconstructed projection transformation matrix is:

[0022] Where, is the projective transformation matrix after reconstruction.

[0023] Optionally, the dynamically converting each basis window into an arbitrary quadrilateral using the reconstructed projective transformation matrix includes: Determine the four vertices on the base window to be converted into an arbitrary quadrilateral, and obtain the original coordinates of the four vertices; The original coordinates of the four vertices are converted into three-dimensional homogeneous coordinates, and the three-dimensional homogeneous coordinates of the base window vertices before the transformation are calculated using the reconstructed projective transformation matrix to obtain the three-dimensional homogeneous coordinates of the base window vertices after the transformation; According to the three-dimensional homogeneous coordinates of the base window vertices after transformation, the positions of the four vertices of the base window after transformation are determined, and after the positions of the transformed vertices are determined, the original coordinates of any pixel point in the base window are obtained; The original coordinates of any pixel point are converted into three-dimensional homogeneous coordinates, and the reconstructed projective transformation matrix is ​​used to calculate the three-dimensional homogeneous coordinates of any pixel point in the base window before the transformation, and the three-dimensional homogeneous coordinates of any pixel point in the base window after the transformation are obtained. The function expression of the coordinate transformation of any pixel point is:

[0024]

[0025] Where, is the transformed coordinate of any pixel point in the base window after transformation, is the transformed three-dimensional homogeneous coordinate of the k-th pixel in the base window, is the three-dimensional homogeneous coordinate of any pixel point in the base window before transformation; The four vertices of the basic window and the transformed three-dimensional homogeneous coordinates of all pixels in the basic window are combined to obtain an arbitrary quadrilateral after the transformation of the basic window.

[0026] Optionally, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism, including: Extract the rotation angle from the rotation transformation matrix corresponding to each basis window; The rotation angle difference is calculated based on the rotation angle of any two basic windows. The function expression of the rotation angle difference is:

[0027] Where, is the rotation angle difference, is the rotation angle of the i-th basic window after transformation, is the rotation angle of the j-th basic window after transformation; The calculated rotation angle difference is used as the rotation perception item, and the quadrilateral attention mechanism is introduced in combination with other features to obtain the variable window attention mechanism. The function expression of the variable window attention mechanism is:

[0028] Where, is a variable window attention mechanism, is the normalization function, d is the vector dimension, is the learnable rotation sensitivity coefficient, is the dot product of the query token and the key token.

[0029] The present invention also provides a rotation-aware enhanced variable window attention super-resolution system, comprising: The original remote sensing image processing module is used to collect and obtain the original remote sensing images of the target object and perform enhancement preprocessing on the collected original remote sensing images; The feature vector extraction module is used to divide the pre-processed original remote sensing image into several basic windows and extract the feature vector from each basic window; The projective transformation matrix decomposition module is used to define the projective transformation matrix, use quadrilaterals to predict the proxy parameters of scaling, rotation, shearing, translation and projection in the projective transformation matrix according to the extracted eigenvectors, and decompose the projective transformation matrix into corresponding transformation matrices according to the predicted proxy parameters; A basic window transformation module is used to reconstruct a projective transformation matrix based on the decomposed transformation matrix, and dynamically transform each basic window into an arbitrary quadrilateral using the reconstructed projective transformation matrix; The attention mechanism establishment module is used to extract the rotation angle difference between pixels at the same position in any two base windows after transformation from the rotation transformation matrix based on the dynamic transformation of the base window. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. A network architecture building module, which is used to embed the variable window attention mechanism into the SwinIR network and apply it to the feature extraction layer to establish the architecture of a high-precision super-resolution model; The model building module is used to collect and obtain a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain the trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; The image resolution processing module is used to acquire remote sensing images of the target object in real time, input the acquired real-time remote sensing images into the high-precision super-resolution model, and output a high-resolution image containing the target sampling area.

[0030] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of a rotation-aware enhanced variable window attention super-resolution method.

[0031] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of a rotation-aware enhanced variable window attention super-resolution method.

[0032] Compared with the prior art, the present invention has the following beneficial effects: By integrating image preprocessing, projective transformation, feature extraction, an attention mechanism, and super-resolution techniques, dynamic processing of remote sensing images is achieved. First, image quality is enhanced through preprocessing. The image is then segmented into multiple basis windows, from which feature vectors are extracted. These feature vectors are then used to predict proxy parameters for the projective transformation matrix, which is then decomposed into transformation matrices such as scaling, rotation, shearing, and translation. The reconstructed projective transformation matrix is ​​used to dynamically transform the basis window into an arbitrary quadrilateral. Furthermore, the rotation angle difference is extracted through a quadrilateral attention mechanism and introduced into a variable window attention mechanism, enhancing the model's dynamic adaptability. Finally, this mechanism is embedded in the SwinIR network to construct a high-precision super-resolution model. Evaluation on training and test sets demonstrates that the model is capable of processing remote sensing images in real time and outputting high-resolution images of the target area, significantly improving the efficiency and accuracy of remote sensing image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of a method for a variable window attention super-resolution method provided by an embodiment of the present invention; Figure 2 A schematic diagram comparing the variable window attention super-resolution method provided by an embodiment of the present invention and existing methods in super-resolution reconstruction tasks; Figure 3 A schematic diagram of an original remote sensing image provided by an embodiment of the present invention being averagely segmented using a common window under a variable window; Figure 4A schematic diagram of arbitrarily segmenting an original remote sensing image using a special-shaped window under a variable window provided by an embodiment of the present invention; Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0034] The symbols in the accompanying drawings represent the following: 100. Electronic device; 101. Memory; 102. Process gas; 103. Computer program; 104. Communication bus. DETAILED DESCRIPTION

[0035] In order to further understand the content of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0036] The present invention provides a rotation-aware enhanced variable window attention super-resolution method, such as Figure 1 As shown, the following steps are included: S1. Acquire the original remote sensing image of the target object and perform enhancement preprocessing on the acquired original remote sensing image; S2, dividing the preprocessed original remote sensing image into several basic windows, and extracting a feature vector from each basic window; S3. defining a projective transformation matrix, using a quadrilateral to predict proxy parameters of scaling, rotation, shearing, translation, and projection in the projective transformation matrix according to the extracted eigenvectors, and decomposing the projective transformation matrix into corresponding transformation matrices according to the predicted proxy parameters; S4. Reconstructing a projective transformation matrix according to the decomposed transformation matrix, and dynamically transforming each basis window into an arbitrary quadrilateral using the reconstructed projective transformation matrix; S5. According to the dynamic transformation of the base window, the rotation angle difference between the pixels at the same position in any two base windows after transformation is extracted from the rotation transformation matrix. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. S6. Embed the variable window attention mechanism into the SwinIR network and apply it to the feature extraction layer to establish the architecture of a high-precision super-resolution model; S7. Acquire a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain a trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; S8. Acquire remote sensing images of the target object in real time, input the acquired real-time remote sensing images into a high-precision super-resolution model, and output a high-resolution image containing the target sampling area.

[0037] SwinIR is a Swin Transformer-based image super-resolution (SR) network. It integrates image preprocessing, projective transformation, feature extraction, an attention mechanism, and super-resolution techniques to achieve dynamic processing of remote sensing images. First, preprocessing enhances image quality. The image is then segmented into multiple basis windows, from which feature vectors are extracted. These feature vectors are then used to predict proxy parameters for the projective transformation matrix, which is then decomposed into transformation matrices for scaling, rotation, shearing, and translation. The reconstructed projective transformation matrix is ​​used to dynamically transform the basis window into an arbitrary quadrilateral. Furthermore, the quadrilateral attention mechanism is used to extract the rotation angle difference and introduce it into the variable window attention mechanism, enhancing the model's dynamic adaptability. Finally, this mechanism is embedded in the SwinIR network to construct a high-precision super-resolution model. Evaluation on training and test sets demonstrates that the model can process remote sensing images in real time and output high-resolution images of the target area, significantly improving the efficiency and accuracy of remote sensing image analysis.

[0038] Furthermore, the acquired original remote sensing images are subjected to enhancement preprocessing, including: According to the collected original remote sensing image, grayscale processing is performed on the original remote sensing image to obtain a grayscale image; Based on wavelet transform, the grayscale image is decomposed into wavelet coefficients of different frequencies, and the high-frequency coefficients are thresholded. The image is reconstructed using the high-frequency coefficients after thresholding to obtain the denoised original remote sensing image.

[0039] Furthermore, the preprocessed original remote sensing image is segmented into several basic windows, and feature vectors are extracted from each basic window, including: According to the predefined window size, the preprocessed original remote sensing image is divided into multiple basic windows of the same window size; Feature extraction is performed on each basic window, and the extracted features are converted into query tokens, key tokens, and value tokens through linear projection. The function expression of linear projection is:

[0040] Where Q represents the query token, K represents the key token, and V represents the value token. represents the linear transformation function, Represents features within the base window; The query token, key token, and value token corresponding to each basis window are concatenated to obtain the feature vector corresponding to the basis window.

[0041] Furthermore, a projective transformation matrix is ​​defined, and the proxy parameters of scaling, rotation, shearing, translation, and projection in the projective transformation matrix are predicted using quadrilaterals according to the extracted feature vectors, including: Define the projective transformation matrix. The function expression of the projective transformation matrix is:

[0042] Where, Define scaling, rotation, and shear transformations, Define the translation transformation, is the projection vector, which defines how the observer's perspective changes when the depth dimension changes; Collect existing remote sensing images with known projective transformation matrices, and build a neural network model consisting of an average pooling layer, a LeakyReLU activation layer, and a convolutional layer. Extract features from the existing remote sensing images and input them into the neural network model for training to obtain a quadrilateral prediction model. The feature vector extracted from the basic window is input into the quadrilateral prediction model, and the predicted proxy parameters of scaling, rotation, shearing, translation and projection are output. The function expression of the proxy parameters is:

[0043] Where, is the nth type of proxy parameter, is the convolutional layer, is the activation layer, is the average pooling layer, is the eigenvector after linear projection.

[0044] Furthermore, a projective transformation matrix is ​​reconstructed according to the decomposed transformation matrix, and each basis window is dynamically transformed into an arbitrary quadrilateral using the reconstructed projective transformation matrix, including: Apply the predicted proxy parameters of scaling, rotation, shearing, translation and projection to the defined projection transformation matrix, respectively, to obtain the scaling transformation matrix, rotation transformation matrix, shearing transformation matrix, translation transformation matrix and projection transformation matrix; The function expression of the scaling transformation matrix is:

[0045] Where, is the scaling transformation matrix, Is the scaling factor of the base window in the x-axis direction, Is the scaling factor of the base window in the y-axis direction; The function expression of the rotation transformation matrix is:

[0046] Where, is the rotation transformation matrix, Is the rotation angle of the base window; The function expression of the shear transformation matrix is:

[0047] Where, is the rotation transformation matrix, is the shear factor of the base window in the x-axis direction, is the shear factor of the base window in the y-axis direction; The function expression of the translation transformation matrix is:

[0048] Where, is the translation transformation matrix, is the translation amount of the base window in the x-axis direction, is the translation amount of the base window in the y-axis direction, is the translation factor; The function expression of the projection transformation matrix is:

[0049] Where, is the projection transformation matrix, is the projection of the basic window in the x-axis direction, Is the projection amount of the basic window in the y-axis direction; According to the decomposed scaling transformation matrix, rotation transformation matrix, shear transformation matrix, translation transformation matrix and projection transformation matrix, the reconstructed projection transformation matrix is ​​established. The function expression of the reconstructed projection transformation matrix is:

[0050] Where, is the projective transformation matrix after reconstruction.

[0051] The base window is used as a reference. Based on the features within the base window, it is transformed into a target quadrilateral using a projective transformation matrix. This quadrilateral covers areas of varying positions, sizes, orientations, and shapes, thus accommodating diverse targets. Because the projective transformation does not preserve parallelism, length, or orientation, the generated quadrilateral is highly flexible in position, size, orientation, and shape, enabling it to effectively cover targets of varying sizes, orientations, and shapes. The projective transformation matrix is ​​typically represented by a transformation matrix with eight parameters.

[0052] Furthermore, each basis window is dynamically transformed into an arbitrary quadrilateral using the reconstructed projective transformation matrix, including: Determine the four vertices on the base window to be converted into an arbitrary quadrilateral, and obtain the original coordinates of the four vertices; The original coordinates of the four vertices are converted into three-dimensional homogeneous coordinates, and the three-dimensional homogeneous coordinates of the base window vertices before the transformation are calculated using the reconstructed projective transformation matrix to obtain the three-dimensional homogeneous coordinates of the base window vertices after the transformation; According to the three-dimensional homogeneous coordinates of the base window vertices after transformation, the positions of the four vertices of the base window after transformation are determined, and after the positions of the transformed vertices are determined, the original coordinates of any pixel point in the base window are obtained; The original coordinates of any pixel point are converted into three-dimensional homogeneous coordinates, and the reconstructed projective transformation matrix is ​​used to calculate the three-dimensional homogeneous coordinates of any pixel point in the base window before the transformation, and the three-dimensional homogeneous coordinates of any pixel point in the base window after the transformation are obtained. The function expression of the coordinate transformation of any pixel point is:

[0053]

[0054] Where, is the transformed coordinate of any pixel point in the base window after transformation, is the transformed three-dimensional homogeneous coordinate of the k-th pixel in the base window, is the three-dimensional homogeneous coordinate of any pixel point in the base window before transformation; The four vertices of the basic window and the transformed three-dimensional homogeneous coordinates of all pixels in the basic window are combined to obtain an arbitrary quadrilateral after the transformation of the basic window.

[0055] Furthermore, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism, and a variable window attention mechanism is established, including: Extract the rotation angle from the rotation transformation matrix corresponding to each basis window; The rotation angle difference is calculated based on the rotation angle of any two basic windows. The function expression of the rotation angle difference is:

[0056] Where, is the rotation angle difference, is the rotation angle of the i-th basic window after transformation, is the rotation angle of the j-th basic window after transformation; The calculated rotation angle difference is used as the rotation perception item, and the quadrilateral attention mechanism is introduced in combination with other features to obtain the variable window attention mechanism. The function expression of the variable window attention mechanism is:

[0057] Where, is a variable window attention mechanism, is the normalization function, d is the vector dimension, is the learnable rotation sensitivity coefficient, is the dot product of the query token and the key token.

[0058] Among them, the Deformable Quadrangle Attention (DQA) method is proposed, which enables the model to dynamically determine the position, size, direction and shape of each window.

[0059] When the two windows rotate in similar directions, When , the attention score is enhanced, promoting the feature fusion of targets with similar directions; when the rotation direction difference is large, When , the attention score decays, suppressing the interference of irrelevant features.

[0060] Finally, a rotation-aware, enhanced variable-window attention mechanism is embedded in the SwinIR network, replacing the original fixed-window attention calculation to achieve high-precision super-resolution reconstruction of rotated objects. Specifically, arbitrary quadrilateral windows are dynamically generated through projective transformations, and the output features of all windows are concatenated to obtain the target sampling area.

[0061] The present invention also provides a rotation-aware enhanced variable window attention super-resolution system, comprising: The original remote sensing image processing module is used to collect and obtain the original remote sensing images of the target object and perform enhancement preprocessing on the collected original remote sensing images; The feature vector extraction module is used to divide the pre-processed original remote sensing image into several basic windows and extract the feature vector from each basic window; The projective transformation matrix decomposition module is used to define the projective transformation matrix, use quadrilaterals to predict the proxy parameters of scaling, rotation, shearing, translation and projection in the projective transformation matrix according to the extracted eigenvectors, and decompose the projective transformation matrix into corresponding transformation matrices according to the predicted proxy parameters; A basic window transformation module is used to reconstruct a projective transformation matrix based on the decomposed transformation matrix, and dynamically transform each basic window into an arbitrary quadrilateral using the reconstructed projective transformation matrix; The attention mechanism establishment module is used to extract the rotation angle difference between pixels at the same position in any two base windows after transformation from the rotation transformation matrix based on the dynamic transformation of the base window. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. A network architecture building module, which is used to embed the variable window attention mechanism into the SwinIR network and apply it to the feature extraction layer to establish the architecture of a high-precision super-resolution model; The model building module is used to collect and obtain a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain the trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; The image resolution processing module is used to acquire remote sensing images of the target object in real time, input the acquired real-time remote sensing images into the high-precision super-resolution model, and output a high-resolution image containing the target sampling area.

[0062] As described above, combined with the results shown in Tables 1 and 2, to validate the effectiveness of our proposed method, we conducted a comprehensive comparison with leading super-resolution methods based on convolutional neural networks (CNNs) and vision transformers (ViTs). The experimental setup employed two super-resolution scaling factors, 2x and 4x, for evaluation, and directly used the pre-trained models provided by the authors of each method to ensure fairness. Among CNN-based methods, we selected EDSR (Enhanced Deep Super-Resolution), RCAN (Recursive Convolutional Network), DPSR (Deep Parametric Super-Resolution), and IMDN (Image Multi-Scale Deep Network). Among Transformer-based methods, we compared LKFormer (Long-Range former with Long Distance Attention Mechanism), SwinIR (Swin Image Restoration), and TTST (Spatiotemporal Transformer). We used Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) for quantitative analysis, objectively measuring the performance of each method by calculating the PSNR and SSIM values ​​between the high-resolution ground truth image and the super-resolution reconstructed image. Experimental results show that our proposed method based on the variable window attention mechanism outperforms these comparative methods in terms of rotation perception and super-resolution reconstruction quality while maintaining computational efficiency, especially showing significant advantages when processing remote sensing images with complex geometric structures.

[0063] Table 1 Comparison of reconstruction performance between the present invention and existing super-resolution methods at 2x downsampling rate

[0064] Quantitative Evaluation: Our proposed rotation-aware enhanced variable window attention super-resolution method demonstrates significant advantages in quantitative evaluations. As shown in Table 1, on the UC Merced Land-Use dataset, with a 2x downsampling task, our method achieves a PSNR of 36.35dB and an SSIM of 0.9871 with only 2.15M parameters. While maintaining a lightweight model, this method achieves a 0.46dB PSNR improvement over the traditional CNN method EDSR and a 0.21dB improvement over the Transformer baseline SwinIR, while also achieving the best SSIM. Notably, on the NWPU (Northwestern Polytechnical University) dataset, our method achieves an SSIM of 0.9619, significantly outperforming SwinIR's 0.9386, demonstrating its ability to reconstruct complex ground structures. Compared to the larger TTST (18.94M parameters), our method achieves an 8.8x improvement in model efficiency while remaining competitive. These data fully demonstrate that through the combination of projective transformation matrix decomposition and rotation perception mechanism, the present invention effectively improves the super-resolution reconstruction accuracy and geometric consistency of rotating objects while maintaining computational efficiency.

[0065] Table 2 Comparison of the reconstruction performance of the present invention and existing super-resolution methods for different ground objects at 2x downsampling rate

[0066] As shown in Table 2, the rotation-aware enhanced variable window attention super-resolution method proposed in this paper shows comprehensive advantages in the quantitative evaluation of 13 typical ground object scenes, surpassing all comparison models with an average PSNR of 34.29dB, which is 0.18dB and 0.26dB higher than the traditional CNN methods EDSR (34.11dB) and RCAN (34.03dB), respectively. At the same time, it achieves the best performance in 9 specific scenes (such as buildings 35.20dB, golf courses 37.13dB, highways 33.01dB, etc.), especially in the reconstruction of structured objects. The performance of the proposed method is outstanding (+0.22dB for buildings and +0.29dB for highways), which verifies the strong adaptability of the projective transformation matrix decomposition and rotation perception mechanism to geometrically deformed targets. Although the improvement is small (0.06-0.09dB) in some natural scenes (such as 32.31dB in forests and 29.75dB in parks), the proposed method still maintains a stable advantage and significantly outperforms the Transformer model TTST (18.94M) with a lightweight parameter count of only 2.15M, fully demonstrating its dual advantages in model efficiency and reconstruction accuracy.

[0067] like Figure 2-Figure 4As shown in Figure 2, the reconstruction results of the proposed method are compared with those of the existing super-resolution method at a 2x downsampling rate. Qualitative evaluation: The rotation-aware variable window attention method proposed in this paper significantly improves the reconstruction quality of rotating objects in the super-resolution reconstruction task. Taking the aircraft target in the dataset as an example ( Figure 3 and Figure 4 ), the images generated by this method not only maintain clearer texture details (such as wing skin seams and engine air inlet structures), but also effectively avoid the blurred outlines of the IMDN method and the blocking artifacts of LKFormer. Compared with Transformer-based methods (SwinIR and TTST), this method exhibits stronger geometric fidelity for objects with distinct directional characteristics, such as aircraft, due to its explicit modeling of projective transformations and rotational perception mechanisms. The reconstruction results are closer to the real high-resolution image in terms of edge sharpness and structural integrity. This advantage is particularly evident in the transition area between the aircraft fuselage and the background, verifying the adaptability of the variable quadrilateral window to rotating targets. The present invention also discloses a rotation-aware enhanced variable window attention super-resolution system, comprising: The original remote sensing image processing module is used to collect and obtain the original remote sensing images of the target object and perform enhancement preprocessing on the collected original remote sensing images; The feature vector extraction module is used to divide the pre-processed original remote sensing image into several basic windows and extract the feature vector from each basic window; The projective transformation matrix decomposition module is used to define the projective transformation matrix, use quadrilaterals to predict the proxy parameters of scaling, rotation, shearing, translation and projection in the projective transformation matrix according to the extracted eigenvectors, and decompose the projective transformation matrix into corresponding transformation matrices according to the predicted proxy parameters; A basic window transformation module is used to reconstruct a projective transformation matrix based on the decomposed transformation matrix, and dynamically transform each basic window into an arbitrary quadrilateral using the reconstructed projective transformation matrix; The attention mechanism establishment module is used to extract the rotation angle difference between pixels at the same position in any two base windows after transformation from the rotation transformation matrix based on the dynamic transformation of the base window. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. A network architecture building module, which is used to embed the variable window attention mechanism into the SwinIR network and apply it to the feature extraction layer to establish the architecture of a high-precision super-resolution model; The model building module is used to collect and obtain a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain the trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; The image resolution processing module is used to acquire remote sensing images of the target object in real time, input the acquired real-time remote sensing images into the high-precision super-resolution model, and output a high-resolution image containing the target sampling area.

[0068] like Figure 5 As shown, the present invention also provides an electronic device 100 for a rotation-aware enhanced variable window attention super-resolution method; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0069] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the rotation-awareness-enhanced variable window attention super-resolution method described in Example 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data (such as audio data) created based on the use of the electronic device 100. In addition, the memory 101 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device.

[0070] The at least one processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor, etc. The processor 102 is the control center of the electronic device 100 and connects various parts of the entire electronic device 100 using various interfaces and lines.

[0071] The memory 101 in the electronic device 100 stores multiple instructions to implement a rotation-aware enhanced variable window attention super-resolution method.

[0072] The present invention also provides a module / unit integrated in the electronic device 100, which can be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as an independent product. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).

[0073] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A rotation-aware enhanced variable window attention super-resolution method, characterized by: The steps include: Acquire the original remote sensing images of the target object and perform enhancement preprocessing on the acquired original remote sensing images; The preprocessed original remote sensing image is divided into several basic windows, and a feature vector is extracted from each basic window; Define a projective transformation matrix, use quadrilaterals to predict proxy parameters of scaling, rotation, shearing, translation, and projection in the projective transformation matrix based on the extracted eigenvectors, and decompose the projective transformation matrix into corresponding transformation matrices based on the predicted proxy parameters; Reconstruct the projective transformation matrix according to the decomposed transformation matrix, and use the reconstructed projective transformation matrix to dynamically transform each basis window into an arbitrary quadrilateral; According to the dynamic transformation of the basic window, the rotation angle difference between the pixels at the same position in any two basic windows after transformation is extracted from the rotation transformation matrix. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. The variable window attention mechanism is embedded into the SwinIR network and applied to the feature extraction layer to establish the architecture of a high-precision super-resolution model; Acquire a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain the trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; The remote sensing image of the target object is acquired in real time, and the acquired real-time remote sensing image is input into the high-precision super-resolution model to output a high-resolution image containing the target sampling area.

2. The rotation-aware enhanced variable window attention super-resolution method according to claim 1, characterized in that: The enhanced preprocessing of the acquired original remote sensing image includes: According to the original remote sensing image acquired, grayscale processing is performed on the original remote sensing image to obtain a grayscale image; Based on wavelet transform, the grayscale image is decomposed into wavelet coefficients of different frequencies, and the high-frequency coefficients are thresholded. The image is reconstructed using the high-frequency coefficients after thresholding to obtain the denoised original remote sensing image.

3. The rotation-aware enhanced variable window attention super-resolution method according to claim 1, characterized in that: The pre-processed original remote sensing image is divided into a number of basic windows, and a feature vector is extracted from each basic window, including: According to the predefined window size, the preprocessed original remote sensing image is divided into multiple basic windows of the same window size; Feature extraction is performed on each basic window, and the extracted features are converted into query tokens, key tokens, and value tokens through linear projection. The function expression of linear projection is: Where Q represents the query token, K represents the key token, and V represents the value token. represents the linear transformation function, Represents features within the base window; The query token, key token, and value token corresponding to each basis window are concatenated to obtain the feature vector corresponding to the basis window.

4. The rotation-aware enhanced variable window attention super-resolution method according to claim 3, characterized in that: The projective transformation matrix is ​​defined, and the proxy parameters of scaling, rotation, shearing, translation, and projection in the projective transformation matrix are predicted using a quadrilateral according to the extracted eigenvectors, including: Define the projective transformation matrix. The function expression of the projective transformation matrix is: Where, Define scaling, rotation, and shear transformations, Define the translation transformation, is the projection vector, which defines how the observer's perspective changes when the depth dimension changes; Collect existing remote sensing images with known projective transformation matrices, and build a neural network model consisting of an average pooling layer, a LeakyReLU activation layer, and a convolutional layer. Extract features from the existing remote sensing images and input them into the neural network model for training to obtain a quadrilateral prediction model. The feature vector extracted from the basic window is input into the quadrilateral prediction model, and the predicted proxy parameters of scaling, rotation, shearing, translation and projection are output. The function expression of the proxy parameters is: Where, is the nth type of proxy parameter, is the convolutional layer, is the activation layer, is the average pooling layer, is the eigenvector after linear projection.

5. The rotation-aware enhanced variable window attention super-resolution method according to claim 4, characterized in that: The step of reconstructing a projective transformation matrix according to the decomposed transformation matrix and dynamically converting each basis window into an arbitrary quadrilateral using the reconstructed projective transformation matrix includes: Apply the predicted proxy parameters of scaling, rotation, shearing, translation and projection to the defined projection transformation matrix, respectively, to obtain the scaling transformation matrix, rotation transformation matrix, shearing transformation matrix, translation transformation matrix and projection transformation matrix; The function expression of the scaling transformation matrix is: Where, is the scaling transformation matrix, Is the scaling factor of the base window in the x-axis direction, Is the scaling factor of the base window in the y-axis direction; The function expression of the rotation transformation matrix is: Where, is the rotation transformation matrix, Is the rotation angle of the base window; The function expression of the shear transformation matrix is: Where, is the rotation transformation matrix, is the shear factor of the base window in the x-axis direction, is the shear factor of the base window in the y-axis direction; The function expression of the translation transformation matrix is: Where, is the translation transformation matrix, is the translation amount of the base window in the x-axis direction, is the translation amount of the base window in the y-axis direction, is the translation factor; The function expression of the projection transformation matrix is: Where, is the projection transformation matrix, is the projection of the basic window in the x-axis direction, Is the projection amount of the basic window in the y-axis direction; According to the decomposed scaling transformation matrix, rotation transformation matrix, shear transformation matrix, translation transformation matrix and projection transformation matrix, the reconstructed projection transformation matrix is ​​established. The function expression of the reconstructed projection transformation matrix is: Where, is the projective transformation matrix after reconstruction.

6. The rotation-aware enhanced variable window attention super-resolution method according to claim 5, characterized in that: The method of dynamically converting each basis window into an arbitrary quadrilateral using the reconstructed projective transformation matrix includes: Determine the four vertices on the base window to be converted into an arbitrary quadrilateral, and obtain the original coordinates of the four vertices; The original coordinates of the four vertices are converted into three-dimensional homogeneous coordinates, and the three-dimensional homogeneous coordinates of the base window vertices before the transformation are calculated using the reconstructed projective transformation matrix to obtain the three-dimensional homogeneous coordinates of the base window vertices after the transformation; According to the three-dimensional homogeneous coordinates of the base window vertices after transformation, the positions of the four vertices of the base window after transformation are determined, and after the positions of the transformed vertices are determined, the original coordinates of any pixel point in the base window are obtained; The original coordinates of any pixel point are converted into three-dimensional homogeneous coordinates, and the reconstructed projective transformation matrix is ​​used to calculate the three-dimensional homogeneous coordinates of any pixel point in the base window before the transformation, and the three-dimensional homogeneous coordinates of any pixel point in the base window after the transformation are obtained. The function expression of the coordinate transformation of any pixel point is: Where, is the transformed coordinate of any pixel point in the base window after transformation, is the transformed three-dimensional homogeneous coordinate of the k-th pixel in the base window, is the three-dimensional homogeneous coordinate of any pixel point in the base window before transformation; The four vertices of the basic window and the transformed three-dimensional homogeneous coordinates of all pixels in the basic window are combined to obtain an arbitrary quadrilateral after the transformation of the basic window.

7. The rotation-aware enhanced variable window attention super-resolution method according to claim 5, characterized in that: The extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism, including: Extract the rotation angle from the rotation transformation matrix corresponding to each basis window; The rotation angle difference is calculated based on the rotation angle of any two basic windows. The function expression of the rotation angle difference is: Where, is the rotation angle difference, is the rotation angle of the i-th basic window after transformation, is the rotation angle of the j-th basic window after transformation; The calculated rotation angle difference is used as the rotation perception item, and the quadrilateral attention mechanism is introduced in combination with other features to obtain the variable window attention mechanism. The function expression of the variable window attention mechanism is: Where, is a variable window attention mechanism, is the normalization function, d is the vector dimension, is the learnable rotation sensitivity coefficient, is the dot product of the query token and the key token.

8. A rotation-aware enhanced variable window attention super-resolution system, based on the rotation-aware enhanced variable window attention super-resolution method according to any one of claims 1 to 7, characterized in that: include: The original remote sensing image processing module is used to collect and obtain the original remote sensing images of the target object and perform enhancement preprocessing on the collected original remote sensing images; The feature vector extraction module is used to divide the pre-processed original remote sensing image into several basic windows and extract the feature vector from each basic window; The projective transformation matrix decomposition module is used to define the projective transformation matrix, use quadrilaterals to predict the proxy parameters of scaling, rotation, shearing, translation and projection in the projective transformation matrix according to the extracted eigenvectors, and decompose the projective transformation matrix into corresponding transformation matrices according to the predicted proxy parameters; A basic window transformation module is used to reconstruct a projective transformation matrix based on the decomposed transformation matrix, and dynamically transform each basic window into an arbitrary quadrilateral using the reconstructed projective transformation matrix; The attention mechanism establishment module is used to extract the rotation angle difference between pixels at the same position in any two base windows after transformation from the rotation transformation matrix based on the dynamic transformation of the base window. Based on the quadrilateral attention mechanism, the extracted rotation angle difference is introduced as a rotation perception item into the quadrilateral attention mechanism to establish a variable window attention mechanism. A network architecture building module, which is used to embed the variable window attention mechanism into the SwinIR network and apply it to the feature extraction layer to establish the architecture of a high-precision super-resolution model; The model building module is used to collect and obtain a remote sensing image dataset of the target object, divide the remote sensing image dataset into a training set and a test set, use the training set to input the architecture of the high-precision super-resolution model to obtain the trained high-precision super-resolution model, and use the test set to test and evaluate the trained high-precision super-resolution model; The image resolution processing module is used to acquire remote sensing images of the target object in real time, input the acquired real-time remote sensing images into the high-precision super-resolution model, and output a high-resolution image containing the target sampling area.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a rotation-aware enhanced variable window attention super-resolution method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a rotation-aware enhanced variable window attention super-resolution method according to any one of claims 1 to 7 are implemented.

Images