Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness

Abstract

This invention relates to a Gaussian super-resolution reconstruction method based on physically consistent sparsity and frequency awareness, comprising: performing feature extraction and saliency prediction on the spacecraft image to be processed to obtain a multi-scale deep feature tensor and a foreground saliency mask; using the foreground saliency mask to perform spatial feature modulation on the multi-scale deep feature tensor to obtain a modulated multi-scale deep feature tensor; decoding the feature vector corresponding to each spatial position of the modulated multi-scale deep feature tensor to predict the corresponding Gaussian kernel parameters; based on the foreground saliency mask, performing a physically consistent hard gating operation on the opacity of all Gaussian kernel parameters to generate a sparsified set of Gaussian kernel parameters; and based on the sparsified set of Gaussian kernel parameters, obtaining the final super-resolution result image through differentiable Gaussian sputtering. This method can obtain high-quality arbitrary-scale super-resolution reconstruction results while meeting the high-performance requirements of spaceborne edge devices.

Description

Technical Field

[0001] This invention belongs to the field of aerospace image processing technology, specifically involving a Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness. Background Technology

[0002] Spacecraft super-resolution (SR) imaging technology is a key means to improve the accuracy of on-orbit visual perception and plays an important role in missions such as autonomous rendezvous and docking, space debris monitoring, and on-orbit status inspection of satellite components. However, due to the limitations of the physical aperture of spaceborne cameras, the long imaging distance, and the strict constraints of the downlink bandwidth between Earth and spacecraft, directly acquired spacecraft images are often low in resolution and accompanied by noise. Therefore, there is an urgent need to recover high-resolution details at the receiving end through advanced computational imaging methods.

[0003] Existing deep learning super-resolution methods mainly follow two technical routes: one is based on convolutional neural networks (CNNs), such as RCAN and SwinIR, which learn complex mappings from low to high resolution through deep networks; the other is based on implicit neural representations (INRs), such as LIIF, which treats images as functions of continuous coordinates to color values, thereby achieving reconstruction at arbitrary scales. In addition, recently emerging super-resolution methods based on Gaussian sputtering (such as GaussianSR) borrow from the idea of ​​novel view synthesis, utilizing differentiable two-dimensional Gaussian primitives for image representation and reconstruction, demonstrating flexibility and potential in natural scenes.

[0004] However, when these methods are applied to the specific field of spacecraft imaging, their limitations become apparent: First, spacecraft images have a distinct binary physical structure of "foreground (spacecraft) - background (deep space)," with a large proportion of background and sparse information. Existing methods perform uniform calculations on the entire image, resulting in significant computational redundancy in the background region and easily introducing false noise or artifacts in the deep space region, thus compromising the physical consistency of the imaging. Second, as rigid bodies, spacecraft have a large number of regular straight edges and high-frequency surface textures (such as the edges of solar panels and the wrinkles of multi-layered thermal insulation materials). Existing Gaussian methods lack explicit constraints on the geometry of the Gaussian kernel, making it difficult to accurately fit the anisotropic edge structure, resulting in blurry or jagged reconstruction results. At the same time, commonly used spatial domain loss functions (such as L1 loss) tend to smooth high-frequency details, leading to the loss of material texture and an unrealistic "plastic" appearance of the reconstructed object surface.

[0005] In summary, existing technologies for processing spacecraft images generally face a significant contradiction between processing efficiency and reconstruction quality: on the one hand, uniform computation across the entire image generates a substantial computational load in the large deep-space background region and easily introduces artifacts into the background; on the other hand, for the spacecraft body, its regular rigid edges are prone to jaggedness or blurring after reconstruction, while the fine high-frequency material textures on the surface tend to be smoothed or even lost. These shortcomings collectively limit the practical application effectiveness of super-resolution technology in high-precision tasks such as spacecraft visual perception. Summary of the Invention

[0006] To address the aforementioned problems in existing technologies, this invention provides a Gaussian super-resolution reconstruction method based on physically consistent sparsity and frequency awareness. The technical problem to be solved by this invention is achieved through the following technical solution: This invention provides a Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness, comprising: inputting the acquired spacecraft image to be processed into a trained end-to-end super-resolution network to perform the following steps: S1: Using the feature extractor and saliency branch inside the network, feature extraction and saliency prediction are performed on the acquired spacecraft images to be processed, respectively, to obtain multi-scale deep feature tensors and foreground saliency masks; S2: Input the foreground saliency mask into the spatial feature transformation module inside the network to use the foreground saliency mask to perform spatial feature modulation on the multi-scale deep feature tensor to obtain the modulated multi-scale deep feature tensor; S3: Input the modulated multi-scale deep feature tensor into the anisotropic Gaussian parameter prediction head inside the network, decode the feature vector corresponding to each spatial position of the modulated multi-scale deep feature tensor, and predict the corresponding Gaussian kernel parameter. The Gaussian kernel parameter includes opacity. Each spatial position corresponds one-to-one with the grid point of the modulated multi-scale deep feature tensor. S4: Based on the foreground saliency mask, perform a physical consistency hard gating operation on the opacity of all Gaussian kernel parameters, forcibly setting the opacity of the background region to zero, and generating a sparsified set of Gaussian kernel parameters that retains only the foreground region. S5: Based on the sparsified Gaussian kernel parameter set, the final super-resolution result image is obtained through differentiable Gaussian sputtering.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: To address the significant trade-off between processing efficiency and reconstruction quality in existing spacecraft image processing methods, this invention provides a Gaussian super-resolution reconstruction method based on physically consistent sparsity and frequency awareness. By introducing a reconstruction paradigm that combines physically aware sparsity with multi-scale frequency decoupling, a hard-gating operation on the Gaussian kernel parameters is performed during the inference phase using a foreground saliency mask generated by the network itself. This directly eliminates invalid computations in the background region, thereby significantly reducing the computational load and fundamentally eliminating background artifacts while maintaining high-fidelity foreground reconstruction. This ensures that the reconstruction result matches the spacecraft image perfectly. The inherent foreground-background binary physical structure is consistent. At the same time, during the training phase, an innovative design of gradient alignment loss and focal frequency loss is used for multi-task joint optimization. This forces the anisotropic Gaussian kernel predicted by the network to accurately fit the edge direction of the rigid body, so as to achieve sub-pixel-level sharpening reconstruction of the edges of rigid components such as solar panels and cabins. This avoids the blurring and jaggedness caused by traditional methods. Furthermore, it dynamically focuses on restoring the spectral energy of high-frequency textures to suppress material smoothing. Ultimately, it achieves high-quality arbitrary-scale super-resolution reconstruction results with sharp edges, clear textures, and physical consistency while meeting the high-efficiency requirements of spaceborne edge devices. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating the Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness provided in an embodiment of the present invention. Figure 2 This is a data flow diagram of the Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the physical consistency sparsification and feature modulation module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the principle of gradient alignment loss and Gaussian kernel geometric constraints provided in an embodiment of the present invention. Detailed Implementation

[0009] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0010] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0011] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0012] It should be noted that the terms "low resolution" and "high resolution" in the following text are relative concepts. The specific absolute value of resolution they refer to depends on the actual application scenario and the set goals. In this invention, "low-resolution image" generally refers to an input image whose clarity needs to be improved due to hardware or transmission limitations; while "high-resolution image" refers to an image with a resolution greater than that of a low-resolution image, or an image with richer spatial details output after processing by the method of this invention. The relative relationship between the two is quantified and defined by the parameter of "super-resolution scale".

[0013] The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness proposed in this invention will now be described in detail with reference to the accompanying drawings.

[0014] Figure 1 This is a flowchart illustrating the Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness provided in this embodiment of the invention. Figure 2 This is a data flow diagram of the Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness provided in this embodiment of the invention. For example... Figure 1-2 As shown, the method includes: The acquired spacecraft images to be processed are input into the trained end-to-end super-resolution network to perform the following steps: S1: Using the feature extractor and saliency branch inside the network, feature extraction and saliency prediction are performed on the acquired spacecraft images to be processed, respectively, to obtain multi-scale deep feature tensors and foreground saliency masks; S2: Input the foreground saliency mask into the spatial feature transformation module inside the network to use the foreground saliency mask to perform spatial feature modulation on the multi-scale deep feature tensor, and obtain the modulated multi-scale deep feature tensor. S3: Input the modulated multi-scale deep feature tensor into the anisotropic Gaussian parameter prediction head inside the network, decode the feature vector corresponding to each spatial position of the modulated multi-scale deep feature tensor, and predict the corresponding Gaussian kernel parameters. The Gaussian kernel parameters include opacity. Each spatial position corresponds one-to-one with the grid point of the modulated multi-scale deep feature tensor. S4: Based on the foreground saliency mask, perform a physically consistent hard gating operation on the opacity of all Gaussian kernel parameters, forcibly setting the opacity of the background region to zero, and generating a sparsified set of Gaussian kernel parameters that retains only the foreground region. S5: Based on the sparsified Gaussian kernel parameter set, the final super-resolution image is obtained through differentiable Gaussian sputtering.

[0015] Here, the trained end-to-end super-resolution network consists of the following components: Figure 2 The modules shown are: a feature extractor for extracting multi-scale deep features, a saliency detection branch for generating a foreground saliency mask, a spatial feature transformation module for modulating features using the foreground saliency mask, a Gaussian parameter prediction head for predicting anisotropic Gaussian kernel parameters, and a rasterizer for rendering based on Gaussian kernel parameters using differentiable Gaussian sputtering; wherein, during inference, the end-to-end super-resolution network performs physical consistency hard gating on the Gaussian kernel opacity according to the saliency mask to achieve sparsity calculation of the background region.

[0016] In practical deployments, a well-trained end-to-end super-resolution network can efficiently and faithfully reconstruct input low-resolution spacecraft images. Through internally learned saliency priors, it automatically identifies and removes Gaussian kernels corresponding to deep space backgrounds during inference, thereby significantly reducing computational load, eliminating background artifacts, and greatly improving processing speed on spaceborne or edge devices. At the same time, the network's enhanced anisotropic geometric constraints and multi-scale frequency perception capabilities can effectively recover the sharp straight edges of the rigid structure of the spacecraft and the subtle high-frequency material textures on the surface, significantly improving the visual clarity and material realism of key components such as solar panels and docking mechanisms, and ultimately outputting high-resolution images of any specified scale that are rich in detail and physically consistent.

[0017] The processing steps of the trained end-to-end super-resolution network for spacecraft images are described below.

[0018] Specifically, the feature extractor comprises a residual dense network, and the saliency detection branch comprises a lightweight network consisting of at least one convolutional layer; for example, the feature extractor employs a backbone network composed of stacked standard residual dense network (RDU) modules. This network may contain multiple cascaded residual dense blocks, each of which extracts multi-scale local features through dense connections and fuses global contextual information through long skip connections across blocks, thereby outputting a multi-scale deep feature tensor that combines detailed and semantic information. The saliency detection branch, for example, consists of a lightweight three-layer cascaded convolutional layer; the first convolutional layer can use a larger convolutional kernel (e.g., 7x7) for receptive field expansion and preliminary feature extraction; the second convolutional layer can use a regular convolution (e.g., 3x3) to further fuse information; the third convolutional layer can use a 1x1 convolution to reduce the number of channels to 1, followed by a sigmoid activation function, ultimately outputting a single-channel foreground saliency probability mask with the same size as the input image. This design, while ensuring sufficient perceptual capability, greatly reduces the number of parameters and computational cost, meeting real-time requirements.

[0019] Furthermore, S1 includes: using a residual dense network to extract multi-scale deep features from the spacecraft image to be processed, and outputting a multi-scale deep feature tensor; using a lightweight network to process the spacecraft image to be processed, and outputting a single-channel foreground saliency mask, wherein the value of each pixel in the foreground saliency mask represents the probability that the pixel belongs to the foreground spacecraft region.

[0020] For example, residual dense networks, through their internal residual and dense connection structures, progressively abstract and fuse input images at multiple levels, thereby capturing multi-scale visual features from local details to global semantics. The final output multi-scale deep feature tensor is a three-dimensional data structure with multiple channels and a specific spatial dimension, encoding rich texture, shape, and contextual information necessary for subsequent reconstruction. This multi-scale deep feature tensor is a three-dimensional tensor in terms of data form. Where c is the number of channels, each channel can be considered a "feature detector" specifically responding to a particular visual pattern (such as edges in a specific direction, textures with a specific period, or shapes of a specific category). Spatial dimension It still maintains a clear correspondence with the input image, forming a dense feature grid. Each position on the grid... A C-dimensional vector is a comprehensive description of the corresponding region in the input image.

[0021] This lightweight network efficiently processes the input image using a feedforward approach. Its final layer maps the output to a single-channel probability map, i.e., a foreground saliency mask, through a sigmoid activation function. This mask is spatially aligned with the input image, where the value at each pixel location (between 0 and 1) represents the confidence probability that the model determines that the pixel belongs to a foreground spacecraft target.

[0022] It should be noted that the two sub-processes share the same input image, but their calculations are independent of each other, ultimately producing feature tensors and probability masks that need to be used together in subsequent processes.

[0023] S2 includes: inputting the foreground saliency mask into the spatial feature transformation module to predict affine transformation parameters; using the affine transformation parameters to perform channel-wise scaling and bias transformation on the multi-scale deep feature tensor to enhance the features of the foreground region and suppress the features of the background region, thereby obtaining the modulated multi-scale deep feature tensor.

[0024] Specifically, the spatial feature transformation module receives a foreground saliency mask as conditional input. It encodes and analyzes this mask through a small parameter prediction network (typically composed of several convolutional layers) to predict a set of affine transformation parameters matching the number of channels in the multi-scale deep feature tensor. This set of parameters includes a scaling factor γ and a bias term β, where γ is a vector with the same dimension as the feature tensor channels, and β is a vector with the same dimension as the feature tensor channels. Subsequently, the module performs feature modulation. For each spatial location (h, w) and each channel c in the multi-scale deep feature tensor, the following formula is applied: ; in, It refers to the modulated multi-scale deep feature tensor. It refers to multi-scale deep feature tensors.

[0025] Here, features corresponding to high-probability (foreground) regions in the mask are selectively enhanced and activated, while features of low-probability (background) regions are suppressed or normalized. In this way, the spatial feature transformation module directly injects high-level semantics (foreground / background distinction) into the deep feature representation, guiding the subsequent Gaussian parameter prediction head to focus more on the reconstruction of the foreground target and initially weakening the response of the background region, laying the foundation for achieving physically consistent sparsity.

[0026] Here, it's important to note that the anisotropic Gaussian parameter prediction head in S3 is a lightweight neural network module (typically composed of several convolutional or fully connected layers). Its core function is to "decode" the input feature vector, mapping it to a set of geometric and appearance parameters with clear physical meaning required to define the two-dimensional Gaussian kernel. For each spatial location (i.e., a point on the feature grid) in the modulated multi-scale deep feature tensor, its corresponding c-dimensional feature vector is independently input into the prediction head. The prediction head outputs a structured parameter tuple for that location, which fully defines a two-dimensional Gaussian kernel placed near that location. These parameters include: 1) Center offset: A two-dimensional vector (Δx, Δy) represents the sub-pixel-level fine offset of the Gaussian kernel center relative to the standard position of the feature grid point. This allows the reconstruction to overcome the resolution limitations of the input feature grid.

[0027] 2) Covariance matrix parameters: Used to define an anisotropic two-dimensional Gaussian kernel shape. This is typically achieved by predicting a rotation angle θ and two non-negative scaling factors (sx, sy). The rotation angle controls the orientation of the Gaussian kernel, while different scaling factors cause it to stretch to different degrees in the x and y directions, thus enabling the representation of various shapes from circles to extremely elongated ellipses, which is key to accurately fitting the edges of straight lines.

[0028] 3) Color coefficient: a three-dimensional vector (R, G, B) that defines the color value contributed by the Gaussian kernel.

[0029] 4) Opacity: A scalar value α, ranging from 0 to 1, representing the transparency (or visibility) of the Gaussian kernel.

[0030] After obtaining the Gaussian kernel parameters corresponding to each spatial location, execute S4.

[0031] Figure 3 This is a schematic diagram of the structure of the physical consistency sparsity and feature modulation module provided in an embodiment of the present invention. Figure 3 As shown, here, the lightweight network consists of two convolutional layers; S4 includes: binarizing the foreground saliency mask according to a preset saliency threshold to generate a binary mask; multiplying the binary mask element-wise with the opacity of each Gaussian kernel parameter to force the opacity of the background region to be set to zero, thus obtaining a set of Gaussian kernel parameters after dot product processing; removing all Gaussian kernel parameters with zero opacity from the set of Gaussian kernel parameters after dot product processing to form a sparsed set of Gaussian kernel parameters.

[0032] Specifically, based on a pre-set saliency threshold τ (e.g., τ=0.1), the continuous-value foreground saliency mask M is binarized to generate a hard decision mask. ;For example, ,but , ,but Where p represents the spatial location on the feature map.

[0033] Then, the binary mask is applied. The dot product operation is performed on the Gaussian kernel opacity parameter α predicted by S3 for each position (i.e., element-wise). The specific calculation formula is as follows: ; in, It is the transparency after dot product processing. It refers to the opacity at spatial location p. It is a binary mask at spatial location p.

[0034] Here, for those judged as background ( The position of the () regardless of its original opacity value, its output Forced to be set to 0; for foreground positions ( The opacity remains unchanged. This multiplication operation is called "physically consistent hard gating," which is mathematically equivalent to turning off the visibility of all Gaussian kernels in the background region.

[0035] After hard gating, the opacity parameter of the Gaussian kernel corresponding to all background positions is zero. In actual engineering implementations (especially on resource-constrained edge devices), a key optimization operation is also performed: directly based on... The complete set of Gaussian kernel parameters is indexed and filtered, retaining only... All parameters corresponding to the foreground position (including center, covariance, color, and opacity) constitute a completely new sparse Gaussian kernel parameter set with a significantly reduced number of parameters. .

[0036] Here, the Gaussian kernel parameters also include: center position offset, covariance matrix parameters used to control shape and orientation, and color coefficients; S5 includes: Based on the specified target super-resolution scale, a sampling coordinate grid in high-resolution space is determined. The sampling coordinate grid is calculated based on the resolution size of the spacecraft image to be processed and the target super-resolution size. Each Gaussian kernel parameter in the sparsified Gaussian kernel parameter set is projected onto the sampling coordinate grid according to its corresponding center position offset and covariance matrix parameter. Through a differentiable Gaussian sputtering algorithm, the color coefficients and opacities of all Gaussian kernel parameters on the sampling coordinate grid are mixed and calculated in depth order to generate the final super-resolution result image.

[0037] For example, a sampling coordinate grid is first constructed, assuming the user-specified target super-resolution scale is s, and the resolution size of the spacecraft image to be processed is (H lr W lr If the target super-resolution size is H, then the target super-resolution size is expressed as (H). hr W hr )=(s*H lr , s*W lr Subsequently, within a continuous coordinate range of the high-resolution image space, a uniformly discrete sampling coordinate grid covering the entire output region is generated. This grid consists of (H... hr ×W hr The grid consists of 1,000 coordinate points, each of which corresponds precisely to a pixel in the output image. Since the Gaussian function is continuously defined, changing the density of this grid (i.e., changing s) directly controls the output resolution without retraining or adjusting network parameters, thus achieving true arbitrary-scale super-resolution.

[0038] For each Gaussian kernel in the sparse Gaussian kernel parameter set, the spatial distribution range of the two-dimensional Gaussian kernel on the image plane is calculated based on its center position μ (obtained by adding the predicted offset (Δx, Δy) to the base grid position) and the covariance matrix Σ (calculated by the rotation angle θ and the scaling factor (sx, sy)). For each sampling grid point (i.e. pixel) falling within the influence region, the weight of the Gaussian kernel at this point is calculated, which means that a continuous Gaussian kernel is "splashed" onto the discrete pixel grid.

[0039] All Gaussian kernels are ordered according to a preset or learnable depth value (or based on their center position). For each pixel, the contributions of all Gaussian kernels covering that pixel are alpha-mixed in reverse depth order. Because the set of Gaussian kernel parameters used for rendering is sparse (containing only foreground objects) and each Gaussian kernel only affects local pixels, this rendering process is significantly more computationally efficient than traditional full-image upsampling networks. Furthermore, the anisotropic covariance matrix allows the Gaussian kernels to accurately fit edge contours.

[0040] The above describes the processing steps of a trained end-to-end super-resolution network. The training process of the end-to-end super-resolution network model will now be described in detail.

[0041] The end-to-end super-resolution network is trained in the following way: Obtain the training dataset, which includes multiple pairs of spacecraft images. Each pair of spacecraft images consists of a first-resolution image and a second-resolution image of the same scene; the first resolution is smaller than the second resolution. Initialize all trainable parameters of the end-to-end super-resolution network. During multiple rounds of iterative training, the following steps are repeated: T1: Sample a batch of first-resolution images from the training dataset, input them into the current network, and execute S1-S5 in claim 1 to obtain the corresponding predicted image batch; wherein, the corresponding multiple second-resolution images are used as the real image batch; T2: Calculate the pixel-level difference between the predicted image batch and the real image batch as the basic reconstruction loss; T3: Based on the covariance matrix parameters of all Gaussian kernel parameters corresponding to the predicted image batch, calculate the gradient alignment loss between the predicted image batch and the real image batch in the edge region; T4: Transform the predicted image batch and the real image batch to the frequency domain, calculate the difference in energy distribution between the predicted image batch and the real image batch in the frequency domain, and obtain the focal frequency loss; T5: Weighted fusion of basic reconstruction loss, gradient alignment loss, and focal frequency loss yields a hybrid loss; T6: Based on the hybrid loss, update all trainable parameters in the current model in reverse; Training is stopped when the network's performance on the validation set converges, and the trained end-to-end super-resolution network is obtained.

[0042] It should be noted that all trainable parameters of an end-to-end super-resolution network refer to all variables that need to be adjusted using the gradient descent algorithm throughout the complete forward computation path of the network, from feature extraction, saliency assessment, feature modulation to Gaussian parameter prediction. For example, this includes parameters within the following modules: 1. Feature extractor: The weights of all convolutional layers in the residual dense network, the parameters of the batch normalization layer (if used), and the parameters of the relevant connection layers.

[0043] 2. Significance Detection Branch: Weights and biases of all convolutional layers (or other layers) in this lightweight network.

[0044] 3. Spatial Feature Transformation Module: All weights and biases in a small network used to predict affine transformation parameters from a saliency mask.

[0045] 4. Anisotropic Gaussian parameter prediction head: Used to decode modulated features into all weights and biases in the network layer with Gaussian kernel parameters (center, covariance, color, opacity).

[0046] The parameters mentioned above are initialized at the beginning of training, and then jointly optimized using the backpropagation algorithm by minimizing the mixture loss function. After training is complete, these parameters are fixed, forming a "trained" network used for efficient reconstruction during the inference phase.

[0047] Figure 4This is a schematic diagram illustrating the principle of gradient alignment loss and Gaussian kernel geometric constraints provided in an embodiment of the present invention. Figure 4 As shown, the left side shows the directional difference between the gradient direction in the real image and the corresponding predicted principal axis direction in the gradient field, and the right side shows the flowchart for calculating the gradient alignment loss.

[0048] Reference Figure 4 Therefore, T3 includes: calculating the gradient field of each real image in the batch of real images using an edge detection operator, obtaining the gradient magnitude and gradient direction of each pixel; filtering out pixels whose gradient magnitude exceeds a preset edge threshold to form an edge pixel set; for each edge pixel in the edge pixel set, determining its corresponding Gaussian kernel parameter in the batch of predicted images based on spatial mapping relationships; calculating the prediction principal axis direction of the covariance matrix of the Gaussian kernel parameters associated with each edge pixel; for each edge pixel, calculating the directional difference between its gradient direction in the real image and its corresponding prediction principal axis direction; and summing up the directional differences of all edge pixels to obtain the gradient alignment loss.

[0049] In one possible implementation, the expression for calculating the directional difference corresponding to each edge pixel is: ,in, The Gaussian kernel predicts the principal axis direction. It represents the gradient direction of the actual image.

[0050] It should be noted that the spatial mapping relationship is determined by the coordinate transformations during the network's forward propagation (i.e., executing S1-S5). Since the network output (predicted image) and the ground truth label (real image) have the same spatial dimensions, this mapping is essentially a one-to-one correspondence between positions at the same resolution. That is, the gradient information of the pixel with coordinates (h, w) in the real image will be used to constrain the geometry of the Gaussian kernel parameters predicted by the network at the corresponding position (h, w) in the feature map.

[0051] Furthermore, in practice, the directional difference is measured by calculating the absolute sine of the angle between the Gaussian kernel prediction principal axis direction θ and the real image gradient direction φ. The angle is minimized to 0 when the two directions are parallel and maximized to 1 when they are perpendicular, thus minimizing the alignment loss and forcing the Gaussian kernel's extension direction to align with the tangent direction of the image edge. The calculated directional difference (loss) is then transformed into the gradient of the relevant layer weights in the anisotropic Gaussian parameter prediction head through backpropagation, driving the network to learn and predict anisotropic Gaussian kernels that are geometrically aligned with the image edge structure.

[0052] It should be noted that when calculating the gradient alignment loss in batch training mode, the above-mentioned directional difference calculation and summation are performed independently and in parallel for the corresponding positions of all samples in the batch. The final gradient alignment loss is the average or sum of the directional differences of all samples and all edge pixels in the batch.

[0053] Here, T4 includes: performing two-dimensional fast Fourier transform on the batch of real images and the batch of predicted images respectively to obtain the corresponding real spectrum and predicted spectrum; calculating the amplitude spectrum difference between the real spectrum and the predicted spectrum; generating a dynamic weight matrix based on the amplitude spectrum difference; and using the dynamic weight matrix to perform weighted calculation on the amplitude spectrum difference to obtain the focal frequency loss.

[0054] Here, the focal frequency loss function essentially acts as the "attention" module of frequency domain awareness, and the key innovation of focal frequency loss is "generating a dynamic weight matrix based on amplitude spectrum differences." The specific mechanism is as follows: using the amplitude spectrum differences themselves as a basis, a nonlinear transformation (e.g., exponentiation) is used to generate the weight matrix. , For amplitude spectrum differences, The focus factor is set to 1.0. The core of this design is that the more significant the difference in a frequency component, the higher the value of its corresponding weight matrix will be. This allows the model to obtain a larger gradient signal during backpropagation to correct these difficult-to-recover frequency components (especially the high-frequency components representing texture), thereby forcing the network's learning focus to shift towards high-frequency details.

[0055] It's important to note that calculating the amplitude spectrum difference quantitatively reflects the deviation between the reconstructed image and the ground truth image in terms of frequency domain energy distribution. Each value on the amplitude spectrum difference map represents the absolute difference between the energy intensity of the predicted image and the energy intensity of the ground truth image at a specific spatial frequency. The larger the difference value, the greater the reconstruction error of the model at that frequency component. The overall level of the difference map directly measures the deficiencies of the reconstructed image in terms of global consistency in the frequency domain. Furthermore, in the Fourier spectrum of an image, low-frequency components (near the center) correspond to smooth regions and macroscopic contours, while high-frequency components (far from the center) correspond to edges, details, and complex textures. Therefore, large values ​​appearing in the high-frequency region (outer periphery) of the amplitude spectrum difference map directly and objectively quantify the degree of failure of the model in recovering high-frequency texture details (such as wrinkles in spacecraft MLI materials and the microstructure of surface markings). This is precisely the root cause of the "plastic-like" smoothness phenomenon in the reconstruction results, which is ignored or insufficiently penalized by traditional spatial domain losses (such as L1 Loss).

[0056] Based on the above, the formula for calculating the mixture loss for each iteration is as follows: ; in, It is the mixed loss corresponding to each iteration. It is a loss from basic reconstruction. It is gradient alignment loss. It is focal frequency loss. and It is a preset value, for example It is 0.05. It is 0.1.

[0057] To address the significant trade-off between processing efficiency and reconstruction quality in existing spacecraft image processing methods, this invention provides a Gaussian super-resolution reconstruction method based on physically consistent sparsity and frequency awareness. By introducing a reconstruction paradigm that combines physically aware sparsity with multi-scale frequency decoupling, a hard-gating operation on the Gaussian kernel parameters is performed during the inference phase using a foreground saliency mask generated by the network itself. This directly eliminates invalid computations in the background region, thereby significantly reducing the computational load and fundamentally eliminating background artifacts while maintaining high-fidelity foreground reconstruction. This ensures that the reconstruction result matches the spacecraft image perfectly. The inherent foreground-background binary physical structure is consistent. At the same time, during the training phase, an innovative design of gradient alignment loss and focal frequency loss is used for multi-task joint optimization. This forces the anisotropic Gaussian kernel predicted by the network to accurately fit the edge direction of the rigid body, so as to achieve sub-pixel-level sharpening reconstruction of the edges of rigid components such as solar panels and cabins. This avoids the blurring and jaggedness caused by traditional methods. Furthermore, it dynamically focuses on restoring the spectral energy of high-frequency textures to suppress material smoothing. Ultimately, it achieves high-quality arbitrary-scale super-resolution reconstruction results with sharp edges, clear textures, and physical consistency while meeting the high-efficiency requirements of spaceborne edge devices.

[0058] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness, characterized in that, include: The acquired spacecraft images to be processed are input into the trained end-to-end super-resolution network to perform the following steps: S1: Using the feature extractor and saliency branch inside the network, feature extraction and saliency prediction are performed on the acquired spacecraft images to be processed, respectively, to obtain multi-scale deep feature tensors and foreground saliency masks; S2: Input the foreground saliency mask into the spatial feature transformation module inside the network to use the foreground saliency mask to perform spatial feature modulation on the multi-scale deep feature tensor to obtain the modulated multi-scale deep feature tensor; S3: Input the modulated multi-scale deep feature tensor into the anisotropic Gaussian parameter prediction head inside the network, decode the feature vector corresponding to each spatial position of the modulated multi-scale deep feature tensor, and predict the corresponding Gaussian kernel parameter. The Gaussian kernel parameter includes opacity. Each spatial position corresponds one-to-one with the grid point of the modulated multi-scale deep feature tensor. S4: Based on the foreground saliency mask, perform a physical consistency hard gating operation on the opacity of all Gaussian kernel parameters, forcibly setting the opacity of the background region to zero, and generating a sparsified set of Gaussian kernel parameters that retains only the foreground region. S5: Based on the sparsified Gaussian kernel parameter set, the final super-resolution image is obtained through differentiable Gaussian sputtering.

2. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 1, characterized in that, The feature extractor includes a residual dense network, and the saliency detection branch includes a lightweight network consisting of at least one convolutional layer; S1 includes: Using the residual dense network, multi-scale deep features of the spacecraft image to be processed are extracted and output as the multi-scale deep feature tensor; The lightweight network is used to process the spacecraft image to be processed, and outputs a single-channel foreground saliency mask, wherein the value of each pixel in the foreground saliency mask represents the probability that the pixel belongs to the foreground spacecraft region.

3. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 1, characterized in that, S2 includes: The foreground saliency mask is input into the spatial feature transformation module to predict the affine transformation parameters; The multi-scale deep feature tensor is subjected to channel-wise scaling and bias transformation using the affine transformation parameters to enhance the features of the foreground region and suppress the features of the background region, thereby obtaining the modulated multi-scale deep feature tensor.

4. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 1, characterized in that, S4 includes: Based on a preset saliency threshold, the foreground saliency mask is binarized to generate a binary mask; The binary mask is multiplied element-wise with the opacity of each Gaussian kernel parameter to force the opacity of the background region to be set to zero, thus obtaining the set of Gaussian kernel parameters after dot product processing. Remove all Gaussian kernel parameters with zero opacity from the Gaussian kernel parameter set after the dot product processing to form the sparsified Gaussian kernel parameter set.

5. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 1, characterized in that, The Gaussian kernel parameters also include: center position offset, covariance matrix parameters for controlling shape and orientation, and color coefficients; S5 includes: Based on the specified target super-resolution scale, a sampling coordinate grid in high-resolution space is determined; wherein, the sampling coordinate grid is calculated based on the resolution size of the spacecraft image to be processed and the target super-resolution size; Each Gaussian kernel parameter in the sparsified Gaussian kernel parameter set is projected onto the sampling coordinate grid according to its corresponding center position offset and covariance matrix parameter. The color coefficients and opacities of all Gaussian kernel parameters on the sampling coordinate grid are mixed and calculated in depth order using a differentiable Gaussian sputtering algorithm to generate the final super-resolution image.

6. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 1, characterized in that, The end-to-end super-resolution network was trained in the following way: Obtain a training dataset, which includes multiple pairs of spacecraft images. Each pair of spacecraft images consists of a first-resolution image and a second-resolution image of the same scene; the first resolution is smaller than the second resolution. Initialize all trainable parameters of the end-to-end super-resolution network; During multiple rounds of iterative training, the following steps are repeated: T1: Sample a batch of first-resolution images from the training dataset, input them into the current network, and execute S1-S5 as described in claim 1 to obtain the corresponding predicted image batch; wherein, the corresponding multiple second-resolution images are used as the real image batch; T2: Calculate the pixel-level difference between the predicted image batch and the real image batch as the basic reconstruction loss; T3: Based on the covariance matrix parameters of all Gaussian kernel parameters corresponding to the predicted image batch, calculate the gradient alignment loss between the predicted image batch and the real image batch in the edge region; T4: Convert the predicted image batch and the real image batch to the frequency domain, calculate the difference in frequency domain energy distribution between the predicted image batch and the real image batch, and obtain the focal frequency loss; T5: Weighted fusion of the basic reconstruction loss, the gradient alignment loss, and the focal frequency loss yields a hybrid loss; T6: Based on the hybrid loss, update all trainable parameters in the current model in reverse; When the network's performance on the validation set converges, training is stopped, and the trained end-to-end super-resolution network is obtained.

7. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 6, characterized in that, The T3 includes: The gradient field of each real image in the batch of real images is calculated using an edge detection operator to obtain the gradient magnitude and gradient direction of each pixel. Pixels whose gradient magnitude exceeds a preset edge threshold are selected to form an edge pixel set; For each edge pixel in the set of edge pixels, the Gaussian kernel parameter corresponding to it in the batch of predicted images is determined according to the spatial mapping relationship; Calculate the prediction principal axis direction of the covariance matrix of the Gaussian kernel parameters associated with each edge pixel; For each edge pixel, calculate the directional difference between its gradient direction in the real image and the corresponding predicted principal axis direction; The gradient alignment loss is obtained by summing up the directional differences of all edge pixels.

8. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 6, characterized in that, The T4 includes: Two-dimensional fast Fourier transform is performed on the real image batch and the predicted image batch respectively to obtain the corresponding real spectrum and predicted spectrum; Calculate the amplitude spectrum difference between the actual spectrum and the predicted spectrum; A dynamic weight matrix is ​​generated based on the amplitude spectrum differences; The focal frequency loss is obtained by weighting the amplitude spectrum difference using the dynamic weight matrix.

9. The Gaussian super-resolution reconstruction method based on physical consistency sparsity and frequency awareness according to claim 6, characterized in that, The formula for calculating the mixture loss for each iteration is: ； in, It is the mixture loss corresponding to each iteration. It is a loss from basic reconstruction. It is gradient alignment loss. It is focal frequency loss. and It is the default value.

Images