Visible light remote sensing image data enhancement method and system based on unified domain modeling

Abstract

The invention discloses a visible light remote sensing image data enhancement method and system based on unified domain modeling, and belongs to the field of image data enhancement. The method comprises the following steps: modeling based on a physical degradation process of an optical transmission link to obtain a point spread function; judging whether the image size of the clear image is greater than a set threshold; and under the condition that the image size of the clear image is not greater than a set threshold value, acting the point spread function on the clear image, and obtaining a data enhancement result of the clear image in combination with spatial variable convolution. The method is designed for remote sensing visible light image data enhancement, and remote sensing image real degradation modeling, controllable degradation synthesis and high-fidelity data construction are achieved.

Description

Technical Field

[0001] This invention relates to the field of image data enhancement, and more specifically to a method and system for enhancing visible light remote sensing image data based on unified domain modeling. Background Technology

[0002] In the field of remote sensing image processing, data augmentation refers to a technique that expands the original training data through a series of strategic transformation methods (geometric transformation, spectral perturbation, image degradation modeling, etc.) without changing the semantic information of the original image, generating representative new samples to increase data diversity, improve model generalization ability, and reduce the risk of overfitting.

[0003] For remote sensing tasks, image quality is often affected by complex degradation factors in the imaging process. These factors collectively lead to spatially variable, non-uniform, and complexly coupled blurry features in images. Traditional data augmentation methods such as rotation, flipping, and noise addition can improve model generalization, but they cannot effectively simulate the actual degradation processes in remote sensing images. This results in insufficient robustness of the model when facing complex real-world scenes, limiting its generalization performance. Therefore, in remote sensing image data augmentation, constructing images that conform to the real degradation distribution is crucial for improving the quality of augmented data, enhancing model adaptability, and improving task performance.

[0004] The remote sensing data generated by this method has wide application value in many fields, including but not limited to: training data synthesis for remote sensing image restoration models such as deblurring, dehazing, and super-resolution; data augmentation and evaluation benchmark construction for tasks such as remote sensing image recognition and change detection; and simulation data completion and expansion in scenarios such as cross-domain recognition and few-shot learning.

[0005] Existing degraded image generation methods can be divided into three categories: deep learning-based generation methods, point spread function (PSF)-based generation methods, and methods that combine PSF modeling with deep learning.

[0006] Deep learning-based generative methods rely on existing image data, utilizing structures such as convolutional neural networks (CNNs), generative adversarial networks (GANs), and diffusion models to learn the feature distribution of images, directly generating image samples with specific semantic or distributional characteristics in an end-to-end manner. These methods are suitable for image enhancement and generation tasks under large-scale data conditions, possessing advantages such as strong adaptability and learning ability. However, they suffer from weak interpretability and insufficient controllability of degradation in simulating specific physical imaging processes and controlling blur mechanisms or degradation causes. In particular, deep learning-based image generation methods mainly rely on neural networks to learn image distribution characteristics, without considering the physical processes in the actual remote sensing image imaging chain, lacking physical interpretability. The blur degree and degradation causes of the synthesized images are uncontrollable, and training depends on a large amount of high-quality remote sensing data, which is difficult to obtain and has insufficient coverage.

[0007] Generative methods based on point spread function (PSF) modeling simulate the PSF of an image imaging system, performing degradation operations on an ideal image to obtain an image that approximates the real imaging process. Specifically, PSF-based generative methods can be divided into three categories: non-parametric models, parametric models, and optical simulation models. Non-parametric models model the blurring process as a point spread function (PSF) distribution in two-dimensional space, obtaining spatially varying PSFs through sparse sampling in the field of view. Parametric models describe the PSF distribution using finite parameters (such as Gaussian kernels, ellipsoidal functions, Efficient Filter Flow, Zernike polynomials, and Seidel aberration models). Optical simulation models, based on the actual lens structure of the optical system, combine ray tracing or diffraction propagation models to accurately calculate the PSF distribution under specific configurations. However, while PSF-based generative methods have a certain physical basis, they also have several limitations. Nonparametric models employ sparse, independently sampled PSF distributions, neglecting the structural relationships between image pixels and the continuity of the field of view. Parametric models, such as Gaussian kernels and Zernike polynomials, can only fit the degradation process with a limited number of parameters, resulting in limited adaptability and difficulty in accurately describing the causes of high-dimensional, multi-source blurring. While optical simulation models have high accuracy, they heavily rely on complete lens parameters and are mostly based on ideal optical systems, making it difficult to cover the actual complex factors in remote sensing imaging, such as the reflectivity of ground objects, atmospheric disturbances, and system assembly errors.

[0008] With the development of computational imaging and deep learning technologies, the integration of point spread function (PSF) modeling with deep learning has become a trend. PSF, as an important tool for modeling imaging degradation processes, is increasingly being incorporated into the design and training of neural networks. This method of fusing physically interpretable PSF modeling with neural networks can effectively improve the accuracy and generalization ability of image degradation modeling. On the one hand, PSF simulation based on optical principles provides the network with accurate degradation priors; on the other hand, the powerful learning ability of neural networks helps to compensate for and optimize degradation processes that are spatially complex and difficult to model directly. Through this fusion approach, the system can achieve more realistic imaging degradation modeling and end-to-end learning. While the integration of PSF and deep learning can improve the realism and interpretability of degradation modeling and compensate for and optimize complex degradation processes that are difficult to model directly, most of these methods are based on general vision task designs, focusing on optical aberrations introduced by lens systems, and neglecting the more complex degradation factors in the remote sensing image imaging chain. Therefore, existing methods have significant shortcomings in realistic modeling, controllable synthesis, and high-fidelity data construction of multi-source physical degradation in remote sensing images. Summary of the Invention

[0009] To address the problems existing in the prior art, the present invention aims to provide a method and system for enhancing visible light remote sensing image data based on unified domain modeling. Designed specifically for enhancing visible light remote sensing image data, it achieves realistic degradation modeling, controllable degradation synthesis, and high-fidelity data construction of remote sensing images, providing a more unified, interpretable, and adaptable method for generating remote sensing image degradation models. The unified domain has two meanings: first, it unifies the consideration of physical optics and geometric optics in terms of light characteristics, utilizing both ray tracing and wavefront distribution; second, it considers the entire domain of light transmission from object to image, simultaneously taking into account the combined effects of ground microstructure, atmospheric disturbances, and optical aberrations.

[0010] To achieve the above objectives, the technical solution of the present invention includes the following:

[0011] A visible light remote sensing image data augmentation method based on unified domain modeling, the method comprising:

[0012] The point spread function is obtained by modeling the physical degradation process of the optical transmission link.

[0013] Determine whether the size of a clear image exceeds a set threshold;

[0014] If the image size of the clear image is no larger than a set threshold, the point spread function is applied to the clear image, and spatially variable convolution is combined to obtain the data augmentation result of the clear image.

[0015] Furthermore, the pixel diffusion function is obtained by modeling the physical degradation of the optical transmission link, including:

[0016] Based on the wavefront perturbation function of ground object reflection, the wavefront perturbation function of environmental propagation, and the aberration function within the optical system, the total wavefront aberration function is obtained.

[0017] Based on the total wavefront aberration function and amplitude function, the pupil function at the exit pupil position of the optical system is obtained;

[0018] Perform a two-dimensional Fourier transform on the pupil function to calculate the complex amplitude distribution on the image plane;

[0019] Obtain the pixel spread function, which is the squared modulus of the complex amplitude distribution of the image plane.

[0020] Furthermore, the generation process of the wavefront perturbation function reflected by the ground object includes:

[0021] A target power spectrum is constructed in the frequency domain, which satisfies a two-dimensional Gaussian process power distribution.

[0022] Generate a complex white noise field;

[0023] Based on the target power spectrum and the complex white noise field, the modulation spectrum amplitude of the ground object reflection is obtained;

[0024] Perform a two-dimensional inverse Fourier transform on the modulation spectrum amplitude of ground object reflection to obtain the wavefront perturbation function of ground object reflection.

[0025] Furthermore, the generation process of the wavefront perturbation function of the environmental propagation includes:

[0026] Construct the Kolmogorov power spectral density function;

[0027] Generate a complex white noise spectral field with a unit Gaussian distribution in amplitude and a uniform phase distribution in the interval [0, 2π).

[0028] Based on the Kolmogorov power spectral density function and the complex white noise field, the modulation spectrum amplitude of environmental propagation is obtained;

[0029] Perform a two-dimensional inverse Fourier transform on the modulation spectrum amplitude of the environmental propagation to obtain the wavefront perturbation function of the environmental propagation.

[0030] Furthermore, the generation process of the aberration function within the optical system includes:

[0031] Acquire structural parameters and assembly disturbance information of the optical system;

[0032] The structural parameters and assembly perturbation information are input into a multilayer perceptron to obtain the weight 'a' for each Zernike mode. j , j is a positive integer; where, training the multilayer perceptron is based on the mean square error between the true weights and the predicted weights of the Zernike pattern;

[0033] Based on the weight a j By weighting each Zernike mode, the aberration function inside the optical system is obtained.

[0034] Furthermore, the point spread function is applied to the sharpened image, and spatially variable convolution is combined to obtain the data augmentation result of the sharpened image, including:

[0035] Extracting local image patches around pixel location (x,y) in a sharp image x,y ;

[0036] The convolution kernel corresponding to the pixel position (x, y) is selected based on the point spread function;

[0037] Based on the convolution kernel and the image local patch x,y Perform pixel-by-pixel weighted summation to obtain the data augmentation result of the clear image.

[0038] Furthermore, the method also includes:

[0039] If the image size of the clear image is larger than a set threshold, the clear image is divided into blocks based on the image size and the GPU memory size;

[0040] For each small block, the point diffusion function is applied to the small block in parallel processing on different GPU threads, and spatially variable convolution is combined to obtain the data augmentation result of the small block;

[0041] The data augmentation results of the small data blocks are merged to obtain the data augmentation result of the clear image.

[0042] Furthermore, the sharp image is divided into blocks based on its image size and GPU memory size, including:

[0043] Obtain the dimensions H×W of the clear image, the patch size h×w, and the overlap size between patches o. h ×o w Where H represents the height of the sharp image, W represents the width of the sharp image, h represents the height of the patch, w represents the width of the patch image, and o h Indicates the height of the overlapping region, o w Indicates the width of the overlapping region;

[0044] Based on the dimensions H×W of the clear image, the patch size h×w, and the size of the overlapping area between patches.h ×o w Calculate the number of small blocks M in the vertical direction and the number of small blocks N in the horizontal direction;

[0045] If each small block is represented by a two-dimensional index (m,n), then the (m,n)th small block I patch (m,n)=I large [m×(ho h ):min(m×(ho h )+h,H),n×(wo w ):min(n×(wo w )+w,W)];where, I large Represents a clear image, m = 0, 1, ..., M-1, n = 0, 1, ..., N-1.

[0046] Furthermore, the data augmentation results of the smaller data patches are merged to obtain the data augmentation result for the clearer image, including:

[0047] Get small block I patch Data augmentation results for (m,n) I blur_patch (m,n); where the data augmentation result I blur_patch The dimensions of (m,n) are h′×w′, where h′ represents the data augmentation result I. blur_patch The height of (m,n) and w′ represent the data augmentation result I. blur_patch The width of (m,n);

[0048] For any pixel position (x, y) in the data augmentation result of a sharp image, based on that pixel position (x, y) and the data augmentation result I... blur_patch The dimensions h′×w′ of (m,n) and the size of the overlapping area between the small blocks o h ×o w This determines the block to which the pixel position (x, y) belongs;

[0049] If the pixel position (x, y) belongs only to the data augmentation result I blur_patch (m,n), then based on the pixel position (x,y), the two-dimensional index (m,n), and the data augmentation result I blur_patch The dimensions of (m,n) and the size of the overlapping area between the smaller blocks. h ×o w Determine the pixel position (x, y) in the data augmentation result I. blur_patch Local coordinates (x) within (m,n) local ,y local ), and set the local coordinates (x local ,y local The pixel corresponding to ) is taken as the pixel at pixel position (x,y);

[0050] If the pixel position (x, y) belongs to the data augmentation result I blur_patch (m,n) and data augmentation results I blur_patch (m′,n′), then obtain the pixel position (x,y) in the small block I respectively. blur_patch Local coordinates (x) within (m,n) local ,y local ) and in small block I blur_patch Local coordinates (x′) within (m′, n′) local ,y′ local After that, based on the local coordinates (x) local ,y local ) to data augmentation results I blur_patch The distance between the centers of (m,n) and the local coordinates (x′) local ,y′ local ) to data augmentation results I blur_patch The first weight w1 and the second weight w2 are calculated based on the distance between the centers of (m′, n′), and the data augmentation result I is used as the basis for the calculation. blur_patch (m,n), local coordinates (x) local ,y local ), second weight w2, data augmentation result I blur_patch (m′,n′) and local coordinates (x′) ocal ,y′ local Determine the pixel at position (x, y);

[0051] When m = M-1 or n = N-1, the corresponding index range is adjusted according to the actual situation to ensure that it does not exceed the limit.

[0052] A visible light remote sensing image data augmentation system based on unified domain modeling, the system comprising:

[0053] The physical degradation modeling module is used to model the physical degradation process based on the optical transmission link and obtain the point spread function;

[0054] The image synthesis module is used to determine whether the image size of the clear image is greater than a set threshold. If the image size of the clear image is not greater than the set threshold, the point spread function is applied to the clear image, and spatial variable convolution is combined to obtain the data augmentation result of the clear image.

[0055] Compared with the prior art, the present invention has at least the following beneficial effects.

[0056] 1) Introduce a unified domain degradation path based on wavefront physics modeling, especially the core formula for wavefront perturbation modeling: Φ total (x,y)=Φ reflect(x,y)+Φ env (x,y)+Φ optics (x,y). This model considers the entire light propagation path from object to image. Through this unified modeling expression, combined with wavefront superposition, phase screen simulation, Zernike higher-order aberrations, and ray tracing, the physical interpretability of degradation modeling is significantly enhanced. The simulation results can reflect the combined effects of ground microstructure, atmospheric disturbances, and optical aberrations, improving model generalization ability while maintaining degradation realism and supporting high-fidelity, controllable image enhancement data synthesis.

[0057] 2) This invention employs a pixel-aware spatially variable convolution mechanism. Unlike global convolution strategies, this invention precisely generates a local PSF for each pixel based on the physical input, maintaining the continuity of the modeling space.

[0058] 3) This invention introduces an intelligent estimation module into the physical modeling process. By introducing a lightweight MLP or CNN network to dynamically estimate parameters that are not directly obtainable, such as aberration coefficients aj, the adaptability and practicality of the physical modeling module are significantly improved. This method maintains the physical consistency of the modeling framework while enhancing the system's parameter estimation and anti-interference capabilities through intelligent learning, exhibiting stronger stability and generalization in unknown or low-data-dependency scenarios.

[0059] 4) This invention designs a parallel acceleration module for large-size remote sensing images. By combining an overlapping block strategy with a GPU parallel processing mechanism, it achieves efficient degradation simulation and image synthesis for large-size remote sensing images, which not only improves processing efficiency but also ensures spatial continuity and physical consistency in the image degradation process, making it suitable for large-scale remote sensing data generation and industrial-grade deployment needs. Attached Figure Description

[0060] Figure 1 The main functional modules of the visible light remote sensing image data augmentation method based on unified domain modeling.

[0061] Figure 2 Physical degradation modeling process based on optical transmission links.

[0062] Figure 3 MLP model predicts Zernike coefficient

[0063] Figure 4 Pixel-Location-Aware Spatial Variable Convolution Image Synthesis Module Process

[0064] Figure 5 Single large-size image block parallel processing flow Detailed Implementation

[0065] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0066] like Figure 1 As shown, the visible light remote sensing image data augmentation method based on unified domain modeling mainly consists of three parts: physical degradation modeling based on optical transmission links, a spatially variable convolution image synthesis module based on pixel location awareness, and a parallel acceleration module. The physical degradation modeling part is based on the unified domain modeling concept, considering the perturbation effects of light propagating from the target object through the atmospheric environment to the optical system. The image synthesis part achieves a more realistic remote sensing image degradation effect by constructing a pixel-independent point spread function (PSF) and performing spatially variable convolution. The parallel acceleration module can efficiently generate degraded images by performing block-based parallel processing on single large-size remote sensing images, meeting the needs of large-scale data.

[0067] I. Physical Degradation Modeling Module.

[0068] Figure 2 This section demonstrates physical degradation modeling based on optical transmission links. The process begins with the light signal reflecting off the ground surface, passing through the atmospheric environment and mixed atmospheric particles, until it enters the entrance pupil of the optical system, and finally forms an image via the optical path. The full-path degradation is illustrated by the following model.

[0069] Ground feature reflection is the first source of degradation in the remote sensing image formation process, mainly caused by optical path differences due to surface microstructures (such as roughness and height variations). Incident light is reflected at different micro-surfaces; due to variations in the normal direction and height of the microstructures, local phase perturbations occur on the reflected wavefront. Since real terrain data is difficult to obtain, to avoid relying on real terrain data for modeling, a stochastic process modeling method is adopted, representing the influence of the ground feature surface on the wavefront as an equivalent random phase perturbation function in two-dimensional space.

[0070] Specifically, assuming the ground object reflected wavefront disturbance Φ reflect (x,y) is a stationary two-dimensional Gaussian random field with zero mean and spatial correlation. Its covariance function can be written as:

[0071]

[0072] Where, σ 2 denoted by , l represents the intensity of the phase disturbance (controlling the degree of degradation of ground reflection), l represents the spatial correlation length (controlling the spatial rate of change of the disturbance), and r is the Euclidean distance between the two points.

[0073] In order to generate Φ that satisfies the above statistical properties reflect(x,y) is modeled using the spectral sampling method (spectral synthesis method), and the specific process is as follows:

[0074] 1) Construct the target power spectrum S(f) in the frequency domain x ,f y To make it satisfy the power distribution of a two-dimensional Gaussian process:

[0075]

[0076] Among them, f x ,f y This represents the spatial frequency variables corresponding to x and y, used for frequency domain modeling.

[0077] 2) Generate a complex white noise field

[0078]

[0079] Among them, A(f) x ,f y ) is a Gaussian random variable with unit amplitude, θ(f) x ,f y ) is a uniformly distributed phase in the interval [0, 2π), and k represents the imaginary unit.

[0080] 3) Modulation spectrum amplitude:

[0081]

[0082] 4) Perform a two-dimensional inverse Fourier transform on it to obtain Φ in the spatial domain. reflect (x,y):

[0083]

[0084] in, denoted as 2D inverse Fourier transform, and R represents the real number operation.

[0085] Environmental propagation is a significant source of degradation in remote sensing image formation. This is primarily due to physical disturbances in the atmosphere, such as turbulence, refractive index fluctuations, and changes in temperature and humidity, which cause phase distortion in light waves during propagation. This results in a non-flat wavefront, leading to image blurring, flickering, and degradation. To model these wavefront disturbances, the Kolmogorov turbulence model is used for physical modeling. This model assumes that atmospheric turbulence is isotropic and uniform within the inertial subinterval, and that refractive index disturbances can be considered as zero-mean random fields, with their spatial structure described by a structure function.

[0086] According to the Kolmogorov turbulence model, the structure function of phase perturbations in the atmosphere is:

[0087]

[0088] Where r is the positional difference between any two points, and r0 is the Fried parameter, which represents the coherence length scale and determines the strength of the wavefront disturbance.

[0089] To generate a phase perturbation wavefront Φ that satisfies the above structure function env (x,y) is simulated using the spectral sampling (spectral synthesis) method to create the Kolmogorov phase screen. The specific process is as follows:

[0090] 1) Construct the Kolmogorov power spectral density function S(f x ,f y ):

[0091]

[0092] 2) Generate complex white noise spectral field The amplitude is a unit Gaussian distribution, and the phase is a uniform distribution in the interval [0, 2π).

[0093] 3) Modulation spectrum amplitude:

[0094]

[0095] 4) Perform a two-dimensional inverse Fourier transform to obtain the real-valued phase perturbation:

[0096]

[0097] Optical System Section. Aberrations within the optical system are a significant source of remote sensing image degradation, primarily caused by manufacturing and assembly errors in optical components such as lenses and mirrors, as well as optical axis deviation. These errors distort the wavefront relative to an ideal spherical wavefront, ultimately leading to blurred, shifted, or out-of-focus images. To more closely approximate the actual physical process, a ray tracing method is employed, using the wavefront perturbations from ground object reflections and environmental propagation as the initial wavefront input to the entrance pupil, thus uniformly modeling the wavefront evolution process after it enters the optical system.

[0098] During the tracking process, the input wavefront Φ is first constructed. in (x,y):

[0099] Φ in (x,y)=Φ reflect (x,y)+Φ env (x,y)

[0100] This entrance pupil wavefront will serve as the wavefront at the entrance of the optical system, propagating through the lens or mirror assembly. To model the aberrations introduced within the optical system (aberrations caused by manufacturing, assembly, and other errors), the Zernike polynomial is used to represent the difference between the ideal and actual wavefronts:

[0101] Φ optics (x,y)=∑a j ·Z j (x,y)

[0102] Among them, Z j (x,y) is the j-th Zernike pattern, a j It is the weight coefficient corresponding to this pattern.

[0103] Ultimately, the total wavefront at the exit pupil is Φ total (x,y):

[0104] Φ total (x,y)=Φ in (x,y)+Φ optics (x,y)

[0105] In ray tracing, each mirror or lens surface of the system can be modeled using optical design software. This is due to the Zernike coefficient α. j Since real-time data acquisition and precise calibration are difficult, multilayer perceptrons (MLPs) are used for prediction, such as... Figure 3 As shown in the diagram, this network takes the structural parameters and assembly perturbation information of the optical system as input and outputs the corresponding Zernike coefficient vector to model and compensate for wavefront distortion caused by optical system errors. The training process of the model is shown below.

[0106] 1) Obtain the training dataset Where x i For the optical system structural parameters and assembly disturbance information, a i Corresponding Zernike coefficient vector;

[0107] 2) Initialize the neural network parameters θ and learning rate η.

[0108] 3) Entering multiple training cycles:

[0109] A. For each data batch in the current round

[0110] i. Input the optical system parameters x of the current batch into the neural network model. Obtain the Zernike coefficient vector predicted by the model.

[0111] ii. By calculating the predicted value The mean square error between the true value 'a' and the mean square error is used to obtain the mean square error loss.

[0112] iii. Loss based on mean square error The gradient of parameter θ is calculated using the backpropagation algorithm, and parameter θ is updated according to the learning rate η.

[0113] B. After completing the training of all data batches in the current round, proceed to the next round of training and repeat the above steps until the model converges or reaches the preset training round.

[0114] 4) Once training is complete, output the final neural network model.

[0115] The dataset used for training It can be simulated and exported under different manufacturing tolerance and assembly error settings using optical simulation software (such as Zemax).

[0116] The overall model is combined. The above three parts are added to form a unified expression for the total wavefront perturbation in the unified domain. The complex amplitude distribution function of the optical system's exit pupil plane is constructed, and the point spread function (PSF) is calculated. The specific steps are as follows:

[0117] 1) Total wavefront disturbance superimposed with Φ total (x,y):

[0118] Φ total (x,y)=Φ reflect (x,y)+Φ env (x,y)+Φ optics (x,y)

[0119] 2) Construct the pupil function P(x,y). The pupil function is derived from the wavefront aberration function Φ at the exit pupil position. total The equations (x, y) and the amplitude function A(x, y) constitute:

[0120]

[0121] Where λ represents wavelength.

[0122] 3) Perform a two-dimensional Fourier transform on the pupil function to calculate the complex amplitude distribution U(x,y) on the image plane.

[0123]

[0124] 4) Point Spread Function (PSF): PSF(x,y) is the squared modulus of the complex amplitude distribution.

[0125] PSF(x,y)=[U(x,y)] 2

[0126] The resulting PSF characterizes the energy diffusion response of a point target in the scene after passing through the optical system, and is used for subsequent image degradation synthesis.

[0127] II. Image Synthesis Module.

[0128] like Figure 4 As shown, to satisfy the spatial variation characteristics of the PSF (Per-Pixel Spread Function), the image synthesis module takes the per-pixel spread function (PSF) distribution obtained from the aforementioned modeling and applies it to the clear image according to a unified domain structure to complete the realistic degradation simulation of the remote sensing image. This module uses each pixel as the basic unit, generating a dedicated PSF based on the physical input at each location and the degradation modeling results, and completes the image reconstruction process through spatially variable convolution. The entire process includes: spatially position-aware PSF generation, continuous mapping structure preservation, and spatially variable convolution implementation, ensuring the consistency between image degradation and the real remote sensing imaging process at the physical level.

[0129] Spatial location-aware PSF generation. For each pixel location (x, y) in the image, a PSF for that location is generated based on its corresponding physical modeling input. x,y .

[0130] Spatially variable convolution implementation. Perform spatially variable convolution on the potentially ideal image, i.e.:

[0131]

[0132] Among them, I ideal Representing the potential ideal image, I blur Let (i,j) represent the synthesized degraded image, and (i,j) represent the PSF. x,y For each pixel position in the PSF, each pixel position (x, y) in the PSF is related to the PSF. x,y Related. This process can be divided into the following steps:

[0133] 1) Extract local image patches around pixel location (x, y):

[0134] Patch x,y =I ideal [xk:x+k,yk:y+k]

[0135] Where k is half the size of the convolution kernel;

[0136] 2) Select the PSF kernel corresponding to position (x, y) x,y (i,j), with a size of (2k+1)×(2k+1), is derived from the calculation in the previous stage;

[0137] 3) Perform a pixel-by-pixel weighted summation to obtain the degraded pixel values:

[0138]

[0139] III. Parallel Acceleration Module.

[0140] To improve the efficiency of large-scale remote sensing image data synthesis, this invention introduces a GPU-oriented parallel acceleration mechanism while maintaining a unified domain data structure, constructing an efficient degradation simulation process from modeling to synthesis. The segmentation strategy for parallel processing of single large-size images is as follows: Figure 5 As shown, when processing a single large-size remote sensing image, the image synthesis module, which satisfies the PSF spatial variation characteristics, divides the large image into small blocks for independent processing, and finally merges the results to achieve efficient image degradation simulation. Considering that boundary information may affect the processing results during block processing, overlapping regions will be introduced into the block strategy.

[0141] 1) Large-size remote sensing image segmentation strategy

[0142] When processing single large-size remote sensing images, the block size is determined based on the image size and GPU memory capacity. During partitioning, boundaries are considered to avoid information loss, and overlapping regions are set to ensure information continuity between blocks. (Large-size remote sensing image I) large The dimensions are H×W (height H, width W), the block size is h×w (height h, width w), and the overlapping area size is o. h ×o w (height is o) h Width is o w If the vertical offset of adjacent small blocks is ho, then the offset of adjacent small blocks is ho. h The horizontal offset is wo w The number of blocks M in the vertical direction and the number of blocks N in the horizontal direction are respectively:

[0143]

[0144] in This represents rounding up. Each small block is represented by a two-dimensional index (m,n), where m = 0, 1, ..., M-1, and n = 0, 1, ..., N-1. Then the (m,n)th small block I... patch (m,n) can be obtained using the following formula:

[0145] I patch (m,n)=I large [m×(ho h ):min(m×(ho h )+h,H),n×(wo w ):min(n

[0146] ×(wo w )+w,W)]

[0147] Each small piece I patch (m,n) can overlap with adjacent small blocks to a certain extent at the boundary, thus ensuring that the pixels at the boundary can obtain more complete neighborhood information.

[0148] 2) Parallel processing based on physical modeling

[0149] This allows each small block to independently complete the degradation parameter estimation in different computing units, ensuring that the overall processing is efficient and conforms to the characteristics of each small block.

[0150] PSF generation: For each small block I patch For each pixel position (x, y) in (m, n), generate the PSF at that position based on its corresponding physical modeling input. x,y Since each small block is processed independently, it can be completed in parallel on different GPU threads.

[0151] Spatially variable convolution: Performs a spatially variable convolution operation on each small block. Let the small block I... patch Taking (m,n) as an example, for pixel position (x,y), extract the surrounding local image patch. (x,y) (m,n)=I patch_ideal (m,n)[xk:x+k,yk:y+k](where k is half of the convolution kernel, I) ideal_patch (m,n) represents the potential ideal image of this block, and the PSF kernel corresponding to position (x,y) is selected. (x,y) (i,j), and then perform a pixel-wise weighted summation to obtain the degraded pixel values:

[0152]

[0153] This process allows for parallel processing of pixels within each small block across different GPU threads, and the final processing result I is obtained. blur_patch (m,n).

[0154] 3) Block merging and post-processing

[0155] After all the small blocks have been processed, the processed small block I needs to be... blur_patch (m,n) are merged into a complete degenerate image I blur_large Because overlapping areas exist during the segmentation process, these overlapping areas need to be processed during merging to ensure image continuity. Assume the processed small block I... blur_patch The dimensions of (m,n) are h′×w′ (including overlapping regions), for the merged image I blur_large For any pixel position (x, y) in the image, its corresponding block and its position within the block can be determined using the following formula:

[0156]

[0157] x local =xm×(h′-o h )

[0158] y local =yn×(w′-o w )

[0159] in, This indicates a floor operation. For pixels in overlapping regions, a weighted average method is used for merging. If pixels (x, y) in the overlapping region both belong to smaller block I... blur_patch (m,n) and I blur_patch The merged pixel values ​​(m′, n′) can be calculated using the following formula:

[0160]

[0161] Among them, I blur_patch (m,n), I blur_patch (m′, n′) are small blocks I respectively patch (m,n) and I patch The result after processing (m′,n′), (x local ,y local ) and (x′ ocal ,y′ ocal h×w represents the local coordinates of a pixel within its respective small block. w1 and w2 are weights determined based on factors such as the distance from the pixel to the center of its respective small block, ensuring a natural transition in the overlapping area of ​​the merged image. Meanwhile, when m = M-1 or n = N-1, the size of the small block may be smaller than h×w. In this case, the index range needs to be adjusted according to the actual situation to ensure it does not exceed the limits.

[0162] In summary, this invention constructs a unified domain wavefront modeling mechanism. Traditional image degradation modeling primarily considers only aberrations introduced by the optical system, neglecting other key degradation factors in the remote sensing imaging chain. This invention unifies ground object reflection disturbances, environmental propagation disturbances, and optical system fabrication and adjustment errors under the same wavefront phase disturbance expression, achieving consistent path and logically continuous end-to-end modeling.

[0163] This invention employs a wavefront perturbation simulation method based on spectral sampling. A two-dimensional random phase perturbation field is generated using spectral sampling to simulate microstructural perturbations caused by ground object reflection (Gaussian random field) and turbulent perturbations in atmospheric propagation (Kolmogorov structure function), respectively, thus giving the wavefront perturbation controllable spatial correlation and statistical properties.

[0164] This invention introduces a fusion mechanism of ray tracing and wavefront perturbation modeling. By embedding the total wavefront perturbation into the ray tracing process for propagation simulation, a realistic simulation of the entire process from the perturbation source to optical imaging is achieved. This not only retains the accuracy advantage of ray tracing in geometric propagation paths but also integrates the accuracy of wavefront modeling in physical optics.

[0165] This invention introduces neural networks for intelligent parameter estimation. For degradation parameters that cannot be directly obtained or measured in real time, a lightweight neural network model is constructed for prediction and estimation, and then fused with a physical model to achieve collaborative modeling driven by data and physics.

[0166] This invention employs a pixel-wise spatially variable convolution image synthesis mechanism. Based on the PSF generated per pixel during the modeling stage, a spatially variable convolution operation is constructed to perform pixel-wise local degradation on the image, achieving spatially continuous and realistic degradation image generation.

[0167] This invention performs parallel processing of visible light remote sensing image data. For a single large-size remote sensing image, an overlapping segmentation strategy is used to divide it into multiple small blocks for independent processing. After performing feature extraction, intelligent parameter estimation, and degradation processing on each small block, the images are then merged.

Claims

1. A method for enhancing visible light remote sensing image data based on unified domain modeling, characterized in that, The method includes: The point spread function is obtained by modeling the physical degradation process of the optical transmission link. Determine whether the size of a clear image exceeds a set threshold; If the image size of the clear image is no larger than a set threshold, the point spread function is applied to the clear image, and spatially variable convolution is combined to obtain the data augmentation result of the clear image.

2. The method according to claim 1, characterized in that, The pixel diffusion function is obtained by modeling the physical degradation of the optical transmission link, including: Based on the wavefront perturbation function of ground object reflection, the wavefront perturbation function of environmental propagation, and the aberration function within the optical system, the total wavefront aberration function is obtained. Based on the total wavefront aberration function and amplitude function, the pupil function at the exit pupil position of the optical system is obtained; Perform a two-dimensional Fourier transform on the pupil function to calculate the complex amplitude distribution on the image plane; Obtain the pixel spread function, which is the squared modulus of the complex amplitude distribution of the image plane.

3. The method according to claim 2, characterized in that, The generation process of the wavefront perturbation function reflected by the ground object includes: A target power spectrum is constructed in the frequency domain, which satisfies a two-dimensional Gaussian process power distribution. Generate a complex white noise field; Based on the target power spectrum and the complex white noise field, the modulation spectrum amplitude of the ground object reflection is obtained; Perform a two-dimensional inverse Fourier transform on the modulation spectrum amplitude of ground object reflection to obtain the wavefront perturbation function of ground object reflection.

4. The method according to claim 2, characterized in that, The generation process of the wavefront perturbation function of the environmental propagation includes: Construct the Kolmogorov power spectral density function; Generate a complex white noise spectral field with a unit Gaussian distribution in amplitude and a uniform phase distribution in the interval [0, 2π). Based on the Kolmogorov power spectral density function and the complex white noise field, the modulation spectrum amplitude of environmental propagation is obtained; Perform a two-dimensional inverse Fourier transform on the modulation spectrum amplitude of the environmental propagation to obtain the wavefront perturbation function of the environmental propagation.

5. The method according to claim 2, characterized in that, The generation process of the aberration function within the optical system includes: Acquire structural parameters and assembly disturbance information of the optical system; The structural parameters and assembly perturbation information are input into a multilayer perceptron to obtain the weight 'a' for each Zernike mode. j , j is a positive integer; where, training the multilayer perceptron is based on the mean square error between the true weights and the predicted weights of the Zernike pattern; Based on the weight a j By weighting each Zernike mode, the aberration function inside the optical system is obtained.

6. The method according to claim 1, characterized in that, Applying the point spread function to the sharpened image and combining it with spatially variable convolution yields the data augmentation result for that image, including: Extracting local image patches around pixel location (x,y) in a sharp image x,y ; The convolution kernel corresponding to the pixel position (x, y) is selected based on the point spread function; Based on the convolution kernel and the image local patch x,y Perform pixel-by-pixel weighted summation to obtain the data augmentation result of the clear image.

7. The method according to claim 1, characterized in that, The method further includes: If the image size of the clear image is larger than a set threshold, the clear image is divided into blocks based on the image size and the GPU memory size; For each small block, the point diffusion function is applied to the small block in parallel processing on different GPU threads, and spatially variable convolution is combined to obtain the data augmentation result of the small block; The data augmentation results of the small data blocks are merged to obtain the data augmentation result of the clear image.

8. The method according to claim 7, characterized in that, The sharp image is divided into blocks based on its size and GPU memory capacity, including: Obtain the dimensions H×W of the clear image, the patch size h×w, and the overlap size between patches o. h ×o w Where H represents the height of the sharp image, W represents the width of the sharp image, h represents the height of the patch, w represents the width of the patch image, and o h Indicates the height of the overlapping region, o w Indicates the width of the overlapping region; Based on the dimensions H×W of the clear image, the patch size h×w, and the size of the overlapping area between patches. h ×o w Calculate the number of small blocks M in the vertical direction and the number of small blocks N in the horizontal direction; If each small block is represented by a two-dimensional index (m,n), then the (m,n)th small block I patch (m,n)=I large [m×(ho h ):min(m×(ho h )+h,H),n×(wo w ):min(n×(wo w )+w,W)];where, I large Represents a clear image, m = 0, 1, ..., M-1, n = 0, 1, ..., N-1.

9. The method according to claim 8, characterized in that, The data augmentation results of the smaller data patches are merged to obtain the data augmentation result for the clearer image, including: Get small block I patch Data augmentation results for (m,n) I blur_patch (m,n); where the data augmentation result I blur_patch The dimensions of (m,n) are h′×w′, where h′ represents the data augmentation result I. blur_patch The height of (m,n) and w′ represent the data augmentation result I. blur_patch The width of (m,n); For any pixel position (x, y) in the data augmentation result of a sharp image, based on that pixel position (x, y) and the data augmentation result I... blur_patch The dimensions h′×w′ of (m,n) and the size of the overlapping area between the small blocks o h ×o w This determines the block to which the pixel position (x, y) belongs; If the pixel position (x, y) belongs only to the data augmentation result I blur_patch (m,n), then based on the pixel position (x,y), the two-dimensional index (m,n), and the data augmentation result I blur_patch The dimensions of (m,n) and the size of the overlapping area between the smaller blocks. h ×o w Determine the pixel position (x, y) in the data augmentation result I. blur_patch Local coordinates (x) within (m,n) local ,y local ), and set the local coordinates (x local ,y local The pixel corresponding to ) is taken as the pixel at pixel position (x,y); If the pixel position (x, y) belongs to the data augmentation result I blur_patch (m,n) and data augmentation results I blur_patch (m′,n′), then obtain the pixel position (x,y) in the small block I respectively. blur_patch Local coordinates (x) within (m,n) local ,y local ) and in small block I blur_patch Local coordinates (x′) within (m′, n′) local ,y′ local After that, based on the local coordinates (x) local ,y local ) to data augmentation results I blur_patch The distance between the centers of (m,n) and the local coordinates (x′) local ,y′ local ) to data augmentation results I blur_patch The first weight w1 and the second weight w2 are calculated based on the distance between the centers of (m′, n′), and the data augmentation result I is used as the basis for the calculation. blur_ p atch (m,n), local coordinates (x) local ,y local ), second weight w2, data augmentation result I blur_patch (m′,n′) and local coordinates (x′) local ,y′ lopcal Determine the pixel at position (x, y); When m = M-1 or n = N-1, the corresponding index range is adjusted according to the actual situation to ensure that it does not exceed the limit.

10. A visible light remote sensing image data enhancement system based on unified domain modeling, characterized in that, The system includes: The physical degradation modeling module is used to model the physical degradation process based on the optical transmission link and obtain the point spread function; The image synthesis module is used to determine whether the image size of the clear image is greater than a set threshold. If the image size of the clear image is not greater than the set threshold, the point spread function is applied to the clear image, and spatial variable convolution is combined to obtain the data augmentation result of the clear image.

Images