A visual compensation method under starlight conditions

Abstract

This application belongs to the field of visual compensation technology and provides a visual compensation method under starlight conditions. Through pre-sampling and grayscale variance statistics, it achieves the determination of starlight compensation intervals and the dynamic division of target and background regions in the imaging plane. It adopts a sampling method with different exposure time series for the target and background regions to obtain multiple frames of original sampled data and pixel integration time. Based on the pixel integration time, it constructs a spatially variable gain matrix and completes inter-frame registration and gain normalization processing to separate signal and noise components in the image. The signal component is used as a sparse sampling stream, and the pixel variance distribution of the noise component is used as a hyperparameter of the variational inference algorithm. Image reconstruction is completed through probability density gradient estimation, making the variational inference process match the actual noise distribution characteristics under starlight conditions, thereby improving the quality and reliability of visual imaging under starlight conditions.

Description

Technical Field

[0001] This application relates to the field of visual compensation technology, and in particular to a visual compensation method under starlight conditions. Background Technology

[0002] With the popularization of applications such as nighttime security monitoring, vehicle-mounted starlight night vision, and outdoor low-altitude detection, starlight environment visual compensation technology has become an important research direction in the field of low-light imaging. Most existing technologies revolve around photon accumulation, noise suppression, and image enhancement, including using fixed long exposure to increase photon reception to enhance weak signals, using global fixed gain to amplify the grayscale of the image, and relying on multi-frame superposition and fusion to achieve noise smoothing, which improves the imaging clarity in starlight environments to a certain extent.

[0003] While the aforementioned existing technologies can achieve basic low-light imaging compensation, they have significant shortcomings in scenarios with significant differences, such as starlight and extreme low illumination. Existing solutions generally adopt a globally uniform exposure strategy, which cannot simultaneously meet the dual requirements of short exposure to prevent motion blur of dynamic targets and long exposure to accumulate weak signals of static backgrounds, easily leading to target motion blur or excessive accumulation of background noise. Most of them will perform invalid compensation in non-starlight scenarios, resulting in wasted computing power, or fail to activate targeted compensation in starlight scenarios, leading to imaging failure.

[0004] Global gain amplification amplifies photon noise simultaneously, resulting in a low signal-to-noise ratio in the image. It can even lead to problems such as loss of weak signal details, severe noise residue, and blurred edges in the reconstructed image, making it difficult to meet the requirements of high-quality visual imaging under starlight. In view of the shortcomings of the above-mentioned existing technologies, the technical problem to be solved by this application is how to overcome the motion blur caused by long exposure and the insufficient signal-to-noise ratio caused by short exposure under starlight, so as to achieve high-quality visual compensation of the image. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of the prior art and provide a visual compensation method under starlight environment. The method includes: controlling the image sensor to presample with a first exposure time sequence, calculating the gray scale variance of pixels between multiple frames, determining whether the current location is in the starlight compensation range based on the spatial distribution of the gray scale variance, and dynamically dividing the imaging plane into target area and background area.

[0006] A second exposure time series sampling is performed on the target area, and a third exposure time series sampling is performed on the background area to obtain multiple frames of raw sampling data and the integration time of different pixels;

[0007] A spatially variable gain matrix is ​​constructed based on the pixel integral time. Inter-frame registration and gain normalization are performed on the multi-frame original sampled data according to the spatially variable gain matrix to decompose the signal component and noise component.

[0008] The signal components are used as a sparse sampling stream, and the pixel variance distribution of the noise components is used as the hyperparameter of the variational inference algorithm. The probability density gradient within different pixels is estimated based on the variational inference algorithm to reconstruct the compensated image.

[0009] Optionally, the gray-level variance of the statistical pixels across multiple frames includes:

[0010] Based on presampling, multiple frames of images are acquired, the temporal gray-level variance of each pixel in the multiple frames of images is calculated, and the corresponding spatial gray-level variance is calculated using a preset local spatial neighborhood.

[0011] Based on the Poisson distribution characteristics of photon noise under starlight environment, nonlinear weighted fusion of temporal gray-level variance and spatial gray-level variance is performed to obtain the gray-level variance of pixels among multiple frames.

[0012] Optionally, dynamically dividing the imaging plane into a target region and a background region includes:

[0013] If it is determined that the current area is within the starlight compensation range, the spatial distribution of the grayscale variance is traversed through a sliding window to filter out candidate windows with variance dispersion greater than the gradient threshold.

[0014] Based on the temporal gray-level variance, the candidate window is subjected to multi-frame temporal fluctuation verification to determine the target window. Adjacent target windows are subjected to connected component fusion processing. The fused region is determined as the target region, and the remaining region of the imaging plane is determined as the background region.

[0015] Optionally, determining whether it is within the starlight compensation range includes:

[0016] Based on the gray-level variance of pixels across multiple frames, determine the spatial distribution of gray-level variance on the imaging plane.

[0017] Extract the mean variance and variance dispersion of the grayscale variance, and combine them with the Poisson distribution characteristics of photon noise under starlight environment to set the judgment threshold of starlight compensation interval.

[0018] If both the mean variance and the variance dispersion are within the judgment threshold range, then it is determined that the current location is within the starlight compensation interval; otherwise, it is determined that the current location is not within the starlight compensation interval.

[0019] Optionally, the step of acquiring multiple frames of raw sampled data and the integration time for different pixels includes:

[0020] A second exposure time series is dynamically configured for the target area, and a third exposure time series is dynamically configured for the background area;

[0021] Multi-frame image sampling is performed according to the configured exposure time series to obtain multiple frames of raw sampling data, and the actual exposure time of each pixel is recorded synchronously as the integration time.

[0022] The integration time of a pixel is calibrated by taking into account the Poisson distribution characteristics of photon noise under starlight conditions, and the integration time of different pixels after calibration is obtained.

[0023] Optionally, the exposure duration in the second exposure time series is shorter than that in the first exposure time series, and the exposure duration in the third exposure time series is longer than that in the first exposure time series.

[0024] The exposure time of the target area and the corresponding area in the background area are determined based on the temporal gray-scale variance, wherein the larger the temporal gray-scale variance, the shorter the exposure time configured for the corresponding area.

[0025] The number of frames sampled for the multi-frame image is equal to the number of presampled frames for the first exposure time series.

[0026] Optionally, constructing the space-variable gain matrix includes:

[0027] The gain coefficients of the target area and the background area are determined respectively, and the noise correction coefficient of the pixel is calculated in combination with the Poisson distribution characteristics of photon noise under starlight environment to calibrate the gain coefficients.

[0028] Using the pixel coordinates of the imaging plane as matrix indices, the gain coefficients of different pixels after calibration are filled into the corresponding positions to form an initial gain matrix;

[0029] Based on the inter-frame grayscale fluctuation characteristics of the original sampled data of the multiple frames, the initial gain matrix is ​​iteratively optimized between frames to obtain the spatially variable gain matrix.

[0030] Optionally, the gain coefficient of the target region is negatively correlated with the integration time and positively correlated with the temporal gray-level variance; the gain coefficient of the background region is positively correlated with the integration time and negatively correlated with the temporal gray-level variance.

[0031] Optionally, decomposing the signal components and noise components includes:

[0032] Based on the spatially variable gain matrix, gain weighting is performed on the gray values ​​of different pixels in the multi-frame original sampling data, and inter-frame registration is performed on the target region and the background region respectively.

[0033] Gain normalization is performed on the registered multi-frame raw sampling data based on the pixel integration time.

[0034] By combining the Poisson distribution characteristics of photon noise under starlight conditions with the inter-frame grayscale fluctuation characteristics of pixels, the noise judgment threshold is dynamically determined, and the normalized data is decomposed into signal components and noise components.

[0035] Optionally, the reconstructed output compensated image includes:

[0036] The signal components are used as a sparse sampling stream, and the sparse sampling stream is subjected to regional sparsity enhancement processing.

[0037] The pixel variance distribution of the noise component is used as the hyperparameter of the variational inference algorithm, and the hyperparameter is dynamically adjusted and calibrated by combining the variance dispersion and the noise correction coefficient.

[0038] The probability density gradients in different pixels in the target and background regions are estimated based on the variational inference algorithm, and the reconstruction process is constrained by combining the correlation between the pixel integration time and the gain coefficient.

[0039] Image reconstruction is completed based on the estimated results of the adjusted hyperparameters and probability density gradient. Inter-frame fluctuation verification and signal-to-noise ratio verification are performed on the reconstructed image, and the compensated image is output.

[0040] Compared with the prior art, the beneficial effects of this application are as follows: by pre-sampling and gray-scale variance statistics, the starlight compensation interval is automatically determined and the target area and background area in the imaging plane are dynamically divided, the applicable scenarios for starlight compensation are accurately identified, the spatial basis for subsequent regional difference processing is provided, the spatial distribution characteristics of photon noise under starlight environment are matched, and the scene adaptability of the overall compensation scheme is improved.

[0041] A sampling method with different exposure time series for the target area and the background area is adopted to take into account the dual needs of short exposure for dynamic targets to prevent motion blur and long exposure for the background area to increase the accumulation of weak photons. Multiple frames of raw sampling data and pixel integration time are acquired simultaneously to solve the technical contradiction that a single exposure sequence cannot adapt to the imaging characteristics of different areas, and to provide basic data for subsequent gain matrix construction.

[0042] The spatially variable gain matrix is ​​constructed based on the pixel integral time, and inter-frame registration and gain normalization are completed to achieve precise gain control, avoid noise amplification caused by global gain, improve the spatial alignment accuracy of multi-frame data, effectively separate signal components and noise components in the image, reduce component mixing, and provide signal basis and accurate noise distribution basis for subsequent image reconstruction.

[0043] By using the signal components as a sparse sampling stream and the pixel variance distribution of the noise components as the hyperparameters of the variational inference algorithm, image reconstruction is completed through probability density gradient estimation. This allows the variational inference process to adaptively match the actual noise distribution characteristics under starlight conditions. During the reconstruction process, weak signal details are effectively preserved and photon noise interference is suppressed. The output compensated image has a higher signal-to-noise ratio, a more complete target outline, and clearer background details, thus improving the overall quality and reliability of visual imaging under starlight conditions. Attached Figure Description

[0044] Figure 1 This application provides a flowchart of a visual compensation method under starlight conditions.

[0045] Figure 2 A logic flowchart for constructing an air-variable gain matrix is ​​provided for embodiments of this application;

[0046] Figure 3 A logical flowchart of the reconstructed and compensated image is provided for the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more pairs. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0048] like Figure 1 This application provides a visual compensation method under starlight conditions, the method comprising:

[0049] S1. Control the image sensor to presample using the first exposure time sequence, calculate the gray-level variance of pixels across multiple frames, determine whether the current location is within the starlight compensation range based on the spatial distribution of the gray-level variance, and dynamically divide the imaging plane into the target area and the background area.

[0050] Furthermore, the statistical variance of grayscale pixels across multiple frames includes:

[0051] The image sensor is controlled to presample using the first exposure time sequence, and multiple frames of images are acquired based on the presampled images. The temporal gray-level variance of each pixel in the multiple frames of images is calculated, and the corresponding spatial gray-level variance is calculated using a preset local spatial neighborhood.

[0052] Based on the Poisson distribution characteristics of photon noise under starlight environment, nonlinear weighted fusion of temporal gray-level variance and spatial gray-level variance is performed to obtain the gray-level variance of pixels among multiple frames.

[0053] Under starlight conditions, image sensors using fixed exposure time presampling are prone to grayscale saturation or weak signal loss, and Poisson noise interference can cause grayscale value distortion. Furthermore, single temporal or spatial variance statistics have inherent limitations; temporal variance can only reflect pixel temporal fluctuations, and spatial variance can only reflect differences in pixel spatial distribution, failing to comprehensively characterize pixel grayscale variation characteristics. For an ambient illumination of 10... -3 For typical starlight environments, a CMOS image sensor with a pixel size of 1.4μm × 1.4μm and a dark current ≤10e is used. - / pixel / s controls the presampling based on the first exposure time sequence. The presampling must follow the principles of avoiding grayscale saturation, maximizing the retention of weak signals, and suppressing the accumulation of Poisson noise.

[0054] The first exposure time series was set to 5 consecutive pre-sampling frames, with the exposure duration of each frame adjusted alternately in the following order: 10ms, 12ms, 10ms, 12ms, and 10ms. This was to avoid noise accumulation caused by fixed exposure durations and to ensure that the exposure duration of each frame was within the sensor's linear response range, preventing grayscale saturation. The pre-sampling frame rate was set to 30fps, and the image resolution was 1920×1080. Dark current correction was performed on each frame, with the correction value based on the sensor's preset dark current reference value and adjusted according to the current ambient temperature (25℃) to minimize the interference of dark current on grayscale values. After pre-sampling, 5 consecutive pre-sampling images were acquired, denoted as I1, I2, I3, I4, and I5. The pixel grayscale value range in each frame was 0-255. Under starlight conditions, most pixel grayscale values ​​were concentrated between 10 and 50, which is consistent with weak signal characteristics.

[0055] Temporal gray-level variance is used to characterize the temporal fluctuation of gray-level values ​​of a single pixel across multiple pre-sampled images, reflecting the dynamic changes of the scene region corresponding to the pixel. For any pixel P (coordinates (x,y), x∈[0,1919], y∈[0,1079]) in the imaging plane, the gray-level values ​​of this pixel in 5 pre-sampled images are extracted and denoted as G1(x,y), G2(x,y), G3(x,y), G4(x,y), and G5(x,y). The temporal gray-level variance σt is calculated using the mean square error method. 2 First, calculate the mean value μt of the grayscale values ​​of the pixel across 5 frames. Then, calculate the sum of squares of the differences between each grayscale value and the mean value. Finally, divide by the number of frames to obtain the temporal grayscale variance.

[0056] Taking pixel P(100,200) as an example, its grayscale values ​​in the 5 presampled images are 20, 22, 19, 21, and 20 respectively, yielding μt=20.4 and σt. 2 =1.04; If the pixel is a dynamic target area pixel, and the gray values ​​are 25, 30, 22, 28, 24 respectively, then μt = 25.8, σt2 =8.16, which is significantly greater than the temporal variance of static region pixels, thus preliminarily distinguishing between dynamic and static regions.

[0057] Spatial gray-level variance is used to characterize the difference in gray-level distribution of a single pixel within its local spatial neighborhood, reflecting the spatial gray-level abrupt changes in the scene area corresponding to the pixel. A 3×3 neighborhood is preset to adapt to the weak signal characteristics of starlight environments and avoid excessive interference from noisy pixels in the calculation results. Centered on pixel P(x,y), eight neighboring pixels are selected, and the gray-level values ​​of these nine pixels in the intermediate frame I3 are extracted. The mean square error method is used to calculate the spatial gray-level variance σs of this 3×3 neighborhood. 2 The calculation logic is consistent with the temporal variance. First, the mean value μs of the gray values ​​of the 9 pixels is calculated. Then, the sum of squares of the differences between the gray value of each pixel and the mean is calculated. Finally, the spatial variance is obtained by dividing by the number of neighboring pixels.

[0058] Taking P(100,200) as an example, the gray values ​​of the pixels in its 3×3 neighborhood in I3 are (19,20,21), (20,20,22), and (18,19,20) respectively. The calculated values ​​are μs≈19.89 and σs. 2 ≈1.21; If the pixel is an edge pixel with neighborhood gray values ​​of (25,30,28), (20,22,26), and (15,18,22), then μs≈23.11, σs 2 The value is approximately 18.78, which is significantly greater than the spatial variance of pixels in non-edge regions, thus preliminarily distinguishing between edge and non-edge regions.

[0059] This avoids grayscale saturation or weak signal loss, reduces Poisson noise accumulation, and ensures that the presampled image truly reflects the pixel grayscale characteristics under starlight conditions; temporal grayscale variance captures pixel temporal fluctuations, providing temporal feature basis for target area selection; spatial grayscale variance captures differences in pixel spatial distribution, providing spatial feature basis for target area connected domain fusion; and improves the accuracy of variance statistics and noise resistance, adapting to the characteristics of low illumination and high noise under starlight conditions.

[0060] Temporal and spatial gray-level variances characterize pixel gray-level changes from temporal and spatial dimensions, respectively. However, linear fusion methods cannot adapt to the Poisson distribution characteristics of photon noise under starlight conditions. Under starlight conditions, photon noise follows a Poisson distribution, and the noise variance is proportional to the signal intensity (mean pixel gray-level). Low gray-level pixels (weak signals, small noise variance) are weighted more heavily on temporal gray-level variance to highlight temporal fluctuations and facilitate the differentiation of dynamic targets. High gray-level pixels (relatively strong signals, large noise variance) are weighted more heavily on spatial gray-level variance to highlight spatial distribution characteristics and facilitate the differentiation of targets from background edges. A nonlinear correction term is introduced to avoid fusion deviation caused by linear weight superposition. For example, when the temporal and spatial differences are large, the weight deviation is reduced to prevent a single variance from dominating the fusion result. When the temporal and spatial differences are small, the stability of the fusion result is enhanced, noise interference is suppressed, and the fusion result is ensured to conform to the Poisson noise distribution law.

[0061] Specifically, let the gray-level variance of pixel P(x,y) be σ. 2 The time-domain gray-level variance is σt 2 The spatial gray-level variance is σs 2 The weights are wt (time domain weight) and ws (spatial domain weight), respectively, and the sum of the weights is 1. To adapt to the starlight environment, the nonlinear correction term k takes a value of 0.05-0.1. In this embodiment, k=0.05 is selected, i.e., σ 2 =wt×σt 2 +ws×σs 2 +k×(σt 2 ×σs 2 ) / (σt 2 +σs 2 The weights are dynamically adjusted based on the pixel grayscale mean μ. The pixel grayscale mean μ is referenced to the mean of the grayscale variance in the time domain, taking into account the temporal signal strength. When μ≤20 (low grayscale value), wt=0.6 and ws=0.4, focusing on the temporal variance; when 20<μ<40 (medium grayscale value), wt=0.5 and ws=0.5, balancing temporal and spatial features; when μ≥40 (high grayscale value), wt=0.4 and ws=0.6, focusing on the spatial variance.

[0062] Taking a low grayscale pixel (μ=20.4, corresponding to the aforementioned pixel P(100,200)) as an example, σt 2 =1.04, σs 2 =1.21, wt=0.6, ws=0.4, k=0.05, substituting these values ​​gives σ 2 ≈0.6×1.04+0.4×1.21+0.05×(1.04×1.21) / (1.04+1.21)≈1.1315, this pixel is a static background non-edge pixel, and the variance after fusion is small, which is consistent with its characteristics; the dynamic target area pixel (μ=25.8, σt)2 =8.16, σs 2 =3.52, wt=0.5, ws=0.5), substituting these values, we get σ 2 ≈0.5×8.16+0.5×3.52+0.05×(8.16×3.52) / (8.16+3.52)≈5.96, the variance after fusion is relatively large, highlighting its dynamic temporal fluctuation characteristics; target and background edge pixels (μ=23.11, σt) 2 =2.85, σs 2 =18.78, wt=0.5, ws=0.5), substituting these values ​​gives σ 2 The variance after fusion is relatively large, which highlights the differences in its spatial distribution. ≈0.5×2.85+0.5×18.78+0.05×(2.85×18.78) / (2.85+18.78)≈10.94.

[0063] This approach adapts to the noise characteristics of pixels with different grayscale values, avoiding the limitations of fixed linear fusion and improving the accuracy of grayscale variance. It also avoids a single variance dominating the fusion result, while suppressing Poisson noise interference and enhancing the stability of the fusion result. The grayscale variance retains the temporal fluctuation characteristics of the temporal domain variance and the spatial distribution characteristics of the spatial domain variance, comprehensively and accurately characterizing the grayscale change characteristics of pixels under starlight conditions, providing more reliable data support for subsequent steps. It also solves the problems of low accuracy and weak noise resistance under starlight conditions, thus avoiding affecting the overall visual compensation scheme.

[0064] Specifically, dynamically dividing the imaging plane into target and background regions includes:

[0065] If it is determined that the current area is within the starlight compensation range, the spatial distribution of grayscale variance is traversed through a sliding window to filter out candidate windows with variance dispersion greater than the gradient threshold.

[0066] Based on the temporal gray-level variance, the candidate window is checked for temporal fluctuations over multiple frames to determine the target window. Adjacent target windows are then fused using connected component fusion. The fused region is then determined as the target region, and the remaining region of the imaging plane is determined as the background region.

[0067] Under starlight conditions, the spatial distribution of pixel grayscale variance is non-uniform, and the grayscale variance of noise pixels is prone to local anomalies. If region division is performed directly based on the variance of a single pixel, a large number of noise pixels will be misclassified as target pixels, reducing the accuracy of region division. It is confirmed that the current location is within the starlight compensation range, and the spatial distribution map of grayscale variance on the imaging plane is determined based on the grayscale variance. This distribution map has the same size as the imaging plane, and the grayscale value of each pixel in the map corresponds to the grayscale variance of the pixel at that location.

[0068] The size setting of the sliding window needs to take into account both spatial resolution and noise resistance. Considering the target size characteristics and image resolution in the starlight environment, a 5×5 sliding window is selected. If the target size is small, the window is adjusted to 3×3; if the target size is large, the window is adjusted to 7×7. In this embodiment, a 5×5 window is selected, and the window step size is set to 1 pixel to ensure that there are no omissions or repetitions in the traversal process, covering the entire imaging plane (x∈[0,1919], y∈[0,1079]). The effective traversal range of the window is x∈[2,1917], y∈[2,1077], to avoid the window exceeding the boundary of the imaging plane.

[0069] Variance dispersion is the degree of dispersion of the gray-level variance of all pixels within a sliding window. It is used to characterize the uniformity of the distribution of gray-level variance within the window. Within the window corresponding to the target region, due to the large fluctuation of pixel gray-level due to target movement, the variance dispersion will be significantly higher than that of the background region window. That is, for each sliding window, the gray-level variance (σ1) of 25 pixels within the window is extracted. 2 ,σ2 2 ,...,σ 25 2 Calculate the mean μ_σ of the 25 variances, then calculate the sum of the absolute differences between each variance and the mean, and divide by the number of variances to obtain the variance dispersion D, i.e., D=Σ|σi 2 -μ_σ| / 25 (i=1 to 25, representing the i-th pixel).

[0070] Based on the noise characteristics of starlight environments, and using the extracted mean and variance dispersion of grayscale variance, a gradient threshold is determined through statistical methods to ensure effective differentiation between target windows and background windows, avoiding misjudgment caused by noise interference. Considering the distribution range of pixel grayscale variance (1.13-10.94) in the above example starlight environment, a gradient threshold of 2.5 is set. Through multiple experiments, it is verified that windows with significantly higher variance dispersion than the background area can be effectively screened out, and misjudgment of noise windows can be suppressed.

[0071] The variance dispersion D of each sliding window is compared with the gradient threshold. If D > 2.5, the window is considered a candidate window, indicating that it contains the target region features. If D ≤ 2.5, the window is considered a background window and excluded from the candidate windows. A 5×5 sliding window with coordinates (100, 200) in the imaging plane is selected. The μ_σ of the 25 pixels in the window is approximately 1.86, and D is approximately 1.2, which is less than the gradient threshold, so it is selected as a background window. A 5×5 sliding window with coordinates (300, 400) (corresponding to the dynamic target region) is selected. The μ_σ of the 25 pixels in the window is approximately 6.82, and D is approximately 3.2, which is greater than the gradient threshold, so it is selected as a candidate window. A 5×5 sliding window with coordinates (500, 600) (containing noise pixels) is selected. The μ_σ of the 25 pixels in the window is approximately 2.15, and D is approximately 2.3, which is less than the gradient threshold, so it is selected as a background window.

[0072] This enables the extraction of local features from the spatial distribution of grayscale variance, avoiding misjudgments caused by single pixel identification; it accurately captures the local variance aggregation characteristics of the target area, adapts to the noise distribution patterns of starlight environments, effectively filters out candidate windows with target features, suppresses noise window interference, and improves the accuracy and noise resistance of candidate window selection.

[0073] Among the candidate windows selected, there may still be a small number of noisy windows. These windows do not have the temporal fluctuation characteristics of the target area. If they are directly used as target windows, it will lead to misjudgment of the target area. At the same time, the real target area may be covered by multiple adjacent candidate windows. If no fusion processing is performed, the target area will be divided into scattered small blocks, making it impossible to achieve the overall differentiated exposure of the target area in the future. Temporal gray-level variance can characterize the temporal fluctuation characteristics of pixels. Due to the dynamic movement of pixels in the target area, their multi-frame temporal fluctuation is significantly greater than that of pixels in the background area. On the other hand, the pixel temporal fluctuation of noise windows is irregular and the overall amplitude is small. Based on this difference, the temporal verification of candidate windows is realized, and noise windows are eliminated.

[0074] Specifically, the temporal gray-level variance of all pixels within each candidate window is extracted, and the mean μ_tw of the temporal gray-level variance of all pixels within the window is calculated as the temporal fluctuation value of the window. At the same time, a temporal verification threshold is set, which is based on the difference in temporal variance between the target area and the background area under starlight. Combining the distribution range of temporal gray-level variance (1.04-8.16) in the example above, the temporal verification threshold is set to 3 to ensure that the target window and the noise window can be effectively distinguished.

[0075] The temporal fluctuation value μ_tw of each candidate window is compared with the temporal verification threshold. If μ_tw > 3, the candidate window is determined to be the target window, confirming that it contains the real target region. If μ_tw ≤ 3, the candidate window is determined to be a noise window and is removed. For the candidate window with coordinates (300, 400) obtained in the previous screening, the temporal gray-level variance of 25 pixels in the window is extracted, and μ_tw ≈ 4.2 is calculated. Since it is greater than the temporal verification threshold, it is determined to be the target window. If the μ_tw of a candidate window (coordinates 400, 500) is ≈ 2.8, which is less than the temporal verification threshold, it is determined to be a noise window and is removed. For the candidate window with coordinates (302, 402) (adjacent to the target window with coordinates 300, 400), μ_tw ≈ 4.5 is calculated. Since it is greater than the temporal verification threshold, it is determined to be the target window.

[0076] Connected-domain fusion merges adjacent target windows into a single target region, preventing the target region from being fragmented into small pieces and ensuring that subsequent exposures cover the entire target region. It employs an 8-neighbor connectivity criterion: if two target windows are adjacent (including vertically, horizontally, and diagonally), they are considered to be in the same connected domain and fused; if two target windows are not adjacent, they are considered to be in different connected domains and treated as separate target regions. If the connected domain area is too small, it is considered a noise region and discarded. The minimum connected domain area is set to 9 pixels to avoid interference from small, noisy connected domains.

[0077] Traverse all target windows and record the coordinate range of each target window (coordinates of the top left corner (x1, y1) and the bottom right corner (x2, y2)). For any two target windows, determine whether their coordinate ranges overlap or are adjacent. If so, merge the coordinate ranges of the two windows to form a new target window. Repeat this process until all target windows have completed the connected component judgment and fusion, resulting in several target connected components. Determine the region corresponding to each target connected component as the target region, and determine the region in the imaging plane that is not covered by any target connected component as the background region.

[0078] For example, the two target windows with coordinates (300, 400) and (302, 402) obtained above have coordinate ranges from (300-2, 400-2) to (300+2, 400+2) and (302-2, 402-2) to (302+2, 402+2) respectively. The two windows are diagonally adjacent and are determined to be the same connected component. After fusion, the coordinate range of the new target window is (298, 398) to (304, 404), covering the entire area of ​​the original two windows. If there is another target window (coordinates 310, 410), its coordinate range is not adjacent to the above-fused window, and its own connected component area is greater than 9 pixels, it is determined to be another target area. In the imaging plane, except for the above two target areas, the remaining areas are determined as background areas, realizing the complete division between target areas and background areas.

[0079] This process eliminates noisy windows in the candidate window, improves the accuracy of the target window, and avoids misjudgment of the target area due to noise interference. It also merges adjacent target windows into a complete target area, preventing the target area from being segmented and suppressing interference from small noise connected domains. This ensures the logical coherence and accuracy of the region division. The determined target area and background area are the core basis for subsequent processing, determining the configuration range and accuracy of the second and third exposure time series, indirectly affecting the construction of the spatially variable gain matrix, and influencing the final effect of the entire visual compensation scheme.

[0080] The determination of whether a region is within the starlight compensation range includes:

[0081] Based on the gray-level variance of pixels across multiple frames, determine the spatial distribution of gray-level variance on the imaging plane.

[0082] Extract the mean and variance dispersion of the grayscale variance, and combine them with the Poisson distribution characteristics of photon noise under starlight environment to set the judgment threshold of starlight compensation interval.

[0083] If both the mean variance and the variance dispersion are within the judgment threshold range, then it is determined that the current location is within the starlight compensation interval; otherwise, it is determined that the current location is not within the starlight compensation interval.

[0084] The gray-level variance of a single pixel can only reflect the local gray-level fluctuation characteristics of that pixel and cannot characterize the scene characteristics of the imaging plane as a whole. The determination of the starlight compensation interval needs to be based on the gray-level fluctuation distribution law of the entire imaging plane, rather than the characteristics of a single pixel. Based on the gray-level variance of each pixel in multiple frames of images, a gray-level variance spatial distribution map with the same size as the imaging plane is constructed. The resolution of this distribution map is consistent with the imaging resolution of the image sensor. The gray value of the pixel corresponding to each pixel (x,y) (x∈[0,1919], y∈[0,1079]) in the spatial distribution map directly corresponds to the gray-level variance of that pixel, realizing the correspondence between the pixel gray-level variance and the spatial position of the imaging plane.

[0085] During the construction process, the pixel grayscale variance is normalized, mapping the variance values ​​to an 8-bit grayscale range of 0-255. This facilitates intuitive observation of the spatial distribution and data processing. The normalization logic is σ_norm = 255 × (σ 2 -σ_min) / (σ_max-σ_min), where σ_min is the minimum value of the gray-level variance of all pixels in the imaging plane, and σ_max is the maximum value; combined with the distribution range of pixel gray-level variance (1.13-10.94) in the starlight environment in the previous example, we set σ_min=1.13 and σ_max=10.94 to normalize the variance of each pixel.

[0086] For example, σ of pixel P(100,200) 2 =1.13, after normalization σ_norm=0; σ_norm for pixel P(300,400) 2 =5.96, after normalization σ_norm=255×(5.96-1.13) / (10.94-1.13)≈124; σ_norm of pixel P(500,600) 2 =10.94, after normalization σ_norm=255; in the gray-scale variance spatial distribution map, the area with higher gray value corresponds to the larger gray-scale fluctuation of the corresponding pixel in the imaging plane, and the area with lower gray value corresponds to the smaller gray-scale fluctuation of the corresponding pixel, showing the spatial distribution characteristics of gray-scale fluctuation on the imaging plane.

[0087] This enables the spatial integration of pixel grayscale variance, transforming local pixel characteristics into overall scene features and avoiding judgment bias caused by anomalies in a single pixel. It facilitates subsequent data extraction and threshold setting, while also intuitively presenting the distribution pattern of grayscale fluctuations within the imaging plane, providing basic data for the subsequent extraction of variance mean and variance dispersion.

[0088] Simply observing the spatial distribution of grayscale variance is insufficient to accurately determine whether a scene is within the starlight compensation range. The mean variance characterizes the overall average level of grayscale variance across all pixels in the imaging plane, reflecting the overall intensity of grayscale fluctuations in the scene. In starlight environments, due to the low number of photons and the high proportion of weak signals, the mean variance will be in a specific low range. The variance dispersion characterizes the degree of dispersion of grayscale variance across all pixels in the imaging plane, reflecting the spatial uniformity of grayscale fluctuations. In starlight environments, due to the differences in grayscale fluctuations between the target area and the background area, as well as the random distribution of noise, the variance dispersion will be in a specific medium range.

[0089] Traverse all pixels in the spatial distribution map of grayscale variance, totaling 1920 × 10⁸ = 2073600 pixels, and extract the σ of each pixel. 2 Calculate σ for all pixels 2 The arithmetic mean of the variances is used as the mean of variance μ_σall; then σ is calculated for all pixels. 2 The sum of the absolute differences between μ_σall and the total number of pixels yields the variance dispersion D_σall, i.e., D_σall = Σ|σᵢ 2 -μ_σall| / N (i=1 to N, N=2073600), the calculation logic is consistent with the variance dispersion within the sliding window.

[0090] Based on the aforementioned pixel grayscale variance distribution under starlight conditions, traversing 2,073,600 pixels on the imaging plane, the grayscale variance of each pixel was extracted, and μ_σall≈3.2 was calculated. The sum of the absolute differences between all pixel variances and 3.2 was calculated and divided by the total number of pixels, resulting in D_σall≈2.1. In non-starlight environments, such as normal lighting environments, the overall pixel grayscale fluctuation is small and the distribution is uniform, with μ_σall≈1.0 and D_σall≈0.5. In strong light environments, the pixel grayscale is easily saturated, with μ_σall≈8.5 and D_σall≈3.8, showing significant differences from the parameters of starlight environments, thus enabling scene differentiation.

[0091] The judgment thresholds include the variance mean threshold and the variance dispersion threshold. The two thresholds work together, and only when both the variance mean and variance dispersion are within the corresponding threshold range can it be determined as a starlight compensation interval. The thresholds are set according to the Poisson distribution characteristics of photon noise in starlight environment. In starlight environment, the variance of photon noise is proportional to the signal intensity, which causes the mean and dispersion of pixel grayscale variance to show a fixed distribution range. Through a large number of starlight environment experiments and statistics, combined with the parameter distribution law of non-starlight environment, a reasonable threshold range is set to ensure the distinction between starlight environment and non-starlight environment.

[0092] Based on the Poisson distribution characteristics of photon noise under starlight and the parameter range in the previous example, the variance mean threshold range is set to [1.5, 6.0]. This range covers the variance mean distribution of weak signals and high noise under starlight, while excluding the parameter ranges of normal lighting (mean < 1.5) and strong light environment (mean > 6.0). The variance dispersion threshold range is set to [1.0, 3.0]. This range covers the uneven distribution of grayscale fluctuation under starlight, while excluding the parameter ranges of normal lighting (dispersion < 1.0, uniform distribution) and strong light environment (dispersion > 3.0, excessive fluctuation).

[0093] After setting the threshold, calibration is performed using Poisson noise characteristics to ensure that the threshold adapts to scenes with different starlight intensities. Specifically, when the proportion of low grayscale pixels (μ≤20) in the imaging plane exceeds 80% (weak starlight scene), the lower limit of the variance mean threshold is adjusted to 1.2, and the lower limit of the dispersion threshold is adjusted to 0.8; when the proportion of low grayscale pixels is between 50% and 80% (medium starlight scene), the threshold range remains unchanged; when the proportion of low grayscale pixels is less than 50% (low light scene), the upper limit of the variance mean threshold is adjusted to 5.0, and the upper limit of the dispersion threshold is adjusted to 2.5 to ensure the dynamic adaptability of the threshold. For example, for weak starlight scene, the calibrated variance mean threshold is [1.2, 6.0], and the variance dispersion threshold is [0.8, 3.0]; for medium starlight scene, the thresholds [1.5, 6.0] and [1.0, 3.0] remain unchanged to ensure accurate judgment for scenes with different starlight intensities.

[0094] This comprehensively characterizes the overall grayscale fluctuation level and distribution uniformity of the imaging plane, avoiding the limitations of single-parameter judgment; it ensures that the threshold accurately distinguishes between starlight and non-starlight environments, while the dynamic calibration mechanism adapts to different starlight intensity scenes, improving the flexibility and accuracy of threshold setting, and providing reliable support for subsequent accurate judgment.

[0095] A single parameter cannot accurately distinguish between starlight and non-starlight environments. In some low-light scenes, the mean variance may fall within the threshold range of starlight environments, but the variance dispersion is significantly lower. In some non-starlight scenes with severe noise interference, the variance dispersion may fall within the threshold range of starlight environments, but the mean variance does not meet the requirements. The extracted μ_σall is compared with the mean variance threshold range, and D_σall is also compared with the variance dispersion threshold range. If both μ_σall and D_σall are within the mean variance threshold range, the current area is determined to be within the starlight compensation range, and the subsequent process of dividing the target and background regions is initiated. If either parameter is not within its corresponding threshold range, the current area is determined to be outside the starlight compensation range, and the subsequent visual compensation process is not initiated; only the original sampled image is output to avoid invalid compensation and wasted computational power.

[0096] For example, under starlight conditions, μ_σall=3.2, which is within the threshold range [1.5, 6.0], and D_σall=2.1, which is within the threshold range [1.0, 3.0]. Both parameters meet the requirements, so it is determined that the current location is within the starlight compensation range, and the subsequent region segmentation process is initiated. Under normal lighting conditions, μ_σall=1.0, which is lower than the lower limit of the variance mean threshold of 1.5, and D_σall=0.5, which is lower than the lower limit of the variance dispersion threshold of 1.0, so it is determined that the current location is not within the starlight compensation range, and the compensation process is not initiated. Under strong light conditions, μ_σall=8.5, which is higher than the upper limit of the variance mean threshold, and D_σall=3.8, which is higher than the upper limit of the variance dispersion threshold, so it is determined that the current location is not within the starlight compensation range. Under low-light interference scenarios, μ_σall=2.0, which is within the threshold range, but D_σall=0.8, which is lower than the lower limit of the threshold, so it is determined that the current location is not within the starlight compensation range.

[0097] This avoids the limitations of single-parameter judgment, improves the accuracy of starlight compensation range judgment, effectively distinguishes between starlight and non-starlight environments, and avoids invalid and missed compensation; improves the efficiency of the visual compensation process; the judgment result determines whether the subsequent process is started or not, which is the core of the scene adaptation of the entire visual compensation solution, ensuring that the compensation solution is only started when needed, adapting to the needs of different scenarios, while saving computing power and improving the practicality of the solution.

[0098] S2. Perform a second exposure time series sampling on the target area and a third exposure time series sampling on the background area to obtain multiple frames of raw sampling data and the integration time of different pixels.

[0099] Specifically, the integration time for acquiring multiple frames of raw sampled data and different pixels includes:

[0100] Dynamically configure a second exposure time series for the target area and a third exposure time series for the background area;

[0101] Multi-frame image sampling is performed according to the configured exposure time series to obtain multiple frames of raw sampling data, and the actual exposure time of each pixel is recorded synchronously as the integration time.

[0102] The integration time of a pixel is calibrated by taking into account the Poisson distribution characteristics of photon noise under starlight conditions, and the integration time of different pixels after calibration is obtained.

[0103] In the second exposure time series, the exposure duration is shorter than that of the first exposure time series, and in the third exposure time series, the exposure duration is longer than that of the first exposure time series.

[0104] The exposure duration of the target area and the corresponding area in the background area are determined based on the temporal gray-level variance. The larger the temporal gray-level variance, the shorter the exposure duration of the corresponding area.

[0105] The number of frames sampled in the multi-frame image is equal to the number of presampled frames in the first exposure time series.

[0106] Under starlight conditions, the target area and background area exhibit significant differences in characteristics. The target area is mostly dynamic with a large temporal gray-scale variance. If the same exposure time sequence as the background area is used, excessive exposure can easily lead to target motion blur, affecting subsequent target recognition and compensation effects. The background area is mostly static with a small temporal gray-scale variance. If a short exposure is used, insufficient photon reception will result in weak signals being masked by noise, making it impossible to effectively restore background details. The spatial coordinate range of the divided target area and background area, as well as the statistical temporal gray-scale variance of each region's pixels, are retrieved to dynamically configure the exposure time sequence for each region. The configuration principle is short exposure for the target area and long exposure for the background area, and the larger the temporal gray-scale variance, the shorter the exposure time, ensuring that the number of multi-frame sampling frames is equal to the number of pre-sampled frames of the first exposure time sequence.

[0107] The parameters of the first exposure time series remain the same as before, with 5 frames of alternating exposure adjustment: 10ms, 12ms, 10ms, 12ms, and 10ms respectively. The second and third exposure time series are configured based on these parameters. All exposure durations in the second exposure time series are shorter than the minimum exposure duration (10ms) of the first exposure time series. The single-frame exposure duration is dynamically adjusted according to the temporal gray-level variance of the pixels within the target area. Specifically, the temporal gray-level variance of the pixels within the target area is divided into two levels, σt. 2 ≥5.0 (high volatility, strong dynamics), with an exposure time of 8ms; 1.0 < σt 2 <5.0 (medium fluctuation, weak dynamics), configure the exposure time to 10ms, not exceeding the minimum length of the first exposure time series, to form a 5-frame second exposure time series, taking into account both target anti-mog and weak signal preservation.

[0108] For example, σt of a pixel in a certain area within the target region. 2 =8.16 (high fluctuation), corresponding to a frame exposure time configuration of 8ms; another region pixel σt 2 =4.2 (medium fluctuation), corresponding to a frame exposure duration of 10ms. The second exposure time series are 8ms, 10ms, 8ms, 10ms, and 8ms respectively. All durations are less than 10ms and 12ms of the first exposure time series.

[0109] In the third exposure time series, all exposure durations are longer than the maximum exposure duration (12ms) of the first exposure time series, and the single-frame exposure duration is dynamically adjusted based on the temporal gray-level variance of pixels in the background region; specifically, the temporal gray-level variance of pixels in the background region is divided into two levels, σt 2 ≤1.5 (low fluctuation, strong staticity), with an exposure time of 14ms; 1.5<σt 2 ≤3.0 (medium fluctuation, relatively weak staticity), with an exposure time of 12ms, not less than the maximum duration of the first exposure time series, forming a 5-frame third exposure time series, taking into account both weak background signal enhancement and noise suppression.

[0110] For example, σt of a pixel in a certain area within the background region. 2 =1.04 (low fluctuation), corresponding to a frame exposure time configuration of 14ms; another region pixel σt 2 =2.8 (medium fluctuation), corresponding to a frame exposure duration of 12ms. The third exposure time series are 14ms, 12ms, 14ms, 12ms, and 14ms respectively. All durations are greater than the first exposure time series of 10ms and 12ms.

[0111] During the configuration process, ensure that the frame order of the exposure time series is consistent with the number of the first pre-sampling frames. For the boundary pixels between the target area and the background area, use the mean of the neighborhood temporal gray-level variance to determine the exposure duration to avoid abrupt changes in the exposure configuration of the boundary area. For example, if the mean temporal gray-level variance of the target pixel and the background pixel in the neighborhood of the boundary pixel is 3.5, the exposure duration on the target area side is configured as 10ms and on the background area side as 12ms, thus achieving a smooth transition of the boundary exposure.

[0112] This allows for adaptation to the differences in characteristics between the target and background regions. Short exposures in the target region effectively avoid motion blur, while long exposures in the background region effectively enhance weak background signals. This achieves precise exposure adaptation for pixels, improving the relevance of the sampling data. The number of frames is equal to the number of the first pre-sampling frames, ensuring compatibility between the sampling data and the pre-sampling data, and providing a unified frame structure foundation for subsequent inter-frame registration and gain normalization. It also avoids abrupt changes in the sampling data, improving the continuity of the data.

[0113] The configured exposure time series needs to be converted into multiple frames of raw sampling data through actual sampling by the image sensor. At the same time, if the integration time of each pixel cannot be accurately recorded, it will lead to deviations in the subsequent gain matrix construction, thus affecting the accuracy of signal-noise decomposition and image reconstruction. Based on the configured second and third exposure time series, the same image sensor as mentioned above is controlled to perform multi-frame image sampling. The sampling frame rate is consistent with the first pre-sampling frame rate to ensure the continuity of the sampling process and data compatibility. During sampling, the image sensor executes the second exposure time series for the target area and the third exposure time series for the background area according to the region division results. During the same frame sampling process, the target area and the background area are exposed synchronously to ensure the spatial alignment of the multiple frames of raw sampling data and avoid inter-frame misalignment.

[0114] During each frame sampling process, the actual exposure time of each pixel, i.e. the integration time, is recorded synchronously to ensure the accuracy of the integration time. The recorded integration time corresponds to the pixel coordinates (x, y) and is stored as an integration time matrix. The matrix size is consistent with the imaging plane, and each matrix element corresponds to the integration time of the corresponding pixel for easy retrieval later.

[0115] For example, based on the second and third exposure time series, 5 frames of sampling are performed. In the first frame sampling, the exposure time of the target area is 8ms and the exposure time of the background area is 14ms. The integration time of all pixels in the target area is recorded as 8.0ms and the integration time of all pixels in the background area is recorded as 14.0ms. In the second frame sampling, the exposure time of the target area is 10ms and the exposure time of the background area is 12ms. The integration time of the pixels in the target area is recorded as 10.0ms and the integration time of the pixels in the background area is recorded as 12.0ms. The subsequent 3 frames of sampling synchronously record the integration time according to the corresponding exposure time to obtain 5 frames of original sampling data (denoted as S1-S5) and the integration time matrix with the imaging plane. The integration time range of the target area is 8.0ms-10.0ms and the integration time range of the background area is 12.0ms-14.0ms, which corresponds to the second and third exposure time series.

[0116] During the sampling process, the original sampled data undergoes preliminary dark current correction, consistent with the correction logic of the first pre-sampling. The dark current reference value is finely adjusted based on the ambient temperature of 25℃ to avoid the dark current interfering with the authenticity of the original sampled data, but without changing the grayscale characteristics of the original sampled data, ensuring the accuracy of subsequent signal-noise decomposition. Environmental parameters during the sampling process are also recorded to provide a reference for subsequent integration time calibration.

[0117] This ensures the correspondence between the original sampling data and the integration time, improving the correlation and accuracy of the data; the pixel integration time record provides accurate support for the pixel gain calibration of the spatially variable gain matrix; ensures data compatibility, facilitating subsequent inter-frame registration; and facilitates rapid retrieval in subsequent steps, improving processing efficiency.

[0118] During actual sampling, the image sensor is affected by the Poisson distribution characteristics of photon noise under starlight conditions. The actual exposure time may deviate slightly from the theoretically configured exposure time. The randomness of Poisson noise will cause fluctuations in the number of photons received by the pixel, thus affecting the actual exposure effect. If the initial integration time is directly used for subsequent gain matrix construction, it will lead to gain calibration deviation and amplify noise interference. At the same time, the photon noise intensity of pixels with different gray values ​​is different under starlight conditions. The noise interference of low gray value pixels is more significant, and the small deviation of the integration time has a greater impact on them.

[0119] Based on the statistical pixel grayscale mean μ and the Poisson distribution characteristics of photon noise, an integration time calibration logic is designed. The core of the calibration is to correct the deviation of the initial integration time according to the pixel signal strength, ensuring that the calibrated integration time accurately matches the actual signal strength and noise characteristics of the pixel. That is, let the calibrated integration time be T_cal, the initial integration time be T_init, the previously recorded actual exposure time, the pixel grayscale mean μ, and the calibration coefficient be k_cal based on the Poisson noise characteristics, which is positively correlated with μ. The stronger the signal strength, the smaller the noise influence, and the closer the calibration coefficient is to 1; the weaker the signal strength, the greater the noise influence, and the more the calibration coefficient deviates from 1. Then T_cal = T_init × k_cal, where k_cal = 1 + (μ0 - μ) / μ0, μ0 is the pixel grayscale mean under standard starlight conditions. Combining with the previous example, μ0 = 25 is set to achieve appropriate correction of the integration time of weak signal pixels and suppress noise interference.

[0120] For each pixel, retrieve T_init and μ, and substitute them into the calculation of T_cal. The value of k_cal ranges from 0.8 to 1.2 to ensure that T_cal does not deviate too much from the initial integration time, avoiding over-calibration that could lead to deviation. For example, for a pixel in the target region, T_init = 8.0 ms and μ = 20.4 (weak signal), the calculated k_cal = 1 + (25 - 20.4) / 25 = 1.184, and T_cal = 8.0 × 1.184 ≈ 9.47 ms. Appropriately extend the integration time to compensate for noise interference under weak signals. For pixels in the background region, T_init = 8.0 ms and μ = 20.4 (weak signal). With nit=14.0ms and μ=19.89 (weak signal), we calculate k_cal=1+(25-19.89) / 25=1.204 and T_cal=14.0×1.204≈16.86ms, thus enhancing weak signal reception. For high-fluctuation pixels in the target area, with T_init=8.0ms and μ=25.8 (strong signal), we calculate k_cal=1+(25-25.8) / 25=0.968 and T_cal=8.0×0.968≈7.74ms, thus appropriately shortening the integration time to avoid motion blur and suppress noise.

[0121] After calibration, the calibrated integration time of all pixels is updated to the integration time matrix, replacing the initial integration time, to form the pixel integration time matrix, which is used for the subsequent construction of the spatially varied gain matrix. At the same time, the consistency of the calibrated integration time is checked. If the deviation between the calibrated integration time of a pixel and its adjacent pixels exceeds 1.0 ms, the pixel is recalibrated to ensure the spatial continuity of the integration time and avoid the abnormality of the gain matrix caused by calibration deviation.

[0122] This eliminates initial integration time deviation caused by noise interference, improving integration time accuracy; enables pixel-differential calibration, adapting to the noise characteristics of pixels with different signal intensities; calibration of weak signal pixels can enhance signal reception, while calibration of strong signal pixels can avoid motion blur and noise amplification; ensures the spatial continuity of integration time, avoiding local calibration deviations; and provides support for the construction of spatially varying gain matrices and exposure time series adjustment, improving the accuracy of subsequent signal decomposition and image reconstruction.

[0123] S3. Construct a spatially variable gain matrix based on the pixel integral time, and perform inter-frame registration and gain normalization on the original sampled data of multiple frames according to the spatially variable gain matrix to decompose the signal component and noise component.

[0124] like Figure 2 As shown, constructing the space-variable gain matrix includes:

[0125] The gain coefficients of the target area and the background area are determined separately, and the noise correction coefficient of the pixel is calculated in combination with the Poisson distribution characteristics of photon noise under starlight environment in order to calibrate the gain coefficient.

[0126] Using the pixel coordinates of the imaging plane as matrix indices, the gain coefficients of different pixels after calibration are filled into the corresponding positions to form the initial gain matrix;

[0127] Based on the inter-frame grayscale fluctuation characteristics of multiple frames of original sampling data, the initial gain matrix is ​​iteratively optimized between frames to obtain the spatially variable gain matrix.

[0128] Among them, the gain coefficient of the target region is negatively correlated with the integration time and positively correlated with the time-domain gray variance; the gain coefficient of the background region is positively correlated with the integration time and negatively correlated with the time-domain gray variance.

[0129] Under starlight conditions, the imaging characteristics of the two types of regions are fundamentally different. The target region is mostly a dynamic region with a short integration time and a large temporal gray-level variance, while the background region is mostly a static region with a long integration time and a small temporal gray-level variance. If a unified gain coefficient calculation method is used, it is impossible to take into account the compensation requirements of both. By retrieving the coordinate range of the divided target region and background region, the calibrated pixel integration time, and the statistical pixel temporal gray-level variance, the initial gain coefficients of the target region and background region are calculated separately. Then, the noise correction coefficient is calculated by combining the Poisson noise characteristics to complete the gain coefficient calibration.

[0130] The gain coefficient (G_obj) in the target region is negatively correlated with the calibrated integration time (T_cal) and with the time-domain gray variance (σt). 2 The correlation is positive; the shorter the integration time, the fewer photons the target area receives, thus increasing the gain coefficient to amplify the weak signal; the larger the time-domain gray-level variance, the stronger the target dynamics, thus increasing the gain coefficient to highlight the target signal while avoiding motion blur; that is, G_obj=k1×(σt) 2 / T_cal), where k1 is the gain adjustment coefficient of the target area, which is set to 0.8 in combination with the weak signal characteristics of starlight environment, in order to balance the gain intensity and avoid excessive gain amplifying noise.

[0131] In the background region, the gain coefficient (G_bkg) is positively correlated with the calibrated integration time and negatively correlated with the temporal gray-level variance. A longer integration time results in more photons being received in the background region, thus increasing the gain to enhance weak signals. Conversely, a smaller temporal gray-level variance indicates stronger background staticity and relatively less noise interference, thus reducing the gain coefficient to suppress noise accumulation. That is, G_bkg = k2 × (T_cal / σt) 2 ), where k2 is the background region gain adjustment coefficient, set to 0.6 to avoid excessive gain leading to amplification of background noise.

[0132] Based on the Poisson distribution characteristics of photon noise under starlight conditions, the noise correction coefficient (k_noise) is used to correct the interference of noise on the gain coefficient. Its calculation logic is positively correlated with the pixel gray-scale mean μ. The stronger the signal strength, the smaller the noise interference, and the closer the correction coefficient is to 1; the weaker the signal strength, the more significant the noise interference, and the more the correction coefficient deviates from 1. It is used to moderately reduce the gain coefficient and avoid noise amplification; that is, k_noise=1 / (1+μ / μ_ref), where μ_ref is the pixel gray-scale mean under standard starlight conditions. Based on the previous example, it is set to 25 to ensure that the correction coefficient value ranges from 0.5 to 1.0 and avoids over-correction. Then the calibrated gain coefficient is G_cal=G_init×k_noise, where G_init is the initial gain coefficient of the target area or background area, and G_cal is the calibrated gain coefficient.

[0133] For example, in the target region pixel, T_cal=9.47ms, σt 2 =8.16, μ=25.8, calculate G_obj=0.8×(8.16 / 9.47)≈0.688; k_noise=1 / (1+25.8 / 25)≈0.492, G_cal=0.688×0.492≈0.340, which adapts to the integration time and grayscale fluctuation of the target area while suppressing noise interference; in the background pixel, T_cal=16.86ms, σt 2 =1.04, μ=19.89, calculate G_bkg=0.6×(16.86 / 1.04)≈9.75; k_noise=1 / (1+19.89 / 25)≈0.557, G_cal=9.75×0.557≈5.43, which enhances the weak background signal while avoiding noise amplification; the coordinates of the boundary pixel between the target and the background are (298,398), which belongs to the edge of the target area, T_cal=10.0ms, σt 2 =4.5, μ=22.5, G_obj=0.8×(4.5 / 10.0)=0.36; k_noise=1 / (1+22.5 / 25)=0.526, G_cal=0.36×0.526≈0.189, to achieve a smooth transition of gain in the boundary region and avoid image tomography caused by abrupt gain changes.

[0134] This allows for differentiated gain configuration between the target and background regions. The gain in the target region balances signal amplification and motion blur suppression, while the gain in the background region balances weak signal enhancement and noise suppression. It also specifically eliminates noise interference with the gain coefficient, improving the accuracy of gain calibration. Furthermore, it ensures that the gain coefficient of each pixel can be adapted to its own integration time, grayscale fluctuation, and noise characteristics, providing accurate gain support for subsequent signal-noise decomposition and avoiding image quality degradation caused by sudden gain changes.

[0135] The calibrated pixel gain coefficients are scattered pixel-level parameters and cannot be directly used for gain weighting processing of subsequent multi-frame raw sampling data. Specifically, based on the size of the imaging plane, an initial gain matrix corresponding to the imaging plane is constructed. The number of rows in the matrix corresponds to the y-axis coordinates (0-1079) of the imaging plane, and the number of columns corresponds to the x-axis coordinates (0-1919). Each element of the matrix (G_cal(x,y)) corresponds to the calibrated gain coefficient of the pixel with coordinates (x,y) in the imaging plane, thus realizing the correspondence between pixel coordinates and gain coefficients.

[0136] During the filling process, the calibrated gain coefficients are filled into the corresponding coordinate positions one by one according to the divided target and background regions: the matrix elements within the target region coordinate range are filled with the calibrated gain coefficients of the target region; the matrix elements within the background region coordinate range are filled with the calibrated gain coefficients of the background region; for the boundary pixels between the target and the background, the mean of the calibrated gain coefficients of the neighborhood is used for filling to ensure the smoothness of the boundary region gain matrix and avoid abrupt gain changes; after filling, the integrity of the initial gain matrix is ​​checked. All matrix elements are traversed. If there are unfilled null values ​​or outliers, such as negative gain coefficients or values ​​exceeding the reasonable range of 0.1-10.0, the calibrated gain coefficients of the corresponding pixels are retrieved again for supplementation or correction to ensure the integrity and accuracy of the initial gain matrix.

[0137] For example, the target region pixel at imaging plane coordinates (300, 400) has a calibrated gain coefficient of 0.340, which is filled into the (300, 400) position of the initial gain matrix; the background region pixel at coordinates (100, 200) has a calibrated gain coefficient of 5.43, which is filled into the (100, 200) position of the matrix; the boundary pixel at coordinates (298, 398) has a calibrated gain coefficient of 0.189, which is filled into the corresponding position, while its adjacent boundary pixel at coordinates (297, 397) has a calibrated gain coefficient of 0.192, and the difference between the two is less than 0.01, ensuring a smooth boundary; after filling, the target region element of the initial gain matrix has a value range of 0.189-0.340, and the background region element has a value range of 4.50-5.43, which meets the requirements of differentiated gain configuration and has no null or outlier values.

[0138] This ensures accurate matching between pixel coordinates and gain coefficients; avoids image tomography caused by abrupt gain changes, and improves the spatial continuity of the gain matrix; ensures the reliability of the initial gain matrix and avoids interference from null and outlier values ​​on subsequent iterative optimization; and provides a stable platform for subsequent inter-frame iterative optimization, ensuring the orderly progress of the optimization process.

[0139] The initial gain matrix is ​​constructed based solely on integration time, temporal gray-level variance, and noise characteristics, without considering the inter-frame gray-level fluctuation characteristics of the multi-frame original sampling data. In starlight environments, the multi-frame original sampling data exhibits random inter-frame gray-level fluctuations. Some pixel inter-frame fluctuations may be caused by noise, while others may be caused by minute target movements or changes in background signals. The initial gain matrix cannot adapt to such dynamic fluctuations, and if directly used for subsequent processing, it will lead to deviations in signal-noise decomposition. Specifically, the acquired multi-frame original sampling data (S1-S5, a total of 5 frames) is retrieved, and the gray-level value of each pixel in each frame is extracted. The inter-frame gray-level fluctuation value of each pixel is calculated to characterize the degree of change in pixel gray-level across multiple frames. The larger the fluctuation value, the greater the influence of noise or target movement on the pixel gray-level, and the gain coefficient is adjusted to suppress interference. The smaller the fluctuation value, the more stable the pixel gray-level, and the more stable the gain coefficient can be.

[0140] Extract the grayscale values ​​G1(x,y), G2(x,y), G3(x,y), G4(x,y), and G5(x,y) of pixel (x,y) from the 5 frames of original sampled data. Calculate the standard deviation σ_fluc of these 5 grayscale values, which is the inter-frame grayscale fluctuation value. The larger the standard deviation, the greater the inter-frame grayscale fluctuation, and vice versa. Based on the initial gain matrix, in each iteration, dynamically adjust the gain coefficient of the corresponding pixel according to the pixel's σ_fluc. The iteration terminates when the adjustment amount of the gain coefficient of all pixels is less than a preset threshold, set to 0.01, or when the number of iterations reaches a preset upper limit, set to 5 times, to ensure the efficiency and accuracy of iterative optimization.

[0141] If σ_fluc > ​​preset fluctuation threshold, it indicates that the pixel is significantly affected by noise or target motion. The preset fluctuation threshold is set to 5.0 based on the characteristics of starlight ambient noise. If the pixel is in the target area, the gain coefficient is appropriately reduced (adjustment ratio is 10%) to avoid noise amplification. If the pixel is in the background area, the gain coefficient is appropriately reduced (adjustment ratio is 15%) to suppress noise accumulation. If σ_fluc ≤ preset fluctuation threshold, it indicates that the pixel's grayscale is stable. If the gain coefficient deviates significantly from that of adjacent pixels (deviation > 0.1), the gain coefficient is adjusted (adjustment ratio is 5%) to bring it closer to the gain coefficient of adjacent pixels, ensuring the spatial continuity of the gain matrix. If the deviation is ≤ 0.1, the gain coefficient remains unchanged.

[0142] For example, for the target region pixel (300, 400), the initial gain coefficient is 0.340. The extracted grayscale values ​​from its 5 original sampled frames are 28, 30, 27, 29, and 28 respectively. The calculated σ_fluc ≈ 1.1, which is less than the preset fluctuation threshold. Furthermore, the deviation from the initial gain coefficient of 0.335 of the adjacent pixel (301, 400) is 0.005, which is less than 0.1. Therefore, the gain coefficient remains unchanged. For the background region pixel (100, 200), the initial gain coefficient is 5.43. The extracted grayscale values ​​from its 5 frames are 18, 25, 20, 22, and 19 respectively. The calculated σ_fluc ≈ 2.9, which is less than the preset fluctuation threshold. The deviation from the initial gain coefficient of 5.38 of the adjacent pixel (101, 200) is 0.05, which is less than 0.1. The initial gain coefficient is kept constant at 0.1. For edge pixels (305, 405) in the target area, the initial gain coefficient is 0.320, and the gray values ​​of the 5 frames are 32, 40, 35, 38, and 33 respectively, with σ_fluc≈3.2, which is less than the preset fluctuation threshold. However, its gain coefficient deviates significantly from the adjacent background pixel (306, 406) with a gain coefficient of 5.20. The gain coefficient is adjusted to 0.336 (adjusted by 5%) to ensure spatial continuity. For noise pixels (500, 600) in a certain background area, the initial gain coefficient is 4.80, and the gray values ​​of the 5 frames are 15, 28, 17, 25, and 16 respectively, with σ_fluc≈5.8, which is greater than the preset fluctuation threshold. The gain coefficient is appropriately reduced to 4.08 (adjusted by 15%) to suppress noise amplification.

[0143] After iterative optimization, a spatially variable gain matrix is ​​obtained. This matrix retains the differentiated gain characteristics of the initial gain matrix and adapts to the inter-frame grayscale fluctuation characteristics of multiple frames of original sampled data. The gain coefficients are more accurate and the spatial distribution is smoother. After optimization, the spatially variable gain matrix is ​​verified to ensure that the gain coefficients of all pixels are within a reasonable range (0.1-10.0), and the gain matrix after inter-frame iterative optimization can accurately match the dynamic and noise characteristics of the pixels.

[0144] This allows the spatially variable gain matrix to adapt to dynamic changes between frames, overcoming the static nature of the initial gain matrix; suppressing inter-frame fluctuations caused by noise interference and ensuring the spatial continuity of the gain matrix; balancing optimization accuracy and efficiency, and avoiding deviations caused by excessive iteration; the spatially variable gain matrix adapts to the integration time, temporal grayscale fluctuations, noise characteristics, and dynamic changes between frames for each pixel, providing gain support for subsequent signal-noise decomposition.

[0145] Specifically, decomposing the signal components and noise components includes:

[0146] Gain weighting is performed on the gray values ​​of different pixels in the original sampled data of multiple frames based on the spatially variable gain matrix, and inter-frame registration is performed on the target area and the background area respectively.

[0147] Gain normalization is performed on the registered multi-frame raw sampling data based on the pixel integration time.

[0148] By combining the Poisson distribution characteristics of photon noise under starlight conditions with the inter-frame grayscale fluctuation characteristics of pixels, the noise judgment threshold is dynamically determined, and the normalized data is decomposed into signal components and noise components.

[0149] The spatially varying gain matrix amplifies the effective signal and suppresses noise through pixel-differential gain, but this gain has not yet been applied to the original multi-frame sampled data. At the same time, during multi-frame sampling under starlight conditions, the dynamic motion of the target area and the slight jitter of the image sensor can cause inter-frame misalignment in multi-frame images. If registration is not performed, subsequent inter-frame data fusion and normalization processing will have deviations, affecting the accuracy of signal and noise decomposition. In addition, the inter-frame misalignment characteristics of the target area and the background area are different. The misalignment of the target area is more obvious and dynamic, while the misalignment of the background area is weak and static. Performing inter-frame registration by region can adapt to the differences in characteristics between the two and avoid registration deviation caused by a single registration method.

[0150] First, the constructed spatially varied gain matrix, the acquired 5 frames of original sampling data (S1-S5), and the coordinate ranges of the divided target and background regions are retrieved. Gain weighting processing and inter-frame registration by region are then performed. Gain weighting involves multiplying the gain coefficient of each pixel in the spatially varied gain matrix with the grayscale values ​​of the corresponding pixels and frames in the multi-frame original sampling data to achieve differentiated gain for pixels, amplifying weak signals and suppressing noise. For each frame of original sampling data (S1 to S5), the... For any pixel (x,y) in the imaging plane, the weighted gray value G_w(x,y,k) = G_gray(x,y,k) × G_var(x,y), where G_gray(x,y,k) is the gray value of pixel (x,y) in the original sampling data of the k-th frame (k=1 to 5), and G_var(x,y) is the gain coefficient of pixel (x,y) in the spatially variable gain matrix. k=1 to 5 corresponds to 5 frames of sampling data, ensuring that the gain weighting process is completed for each frame of data.

[0151] For example, the spatially variable gain coefficient G_var = 0.340 for the target region pixel (300, 400), and the grayscale values ​​of the 5 original sampled data frames are 28, 30, 27, 29, and 28 respectively. After weighting, the grayscale values ​​are 28 × 0.340 ≈ 9.52, 30 × 0.340 ≈ 10.20, 27 × 0.340 ≈ 9.18, 29 × 0.340 ≈ 9.86, and 28 × 0.340 ≈ 9.52 respectively. For the background region pixel (100, 200), the spatially variable gain coefficient G_var = 5.43, and the grayscale values ​​of the 5 original sampled data frames are 18, 25, 20, 22, and 19 respectively. After weighting, the grayscale values ​​are 18 × 5.43 ≈ 97.74 and 25 × 5.43 ≈ 1 respectively. 35.75, 20×5.43≈108.60, 22×5.43≈119.46, 19×5.43≈103.17; the spatially variable gain coefficient G_var=0.189 for the boundary pixel (298,398), the original gray values ​​of the 5 frames are 22, 24, 21, 23, 22 respectively, and the weighted gray values ​​are 22×0.189≈4.16, 24×0.189≈4.54, 21×0.189≈3.97, 23×0.189≈4.35, 22×0.189≈4.16 respectively; after weighting, the effective signal in the target area is moderately amplified, the weak signal in the background area is significantly enhanced, and the fluctuation amplitude of the noise component is reduced due to the differential suppression effect of the gain.

[0152] Inter-frame registration eliminates inter-frame misalignment in weighted multi-frame data, ensuring spatial alignment across frames. It employs a region-based registration strategy to accommodate the differences in characteristics between the target and background. In the target region, a feature-point matching-based registration method is used. Due to the high dynamism and significant inter-frame misalignment in the target region, feature points are selected from pixels within the target region with large gray-level fluctuations, i.e., the time-domain gray-level variance σt. 2 For a resolution of ≥5.0, 100-150 feature points are selected per frame. The KNN matching algorithm is used, with the first frame (S1) as the reference frame. The frames S2-S5 are then registered sequentially to correct for misalignment such as translation and rotation between frames, ensuring spatial alignment of data across multiple frames in the target area. The registration accuracy is controlled within one pixel. In the background area, a rigid registration method is used. Because the background area is highly static and the misalignment between frames is slight, the background area of ​​the reference frame (S1) is used as a reference to perform translation correction on the background areas of frames S2-S5, improving registration efficiency and accuracy. The registration deviation does not exceed one pixel.

[0153] For example, taking frame 1 (S1) as the reference frame, the misalignment coordinates of the target region pixel (300, 400) in frame S2 are (301, 400), which means a horizontal offset of 1 pixel. After identifying the misalignment through feature point matching, the entire target region in frame S2 is shifted horizontally to the left by 1 pixel, so that the coordinates of pixel (300, 400) in frame S2 are consistent with those in the reference frame. The misalignment coordinates of the background region pixel (100, 200) in frame S3 are (100, 201), which means a vertical offset of 1 pixel. Through rigid registration, the entire background region in frame S3 is shifted vertically downward by 1 pixel to achieve coordinate alignment. After registration, the spatial consistency of the multi-frame weighted data is significantly improved, avoiding subsequent normalization and decomposition deviations caused by inter-frame misalignment.

[0154] This allows for precise amplification of the effective signals of the target and background, suppression of noise components, highlighting the differences between signals and noise, and laying the foundation for subsequent decomposition. It also adapts to the differences in characteristics between the target and background, ensuring accurate correction of dynamic misalignment in the target area, improving the registration efficiency of the background area, eliminating inter-frame misalignment of multi-frame data, and ensuring spatial consistency of multi-frame data.

[0155] Even after gain weighting and inter-frame registration, multi-frame data is still affected by the integration time. Differences in integration time can lead to significant differences in weighted grayscale values ​​under the same signal strength, making them unsuitable for inter-frame comparison and signal-noise decomposition. For example, a long integration time in the background region can result in a higher weighted grayscale value, which may be misjudged as a strong signal when in fact it is due to signal superposition caused by an excessively long integration time. By retrieving the calibrated integration time matrix (T_cal(x,y)), the weighted grayscale value of each pixel in each frame of weighted data after inter-frame registration is normalized. This is done by dividing the weighted grayscale value by the calibrated integration time of the corresponding pixel, eliminating the influence of integration time on the grayscale value. This ensures that the normalized grayscale value (G_norm(x,y,k)) uniformly represents the signal strength and noise characteristics of the pixel per unit time, facilitating inter-frame comparison and decomposition.

[0156] That is, G_norm(x,y,k)=G_w(x,y,k) / T_cal(x,y), where G_w(x,y,k) is the weighted gray value of the k-th frame (x,y) after inter-frame registration, and T_cal(x,y) is the calibrated integration time of the pixel (x,y). k=1 to 5 corresponds to 5 frames of data, ensuring that each frame and each pixel completes the normalization process; so that the gray values ​​of all pixels are mapped to the bit time scale, eliminating the influence of integration time differences, and making the gray values ​​of the target area and the background area comparable.

[0157] For example, the target area pixel (300, 400) has a T_cal of 9.47ms. After registration and weighting, the grayscale values ​​are 9.52, 10.20, 9.18, 9.86, and 9.52, respectively. After normalization, the grayscale values ​​are 9.52 / 9.47≈1.01, 10.20 / 9.47≈1.08, 9.18 / 9.47≈0.97, 9.86 / 9.47≈1.04, and 9.52 / 9.47≈1.01, respectively. The normalized grayscale values ​​fluctuate smoothly, reflecting the true signal characteristics of the target area.

[0158] The background region pixel (100, 200) has a T_cal of 16.86 ms. After registration and weighting, the grayscale values ​​are 97.74, 135.75, 108.60, 119.46, and 103.17, respectively. After normalization, the grayscale values ​​are approximately 97.74 / 16.86 ≈ 5.80, 135.75 / 16.86 ≈ 8.05, 108.60 / 16.86 ≈ 6.44, 119.46 / 16.86 ≈ 7.08, and 103.17 / 16.86 ≈ 6.12, respectively. After normalization, the grayscale value can accurately reflect the weak signal characteristics of the background area, avoiding misjudgment due to excessively high grayscale value caused by excessively long integration time; the T_cal=10.0ms of the boundary pixel (298,398) has grayscale values ​​of 4.16, 4.54, 3.97, 4.35, and 4.16 after registration and weighting, respectively, and grayscale values ​​of 0.42, 0.45, 0.40, 0.44, and 0.42 after normalization, respectively. The normalized grayscale values ​​are smoothly connected with the normalized grayscale values ​​of adjacent target and background pixels, avoiding abrupt changes in the value of the boundary area.

[0159] After normalization, the normalized gray values ​​of all pixels are checked for range to ensure that the normalized gray values ​​are within a reasonable range (0.1-10.0). If there are outliers, such as a normalized gray value > 10.0, it may be due to an insufficient integration time or an abnormal weighted gray value. In this case, the weighted gray value and integration time of the corresponding pixel are retrieved again, and the normalized gray value is recalculated to ensure the accuracy of the normalized data.

[0160] This eliminates the impact of pixel integration time differences, providing a unified benchmark for setting subsequent dynamic noise thresholds and signal-noise decomposition; it accurately reflects the true signal strength and noise characteristics of pixels, avoiding misjudgments in decomposition caused by integration time differences; range verification ensures the accuracy of normalized data, providing reliable support for subsequent decomposition steps.

[0161] Noise in starlight environments is mainly Poisson noise, whose distribution characteristics are positively correlated with the signal intensity of pixels. Furthermore, the inter-frame grayscale fluctuation characteristics of different pixels differ. If a fixed threshold is used, it cannot adapt to the signal intensity, noise characteristics, and inter-frame fluctuation differences of different pixels, which can easily lead to misjudgment. For example, the signal of a weak signal pixel may be misjudged as noise, and the noise of a strong noise pixel may be misjudged as a signal. The solution involves retrieving normalized multi-frame data, statistically analyzing the pixel grayscale mean μ, and the pixel inter-frame grayscale fluctuation value σ_fluc, and then dynamically determining the noise judgment threshold and decomposing the signal and noise components.

[0162] By combining the characteristics of Poisson noise and the features of inter-frame grayscale fluctuations, the noise judgment threshold is dynamically set. The stronger the pixel signal intensity, the larger the Poisson noise variance, and the noise judgment threshold is appropriately increased to avoid misjudging noise as a signal. The larger the inter-frame grayscale fluctuation of the pixel, if it is a target area, the noise judgment threshold is appropriately increased to preserve the target signal fluctuation. If it is a background area, the noise judgment threshold is appropriately decreased to suppress noise fluctuations. At the same time, a threshold correction coefficient is introduced to balance signal preservation and noise suppression.

[0163] For each pixel (x,y), the noise threshold Th(x,y) = α × μ(x,y) + β × σ_fluc(x,y), where α is a noise correction coefficient of 0.3 set in combination with the Poisson noise characteristics of starlight environment to adapt to the positive correlation between noise variance and signal strength, β is the inter-frame fluctuation correction coefficient, β = 0.2 for target area and β = 0.1 for background area to adapt to the difference in fluctuation characteristics between the two, μ(x,y) is the gray mean of pixel (x,y), and σ_fluc(x,y) is the inter-frame gray fluctuation value of pixel (x,y).

[0164] For example, the target region pixel (300, 400) has μ=25.8, σ_fluc≈1.1, and belongs to the target region. β=0.2, and Th=0.3×25.8+0.2×1.1=7.74+0.22=7.96. The normalized gray values ​​are 1.01, 1.08, 0.97, 1.04, and 1.01, all less than the noise threshold. This indicates that the signal component accounts for a low proportion in the normalized data of this pixel, and the fluctuations are mainly noise. Subsequent decomposition should focus on suppressing noise. Sound; for the background region pixel (100,200), μ=19.89, σ_fluc≈2.9, belonging to the background region, β=0.1, calculate Th=0.3×19.89+0.1×2.9=5.97+0.29=6.26; after normalization, the gray values ​​are 5.8, 8.05, 6.44, 7.08, 6.12 respectively, where the part greater than the noise judgment threshold is the signal component, and the part less than or equal to the noise judgment threshold is the noise component, accurately distinguishing between signal and noise.

[0165] For the weak signal pixel (500, 600) in the background area, μ=15.0, σ_fluc≈3.5, β=0.1, and Th=0.3×15+0.1×3.5=4.5+0.35=4.85. After normalization, the gray values ​​are 4.2, 5.1, 4.6, 4.9, and 4.3 respectively. Among them, 5.1 and 4.9 are greater than the noise judgment threshold and are signal components, while the rest are noise components, effectively preserving weak signals and avoiding misjudgment. After the noise judgment threshold is set, its rationality is checked. If the noise judgment threshold of a certain pixel is less than 0.5 (weak signal, low noise), it is adjusted to 0.5 to avoid the threshold being too low and causing noise to be misjudged as signal. If the noise judgment threshold is greater than 10 (strong signal, high noise), it is adjusted to 10 to avoid the threshold being too high and causing signal to be misjudged as noise.

[0166] Iterate through the normalized grayscale values ​​G_norm(x,y,k) (k=1 to 5) of each pixel (x,y) for 5 frames. If G_norm(x,y,k) > Th(x,y), then the portion of the grayscale value of that pixel in that frame that exceeds the noise threshold is the signal component S(x,y,k), i.e., S(x,y,k) = G_norm(x,y,k) - Th(x,y); the portion that is at or below the noise threshold is the noise component N(x,y,k), i.e., N(x,y,k) = G_norm(x,y,k) - Th(x,y). (x,y,k)-S(x,y,k) (If G_norm(x,y,k)≤Th(x,y), then S(x,y,k)=0, N(x,y,k)=G_norm(x,y,k)); After decomposition, the signal and noise components are smoothed between frames. The signal components are smoothed by the mean of the frames to reduce the fluctuation between frames and retain the true signal. The noise components are smoothed by the median of the frames to suppress the interference of random noise and ensure that the decomposed signal components are pure and the noise components can be suppressed by subsequent steps.

[0167] For example, the background region pixel (100, 200) has a Th=6.26. The normalized grayscale values ​​of the five frames are 5.80, 8.05, 6.44, 7.08, and 6.12, respectively. After decomposition, the signal components are 0, 8.05-6.26=1.79, 6.44-6.26=0.18, 7.08-6.26=0.82, and 0, respectively; the noise components are 5.80, 6.26, 6.26, 6.26, and 6.12, respectively. After inter-frame mean smoothing of the signal components, the smoothed signal component (0+1.79+0.18+0.82+0) / 5≈0.56, and the noise component after median smoothing is 6.26. This preserves the weak background signal and suppresses noise fluctuations. The weak signal pixel (500, 600) in the scene area has a Th=4.85, and the normalized gray values ​​of the 5 frames are 4.20, 5.10, 4.60, 4.90, and 4.30 respectively. After decomposition, the signal components are 0, 0.25, 0, 0.05, and 0 respectively. The mean after smoothing between frames is 0.06, and the median after smoothing of the noise component is 4.30. Weak signals are preserved to avoid being misjudged as noise. The target area pixel (300, 400) has a Th=7.96, and the normalized gray values ​​of the 5 frames are all less than the threshold. After decomposition, the signal components are all 0, and the noise components are all the corresponding normalized gray values. The median after smoothing between frames is 1.01. It is identified that there is no obvious effective signal in this pixel, and the fluctuations are all noise.

[0168] This allows it to adapt to the signal strength, noise characteristics, and fluctuation differences of different pixels, avoiding misjudgment caused by fixed thresholds; accurately separate superimposed signals and noise, retain the effective signals of the target and background, and accurately extract noise components; reduce the interference of inter-frame fluctuations and random noise, ensuring that the decomposed signal components are pure and the noise components are suppressed; and provide core input for subsequent image reconstruction, determining the quality of image reconstruction and avoiding subsequent visual compensation effects.

[0169] S4. The signal components are used as a sparse sampling stream, and the pixel variance distribution of the noise components is used as the hyperparameter of the variational inference algorithm. The probability density gradient in different pixels is estimated based on the variational inference algorithm to reconstruct the compensated image.

[0170] like Figure 3 As shown, the reconstructed and compensated image includes:

[0171] The signal components are treated as a sparse sampling stream, and the sparse sampling stream is subjected to regional sparsity enhancement processing.

[0172] The pixel variance distribution of the noise component is used as the hyperparameter of the variational inference algorithm, and the hyperparameter is dynamically adjusted and calibrated by combining the variance dispersion and the noise correction coefficient.

[0173] The probability density gradients in different pixels in the target and background regions are estimated based on the variational inference algorithm, and the reconstruction process is constrained by combining the correlation between the pixel integration time and the gain coefficient.

[0174] Image reconstruction is completed based on the estimated results of the adjusted hyperparameters and probability density gradient. Inter-frame fluctuation verification and signal-to-noise ratio verification are performed on the reconstructed image, and the compensated image is output.

[0175] Although the decomposed signal components have removed most of the noise, the signals under starlight are inherently sparsity. If they are directly used as reconstruction input, the reconstructed image will be blurry, the target details will be unclear, and the weak background signals will not be effectively presented. The signal components S(x,y,k) (k=1 to 5) of all pixels and all frames obtained by decomposition are retrieved and integrated into a sparse sampling stream. Using the pixel coordinates (x,y) of the imaging plane as the index and the inter-frame sequence of the signal components of multiple frames as the temporal dimension, a two-dimensional sparse sampling stream of space and time is formed. This sampling stream contains only signal information and removes noise interference. Based on the segmented target area and background area, a differentiated sparse enhancement strategy is adopted to adapt to the difference in the sparse characteristics of the signals of the two.

[0176] The signal components in the target area exhibit local clustering and inter-frame fluctuations. Sparse enhancement strengthens the target edges and amplifies the target signal while preserving the target's dynamic characteristics and avoiding blurring of the target edges during the enhancement process. An adaptive sparse enhancement algorithm is employed, taking the signal component of each pixel within the target area as the core and selecting the signal components within its 8-neighborhood. The mean value of the neighborhood signal is calculated. If the current pixel signal component is less than the neighborhood mean, it is appropriately amplified. The amplification factor is dynamically adjusted based on the temporal gray-level variance; the larger the temporal gray-level variance, the larger the amplification factor, highlighting the target's dynamic characteristics. If the current pixel signal component is greater than or equal to the neighborhood mean, it remains unchanged to avoid signal saturation.

[0177] The signal components in the background region are weak and discrete with smooth inter-frame fluctuations. Sparse enhancement amplifies weak signals, suppresses signal discreteness, and highlights background details while avoiding amplifying residual noise. A weak signal adaptive enhancement algorithm is employed, dynamically adjusting the enhancement coefficient based on the pixel's gray-level mean. A smaller gray-level mean results in a larger enhancement coefficient, emphasizing the amplification of weak signals; conversely, a larger gray-level mean results in a smaller enhancement coefficient, avoiding distortion caused by excessive signal enhancement. For example, after decomposing the target region pixel (300, 400), all signal components are 0, and its 8-neighborhood... The inter-frame mean of the signal component of pixel (300, 401) is 0.3. The signal component of the current pixel is less than the mean of the neighborhood, but its own σt²=8.16. The amplification factor is set to 1.5. After enhancement, the signal component is still 0, that is, there is no effective signal, thus avoiding false enhancement in areas without signal. The inter-frame mean of the signal component of the adjacent target pixel (301, 400) is 0.4. Its own signal component is 0.3, which is less than the mean of the neighborhood. The amplification factor is 1.4. After enhancement, the signal component is 0.3×1.4=0.42, thus enhancing the target signal.

[0178] The background region pixel (100, 200) has a signal component mean of 0.56 after decomposition, a pixel grayscale mean μ=19.89, and an enhancement coefficient set to 1.2. After enhancement, the signal component mean of the signal component is 0.56×1.2=0.67, effectively amplifying the weak background signal. The background weak signal pixel (500, 600) has a signal component mean of 0.06, a grayscale mean μ=15.0, and an enhancement coefficient set to 1.8. After enhancement, the signal component mean of the signal component is 0.06×1.8=0.11, highlighting faint background details. Boundary pixels ( The inter-frame mean of the signal components (298,398) is 0.08. Combining the target and background enhancement strategies, a compromise enhancement coefficient of 1.3 is adopted, and the enhanced signal component is 0.08×1.3=0.10, which realizes a smooth transition of signal enhancement in the boundary area and avoids edge discontinuity. After the enhancement is completed, the enhanced sparse sampling stream is checked. If the enhanced signal component of a certain pixel exceeds the reasonable range (0.01-5.0), the enhancement coefficient is readjusted to avoid signal saturation or insufficient enhancement, and to ensure that the enhanced signal component has the real signal characteristics and reconstruction adaptability.

[0179] This enables signal integration, facilitating subsequent enhancement and reconstruction processing; it adapts to the differences in signal sparsity between the target and the background, enhancing the target region to strengthen target edges and dynamic features, and enhancing the background region to highlight weak signals and details, thus resolving reconstruction blur caused by weak signals; it avoids signal saturation and insufficient enhancement, ensuring that the enhanced signal components are pure and effective, guaranteeing the quality of the enhanced signal, and providing an input basis for subsequent image reconstruction.

[0180] Variational inference is an algorithm for image reconstruction under starlight conditions. The setting of hyperparameters determines the noise suppression accuracy and reconstruction effect. Inappropriate hyperparameters can lead to residual noise, signal distortion, or blurred details in the reconstructed image. The algorithm retrieves the noise components N(x,y,k) of all decomposed pixels and all frames, calculates the noise variance distribution of each pixel, and calculates the variance σ_n of the noise components of each pixel (x,y) across 5 frames. 2 (x,y) is used to reflect the noise intensity and distribution of the pixel. This noise variance distribution is used as the initial hyperparameter θ_init(x,y) of the variational inference algorithm, i.e., θ_init(x,y)=σ_n 2 (x,y).

[0181] By combining D_σall and k_noise(x,y), the initial hyperparameters are dynamically adjusted and calibrated. Specifically, the larger the variance dispersion and the more uneven the noise distribution, the more moderately the hyperparameters are increased to enhance noise suppression; conversely, the smaller the noise correction coefficient and the greater the noise intensity, the more moderately the hyperparameters are increased to improve the noise suppression weight. Simultaneously, a calibration factor is introduced to avoid signal distortion caused by excessive hyperparameter adjustment. Therefore, the calibrated hyperparameters are: θ_cal(x,y) = θ_init(x,y) × (1 + γ × D_σall) ×(1-k_noise(x,y)), where γ is the variance dispersion correction factor, which is set to 0.2 in combination with the characteristics of starlight environmental noise to balance the impact of variance dispersion on hyperparameters and ensure that the hyperparameter adjustment range is reasonable; k_noise(x,y) is the noise correction coefficient of pixel (x,y) (taken as 0.492-0.557 in the above example), with a value range of 0.5-1.0, ensuring that the value of (1-k_noise(x,y)) is in the range of 0-0.5 to avoid over-adjustment.

[0182] For example, σ_n of the background region pixel (100, 200) 2 =6.26, θ_init=6.26; D_σall=2.1, k_noise=0.557, substituting into the calculation, θ_cal=6.26×(1+0.2×2.1)×(1-0.557)=6.26×1.42×0.443≈6.26×0.629≈3.94, adapting to the noise intensity and distribution characteristics of this pixel, enhancing noise suppression; σ_n of the target region pixel (300,400) 2 =1.01, θ_init=1.01; D_σall=2.1, k_noise=0.492, θ_cal=1.01×(1+0.2×2.1)×(1-0.492)=1.01×1.42×0.508≈1.01×0.721≈0.728, adapting to the low noise characteristics of the target area and avoiding excessive suppression leading to signal loss; σ_n of weak signal pixels (500, 600) in the background area2 =4.30, θ_init=4.30; D_σall=2.1, k_noise=0.580, θ_cal=4.30×1.42×(1-0.580)=4.30×1.42×0.42≈4.30×0.596≈2.56, balancing weak signal preservation and noise suppression.

[0183] After calibration, the consistency of the calibrated hyperparameters of all pixels is checked. If the hyperparameter of a certain pixel deviates from that of the adjacent pixels by more than 1.0, the pixel is recalibrated to ensure the spatial continuity of the hyperparameters and avoid tomography of the reconstructed image caused by abrupt changes in hyperparameters. At the same time, the hyperparameter values ​​are ensured to be in the range of 0.5-5.0 to avoid insufficient noise suppression due to excessively small hyperparameters or signal distortion due to excessively large hyperparameters.

[0184] This ensures the initial fit between hyperparameters and pixel noise characteristics, conforming to the Poisson noise distribution pattern; enables hyperparameters to adapt to differences in noise intensity and distribution uniformity of different pixels, improving the noise suppression accuracy of variational inference algorithms; and ensures the spatial continuity of hyperparameters, avoiding discontinuities in the reconstructed image.

[0185] Reconstruction solely through probability density gradient estimation is prone to reconstruction bias. A variational inference algorithm based on θ_cal(x,y) is used to estimate the probability density gradient within pixels of both the target and background regions, adapting to the differences in signal characteristics between the two. Since the signal in the target region exhibits dynamic fluctuations and clear edges, probability density gradient estimation focuses on capturing the grayscale change trend at the target edges. A gradient enhancement estimation method is employed to calculate the gradient of the enhanced signal components of pixels within the target region, amplifying the gradient values ​​in the edge regions and highlighting the target edge features. Furthermore, the gradient estimation results are corrected by incorporating changes in signal components between frames, avoiding gradient bias caused by dynamic target motion.

[0186] In the background region, the signal is static, weak, and uniformly distributed. The probability density gradient estimation focuses on capturing the gray-level changes in background details. A smoothing gradient estimation method is adopted to reduce the impact of random fluctuations on the gradient, amplify the gradient value in the weak signal region, and highlight background details. For static background pixels, inter-frame gradient mean smoothing is used to improve the stability of gradient estimation. For each pixel (x,y), based on the inter-frame sequence of the enhanced signal components, the spatial gradient (x-direction, y-direction) and temporal gradient (inter-frame direction) of its gray-level value are calculated. The spatial gradient and temporal gradient are weighted and fused to obtain the probability density gradient ∇P(x,y). The spatial gradient weight for the target region is 0.7 and the temporal gradient weight is 0.3 to highlight spatial edges and dynamic features. The spatial gradient weight for the background region is 0.5 and the temporal gradient weight is 0.5 to balance details and stability.

[0187] By combining correlations to construct reconstruction constraints, the reconstruction process of variational inference is constrained. The core constraint logic is that G_var(x,y) and T_cal(x,y) are negatively correlated in the target region. During reconstruction, if the pixel integration time is shorter, i.e., the target dynamics are stronger, the smoothness of the constraint probability density gradient is lower, preserving the dynamic edges of the target. If the gain coefficient is larger, i.e., the signal amplification is stronger, the deviation of the constraint gradient estimation is smaller, ensuring the accuracy of signal reconstruction. In the background region, G_var(x,y) and T_cal(x,y) are positively correlated. During reconstruction, if the pixel integration time is longer, i.e., the background signal is stronger, the smoothness of the constraint probability density gradient is higher, suppressing residual background noise. If the gain coefficient is larger, i.e., the weak signal amplification is stronger, the sensitivity of the constraint gradient estimation is higher, ensuring accurate reconstruction of background details.

[0188] Specifically, ∇P_constraint(x,y)=∇P(x,y)×λ(x,y), where λ(x,y) is the constraint coefficient. For the target region, λ(x,y)=G_var(x,y) / T_cal(x,y), and for the background region, λ(x,y)=G_var(x,y)×T_cal(x,y). The probability density gradient is adjusted through the constraint coefficient to achieve targeted constraints in the reconstruction process; for example, for pixels in the target region (3... In the background region (pixels 00, 400), with ∇P = 0.35, G_var = 0.340, T_cal = 9.47ms, λ = 0.340 / 9.47 ≈ 0.036, and ∇P_constraint = 0.35 × 0.036 ≈ 0.012, the gradient smoothing is moderately reduced to preserve the dynamic edges of the target. In the background region (pixels 100, 200), ∇P = 0.28, G_var = 5.43, and T_cal = 16. 86ms, λ=5.43×16.86≈91.55, ∇P_constraint=0.28×91.55≈25.63, improve gradient smoothness, suppress residual background noise, and highlight weak signal details; in the boundary pixel (298,398), ∇P=0.15, G_var=0.189, T_cal=10.0ms, combined with the target and background constraint logic, λ=0.189×10.0=1.89 (compromise constraint), ∇P_constraint=0.15×1.89≈0.28, to achieve a smooth transition of boundary region reconstruction constraints and avoid edge discontinuities; after gradient estimation and reconstruction constraints are completed, the rationality of the constrained probability density gradient is checked. If the gradient value of a pixel exceeds the reasonable range (0.01-30.0), the constraint coefficients are readjusted to ensure that the gradient estimation fits the signal characteristics and effectively constrains the reconstruction process.

[0189] This allows us to capture the spatial and temporal trends of pixel grayscale, providing clear gradient support for image reconstruction; adapt to the differences in signal characteristics between the target and the background, highlighting target edges and background details; avoid reconstruction deviations, ensuring that the reconstruction results match the characteristics of the target and the background, and enhancing the accuracy of signal restoration; and provide a gradient constraint basis for subsequent image reconstruction.

[0190] The reconstructed image may have problems such as excessive inter-frame fluctuations and substandard signal-to-noise ratio. If output directly, it cannot meet the visual compensation quality requirements. Based on the calibrated hyperparameters, constrained probability density gradients, and enhanced sparse sampling streams, a variational inference algorithm is initiated to complete the image reconstruction. Using the enhanced signal components as the basis, the probability density gradient as the spatial distribution guide, and the hyperparameters as noise suppression constraints, the variational inference iterative optimization reconstructs the multi-frame sparse signal components into a single-frame image, maximizing the suppression of noise interference. During the reconstruction process, the characteristic differences between the target region and the background region are maintained. The target region focuses on clear edges and preservation of dynamic features, while the background region focuses on complete details and noise suppression.

[0191] After reconstruction, a single-frame initial reconstructed image is obtained. The gray value of each pixel corresponds to the reconstructed signal strength, achieving the reconstruction goals of weak signal enhancement, noise suppression, and clear target. For example, in the target region, based on the enhanced signal components and constrained gradients of pixel (300, 400) and its neighborhood, variational inference iterative optimization results in a pixel gray value of 0.45 after reconstruction. The target edge is clear, with no ghosting or noise residue, and it fits the dynamic characteristics of the target. In the background region, the pixel (100, 200) has a gray value of 0.67 after reconstruction. The weak background signal is effectively enhanced, details are clear, and there is no obvious noise residue. In the boundary region, the gray value transitions smoothly after reconstruction, without discontinuities or abrupt gray value changes, and the overall image has good coherence.

[0192] Inter-frame fluctuation verification ensures the stability of the reconstructed image and avoids inter-frame flicker. It retrieves the reconstructed single-frame image and the original sampled data and signal components of multiple frames, calculates the inter-frame grayscale fluctuation value between the reconstructed image and the signal components of each frame. If the fluctuation value is greater than the preset fluctuation threshold, it indicates that the reconstructed image deviates too much from the original signal components, and there is an abnormal inter-frame fluctuation. The hyperparameters and gradient constraints are readjusted, and the image is reconstructed. Otherwise, it indicates that the inter-frame fluctuation is stable and the reconstructed image is stable.

[0193] Signal-to-noise ratio (SNR) verification ensures the noise suppression effect of the reconstructed image; the higher the SNR, the better the image quality. The SNR of the reconstructed image is calculated as SNR = 10 × lg(mean signal component / mean noise component), where the mean signal component is the inter-frame mean of the enhanced signal components, and the mean noise component is the inter-frame mean of the calibrated noise components. Considering the visual compensation requirements of starlight environments, a SNR threshold of 15 dB is set. If SNR ≥ 15 dB, noise suppression is effective, and the image quality meets the standard. If SNR < 15 dB, residual noise exists, requiring readjustment of hyperparameters (e.g., increasing hyperparameters) and reconstructing.

[0194] For example, the inter-frame fluctuation check shows that the inter-frame grayscale fluctuation values ​​of the reconstructed image and the 5 frame signal components are 1.2, 1.5, 1.3, 1.4, and 1.2 respectively, all less than the preset fluctuation threshold, indicating stable inter-frame fluctuation and meeting the stability requirements. The signal-to-noise ratio (SNR) check shows that the mean value of the enhanced signal component between frames is 0.42, and the mean value of the calibrated noise component between frames is 0.02. The calculated SNR is 10 × lg(0.42 / 0.02) = 10 × lg21 ≈ 13.2 dB, slightly lower than the SNR threshold of 15 dB. After readjusting the hyperparameters and increasing the average hyperparameter of the background region by 0.5, the SNR after reconstruction is 10 × lg(0.42 / 0.015) ≈ 14.5 dB, which still does not meet the standard. After readjusting the hyperparameters again, the SNR is 10 × lg(0.42 / 0.012) ≈ 15.5 dB, meeting the quality requirements.

[0195] After the verification is passed, the grayscale range of the reconstructed image is adjusted to map the grayscale values ​​to the standard grayscale level of 0-255 to ensure that the image brightness is moderate and the details are clear. Then the compensated image is output. If the verification fails, the process of adjusting parameters, reconstructing and verifying is repeated until the quality requirements are met.

[0196] This enables precise reconstruction of sparse signals, effectively suppressing noise interference, preserving target edges and background details, and adapting to the imaging needs of starlight environments. Inter-frame fluctuation verification ensures the stability of the reconstructed image, avoids inter-frame flicker, and improves the visual experience. Signal-to-noise ratio verification ensures noise suppression effectiveness and guarantees image quality. Grayscale range adjustment makes the output image meet standard visual requirements, completing the closed loop of the entire visual compensation process. The compensated image is the final result of the entire visual compensation method. If the reconstructed image fails verification and is not corrected, it will lead to a blurry output image, residual noise, and inter-frame flicker, making it impossible to achieve effective visual compensation under starlight environments. If the verification is successful and a clear and stable compensated image is output, the imaging pain points of low illumination, high noise, and weak signals under starlight environments are successfully solved.

Claims

1. A visual compensation method under starlight conditions, characterized in that, include: The image sensor is controlled to presample using the first exposure time sequence, and the gray-level variance of pixels in multiple frames is statistically analyzed. Based on the spatial distribution of the gray-level variance, it is determined whether the current area is in the starlight compensation range, and the imaging plane is dynamically divided into the target area and the background area. A second exposure time series sampling is performed on the target area, and a third exposure time series sampling is performed on the background area to obtain multiple frames of raw sampling data and the integration time of different pixels; A spatially variable gain matrix is ​​constructed based on the pixel integral time. Inter-frame registration and gain normalization are performed on the multi-frame original sampled data according to the spatially variable gain matrix to decompose the signal component and noise component. The construction of the space-variable gain matrix includes: The gain coefficients of the target area and the background area are determined respectively, and the noise correction coefficient of the pixel is calculated in combination with the Poisson distribution characteristics of photon noise under starlight environment to calibrate the gain coefficients. Using the pixel coordinates of the imaging plane as matrix indices, the gain coefficients of different pixels after calibration are filled into the corresponding positions to form an initial gain matrix; Based on the inter-frame grayscale fluctuation characteristics of the original sampled data of the multiple frames, the initial gain matrix is ​​iteratively optimized by inter-frame to obtain the spatially variable gain matrix. The signal components are used as a sparse sampling stream, and the pixel variance distribution of the noise components is used as the hyperparameter of the variational inference algorithm. The probability density gradient within different pixels is estimated based on the variational inference algorithm to reconstruct the compensated image.

2. The visual compensation method under starlight environment as described in claim 1, characterized in that, The gray-level variance of the statistical pixels across multiple frames includes: Based on presampling, multiple frames of images are acquired, the temporal gray-level variance of each pixel in the multiple frames of images is calculated, and the corresponding spatial gray-level variance is calculated using a preset local spatial neighborhood. Based on the Poisson distribution characteristics of photon noise under starlight environment, nonlinear weighted fusion of temporal gray-level variance and spatial gray-level variance is performed to obtain the gray-level variance of pixels among multiple frames.

3. The visual compensation method under starlight environment as described in claim 2, characterized in that, The step of dynamically dividing the imaging plane into target region and background region includes: If it is determined that the current area is within the starlight compensation range, the spatial distribution of the grayscale variance is traversed through a sliding window to filter out candidate windows with variance dispersion greater than the gradient threshold. Based on the temporal gray-level variance, the candidate window is subjected to multi-frame temporal fluctuation verification to determine the target window. Adjacent target windows are subjected to connected component fusion processing. The fused region is determined as the target region, and the remaining region of the imaging plane is determined as the background region.

4. The visual compensation method under starlight environment as described in claim 3, characterized in that, Determining whether the starlight compensation range is within the specified range includes: Based on the gray-level variance of pixels across multiple frames, determine the spatial distribution of gray-level variance on the imaging plane. Extract the mean variance and variance dispersion of the grayscale variance, and combine them with the Poisson distribution characteristics of photon noise under starlight environment to set the judgment threshold of starlight compensation interval. If both the mean variance and the variance dispersion are within the judgment threshold range, then it is determined that the current location is within the starlight compensation interval; otherwise, it is determined that the current location is not within the starlight compensation interval.

5. A visual compensation method under starlight conditions as described in claim 3, characterized in that, The integration time for acquiring multiple frames of raw sampled data and different pixels includes: A second exposure time series is dynamically configured for the target area, and a third exposure time series is dynamically configured for the background area; Multi-frame image sampling is performed according to the configured exposure time series to obtain multiple frames of raw sampling data, and the actual exposure time of each pixel is recorded synchronously as the integration time. The integration time of a pixel is calibrated by taking into account the Poisson distribution characteristics of photon noise under starlight conditions, and the integration time of different pixels after calibration is obtained.

6. The visual compensation method under starlight environment as described in claim 5, characterized in that, The exposure durations in the second exposure time series are all shorter than those in the first exposure time series, while the exposure durations in the third exposure time series are all longer than those in the first exposure time series. The exposure time of the target area and the corresponding area in the background area are determined based on the temporal gray-scale variance, wherein the larger the temporal gray-scale variance, the shorter the exposure time configured for the corresponding area. The number of frames sampled for the multi-frame image is equal to the number of presampled frames for the first exposure time series.

7. The visual compensation method under starlight environment as described in claim 1, characterized in that, The gain coefficient of the target region is negatively correlated with the integration time and positively correlated with the time-domain gray-level variance; the gain coefficient of the background region is positively correlated with the integration time and negatively correlated with the time-domain gray-level variance.

8. A visual compensation method under starlight conditions as described in claim 1, characterized in that, Decomposing the signal components and noise components includes: Based on the spatially variable gain matrix, gain weighting is performed on the gray values ​​of different pixels in the multi-frame original sampling data, and inter-frame registration is performed on the target region and the background region respectively. Gain normalization is performed on the registered multi-frame raw sampling data based on the pixel integration time. By combining the Poisson distribution characteristics of photon noise under starlight conditions with the inter-frame grayscale fluctuation characteristics of pixels, the noise judgment threshold is dynamically determined, and the normalized data is decomposed into signal components and noise components.

9. A visual compensation method under starlight conditions as described in claim 8, characterized in that, The reconstructed and compensated image includes: The signal components are used as a sparse sampling stream, and the sparse sampling stream is subjected to regional sparsity enhancement processing. The pixel variance distribution of the noise component is used as the hyperparameter of the variational inference algorithm, and the hyperparameter is dynamically adjusted and calibrated by combining the variance dispersion and the noise correction coefficient. The probability density gradients in different pixels in the target and background regions are estimated based on the variational inference algorithm, and the reconstruction process is constrained by combining the correlation between the pixel integration time and the gain coefficient. Image reconstruction is completed based on the estimated results of the adjusted hyperparameters and probability density gradient. Inter-frame fluctuation verification and signal-to-noise ratio verification are performed on the reconstructed image, and the compensated image is output.

Images