A scattering imaging method based on partial scene prior

Abstract

The application discloses a scattering imaging method based on partial scene prior, adopts an imaging lens, a prior target and a monitoring camera, modifies a probability model of a noise part on the basis of a traditional atmospheric scattering model, estimates the noise by means of a preset prior target in a monitoring scene, develops a corresponding algorithm according to the estimation to restore an image of a field of view monitoring area, and realizes cloud-penetrating and fog-penetrating imaging to a certain extent. Compared with a traditional image enhancement algorithm, the application does not need gray scale stretching and directly restores a high-contrast image from a physical model; compared with a traditional image restoration algorithm, the application can restore an image invisible to human eyes in a super-visibility range.

Description

Technical Field

[0001] This invention belongs to the field of scattering imaging, particularly passive scattering imaging under conditions of partial prior information; specifically, it is a scattering imaging method based on partial scene priors. Background Technology

[0002] In fields such as security, transportation, and remote sensing, the use of imaging equipment for all-weather monitoring of fixed scenes is of great significance. However, in harsh environments with scattering media such as heavy fog, dust, and dense smoke, it is difficult to obtain clear images. This is mainly because the scattering effect of particles in the atmosphere reduces the signal-to-noise ratio when the camera sensor acquires images, resulting in image quality degradation.

[0003] Atmospheric scattering imaging research mainly falls into two categories: image enhancement algorithms based on image features and image restoration based on physical principles. The former primarily uses contrast stretching to enhance the display effect, while the latter aims to study the physical mechanism of scattering imaging and construct mathematical models. Traditional atmospheric scattering models explain the imaging process and information composition of foggy images, suggesting that the main reasons for image degradation acquired by imaging systems include:

[0004] 1. The reflected light from the target scene is absorbed and scattered by particles in the atmosphere, causing energy attenuation, which reduces the portion of the signal received by the camera sensor.

[0005] 2. Ambient light such as sunlight is scattered by particles in the atmosphere, forming stray light, which constitutes the noise part when the sensor collects data. In strong scattering environments, this noise can even be greater than the target reflected light, introducing additive noise that reduces image contrast.

[0006] In traditional image dehazing techniques, image enhancement, as a pure image processing method without a physical model, improves contrast through grayscale stretching. This can be compared with physically-based image restoration methods to verify their accuracy and effectiveness. As can be seen from the above formula, the degradation of foggy images can be attributed to the attenuation of light intensity in the target scene and additive noise from skylight that does not contain scene information. In strong scattering environments, the latter is far greater than the former and is the main cause of the degradation.

[0007] When the imaging environment contains scattering media such as heavy fog, dust, or dense smoke, the scattering effect of these particles reduces the reflected light from the target scene monitored by the camera, and increases stray light entering the camera through the scattering particles. This significantly reduces the light signal-to-noise ratio (SNR) of the imaging device's sensor, making the target scene unrecognizable. In strong scattering environments, it is difficult to perform all-weather image and video monitoring of target scenes. Firstly, long exposures or multi-frame stacking are needed to improve the SNR, but this is time-consuming and has very limited effectiveness. Secondly, traditional image restoration methods based on principles such as dark channels require multiple measurements, making real-time monitoring difficult. Thirdly, most traditional image restoration methods rely on certain image features, making it difficult to achieve ultra-visible scattering imaging.

[0008] The patent "A Polarization Imaging Dehazing Method Combining Dark Channel Prior Principle (CN 105139347 A)" proposes a polarization imaging dehazing method combining the dark channel prior principle, including the following steps: First, a polarization image is acquired through polarization imaging technology, thereby obtaining the linearly polarized Stokes vector of the scene; second, the sky region is selected from the obtained total light intensity image of the scene, and the degree of polarization, polarization angle, and intensity of atmospheric scattered light at infinity are estimated based on this region; third, the intensity of atmospheric scattered light at each pixel of the entire image is calculated based on the estimated degree of polarization and polarization angle, and combined with the intensity of atmospheric scattered light at infinity, the intensity of the target light after dehazing is calculated according to the physical model of polarization dehazing imaging; finally, the image quality is evaluated using the gray-level entropy function, and the bias coefficient of the intensity of atmospheric scattered light at infinity is automatically optimized to obtain the dehazed image. However, this method has the following problems:

[0009] 1. Requires multiple data collections and is not suitable for moving objects or changing scenarios;

[0010] 2. A sky area needs to be reserved in the field of view to estimate global atmospheric light, which limits its application scenarios;

[0011] 3. Relying on image features, the restoration effect can only go from unclear to slightly clear, and it is difficult to break through the visibility range to restore objects from a completely invisible scattering environment. Summary of the Invention

[0012] To address the aforementioned problems and shortcomings, this invention proposes a scattering imaging method based on partial scene priors. It involves pre-setting partial prior targets in the monitored scene, estimating noise models and parameters through the degradation of these prior targets, and then constructing images of other regions in the field of view using the same type of noise to inversely solve for the noise. This invention requires only a single shot, replacing the traditional method of acquiring and stacking multiple frames to improve the signal-to-noise ratio with multiple estimations and iterative optimization. It achieves scattering imaging in completely invisible ultra-visible ranges without the need for image enhancement such as grayscale stretching, enabling real-time monitoring of target scenes in strong scattering environments. This invention relates to the correction of atmospheric scattering models, the estimation of noise parameters using partial scene priors, and the construction of noise-inverse solutions for unknown images.

[0013] To address the challenge of real-time image monitoring in environments with strong scattering, the technical solution of this invention is as follows:

[0014] A scattering imaging method based on partial scene priors, employing an imaging lens, a scene prior target, and a camera, is characterized by the following steps:

[0015] ① Set up a priori target in the imaging scene, and use the degradation of the priori target in foggy weather to estimate the probability distribution model and parameters of noise, including visibility, noise type, mean and variance, as follows:

[0016] Extract the prior target region I from the acquired foggy image. prior (x), the formula is as follows:

[0017] I prior (x)=J prior (x)*t(x)+N<A(1-t(x))>

[0018] In the formula, J prior (x) represents the prior target reflective pattern region cropped from the acquired foggy image, x is the spatial coordinate of the image pixel, t(x) is the transmittance at spatial coordinate x, A is the global atmospheric light, i.e., atmospheric scattered light at infinity, and N<A(1-t(x))> The corrected noise distribution probability model;

[0019] Estimate the mean brightness of the pure black background region of the prior target. Its variance is denoted as var(I) airlight );

[0020] Estimate the mean brightness of the region of the prior target reflective pattern. The formula is as follows:

[0021]

[0022] In the formula, The mean intensity of the a priori reflective pattern region of the target before scattering.

[0023] The transmittance t(x) at spatial coordinate x is estimated using the following formula:

[0024]

[0025] ② Construct image information of the unknown region by inverse solving the noise of the same type:

[0026] Based on the mean of the pure black background region of the prior target and variance var(I airlight Construct random noise with the same parameters:

[0027]

[0028] Its spectral distribution is as follows:

[0029] F ′ niose =F{N'<A(1-t(x))>}

[0030] Based on the above-constructed clear diagram of the unknown monitoring scenario using noise inversion, a similar noise inversion method is obtained. target :

[0031]

[0032] Among them I target This represents an unknown monitoring scene area in the fog map;

[0033] ③ Repeatedly solve the problem at the allowed frame rate, and based on the noise parameters estimated in the previous step, repeatedly generate n noise parameters of the same type N. i <A(1-t(x))> (i = 1, 2, ..., n), iterate multiple times, saving the image locally, and finally average the results of the previous multiple solutions to obtain the optimized restored image S(J). target ):

[0034]

[0035] Among them, F i niose To estimate the noise N multiple times i <A(1-t(x))> Spectral distribution.

[0036] Specifically, a scattering imaging method based on partial scene priors includes:

[0037] 1. Set up a traditional surveillance imaging system;

[0038] 2. In clear weather, focus on the target scene area being monitored;

[0039] 3. Design a priori targets with sparse frequency domain structures of appropriate size based on the field of view;

[0040] 4. Place prior targets within the monitored scene;

[0041] 5. In clear weather, calibrate the prior target intensity and calculate its frequency domain distribution;

[0042] 6. Collect fog images with noise in foggy conditions;

[0043] 7. Calculate the spectral degradation of the prior target portion of the fog map, and estimate the noise type and specific parameters;

[0044] 8. Construct noise of the same type and inversely solve the image of the unknown region;

[0045] 9. Set a certain number of random variables and iterations, and solve the fog map repeatedly.

[0046] 10. Utilize the inverse solution results of multiple noise estimations to jointly optimize and recover the image of the unknown region.

[0047] Preferably, the lens is a telephoto lens that can block some stray light, and is equipped with a theodolite and an electric focuser for easy and quick alignment and focusing;

[0048] Preferably, the prior target pattern and background of the scene are designed with high reflectivity and contrast, and the designed pattern has a sparse structure and directionality in the spectral domain, so the energy is concentrated, which makes it easier to estimate its degradation degree and thus facilitates fitting of noise parameters;

[0049] Preferably, the camera is a 16-bit large bit depth, high sensitivity, and low noise camera with readout noise of 1.0 med e- and dark current of less than 0.5 e- / pixel / s.

[0050] Under the same visibility, the noise term A(1-t(x)) in the atmospheric scattering model is traditionally considered a constant, but in reality, it is not uniform and fluctuates in different magnitudes in both space and time. First, we assume it conforms to a certain probability model distribution, and then modify the atmospheric scattering model as follows:

[0051] I(x)=J(x)t(x)+N>A(1-t(x))>

[0052] Where N<A(1-t(x))> To correct the probabilistic model of noise distribution, it can generally be estimated as Gaussian noise. Preferably, considering different scattering environments and camera internal noise, it can be fitted as a composite model of multiple noise combinations. By estimating the degradation mode of a pre-set target in the scene, the type and parameters of noise can be obtained. Based on this, the same type of noise is repeatedly constructed within a certain range to iteratively solve other areas of the monitoring field of view, achieving contrast enhancement without grayscale stretching, and recovering completely invisible fog maps solely based on the physical model.

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] 1) By modifying the atmospheric scattering model and using the noise parameters estimated by the prior target, the unknown region image is solved by repeatedly constructing the same noise model. This replaces the traditional method of improving the signal-to-noise ratio by relying on multiple frame superpositions, and overcomes the problems that it can only average the random part of the noise, is time-consuming, and is not suitable for dynamic scenes.

[0055] 2) It can simultaneously estimate both the background and random components of noise, overcoming the problem that the traditional superposition averaging method has limited improvement effect when scattering is severe;

[0056] 3) Compared to traditional image restoration methods that rely on physical models, it can achieve the restoration of scenes invisible to the human eye that exceed the visibility range;

[0057] 4) Foggy image restoration can be achieved without image enhancement and denoising algorithms. In particular, traditional image algorithms and methods such as deep learning can be used to further optimize the restoration results. Attached Figure Description

[0058] Figure 1 This is a schematic diagram illustrating the principle of a scattering imaging method based on partial scene priors.

[0059] Figure 2 This is a flowchart of an algorithm for noise estimation and recovery of unknown targets.

[0060] Figure 3 These are images showing the results of fog image processing and contrast enhancement. Detailed Implementation

[0061] This invention provides a scattering imaging method based on partial scene priors. By pre-setting a priori target in the monitored scene to estimate the noise type and parameters, a noise model is constructed based on this target to solve for images of other monitored areas, achieving real-time monitoring in strong scattering environments. This invention requires only a single measurement, replacing traditional multi-frame overlay with multiple noise estimations, saving time and improving real-time performance and effectiveness. Furthermore, this invention utilizes only a physical model to construct a noise model to solve for fog images, achieving scattering imaging recovery results in the ultra-visible range without grayscale stretching.

[0062] In this embodiment of the invention, the application scenario is when traditional imaging faces the problem of image blurring due to scattering effects caused by various pollutant particles such as fog, dust, and smoke in the environment. For example... Figure 1As shown, scattering particles in the air partially block the reflected light from the target scene, reducing the signal strength received by the camera sensor. On the other hand, sunlight enters the camera directly through scattering by these particles; this light, known as atmospheric scattered light or stray light, adds a layer of background noise to the image. In strong scattering environments, this additive noise can be much greater than the target scene signal. The combined effect of these two factors leads to a reduced signal-to-noise ratio when the camera captures data in foggy conditions, resulting in low image contrast and an inability to distinguish the target scene.

[0063] In this embodiment of the invention, the noise term in the traditional atmospheric scattering model is first modified and rewritten as a probability distribution model; then, using a pre-defined prior target in the scene, the noise model and parameters are estimated by utilizing its degradation mode in foggy weather, such as... Figure 2 As shown, the unknown monitoring area is then constructed based on the estimated noise parameters. The solution is repeatedly performed at the allowed frame rate. The fog map is iteratively optimized by combining the multiple solutions to further improve the recovery effect. The specific number of iterations can be adjusted according to the algorithm time and the required frame rate.

[0064] The specific implementation steps are as follows:

[0065] 1. Connect the imaging lens, adapter firmware, and camera;

[0066] 2. Connect the camera and computer power supply, and open the camera operation software to preview the real-time interface;

[0067] 3. Connect the power supply to the theodolite of the astronomical telescope and locate the target scene;

[0068] 4. Connect the power supply to the electronic focuser and adjust the handle to focus on the target scene;

[0069] 5. Based on the field of view, design a priori target of appropriate size. In this embodiment, a sparse diffuse scattering target with frequency domain mainly distributed in four directions is selected, and the background is pure black high-absorption velvet cloth.

[0070] 6. Fix the prior target in the scene;

[0071] 7. Under clear conditions, acquire images of a scene with a pre-set target;

[0072] 8. From the acquired image, extract the prior target region, denoted as: J prior (x), the black velvet background outside its pattern can be regarded as a value of "0".

[0073] 9. In the absence of scattering, record the contrast between the prior target pattern and the background, and calculate the spectral distribution;

[0074] 10. Add a target to the location monitoring area, simulating a new target appearing in foggy weather;

[0075] 11. Under clear conditions, sample prior targets and simulated unknown new targets in the scene as ground truth for fog map reconstruction contrast;

[0076] 12. In foggy conditions, turn on the visibility meter to record and synchronize the clock with the camera;

[0077] 13. Collect images of air quality degradation during foggy weather;

[0078] 14. Extract the prior target region, denoted as:

[0079] I prior (x)=J prior (x)*t(x)+N<A(1-t(x))>

[0080] 15. Estimate the average brightness of the pure black background region of the prior target as the increased atmospheric scattered light.

[0081] N<A(1-t(x))> The mean, denoted as: Its variance is denoted as var(I) airlight );

[0082] 16. Based on the average brightness of the prior target reflective pattern area, it is estimated to be the result of signal attenuation and noise superposition, denoted as:

[0083]

[0084] In the formula, The mean intensity of the a priori target reflective pattern region before scattering;

[0085] 17. The estimated transmittance is calculated as follows:

[0086]

[0087] in, J is the average intensity of the a priori target reflective pattern region before scattering, i.e., J in the formula prior The average brightness of the reflective pattern area of ​​(x);

[0088] 18. Convert the calculated transmittance results into visibility according to the national standard document (GB / T 35223-2017), and compare them with the measured visibility data to verify their accuracy;

[0089] 19. Noise N<A(1-t(x))> When estimating transmittance, noise can be treated as a constant. In this case, the image reconstruction is specifically a probabilistic model, using a Gaussian noise model as the reference model, through J... prior (x) Estimate the mean and variance of the Gaussian noise;

[0090] 20. Based on the average brightness of the pure black background area of ​​the prior target and variance var(I airlight Construct random noise with the same parameters:

[0091]

[0092] 21. Its spectral distribution is as follows:

[0093] F ′ niose =F{N'<A(1-t(x))>}

[0094] 22. The new objective for the unknown region estimated in the previous step using the noise inverse solution:

[0095]

[0096] 23. Solve repeatedly at the allowed frame rate. Based on the noise parameters estimated in the previous step, repeatedly generate n random noises of the same type N. i <A(1-t(x))> (i = 1, 2, ..., n), iterate multiple times, saving the image locally, and finally average the results of the previous multiple solutions to obtain the optimized restored image:

[0097]

[0098] Where F i niose To estimate the noise N multiple times i <A(1-t(x))> The spectral distribution of S(J) target The result is obtained from multiple estimations and optimizations.

[0099] 24. Standardize the noise levels of the restored image and the original fog image to the same level;

[0100] 25. Take 20 pixels horizontally from the newly added target and calculate the average value. Draw the grayscale distribution of the original image and the restored image to show the contrast enhancement effect.

[0101] 26. Output the original fog map, restored map, truth map, and grayscale comparison before and after restoration, and label the experimental environment: estimated visibility and measured visibility.

[0102] In embodiments of the present invention, such as Figure 3 As shown, a test was conducted on a real-world scene 500 meters away. The example image shows the recovery result at a visibility of 416 meters. It can be seen that without grayscale stretching, the image contrast can be significantly improved simply by solving the estimated noise model. It should be noted that due to the preferred lens and camera module, there are still blurred outlines even under conditions of extremely low visibility where they are invisible to the human eye. This effect is also attributed to the design of the invention, but the contrast is very low.

[0103] As can be seen from the contrast difference graph, the original image has very low contrast compared to the restored image, appearing almost as a flat straight line. Even when visibility further decreases to the point where the camera-captured image is completely invisible, the invention can still recover the object's outline to a certain extent.

[0104] The above embodiments demonstrate the results of solving the fog map using only a modified physical model, aiming to illustrate the contrast enhancement effect of the invention and the advantage of multiple iterations instead of traditional multi-frame overlay. Traditional image enhancement and denoising algorithms can be used as further optimization methods, but are not shown here.

[0105] In this embodiment of the invention, the relevant steps have been specialized or simplified, and are not the only technical means, including:

[0106] 1. The noise fitting method uses a Gaussian model, but other models or multiple models can be selected for estimation based on the actual distribution.

[0107] 2. The noise generation process uses the same distribution of estimated noise parameters. However, due to the estimation error of the visibility meter and the dynamic randomness of scattering, the various parameters of the noise can be given a certain degree of randomness to construct a new noise inverse solution of the unknown area image.

[0108] 3. Transmittance t(x) can be given not only from image brightness estimation, but also directly from the visibility meter. Especially under strong scattering conditions, the signal strength is even less than the random amount of noise, resulting in a negative calculation result. In this case, it can only be given by the visibility meter.

[0109] 4. The image of the target in the unknown region estimated multiple times is processed by superimposed averaging. This is only for comparison with the traditional multi-frame superposition method. Alternatively, joint optimization methods such as comparing the similarity of each neighborhood to determine the authenticity of the restoration can be selected.

[0110] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be included in the scope of the present invention.

Claims

1. A method for scatter imaging based on partial scene priors, characterized in that, Comprise the following steps: ① Set prior target in imaging scene, estimate distribution probability model and parameters of noise by analyzing degeneration characteristics of prior target in fog day, the parameters include visibility, noise type, mean and variance, as follows: Cutting the prior target region I from the collected fog image prior (x), as follows: I prior (x) = J prior (x) * t(x) + N < A (1 - t(x)) > In the formula, J prior (x) is the prior target retro-reflective pattern region intercepted in the collected fog image, x is the spatial position coordinate of the image pixel, t(x) is the transmittance at the spatial position coordinate x, A is the global atmospheric light, i.e., the atmospheric scattering light at infinity, and N<A(1-t(x))> is the modified noise distribution probability model. Estimating the mean luminance of a pure black background region of a prior target The variance of I is denoted var(I airlight ). Estimating the mean luminance of a prior target retro-reflective pattern area The formula is as follows: In the formula, is the average intensity of the pre-scattering prior target reflection pattern region, Estimate transmittance t(x) at spatial position coordinate x, formula as follows: ② Construct image information of unknown area of same type noise inverse solution: According to the brightness mean value of the prior target pure black background region and variance var(I airlight ), construct the same parameter random noise: Its spectral distribution is: F ′ niose = F{N' < A(1 - t(x))>} The clear image J of the unknown monitoring scene is not solved according to the same type of noise above the structure target : where I target is an unknown monitoring scene region in the fog map; ③ In the allowed frame rate, solve multiple times, according to the estimated noise parameters in the last step, randomly generate n same type noise N i <A(1-t(x))>(i=1,2…n), iterative solve multiple times, and save the image to the local, finally, superimpose and average the results of the last step multiple times, get the optimized recovery result S(J target ): where F i niose is the noise N i the spectral distribution of <A(1-t(x))>.

2. The scatter imaging method based on partial scene prior according to claim 1, characterized in that, The prior target is high-contrast image with pure black background and reflective pattern, and has sparse frequency domain distribution, obvious directionality and energy concentration.

3. The scatter imaging method based on partial scene prior according to claim 1 or 2, characterized in that, The prior target is diffuse scattering target, reflective target with light field parameter modulation, self-luminous target, self-luminous target with light field parameter modulation, and the prior target can be more than one, and the acquisition system can adopt multi-camera array.

4. The scatter imaging method based on partial scene prior according to claim 1, characterized in that, The noise distribution probability model comprises one or more combinations of Gaussian noise, Rayleigh noise, Gamma noise, Poisson noise, exponential noise, uniform noise or salt and pepper noise.

Images