V-CPA computational photonic imaging system and control method for low-light scenarios
By selecting spectral bands, adjusting the image sensor orientation, and modulating light energy under low-light conditions, combined with digital distortion correction, the problems of unreasonable light energy resource allocation and insufficient environmental adaptability in existing low-light imaging schemes are solved, achieving efficient and accurate low-light imaging results.
Patent Information
- Application Number
- CN202611295018.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-25
- Publication Date
- 2026-11-17
AI Technical Summary
Existing low-light imaging solutions pursue multiple indicators at the physical optics level, such as full spectrum, wide-angle adaptability, full-frame uniform light and low distortion, resulting in unreasonable allocation of light energy resources, insufficient light intake and signal conversion rate, and a lack of dynamic adaptability to environmental changes.
By determining the target spectral band and performing wavelength-selective filtering, dynamically adjusting the image sensor's orientation, performing spatial light energy modulation and compensation, and combining digital distortion correction, a target light field is formed and imaging quality is optimized.
It significantly improves light intake and transmittance under low-light conditions, enhances photoelectric conversion efficiency and imaging quality, possesses high distortion correction accuracy and environmental adaptability, and achieves system miniaturization and efficient imaging.
Smart Images

Figure CN123123743A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence vision and image sensing technology, and more specifically, to a V-CPA computational photon imaging system and control method for low-light scenarios. Background Technology
[0002] High-quality imaging under low-light conditions is a recognized technical challenge in the field of optical imaging. Existing solutions typically rely on purely physical optical methods such as increasing the number of lenses and introducing aspherical lenses to simultaneously pursue full-spectrum imaging and low geometric distortion at the optical level, but they suffer from the following technical problems: First, in order to eliminate the chromatic aberration of light in different wavelengths, existing lenses need to stack a large number of lenses. After the incident light passes through multiple layers of glass media, the transmittance is gradually attenuated, and the effective light energy reaching the image sensor is very limited, which restricts the amount of light entering the lens in low-light scenes.
[0003] Second, the optimal incident angle of light relative to the image sensor varies for different wavelengths, while the sensor orientation of existing cameras remains fixed and cannot be adaptively adjusted for the incident light wavelength, resulting in a decrease in photoelectric conversion efficiency in edge areas and mismatched wavelengths, and insufficient utilization of weak light signals.
[0004] Third, existing optical systems are designed with the goal of achieving absolute uniform light across the entire image, which usually comes at the cost of sacrificing overall brightness and contrast. This results in the image quality of the core imaging area not being prioritized and the allocation of light energy resources being unreasonable.
[0005] Fourth, in order to correct optical distortion, existing lenses need to add aspherical lenses. These lenses block some incident light and limit the maximum usable aperture, further restricting low-light imaging capabilities.
[0006] In addition, many existing digital distortion correction schemes are based on fixed optical structures with pre-calibrated parameters, and adopt an open-loop, one-time correction method. They do not consider the residual distortion introduced by the dynamic adjustment of the sensor attitude, and also lack a closed-loop evaluation and iterative correction mechanism for the correction effect. They are difficult to adapt to the dynamic changes in environmental conditions during continuous low-light imaging, and the correction accuracy and stability are limited.
[0007] In summary, existing low-light imaging solutions simultaneously pursue multiple indicators at the physical optics level, such as full-spectrum, wide-angle adaptability, full-frame uniform light, and low distortion. These indicators are mutually restrictive and generally rely on increasing the complexity of optical components to achieve them, which further compresses the amount of light received and the signal conversion rate, and also limits the miniaturization and large aperture of the system.
[0008] Therefore, how to balance light intake, signal conversion rate, core image illumination uniformity and distortion correction accuracy through optics and algorithms while reducing the dependence on the complexity of physical optical devices, and enabling the system to dynamically adapt to environmental changes, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0009] To overcome a series of shortcomings in existing technologies, the purpose of this application is to provide a V-CPA computational photon imaging control method for low-light scenarios, comprising the following steps: The target spectral band is determined based on the low-light environment information of the scene to be imaged, and the incident light is wavelength selectively filtered to allow effective light within the target spectral band to pass through. Based on the optical response characteristics corresponding to the target spectral band, adjust the attitude parameters of the image sensor relative to the optical axis so that the incident light is incident on the image sensor at the optimal incident angle. The core imaging area is determined based on the image to be output, and the incident light energy distribution is spatially modulated based on the target illumination uniformity requirements corresponding to the core imaging area. The modulated illumination distribution is compensated to form a target light field that meets the target illumination uniformity requirements within the core imaging region; The system acquires raw image data from an image sensor, performs physical optical distortion analysis on the raw image data, identifies residual distortion features caused by attitude adjustment, and performs digital distortion compensation on the raw image data based on the residual distortion features to generate a distortion-corrected image.
[0010] In some embodiments, the method for determining the target spectral band is as follows: Based on the illuminance sensing data, color temperature distribution data, and background radiation noise data of the scene to be imaged, a feature vector of the low-light environment is constructed. Based on the feature vector of the low-light environment, a set of candidate spectral bands is determined; Obtain the signal-to-noise ratio, effective photon flux, and imaging contrast for each candidate spectral band in the candidate spectral band set. Based on the predicted signal-to-noise ratio, effective photon flux and imaging contrast of each candidate spectral band, a comprehensive score is obtained for each candidate spectral band. The candidate spectral band with the highest comprehensive score is selected as the target spectral band.
[0011] In some embodiments, the method for determining the core imaging region based on the image to be output is as follows: Based on the subject position, aspect ratio, and edge cropping range of the image to be output, the image to be output is divided into regions. Based on a pre-defined human visual sensitivity distribution model, the visual importance weights corresponding to each segmented region are determined. The target region is selected based on the comparison between the visual importance weight and the preset weight threshold, and adjacent target regions are combined to form a continuous region. The boundaries of the continuous region are smoothed, and the processed continuous region is determined as the core imaging region.
[0012] In some embodiments, the method for spatially modulating the energy distribution of incident light is as follows: Based on the target illumination uniformity requirements of each divided region within the core imaging area and the current incident light energy distribution, a modulation coefficient matrix is determined. The modulation coefficient matrix includes the light energy transmittance adjustment amount or reflectance adjustment amount of each pixel unit. The incident light is spatially energy modulated based on the modulation coefficient matrix to obtain modulated light. The optical energy of the modulated light and its energy distribution feedback signal are collected, and the actual optical energy distribution is determined based on the energy distribution feedback signal; The actual light energy distribution is compared with the target illumination uniformity requirement to determine the illumination uniformity deviation; The modulation coefficient correction amount is determined based on the illumination uniformity deviation, and the modulation coefficient matrix is updated based on the modulation coefficient correction amount. Spatial energy modulation and feedback correction are repeated until the actual light energy distribution meets the target illumination uniformity requirements.
[0013] In some embodiments, the method for forming the target light field is as follows: The core imaging area is divided into multiple compensation sub-units. The illumination compensation amount is determined based on the difference between the actual illumination intensity and the desired illumination intensity of each compensation sub-unit, and the supplementary lighting parameters are determined accordingly. Based on the supplementary lighting parameters, local light energy is supplemented to each compensation subunit to obtain the light intensity distribution and color temperature distribution after supplementary lighting; The actual illumination uniformity is determined based on the illumination intensity distribution, and compared with the target illumination uniformity requirement to determine the illumination compensation correction amount. The color temperature distribution is compared with the preset color temperature consistency requirements, and the adjustment amount of the emission spectrum ratio is determined based on the color temperature deviation. The supplementary lighting parameters are updated and iteratively compensated based on the illumination compensation correction amount and the emission spectrum ratio adjustment amount until the actual illumination uniformity and color temperature distribution meet the corresponding requirements. The resulting light field is then determined as the target light field.
[0014] In some embodiments, the method for determining residual distortion features is as follows: Obtain the preset feature points and their corresponding image coordinates from the original image data; Based on the current pose parameters of the image sensor and the pre-established optical system distortion model, the theoretical coordinates corresponding to each preset feature point are determined. Based on the coordinate deviation between the image coordinates and the corresponding theoretical coordinates, determine the distortion offset vector of each preset feature point; Interpolation fitting is performed on each distortion offset vector to construct a continuous distortion offset field, and residual distortion features are determined based on the continuous distortion offset field.
[0015] In some embodiments, the method for generating the distortion-corrected image is as follows: The inverse mapping parameters corresponding to the pixel positions of each image are determined based on the continuous distortion offset field, and the original image sampling coordinates corresponding to the pixel positions of each image are determined based on the inverse mapping parameters. Obtain the corresponding neighboring pixel data based on the sampling coordinates of each original image, and perform interpolation calculations to determine the corresponding corrected pixel value; Determine whether the sampling coordinates of each original image exceed the range of the original image. The sampling coordinates that exceed the range of the original image are identified as out-of-bounds sampling coordinates. Boundary filling is performed on the out-of-bounds sampling coordinates, and the corresponding filling pixel values are determined. A distortion-corrected image is generated based on the correction pixel value or fill pixel value corresponding to each image pixel position.
[0016] In some embodiments, it also includes: The quality of the distortion-corrected image is evaluated to determine the distortion residual corresponding to the current distortion compensation result and to determine whether the distortion residual meets the preset compensation conditions. If the conditions are not met, the digital distortion compensation parameters are updated based on the distortion residuals, and the distortion compensation process is repeated until the preset compensation conditions are met. If satisfied, output the distortion-corrected image of the corresponding frame; During continuous low-light imaging, the target spectral band adjustment, incident angle optimization, light field modulation, and distortion compensation processes are repeatedly executed based on environmental changes in subsequent acquisition frames to achieve dynamic optimization of imaging quality.
[0017] In some embodiments, the method for determining the distortion residual is as follows: Obtain multiple sets of feature points and their corresponding image coordinates in the distortion-corrected image; Calculate the actual geometric relationship parameters between each set of feature points based on the image coordinates of each set of feature points. Obtain the theoretical geometric relationship parameters corresponding to each group of feature points, and determine the corresponding geometric deviation parameters based on the deviation between the actual geometric relationship parameters and the theoretical geometric relationship parameters; The geometric deviation parameters corresponding to each group of feature points are normalized to obtain the normalized geometric deviation parameters. The distortion residuals corresponding to the distortion-corrected image are obtained by weighting each normalized geometric deviation parameter according to the preset weight coefficients.
[0018] The purpose of this application is also to provide a V-CPA computational photon imaging system for low-light scenarios, for implementing the above-mentioned control methods, including: The spectral filtering module is used to determine the target spectral band and perform wavelength-selective filtering on the incident light; The attitude adjustment module is used to adjust the attitude of the image sensor relative to the optical axis according to the optical response characteristics of the target spectral band, so that the incident light is incident at the optimal angle. The light field modulation module is used to spatially modulate and compensate the energy distribution of incident light according to the illumination uniformity requirements of the core imaging area, so as to form the target light field. An image sensor is used to acquire raw image data under the target light field; The distortion compensation module is used to analyze the residual distortion caused by pose adjustment in the original image data, perform digital compensation and evaluate the distortion residual. If the preset conditions are not met, iterative compensation is performed until the conditions are met and then the distortion-corrected image is output. The control module is used to coordinate the control of the above modules according to environmental changes during continuous low-light imaging, and to perform band selection, angle optimization, light field modulation and distortion compensation in a loop to achieve dynamic optimization of imaging quality.
[0019] Compared with the prior art, this application has the following beneficial effects: Firstly, by determining the target spectral band based on the feature vector of the low-light environment and performing wavelength selective filtering, only the core spectral band required for imaging is allowed to pass through. This eliminates the need for a large number of chromatic aberration correction lenses in traditional optical solutions to cover the entire spectrum, thereby reducing optical path loss at the physical level and significantly improving the amount of light entering and the transmittance under low-light conditions.
[0020] Secondly, by dynamically adjusting the image sensor orientation according to the optimal incident angle corresponding to the target spectral band, the incident light is always incident on the photosensitive surface at the optimal angle. Compared with the traditional approach with fixed orientation, this can effectively improve the photoelectric conversion efficiency of weak light signals and improve the image signal-to-noise ratio.
[0021] Third, by performing targeted spatial light energy modulation and active supplemental lighting on the core imaging area, the uniformity and contrast of illumination in the core image area are prioritized within a limited energy budget. Compared with the traditional solution that aims for absolute uniformity, it can achieve better image quality of key images under the same hardware conditions.
[0022] Fourth, by establishing a continuous offset field model of residual distortion and combining it with a closed-loop iterative evaluation mechanism for digital distortion compensation, it is possible to accurately compensate for residual distortion introduced by attitude adjustment and large aperture light transmission. Compared with the traditional open-loop, one-time digital correction scheme, it has higher distortion correction accuracy and stronger environmental adaptability.
[0023] Fifth, the control module coordinates and schedules modules such as spectral filtering, attitude adjustment, light field modulation, and distortion compensation. During continuous low-light imaging, the system adaptively triggers the recalculation of the corresponding modules according to environmental changes, so that the system can avoid unnecessary recalculation while ensuring imaging quality, and balance processing real-time performance with engineering feasibility.
[0024] Sixth, since the present invention allows for a certain degree of distortion and non-absolutely uniform illumination in the optical system at the physical level, and then improves the final imaging quality through algorithm-level compensation, it can achieve better low-light imaging effects than traditional solutions with a smaller optical volume and a larger aperture, and has good miniaturization potential and engineering scalability. Attached Figure Description
[0025] Figure 1 A schematic diagram of the overall process of the V--CPA computational photon imaging control method for low-light scenarios provided in this embodiment of the invention; Figure 2 This is a flowchart illustrating the method for determining the target spectral band in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the image sensor attitude parameter adjustment method in an embodiment of the present invention. Figure 4 This is a schematic flowchart of the incident light energy distribution spatial modulation and optical field compensation method in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the residual distortion analysis and distortion compensation iterative method in an embodiment of the present invention. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some embodiments of this invention, but not all embodiments.
[0027] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0028] The embodiments and directional terms described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0029] In a broad embodiment of the present invention, the V-CPA computational photon imaging control method for low-light scenarios includes the following steps: The target spectral band is determined based on the low-light environment information of the scene to be imaged, and the incident light is wavelength selectively filtered to allow effective light within the target spectral band to pass through. Based on the optical response characteristics corresponding to the target spectral band, adjust the attitude parameters of the image sensor relative to the optical axis so that the incident light is incident on the image sensor at the optimal incident angle. The core imaging area is determined based on the image to be output, and the incident light energy distribution is spatially modulated based on the target illumination uniformity requirements corresponding to the core imaging area. The modulated illumination distribution is compensated to form a target light field that meets the target illumination uniformity requirements within the core imaging region; The raw image data acquired by the image sensor is obtained, physical optical distortion analysis is performed on the raw image data to determine the residual distortion features caused by attitude adjustment, and digital distortion compensation is performed on the raw image data based on the residual distortion features to generate a distortion-corrected image. The quality of the distortion-corrected image is evaluated to determine the distortion residual corresponding to the current distortion compensation result and to determine whether the distortion residual meets the preset compensation conditions. If the conditions are not met, the digital distortion compensation parameters are updated based on the distortion residuals, and the distortion compensation process is repeated until the preset compensation conditions are met. If satisfied, output the distortion-corrected image of the corresponding frame; During continuous low-light imaging, the target spectral band adjustment, incident angle optimization, light field modulation, and distortion compensation processes are repeatedly executed based on environmental changes in subsequent acquisition frames to achieve dynamic optimization of imaging quality.
[0030] In some embodiments, the method for determining the target spectral band is as follows: Based on the illuminance sensing data, color temperature distribution data, and background radiation noise data of the scene to be imaged, a feature vector of the low-light environment is constructed. Based on the feature vector of the low-light environment, a set of candidate spectral bands is determined; Obtain the signal-to-noise ratio, effective photon flux, and imaging contrast for each candidate spectral band in the candidate spectral band set. Based on the predicted signal-to-noise ratio, effective photon flux and imaging contrast of each candidate spectral band, a comprehensive score is obtained for each candidate spectral band. The candidate spectral band with the highest comprehensive score is selected as the target spectral band.
[0031] In some embodiments, the method for adjusting the attitude parameters of the image sensor relative to the optical axis is as follows: Obtain a pre-established image sensor response angle characteristic database, which includes multiple discrete spectral band sampling points, as well as the optimal incident angle, quantum efficiency, and pixel crosstalk coefficient corresponding to each discrete spectral band sampling point; Obtain the center wavelength of the target spectral band and determine at least two discrete spectral band sampling points adjacent to the center wavelength; The optimal incident angle of the target spectral band is determined by interpolation calculation based on the optimal incident angles corresponding to at least two discrete spectral band sampling points. Based on the optimal incident angle of the target and the spatial attitude relationship between the image sensor and the optical axis, the target attitude parameters corresponding to the image sensor are determined. The target attitude parameters include pitch angle, yaw angle and roll angle. Generate attitude adjustment instructions based on the target attitude parameters, and adjust the attitude of the image sensor relative to the optical axis based on the attitude adjustment instructions; Obtain the actual attitude parameters of the adjusted image sensor and calculate the attitude deviation between the actual attitude parameters and the target attitude parameters; The attitude correction amount is determined based on the attitude deviation, and the attitude of the image sensor is corrected based on the attitude correction amount.
[0032] In some embodiments, the method for determining the core imaging region based on the image to be output is as follows: Based on the subject position, aspect ratio, and edge cropping range of the image to be output, the image to be output is divided into regions. Based on a pre-defined human visual sensitivity distribution model, the visual importance weights corresponding to each segmented region are determined. The target region is selected based on the comparison between the visual importance weight and the preset weight threshold, and adjacent target regions are combined to form a continuous region. The boundaries of the continuous region are smoothed, and the processed continuous region is determined as the core imaging region.
[0033] In some embodiments, the method for spatially modulating the energy distribution of incident light is as follows: Based on the target illumination uniformity requirements of each divided region within the core imaging area and the current incident light energy distribution, a modulation coefficient matrix is determined. The modulation coefficient matrix includes the light energy transmittance adjustment amount or reflectance adjustment amount of each pixel unit. The incident light is spatially energy modulated based on the modulation coefficient matrix to obtain modulated light. The optical energy of the modulated light and its energy distribution feedback signal are collected, and the actual optical energy distribution is determined based on the energy distribution feedback signal; The actual light energy distribution is compared with the target illumination uniformity requirement to determine the illumination uniformity deviation; The modulation coefficient correction amount is determined based on the illumination uniformity deviation, and the modulation coefficient matrix is updated based on the modulation coefficient correction amount. Spatial energy modulation and feedback correction are repeated until the actual light energy distribution meets the target illumination uniformity requirements.
[0034] In some embodiments, the method for forming the target light field is as follows: The core imaging area is divided into multiple compensation sub-units. The illumination compensation amount is determined based on the difference between the actual illumination intensity and the desired illumination intensity of each compensation sub-unit, and the supplementary lighting parameters are determined accordingly. Based on the supplementary lighting parameters, local light energy is supplemented to each compensation subunit to obtain the light intensity distribution and color temperature distribution after supplementary lighting; The actual illumination uniformity is determined based on the illumination intensity distribution, and compared with the target illumination uniformity requirement to determine the illumination compensation correction amount. The color temperature distribution is compared with the preset color temperature consistency requirements, and the adjustment amount of the emission spectrum ratio is determined based on the color temperature deviation. The supplementary lighting parameters are updated and iteratively compensated based on the illumination compensation correction amount and the emission spectrum ratio adjustment amount until the actual illumination uniformity and color temperature distribution meet the corresponding requirements. The resulting light field is then determined as the target light field.
[0035] In some embodiments, the method for determining residual distortion features is as follows: Obtain the preset feature points and their corresponding image coordinates from the original image data; Based on the current pose parameters of the image sensor and the pre-established optical system distortion model, the theoretical coordinates corresponding to each preset feature point are determined. Based on the coordinate deviation between the image coordinates and the corresponding theoretical coordinates, determine the distortion offset vector of each preset feature point; Interpolation fitting is performed on each distortion offset vector to construct a continuous distortion offset field, and residual distortion features are determined based on the continuous distortion offset field.
[0036] In some embodiments, the method for generating the distortion-corrected image is as follows: The inverse mapping parameters corresponding to the pixel positions of each image are determined based on the continuous distortion offset field, and the original image sampling coordinates corresponding to the pixel positions of each image are determined based on the inverse mapping parameters. Obtain the corresponding neighboring pixel data based on the sampling coordinates of each original image, and perform interpolation calculations to determine the corresponding corrected pixel value; Determine whether the sampling coordinates of each original image exceed the range of the original image. The sampling coordinates that exceed the range of the original image are identified as out-of-bounds sampling coordinates. Boundary filling is performed on the out-of-bounds sampling coordinates, and the corresponding filling pixel values are determined. A distortion-corrected image is generated based on the correction pixel value or fill pixel value corresponding to each image pixel position.
[0037] In some embodiments, the method for determining the distortion residual is as follows: Obtain multiple sets of feature points and their corresponding image coordinates in the distortion-corrected image; Calculate the actual geometric relationship parameters between each set of feature points based on the image coordinates of each set of feature points. Obtain the theoretical geometric relationship parameters corresponding to each group of feature points, and determine the corresponding geometric deviation parameters based on the deviation between the actual geometric relationship parameters and the theoretical geometric relationship parameters; The geometric deviation parameters corresponding to each group of feature points are normalized to obtain the normalized geometric deviation parameters. The distortion residuals corresponding to the distortion-corrected image are obtained by weighting each normalized geometric deviation parameter according to the preset weight coefficients.
[0038] In some embodiments, the digital distortion compensation parameters are updated based on the distortion residual, and the distortion compensation process is repeated, specifically including: When the distortion residual is greater than the preset residual threshold, the parameter adjustment direction and parameter adjustment step size are determined based on the distortion residual and the pre-established distortion compensation parameter iterative model. The digital distortion compensation parameters are updated according to the parameter adjustment direction and parameter adjustment step size, and distortion compensation is performed based on the updated digital distortion compensation parameters to obtain the updated distortion-corrected image. The updated distortion-corrected image is quality-assessed to determine the updated distortion residuals. When the updated distortion residual is greater than the preset residual threshold and the current iteration number has not reached the preset maximum iteration number, the update of the digital distortion compensation parameters, distortion compensation, and quality assessment are repeated. When the updated distortion residual value is less than or equal to the preset residual threshold, or when the current iteration number reaches the preset maximum iteration number, the updated distortion-corrected image is determined as the output image.
[0039] In some embodiments, during continuous low-light imaging, the target spectral band adjustment, incident angle optimization, light field modulation, and distortion compensation processes are repeatedly executed based on environmental changes in subsequent acquisition frames, specifically including: Obtain the real-time low-light environment feature vector corresponding to the current subsequent acquisition frame, and determine the environmental change state based on the comparison result between the real-time low-light environment feature vector and the preset environmental change threshold. When the magnitude of environmental change exceeds the preset environmental change threshold, the target spectral band, image sensor attitude parameters, spatial light modulation coefficient matrix and digital distortion compensation parameters are re-determined based on the real-time weak light environment feature vector, and spectral range adjustment, attitude adjustment, light energy modulation and distortion compensation are performed. When the magnitude of environmental change does not exceed the preset environmental change threshold, the target spectral band, image sensor attitude parameters, and spatial light modulation coefficient matrix are maintained, and the digital distortion compensation parameters are updated according to the real-time low light environment feature vector. Based on the updated digital distortion compensation parameters, distortion compensation is performed on the current subsequent acquisition frame, and the next subsequent acquisition frame is acquired, repeating the above process.
[0040] The purpose of this application is also to provide a V-CPA computational photon imaging system for low-light scenarios, for implementing the above-mentioned control method, including: The spectral filtering module is used to determine the target spectral band and perform wavelength-selective filtering on the incident light; The attitude adjustment module is used to adjust the attitude of the image sensor relative to the optical axis according to the optical response characteristics of the target spectral band, so that the incident light is incident at the optimal angle. The light field modulation module is used to spatially modulate and compensate the energy distribution of incident light according to the illumination uniformity requirements of the core imaging area, so as to form the target light field. An image sensor is used to acquire raw image data under the target light field; The distortion compensation module is used to analyze the residual distortion caused by pose adjustment in the original image data, perform digital compensation and evaluate the distortion residual. If the preset conditions are not met, iterative compensation is performed until the conditions are met and then the distortion-corrected image is output. The control module is used to coordinate the control of the above modules according to environmental changes during continuous low-light imaging, and to perform band selection, angle optimization, light field modulation and distortion compensation in a loop to achieve dynamic optimization of imaging quality.
[0041] The preferred embodiments of the present invention are described below with reference to the accompanying drawings, and the present invention will be further described in detail.
[0042] The V-CPA computational photonic imaging system and control method for low-light scenarios involved in this embodiment are mainly applied to low-light (low-illuminance) imaging scenarios, such as nighttime vehicle perception, security monitoring, and machine vision recognition. By performing wavelength selective filtering on the incident light, dynamically adjusting the image sensor's attitude, and spatially modulating and compensating for the incident light energy distribution, combined with closed-loop iteration of digital distortion compensation, high-quality low-light imaging is achieved while ensuring the amount of light received and the signal conversion rate.
[0043] like Figure 1 As shown, the V-CPA computational photonic imaging control method for low-light scenes provided in this embodiment is applied to a V-CPA computational photonic imaging system for low-light scenes, which includes a spectral filtering module, an attitude adjustment module, a light field modulation module, an image sensor, and a distortion compensation module. The method includes the following steps: determining the target spectral band based on the low-light environment information of the scene to be imaged, and performing wavelength-selective filtering on the incident light to ensure that effective light within the target spectral band passes through the filtering unit; adjusting the attitude parameters of the image sensor relative to the optical axis according to the optical response characteristics corresponding to the target spectral band, so that the incident light is incident on the photosensitive surface of the image sensor at the optimal incident angle; determining the core imaging area according to the image to be output, and spatially modulating and compensating the incident light energy distribution based on the target illumination uniformity requirements corresponding to the core imaging area. A target light field is formed within the imaging area; the original image data collected by the image sensor under the target light field is acquired, and physical optical distortion analysis is performed on the original image data to determine the residual distortion features caused by attitude adjustment. Based on the residual distortion features, digital distortion compensation is performed on the original image data to generate a distortion-corrected image; the distortion-corrected image is quality-assessed to determine the distortion residual corresponding to the current distortion compensation result, and it is determined whether the distortion residual meets the preset compensation conditions. If not, the digital distortion compensation parameters are updated according to the distortion residual, and the distortion compensation process is repeated until the preset compensation conditions are met, and then the distortion-corrected image of the corresponding frame is output; during continuous low-light imaging, the target spectral band adjustment, incident angle optimization, light field control, and distortion compensation processes are repeatedly executed according to the environmental changes of subsequent acquisition frames to achieve dynamic optimization of imaging quality.
[0044] like Figure 2 As shown, in a specific application scenario, the method for determining the target spectral band includes: firstly, acquiring illuminance sensing data, color temperature distribution data, and background radiation noise data of the scene to be imaged, and then constructing a low-light environment feature vector. Its expression is shown in formula (1): (1) In the formula, This represents the illuminance sensing data of the scene to be imaged. Its physical meaning is the ambient light illuminance of the scene, and the unit is lux. It is derived from real-time sampling by the ambient light sensor. This represents the scene color temperature distribution data, which physically represents the correlated color temperature of the dominant light source. The unit is Kelvin, and it is derived from color temperature sensors or multispectral presampling estimation. This represents background radiation noise data, which physically represents the level of noise caused by the superposition of dark current and background stray light in the current environment. The unit is the number of electrons, and it is derived from the sensor's dark field calibration data.
[0045] Based on low light environment feature vector By combining a pre-established spectral-environment mapping database, a set of candidate spectral bands is determined. For each candidate spectral band in the candidate spectral band set ( ), calculate their signal-to-noise ratio estimates respectively. Estimated effective photon flux and estimated image contrast Since the three estimated values have different dimensions (signal-to-noise ratio is a dimensionless ratio, photon flux is the number of photons per square millimeter per second, and contrast ratio is a dimensionless ratio but its range is different from the previous two), they cannot be directly weighted and summed. Therefore, it is necessary to first normalize the three estimated values according to formula (2) with the maximum value in the candidate band set as the benchmark, so as to obtain the dimensionless normalized signal-to-noise ratio, normalized photon flux, and normalized contrast ratio: (2) In the formula, Represent , , Three types of estimated values (independently normalized for each item); Candidate spectral bands The corresponding normalized values range from 0 to 1. Based on the normalization, the three normalization indices are weighted and fused according to formula (3) to obtain the candidate spectral bands. Corresponding overall score : (3) In the formula, , , These are the weighting coefficients for the three normalized indicators: signal-to-noise ratio (SNR), photon flux, and imaging contrast. The sum of these three coefficients is 1. Their specific values are pre-calibrated based on the application scenario's emphasis on SNR, light flux, and contrast. For example, in automotive night vision scenarios, where SNR is more important, a weighting coefficient of 1 can be used. =0.5、 =0.3、 =0.2. A comprehensive score is applied to the candidate spectral band set. The highest value is determined as the target spectral band. That is, it satisfies formula (4): (4) For example, in a low-light scenario in a vehicle at night, the ambient illuminance... Approximately 0.05 lx, color temperature The candidate spectral band set is approximately 4500 K, and includes two candidates: the near-infrared band (around 850 nm) and a wide visible band. After normalization and weighted calculation, the corresponding near-infrared band... ≈0.82, corresponding to the wide band of visible light ≈0.61, and the near-infrared band is selected as the target spectral band according to formula (4). Subsequently, the spectral filtering module is based on the target spectral band. The corresponding center wavelength and bandwidth are controlled to allow effective light in that band to pass through while suppressing light in other bands, thereby reducing ineffective optical path loss at the physical level and improving overall transmittance.
[0046] like Figure 3 As shown, after determining the target spectral band, the attitude parameters of the image sensor relative to the optical axis need to be adjusted according to the optical response characteristics corresponding to the target spectral band. Specifically, an image sensor response angle characteristic database is pre-established, which contains multiple discrete spectral band sampling points. ( ), and the optimal incident angle corresponding to the sampling points of each discrete spectral band. Quantum efficiency and pixel crosstalk coefficient Obtain the target spectral band. The center wavelength was determined, and two discrete spectral band sampling points adjacent to this center wavelength were identified in the response angle characteristic database. and (satisfy Then, according to formula (5), linear interpolation is performed on the corresponding optimal incident angle to obtain the optimal incident angle of the target spectral band. : (5) In the formula, , Sampling points , The corresponding optimal angle of incidence, in degrees (°); The center wavelength of the target spectral band is expressed in nanometers (nm). When the center wavelength of the target spectral band is exactly equal to the wavelength of a discrete sampling point, the optimal incident angle corresponding to that sampling point is directly taken as... Interpolation is no longer performed. Based on the target's optimal angle of incidence... And the pre-calibrated spatial attitude relationship between the image sensor and the optical axis, and the target attitude parameters corresponding to the image sensor, including the pitch angle, are calculated according to formula (6). Yaw angle and roll angle : (6) In the formula, Let be the azimuth angle of the incident light in the image sensor coordinate system. The function f(·) is a pre-calibrated attitude calculation function obtained through optical simulation and experimental measurement, used to map the optimal incident angle and azimuth angle of the target to the three-axis attitude angles required by the drive mechanism. The control module determines the target attitude parameters based on these parameters. , , An attitude adjustment command is generated to drive a micro-displacement actuator equipped with an image sensor to complete the attitude adjustment; after the adjustment is completed, the actual attitude parameters of the image sensor are obtained through an angle encoder. , , And calculate the attitude deviation according to formula (7). : (7) In the formula, This represents the overall attitude deviation between the actual attitude parameters and the target attitude parameters, expressed in degrees (°). Greater than the preset attitude deviation threshold (Example value 0.05°) When the deviation direction is taken, the control module generates an attitude correction amount based on the deviation direction and performs closed-loop correction on the actuator until the attitude deviation meets the requirements, thereby ensuring that the incident light is at the optimal incident angle to the target. Stable incident light onto the photosensitive surface of the image sensor ensures that light of different wavelengths strikes the pixel unit at the optimal angle, improving the conversion efficiency of weak light signals.
[0047] like Figure 4 As shown, based on the completed posture adjustment, the core imaging area is further determined according to the image to be output, and the incident light energy distribution within the core imaging area is spatially modulated and compensated. Specifically, the image is divided into regions based on the subject position, aspect ratio, and edge cropping range of the image to be output, and each region is calculated according to formula (8) based on the preset human visual sensitivity distribution model. Visual importance weight : (8) In the formula, Indicates the first The normalized distance weights of each region relative to the center of the image are increased as they are closer to the center. Indicates the first The salience weights of each region are given by a pre-calibrated human visual sensitivity distribution model; , These are the corresponding weighting coefficients, and their sum is 1. The visual importance weights are... Greater than the preset weight threshold The divided regions are selected as target regions, and adjacent target regions are combined and their boundaries are smoothed to obtain the core imaging region.
[0048] For each pixel unit within the core imaging region, the modulation coefficient matrix is determined according to formula (9) based on the target illumination uniformity requirements and the current incident light energy distribution. The modulation coefficient corresponding to the pixel unit with coordinates (x, y) It should be noted that spatial light modulation units (such as digital micromirror arrays or liquid crystal light valves) suppress overly bright areas by reducing local transmittance or reflectance. Physically, the transmittance cannot exceed 1, therefore the modulation coefficient must be limited to between 0 and 1. For areas where the actual illumination intensity is lower than the target value, amplification compensation is not performed through the modulation coefficient, but rather active energy replenishment is provided by the supplementary lighting unit in subsequent steps. (9) In the formula, This represents the target illumination intensity value corresponding to the target illumination uniformity requirement within the core imaging area, expressed in candela per square meter. This represents the illumination intensity value corresponding to the actual incident light energy of the pixel unit at coordinates (x, y); min(·, 1) indicates that the upper limit constraint of the modulation coefficient is 1 to avoid exceeding the physically realizable range of the spatial light modulation unit. Modulation coefficient matrix The incident light is modulated spatially by the spatial light modulation unit to obtain modulated light. Then, the energy distribution feedback signal of the modulated light is collected by the photoelectric detection feedback unit to obtain the actual light energy distribution I_actual(x,y). The overall illumination uniformity deviation of the core imaging area is calculated according to formula (10). : (10) In the formula, This represents the total number of pixel units participating in the statistics within the core imaging region; For the first The actual illumination intensity of each pixel unit. Due to factors such as nonlinear response, device calibration error, and pixel crosstalk in the spatial light modulation unit, the modulation coefficient obtained by formula (9) is an open-loop estimate. There is usually still a deviation between the actual modulation effect and the target value. Therefore, closed-loop feedback correction needs to be introduced: when the overall illumination uniformity deviation Greater than the preset uniformity deviation threshold (Example value 5%), for pixel units whose deviation exceeds the allowable range, the modulation coefficient correction amount is calculated according to formula (11): (11) In the formula, This is a preset correction gain coefficient (between 0 and 1) used to control the convergence speed of a single iteration and avoid overshoot of the modulation coefficient. The modulation coefficient correction amount... Superimposed on the original modulation coefficient Then, the amplitude is still limited to the range of 0 to 1 to obtain the updated modulation coefficient matrix, and the spatial energy modulation and feedback correction are repeated until the actual light energy distribution meets the target illumination uniformity requirements.
[0049] Based on this, the core imaging region is divided into several compensation sub-units (numbered as follows). For compensation sub-units that still cannot reach the target illumination intensity after modulation coefficient limiting (i.e., regions where the actual illumination intensity is lower than the target value), the illumination compensation amount is determined based on the difference between the actual illumination intensity and the desired illumination intensity, and the first sub-unit is determined in a proportional-integral manner according to formula (12). The adjustment amount of the supplementary lighting parameters corresponding to each compensation subunit : (12) In the formula, For the first The deviation between the actual illumination intensity and the desired illumination intensity of each compensation sub-unit , These are the proportional gain coefficient and the integral gain coefficient, respectively, and their values are pre-calibrated based on the response speed of the supplementary light actuator. For discrete time sequence numbers. For the current moment. Adjustment amount based on supplementary lighting parameters. Local light energy is supplemented to the corresponding compensation subunit to obtain the illumination intensity distribution and color temperature distribution after supplementation. The actual illumination uniformity is determined based on the illumination intensity distribution after supplementation and compared with the target illumination uniformity requirement to determine the illumination compensation correction amount. At the same time, the color temperature distribution is compared with the preset color temperature consistency requirement, and the emission spectrum ratio adjustment amount is determined based on the color temperature deviation. The two together drive the iterative update of the supplementation parameters until the actual illumination uniformity and color temperature distribution meet the corresponding requirements. The light field formed at this time is determined as the target light field.
[0050] like Figure 5 As shown, after the image sensor acquires raw image data under the target light field, the spatial position of the image sensor relative to the optical axis is changed due to attitude adjustment. This introduces residual distortion into the raw image data, which needs to be identified and compensated for. Specifically, multiple preset feature points and their image coordinates in the raw image data are obtained. , Based on the current attitude parameters of the image sensor , , Based on the pre-established optical system distortion model, the theoretical coordinates corresponding to each preset feature point are determined. , And calculate the first according to formula (13). Distortion offset vector of each feature point : (13) In the formula, , The first The theoretical image coordinates of each feature point are calculated based on the optical system distortion model. , Let be the actual image coordinates of the feature point in the original image data. Interpolation fitting is performed on the distortion offset vectors of all feature points to construct a continuous distortion offset field D(x,y), which is expressed using radial basis function interpolation according to formula (14): (14) In the formula, For the first The interpolation weight coefficients corresponding to each feature point are obtained by least-squares solving the known distortion offset vector; φ(·) is a preset radial basis kernel function (e.g., thin plate spline kernel function). The continuous distortion offset field D(x,y) represents the residual distortion features at each location in the image and is used for subsequent inverse mapping correction.
[0051] Based on the continuous distortion offset field D(x,y), the inverse mapping parameters corresponding to each image pixel position are determined, and the original image sampling coordinates (x′,y′) corresponding to each pixel position are obtained; bilinear interpolation is performed on the pixel data in the neighborhood of the sampling coordinates, and the corresponding corrected pixel value is determined according to formula (15). (x,y): (15) In the formula, (x′, y′) represents the i-th sampled coordinate in the neighborhood of the back-mapping sampling coordinates. The original pixel value of the neighboring pixels, The four weights are the corresponding bilinear interpolation weights, and their sum is 1. When the inverse mapping sampling coordinates exceed the range of the original image, they are identified as out-of-bounds sampling coordinates, and boundary padding is performed on these coordinates to obtain the corresponding padding pixel values. Based on the correction pixel values or padding pixel values corresponding to each image pixel position, a distortion-corrected image is generated.
[0052] The distortion-corrected image is quality-assessed to determine the distortion residuals. Specifically, multiple sets of feature points and their image coordinates are obtained from the distortion-corrected image, and the actual geometric relationship parameters between each set of feature points are calculated. (e.g., feature point spacing or angle), and obtain the corresponding theoretical geometric relationship parameters. Calculate the first according to formula (16) Normalized geometric deviation parameters corresponding to the group of feature points To avoid instability in the denominator caused by individual theoretical geometric parameters (such as the theoretical angle between collinear feature points) approaching zero, a preset protective constant much smaller than the typical value is introduced into the denominator. : (16) In the formula, For the first Theoretical geometric relation parameters corresponding to the group of feature points Its actual geometric relationship parameters in the distortion-corrected image. This is a preset protection constant (its value is much smaller than the normal range of the theoretical geometric relationship parameters for each group, to avoid abnormal amplification of deviation parameters due to division by zero or an excessively small denominator). Based on preset weighting coefficients... The normalized geometric deviation parameters of each group are weighted and summed, and the distortion residuals corresponding to the distortion-corrected image are obtained according to formula (17). : (17) In the formula, The number of feature point groups participating in the statistics, and the weighting coefficient. The sum is 1, and can be set differently based on the importance of each group of feature points in the image. This will affect the distortion residual. Compared with the preset residual threshold Comparison: When Greater than If the current distortion compensation result does not meet the preset compensation conditions, the digital distortion compensation parameters (i.e., the interpolation weighting coefficients of the continuous distortion migration field) are updated based on the distortion residual. Due to distortion residuals It relies on multiple non-analytical steps such as feature point extraction, reverse mapping, and interpolation correction. and Since there is no explicitly differentiable analytical expression between them, the central difference method is used according to formula (18) to... about The sensitivity is numerically approximated, and the adjustment direction and step size of the compensation parameters are determined accordingly to achieve gradient updates of the compensation parameters: (18) In the formula, For the first The continuous distortion migration field at the nth iteration Interpolation weight coefficients, These are the updated interpolation weight coefficients; The preset small perturbation amount is used to adjust the first... The interpolation weight coefficients are subjected to tentative perturbations in both positive and negative directions. The distortion residuals obtained by re-performing the reverse mapping, interpolation correction, and quality assessment before and after the perturbation are approximately determined. about The local trend of change (numerical gradient); The preset iteration step size coefficient (learning rate) is determined based on the distortion residual. The convergence speed is adaptively adjusted. Taking values that are too large can easily lead to iterative oscillations, while taking values that are too small will result in slow convergence. (Example) The initial value is 0.1, and it is dynamically adjusted according to the decrease ratio of the distortion residual between two adjacent iterations. Inverse mapping and interpolation correction are re-executed based on the updated digital distortion compensation parameters to obtain the updated distortion-corrected image. Quality assessment is then repeated to determine the updated distortion residual. When the updated distortion residual is less than or equal to a preset residual threshold... Or the current iteration count has reached the preset maximum iteration count. When the example takes 10 values, stop the iteration and determine the distortion-corrected image at this time as the output image.
[0053] In continuous low-light imaging, to adapt to dynamic changes in scene lighting conditions, this embodiment further repeats the target spectral band adjustment, incident angle optimization, light field modulation, and distortion compensation processes based on environmental changes in subsequent acquisition frames, thereby achieving dynamic optimization of imaging quality. Specifically, it obtains the real-time low-light environment feature vector corresponding to the current subsequent acquisition frame. (Note its weight as) , , Since the dimensions and typical value ranges of the three components of this feature vector are different (illuminance, color temperature, and noise electron number, respectively), directly calculating the Euclidean distance will lead to the component with the larger numerical magnitude dominating the overall judgment. Therefore, it is necessary to first normalize according to the formula (19) based on the preset typical variation range of each component, and then calculate the normalized comprehensive environmental change. : (19) In the formula, , , These are the differences between the real-time low-light environment feature vector and the low-light environment feature vector used in the previous determination of the target spectral band, respectively, in terms of the three components of illuminance, color temperature, and background noise. , , These are pre-calibrated typical variation ranges for illuminance, color temperature, and background noise components, used to convert the differences between these three components with different dimensions into dimensionless quantities that can be directly compared and superimposed. This results in a normalized comprehensive environmental change. Compared with the preset environmental change threshold (Dimensionless, example value 0.15) Compare: When Greater than When the environmental change exceeds a preset environmental change threshold, the system determines that the change is based on the real-time low-light environment feature vector. The target spectral band is redefined according to formulas (1) to (4), and the image sensor attitude parameters, spatial light modulation coefficient matrix, and digital distortion compensation parameters are redefined according to formulas (5) to (18). Spectral range adjustment, attitude adjustment, light energy modulation, and distortion compensation are then re-executed. Not greater than If the environmental change is determined to be within a preset environmental change threshold, then the current target spectral band, image sensor attitude parameters, and spatial light modulation coefficient matrix remain unchanged. Only the real-time low-light environment feature vector is considered. Update the digital distortion compensation parameters according to formulas (16) to (18), and perform distortion compensation on the current subsequent acquisition frame based on the updated digital distortion compensation parameters; after completing the current frame processing, obtain the next subsequent acquisition frame, and repeat the above judgment and processing process, so as to reduce unnecessary repeated calculation overhead and improve the real-time processing efficiency of the system while ensuring the continuous low light imaging quality.
[0054] Corresponding to the above method, this embodiment also provides a V-CPA computational photon imaging system for low-light scenarios, including a spectral filtering module, an attitude adjustment module, a light field modulation module, an image sensor, a distortion compensation module, and a control module. The spectral filtering module is used to perform the aforementioned target spectral band determination and wavelength-selective filtering process. Its hardware can be a dynamic filtering unit based on a liquid crystal tunable filter or an acousto-optic tunable filter, capable of adjusting the transmission band in real time according to the target spectral band parameters issued by the control module. The attitude adjustment module is used to perform the aforementioned image sensor attitude parameter adjustment process. Its hardware can be a three-axis micro-displacement actuator (e.g., a voice coil motor or a piezoelectric ceramic drive platform) combined with an angle encoder to form a closed-loop control. The light field modulation module is used to perform the aforementioned incident light energy distribution spatial modulation and compensation process. Its hardware can be a digital micromirror array, A liquid crystal light valve or adjustable array light source, in conjunction with a photoelectric detection feedback unit, achieves closed-loop modulation; an image sensor is used to acquire raw image data under the target light field; a distortion compensation module is used to perform the aforementioned residual distortion analysis, digital distortion compensation, and iterative evaluation process, which can be implemented by an image processing chip with parallel computing capabilities (such as a field-programmable gate array or a graphics processor); a control module is used to coordinately control the spectral filtering module, attitude adjustment module, light field modulation module, and distortion compensation module according to environmental changes during continuous low-light imaging, cyclically performing band selection, angle optimization, light field modulation, and distortion compensation to achieve dynamic optimization of imaging quality.
[0055] It should be noted that the data interaction between the above modules follows a unified timing control: the control module first issues the target spectral band parameters and target attitude parameters based on the low-light environment feature vector of the current frame; the spectral filtering module and attitude adjustment module execute wavelength filtering and attitude adjustment in parallel; after the attitude adjustment is completed, the light field control module performs spatial energy modulation and feedback compensation based on the core imaging area to form the target light field; after the target light field stabilizes, the image sensor triggers exposure, acquires raw image data and transmits it to the distortion compensation module; after the distortion compensation module completes residual distortion analysis, digital compensation and iterative evaluation, it outputs the distortion-corrected image and feeds back the state variables such as the distortion residual, illumination uniformity deviation and attitude deviation of the current frame to the control module as the basis for judging environmental changes and updating parameters in the next frame, thus forming a complete closed loop in which data acquisition, processing and analysis, result output and feedback optimization are interconnected.
[0056] In summary, this embodiment achieves increased light intake at the optical level through targeted filtering, adaptive matching of the incident angle through dynamic attitude adjustment, and ensures uniform illumination of key images through core area light field modulation and compensation. Furthermore, it offsets physical distortions introduced by attitude adjustment and large aperture light transmission through closed-loop iteration of digital distortion compensation. These four elements work synergistically, combining hardware and software, to fully release the light intake and signal conversion rate under low-light conditions while ensuring image quality in the core imaging area. Compared to traditional optical design schemes that rely on increasing the number of lenses to achieve both full spectrum and low distortion, the V-CPA computational photon imaging system and control method provided in this embodiment for low-light scenarios can achieve better low-light imaging effects with a smaller optical volume and a larger aperture, demonstrating good feasibility and engineering scalability.
[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A V-CPA computational photon imaging control method for low-light scenarios, characterized in that, Includes the following steps: The target spectral band is determined based on the low-light environment information of the scene to be imaged, and the incident light is wavelength selectively filtered to allow effective light within the target spectral band to pass through. Based on the optical response characteristics corresponding to the target spectral band, adjust the attitude parameters of the image sensor relative to the optical axis so that the incident light is incident on the image sensor at the optimal incident angle. The core imaging area is determined based on the image to be output, and the incident light energy distribution is spatially modulated based on the target illumination uniformity requirements corresponding to the core imaging area. The modulated illumination distribution is compensated to form a target light field that meets the target illumination uniformity requirements within the core imaging region; The system acquires raw image data from an image sensor, performs physical optical distortion analysis on the raw image data, identifies residual distortion features caused by attitude adjustment, and performs digital distortion compensation on the raw image data based on the residual distortion features to generate a distortion-corrected image.
2. The control method according to claim 1, characterized in that, The method for determining the target spectral band is as follows: Based on the illuminance sensing data, color temperature distribution data, and background radiation noise data of the scene to be imaged, a feature vector of the low-light environment is constructed. Based on the feature vector of the low-light environment, a set of candidate spectral bands is determined; Obtain the signal-to-noise ratio, effective photon flux, and imaging contrast for each candidate spectral band in the candidate spectral band set. Based on the predicted signal-to-noise ratio, effective photon flux and imaging contrast of each candidate spectral band, a comprehensive score is obtained for each candidate spectral band. The candidate spectral band with the highest comprehensive score is selected as the target spectral band.
3. The control method according to claim 1, characterized in that, The method for determining the core imaging region based on the image to be output is as follows: Based on the subject position, aspect ratio, and edge cropping range of the image to be output, the image to be output is divided into regions. Based on a pre-defined human visual sensitivity distribution model, the visual importance weights corresponding to each segmented region are determined. The target region is selected based on the comparison between the visual importance weight and the preset weight threshold, and adjacent target regions are combined to form a continuous region. The boundaries of the continuous region are smoothed, and the processed continuous region is determined as the core imaging region.
4. The control method according to claim 3, characterized in that, The method for spatially modulating the energy distribution of incident light is as follows: Based on the target illumination uniformity requirements of each divided region within the core imaging area and the current incident light energy distribution, a modulation coefficient matrix is determined. The modulation coefficient matrix includes the light energy transmittance adjustment amount or reflectance adjustment amount of each pixel unit. The incident light is spatially energy modulated based on the modulation coefficient matrix to obtain modulated light. The optical energy of the modulated light and its energy distribution feedback signal are collected, and the actual optical energy distribution is determined based on the energy distribution feedback signal; The actual light energy distribution is compared with the target illumination uniformity requirement to determine the illumination uniformity deviation; The modulation coefficient correction amount is determined based on the illumination uniformity deviation, and the modulation coefficient matrix is updated based on the modulation coefficient correction amount. Spatial energy modulation and feedback correction are repeated until the actual light energy distribution meets the target illumination uniformity requirements.
5. The control method according to claim 3, characterized in that, The method for forming the target light field is as follows: The core imaging area is divided into multiple compensation sub-units. The illumination compensation amount is determined based on the difference between the actual illumination intensity and the desired illumination intensity of each compensation sub-unit, and the supplementary lighting parameters are determined accordingly. Based on the supplementary lighting parameters, local light energy is supplemented to each compensation subunit to obtain the light intensity distribution and color temperature distribution after supplementary lighting; The actual illumination uniformity is determined based on the illumination intensity distribution, and compared with the target illumination uniformity requirement to determine the illumination compensation correction amount. The color temperature distribution is compared with the preset color temperature consistency requirements, and the adjustment amount of the emission spectrum ratio is determined based on the color temperature deviation. The supplementary lighting parameters are updated and iteratively compensated based on the illumination compensation correction amount and the emission spectrum ratio adjustment amount until the actual illumination uniformity and color temperature distribution meet the corresponding requirements. The resulting light field is then determined as the target light field.
6. The control method according to claim 1, characterized in that, The method for determining residual distortion features is as follows: Obtain the preset feature points and their corresponding image coordinates from the original image data; Based on the current pose parameters of the image sensor and the pre-established optical system distortion model, the theoretical coordinates corresponding to each preset feature point are determined. Based on the coordinate deviation between the image coordinates and the corresponding theoretical coordinates, determine the distortion offset vector of each preset feature point; Interpolation fitting is performed on each distortion offset vector to construct a continuous distortion offset field, and residual distortion features are determined based on the continuous distortion offset field.
7. The control method according to claim 6, characterized in that, The method for generating distortion-corrected images is as follows: The inverse mapping parameters corresponding to the pixel positions of each image are determined based on the continuous distortion offset field, and the original image sampling coordinates corresponding to the pixel positions of each image are determined based on the inverse mapping parameters. Obtain the corresponding neighboring pixel data based on the sampling coordinates of each original image, and perform interpolation calculations to determine the corresponding corrected pixel value; Determine whether the sampling coordinates of each original image exceed the range of the original image. The sampling coordinates that exceed the range of the original image are identified as out-of-bounds sampling coordinates. Boundary filling is performed on the out-of-bounds sampling coordinates, and the corresponding filling pixel values are determined. A distortion-corrected image is generated based on the correction pixel value or fill pixel value corresponding to each image pixel position.
8. The control method according to claim 1, characterized in that, Also includes: The quality of the distortion-corrected image is evaluated to determine the distortion residual corresponding to the current distortion compensation result and to determine whether the distortion residual meets the preset compensation conditions. If the conditions are not met, the digital distortion compensation parameters are updated based on the distortion residuals, and the distortion compensation process is repeated until the preset compensation conditions are met. If satisfied, output the distortion-corrected image of the corresponding frame; During continuous low-light imaging, the target spectral band adjustment, incident angle optimization, light field modulation, and distortion compensation processes are repeatedly executed based on environmental changes in subsequent acquisition frames to achieve dynamic optimization of imaging quality.
9. The control method according to claim 8, characterized in that, The method for determining the distortion residual is as follows: Obtain multiple sets of feature points and their corresponding image coordinates in the distortion-corrected image; Calculate the actual geometric relationship parameters between each set of feature points based on the image coordinates of each set of feature points. Obtain the theoretical geometric relationship parameters corresponding to each group of feature points, and determine the corresponding geometric deviation parameters based on the deviation between the actual geometric relationship parameters and the theoretical geometric relationship parameters; The geometric deviation parameters corresponding to each group of feature points are normalized to obtain the normalized geometric deviation parameters. The distortion residuals corresponding to the distortion-corrected image are obtained by weighting each normalized geometric deviation parameter according to the preset weight coefficients.
10. A V-CPA computational photon imaging system for low-light scenarios, used to implement the control method as described in any one of claims 1-9, characterized in that, include: The spectral filtering module is used to determine the target spectral band and perform wavelength-selective filtering on the incident light; The attitude adjustment module is used to adjust the attitude of the image sensor relative to the optical axis according to the optical response characteristics of the target spectral band, so that the incident light is incident at the optimal angle. The light field modulation module is used to spatially modulate and compensate the energy distribution of incident light according to the illumination uniformity requirements of the core imaging area, so as to form the target light field. An image sensor is used to acquire raw image data under the target light field; The distortion compensation module is used to analyze the residual distortion caused by pose adjustment in the original image data, perform digital compensation and evaluate the distortion residual. If the preset conditions are not met, iterative compensation is performed until the conditions are met and then the distortion-corrected image is output. The control module is used to coordinate the control of the above modules according to environmental changes during continuous low-light imaging, and to perform band selection, angle optimization, light field modulation and distortion compensation in a loop to achieve dynamic optimization of imaging quality.
