Security camera full-color night vision imaging optimization method and system in light-free environment

Abstract

The present application relates to a security camera full-color night vision imaging optimization method and system in a non-light-supplemented environment, which collects visible light RAW and weak near-infrared dual-channel images without auxiliary light, and synchronously acquires the sensor pixel photosensitive amplitude; a variable sliding window is constructed based on the lens dispersion radius, the dual-channel image features are hierarchically disassembled and the distortion offset is corrected; cross-channel feature fusion is realized by dynamic weighting based on layer information entropy, and photon shot noise is removed by annular neighborhood residual compensation; combined with the self-developed gray drop coefficient and multi-level dynamic threshold spiral sampling, the dark light and conventional imaging partitions are divided; the partition-bound photosensitive data is used to respectively correct the independent white balance nonlinear gain of the dark light area and slightly adjust the color of the conventional area, and the full-color image is output after splicing. The present application does not need to add light supplementing devices, relies on partition correction of optical parameters and actual measurement photosensitive data, effectively improves the defects of weak light color deviation and large noise, and improves the quality of full-color night vision imaging in zero light supplementing scenes.

Description

Technical Field

[0001] This invention belongs to the field of security camera and image processing technology, and particularly relates to a method and system for optimizing full-color night vision imaging of security cameras in environments without supplemental lighting. Background Technology

[0002] Security night vision cameras generally rely on infrared fill lights and white light fill devices to supplement the amount of light entering the low-light environment in order to achieve night imaging. However, fill light accessories have drawbacks such as high hardware cost, high power consumption, light pollution at night, and inability to be used in concealed security scenarios. Therefore, the industry has gradually developed a zero-fill light full-color night vision camera solution.

[0003] Existing night vision imaging technologies without supplemental lighting mainly fall into two categories. One type relies solely on gain enhancement of a single-channel visible light image, resulting in a sharp drop in the signal-to-noise ratio in dark areas under low-light conditions, dense noise, and widespread color distortion. The other type simply superimposes the pixel values ​​of visible light and near-infrared images for fusion, without considering lens optical parameters and actual photosensitive data for correction. This leads to severe interference from lens dispersion, stray light, and photon shot noise, resulting in misaligned edge dispersion and loss of detail in dark areas after fusion. Furthermore, existing technologies employ a unified white balance and global gain control strategy, failing to distinguish between low-light, low-signal-to-noise-ratio areas and areas of normal brightness. Dark areas and normal imaging areas share the same set of color correction parameters, leading to an imbalance in the RGB channel gain ratio in dark areas under low light, resulting in poor full-color reproduction and failing to meet the practical needs of all-weather, zero-supplemental-light, high-definition, full-color imaging in security applications. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for optimizing full-color night vision imaging of security cameras in environments without supplemental lighting, so as to solve the problems mentioned in the background art.

[0005] In view of this, the present invention provides a method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment. The method includes: acquiring dual-channel raw imaging data of a security camera in a low-light scene with zero supplementary lighting, wherein the dual-channel raw imaging data includes a visible light channel RAW image and a weak near-infrared non-supplementary light sensing image, and simultaneously acquiring pixel photosensitive amplitude parameters output by the multi-channel photosensitive sensor built into the camera module. Based on the spatial neighborhood dynamic weighted mapping rule, the visible light channel RAW image and the weak near-infrared unassisted light sensing image are respectively decomposed into layered pixel features to generate layered feature maps of the two types of images. A multi-dimensional coupled entropy weighted fusion model is used to perform cross-channel fusion processing on the two sets of hierarchical feature maps to generate a fusion benchmark image of the scene without supplementary lighting. Based on the pixel dynamic gain threshold criterion, the dark area features of the fused benchmark image are verified, and the dark light low signal-to-noise ratio region and the conventional imaging region are divided in the image. By combining photosensitive amplitude parameter zoning adaptive color restoration correction, full-color night vision imaging optimization output is achieved.

[0006] In a further embodiment of the present invention, the visible light channel RAW image and the weak near-infrared unassisted light sensing image are respectively decomposed into layered pixel features based on the spatial neighborhood dynamic weighted mapping rule to generate layered feature maps for the two types of images. This includes: delineating a variable-size sliding window neighborhood based on the optical diffusion radius of the camera lens, extracting surface color primitive features and bottom brightness detail features for the visible light channel RAW image window by window to form a visible light dual-layer feature map; A sliding window for weak near-infrared image adaptation is constructed synchronously with reference to the scaling ratio of the neighborhood of the visible smooth window. Near-infrared contour structure features and dark-light latent pixel features are extracted to form a near-infrared double-layer feature map. The sliding window boundary offset of the two types of spectra is dynamically corrected based on the lens's factory optical distortion coefficient, eliminating the feature misalignment error caused by lens stray light under zero illumination.

[0007] In a further embodiment of the present invention, the method of using a multi-dimensional coupled entropy weighted fusion model to perform cross-channel fusion processing on the two sets of layered feature maps to generate a fusion reference image of a scene without supplementary lighting includes: calculating the local neighborhood information entropy of each layer of visible light spectrum and each layer of near-infrared spectrum respectively, and generating independent weighting coefficients for the corresponding layers based on the entropy values; Construct a layer cross-matching index table, and pre-fuse the visible light color primitives and near-infrared contour pixels at the same spatial location by superimposing pixel values ​​according to weighting coefficients to obtain the initial fused image; The neighboring pixel residual compensation operation is performed on the initial fused image to remove abrupt pixels caused by photon shot noise under zero supplementary lighting conditions, and finally the fused reference image is output.

[0008] In this invention, a further implementation is to verify the dark area features of the fusion reference image based on the pixel dynamic gain threshold criterion, and to divide the dark light low signal-to-noise ratio region and the conventional imaging region in the image, including: setting a multi-level dynamic gain threshold with the single pixel light sensitivity limit value as a reference, and using spiral pixel traversal sampling along the row and column direction of the fusion reference image. During the sampling process, the grayscale difference coefficient between the sampled pixel and its 8 neighboring pixels is simultaneously calculated, and each pixel is partitioned and marked in combination with the gain threshold. After summarizing consecutive pixel blocks with the same label and merging scattered micro blocks, the system is divided into low-light low signal-to-noise ratio partition and regular imaging partition. The grayscale difference coefficient is calculated using the following formula: In the formula: S (i,j)Let P(i,j) be the grayscale difference coefficient of the pixel at coordinate (i,j), P(i,j) be the grayscale value of the pixel at the target position, and P(i+m,j+n) be the grayscale value of the eight neighboring pixels. (i,j) is the local entropy of the target pixel, (Emax is the maximum local entropy within the partition).

[0009] In a further embodiment of the present invention, the full-color night vision imaging optimization output is completed by combining the photosensitive amplitude parameter zoning adaptive color restoration correction, including: retrieving the synchronously acquired multi-channel photosensitive amplitude parameters and binding and mapping the sensor photosensitive amplitude with the low light low signal-to-noise ratio zoning one by one; For low-signal-to-noise ratio low-light zones, independent white balance offset correction is adopted for each zone, and the gain ratio of the RGB three channels is dynamically adjusted based on the fluctuation of photosensitive amplitude. The conventional imaging zone adopts a global color fine-tuning strategy, which slightly corrects the color cast based on the average sensitivity of the entire image; The low-light image after partition correction is stitched together with the regular image after fine-tuning to output a full-color night vision optimized image with zero supplementary light.

[0010] In a further embodiment of the present invention, a partition-independent white balance offset correction is adopted for low light and low signal-to-noise ratio partitions, and the gain ratio of the RGB three channels is dynamically adjusted based on the photosensitive amplitude fluctuation. This includes: statistically analyzing the mean and amplitude fluctuation variance of all photosensitive amplitudes within the bound partition, and using the fluctuation variance as the gain adjustment weight. Set the original reference gain coefficients for the RGB three colors, and then use the adjustment weights to perform non-linear scaling on the reference gain of the three colors respectively; The original RGB pixel values ​​of each partition are replaced pixel by pixel based on the scaled real-time gain coefficient, eliminating the whitening and greening differences in the image caused by low light environments with zero fill light.

[0011] In a further embodiment of the present invention, before acquiring the dual-channel raw imaging data of the security camera in a low-light scene with zero supplementary lighting, an optical preprocessing step is also included: pre-compensating the pixel dispersion of the lens imaging path by adjusting the chromatic dispersion parameters of the camera optical lens in advance. By correcting the original pixel position of the dual-channel image through the lens dispersion coefficient, edge color dispersion distortion caused by dispersion under zero-light conditions is avoided.

[0012] In a further embodiment of the present invention, the pixel photosensitive amplitude parameters output by the built-in multi-channel photosensitive sensor of the camera module are acquired simultaneously, including: the sensor collects photosensitive charge values ​​in real time according to the visible light band and the near-infrared weak light band, and converts the charge values ​​into standardized photosensitive amplitude values. A photosensitive amplitude matrix is ​​constructed according to the row and column arrangement order of the image pixels, so that the amplitude matrix and the pixel coordinates of the dual-channel image are stored in a one-to-one correspondence.

[0013] In a further embodiment of the present invention, a neighboring pixel residual compensation operation is performed on the initial fused image to remove abrupt pixels caused by photon shot noise under zero supplementary lighting conditions, including: selecting an annular neighborhood centered on the target pixel in the initial fused image and calculating the average gray value of the pixels in the annular neighborhood. The deviation between the gray level of the target pixel and the average gray level is compared. If the deviation exceeds the preset residual threshold, it is determined to be a pixel with a sudden noise change. The residual compensation noise reduction is achieved by replacing the grayscale of abruptly changed pixels with the average grayscale value of the annular neighborhood.

[0014] In a further embodiment of the present invention, a method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment is implemented. The system includes: a dual-channel image acquisition module for acquiring visible light channel RAW images and weak near-infrared non-supplementary light sensing images of security cameras in low-light scenes with zero supplementary lighting. The photosensitive parameter acquisition module is used to synchronously acquire the pixel photosensitive amplitude parameters output by the multi-channel photosensitive sensor built into the camera module; The hierarchical decomposition module is used to dynamically weighted map image features in the spatial neighborhood and generate hierarchical feature maps. The cross-channel fusion module is used for multi-dimensional coupled entropy weighted fusion to generate a fused baseline image; The partition determination module is used for dynamic gain threshold verification to divide the dark light partition into a regular partition; The color correction output module is used for zone-adaptive color restoration correction, outputting an optimized full-color night view image.

[0015] The beneficial effects of this invention are: This invention acquires visible light RAW and weak near-infrared dual-channel images without supplemental lighting, simultaneously matches the photosensitive amplitude parameters of multi-channel sensors, and dynamically constructs a variable sliding window based on the lens diffusion radius and distortion coefficient to complete the layered feature decomposition. It corrects the feature shift caused by stray light and optical distortion from the imaging source. Combined with coupling entropy weighted cross-channel fusion and annular neighborhood residual compensation noise reduction, it effectively suppresses photon shot noise in zero-supplement lighting environment, solves the defects of dual-channel fusion misalignment and edge dispersion distortion, and significantly improves the detail integrity of low-light images.

[0016] This invention utilizes spiral sampling and grayscale difference coefficient formula partitioning to accurately divide low-light, low signal-to-noise ratio zones into regular imaging zones. Based on the partition-bound mean photosensitive amplitude and fluctuation variance, it adaptively adjusts the RGB gain of each region nonlinearly. The low-light zone features independent white balance offset correction, while the regular zone features lightweight color fine-tuning. This differentiated color control of the zones fundamentally improves the problem of whiteness and greenness in low-light images, significantly enhancing the accuracy of full-color reproduction in night vision images without supplemental lighting. Attached Figure Description

[0017] Figure 1This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0018] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," and "horizontal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0019] This embodiment provides an optimization method for full-color night vision imaging of security cameras in environments without supplemental lighting. The method includes: acquiring dual-channel raw imaging data of the security camera in low-light scenes with zero supplemental lighting. The dual-channel raw imaging data includes visible light channel RAW images and weak near-infrared images without supplemental light sensing. Simultaneously, the pixel photometric amplitude parameters output by the multi-channel photosensitive sensor built into the camera module are acquired. Image acquisition is carried out under the condition of natural low light at night and no artificial supplemental lighting source. All active light-emitting supplemental lighting devices such as infrared lamps and white light supplemental lights are removed from the camera hardware, and the pre-processing functions of automatic gain enhancement and global noise reduction built into the imaging chip are turned off. The visible light channel directly acquires Bayer format RAW raw images in the 400nm-650nm natural visible light band, fully preserving the weak pixel charge data in low-light environments and avoiding the compression and loss of dark detail information caused by pre-processing. The weak near-infrared channel switches to the 780nm-920nm transmittance level through an electronically controlled filter at the front of the lens, without activating any auxiliary light source, passively acquiring near-infrared RAW images solely based on ambient stray light and trace infrared radiation. The camera module is equipped with a multi-channel photosensitive sensor that is one-to-one matched with the pixel position of the imaging CMOS. The sensor and image output are controlled by the same timing clock, ensuring that the full-frame pixel photoelectric signal acquisition is completed simultaneously with each frame output. The sensor converts the acquired photoelectric raw charge signals into standardized pixel photosensitive amplitude values ​​in the 0-255 range through analog-to-digital conversion, achieving simultaneous retention of image information and photosensitive physical data. By combining native RAW direct sampling with hardware-synchronized sampling of light-sensing parameters, the material costs and power consumption of supplementary lighting hardware are eliminated, and the real physical light-sensing data can provide a reliable basis for subsequent image processing. Compared with the traditional solution that relies solely on image pixels for algorithm deduction, the data accuracy is significantly improved.

[0020] Based on a dynamic weighted mapping rule for spatial neighborhood, layered pixel feature decomposition is performed on RAW images of the visible light channel and weak near-infrared images without auxiliary light sensing, generating layered feature maps for each type of image. Abandoning the industry-standard method of using fixed-size windows for feature extraction, this method dynamically adjusts the sampling window size based on real-time lens optical parameters. Features are layered according to image information attributes, including color, brightness, contour, and weakly sensitive latent pixels. Each feature data type is stored independently in separate layers, physically separating and preserving color and detail information. This layered decomposition mode can separately retain weakly lit pixels that are directly discarded as noise by conventional image processing algorithms, maximizing the extraction of image details contained in limited light intake under zero-light conditions, and effectively improving the ability to retain details in low-light images.

[0021] A multi-dimensional coupled entropy weighted fusion model is employed to perform cross-channel fusion processing on the two sets of layered feature maps, generating a fusion baseline image for a scene without supplementary lighting. Instead of using fixed weighting coefficients for dual-channel pixel fusion, the fusion weights are dynamically generated in real-time based on the information entropy corresponding to the detail richness of each layer. In areas with intact color, the fusion ratio of the visible light layer is increased, while in areas with missing dark contours, the weight of the near-infrared layer is automatically increased. After fusion, targeted residual compensation and noise reduction processing are applied. This dynamically coupled weighted fusion method can adapt to changes in brightness in different areas of the image, balancing the restoration of native visible light colors with the supplementation of near-infrared dark contours. The point-to-point noise reduction method also avoids the problems of blurred edges and texture loss caused by traditional global filtering.

[0022] Based on the pixel dynamic gain threshold criterion, the dark area features of the fused benchmark image are verified, dividing the image into low-signal-to-noise ratio (SNR) dark areas and regular imaging areas. Multi-level brightness thresholds are dynamically generated based on the inherent minimum sensitivity limit of the image sensor. These threshold values ​​change synchronously with the average sensitivity level of the entire image. This, combined with a sampling method that spirals outward from the image center and a self-developed grayscale difference calculation formula, is used to jointly determine pixel attributes. Finally, connected component processing is used to regularize scattered pixel blocks. Spiral sampling prioritizes fine-grained partitioning of key monitoring areas in the image center. The combination of multi-level dynamic thresholds and grayscale difference effectively resists random clutter interference in low-light environments. The partition boundaries accurately match the transition between light and dark areas in the real scene, improving the accuracy of area division.

[0023] By combining photosensitive amplitude parameters with zoned adaptive color restoration correction, full-color night vision imaging optimization output is achieved. Using previously collected hardware-measured photosensitive data and zone locations, two independent color correction logics are configured for low-light and normal-brightness areas respectively. A separate white balance correction rule is set for low-light areas, while only minor color adjustments are made for normal-brightness areas. After zone correction, a gradient smoothing process is applied at the image stitching area. Zoned differential correction fundamentally solves the color difference defects commonly found in low-light images in zero-supplementary-light environments, such as a tendency towards whitening and greening, while simultaneously preventing color distortion in normal-brightness images due to over-color adjustment, ensuring consistent color performance across the entire image.

[0024] In this embodiment, the layered pixel feature decomposition of the visible light channel RAW image and the weak near-infrared image without auxiliary light sensing, based on the spatial neighborhood dynamic weighted mapping rule, generates layered feature maps for each type of image. This includes: defining a variable-size sliding window neighborhood based on the optical blur radius of the camera lens; extracting surface color primitive features and bottom brightness detail features for each sliding window of the visible light channel RAW image to form a visible light dual-layer feature map; the camera master control retrieves the measured optical blur radius parameters of the lens corresponding to different focal lengths and aperture settings stored in the device firmware, and calculates the actual pixel size of the sliding window by combining the device's real-time focus and aperture opening values. The sliding window specifications dynamically change with the lens's working state. Within a single sliding window, the surface layer is based on the Bayer RGB pixel cluster, and the statistical pixel distribution interval and pixel extreme values ​​of each channel are used as color primitive features; the bottom layer generates brightness detail features by statistically analyzing the gray-level gradient abrupt change points and gradient change amplitude of adjacent pixels. The two types of features are stored separately to form a visible light dual-layer feature map. The variable sliding window can match the real-time diffusion changes of the lens. Compared with fixed window sampling, it can reduce the feature extraction deviation caused by lens stray light and improve the accuracy of color and detail feature extraction.

[0025] A near-infrared image adaptation sliding window is constructed synchronously with the visible smooth window neighborhood scaling ratio. Near-infrared contour structure features and dark-light latent pixel features are extracted to form a near-infrared dual-layer feature map. The inherent field-of-view scaling factor of near-infrared imaging is retrieved and combined with the current visible light sliding window size to calculate the near-infrared-adaptive sliding window, eliminating window matching errors caused by inconsistent fields of view in different photosensitive bands. Within the adaptation sliding window, the shape information of the object is extracted based on the boundary of pixel grayscale abrupt changes as contour structure features. Weak-response pixels with grayscale values ​​in the range from the lower limit of sensitivity to 15% of the full range are individually selected as dark-light latent pixel features. These two types of features are combined to form the near-infrared dual-layer feature map. The adaptation scaling window avoids spatial misalignment of features from different channels, and the separate inclusion of weakly sensitive latent pixels can fully utilize the trace ambient light in zero-supplement lighting environments to uncover imaging details in dark areas.

[0026] The offset of the sliding window boundaries of the two types of image maps is dynamically corrected based on the lens's factory optical distortion coefficient, eliminating feature misalignment errors caused by stray light from the lens under zero illumination. Lens radial and tangential distortion calibration parameters are pre-programmed into the firmware. After feature map generation, the coordinates of the four sides of each sliding window are offset according to the distortion offset pixel values, correcting the actual pixel capture range of the sliding window. This pre-compensation for coordinate offsets caused by optical distortion during feature generation prevents pixel ghosting and blurred image edges during subsequent image fusion.

[0027] In this embodiment, the multi-dimensional coupled entropy weighted fusion model is used to perform cross-channel fusion processing on the two sets of layered feature maps to generate a fusion benchmark image for a scene without supplementary lighting. This includes: calculating the local neighborhood information entropy of each layer of the visible light spectrum and each layer of the near-infrared spectrum, and generating independent weighting coefficients for the corresponding layers based on the entropy values; using the aforementioned dynamic variable sliding window as the basic calculation unit for information entropy, calculating the local information entropy layer by layer and window by window for the visible light dual-layer spectrum and the near-infrared dual-layer spectrum, normalizing all entropy values ​​to the 0-1 value range, and directly using the normalized result as the real-time fusion weight of the corresponding layer. By dynamically allocating layer weights based on local information entropy, image areas with rich details automatically obtain a higher fusion ratio, realizing adaptive adjustment of fusion parameters according to the image content.

[0028] A layer cross-matching index table is constructed. Visible light color primitives and near-infrared contour pixels at the same spatial location are pre-fused by superimposing pixel values ​​according to weighted coefficients to obtain an initial fused image. A layer cross-matching index table is then built using the image pixel planar coordinates as the unique index key. This table uniformly records pixel coordinates, the original pixel values ​​of each layer, and the corresponding real-time weighted coefficients. Following the weighted summation operation rules, pixels from multiple layers at the same coordinate are superimposed to generate an initial fused image without noise reduction. Standardized indexes bind coordinates to various operation parameters, avoiding cross-layer pixel matching errors and improving the accuracy of pixel fusion operations.

[0029] The initial fused image undergoes neighborhood pixel residual compensation calculations to remove abrupt pixel changes caused by photon shot noise under zero-supplementation conditions, ultimately outputting a fused baseline image. A subsequently defined annular neighborhood judgment rule is used to screen for pulse-type noise points within the image. Only pixels identified as noise are replaced with grayscale values, while the remaining valid pixels retain their original fused values. This point-to-point noise replacement scheme removes shot noise caused by random photon motion while preserving fine image textures, preventing detail loss due to large-scale smoothing filtering.

[0030] In this embodiment, the dark area features of the fusion reference image are verified based on the pixel dynamic gain threshold criterion. The image is divided into dark-light, low signal-to-noise ratio zones and regular imaging zones. This includes: setting multi-level dynamic gain thresholds with the single-pixel light sensitivity limit as a reference; using spiral pixel traversal sampling along the row and column directions of the fusion reference image; and establishing three linked thresholds—dark light judgment threshold, intermediate transition threshold, and regular brightness threshold—based on the minimum photosensitive charge parameter calibrated at the imaging sensor's factory. These three thresholds rise and fall synchronously with the average photosensitive amplitude of the entire image. The sampling starting point is fixed at the center pixel of the image, and the entire image is traversed clockwise outwards. The three-level dynamic thresholds can adapt to the overall brightness fluctuations of the image, and the spiral sampling logic prioritizes and refines the key monitoring areas in the center of the image, improving the zoning accuracy of critical areas.

[0031] During sampling, the grayscale difference coefficient between the sampled pixel and its eight neighboring pixels is simultaneously calculated. This coefficient, combined with a gain threshold, is used to partition and label each pixel. After each pixel is sampled, a preset calculation formula is used to calculate the grayscale difference coefficient. The calculated coefficient is then compared with a three-level dynamic threshold to complete the pixel attribute labeling. This method, which integrates pixel grayscale and local entropy parameters for judgment, offers stronger resistance to random noise interference in low light compared to the traditional method that relies solely on grayscale threshold partitioning.

[0032] By aggregating consecutive pixel blocks with the same label and merging scattered micro-blocks, two imaging zones are created: a low-light, low-signal-to-noise ratio zone and a regular imaging zone. An 8-connectivity algorithm is used to group spatially consecutive pixels with the same label into independent blocks. Isolated micro-blocks with fewer than five pixels are merged into adjacent main subject zones, resulting in two large, complete imaging zones. Merging these fragmented blocks reduces the frequency of switching computational logic during subsequent color correction, lowers the computational overhead of the embedded processing chip, and facilitates real-time image processing deployment.

[0033] The grayscale difference coefficient is calculated using the following formula: In the formula: S(i,j) is the gray level difference coefficient of the pixel at coordinate (i,j), P(i,j) is the gray level value of the pixel at the target position, P(i+m,j+n) is the gray level value of the eight neighboring pixels, E(i,j) is the local information entropy of the target pixel, and Emax is the maximum local information entropy within the partition.

[0034] In this embodiment, adaptive color restoration correction based on photosensitive amplitude parameters is combined to achieve optimized output of full-color night vision imaging. This includes: retrieving synchronously acquired multi-channel photosensitive amplitude parameters and mapping the sensor's photosensitive amplitude to low-light, low-signal-to-noise-ratio zones; retrieving a previously stored two-dimensional photosensitive amplitude matrix and, using pixel row and column coordinate indexing, binding the photosensitive data within the matrix point-to-point to the corresponding zones. Low-light zones and regular imaging zones each collect all photosensitive data within their respective blocks, with data from different zones isolated and independently participating in correction parameter calculations. By binding zones to hardware-measured photosensitive data, color correction parameters are generated in real-time from actual photosensitive conditions, overcoming the shortcomings of traditional algorithms that rely on fixed parameters for color adjustment, resulting in poor environmental adaptability.

[0035] For low-signal-to-noise ratio areas in low-light conditions, independent white balance shift correction is employed, dynamically adjusting the gain ratio of the RGB three channels based on fluctuations in photosensitive amplitude. A separate independent white balance calculation channel is allocated for low-light areas, generating RGB three-color gain adjustment parameters in real time based on the fluctuation amplitude of photosensitive data within the area, with the gain values ​​of the three color channels being independently controlled. This zone-based white balance can accurately match the photosensitive characteristics of low-light environments, specifically improving color cast issues in dark areas of scenes with zero fill light.

[0036] The standard imaging zones employ a global color fine-tuning strategy, subtly correcting color cast based on the average ISO value of the entire image. The standard imaging areas maintain a unified white balance benchmark across the entire image, fine-tuning RGB channel parameters only within a small range based on the average ISO value of the entire frame, strictly limiting the magnitude of color correction. This small-scale fine-tuning mode, while slightly correcting inherent color casts in bright areas, avoids oversaturation and distortion issues in images with normal brightness.

[0037] The low-light image after zonal correction is stitched together with the fine-tuned regular image to output a zero-light full-color night vision optimized image. The two types of zonal-corrected images are stitched together in situ according to their original pixel spatial coordinates. At the boundary between the zonal areas, adjacent pixels are smoothed using gradient interpolation to eliminate the visible boundary lines caused by the zonal stitching. This boundary smoothing ensures a natural and continuous transition between light and dark areas across the entire image, improving the overall visual appeal of the final product.

[0038] In this embodiment, independent white balance shift correction is adopted for low-light, low signal-to-noise ratio zones. The gain ratio of the RGB three channels is dynamically adjusted based on the fluctuation of photosensitive amplitude. This includes: statistically analyzing the mean and amplitude fluctuation variance of all photosensitive amplitudes within the bound zone, using the fluctuation variance as the gain adjustment weight; traversing all bound photosensitive amplitude data within the low-light zone, obtaining the zone's mean amplitude through arithmetic mean calculation, and using the data fluctuation variance to characterize the degree of light and dark fluctuations within the zone; the larger the variance value, the higher the corresponding RGB gain adjustment weight. By dynamically controlling the color adjustment weight based on the actual fluctuation of photosensitive data, it can adapt to uneven low-light environments at night, automatically increasing the correction intensity in areas with drastic light and dark fluctuations.

[0039] A set of original RGB three-color reference gain coefficients is established, and non-linear scaling is applied to the three-color reference gain using adjustment weights. The original R, G, and B color reference gain coefficients are pre-calibrated and stored at the camera's manufacturing stage. Based on the aforementioned gain adjustment weights, non-linear calculations are used to scale the three-color reference gain separately, with the scaling operations for the red, green, and blue color channels being independent of each other. This channel-specific non-linear scaling can accurately compensate for the inherent defect of uneven RGB three-color light sensitivity in low-light environments.

[0040] The original RGB pixel values ​​of each zone are replaced pixel by pixel based on the scaled real-time gain coefficient, eliminating the whitening and greening biases caused by low-light environments with zero supplementary lighting. All pixels in the dark light zone are traversed, and the scaled real-time RGB gain coefficient is multiplied by the corresponding pixel's three-channel original values ​​to refresh the image color data from the pixel level. Color difference correction is performed directly at the pixel level, achieving natural full-color restoration in dark areas without any auxiliary lighting throughout the process.

[0041] In this embodiment, before acquiring the dual-channel raw imaging data of the security camera in a low-light scene with zero supplementary lighting, an optical preprocessing step is also included: pre-compensating for pixel dispersion in the lens imaging path by adjusting the camera's optical lens dispersion parameters; the device firmware pre-stores the lens dispersion calibration coefficients corresponding to the visible light and weak near-infrared bands; at the moment of imaging startup, the main control reads the current focal length and aperture parameters of the lens, matches the corresponding dispersion data, and completes pixel coordinate compensation in advance during the RAW image pixel forming stage. This pre-processing dispersion compensation counteracts the pixel position shift caused by lens dispersion from the sensor head, and compared to performing dispersion correction after imaging, it does not lose the original weak pixel information in dark areas.

[0042] By correcting the original pixel positions of the dual-channel image using the lens dispersion coefficient, edge color dispersion distortion caused by dispersion under zero-light conditions is avoided. The visible light channel and the weak near-infrared channel are each assigned their corresponding wavelength dispersion coefficients to independently correct the horizontal and vertical coordinate offsets of the pixels. This dual-channel differential dispersion correction can resolve edge color fringing and ghosting defects in the blended image caused by inconsistencies in the dispersion coefficients of the two photosensitive wavelengths.

[0043] In this embodiment, the pixel photosensitive amplitude parameters output by the built-in multi-channel photosensitive sensor of the camera module are acquired simultaneously. This includes: the sensor acquiring photosensitive charge values ​​in real time according to the visible light band and the near-infrared weak light band, and converting the charge values ​​into standardized photosensitive amplitudes; the sensor internally divides into two independent photosensitive sub-units for visible light and near-infrared light, and the photoelectric charge acquisition is completed separately for each channel. The two analog charge signals are uniformly converted to a standardized amplitude range of 0-255 by an analog-to-digital conversion circuit. Independent sampling of each channel can accurately distinguish the photosensitive differences between the two bands, providing sub-measured data support for subsequent zone-based differentiated color adjustment.

[0044] A photosensitive amplitude matrix is ​​constructed according to the row and column arrangement of image pixels, achieving a one-to-one correspondence between the amplitude matrix and the pixel coordinates of the dual-channel image. The row and column numbers of the photosensitive amplitude matrix perfectly match the XY coordinates of the image pixels, and images of the same frame and their corresponding amplitude matrices are stored in the device's local storage unit with a unified timestamp. This one-to-one coordinate binding storage format facilitates rapid data retrieval during subsequent partition correction and adapts to the low-computing-power real-time processing requirements of the camera's embedded chip.

[0045] In this embodiment, a neighboring pixel residual compensation operation is performed on the initial fused image to remove abrupt pixels caused by photon shot noise under zero-supplementation lighting. This includes: selecting a ring-shaped neighborhood centered on the target pixel in the initial fused image and calculating the average gray value of the pixels within the ring-shaped neighborhood; constructing a hollow ring-shaped sampling region centered on the pixel to be detected and surrounded by eight neighboring pixels, and removing the central target pixel when calculating the average gray value of the region. The ring-shaped sampling structure avoids interference from the pixels to be detected, making the calculation benchmark of the average gray value more objective and improving the accuracy of noise detection.

[0046] The target pixel's grayscale is compared to the average grayscale. If the deviation exceeds a preset residual threshold, the pixel is identified as a noise mutation pixel. The residual threshold is dynamically adjusted synchronously with the average photosensitive amplitude of the entire image. The threshold range is automatically widened in dark areas to prevent weak, effective dark pixels from being misclassified as noise. The dynamic threshold adapts to changes in image brightness, maximizing the preservation of effective pixel information in dark areas while efficiently reducing noise.

[0047] The method replaces the grayscale of pixels with abrupt changes using the average grayscale value of the circular neighborhood, thus achieving residual compensation noise reduction. Only pixels identified as impulse noise undergo grayscale replacement; the remaining pixels retain their original fused grayscale values. This single-point fixed-point replacement noise reduction mode preserves the original texture details of the image to the greatest extent, overcoming the blurring effect caused by traditional global filtering.

[0048] This embodiment provides a full-color night vision imaging optimization system for security cameras in environments without supplemental lighting. It implements a method for optimizing full-color night vision imaging in such environments. The system includes: a dual-channel image acquisition module for acquiring visible light RAW images and weak near-infrared images without supplemental lighting in low-light scenarios; the module integrates an electronically controlled filter switching device and a supplemental lighting hardware power-off control circuit. In night vision mode, the hardware directly cuts off the power supply to all supplemental lighting components and simultaneously drives the filter to switch to the near-infrared transmission level, ensuring the entire system operates without supplemental lighting from a hardware structure perspective. This system is suitable for concealed security scenarios where supplemental lighting cannot be deployed.

[0049] The photosensitive parameter acquisition module is used to synchronously acquire the pixel photosensitive amplitude parameters output by the built-in multi-channel photosensitive sensor of the camera module. The module and the dual-channel image acquisition module are connected to the same hardware synchronization clock to ensure that the photosensitive sampling action and image acquisition timing are completely synchronized, avoid timing misalignment between image data and photosensitive parameters, and ensure accurate subsequent coordinate binding and mapping.

[0050] The layered decomposition module is used for dynamically weighted mapping of spatial neighborhood to decompose image features and generate layered feature maps. The module has a built-in subroutine for real-time retrieval of lens optical parameters. During operation, it automatically reads relevant parameters such as focal length, aperture, distortion, and chromatic aberration, and drives a variable sliding window to complete the layered feature extraction calculation. Optical parameters and algorithms are linked in real time, and the entire processing logic can adaptively adjust according to lens conditions.

[0051] The cross-channel fusion module is used for multi-dimensional coupled entropy weighted fusion to generate a fused baseline image. Internally, the module consists of three independent subroutines: layer index construction, hierarchical information entropy calculation, and ring-shaped neighborhood residual compensation. These subroutines complete the fusion and noise reduction operations step-by-step. This modular design facilitates segmented scheduling of hardware computing power by the main control chip, adapting to the 25fps real-time video output requirements of security cameras.

[0052] The partitioning module is used for dynamic gain threshold verification to distinguish between dark and normal partitions. The module embeds the spiral pixel sampling logic and grayscale difference coefficient calculation formula, leveraging the chip's built-in floating-point unit to accelerate numerical calculations, completing full-image pixel partitioning in milliseconds. Hardware-accelerated self-developed calculation formulas effectively shorten partitioning time and ensure real-time algorithm implementation.

[0053] The color correction output module is used for zone-specific adaptive color restoration correction, outputting optimized full-color night vision images. The module features a dual-branch processing program: independent white balance calculation for low-light zones and minor color fine-tuning for regular zones. After zone correction, it automatically completes smooth image stitching and video encoding output. The integrated hardware and software design allows the entire system to be directly mass-produced and installed in various security camera devices, eliminating the need for supplementary lighting hardware while achieving all-weather, zero-supplementary-light full-color night vision imaging.

[0054] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for optimizing full-color night vision imaging in security cameras under conditions without supplemental lighting, characterized in that, The method includes: acquiring dual-channel raw imaging data of a security camera in a low-light scene with zero supplementary light, wherein the dual-channel raw imaging data includes a visible light channel RAW image and a weak near-infrared image without supplementary light sensing, and simultaneously acquiring pixel photosensitive amplitude parameters output by the multi-channel photosensitive sensor built into the camera module; Based on the spatial neighborhood dynamic weighted mapping rule, the visible light channel RAW image and the weak near-infrared unassisted light sensing image are respectively decomposed into layered pixel features to generate layered feature maps of the two types of images. A multi-dimensional coupled entropy weighted fusion model is used to perform cross-channel fusion processing on the two sets of hierarchical feature maps to generate a fusion benchmark image of the scene without supplementary lighting. Based on the pixel dynamic gain threshold criterion, the dark area features of the fused benchmark image are verified, and the dark light low signal-to-noise ratio region and the conventional imaging region are divided in the image. By combining photosensitive amplitude parameter zoning adaptive color restoration correction, full-color night vision imaging optimization output is achieved.

2. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 1, characterized in that, The spatial neighborhood dynamic weighted mapping rule is used to perform layered pixel feature decomposition on visible light channel RAW images and weak near-infrared unassisted light sensing images to generate layered feature maps for the two types of images. This includes: delineating a variable-size sliding window neighborhood based on the optical diffusion radius of the camera lens, extracting surface color primitive features and bottom brightness detail features for visible light channel RAW images window by window to form a visible light dual-layer feature map. A sliding window for weak near-infrared image adaptation is constructed synchronously with reference to the scaling ratio of the neighborhood of the visible smooth window. Near-infrared contour structure features and dark-light latent pixel features are extracted to form a near-infrared double-layer feature map. The sliding window boundary offset of the two types of spectra is dynamically corrected based on the lens's factory optical distortion coefficient, eliminating the feature misalignment error caused by lens stray light under zero illumination.

3. The method for optimizing full-color night vision imaging for security cameras in a non-supplementary lighting environment according to claim 2, characterized in that, The method employs a multi-dimensional coupled entropy weighted fusion model to perform cross-channel fusion processing on the two sets of layered feature maps to generate a fusion benchmark image for a scene without supplementary lighting. This includes: calculating the local neighborhood information entropy of each layer of visible light and each layer of near-infrared light, and generating independent weighting coefficients for the corresponding layers based on the entropy values. Construct a layer cross-matching index table, and pre-fuse the visible light color primitives and near-infrared contour pixels at the same spatial location by superimposing pixel values ​​according to weighting coefficients to obtain the initial fused image; The neighboring pixel residual compensation operation is performed on the initial fused image to remove abrupt pixels caused by photon shot noise under zero supplementary lighting conditions, and finally the fused reference image is output.

4. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 1, characterized in that, Based on the pixel dynamic gain threshold criterion, the dark area features of the fusion reference image are verified, and the dark light low signal-to-noise ratio region and the regular imaging region are divided into the image. This includes: setting a multi-level dynamic gain threshold with the single pixel light sensitivity limit as a reference, and using spiral pixel traversal sampling along the row and column direction of the fusion reference image. During the sampling process, the grayscale difference coefficient between the sampled pixel and its 8 neighboring pixels is simultaneously calculated, and each pixel is partitioned and marked in combination with the gain threshold. After summarizing consecutive pixel blocks with the same label and merging scattered micro blocks, the system is divided into low-light low signal-to-noise ratio partition and regular imaging partition. The grayscale difference coefficient is calculated using the following formula: In the formula: S (i,j) Let P(i,j) be the grayscale difference coefficient of the pixel at coordinate (i,j), P(i,j) be the grayscale value of the pixel at the target position, and P(i+m,j+n) be the grayscale value of the eight neighboring pixels. (i,j) is the local entropy of the target pixel, (Emax is the maximum local entropy within the partition).

5. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 4, characterized in that, By combining the photosensitive amplitude parameter zoning adaptive color restoration correction, the full-color night vision imaging optimization output is completed, including: retrieving the synchronously acquired multi-channel photosensitive amplitude parameters and binding and mapping the sensor photosensitive amplitude with the low light low signal-to-noise ratio zoning one by one; For low-signal-to-noise ratio low-light zones, independent white balance offset correction is adopted for each zone, and the gain ratio of the RGB three channels is dynamically adjusted based on the fluctuation of photosensitive amplitude. The conventional imaging zone adopts a global color fine-tuning strategy, which slightly corrects the color cast based on the average sensitivity of the entire image; The low-light image after partition correction is stitched together with the regular image after fine-tuning to output a full-color night vision optimized image with zero supplementary light.

6. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 5, characterized in that, For low-signal-to-noise ratio low-light zones, independent white balance offset correction is adopted for each zone. The gain ratio of the three RGB channels is dynamically adjusted based on the fluctuation of photosensitive amplitude. This includes: statistically analyzing the mean and amplitude fluctuation variance of all photosensitive amplitudes in the bound zone, and using the fluctuation variance as the gain adjustment weight. Set the original reference gain coefficients for the RGB three colors, and then use the adjustment weights to perform non-linear scaling on the reference gain of the three colors respectively; The original RGB pixel values ​​of each partition are replaced pixel by pixel based on the scaled real-time gain coefficient, eliminating the whitening and greening differences in the image caused by low light environments with zero fill light.

7. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 1, characterized in that, Before acquiring dual-channel raw imaging data of security cameras in low-light scenarios with zero supplementary lighting, there is also an optical preprocessing step: pre-compensating pixel dispersion of the lens optical parameters in advance. By correcting the original pixel position of the dual-channel image through the lens dispersion coefficient, edge color dispersion distortion caused by dispersion under zero-light conditions is avoided.

8. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 1, characterized in that, Simultaneously acquire pixel photosensitive amplitude parameters output by the built-in multi-channel photosensitive sensor of the camera module, including: the sensor collects photosensitive charge values ​​in real time according to the visible light band and the near-infrared weak light band, and converts the charge values ​​into standardized photosensitive amplitude values; A photosensitive amplitude matrix is ​​constructed according to the row and column arrangement order of the image pixels, so that the amplitude matrix and the pixel coordinates of the dual-channel image are stored in a one-to-one correspondence.

9. The method for optimizing full-color night vision imaging of security cameras in a non-supplementary lighting environment according to claim 3, characterized in that, The neighboring pixel residual compensation operation is performed on the initial fused image to remove abrupt pixels caused by photon shot noise under zero supplementary lighting conditions. This includes: selecting a ring-shaped neighborhood centered on the target pixel in the initial fused image and calculating the average gray value of the pixels in the ring-shaped neighborhood. The deviation between the gray level of the target pixel and the average gray level is compared. If the deviation exceeds the preset residual threshold, it is determined to be a pixel with a sudden noise change. The residual compensation noise reduction is achieved by replacing the grayscale of abruptly changed pixels with the average grayscale value of the annular neighborhood.

10. A full-color night vision imaging optimization system for security cameras in environments without supplemental lighting, characterized in that: The system is used to perform the method according to any one of claims 1-9, the system comprising: a dual-channel image acquisition module for acquiring visible light channel RAW images and weak near-infrared images without auxiliary light sensing from a security camera in a low-light scene with zero supplementary lighting; The photosensitive parameter acquisition module is used to synchronously acquire the pixel photosensitive amplitude parameters output by the multi-channel photosensitive sensor built into the camera module; The hierarchical decomposition module is used to dynamically weighted map image features in the spatial neighborhood and generate hierarchical feature maps. The cross-channel fusion module is used for multi-dimensional coupled entropy weighted fusion to generate a fused baseline image; The partition determination module is used for dynamic gain threshold verification to divide the dark light partition into a regular partition; The color correction output module is used for zone-adaptive color restoration correction, outputting an optimized full-color night view image.

Images