A flicker detection circuit and method of operation thereof

Abstract

This invention discloses a flicker detection circuit and method for CMOS image sensors, belonging to the field of integrated circuits and CMOS image sensor technology. It aims to address the shortcomings of existing technologies, such as inaccurate detection of high-frequency light source flicker, slow response speed, and the need for additional hardware that increases cost and chip area. This invention divides the pixel array into multiple periodically arranged flicker detection groups covering the entire field of view. The floating diffusion nodes of the pixel units within each group are electrically coupled to each other, and the signal reading is completed using the main image rolling shutter readout timing. Through an independent timing control circuit, the single exposure time of the flicker detection group is limited to within the readout time of a single row of pixels, achieving high-frequency sampling. Then, through intra-frame spatial and inter-frame temporal dual-dimensional signal processing, the fundamental frequency of light source flicker is accurately identified and the exposure time is adjusted. This invention requires no additional hardware, is compatible with mainstream pixel architectures, can achieve high-frequency flicker detection at the tens of kilohertz level, can complete preliminary identification in a single frame, has fast response, high accuracy, strong anti-interference ability, and is adaptable to various imaging terminals.

Description

Technical Field

[0001] This invention relates to the field of image sensors, and more particularly to a flicker detection circuit for a CMOS image sensor and its operation method. Background Technology

[0002] Complementary metal-oxide-semiconductor (CMOS) image sensors, with their advantages of high integration, low power consumption, and fast response speed, have become core components in imaging systems for smartphones, automotive imaging, security monitoring, and industrial machine vision. As end-user applications continue to demand higher image quality and dynamic scene adaptability, the flicker-proof imaging performance of image sensors in complex lighting environments has become a key indicator of a device's core competitiveness. In recent years, lighting and display technologies have rapidly evolved, with LED light sources having completely replaced traditional power frequency light sources such as incandescent and halogen lamps, becoming the mainstream lighting and display units in everyday, automotive, and industrial scenarios. Unlike the fixed power frequency flicker characteristics of traditional light sources, LED light sources generally employ pulse width modulation (PWM) dimming technology, offering a wide flicker frequency range, extending from hundreds of hertz to tens of kilohertz. Furthermore, the dimming frequency and duty cycle vary greatly across different application scenarios, posing new technical challenges to flicker detection and suppression in image sensors.

[0003] Existing traditional anti-flicker technologies can only detect fixed low-frequency flicker of 100Hz or 120Hz caused by industrial AC power, failing to cover the high-frequency flicker band of LED light sources. This can easily lead to a missynchronization between the horizontal exposure cycle of the image sensor's rolling shutter and the on / off cycle of the flickering light source, ultimately forming persistent band-like ripples in the image and severely degrading image quality. Furthermore, traditional flicker detection schemes rely on statistical information from multiple frames to identify flicker frequencies, resulting in slow response times and an inability to quickly adapt to sudden changes in light source in dynamic shooting scenarios, easily leading to exposure adjustment lag.

[0004] Currently, some high-end terminals address high-frequency flicker by incorporating a separate flicker sensor. However, this significantly increases hardware costs and chip layout area, hindering the miniaturization and cost-effectiveness of imaging devices. Furthermore, it suffers from issues such as mismatch between the sensor's field of view and the failure to detect local flicker signals, making widespread adoption in consumer-grade terminals difficult. In addition, purely software-optimized anti-flicker solutions suffer from high computational demands, high processor resource consumption, insufficient high-frequency flicker recognition accuracy, and high false positive rates. These shortcomings fail to address the root causes of high-frequency flicker detection's technical pain points, severely restricting the improvement of image sensor imaging performance in complex lighting environments. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing anti-flicker solutions for CMOS image sensors, such as insufficient high-frequency flicker detection capability, slow response speed, high hardware cost and chip area, and poor architecture adaptability. This invention provides a flicker detection circuit and operation method for CMOS image sensors that does not require additional independent hardware circuits. It can achieve fast and accurate detection of high-frequency flicker by simply optimizing the configuration and timing control of the existing pixel array, and is compatible with various mainstream CMOS image sensor architectures.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: On one hand, the present invention provides a flicker detection circuit for a CMOS image sensor. The CMOS image sensor includes a pixel array, which comprises multiple pixel units arranged in rows and columns. Each pixel unit includes a photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor, and a row selection transistor. The pixel array is divided into multiple flicker detection groups, which are periodically arranged in the pixel array along horizontal and vertical directions and cover the complete field of view of the image sensor. The floating diffusion nodes of all pixel units within the same flicker detection group are electrically coupled to each other. The source follower transistor and the row selection transistor of the flicker detection group are multiplexed using the rolling shutter readout timing of the main image to complete the signal readout. The circuit also includes a timing control circuit, which is electrically connected to the reset transistor and transmission transistor of the flicker detection group. This timing control circuit controls the single exposure time of the flicker detection group to not exceed the single row pixel readout time of the pixel array, making the sampling rate of the flicker detection group equivalent to the row scanning frequency. The circuit also includes a signal processing module, which is electrically connected to the signal output terminal of the flicker detection group. The signal processing module is used to identify the frequency of the flickering light source based on the readout signal of the flicker detection group and adjust the exposure time of the main image.

[0007] Furthermore, the pixel unit is a single-pixel unit or a shared pixel unit in a multi-pixel shared architecture; the single-pixel unit includes a photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor, and a row selection transistor; the shared pixel unit includes multiple photodiodes, multiple transmission transistors, and a floating diffusion node, a reset transistor, a source follower transistor, and a row selection transistor shared by all photodiodes. Furthermore, all pixel units within the same flicker detection group are exposed synchronously to compensate for the conversion gain reduction caused by the coupling of the floating diffusion node.

[0008] Furthermore, at the beginning of a single high-frequency sampling cycle, the timing control circuit coordinates the conduction of the reset transistor and the transmission transistor to open the charge discharge path from the photodiode to the power supply. Subsequently, the turn-off moment of the transmission transistor is taken as the physical starting point of the exposure. When the shutter readout pointer of the main image scans to the corresponding row, the ordinary pixel unit and the row selection transistor of the scintillation detection group in the same row are synchronously turned on to complete the reference signal readout. Then, the timing control circuit triggers the transmission transistor to turn on, transferring the photoelectrons in this exposure cycle to the coupled floating diffusion node and completing the signal output.

[0009] Furthermore, the signal output terminal of the flicker detection group shares a correlated double sampling circuit with the ordinary pixel unit, and the output terminal of the correlated double sampling circuit is connected to the core processing unit of the signal processing module; the signal readout timing of the flicker detection group is consistent with that of the ordinary pixel unit, and there is no need to configure an independent sampling circuit for the flicker detection signal.

[0010] Furthermore, the signal processing module first filters out the DC component and low-frequency drift interference from the sampled sequence, then performs a fast Fourier transform on the processed signal to obtain the spectral distribution. Based on the frequency interval and multiple relationship of significant peaks in the spectral distribution, the fundamental frequency and harmonic frequency of the pulse light source are identified, and the period corresponding to the fundamental frequency is used as the adjustment benchmark for the exposure time of the main image.

[0011] Furthermore, the multiple flicker detection groups are discretely distributed in the pixel array, and the proportion of their total number of pixels is controlled within the restoration threshold of image reconstruction; the multiple flicker detection groups are arranged in a non-collinear staggered manner in the horizontal and vertical directions to break the spatial continuity of heterogeneous pixels.

[0012] Furthermore, the timing control circuit is also configured to support a low-power intermittent sampling mode based on time multiplexing; in scenarios where there is no need to continuously update the light source frequency, the timing control circuit stops outputting the high-frequency drive clock for the flicker detection group and switches the trigger timing of the transmission transistor and reset transistor of the flicker detection group to a low-frequency long exposure mode synchronized with the ordinary pixel unit in the same row; in this mode, the flicker detection group no longer performs ultra-short stroboscopic sampling, but instead performs full-frame integration and readout as a large area of ​​high-sensitivity monochrome pixel.

[0013] Furthermore, the signal processing module also includes an image reconstruction unit, which is used to perform image reconstruction and repair on the pixel region where the flicker detection group is located: in the spatial domain, it prioritizes extracting the high signal-to-noise ratio brightness signal output by the flicker detection group in low-power mode as a guiding feature, or extracts the local brightness gradient information around the flicker detection group as a guiding feature, and performs adaptive direction interpolation on the surrounding normal pixels; in the temporal domain, it extracts the motion vectors of adjacent historical frames and the current frame, projects the normal pixels with corresponding physical space coordinates in the historical frames to the pixel blind area position of the current frame, and realizes the restoration of image details.

[0014] On the other hand, the present invention provides a flicker detection method for a CMOS image sensor, comprising the following steps: First, dividing the pixel array of the CMOS image sensor into multiple flicker detection groups, and arranging the multiple flicker detection groups periodically along the horizontal and vertical directions of the pixel array so that the flicker detection groups cover the complete field of view of the image sensor; electrically coupling the floating diffusion nodes of all pixel units within the same flicker detection group to each other; ensuring that the source follower transistor of the flicker detection group is consistent with the corresponding device of the ordinary pixel unit in the same row, and controlling the row selection transistor to multiplex the shutter readout timing of the main image to complete the signal readout; controlling the single exposure time of the flicker detection group to not exceed the single row pixel readout time of the pixel array, so that the sampling rate of the flicker detection group is equivalent to the row scanning frequency; finally, processing the readout signal of the flicker detection group, identifying the frequency of the flickering light source, and adjusting the exposure time of the main image according to the identification result.

[0015] Compared with existing technologies, this invention has significant advantages: First, it fully embeds the flicker detection mechanism into the pixel array of existing CMOS image sensors, eliminating the need for additional independent flicker sensor hardware, reducing chip layout area and hardware costs. It is also compatible with single-pixel and multi-pixel shared architectures, requiring no major modifications to the original sensor architecture, thus offering strong adaptability and facilitating large-scale mass production. Second, through short exposures within a single line duration and high-frequency sampling synchronized with the line, this invention achieves high-frequency flicker detection at the tens of kilohertz level, overcoming the limitation of traditional solutions that can only detect low-frequency flicker at power frequency. Furthermore, it can accurately identify flicker frequency in a single frame, significantly shortening response time and allowing for rapid adaptation to changes in light sources in dynamic scenes. Finally, through the detection units arranged cyclically across the entire field of view and precise fundamental frequency identification logic, this invention effectively avoids missed detections of local flicker and harmonic mis-locking, reducing the false judgment rate. The overall solution has low computational requirements, does not consume excessive processor resources, and is widely adaptable to various consumer and industrial imaging terminals. Attached Figure Description

[0016] Figure 1 This is a partial structural diagram of a CMOS image sensor circuit. Figure 2This is a timing diagram of the flicker detection circuit of the present invention; Figure 3 This is a schematic diagram illustrating the process of transmitting the flicker detection signal from the CDS to the DSP according to the present invention. Figure 4 This is a flowchart of the core algorithm of the flicker detection method of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The specific embodiments described in this section are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. All technical features disclosed in the present invention, or steps in all methods or processes disclosed, except for mutually exclusive features and / or steps, can be combined in any way.

[0018] The CMOS image sensor involved in this invention comprises a pixel array, a timing control circuit, a column readout circuit, and a signal processing module. The pixel array consists of several pixel units arranged in a regular row-column pattern. Each pixel unit includes a photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor, and a row selection transistor. The pixel units can employ a single-pixel architecture or a multi-pixel sharing architecture such as 4-pixel sharing or 8-pixel sharing. This invention is compatible with both architectures simultaneously without requiring significant modifications to the original core architecture of the sensor.

[0019] In the single-pixel architecture, each pixel unit includes an independent photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor, and a row select transistor. The anode of the photodiode is grounded, the cathode is connected to the source of the transmission transistor, the drain of the transmission transistor is connected to the floating diffusion node, the floating diffusion node is connected to the drain of the reset transistor and the gate of the source follower transistor, the source of the reset transistor is connected to the power supply voltage, the drain of the source follower transistor is connected to the power supply voltage, and the source of the source follower transistor is connected to the drain of the row select transistor. The source of the row select transistor is connected to the column bus as the signal output terminal of the pixel unit.

[0020] In a multi-pixel shared architecture, taking a 4-pixel shared architecture as an example, every 4 photodiodes and 4 transmission transistors share a floating diffusion node, a reset transistor, a source follower transistor, and a row select transistor to form a shared pixel unit. Each photodiode is connected to the source of a corresponding transmission transistor, and the drains of all transmission transistors are connected to the shared floating diffusion node. The floating diffusion node is simultaneously connected to the drain of the shared reset transistor and the gate of the source follower transistor. The source of the reset transistor is connected to the power supply voltage, the drain of the source follower transistor is connected to the power supply voltage, and the source of the source follower transistor is connected to the drain of the shared row select transistor. The source of the row select transistor is connected to the column bus. This invention can be directly adapted to multi-pixel shared architectures of different specifications; only the smallest constituent unit of the flicker detection group needs to be replaced from a single pixel unit to a shared pixel unit. Example 1

[0021] like Figure 1 As shown, the pixel array of the CMOS image sensor contains a large number of ordinary pixel units arranged in rows and columns. The area marked by the red box is the core structure of the flicker detection circuit in this embodiment, which consists of multiple flicker detection groups.

[0022] In the pixel array, multiple flicker detection groups are divided, each consisting of m rows and n columns of pixel units, where m and n are both positive integers. The value of m must meet the sampling requirement for a complete flicker signal cycle: to ensure complete capture of at least one complete cycle of the lowest 100Hz power frequency flicker in the environment, under typical operating conditions where the single-row readout time of the pixel array is 10μs, the minimum number of sampling rows required to complete the signal acquisition for one 100Hz flicker cycle is 1000 rows; therefore, the value of m must be no less than 1000. Simultaneously, to adapt to the subsequent Fast Fourier Transform (FFT) operation rules, m is preferably selected as an integer power of 2. In this embodiment, m is 1024 and n is 2, meaning each flicker detection group consists of 1024 rows and 2 columns of pixel units. In other optional embodiments, m can also be selected as 2048, 4096, or other values ​​to adapt to different sensor total row counts and sampling accuracy requirements. Since the flicker detection group does not contain complete spatial image information, image signal processing (ISP) is needed to repair relevant positions in the output image. To ensure full-field detection while strictly controlling the proportion of flicker-detected pixels within a threshold that can be repaired by image signal processing (e.g., around 3%), this embodiment sets up a flicker detection group every 126 columns of ordinary pixel units horizontally (i.e., a horizontal arrangement period of 128 columns). Vertically, to avoid lateral interpolation streaking and high-frequency moiré patterns caused by long, narrow blind zones, the flicker detection groups are arranged in a staggered "triangular" pattern: specifically, the array is divided into upper, middle, and lower horizontal bands of 1024 rows, and the flicker detection groups in adjacent bands are horizontally shifted by half a cycle (i.e., 64 columns). This results in multiple flicker detection groups being non-collinearly distributed in both the horizontal and vertical directions, disrupting the spatial continuity of heterogeneous pixels and further reducing the difficulty of image reconstruction and the risk of artifacts.

[0023] The timing control circuit is also configured to support a low-power intermittent sampling mode based on time multiplexing. In scenarios where continuous updates of the light source frequency are not required, the timing control circuit stops outputting a high-frequency drive clock for the flicker detection group and switches the trigger timing of the transmission transistor and reset transistor of the flicker detection group to a low-frequency long exposure mode synchronized with the ordinary pixel units in the same row. In this mode, the flicker detection group no longer performs ultra-short stroboscopic sampling, but instead performs full-frame integration and readout as a large-area monochrome pixel with high sensitivity, which significantly reduces the power consumption of high-frequency processing of the system while outputting a clean brightness signal with a high signal-to-noise ratio.

[0024] The signal processing module also includes an image reconstruction unit for performing multi-dimensional image reconstruction on the area where the flicker detection group is located: in the spatial domain, the high signal-to-noise ratio brightness signal output by the flicker detection group in low-power mode is first used as a guiding feature, or the local high-frequency brightness (Y channel) gradient information of the flicker detection group is extracted as a guiding feature, and adaptive directional interpolation is performed on the surrounding adjacent normal RGB pixels; in the temporal domain, the motion vectors of adjacent historical frames and the current frame are extracted, and the normal pixels corresponding to the physical space coordinates in the historical frames are projected to the heterogeneous hole positions in the current frame to achieve lossless image detail restoration.

[0025] like Figure 1 As shown in the red box, the floating diffusion nodes of all pixel units within the same flicker detection group are physically electrically coupled through metal wires. This allows brightness flicker changes within the continuous space covered by the flicker detection group to be uniformly reflected in the readout signal of the entire flicker detection group, significantly improving the acquisition sensitivity of the flicker signal. Simultaneously, all pixel units within the same flicker detection group are exposed synchronously to compensate for the decrease in conversion gain caused by the coupling of the floating diffusion nodes, ensuring the signal-to-noise ratio of the output signal.

[0026] In this embodiment, the pixel units within the scintillation detection group have their color filters removed and replaced with broadband filters. This covers photon capture across the entire visible light spectrum, significantly improving photon capture efficiency and enabling full-spectrum scintillation detection. This avoids the problem of insufficient signal acquisition in low-light scenarios caused by the band limitation of color filters. In other alternative embodiments, the filters in the pixel units within the scintillation detection group can also be directly removed to further improve photon reception efficiency.

[0027] like Figure 1 As shown, the source follower transistor and row selection transistor in the flicker detection group are completely identical to their corresponding transistors in the ordinary pixel units in the same row in terms of physical interconnection and timing control. Specifically, the gate of the row selection transistor in the flicker detection group is connected to the same row gate line as the gate of the row selection transistor in the ordinary pixel unit in the same row, and the power supply and bias circuit of the source follower transistor are completely shared with the ordinary pixel unit. Therefore, the flicker detection group does not need to be configured with a separate row scan shift register, and can directly reuse the readout pointer of the rolling shutter of the main image, and synchronously complete the signal readout following the row readout timing of the main image, without modifying the original sensor's row scan architecture, significantly reducing design and modification costs.

[0028] This embodiment also includes a timing control circuit, such as... Figure 1As shown, the output of the timing control circuit is electrically connected to the gates of all reset transistors and all transmission transistors in the scintillation detection group, respectively, to provide independent high-frequency clock signals for the reset transistors and transmission transistors. The core control logic is to control the single exposure time of each scintillation detection group to not exceed the readout time of a single row of pixels in the pixel array.

[0029] The operating timing sequence of the flicker detection circuit in this embodiment is as follows: Figure 2 As shown, this timing diagram is a comparison of the control signal timing between ordinary imaging pixels and the scintillation detection group of this invention, fully demonstrating the signal control logic within two consecutive single-row readout cycles; the following is a detailed description with specific parameters: In this embodiment, the single-row pixel readout time of the pixel array is 12μs, and the high-frequency clock signal output by the timing control circuit corresponds to... Figure 2 The reset signal, transmission signal, and line selection signal synchronized with the main image, as well as the timing of the rolling shutter readout pointer.

[0030] The specific working timing process is as follows: ① Reset phase: The timing control circuit coordinates the conduction of the reset transistor and the transmission transistor to open the charge discharge path from the photodiode to the power supply; after the reset is completed, the reset signal and the transmission signal jump to low level, the reset transistor and the transmission transistor turn off, the photodiode begins to collect photons and convert them into photoelectrons, and the exposure phase begins. The single exposure time is strictly controlled within 12μs. Figure 2 The design of the flicker detection group with a short exposure time of ≤1H, as indicated in the text, is completely consistent.

[0031] ② Reference signal readout stage: When the shutter readout pointer of the main image scans to the i-th line, the corresponding... Figure 2 When the row selection signal in the i-th row jumps to a high level, the ordinary pixel unit in the i-th row and the row selection transistor in the i-th row of the flash detection group are synchronously turned on, and the reset transistor is synchronously turned off, thus completing the reading of the reset level reference signal.

[0032] ③ Detection signal readout stage: After the reference signal is readout, the timing control circuit triggers the transmission signal to jump to a high level, and the transmission transistor turns on, transferring the photoelectrons collected by the photodiode during this 12μs exposure cycle to the mutually coupled floating diffusion nodes; at this time, the charge voltage on the floating diffusion nodes is output through the source follower transistor and transmitted to the column bus through the turned-on row selection transistor, completing the readout of one flash detection signal. The transmission signal then jumps to a low level. After the signal readout is completed, the reset transistor turns on again to clear the residual charge of this exposure. Then the reset signal jumps to a low level, synchronously starting the next exposure cycle. This exposure and readout process ends, and the next 12μs exposure cycle begins.

[0033] Through the above timing design, the sampling rate of the scintillation detection group is made equivalent to the line scanning frequency, achieving high-frequency sampling synchronized with line readout. Since the rolling shutter CMOS sensor exposes line by line, the exposure time of the N rows of pixels in a single frame image is sequentially delayed according to the line number. Therefore, the brightness sampling value of each row naturally corresponds to continuous sampling points in the time domain, which can capture the brightness fluctuation of the light source over time and provide sufficient time domain data for subsequent frequency domain analysis.

[0034] This embodiment also supports a low-power intermittent sampling mode based on time multiplexing: In scenarios where there is no need to continuously update the light source frequency, the timing control circuit stops outputting the high-frequency drive clock for the flicker detection group and switches the trigger timing of the transmission transistor and reset transistor of the flicker detection group to a low-frequency long exposure mode synchronized with the ordinary pixel unit in the same row; in this mode, the flicker detection group no longer performs ultra-short stroboscopic sampling, but instead performs full-frame integration and readout as a large area of ​​high-sensitivity monochrome pixel, which can significantly reduce the power consumption of high-frequency digital processing of the system, while outputting a clean brightness signal with a high signal-to-noise ratio.

[0035] The flicker detection signal processing flow in this embodiment is combined with Figure 3 and Figure 4 Please provide an explanation. For example... Figure 3 As shown, the signal output of the flicker detection group is connected to the column bus, which is shared by both the flicker detection group and the ordinary pixel units. The analog signal output by the flicker detection group and the image signal from the ordinary pixel units are synchronously fed into a shared correlation double sampling circuit. Correlation double sampling eliminates reset noise and fixed-pattern noise, completing signal noise reduction preprocessing and improving signal accuracy. The signal sampling timing of the flicker detection group is completely consistent with that of the ordinary pixel units, eliminating the need for a separate sampling circuit for the flicker detection signal. The signal after correlation double sampling can be directly transmitted to the digital signal processing module built into the image sensor chip, or it can be transmitted as row data to the application processor of the terminal for further processing.

[0036] After receiving the preprocessed signal, the digital signal processing module follows the instructions. Figure 4 The process shown is divided into two core stages: intra-frame spatial dimension processing and inter-frame temporal dimension processing, to complete flicker frequency identification. The specific steps are as follows: The first stage is intra-frame spatial dimension processing. The core of this stage is to extract frequency domain information from the line sampling data of a single-frame image, complete the spatial dimension signal fusion within the single frame, suppress interference from a single line or local area, and obtain a robust single-frame spectrum. Specifically, this includes: ① Obtaining K groups of N-point temporal sampling sequences: where N is the number of sampling points in the flicker detection group within a single frame, and K is the number of flicker detection groups, with K≥2. Due to the line-by-line exposure characteristic of the rolling shutter, the brightness sampling value of each line corresponds to continuous sampling points in the temporal domain, which can capture the flicker fluctuations of the light source; the design of K≥2 enables simultaneous multi-region sampling across the entire field of view, providing sufficient data sources for subsequent intra-frame spatial dimension signal fusion. ② Time-domain preprocessing: First, DC component removal is performed to remove the average brightness (DC component) from the signal, retaining only the AC component that fluctuates over time—because the core information of stroboscopic flicker is the periodic fluctuation of brightness, and the DC component does not carry frequency information, removing it can significantly highlight the effective fluctuation signal; then, windowing is performed, superimposing a window function on the time-domain signal to suppress spectral leakage in subsequent Fourier transforms, reduce sidelobe interference, and make the frequency detection results more accurate. ③ Fast Fourier Transform (FFT): FFT is performed on each preprocessed row sequence to transform the brightness fluctuation signal in the time domain to the frequency domain, finally obtaining K initial amplitude spectra of single frames. The peak values ​​of the amplitude spectra correspond to the frequency components contained in the signal, that is, the frequency candidates of the light source flicker. ④ Single-frame spatial fusion (frequency domain max pooling): Max pooling is performed on the amplitude spectra of the K single frames in the frequency domain. Its core function is: the sampling signals of different rows are affected by local light sources, image content and noise, resulting in huge differences in signal strength; max pooling can retain the maximum response at each frequency point, enhance the real and global light source flicker signal, suppress the interference of local scene and random noise, and finally output a single frame comprehensive spectrum that is sensitive to local light sources, thus completing the spatial information fusion within a single frame.

[0037] The second stage is inter-frame temporal processing. The core of this stage is to perform temporal signal fusion across multiple frames to further improve the signal-to-noise ratio, filter transient interference, and ultimately lock in a stable and reliable fundamental frequency of the light source. Specifically, this includes: By configuring a historical smoothed spectrum buffer and an infinite impulse response (IIR) recursive filter, the algorithm addresses the issue that ambient light flicker is stable and continuous, while frame changes, transient noise, and motion interference are random and short-lived. The algorithm stores the smoothed spectrum of past frames in the historical spectrum buffer and then uses the IIR recursive filter to weightedly fuse the current frame's composite spectrum with the historical smoothed spectrum. This time-dimensional smoothing filtering significantly suppresses random noise and transient interference, ultimately outputting a high signal-to-noise ratio final decision spectrum.

[0038] After obtaining a high signal-to-noise ratio spectrum, the amplitude peaks in the spectrum are first searched. Then, by calculating the frequency intervals and multiples between multiple significant peaks in the spectrum, the fundamental frequency and harmonics are distinguished and identified, the interference of higher harmonics is eliminated, the fundamental frequency and harmonic peaks are separated, and the true fundamental frequency component of the pulsed light source is locked.

[0039] Finally, the fundamental frequency f0 of the ambient light source is determined and output. The output result can be directly used to adapt and adjust the camera exposure parameters. The exposure time of the main image is adjusted to an integer multiple of the period corresponding to the fundamental frequency, so that the line exposure period of the main image is synchronized with the bright and dark period of the light source, completely eliminating the banded ripples in the image and achieving anti-flicker imaging.

[0040] When performing image reconstruction, the high signal-to-noise ratio brightness signal output by the scintillation detection group in low power mode is preferentially used as the guiding feature, or the local brightness gradient information around the scintillation detection group is extracted as the guiding feature, and adaptive directional interpolation is performed on the surrounding normal pixels. Combined with motion vector projection in the time domain, the image reconstruction resolution and image quality performance in low power mode can be further improved. Example 2

[0041] This embodiment is a specific implementation of a flicker detection method for CMOS image sensors, combined with... Figure 4 The operation flowchart is described in detail below. This method is based on the flicker detection circuit described in Example 1, and the specific steps are as follows: Step S1, Detection group configuration: In the pixel array of the CMOS image sensor, multiple flicker detection groups consisting of m rows and n columns of pixel units are divided. The multiple flicker detection groups are arranged along the horizontal and vertical directions of the pixel array at a fixed period, so that all flicker detection groups cover the complete field of view of the image sensor; wherein, the pixel unit is a single pixel unit or a shared pixel unit of a multi-pixel shared architecture. Step S2, Node Coupling Configuration: All floating diffusion nodes of pixel units within the same scintillation detection group are physically and electrically coupled via metal wires, ensuring that brightness flicker changes within a continuous space are reflected in the readout signal of that scintillation detection group. Simultaneously, all pixel units within the same scintillation detection group are synchronously exposed to compensate for the conversion gain decrease caused by the coupling of the floating diffusion nodes. The value of m must meet the sampling requirements for a complete cycle of the scintillation signal. Under typical conditions with a single-line readout time of 10μs, m is preferentially selected as integer powers of 2, such as 1024, 2048, or 4096, to ensure complete capture of at least one complete cycle of the lowest 100Hz power frequency scintillation. Step S3, Readout Timing Matching: Ensure that the source follower transistor in the flicker detection group and the corresponding device in the same row of ordinary pixel units have completely consistent physical connections and bias configurations. Match the physical connections and timing control of the row selection transistor in the flicker detection group and the row selection transistor in the same row of ordinary pixel units. This allows the flicker detection group to reuse the rolling shutter readout timing of the main image and complete the signal readout synchronously with the row readout process of the main image. Step S4, Exposure Timing Control: An independent high-frequency clock signal is output through the timing control circuit to control the operating timing of the reset transistor and transmission transistor of the flicker detection group. This ensures that the single exposure duration of the flicker detection group does not exceed the readout time of a single row of pixels in the pixel array, making the sampling rate of the flicker detection group equivalent to the line scanning frequency. K groups of N-point temporal sampling sequences are obtained, where N is the number of sampling points in a single frame, and K is the number of flicker detection groups, with K ≥ 2. Specifically, at the beginning of a single high-frequency sampling period, the control circuit collaboratively turns on the reset transistor and transmission transistor to activate... The photodiode is connected to the power supply for charge discharge. Then, the off-time (falling edge) of the transmission transistor is used as the physical starting point for this exposure. When the shutter readout pointer of the main image scans to the corresponding line, the row selection transistors of the same row's ordinary pixel unit and the flicker detection group are simultaneously turned on to complete the reference signal readout. The transmission transistor is then triggered to turn on, transferring the photoelectrons within this exposure cycle to the coupled floating diffusion node and completing the signal output. After signal reading is complete, the reset transistor is turned on to clear the remaining electrons, and then both the transmission transistor and the reset transistor are simultaneously turned off to begin the next exposure cycle. Step S5, Intra-frame Spatial Dimension Processing: The acquired time-domain sampling sequence is preprocessed by removing DC components and windowing. Then, each preprocessed sequence is subjected to Fast Fourier Transform to obtain K initial amplitude spectra of a single frame. Subsequently, the K amplitude spectra of a single frame are subjected to max pooling in the frequency domain to complete the spatial information fusion within a single frame and output the comprehensive spectrum of a single frame. Step S6, Inter-frame temporal dimension processing: The smooth spectrum of past frames is saved through the historical spectrum buffer, and then the comprehensive spectrum of the current frame is weighted and fused with the historical smooth spectrum through the infinite impulse response recursive filter to output the final judgment spectrum with high signal-to-noise ratio; the amplitude peak in the final judgment spectrum is searched to distinguish and identify the fundamental frequency and harmonics, and the fundamental frequency f0 of the ambient light source is locked. Step S7, Exposure Adjustment: Based on the identified fundamental frequency f0, adjust the exposure duration of the main image to an integer multiple of the corresponding period of the fundamental frequency to eliminate the effect of flicker on imaging.

[0042] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A flicker detection circuit for a CMOS image sensor, characterized in that, The CMOS image sensor includes a pixel array comprising multiple pixel units arranged in rows and columns. Each pixel unit includes a photodiode, a transfer transistor, a floating diffusion node, a reset transistor, a source follower transistor, and a row selection transistor. The pixel array is divided into multiple flicker detection groups, which are periodically arranged in the horizontal and vertical directions and cover the entire field of view of the image sensor. The floating diffusion nodes of all pixel units within the same flicker detection group are electrically coupled to each other. The source follower transistor and the row selection transistor of the flicker detection group are multiplexed using the rolling shutter readout timing of the main image to complete signal readout. The sensor also includes a timing control circuit electrically connected to the reset transistor and transfer transistor of the flicker detection group. This circuit controls the single exposure duration of the flicker detection group to not exceed the single row pixel readout time of the pixel array, making the sampling rate of the flicker detection group equivalent to the row scanning frequency. Finally, the sensor includes a signal processing module electrically connected to the signal output terminal of the flicker detection group. This module identifies the frequency of the flickering light source based on the readout signal of the flicker detection group and adjusts the exposure duration of the main image.

2. The flicker detection circuit of the CMOS image sensor according to claim 1, characterized in that, The pixel unit is a single pixel unit or a shared pixel unit of a multi-pixel shared architecture; the single pixel unit includes a photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor and a row select transistor; the shared pixel unit includes multiple photodiodes, multiple transmission transistors, and a floating diffusion node, a reset transistor, a source follower transistor and a row select transistor shared by all photodiodes.

3. The flicker detection circuit of the CMOS image sensor according to claim 1, characterized in that, All pixel units within the same flicker detection group are exposed synchronously to compensate for the decrease in conversion gain caused by the coupling of the floating diffusion nodes.

4. The flicker detection circuit of the CMOS image sensor according to claim 1, characterized in that, The pixel units in the scintillation detection group have their color filters removed or are configured with broadband filters to improve photon capture efficiency and achieve full-band scintillation detection.

5. The flicker detection circuit for a CMOS image sensor according to claim 1, characterized in that, The timing control circuit coordinates the conduction of the reset transistor and the transmission transistor at the beginning of a single high-frequency sampling cycle to open the charge discharge path from the photodiode to the power supply. Subsequently, the moment when the transmission transistor turns off (falling edge) is taken as the physical starting point of the exposure. When the shutter readout pointer of the main image scans to the corresponding line, the ordinary pixel unit and the row selection transistor of the scintillation detection group in the same line are synchronously turned on to complete the reference signal readout. Then, the timing control circuit triggers the transmission transistor to turn on, transferring the photoelectrons in this exposure cycle to the coupled floating diffusion node and completing the signal output.

6. The flicker detection circuit of the CMOS image sensor according to claim 1, characterized in that, The signal output terminal of the flicker detection group shares a correlation dual sampling circuit with the ordinary pixel unit, and the output terminal of the correlation dual sampling circuit is connected to the core processing unit of the signal processing module; the signal readout timing of the flicker detection group is consistent with that of the ordinary pixel unit, and there is no need to configure an independent sampling circuit for the flicker detection signal.

7. The flicker detection circuit of the CMOS image sensor according to claim 1, characterized in that, The signal processing module's processing flow for the readout signal includes: first, filtering out DC components and low-frequency drift interference from the sampled sequence; then, performing a fast Fourier transform on the processed signal to obtain the spectral distribution; identifying the fundamental frequency and harmonic frequency of the pulsed light source based on the frequency intervals and multiples of significant peaks in the spectral distribution; and using the period corresponding to the fundamental frequency as the adjustment benchmark for the main image exposure time.

8. The flicker detection circuit of the CMOS image sensor according to claim 7, characterized in that, The filtering process for the DC component and low-frequency drift interference is to subtract the moving average value from the sampled sequence, or to perform filtering through a digital high-pass filter.

9. The flicker detection circuit of the CMOS image sensor according to claim 1, characterized in that, The signal processing module synthesizes the signals output by different flicker detection groups within the same frame image using max pooling to preserve local flicker information, and obtains the final flicker detection result by smoothing the synthesis of multiple frame signals to reduce the false positive rate.

10. The flicker detection circuit of a CMOS image sensor according to claim 1, characterized by, The multiple flicker detection groups are discretely distributed in the pixel array, and the proportion of their total number of pixels is controlled within the restoration threshold of image reconstruction; the multiple flicker detection groups are arranged in a non-collinear staggered manner in the horizontal and vertical directions to break the spatial continuity of heterogeneous pixels.

11. The flicker detection circuit of a CMOS image sensor according to claim 1, characterized by, The timing control circuit is also configured to support a low-power intermittent sampling mode based on time multiplexing. In scenarios where continuous updates of the light source frequency are not required, the timing control circuit stops outputting a high-frequency drive clock for the flicker detection group and switches the trigger timing of the transmission transistor and reset transistor of the flicker detection group to a low-frequency long exposure mode synchronized with the ordinary pixel units in the same row. In this mode, the flicker detection group no longer performs ultra-short stroboscopic sampling, but instead performs full-frame integration and readout as a large-area monochromatic pixel with high sensitivity. The signal processing module also includes an image reconstruction unit, which is used to perform image reconstruction and repair on the pixel area where the flicker detection group is located: in the spatial domain, the high signal-to-noise ratio brightness signal output by the flicker detection group in the low-power mode is extracted first as a guiding feature, or the local brightness gradient information around the flicker detection group is extracted as a guiding feature, and adaptive directional interpolation is performed on the surrounding normal pixels; in the temporal domain, the motion vectors of adjacent historical frames and the current frame are extracted, and the normal pixels with corresponding physical spatial coordinates in the historical frames are projected to the pixel blind zone position of the current frame.

12. A method of flicker detection for a CMOS image sensor, Its features are, Includes the following steps: Step S1: Divide the pixel array of the CMOS image sensor into multiple flicker detection groups, and arrange the multiple flicker detection groups periodically along the horizontal and vertical directions of the pixel array so that the flicker detection groups cover the complete field of view of the image sensor; the pixel array contains multiple pixel units arranged in rows and columns, and each pixel unit contains a photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor and a row selection transistor; Step S2: Electrically couple the floating diffusion nodes of all pixel units within the same flicker detection group to each other; Step S3: Control the source follower transistor and line selection transistor of the flicker detection group, and use the rolling shutter readout timing of the main image to complete the signal readout; Step S4: Control the single exposure duration of the scintillation detection group to not exceed the single row pixel readout time of the pixel array, so that the sampling rate of the scintillation detection group is equivalent to the row scanning frequency; Step S5: Process the readout signal of the flicker detection group, identify the frequency of the flickering light source, and adjust the exposure time of the main image according to the identification result.

13. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, The pixel unit is a single pixel unit or a shared pixel unit of a multi-pixel shared architecture; the single pixel unit includes a photodiode, a transmission transistor, a floating diffusion node, a reset transistor, a source follower transistor and a row select transistor; the shared pixel unit includes multiple photodiodes, multiple transmission transistors, and a floating diffusion node, a reset transistor, a source follower transistor and a row select transistor shared by all photodiodes.

14. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, In step S2, all pixel units within the same scintillation detection group are controlled to be exposed synchronously to compensate for the decrease in conversion gain caused by the coupling of floating diffusion nodes.

15. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, In step S1, the pixel units in the scintillation detection group remove the color filter or configure a broadband filter to improve photon capture efficiency and achieve full-band scintillation detection.

16. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, The specific control logic of step S4 is as follows: At the beginning of a single high-frequency sampling cycle, the timing control circuit coordinates the conduction of the reset transistor and the transfer transistor to open the charge discharge path from the photodiode to the power supply; subsequently, the turn-off time of the transfer transistor is taken as the physical starting point of this exposure; when the shutter readout pointer of the main image scans to the corresponding line, the row selection transistor of the same row ordinary pixel unit and the flicker detection group is simultaneously turned on to complete the reference signal readout, and then the transfer transistor is triggered to turn on, transferring the photoelectrons in this exposure cycle to the coupled floating diffusion node and completing the signal output; after the signal readout is completed, the reset transistor is turned on to clear the remaining electrons, and then the transfer transistor and the reset transistor are turned off simultaneously to start the next exposure cycle.

17. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, In step S5, before processing the readout signal of the flicker detection group, the readout signal is first denoised by relevant double sampling.

18. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, In step S5, the processing flow of the readout signal includes: first, filtering out DC components and low-frequency drift interference from the sampled sequence; then, performing a fast Fourier transform on the processed signal to obtain the spectral distribution; identifying the fundamental frequency and harmonic frequency of the pulse light source based on the frequency interval and multiple relationship of significant peaks in the spectral distribution; and using the period corresponding to the fundamental frequency as the adjustment benchmark for the exposure time of the main image.

19. The flicker detection method for a CMOS image sensor according to claim 18, characterized in that, The filtering process for the DC component and low-frequency drift interference is to subtract the moving average value from the sampled sequence, or to perform filtering through a digital high-pass filter.

20. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, In step S5, the signals output by different flicker detection groups within the same frame image are synthesized using max pooling to preserve local flicker information, and the final flicker detection result is obtained by smoothing the synthesis of multiple frame signals to reduce the false positive rate.

21. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, In step S1, the multiple flicker detection groups are discretely distributed in the pixel array, and the proportion of their total number of pixels is controlled within the threshold range that can be repaired by image signal processing; the multiple flicker detection groups are arranged in a non-collinear staggered manner in the horizontal and vertical directions to break the spatial continuity of heterogeneous pixels. It also includes a low-power intermittent sampling control step: that is, in scenarios where there is no need to continuously update the light source frequency, the high-frequency drive clock for the flicker detection group is stopped, and the trigger timing of the transmission transistor and reset transistor of the flicker detection group is switched to a low-frequency long exposure mode synchronized with the ordinary pixel unit in the same row; in this mode, the flicker detection group no longer performs ultra-short strobe sampling, but instead performs full-frame integration and readout as a large area of ​​high sensitivity monochrome pixel.

22. The flicker detection method for a CMOS image sensor according to claim 12, characterized in that, After step S5, an image reconstruction step is also included: performing image reconstruction and repair on the pixel region where the flicker detection group is located, extracting local brightness gradient information around the flicker detection group in the spatial domain as guiding features, and performing adaptive direction interpolation on the surrounding normal pixels; extracting motion vectors of adjacent historical frames and the current frame in the temporal domain, and projecting the normal pixels corresponding to the physical space coordinates in the historical frames to the pixel blind zone position of the current frame.

Images