Single-frame image stroboscopic compensation method and electronic device

Abstract

The application relates to a single-frame image stroboscopic compensation method and an electronic device. The method comprises the following steps: during the acquisition of a single-frame original image by an image sensor, a time sequence signal of ambient light intensity is acquired by using a stroboscopic sensor which is time-synchronized with the image sensor; the time sequence parameter of a rolling shutter of the image sensor is acquired, and the exposure time interval corresponding to each row of pixels in the single-frame original image is determined; based on the time sequence signal, the light intensity characteristic quantity received by each row of pixels in the respective exposure time interval is determined; the reference light intensity is determined according to the light intensity characteristic quantity of each row of pixels, and the compensation gain corresponding to each row of pixels is generated; and the corresponding compensation gain is applied to each row of the single-frame original image, so that a stroboscopic-compensated image is obtained. In this way, under the premise of only relying on a single frame of image, the stroboscopic fringe under any light source waveform is accurately corrected from the physical measurement level, the multi-frame alignment problem is completely avoided, and the stroboscopic effect can be effectively eliminated while the clarity of the motion details is completely preserved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and electronic device for flicker compensation of single-frame images. Background Technology

[0002] When using a CMOS (Complementary Metal Oxide Semiconductor) image sensor in an environment illuminated by artificial light sources (such as fluorescent lamps or LED lamps), the actual light-sensing time for each row of pixels is not the same due to the rolling shutter's line-by-line start and end exposure characteristics. When the light source brightness changes periodically over time (e.g., AC-driven lamps produce 100Hz / 120Hz light intensity fluctuations, or PWM (Pulse Width Modulation) dimming LEDs produce flickering at hundreds to thousands of Hz), different rows in the image will capture different phases of the light source waveform, thus forming alternating bright and dark striped flickering patterns. In scenes requiring freezing of moving objects, the exposure time is compressed to an extremely short period (e.g., 1 / 500s to 1 / 2000s), much shorter than the brightness change period of the light source, resulting in a huge difference in the energy received by each row, making the flickering patterns particularly severe, and it cannot be eliminated by extending the exposure time to an integer multiple of the light source period.

[0003] To address image flicker issues in moving scenes, existing technologies primarily employ multi-frame temporal averaging or fusion methods. These methods estimate the flicker frequency and compensate for it by acquiring multiple frames. However, when objects are moving at high speeds, the inconsistencies between frames easily introduce ghosting and blurring. While purely data-driven deep learning methods can operate on a single frame, they lack physical priors and have insufficient generalization ability under extreme exposures or unknown light source waveforms. Some solutions incorporate ambient light sensors to detect light source frequencies; however, adjusting exposure time solely based on this is no longer applicable in moving scenes where short exposures are already mandatory. Therefore, how to achieve flicker compensation using a single frame image, avoid the difficulties of multi-frame alignment, and adapt to arbitrary light source waveforms and high-speed moving objects has become a pressing technical problem. Summary of the Invention

[0004] This application provides a method and electronic device for single-frame image flicker compensation.

[0005] According to a first aspect of the embodiments of this application, a single-frame image flicker compensation method is provided, the method comprising: During the acquisition of a single frame of raw image by the image sensor, a time-series signal of ambient light intensity is acquired using a scintillator that is time-synchronized with the image sensor. Obtain the rolling shutter timing parameters of the image sensor to determine the exposure time interval corresponding to each row of pixels in a single frame of the original image; Based on the time series signal, determine the light intensity characterization of each row of pixels within their respective exposure time interval; The reference light intensity is determined based on the light intensity characterization of each row of pixels, and the compensation gain corresponding to each row of pixels is generated. Apply the corresponding compensation gain to each line of the original single-frame image to obtain the flicker-compensated image.

[0006] According to a second aspect of the embodiments of this application, a single-frame image flicker compensation device is provided, the device comprising: The time-series signal acquisition module is used to acquire the time-series signal of ambient light intensity using a flash sensor synchronized with the image sensor during the acquisition of a single frame of raw image by the image sensor. The exposure time interval determination module is used to obtain the rolling shutter timing parameters of the image sensor and determine the exposure time interval corresponding to each row of pixels in a single frame of the original image. The light intensity characterization module is used to determine the light intensity characterization received by each row of pixels within their respective exposure time intervals based on the time series signal. The compensation gain generation module is used to determine the reference light intensity based on the light intensity characterization of each row of pixels and generate the compensation gain corresponding to each row of pixels. The image compensation module is used to apply corresponding compensation gains to each line of the original single-frame image to obtain the flicker-compensated image.

[0007] According to a third aspect of the embodiments of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0008] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described above in this application.

[0009] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above in this application.

[0010] The single-frame image flicker compensation method and electronic device provided in this application determine the exposure window and corresponding light intensity characterization of each row based on the single-frame ambient light intensity timing signal collected by a flicker sensor synchronized with the image sensor, combined with the rolling shutter timing parameters. Then, a line-by-line compensation gain is generated and applied to the original image. Thus, relying only on a single-frame image, it achieves accurate correction of flicker stripes under arbitrary light source waveforms from the physical measurement level, completely avoiding the multi-frame alignment problem. It is particularly suitable for motion scenes and can effectively eliminate flicker while fully preserving the clarity of motion details. Attached Figure Description

[0011] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 A flowchart of a single-frame image flicker compensation method provided as an exemplary embodiment of this application; Figure 2 A schematic block diagram of the functional modules of a single-frame image flicker compensation device provided in an exemplary embodiment of this application; Figure 3 A structural block diagram of an electronic device provided in an exemplary embodiment of this application; Figure 4 A structural block diagram of a computer system provided for an exemplary embodiment of this application. Detailed Implementation

[0012] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0013] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0017] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0018] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0019] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application; other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0020] This application first provides a single-frame image flicker compensation method, such as... Figure 1 As shown, the method may include the following steps: In step S110, during the acquisition of a single frame of original image by the image sensor, a time-series signal of ambient light intensity is acquired using a scintillation sensor that is time-synchronized with the image sensor.

[0021] During image acquisition, a scintillation sensor is simultaneously activated. A reasonable sampling frequency is set for the scintillation sensor to acquire the light source signal intensity values. Assuming each acquired data set has a length of N, it can be represented as (t0, I0), (t1, I1)…, (t…). n-1 ,I n-1 ), where (t) n-1 ,I n-1 ) represents the sampling time of the nth point and the corresponding light source signal intensity.

[0022] In this embodiment, time synchronization between the image sensor and the scintillation sensor can be achieved by sharing the same clock source, ensuring that the sampling time of each sampling point in the time-series signal is precisely aligned with the exposure time of each row of pixels in subsequent steps. During the acquisition process, the sampling frequency of the scintillation sensor can be set according to the flicker frequency range of the light source to be detected. For example, when the frequency range of the light source to be detected is 40Hz to 500Hz, according to the Nyquist sampling theorem, the sampling frequency can be set to no less than 1000Hz, such as 2kHz or higher, to ensure the accuracy of signal reconstruction. The acquired time-series signal can be represented as a set of discrete data points, each data point containing the sampling time and the corresponding light source signal intensity value.

[0023] In step S120, the rolling shutter timing parameters of the image sensor are obtained to determine the exposure time interval corresponding to each row of pixels in a single frame of the original image.

[0024] Specifically, the shutter timing parameters can include: the absolute time when the first row of pixels begins exposure, the exposure duration of each row of pixels, and the time interval between the start of exposure of two adjacent rows of pixels. Based on these parameters, for the i-th row of pixels in the image, its exposure start time can be determined as the absolute time when the first row begins exposure plus (i-1) times the row interval time, and its exposure end time can be determined as its exposure start time plus the exposure duration, thus obtaining the exact exposure time window for that row of pixels.

[0025] In practice, the absolute time of the first row's exposure, the exposure duration of each row, and the time interval between the start of exposure of two adjacent rows can be obtained based on the rolling shutter parameters of the image sensor, thereby determining the actual exposure time range corresponding to each image row.

[0026] The aforementioned rolling shutter parameters are inherent properties of the CMOS image sensor when operating in rolling shutter mode and can be directly read from the image sensor's register configuration or driver. Specifically, let the image sensor capture a single frame of raw image with H rows, for example, H = 1080. Let t1 be the absolute time at which the first row begins exposure, in seconds. This absolute time refers to the precise moment when the first row of pixels begins to reset and enter the exposure state after the image sensor receives the shooting command. Let T_exp be the exposure time for each row, in seconds. This parameter is manually set by the user or automatically determined by the camera's automatic exposure algorithm based on the scene brightness. For example, in a moving scene to capture instantaneous action, T_exp can be set to 10 milliseconds, or 0.01 seconds. Let linetime be the time interval between the start of exposure of two adjacent rows, in seconds. This parameter is also called the line period and is usually determined by the image sensor's pixel clock frequency and line length; for example, it can be set to 9 microseconds, or 0.0009 seconds.

[0027] Based on the above parameters, the exposure start time t_start for the i-th row of pixels in the image is... i It can be obtained through the formula t_start i = t1 + (i-1) × linetime is calculated, meaning that starting from the exposure start time of the first line, the next line begins exposure after each line cycle. Therefore, the exposure start time of the i-th line is the exposure start time of the first line plus a delay of (i-1) line cycles. The exposure end time of the i-th line is t_end. i It can be obtained through the formula t_end i = t_start i The exposure end time is calculated as the exposure start time plus the exposure duration of that row. Therefore, the time window during which the i-th row of pixels continuously receives ambient light during the exposure is precisely determined as [t_start...]. i , t_end i ].

[0028] Using the above method, based on the absolute exposure time of the first row, and combined with the row interval and exposure duration, the exposure time interval of each row of pixels is obtained by recursion, so that the light-sensing process of each row of pixels is accurately located on a unified time axis, providing a time reference for subsequent matching and integration calculation of the ambient light intensity timing signal collected by the scintillation sensor with the exposure windows of each row.

[0029] In step S130, based on the time series signal, the light intensity characterization received by each row of pixels within its respective exposure time interval is determined.

[0030] Since the scintillation sensor collects discrete time series data, to obtain the light intensity value at any given moment, linear interpolation can be performed on the discrete sampling points to reconstruct a continuous light intensity function. For each row of pixels, the continuous light intensity function is integrated within its corresponding exposure time interval, and the average light intensity of that row is obtained based on the exposure duration of that row. This average light intensity is then used as the light intensity representation of that row.

[0031] In step S140, the reference light intensity is determined based on the light intensity characterization of each row of pixels, and the compensation gain corresponding to each row of pixels is generated.

[0032] The reference light intensity can be determined by statistically analyzing the median of all row light intensity values. Using the median effectively mitigates compensation distortion caused by abnormal light intensity in individual rows. For each row, the ratio of the reference light intensity to the light intensity value of that row is calculated as the compensation gain for that row. To enhance the robustness of the method, a protection mechanism can be introduced: when the light intensity value of a row is less than a preset minimum light intensity threshold, this minimum light intensity threshold is used to replace the original light intensity value in the compensation gain calculation, preventing noise amplification due to excessively low light intensity. Furthermore, the calculated compensation gain can be limited to a preset maximum gain value, such as 3, to further prevent overexposure.

[0033] In step S150, the corresponding compensation gain is applied line by line to the original single-frame image to obtain the flicker-compensated image.

[0034] Specifically, this compensation operation can be performed in the RAW (RAW Domain) of image signal processing. For the pixel located in the i-th row and j-th column of a single frame of the original image, its original pixel value is multiplied by the compensation gain of the i-th row calculated in step S140, and the resulting product is the compensated pixel value. By performing the above operation on all rows, a single frame image with consistent brightness and eliminated flicker stripes can be obtained.

[0035] This embodiment determines the exposure window and corresponding light intensity characterization of each row based on the single-frame ambient light intensity timing signal collected by a scintillation sensor synchronized with the image sensor, combined with the rolling shutter timing parameters. Then, it generates a line-by-line compensation gain and applies it to the original image. Thus, relying only on a single-frame image, it achieves accurate correction of stroboscopic stripes under arbitrary light source waveforms from the physical measurement level, completely avoiding the multi-frame alignment problem. It is particularly suitable for motion scenes, and can effectively eliminate stroboscopic stripes while fully preserving the clarity of motion details.

[0036] This application utilizes a scintillation sensor that is strictly time-synchronized with the image sensor to synchronously acquire real ambient light intensity time-series signals during the exposure of a single frame image. It then establishes a precise mapping relationship between the exposure window of each row of pixels and the light intensity signal by combining the rolling shutter timing parameters. The average light intensity is calculated through line-by-line integration, and an inverse compensation gain is generated. This allows for single-frame line-by-line correction of the original image in the RAW domain. This method completely avoids the challenges of multi-frame alignment, requires no preset light source waveform or frequency, and effectively compensates for power frequency flicker, high-frequency PWM modulation, and even mixed light sources. It is particularly suitable for shooting scenarios with fast-moving objects. Simultaneously, through mechanisms such as median reference and gain limiting, it avoids dark line noise amplification or overexposure. The overall solution has low computational overhead and can be integrated into an ISP (Image Signal Processor) pipeline to meet real-time video processing requirements, effectively eliminating flicker stripes while perfectly preserving the clarity of motion details.

[0037] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to collect the time series signal of ambient light intensity, the above step S110 may specifically include the following steps: In step S111, the sampling frequency of the flicker sensor is set according to the flicker frequency range of the light source to be detected. This sampling frequency is greater than or equal to twice the upper limit of the flicker frequency range.

[0038] Specifically, artificial light sources in different application scenarios have different flicker frequency ranges. For example, fluorescent or incandescent lamps directly driven by AC power typically have a light intensity variation frequency twice that of the mains frequency, i.e., 100Hz or 120Hz; while LED light sources using pulse width modulation for brightness control may have switching frequencies between 200Hz and several kilohertz. To accurately capture the details of brightness variations in these light sources and avoid spectral aliasing during sampling, the sampling frequency of the flicker sensor needs to be set according to the Nyquist sampling theorem. In practical applications, the flicker frequency range of the light source to be detected can be initially determined to be, for example, 40Hz to 500Hz, i.e., the upper limit of the frequency range is 500Hz, then the sampling frequency should be set to at least 1000Hz. To further improve the accuracy of signal reconstruction, the sampling frequency can be set to 2000Hz or higher, thereby obtaining denser discrete sampling points, providing a more accurate data basis for subsequent interpolation calculations and integration.

[0039] This embodiment sets the sampling frequency of the scintillation sensor to be no less than twice the upper limit of the scintillation frequency range of the light source to be detected. This ensures that the ambient light intensity timing signal can be collected completely and without distortion, providing reliable data support for the accurate calculation of the average light intensity within the subsequent line-by-line exposure window, thereby guaranteeing the accuracy of the final scintillation compensation result.

[0040] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to determine the exposure time interval corresponding to each row of pixels in a single frame of original image, the above step S120 may specifically include the following steps: In step S121, the absolute time when the first row of the image sensor begins exposure, the exposure duration of each row, and the time interval between the start of exposure of adjacent rows are obtained.

[0041] The three parameters mentioned above (absolute time, exposure duration per line, and time interval between the start of exposure of adjacent lines) are inherent attributes describing the operation of a rolling shutter. The absolute time of the first line's exposure start refers to the moment when the image sensor, after receiving the shooting command, begins to reset and enter the exposure state for its first line of pixels. The exposure duration per line refers to the time from the start to the end of exposure for a single line of pixels; this parameter is usually determined by user settings or the camera's automatic exposure algorithm. The time interval between the start of exposure of adjacent lines, also known as the line period, refers to the fixed time difference between the start of exposure of the i-th line and the start of exposure of the (i+1)-th line in the image sensor. These parameters can be directly obtained from the image sensor's register configuration information or driver.

[0042] In step S122, for the i-th row, the exposure start time is determined as the absolute time plus i-1 times the time interval, and the exposure end time is determined as the exposure start time plus the exposure duration.

[0043] Specifically, let t1 be the absolute time when the first row begins exposure, ttime be the time interval between the start of exposure of adjacent rows, and T_exp be the exposure duration for each row. Then, for the i-th row pixel in the image, its exposure start time t_start i It can be done through the formula t_start i The exposure end time t_end is calculated as t1 + (i-1) × linetime. i It can be done through the formula t_end i = t_start i The result is calculated using T_exp. Therefore, the time window during which the i-th row pixel continuously receives light flux throughout the entire exposure process is [t_start]. i , t_end i This precise time interval provides a crucial basis for subsequently matching and aligning the row with the light intensity timing signal collected by the scintillation sensor.

[0044] This embodiment obtains the inherent parameters of the rolling shutter of the image sensor and calculates the exact start and end times of exposure for each row of pixels in a row-by-row recursive manner, thereby accurately locating the light-sensing process of each row of pixels on the time axis. The accurate establishment of this time window allows for the subsequent mapping of the light intensity timing signal collected by the scintillator sensor to each row of pixels, laying a solid foundation for the accurate calculation of the row-by-row average light intensity and the reasonable generation of compensation gain, thus ensuring the accuracy and reliability of flicker correction.

[0045] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to determine the light intensity characterization received by each row of pixels within its respective exposure time interval, the above step S130 may specifically include the following steps: Step S131: Perform linear interpolation on the time series signal to obtain a continuous light intensity function.

[0046] Specifically, since the scintillation sensor collects ambient light intensity data discretely, the resulting signal is a time series composed of a finite number of sampling points. Each sampling point contains the sampling time and the corresponding light intensity value. To obtain the light intensity value I(t) corresponding to any time t, it is necessary to interpolate and reconstruct the discrete sampling points. Using a linear interpolation method, for any target time t, two adjacent sampling points are first located in the time series signal, such that t lies between the sampling times of these two sampling points, i.e., tt. k ≤ t< t k +1, and then based on the light intensity values ​​I at these two sampling points k and I k +1, calculate the light intensity value at time t through a linear relationship. Linear interpolation has low computational cost and high real-time performance, and can quickly reconstruct a continuous light intensity function while meeting accuracy requirements, providing a continuous light intensity variation curve for subsequent row-by-row integration.

[0047] Step S132: For each row, integrate the continuous light intensity function within the corresponding exposure time interval.

[0048] Specifically, in step S122, the exposure start time t_start for each row of pixels has been determined. i and exposure end time t_end i This time interval is the window during which the pixels in that row continuously receive light flux. For the i-th row, the continuous light intensity function I(t) obtained in step S131 is set within the interval [t_start]. i t_end i The integral is performed on the image, and the resulting integral value represents the total light energy received by the pixels in that row during exposure. Since different rows may be in different phases of light intensity change due to light source flicker, the integral value corresponding to each row can truly reflect the difference in the actual light sensitivity of that row.

[0049] Step S133: Based on the exposure time of the row, obtain the average light intensity of the row, and use the average light intensity as a light intensity characterization quantity.

[0050] Specifically, for the i-th row, the integral value obtained in step S132 is divided by the exposure duration T_exp of that row. The result is the average light intensity received by the pixels in that row during the exposure period. This average light intensity directly reflects the average illumination level experienced by the pixels in that row within the exposure window. Using it as a light intensity characterization of that row can accurately quantify the brightness differences between rows caused by flicker, providing a basis for subsequent calculation of row-by-row compensation gain.

[0051] In the embodiment, average light intensity It can be calculated using the following formula (1).

[0052] (1) This embodiment reconstructs discrete sampling points into a continuous light intensity function through linear interpolation, and performs integration and averaging within the exposure time interval corresponding to each row of pixels. This accurately corresponds the physical measurement data of the scintillation sensor with the line-by-line exposure process of the image sensor in both time and light energy dimensions. Thus, the real physical measurement value is used as the light intensity characterization of each row, avoiding motion interference and prior light source dependence problems encountered in estimating stroboscopic parameters from the image itself, and providing a reliable foundation for generating accurate line-by-line compensation gain.

[0053] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to determine the reference light intensity according to the light intensity characterization of each row of pixels, the above step S140 may specifically include: In step S141, the median of the light intensity characterization of all rows is used as the reference light intensity.

[0054] Specifically, after completing the calculation in step S130, the light intensity representation value corresponding to each row of pixels in the image has been obtained. These light intensity representation values ​​constitute a numerical sequence of length H, which is equal to the total number of rows in the image. For this sequence, a representative reference light intensity value can be determined using statistical methods. In this embodiment, the median of all row light intensity representation values ​​is selected as the reference light intensity I. refThe median is the value in the middle when the light intensity representation is sorted by magnitude. Compared to the average, the median is less sensitive to extreme values. When some rows in an image have abnormally low light intensity representations due to the light source being completely off or at extremely low brightness within the exposure window, using the average will cause the reference light intensity to be lowered by these outliers, resulting in an overall overestimation of the compensation gain for normal rows. Using the median, however, can effectively resist the interference of such outliers, allowing the reference light intensity to more accurately reflect the average brightness level of the light source during its normal emission phase. Alternatively, the maximum value can be used as the reference light intensity to avoid excessive amplification of noise in dark areas of the image to be compensated; the choice can be made flexibly according to the actual scene requirements.

[0055] This embodiment effectively avoids the interference of outliers caused by individual rows being in the light intensity characterization range or in the extremely low brightness range on the global compensation benchmark by using the median of all row light intensity characterizations as the reference light intensity. This ensures that the generated reference light intensity can truly represent the brightness level of the light source under normal light emission conditions, thereby guaranteeing the stability and rationality of subsequent row-by-row gain calculations, and finally obtaining a flicker-compensated image with uniform brightness and no overexposure or noise amplification.

[0056] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to generate the compensation gain corresponding to each row of pixels, the above step S140 may further include the following steps: In step S142, for each row, the ratio of the reference light intensity to the light intensity representation value of that row is calculated, and the ratio is used as the compensation gain for that row. Specifically, when the light intensity representation value of that row is less than a preset minimum light intensity threshold, the preset minimum light intensity threshold is used instead of the light intensity representation value of that row to calculate the ratio.

[0057] Specifically, in step S141, the reference light intensity I is determined. ref Next, to eliminate the brightness differences caused by varying received light intensities across rows, an inverse compensation gain needs to be generated for each row of pixels. For the i-th row, its compensation gain G is... i The calculation method is based on the reference light intensity I. ref Divide by the light intensity characterization of that row I i G i = I ref / I i When a row of pixels receives a large light intensity within the exposure window due to the light source being in a relatively bright position, the ratio is less than 1, and applying gain will reduce the brightness of that row; conversely, when a row receives a small light intensity due to the light source being in a darker phase, the ratio is greater than 1, and applying gain will increase the brightness of that row, thereby achieving consistent brightness correction across all rows.

[0058] In some cases, the light intensity value collected by the scintillation sensor may be extremely weak or even close to zero. For example, when an LED light source using pulse width modulation is completely off within the exposure window, the light intensity characterization value I corresponding to that row will be very low. i The value will approach zero. If this value is used directly to calculate the compensation gain, it will produce an extremely large gain value approaching infinity, causing severe noise amplification or even overexposure of the pixels in that row after compensation, resulting in a serious degradation of image quality. To solve this problem, this embodiment introduces a preset minimum light intensity threshold I. min As a lower limit of protection, before calculating the compensation gain of the i-th row, the light intensity characterization I of that row is first determined. i Is it less than the preset minimum light intensity threshold I? min , if I_i is less than I min Then, when calculating the gain, use I. min Replace the original I i Participating in division operation, i.e., Gi = I ref / I min Preset minimum light intensity threshold I min The settings can be adjusted according to the actual application scenario and sensor noise level to ensure that the gain value after replacement is within a reasonable range.

[0059] This embodiment calculates the ratio of the reference light intensity to the light intensity characterization quantity for each row as the compensation gain, and adaptively generates a gain value greater than 1 or less than 1 based on the relative magnitude of the actual received light intensity and the reference light intensity of each row. This achieves precise reverse compensation for the brightness difference between rows caused by flicker. At the same time, by introducing a protection mechanism with a preset minimum light intensity threshold, it effectively prevents the problem of abnormal gain amplification caused by extremely low light intensity in individual rows, avoids noise explosion or overexposure in the compensated image, and significantly improves the robustness of the method and the final image quality.

[0060] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to generate the compensation gain corresponding to each row of pixels, the above step S140 may further include the following steps: In step S143, the compensation gain is limited to a preset maximum gain value.

[0061] Specifically, in step S142, the compensation gain G of each row of pixels is calculated. i Afterwards, even though the preset minimum light intensity threshold I was passed... minWhile protection is provided for extremely low light intensity conditions, in certain scenarios, such as when the light source waveform itself has a very deep modulation depth or when the flicker sensor exhibits noise fluctuations at individual sampling points, the obtained compensation gain may still be at a high level. If these large gain values ​​are directly applied to the original image, the pixel values ​​of the corresponding rows will be significantly amplified. At the same time, the inherent background noise and photon shot noise of the image sensor in that row will also be amplified simultaneously, resulting in obvious noise particles or color speckles in the originally darker rows of the compensated image, severely affecting visual quality.

[0062] To address the aforementioned issues, this embodiment imposes an upper limit on the compensation gain. A reasonable maximum gain value G is preset. max For example, G max Set to 3. Before applying the compensation gain to each row, the compensation gain G calculated in step S142 is... i With G max Comparison: If G i Greater than G max Then Gi will be forcibly modified to G. max If G i Less than or equal to G max Then keep G i Unchanged. Preset maximum gain value G max The selection of this value requires a trade-off between flicker removal and noise suppression. If the value is too small, it may result in insufficient compensation for some darker lines and residual flicker stripes. If the value is too large, the noise suppression effect will be weakened. It can be calibrated and set according to the noise performance of the image sensor, the light source characteristics of typical shooting scenes, and the user's tolerance for image quality.

[0063] This embodiment enhances the robustness of the method by setting an upper limit on the compensation gain, based on the minimum light intensity protection provided in step S142. It effectively suppresses the problem of dark noise amplification caused by excessive gain, and achieves a balance between eliminating flicker stripes and maintaining the image signal-to-noise ratio, so that the final output image has good visual cleanliness while maintaining uniform brightness.

[0064] Based on the above embodiments, in another embodiment provided in this application, in order to describe in detail how to apply the corresponding compensation gain to each line of the original single-frame image, the above step S150 may specifically include: In step S151, the original value of the pixel in the i-th row and j-th column of the single-frame original image is multiplied by the compensation gain of the i-th row to obtain the compensated pixel value.

[0065] Specifically, the single-frame raw image is output by the image sensor, typically in RAW format such as a Bayer array, with each pixel location corresponding to a raw photosensitive value. In step S140, a corresponding compensation gain G has been generated for each row of pixels.i This gain value applies to all pixels within that row. For a pixel located in the i-th row and j-th column of the image, let its original pixel value be p. in(i，j) Then the compensated pixel value p out(i， j) Through formula p out(i，j) = p in(i，j) ×G _i Calculated. Since all pixels within the same row share the same row gain G. i This operation ensures that the relative brightness relationships between pixels within the same row remain unaffected, correcting only the overall brightness differences between rows caused by flicker. This compensation operation can be performed in the RAW domain of the image signal processing pipeline, that is, before subsequent processing steps such as de-mosaicing, white balance, and color correction. This avoids adverse effects of flicker stripes on subsequent image processing algorithms and requires no modifications to other parts of the ISP pipeline. By performing the above multiplication operation row by row, a compensated image with consistent brightness and eliminated flicker stripes can be obtained.

[0066] This embodiment achieves brightness correction by multiplying the original value of each row of pixels in the original image by the corresponding compensation gain, using a row-by-row scalar multiplication method. Pixels within the same row share the same gain value. While eliminating inter-row flickering stripes, it perfectly preserves the image details and texture information within the row. It is simple to operate, has minimal computational overhead, and can be seamlessly integrated into the RAW domain processing stage of existing ISP pipelines, meeting the performance requirements of high-resolution image and real-time video stream processing.

[0067] Based on the above embodiments, in another embodiment provided in this application, before acquiring the time-series signal of ambient light intensity, the method may further include the following steps: In step S160, the scintillation sensor and the image sensor are synchronized in time so that the sampling time of the time series signal is aligned with the exposure time of each row of pixels.

[0068] Specifically, the core of this invention lies in precisely matching the ambient light intensity timing signal collected by the scintillation sensor with the exposure time window of each row of pixels in the image sensor, thereby accurately calculating the actual light intensity received by each row of pixels. This matching process relies on strict time synchronization between the two sensors. If the scintillation sensor and the image sensor each use independent clock sources, their time bases may drift or deviate, resulting in unpredictable misalignment between the sampling time and the exposure time. This causes the subsequently calculated average light intensity of each row to fail to accurately reflect the actual light energy received by that row, ultimately leading to deviations in the compensation gain or even correction failure.

[0069] To achieve the aforementioned time synchronization, at the hardware level, the scintillation sensor and the image sensor can share the same clock source. For example, the device's system clock or a dedicated crystal oscillator can simultaneously provide a time reference for both, thus physically ensuring that their time counts advance synchronously. At the software level, a synchronization trigger signal can be sent to the scintillation sensor at the same time the image sensor begins exposure. This trigger moment is used as the starting point for sampling by the scintillation sensor, ensuring that the first sampling point of the time sequence signal is precisely aligned with the start time of the first line of image exposure on the time axis. Through this hardware and software synchronization mechanism, the timestamp of each light intensity sampling point acquired by the scintillation sensor maintains a definite correspondence with the line exposure timeline of the image sensor, providing a reliable guarantee for the accuracy of the light intensity integral calculation within the line-by-line exposure window in step S130.

[0070] This embodiment establishes a time synchronization mechanism between the scintillation sensor and the image sensor during the image acquisition process, ensuring that the sampling time of the ambient light intensity timing signal is precisely aligned with the exposure time of each row of pixels in the image under a unified time coordinate system. This lays a time reference for subsequent line-by-line light intensity calculation and gain compensation generation based on physical measurements, fundamentally guaranteeing the measurement accuracy and correction effect of the stroboscopic compensation method.

[0071] By dividing each functional module according to its corresponding function, this application provides a single-frame image flicker compensation device, which can be a server, a terminal, or a chip applied to a server. Figure 2 A schematic block diagram of the functional modules of a single-frame image flicker compensation device provided for an exemplary embodiment of this application. Figure 2 As shown, the single-frame image flicker compensation device includes: The time series signal acquisition module 21 is used to acquire the time series signal of ambient light intensity using a flash sensor synchronized with the image sensor during the acquisition of a single frame of original image by the image sensor. The exposure time interval determination module 22 is used to obtain the rolling shutter timing parameters of the image sensor and determine the exposure time interval corresponding to each row of pixels in a single frame of the original image. The light intensity characterization determination module 23 is used to determine the light intensity characterization received by each row of pixels within their respective exposure time intervals based on the time series signal. The compensation gain generation module 24 is used to determine the reference light intensity based on the light intensity characterization of each row of pixels and generate the compensation gain corresponding to each row of pixels. The image compensation module 25 is used to apply corresponding compensation gain to the original single-frame image line by line to obtain the flicker-compensated image.

[0072] In another embodiment provided in this application, the time series signal acquisition module 21 is specifically used for: Set the sampling frequency of the flicker sensor according to the flicker frequency range of the light source to be detected; the sampling frequency should be greater than or equal to twice the upper limit of the flicker frequency range.

[0073] In another embodiment provided in this application, the exposure time interval determination module 22 is specifically used for: Obtain the absolute time when the first row of the image sensor begins exposure, the exposure duration of each row, and the time interval between the start of exposure of adjacent rows; For the i-th row, the exposure start time is determined as the absolute time plus i-1 times the time interval, and the exposure end time is determined as the exposure start time plus the exposure duration.

[0074] In another embodiment provided in this application, the light intensity characterization module 23 is specifically used for: Linear interpolation of time-series signals yields continuous light intensity functions; For each row, integrate the continuous light intensity function over the corresponding exposure time interval; Based on the exposure time of the line, the average light intensity of the line is obtained, and the average light intensity is used as a light intensity characterization quantity.

[0075] In another embodiment provided in this application, the compensation gain generation module 24 is specifically used for: The median of the light intensity values ​​across all rows is used as the reference light intensity.

[0076] In another embodiment provided in this application, the compensation gain generation module 24 is specifically used for: For each row, the ratio of the reference light intensity to the light intensity representation of that row is calculated, and the ratio is used as the compensation gain for that row. When the light intensity representation of that row is less than a preset minimum light intensity threshold, the preset minimum light intensity threshold is used to replace the light intensity representation of that row in the calculation of the ratio.

[0077] In another embodiment provided in this application, the compensation gain generation module 24 is specifically used for: The compensation gain is limited to a preset maximum gain value.

[0078] In another embodiment provided in this application, the image compensation module 25 is specifically used for: The original value of the pixel in the i-th row and j-th column of a single frame of the original image is multiplied by the compensation gain of the i-th row to obtain the compensated pixel value.

[0079] In another embodiment provided in this application, before acquiring the time-series signal of ambient light intensity, the device further includes a time synchronization module, specifically used for: The scintillation sensor is synchronized with the image sensor in time so that the sampling time of the time series signal is aligned with the exposure time of each row of pixels.

[0080] The single-frame image flicker compensation device provided in this application determines the exposure window and corresponding light intensity characterization of each row based on the single-frame ambient light intensity timing signal collected by a flicker sensor synchronized with the image sensor, combined with the rolling shutter timing parameters. Then, it generates a line-by-line compensation gain and applies it to the original image. Thus, relying only on a single-frame image, it achieves accurate correction of flicker stripes under arbitrary light source waveforms from the physical measurement level, completely avoiding the multi-frame alignment problem. It is particularly suitable for motion scenes and can effectively eliminate flicker while fully preserving the clarity of motion details.

[0081] This application also provides an electronic device, including: at least one processor; a memory for storing executable instructions of the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method disclosed in the embodiments of this application.

[0082] Figure 3 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. For example... Figure 3 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this application.

[0083] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this application can be implemented by the integrated logic circuitry in the processor 1801 or by software instructions. The processor 1801 can be a general-purpose processor, DSP, ASIC (Application Specific Integrated Circuit), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the above method.

[0084] Furthermore, the various operations / processes according to this application, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 4 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 4 A structural block diagram of a computer system provided for an exemplary embodiment of this application.

[0085] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0086] like Figure 4As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0087] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0088] The computing unit 1901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in the embodiments of this application can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0089] This application also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this application.

[0090] The computer-readable storage medium in this application embodiment may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0091] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0092] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this application.

[0093] In embodiments of this application, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The modules, components, or units described in the embodiments of this application can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0096] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0097] The above description is merely an embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0098] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A method for flicker compensation in a single-frame image, characterized in that, The method includes: During the acquisition of a single frame of raw image by the image sensor, a time-series signal of ambient light intensity is acquired using a flash sensor that is time-synchronized with the image sensor. Obtain the rolling shutter timing parameters of the image sensor to determine the exposure time interval corresponding to each row of pixels in the single frame original image; Based on the time series signal, determine the light intensity characterization of each row of pixels within their respective exposure time intervals; The reference light intensity is determined based on the light intensity characterization of each row of pixels, and the compensation gain corresponding to each row of pixels is generated. The corresponding compensation gain is applied line by line to the original single-frame image to obtain the flicker-compensated image.

2. The single-frame image flicker compensation method according to claim 1, characterized in that, The time-series signal of the acquired ambient light intensity includes: The sampling frequency of the flicker sensor is set according to the flicker frequency range of the light source to be detected; the sampling frequency is greater than or equal to twice the upper limit of the flicker frequency range.

3. The single-frame image flicker compensation method according to claim 1, characterized in that, Determining the exposure time interval corresponding to each row of pixels in the single-frame original image includes: Obtain the absolute time when the first row of the image sensor begins exposure, the exposure duration of each row, and the time interval between the start of exposure of adjacent rows; For the i-th row, the exposure start time is determined as the absolute time plus i-1 times the time interval, and the exposure end time is determined as the exposure start time plus the exposure duration.

4. The single-frame image flicker compensation method according to claim 1, characterized in that, The determination of the light intensity characterization received by each row of pixels within its respective exposure time interval includes: Linear interpolation is performed on the time series signal to obtain a continuous light intensity function; For each row, the continuous light intensity function is integrated over the corresponding exposure time interval; Based on the exposure time of the row, the average light intensity of the row is obtained, and the average light intensity is used as the light intensity characterization quantity.

5. The single-frame image flicker compensation method according to claim 1, characterized in that, Determining the reference light intensity based on the light intensity characterization of each row of pixels includes: The median of the light intensity values ​​of all rows is used as the reference light intensity.

6. The single-frame image flicker compensation method according to claim 1, characterized in that, The generation of compensation gain corresponding to each row of pixels includes: For each row, the ratio of the reference light intensity to the light intensity representation of that row is calculated, and the ratio is used as the compensation gain for that row; wherein, when the light intensity representation of that row is less than a preset minimum light intensity threshold, the preset minimum light intensity threshold is used to replace the light intensity representation of that row to calculate the ratio.

7. The single-frame image flicker compensation method according to claim 1 or 6, characterized in that, The process of generating the compensation gain corresponding to each row of pixels also includes: The compensation gain is limited to a preset maximum gain value.

8. The single-frame image flicker compensation method according to claim 1, characterized in that, The step of applying corresponding compensation gain line by line to the original single-frame image includes: The original value of the pixel in the i-th row and j-th column of the single-frame original image is multiplied by the compensation gain of the i-th row to obtain the compensated pixel value.

9. The single-frame image flicker compensation method according to claim 1, characterized in that, Before acquiring the time-series signal of ambient light intensity, the method further includes: The flicker sensor is synchronized with the image sensor in time so that the sampling time of the time series signal is aligned with the exposure time of each row of pixels.

10. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-9.

Images