Exposure control method and device, computer equipment and computer readable storage medium

Abstract

The invention relates to an exposure control method and device, computer equipment and a computer readable storage medium. The method comprises the following steps: acquiring a line synchronization signal and a frame synchronization signal of an image sensor; the image sensor adopts a roller shutter; determining a frame starting moment of the current frame according to the frame synchronization signal; acquiring a row period, an effective imaging row number and integration time of the image sensor, and calculating a first target moment and a second target moment according to the row period, the effective imaging row number and the integration time; and by taking the frame starting moment as a timing starting point, when timing reaches a first target moment, generating a turn-on control signal of the lighting device corresponding to the image sensor, and when timing reaches a second target moment, generating a turn-off control signal of the lighting device. By adopting the method, the motion distortion can be reduced under the condition that the rolling shutter is not replaced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an exposure control method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] Rolling image sensors start and end integration sequentially by row, resulting in a fixed offset (Line Time, TL) between the exposure start and end times. This can cause a "rolling artifact" in moving scenes.

[0003] In related technologies, a global shutter sensor is used to replace the rolling image sensor. The global shutter sensor can start and stop exposure simultaneously, but it is costly, complex in manufacturing process, and has limited dynamic range and noise performance. Alternatively, global synchronous exposure can be achieved through the exposure window (Strobe / LED OUT) built into the image sensor, but the timing models of different sensors vary greatly, and peripheral compatibility is limited.

[0004] Therefore, there is a need for an image processing solution that can avoid the "jelly effect" more cost-effectively and efficiently. Summary of the Invention

[0005] Based on this, this application provides an exposure control method, apparatus, computer device, and computer-readable storage medium that can avoid the "jelly effect" at a lower cost and with higher efficiency.

[0006] Firstly, this application provides an exposure control method, including:

[0007] The image sensor acquires the line synchronization signal and frame synchronization signal; the image sensor uses a rolling shutter.

[0008] The frame start time of the current frame is determined based on the frame synchronization signal;

[0009] The image sensor's line period, effective number of imaging lines, and integration time are obtained; where the line period represents the time interval between line synchronization signals; the effective number of imaging lines is the effective number of lines in a single frame; and the integration time is the exposure duration of a single line.

[0010] The first target time and the second target time are calculated based on the row period, the number of effective imaging rows, and the integration time; wherein, the first target time includes the time when the integration of the first target row in the current frame begins; and the second target time includes the time when the integration of the second target row in the current frame ends.

[0011] Using the frame start time as the timing start point, when the timing reaches the first target time, an on control signal for the lighting device corresponding to the image sensor is generated, and when the timing reaches the second target time, an off control signal for the lighting device is generated.

[0012] Secondly, this application provides an exposure control device, comprising:

[0013] The first acquisition module is used to acquire the line synchronization signal and frame synchronization signal of the image sensor; the image sensor uses a rolling shutter.

[0014] The determination module is used to determine the frame start time of the current frame based on the frame synchronization signal;

[0015] The second acquisition module is used to acquire the line period, effective number of imaging lines, and integration time of the image sensor; wherein, the line period represents the time interval between line synchronization signals; the effective number of imaging lines is the effective number of lines in a single frame image; and the integration time is the exposure time of a single line.

[0016] The calculation module is used to calculate the first target time and the second target time based on the row period, the number of effective imaging rows, and the integration time; wherein, the first target time includes the time when the integration of the first target row in the current frame begins; and the second target time includes the time when the integration of the second target row in the current frame ends.

[0017] The generation module is used to generate an on control signal for the lighting device corresponding to the image sensor when the frame start time is taken as the timing start point, and to generate an off control signal for the lighting device when the timing reaches the first target time.

[0018] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method described above.

[0019] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0020] Fifthly, this application also provides a computer program product comprising a computer program that, when executed by a processor, implements the steps in the method described above.

[0021] The aforementioned exposure control method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire parameters such as frame synchronization signal, line synchronization signal, line period, effective number of imaging lines, and integration time from the image sensor. Then, based on these parameters, they calculate the start time of integration for the first target line in the current frame (i.e., the first target time) and the end time of integration for the second target line (i.e., the second target time). Finally, using the frame start time as a reference, the illumination device is activated when the first target time is reached and deactivated when the second target time is reached, thereby ensuring that the entire image is exposed within a unified illumination window, effectively reducing motion distortion. This embodiment of the invention, through precise timing modeling and hardware scheduling, controls the external illumination device to emit light within a precisely calculated time window, enabling all lines of the rolling shutter image sensor to integrate within this window, thus simulating the effect of a global shutter. This significantly reduces motion distortion in the image without changing the rolling shutter. Attached Figure Description

[0022] Figure 1 A flowchart illustrating an exposure control method provided in an embodiment of this application;

[0023] Figure 2 A structural block diagram of an exposure control device provided in an embodiment of this application;

[0024] Figure 3 An internal structural diagram of a computer device provided in an embodiment of this application;

[0025] Figure 4 This is an internal structural diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] Explanation of related terms:

[0028] Rolling shutter is an image distortion that occurs when using a rolling shutter image sensor. Global shutter: All pixels on the sensor begin and end exposure at the same instant. This is like taking a "snapshot" of the entire scene with a high-speed camera, perfectly freezing the moment of motion without distortion. However, it is expensive and has disadvantages in certain performance aspects (such as dynamic range). Rolling shutter: Pixels on the sensor are exposed line by line. The first line of exposure ends before the second line begins, the third line follows, and so on, sweeping across the entire sensor like a "rolling shutter." This is how most consumer cameras, mobile phones, and CMOS (Complementary Metal-Oxide-Semiconductor) sensors work. It is inexpensive but has shortcomings when handling moving scenes. Cause of rolling shutter: Because the exposure of a rolling shutter is done line by line, the moment of the scene recorded in each line is slightly different. When shooting a fast-moving object, the object's position on the sensor during the first line of exposure has changed by the time it is exposed during the last line of exposure. In the final composite image, objects that should be straight appear tilted, bent, or twisted, wobbling like jelly, hence the name "jelly effect." For example, if you quickly shake your phone to take a picture or photograph the blades of a high-speed rotating fan, the blades will appear bent.

[0029] FPGA: Field Programmable Gate Array, is an integrated circuit with programmable characteristics that is pre-designed and implemented on a silicon chip. It can be configured into a specified circuit structure according to the designer's needs, without relying on ASIC chips designed and manufactured by chip manufacturers. It is widely used in prototyping, communications, automotive electronics, industrial control, aerospace, data centers and other fields.

[0030] like Figure 1 As shown, this application provides an exposure control method that can be applied to a preset electronic device with communication, data processing, and data storage capabilities. The electronic device may include an FPGA, a MCU (Microcontroller Unit), etc. Taking the application of this method to an FPGA as an example, the method includes the following steps:

[0031] Step 102: Acquire the line synchronization signal and frame synchronization signal of the image sensor; the image sensor uses a rolling shutter.

[0032] The image sensor, such as a CMOS image sensor, converts light signals into electrical signals. The signals output by the image sensor may include: a pixel clock (PIXCLK), with each clock cycle corresponding to the output of one pixel data; a line synchronization signal (XHS or HREF), which remains active during valid pixel data output, marking the start and end of a line; and a frame synchronization signal (XVS or VSYNC), which represents a pulse marking the start of a frame of the image.

[0033] Specifically, considering that the sensor clock domain (PIXCLK) of the image sensor and the master clock domain (SYSCLK) of the FPGA can be asynchronous, directly using these signals will cause metastability problems.

[0034] Therefore, the signal output from the image sensor can be synchronized with the FPGA in time, as follows:

[0035] For the frame synchronization signal, a two-stage D flip-flop is used to synchronize the XVS signal and detect its rising or falling edge (depending on the image sensor protocol) to generate a single-clock-cycle-width frame start pulse (denoted as sof_pulse) within the FPGA master clock domain. This pulse serves as the absolute reference point t0 for the entire frame timing. Correspondingly, the line synchronization signal is synchronized and edge detected to generate a line start pulse, which is used for subsequent line period measurement and line counting.

[0036] Step 104: Determine the frame start time of the current frame based on the frame synchronization signal.

[0037] The frame start time t0 is the reference for calculating subsequent timing. In FPGA, t0 is not an absolute time point, but a counter reset time.

[0038] The process of determining the frame start time may include: upon detecting the sof_pulse pulse generated in step 102, a 32-bit frame timer counter (frame_tick) is cleared to zero, and from this point onward, the counter increments by 1 in each FPGA master clock cycle. Therefore, any moment within a frame can be represented by the current value of frame_tick. For example, frame_tick = 0 represents the frame start time t0.

[0039] Step 106: Obtain the line period, effective number of imaging lines, and integration time of the image sensor; where the line period represents the time interval between line synchronization signals; the effective number of imaging lines is the effective number of lines in a single frame image; and the integration time is the exposure time of a single line.

[0040] The line period (which can be denoted as TL) represents the time interval between the start of integration of two lines, and can be equal to the readout time of one line of pixels. The line period can be determined through dynamic measurement or static configuration. Dynamic measurement uses synchronized XHS pulses to measure the number of FPGA master clock cycles between the start pulses of two consecutive lines. To combat jitter, a moving average filter can be applied to the measurements of multiple consecutive lines to obtain TL_ticks (line period clock count), which is the line period value in FPGA clock ticks. Specifically, if the sensor model is fixed, a known TL value can also be obtained through static configuration of registers (such as CFG_LINE_TICKS).

[0041] The effective number of imaging rows (denoted as N) represents the number of rows actually involved in imaging within a frame. In global shutter mode, this determines the range of rows that need to be illuminated. The effective number of imaging rows can be determined by the sensor's operating mode, such as when ROI (Region of Interest) or Binning / Skipping is set. This value can be set by the upper-level software according to the current sensor configuration via registers (such as CFG_N_LINES). The ROI can be obtained by setting the starting row, starting column, height (number of rows), and width (number of columns) of the region through registers. After the ROI is configured in the software, the image sensor will only scan and read out pixels within this specified region. Binning is a technique that combines the charges of adjacent (2x2, 3x3) pixels into a single "superpixel" before reading it out, either within the sensor or during readout. It is divided into horizontal Binning (merging adjacent columns in the same row) and vertical Binning (merging adjacent rows in the same column). Through pixel binning, the total number of equivalent pixels after binning is reduced, resulting in less readout data, thus shortening readout time and increasing frame rate. Skipping is a technique that systematically skips certain pixels from being read out. For example, 2x2 skipping reads out one pixel every other row and column. Skipping vertically (e.g., skipping every other row) directly reduces the number of effective imaging rows N proportionally. For example, 2:1 vertical skipping will reduce N to half its original value.

[0042] Integration time (denoted as T) int The integral time is used to characterize the actual exposure time of each row of photodiodes. The integral time can be obtained by reading it from the sensor's internal integral time register via an interface such as I2C / SPI and converting it into FPGA clock ticks (EXP_ticks). Specifically, in a system where exposure control is dominated by the FPGA, the integral time (T...) is... int It can be directly configured.

[0043] Step 108: Calculate the first target time and the second target time based on the line period, the number of effective imaging lines and the integration time; wherein, the first target time includes the time when the first target line in the current frame begins integration; the second target time includes the time when the second target line in the current frame ends integration.

[0044] The first target row represents the starting position of the area to be exposed in the current frame, and the second target row represents the ending position of the area to be exposed in the current frame. Specifically, when the area to be exposed is the entire area of ​​the current frame, the first target row can be the last row of the current frame, and the second target row can be the first row of the current frame. In this case, by performing exposure control on the entire area of ​​the current frame, the rolling shutter effect of the current frame can be avoided.

[0045] For example, for row number r (from 0 to N-1), the integration begins at time t. _start(r) = t0 + r*TL, the integration ends at time t. _end(r) =t _start(r) +T int The first target time (ton) is the time when integration begins in the last line of the current frame (the second target line, r=N-1), calculated using the formula: t on =t0+(N-1)*TLF. The implementation logic in the FPGA can include: t on_tick =(N-1)*TL_ticks (because t0 corresponds to frame_tick=0). Correspondingly, the second target time (t off The time when the integration ends for the first line (first target line, r=0) in the current frame can be calculated using the formula t. off =t0+T int .

[0046] Step 110: Taking the frame start time as the timing start point, when the timing reaches the first target time, generate the lighting device turn-on control signal corresponding to the image sensor, and when the timing reaches the second target time, generate the lighting device turn-off control signal.

[0047] In this case, considering that under normal circumstances, due to the integration time, the time when the first row finishes integration will be later than the time when the last row starts integration, the following example illustrates this:

[0048] t=0ms: The frame start time, the first line (line 0) begins integration.

[0049] t=1ms: Line 1 begins integration (at this time, line 0 has already integrated for 1ms).

[0050] t=2ms: Integration begins at line 2.

[0051] t=3ms: Integration begins at line 3.

[0052] t=4ms: Integration begins in the last row (row 4). At this time, the first row (row 0) has integrated for 4ms and needs 6ms to finish. t=10ms: Integration ends in the first row (row 0). At this time, the last row (row 4) has integrated for 6ms (from 4ms to 10ms) and needs another 4ms to finish. It can be seen that during the 6ms time from t=4ms to t=10ms, all 5 rows are integrating simultaneously. Specifically: Row 0: Integrating (4ms -> 10ms); Row 1: Integrating (3ms -> 10ms); Row 2: Integrating (2ms -> 10ms); Row 3: Integrating (1ms -> 10ms); Row 4: Integrating (0ms -> 6ms). For row 4, this is the first 6ms of its integration.

[0053] Therefore, in this embodiment, the time when the integration begins in the last row (row 4) is set as the trigger time of the lighting device's on-control signal, and the time when the integration ends in the first row (row 0) is set as the trigger time of the lighting device's off-control signal. This allows the lighting to be turned on within the aforementioned 6ms window W, ensuring that all five rows experience the same intensity of light. Thus, without needing to change the rolling shutter, the effect of a global shutter is simulated, meaning all pixels are exposed within the same time period. The lighting device can be a light-emitting device such as an LED; this embodiment does not impose any limitations on this.

[0054] This embodiment uses precise timing modeling and hardware scheduling to control the external lighting device to emit light within a precisely calculated time window, so that all rows of the rolling shutter image sensor can be integrated within this window, thereby simulating the effect of a global shutter and eliminating or significantly reducing motion distortion (jelly effect).

[0055] In some embodiments, the first target line in the current frame includes the last line of the current frame; the second target line in the current frame includes the first line of the current frame; the image sensor performs multiple exposure integrations within one frame; the method further includes:

[0056] For each exposure score, obtain the corresponding score time.

[0057] For each exposure integration, the third target moment and the fourth target moment are calculated based on the row period, the number of effective imaging rows, and the integration time of this exposure integration. The third target moment corresponds to the moment when the integration begins in the third target row of the current frame; the third target row in the current frame includes the last row of the current frame. The fourth target moment corresponds to the moment when the integration ends in the fourth target row of the current frame; the fourth target row in the current frame includes the first row of the current frame.

[0058] Taking the frame start time as the timing start point, for each exposure integral, when the timing reaches the third target time corresponding to the exposure integral, an on control signal is generated, and when the timing reaches the fourth target time corresponding to the exposure integral, an off control signal is generated.

[0059] The on and off control signals generated for each exposure integration are combined to drive the illumination device to output multiple light pulses within one frame; wherein each light pulse overlaps with the time window of one exposure integration.

[0060] In particular, considering that the image sensor may be in a high dynamic range (HDR) imaging mode or a working mode that requires multiple exposures, in this mode, the image sensor performs multiple exposure integrations within a frame (for example, a short exposure to capture bright details and a long exposure to capture dark details), and then combines the results of multiple exposures into an HDR image.

[0061] In order to achieve pseudo-global exposure in scenarios where the rolling shutter sensor is used for high dynamic range imaging or requires multiple exposures, in this embodiment, a matching pseudo-global exposure window is calculated for each independent exposure integral, and the illumination device is driven to output corresponding multi-segment light pulses.

[0062] The specific calculation process is as follows: First, determine the multiple exposure parameters. At the beginning of a frame, in addition to obtaining the line period TL and the number of effective lines N, also obtain the number of exposures M of this frame and the integration time T corresponding to each exposure integration. int_i (Where i = 1, 2, ..., M, representing the exponent number), this parameter can be configured via a register (by the upper-level software according to the HDR strategy. For example, for two HDR segments, M = 2, T...). int_1 It may be a short integration time (e.g., 1 ms), T int_2 This is the long integration time (e.g., 10ms).

[0063] Then, an independent pseudo-global window is calculated for each exposure integral. For the i-th exposure integral, the corresponding illumination on and off times are calculated as follows:

[0064] Third target time (t) on_i This corresponds to the moment when the i-th integration begins from the last row (third target row) in the current frame. Calculation formula: t on_i =t0+(N-1)*TL.

[0065] Since the i-th integration of all rows starts simultaneously, the moment when the "last row starts integration" is the same for any integration, which is t0+(N-1)*TL.

[0066] Considering that the first row starts its i-th integration from time t0, after T... int_i The process ends after a certain duration; therefore, the fourth objective time (t) off_i The formula for calculating t is: off_i =t0+T int_i .

[0067] Inside the FPGA, time can be measured by the number of cycles (Ticks) of the system clock (SYSCLK). Therefore, the theoretical time mentioned above needs to be quantized into integer Tick values ​​for hardware comparison.

[0068] TL _ticks : The number of SYSCLK cycles corresponding to the line period TL.

[0069] T int_i_ticks The time T for the i-th integration. int_i The corresponding SYSCLK cycle number.

[0070] CFG_TON_ADV_i: Lamp-on advance compensation amount (in ticks) for the i-th exposure, used to compensate for LED rise delay, etc.

[0071] CFG_TOFF_ADV_i: Lamp-off advance compensation amount (in ticks) for the i-th exposure, used to compensate for LED fall-off delay, etc.

[0072] Based on the above, the FPGA calculates two key comparison values ​​in parallel for each exposure index i:

[0073] Actual light-on comparison value (t) on_tick_i ):

[0074] t on_tick_i =(N-1)*TL _ticks -CFG_TON_ADV_i;

[0075] The characterization starts counting from the frame start point t0 (frame_tick=0). When the global counter equals this value, illumination should be turned on to match the i-th exposure.

[0076] Actual light-off comparison value (t) off_tick_i ):

[0077] t off_tick_i =T int_i_ticks -CFG_TOFF_ADV_i;

[0078] Meaning: When the global counter equals this value, the lighting should be turned off to match the end of the i-th exposure.

[0079] The scheduler monitors the value of the frame start time t0 (when the frame_tick counter is cleared) as a reference. Specifically, when frame_tick equals any t... on_tick_i When (i=1 to M), an "open" reservation event is generated.

[0080] When frame _tick Equal to any t off_tick_i When (i=1 to M), a "close" reservation event is generated. These discrete events are combined into a final LED enable signal through a pre-defined arbitration logic. This signal remains valid from the earliest light-on event to the latest light-off event. The result could be: a continuous wide pulse if the windows of the two exposures overlap; or multiple separate pulses if the windows of the two exposures are temporally separated. In this way, the illumination device is precisely controlled to output multiple light pulses within a single frame, matching the sensor's multiple integration windows, thus effectively freezing moments of motion even in HDR mode.

[0081] By calculating the illumination window separately for each integration, it is ensured that light energy is used only during the sensor-sensitive integration period, avoiding ineffective light emission during non-integration periods and improving illumination efficiency. Simultaneously, precise timing control guarantees that the light intensity received by each row is consistent under different exposures.

[0082] Taking two HDR segments as an example, the timing sequence of sensor exposure and the timing sequence of exposure control corresponding to the embodiments of the present invention are explained as follows:

[0083] Sensor timing includes:

[0084] First integration (short exposure): All rows begin integration almost simultaneously, after T... int_1 Then, the first line ends the integration, and the last line ends a little later.

[0085] Second integration (long exposure): begins immediately after (or after a certain interval), after T int_2 Then it ends.

[0086] The optical pulse timing sequence of this invention includes:

[0087] First pulse segment: from t on_1' Continue until t off_1' This covers the valid window for the first integration.

[0088] Second pulse segment: from t on_2' Continue until t off_2' This covers the effective window for the second integration.

[0089] It should be noted that if t off_1' and t on_2'Very close, the two pulses may merge into a wider pulse, but its physical nature is still for two independent integration processes.

[0090] Traditional HDR technology often suffers from "ghosting" in composite images when capturing moving objects, as the object's position varies at different exposure times. This embodiment addresses this by providing matched pseudo-global illumination for each HDR exposure, freezing the motion state at each instant and significantly reducing motion artifacts in HDR synthesis, thus achieving high-quality dynamic scene HDR imaging. Furthermore, this embodiment utilizes parallel computing and a hardware scheduler in an FPGA to generate multiple pulses, resulting in fast response and precise timing, fully meeting the real-time requirements of high-speed HDR imaging. It demonstrates strong flexibility and scalability, effectively supporting the use of rolling shutter sensors in complex and demanding industrial vision applications.

[0091] In some embodiments, obtaining the line period of an image sensor includes:

[0092] Responding to an external trigger signal;

[0093] Align the external trigger signal with the frame synchronization signal that is closest to the external trigger signal in time to obtain the new frame start time;

[0094] The original frame start time is replaced with the new frame start time.

[0095] In particular, considering that in scenarios such as industrial automation and scientific research, the timing of taking pictures is often determined by external sensors (such as photoelectric switches and encoders) or host computer commands, in order to solve the need for exposure control in response to external events to accurately control the timing of image capture, in this embodiment, the external trigger signal is synchronized with the internal frame timing of the image sensor, and the pseudo-global exposure window is recalculated based on the new timing reference after synchronization, so as to ensure that even in non-fixed frame rate mode, the illumination control can maintain precise alignment with the image integration.

[0096] Specifically, an external trigger signal (ext_trig) is received via a dedicated hardware pin. This signal can be a TTL or LVDS level pulse signal. Considering that the original external trigger signal may contain glitches or have asynchronous issues, it may need to be preprocessed as follows: Glitch filtering: Use a simple digital filter (such as continuous sampling logic) to ensure that only pulses lasting longer than a certain width are considered valid triggers. Cross-clock domain synchronization: Synchronize the glitch-filtered trigger signal to the FPGA's master clock domain (SYSCLK) through two stages of D flip-flops, generating a synchronized trigger signal ext_trig_synced to avoid metastability.

[0097] To ensure a complete and time-stable image is acquired after triggering, the synchronized frame start pulse (sof_pulse, generated by the sensor's XVS signal) and the synchronized external trigger signal (ext_trig_synced) are continuously monitored. The alignment process involves: when a valid rising edge of ext_trig_synced is detected, the system does not immediately use it as the frame start point, but waits for the next upcoming sof_pulse. The moment of the first sof_pulse that is temporally closest to (and follows) the external trigger signal is defined as the new frame start time t. 0_new .

[0098] Specifically, the implementation logic in the FPGA may include:

[0099] The state machine control process is set up as follows:

[0100] Idle state: waiting for ext_trig_synced.

[0101] Triggering wait state: If ext_trig_synced is received, clear the existing frame counter and enter the wait state.

[0102] Synchronization state: In the waiting state, when the next sof_pulse is detected, this instant is taken as the start of a new frame, and the frame timer counter frame_tick is cleared to zero. At this time, t 0_new It was established.

[0103] Replacing the original frame start time includes: from the new t 0_new From time t onwards, all subsequent calculations regarding the pseudo-global exposure window (t) on ,t off All calculations will be based on this. The original frame start times, generated by the free-running frame timing of the sensor, are temporarily replaced.

[0104] It should be noted that this replacement is only effective for the currently triggered frame. After completing the illumination control for the current frame, the system will return to an idle state, waiting for the next external trigger, or switch back to free-running mode (if configured). Finally, pseudo-global exposure control is performed based on the new frame start time:

[0105] From t 0_new (frame_tick=0) Start timing.

[0106] Based on the obtained row period (TL), number of valid rows (N), and integration time (T) int ), calculate t on_tick and t off_tick In frame _tick Count to ton_tick Turn on the lights when the time comes, and count to t. off_tick Turn off the lights when necessary.

[0107] This embodiment closely links external events (such as workpiece arrival) with the image capture time, ensuring that the correct target is captured at the correct time, thus meeting the basic requirements of automated process control. By aligning the trigger signal to the nearest frame start boundary, the sensor is forced to start exposure from a new frame, avoiding image tearing, partial exposure, or incomplete line data that may result from triggering in the middle of a frame, ensuring that every triggered image is time-complete and usable.

[0108] Considering that if illumination is activated directly at the trigger moment without aligning with the sensor frame timing, the illumination time window will be misaligned with the sensor's integration window, leading to pseudo-global exposure failure and even introducing more severe distortion, this method ensures that the illumination control pulse is always calculated based on a stable and complete frame timing sequence. This reliably eliminates the rolling shutter effect even in trigger mode, maintaining the synchronicity of the pseudo-global exposure effect. Furthermore, it provides an interface for interaction with the external world, enabling the pseudo-global exposure system of this invention to be easily integrated into larger automated or synchronous systems, serving as a slave device receiving precise scheduling from the master device.

[0109] In some embodiments, the control process for the lighting device further includes:

[0110] The first time delay between the edge of the level change of the control signal and the moment when the light output of the lighting device reaches the preset intensity is measured;

[0111] The second time delay between the edge of the level change of the off control signal and the moment when the light output of the lighting device decays to the point of being turned off is measured;

[0112] The compensation amount for turning on the lights is determined based on the measurement value of the first time delay, and the compensation amount for turning off the lights is determined based on the measurement value of the second time delay.

[0113] The compensation amount for turning on the lights ahead of the first target time is used to obtain the actual time when the lights are turned on.

[0114] The compensation amount for turning off the lights ahead of the second target time is used to obtain the actual time when the lights are turned off.

[0115] An on control signal is generated when the lights are actually turned on, and an off control signal is generated when the lights are actually turned off.

[0116] Among these factors, considering the rise / fall time of the LED and the electronic delay of the driving circuit, the theoretically calculated illumination window (t) on to t off There may be a deviation between the actual optical output window and the actual output window.

[0117] Therefore, in this embodiment, a closed-loop measurement and adaptive compensation mechanism is introduced to achieve automatic calibration of the delay parameter, thereby ensuring that the pseudo-global exposure effect reaches the optimal level. Specifically, it may include:

[0118] In system calibration mode, the light-on delay (first time delay T_delay_on) can be measured during power-on initialization, periodic maintenance, or triggered by user commands. This is achieved when the FPGA generates a rising edge of an on-control signal (led_en) at any time (or within a specific test frame). Simultaneously, a high-precision timer (which can be a counter) inside the FPGA starts timing. Meanwhile, a high-speed ADC channel continuously acquires the output voltage of the photodiode. When the photodiode signal strength reaches a preset threshold (e.g., 10% or 50% of the stable maximum light intensity), the timer stops. The time from the rising edge of led_en to the light intensity reaching the preset threshold is T_delay_on. This measurement is repeated multiple times and averaged to suppress noise.

[0119] Correspondingly, the light-off delay (second time delay T_delay_off) is measured as follows: After the light output stabilizes, the FPGA generates a falling edge of a turn-off control signal (led_en). The timer starts counting and continues to acquire the photodiode signal. When the photodiode signal strength decays to another preset threshold (e.g., 10% of the maximum light intensity), the timer stops. The time from the falling edge of led_en to the light intensity decaying to the threshold is T_delay_off. Multiple measurements are performed and the average is taken.

[0120] The measured delay needs to be converted into a compensation amount for the theoretical time.

[0121] To ensure the light pulse reaches effective intensity at the theoretical ton time, the light-on command needs to be issued in advance. Therefore, the advance amount should equal the light-on delay, from which the light-on compensation amount (Δt) can be calculated. on The following is an example: Δt on =T_delay_on.

[0122] Correspondingly, in order to make the light pulse at the theoretical t off It begins to decay before time t, and at time t off Even when the lights are almost completely off, a command to turn them off needs to be issued in advance. The advance amount should equal the turn-off delay, and the compensation amount for turning off the lights (Δt) is calculated accordingly. off ): Δt off =T_delay_off.

[0123] The logic implemented in the FPGA includes writing this value into the configuration register CFG_TOFF_ADV.

[0124] Finally, the compensation amount is applied to generate the actual control signal. For example, after exiting calibration mode and entering normal working time, the actual time is calculated using the newly calibrated compensation parameters as follows:

[0125] Actual time when lights are turned on t on_actual =t on -Δt on ;

[0126] Actual time t when lights are turned off off_actual =t off -Δt off ;

[0127] Implementation process in FPGA:

[0128] t on_tick_actual =(N-1)*TL_ticks-CFG_TON_ADV;

[0129] t off_tick_actual =EXP_ticks-CFG_TOFF_ADV;

[0130] Scheduler listener t on_tick_actual and t off_tick_actual The led_en signal is toggled at these times. It should be noted that, to acquire the first and second time delays, a high-speed photodiode or its equivalent sensor (such as another global shutter sensor for measurement) can be integrated into the FPGA and placed in a position where it can receive light from the illumination device.

[0131] This embodiment reduces the alignment error between the illumination window and the sensor integration window to the microsecond or even nanosecond level by directly measuring and compensating for the actual delay of the light output, which is far lower than the accuracy of manual estimation, thereby maximizing the effect of pseudo-global exposure.

[0132] In some embodiments, the method further includes:

[0133] Calculate the time window width between the enable control signal and the disable control signal;

[0134] When the time window width is less than a preset window width threshold, a preset security policy is executed. The preset security policy may include a passive restriction policy and an active adjustment policy; the passive restriction policy includes at least one of disabling the lighting device, limiting the pulse width of the activation control signal, and issuing an alarm; the active adjustment policy includes at least one of increasing the integration time setting, decreasing the effective imaging line number setting, and increasing the operating line frequency of the image sensor.

[0135] Among them, when the system parameters are configured improperly or the working conditions change, resulting in an excessively small or even negative theoretical window width, the pseudo-global exposure will fail, and it may cause abnormalities in the lighting device (such as LED damage caused by too short pulses) or invalid images being captured.

[0136] Therefore, in this embodiment, by monitoring the window width in real time and implementing a hierarchical safety policy, it is ensured that the system always operates in a safe and effective state. Specifically, it can include: at the start of each frame, or whenever key parameters such as the integration time T int , the effective number of rows N, the line period TL, etc. change, the system immediately performs window width calculation. The calculation formula can be as follows: W = T int - (N - 1)*TL;

[0137] Compare the window width with a preset threshold. One or more programmable window width thresholds can be maintained to define different safety levels. Among them, the minimum safety window width (W_min_safe): This is a key threshold. It can be slightly greater than 0, and its value depends on: the physical limit of the lighting device: the minimum pulse width for the LED to emit light stably. The system timing accuracy: Considering clock jitter, compensation errors, etc., a certain margin is required. Image quality requirements: Too short exposure may lead to uneven illumination. Compare the calculated W_ticks with the value of W_min_safe stored in the register CFG_MIN_MIN. According to the comparison result, the system executes different levels of safety policies, and their execution priorities and automation degrees are different.

[0138] Among them, the passive restriction strategy can include: when it is detected that W_ticks < W_min_safe, the system immediately takes protective measures to prevent invalid or dangerous operations. These strategies take effect immediately.

[0139] Prohibit turning on the lighting device, and force the led_en signal to be locked at a low level regardless of the calculation result. It can be when W <= 0 or W is much smaller than the safe value, indicating that pseudo-global exposure is not feasible, and lighting the device will only waste energy and may produce misleading images.

[0140] Limit the pulse width: Ignore the calculated t off_tick , but after turning on the lighting at t on_tick , after a fixed duration equal to W_min_safe, force the lighting to be turned off. It can be when W is positive but less than the safety lower limit, the system outputs a "clamped" safe pulse instead of the theoretically narrow pulse to ensure the stability of the LED operation and the basic brightness uniformity of the image.

[0141] Specifically, an alarm can also be triggered: immediately set the fault flag (such as STS_FAULT_WINDOW_UNDERFLOW) in the status register. The host computer software can detect this flag through interrupts or polling and issue a warning message to the user. This indicates to the user that the current parameter configuration is unreasonable and needs to be adjusted.

[0142] Specifically, proactive adjustment strategies can be adopted to attempt to automatically correct parameters that cause the window width to be too small, fundamentally solving the problem. This strategy can be executed by higher-level control firmware (such as a soft core within the FPGA or an external processor). Specifically, this can include: increasing the integration time setting, such as automatically increasing the integration time value in the sensor register by a step size via the I2C / SPI interface until W > W_min_safe; expanding the window width by extending the integration time of the first row. However, this may reduce the frame rate or cause image overexposure, requiring a trade-off; or reducing the effective imaging row number setting: by modifying the ROI (Region of Interest), reducing the number of effective rows N in the vertical direction, thereby reducing the time difference (N-1)*TL from the start of integration from the first row to the last row, thus quickly expanding the window width, reducing image resolution, but effectively improving the frame rate or ensuring synchronized exposure. Specifically, the operating line frequency of the image sensor can also be increased, such as by configuring the sensor register to reduce the line period TL, thereby reducing (N-1)*TL without sacrificing resolution. Once an anomaly is detected, a passive constraint strategy immediately takes effect to ensure hardware safety. After the passive policy takes effect, the system can trigger an interrupt, and the firmware will determine whether and how to execute the active adjustment policy. For example, the firmware might attempt to fine-tune T. int If the adjustment is successful and the window width returns to normal, the fault flag is cleared and the system continues to work; if the adjustment fails, the lighting is disabled and a serious error is reported.

[0143] The proactive adjustment strategy enables the system to have self-healing and adaptive capabilities. When parameter configurations that pursue high performance (such as high frame rates) reach physical limits, the system can automatically degrade to a safe and feasible mode (such as reducing resolution or increasing exposure) without manual intervention, ensuring the efficiency of continuous production.

[0144] In this embodiment, the passive limiting strategy provides immediate hardware protection, while the active adjustment strategy attempts to optimize performance at a higher level. This hierarchical processing mechanism ensures a safety baseline while maintaining system functionality as much as possible.

[0145] This embodiment effectively avoids pseudo-global exposure attempts under invalid parameters by using window width checks and security strategies, preventing abnormal LED driving and invalid image acquisition, and improving the stability and reliability of the system.

[0146] In some embodiments, acquiring the line synchronization signal and frame synchronization signal of the image sensor includes:

[0147] The line synchronization signal and frame synchronization signal of the image sensor are synchronized from the first clock domain to the second clock domain; wherein the first clock domain is based on the operating clock of the image sensor, and the second clock domain is based on the clock on which the power-on control signal and power-off control signal are generated.

[0148] If the communication between the timing signals generated by the image sensor and the FPGA main control logic is out of sync, it will lead to counter errors, pulse loss, or serious deviations in timing calculation, thereby causing the entire pseudo-global exposure control to fail.

[0149] The first clock domain (sensor clock domain) represents the pixel clock (PIXCLK) output by the image sensor as the clock source. The line synchronization signal (XHS) and frame synchronization signal (XVS) are generated and changed in this clock domain.

[0150] The second clock domain (FPGA control clock domain) represents the FPGA's main system clock (SYSCLK) as the clock source. All calculations, scheduling, and generation of lighting control signals (led_en) for pseudo-global exposure are performed in this clock domain.

[0151] PIXCLK and SYSCLK can be asynchronous, meaning there is no fixed phase relationship between them. When the XHS / XVS signal from the first clock domain directly enters the second clock domain, its transition edge may be very close to the sampling edge of SYSCLK, causing the flip-flops inside the FPGA to enter a metastable state. The output will be in an uncertain oscillating state for a long time, which will lead to errors in subsequent logic.

[0152] Before entering the synchronization chain, the input XHS and XVS signals can be preliminarily processed to filter out any possible high-frequency glitches, such as through a simple sampling window filter.

[0153] First-level trigger: The preprocessed XHS signal is sampled using SYSCLK. The output of this trigger (xhs_sync1) may enter a metastable state.

[0154] The second-stage trigger: xhs_sync1 is resampled using SYSCLK. The metastable probability of its output (xhs_sync2) is greatly reduced, and it can be considered a stable signal under the SYSCLK clock domain. A corresponding two-stage synchronization chain is then applied to the XVS signal to obtain xvs_sync2.

[0155] The synchronized signal (such as xvs_sync2) is delayed by one SYSCLK cycle to obtain xvs_sync2_dly, and then the delayed signal is compared with the undelayed signal.

[0156] Rising edge detection: posedge_pulse=~xvs_sync2_dly&xvs_sync2

[0157] Falling edge detection: negedge_pulse=xvs_sync2_dly&~xvs_sync2

[0158] In the FPGA implementation, the resulting `sof_pulse` (frame start pulse, based on the rising edge of XVS) and `line_start_pulse` (line start pulse, based on the rising edge of XHS) are single-clock-cycle pulses. They are clean and synchronous in the SYSCLK domain, serving as a reliable timing reference. This successfully converts asynchronous sensor synchronization signals into synchronization pulses in the FPGA master clock domain. These pulses are used to drive all subsequent critical modules. Specifically, the `sof_pulse` resets the frame timer counter (`frame_tick`), establishing the absolute time start point `t0` for each frame. The `line_start_pulse` increments the line counter. The line period measurement module dynamically estimates `TL_ticks` by measuring the number of SYSCLK cycles between two consecutive `line_start_pulse`s.

[0159] In this embodiment, a rigorous two-stage trigger synchronization chain is used to reduce the probability of metastability caused by asynchronous sensor signals to an acceptable level, providing a stable and error-free timing foundation for the entire control system.

[0160] In some embodiments, the process of obtaining the line period, effective number of imaging lines, and integration time of the image sensor includes:

[0161] The time interval between the effective edges of multiple consecutive line synchronization signals is statistically analyzed and filtered to obtain the line period of the image sensor.

[0162] The effective number of imaging rows of the image sensor is determined based on the region of interest or pixel skipping mode set by the image sensor.

[0163] The integration time of the image sensor is determined based on the pixel merging mode set by the image sensor.

[0164] In this study, it is considered that the measurement of a single time interval may be subject to noise due to clock jitter, synchronization errors, etc. Filtering can be used to obtain a stable and reliable average row period. Filtering can include moving average filtering. This embodiment does not impose any limitations on this.

[0165] The effective number of imaging lines N is automatically inferred by analyzing the sensor's configuration status. ROI or pixel skipping modes are identified by listening to or reading relevant configuration registers within the sensor via I2C / SPI buses. ROI mode: The effective number of imaging lines N is directly equal to the value of the HEIGHT register. For example, if the full resolution is 1200 lines and HEIGHT is set to 300, then N = 300. Vertical Skipping mode: The effective number of imaging lines N is equal to the number of lines at full resolution divided by the vertical skipping factor. For example, if the full resolution is 1200 lines and a 2:1 vertical skipping ratio is used (reading one line for every two lines), then N = 1200 / 2 = 600. The skipping factor needs to be decoded from the corresponding mode control register. Hardware implementation in FPGA: A small state machine can be implemented within the FPGA to automatically read these registers and calculate the new N value after the sensor configuration changes, updating the CFG_N_LINES register.

[0166] Similarly, by monitoring the sensor configuration register, the system identifies whether vertical binding (e.g., 2x1, 3x1, 4x1, etc.) and its factors are enabled. When vertical binding is enabled, the charge of multiple rows of pixels is merged into one row for reading. The real-time TL_ticks value is obtained through statistical filtering. For example, when 2x1 binding is enabled, the measurement module automatically measures a new, stable TL_ticks value that already includes the timing changes caused by the binding operation.

[0167] This embodiment focuses on an automated and adaptive acquisition mechanism for core timing parameters (line period TL and effective number of imaging lines N). It solves the problem of frequent intervention and reconfiguration by upper-layer software due to parameter changes caused by dynamic switching of sensor operating modes (such as changes in ROI, Binning, etc.). By automatically measuring and identifying these parameters through hardware logic, the system possesses stronger adaptive capabilities, reduces configuration complexity, and improves response speed and reliability.

[0168] In some embodiments, the method further includes:

[0169] Obtain a feedback signal reflecting the current lighting brightness;

[0170] Based on the feedback signal, adjust at least one of the following: the light-on compensation amount, the light-off compensation amount, and the duty cycle of the turn-on control signal.

[0171] This paper addresses the issue of inconsistent illumination brightness caused by aging of lighting components, temperature variations, differences in target object reflectivity, or changes in optical path. Open-loop control struggles to maintain stable image grayscale values ​​over the long term. This embodiment introduces a brightness feedback closed loop to dynamically fine-tune illumination parameters, ensuring stable and ideal image exposure under various operating conditions, thereby improving the accuracy and consistency of detection.

[0172] Specifically, it includes the following:

[0173] Brightness information is obtained through one or more of the following methods to acquire feedback signals reflecting the current lighting brightness:

[0174] Dedicated hardware sensors or image statistics, such as statistics of the current frame image obtained from the image signal processor (ISP) or directly in the FPGA, such as the average pixel value (Y_avg) or grayscale histogram of the entire image or a specific ROI.

[0175] The process of adjusting control parameters based on feedback signals may include:

[0176] The acquired feedback signal is compared with a preset target brightness value (target_brightness) to generate an error signal, which is then used by a control algorithm (such as a PID controller or a simpler proportional regulator) to adjust one or more of the following parameters:

[0177] Adjust the headlight compensation amount (Δt) on ), fine-tuning Δt on The alignment between the starting edge of the illumination pulse and the sensor's integration window can be slightly altered. While the primary purpose is to compensate for delay, this fine-tuning can also slightly affect the effective light energy integration due to the nonlinearity of the sensor's sensitivity during the initial integration phase.

[0178] Adjust the compensation amount for turning off the lights (Δt) off ), and adjusting Δt on Similarly, fine-tuning Δt off It can change the ending edge of the illumination pulse. This is a more effective means of fine-tuning brightness because it directly alters the effective pulse width.

[0179] Implementation: If the brightness is insufficient, Δt can be slightly reduced. off (That is, turning off the lights later) is equivalent to extending the illumination time and increasing the integrated light energy. The reverse is also true. However, it should be noted that this adjustment should not cause the pulse width to exceed the theoretical upper limit of the pseudo-global exposure window.

[0180] Adjusting the duty cycle of the enable control signal (PWM dimming) is a direct and effective method for brightness control. In the lighting driver module, the led_en signal does not directly drive the LED, but serves as an enable signal. During its effective period, a higher frequency PWM signal is output to the LED driver circuit.

[0181] Specifically, a configurable PWM duty cycle register (CFG_PWM_DUTY) is used. The brightness closed-loop controller directly adjusts this duty cycle based on the brightness error. If the brightness is insufficient, the duty cycle is increased, increasing the proportion of time the LED emits light at maximum power within a lighting pulse, thus increasing the total output light energy. Conversely, the duty cycle is decreased if the brightness is high.

[0182] The final adjusted parameters (new Δt) on , Δt off The PWM_DUTY parameter is immediately updated in the corresponding configuration register. The system will use the new parameters to drive the lighting in the next frame or the next control cycle, and will collect brightness feedback again, forming a continuous closed-loop control.

[0183] This embodiment automatically compensates for light decay and environmental changes in the lighting device, ensuring consistent image brightness under the same testing standards for months or even years, greatly reducing maintenance costs and the risk of misjudgment. When faced with different workpieces with vastly different reflectivities, the system can automatically adjust the lighting intensity, enabling each workpiece to be imaged under optimal exposure, thus expanding the equipment's application range.

[0184] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0185] Based on the same inventive concept, this application also provides an exposure control device. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more exposure control device embodiments provided below can be found in the limitations of the exposure control method above, and will not be repeated here.

[0186] like Figure 2As shown, this application embodiment provides an exposure control device 200, including:

[0187] The first acquisition module 202 is used to acquire the line synchronization signal and frame synchronization signal of the image sensor; the image sensor adopts a rolling shutter.

[0188] The determining module 204 is used to determine the frame start time of the current frame based on the frame synchronization signal;

[0189] The second acquisition module 206 is used to acquire the line period, effective number of imaging lines, and integration time of the image sensor; wherein, the line period represents the time interval between line synchronization signals; the effective number of imaging lines is the effective number of lines in a single frame image; and the integration time is the exposure time of a single line.

[0190] The calculation module 208 is used to calculate the first target time and the second target time based on the row period, the effective number of imaging rows, and the integration time; wherein, the first target time includes the time when the integration of the first target row in the current frame begins; and the second target time includes the time when the integration of the second target row in the current frame ends.

[0191] The generation module 210 is used to generate an on control signal for the lighting device corresponding to the image sensor when the timing reaches the first target time, and to generate an off control signal for the lighting device when the timing reaches the second target time, with the frame start time as the timing start point.

[0192] In some embodiments, the first target line in the current frame includes the last line of the current frame; the second target line in the current frame includes the first line of the current frame; the image sensor performs multiple exposure integrations within one frame; the generation module 210 is further configured to:

[0193] For each exposure integration, the integration time corresponding to this exposure integration is obtained; for each exposure integration, based on the row period, the number of effective imaging rows, and the integration time of this exposure integration, the third target time and the fourth target time corresponding to this exposure integration are calculated; wherein, the third target time corresponds to the time when the integration of the third target row in the current frame begins; the third target row in the current frame includes the last row in the current frame; the fourth target time corresponds to the time when the integration of the fourth target row in the current frame ends; the fourth target row in the current frame includes the first row in the current frame;

[0194] Taking the frame start time as the timing start point, for each exposure integral, when the timing reaches the third target time corresponding to the exposure integral, an on control signal is generated, and when the timing reaches the fourth target time corresponding to the exposure integral, an off control signal is generated.

[0195] The on and off control signals generated for each exposure integration are combined to drive the illumination device to output multiple light pulses within one frame; wherein each light pulse overlaps with the time window of one exposure integration.

[0196] In some embodiments, in acquiring the line period of the image sensor, the second acquisition module 206 is specifically used for:

[0197] Responding to an external trigger signal;

[0198] Align the external trigger signal with the frame synchronization signal that is closest to the external trigger signal in time to obtain the new frame start time;

[0199] The original frame start time is replaced with the new frame start time.

[0200] In some embodiments, the generation module 210 is further configured to:

[0201] The first time delay between the edge of the level change of the control signal and the moment when the light output of the lighting device reaches the preset intensity is measured;

[0202] The second time delay between the edge of the level change of the off control signal and the moment when the light output of the lighting device decays to the point of being turned off is measured;

[0203] The compensation amount for turning on the lights is determined based on the measurement value of the first time delay, and the compensation amount for turning off the lights is determined based on the measurement value of the second time delay.

[0204] The compensation amount for turning on the lights ahead of the first target time is used to obtain the actual time when the lights are turned on.

[0205] The compensation amount for turning off the lights ahead of the second target time is used to obtain the actual time when the lights are turned off.

[0206] An on control signal is generated when the lights are actually turned on, and an off control signal is generated when the lights are actually turned off.

[0207] In some embodiments, the exposure control device 200 further includes an adjustment module 212, configured to:

[0208] Calculate the time window width between the on control signal and the off control signal; when the time window width is less than a preset window width threshold, execute a preset safety strategy; the preset safety strategy includes a passive restriction strategy and an active adjustment strategy; the passive restriction strategy includes at least one of prohibiting the lighting device from being turned on, limiting the pulse width of the on control signal, and issuing an alarm; the active adjustment strategy includes at least one of increasing the setting value of the integration time, decreasing the setting value of the effective imaging lines, and increasing the operating line frequency of the image sensor.

[0209] In some embodiments, the first acquisition module 202 is specifically used for: acquiring the line synchronization signal and frame synchronization signal of the image sensor.

[0210] The line synchronization signal and frame synchronization signal of the image sensor are synchronized from the first clock domain to the second clock domain; wherein the first clock domain is based on the operating clock of the image sensor, and the second clock domain is based on the clock on which the power-on control signal and power-off control signal are generated.

[0211] In some embodiments, the second acquisition module 206 is specifically used for: acquiring the line period, effective number of imaging lines, and integration time of the image sensor.

[0212] The time interval between the effective edges of multiple consecutive line synchronization signals is statistically analyzed and filtered to obtain the line period of the image sensor.

[0213] The effective number of imaging rows of the image sensor is determined based on the region of interest or pixel skipping mode set by the image sensor.

[0214] The integration time of the image sensor is determined based on the pixel merging mode set by the image sensor.

[0215] In some embodiments, the adjustment module 212 is further configured to:

[0216] Obtain a feedback signal reflecting the current lighting brightness; based on the feedback signal, adjust at least one of the following: the light-on compensation amount, the light-off compensation amount, and the duty cycle of the turn-on control signal.

[0217] Each module in the aforementioned exposure control device 200 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0218] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements the steps in the exposure control method described above. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0219] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0220] In one exemplary embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0221] In some embodiments, such as Figure 4 The diagram shows the internal structure of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described method embodiments.

[0222] In some embodiments, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0224] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0225] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0226] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An exposure control method characterized by comprising: The method comprises: acquiring a row synchronization signal and a frame synchronization signal of an image sensor; the image sensor adopts a rolling shutter; determining a frame start time of a current frame according to the frame synchronization signal; acquiring a row period, a number of effective imaging rows, and an integration time of the image sensor; the row period represents a time interval between the row synchronization signals; the number of effective imaging rows is a number of effective rows of a single frame image; and the integration time is an exposure time of a single row; calculating a first target time and a second target time according to the row period, the number of effective imaging rows, and the integration time; the first target time includes a time at which a first target row in the current frame starts integration; and the second target time includes a time at which a second target row in the current frame ends integration; generating an opening control signal of an illumination device corresponding to the image sensor when a timer reaches the first target time, and generating a closing control signal of the illumination device when the timer reaches the second target time, with the frame start time as a starting point of the timer.

2. The method of claim 1, wherein, The first target row in the current frame includes a last row of the current frame; the second target row in the current frame includes a first row of the current frame; the image sensor performs multiple exposure integrations within a frame time; and the method further comprises: for each exposure integration, acquiring an integration time corresponding to the exposure integration; for each exposure integration, calculating a third target time and a fourth target time corresponding to the exposure integration according to the row period, the number of effective imaging rows, and the integration time of the exposure integration; the third target time corresponds to a time at which a third target row in the current frame starts the integration; the third target row in the current frame includes the last row of the current frame; the fourth target time corresponds to a time at which a fourth target row in the current frame ends the integration; and the fourth target row in the current frame includes the first row of the current frame; for each exposure integration, generating the opening control signal when the timer reaches the third target time corresponding to the exposure integration, and generating the closing control signal when the timer reaches the fourth target time corresponding to the exposure integration, with the frame start time as a starting point of the timer; combining the opening control signals and the closing control signals generated for each exposure integration to drive the illumination device to output multiple light pulses within the frame time; each light pulse overlaps a time window of one exposure integration.

3. The method of claim 1, wherein, The acquisition of the row period of the image sensor comprises: in response to an external trigger signal; aligning the external trigger signal with a frame synchronization signal closest in time to the external trigger signal to obtain a new frame start time; replacing the original frame start time with the new frame start time.

4. The method of claim 1, wherein, The control process of the illumination device further comprises: measuring a first time delay between a level change edge of the opening control signal and a time at which light output of the illumination device reaches a preset intensity; measure a second time delay between a level change edge of the off control signal and a time when light output of the lighting device decays to off; determine an on compensation according to the measurement of the first time delay, and determine an off compensation according to the measurement of the second time delay; advance the first target time by the on compensation to obtain an actual on time; advance the second target time by the off compensation to obtain an actual off time; generate the on control signal at the actual on time, and generate the off control signal at the actual off time.

5. The method of claim 1, wherein, The method further comprises: calculating a time window width between the on control signal and the off control signal; when the time window width is less than a preset window width threshold, executing a preset safety strategy; the preset safety strategy comprises a passive restriction strategy and an active adjustment strategy; the passive restriction strategy comprises at least one of disabling the lighting device, limiting a pulse width of the on control signal, and performing an alarm; the active adjustment strategy comprises at least one of increasing a set value of the integration time, decreasing a set value of the effective imaging row number, and increasing a working row frequency of the image sensor.

6. The method of claim 1, wherein, The method further comprises: synchronizing the row synchronization signal and the frame synchronization signal of the image sensor from a first clock domain to a second clock domain; wherein the first clock domain is based on a working clock of the image sensor, and the second clock domain is based on a clock on which the on control signal and the off control signal are generated.

7. The method of claim 1, wherein, The method further comprises: statistically counting and filtering a time interval between effective edges of a plurality of continuous row synchronization signals to obtain a row period of the image sensor; determining an effective imaging row number of the image sensor according to a region of interest or a pixel skipping mode set by the image sensor; determining an integration time of the image sensor according to a pixel merging mode set by the image sensor.

8. The method of claim 4, wherein, The method further comprises: obtaining a feedback signal reflecting a current lighting brightness; adjusting at least one of the on compensation, the off compensation, and a duty cycle of the on control signal according to the feedback signal.

9. An exposure control device characterized by comprising: The method further comprises: a first obtaining module configured to obtain a row synchronization signal and a frame synchronization signal of an image sensor; the image sensor adopts a rolling shutter; a determining module configured to determine a frame start time of a current frame according to the frame synchronization signal; a second obtaining module configured to obtain a row period, an effective imaging row number, and an integration time of the image sensor; wherein the row period represents a time interval between the row synchronization signals; the effective imaging row number is an effective row number of a single frame image; and the integration time is an exposure time of a single row. a calculation module, configured to calculate a first target time and a second target time according to the row period, the effective number of imaging rows and the integration time; the first target time comprises a time at which a first target row in the current frame starts to integrate; the second target time comprises a time at which a second target row in the current frame ends to integrate; a generation module, configured to generate an opening control signal of an illumination device corresponding to the image sensor with the frame start time as a timing starting point, and generate a closing control signal of the illumination device when the timing reaches the first target time and the second target time. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that, The processor implements the steps of the method in any one of claims 1 to 8 when executing the computer program.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

Images