FPGA-based short-wave infrared camera fast automatic exposure method and system

Abstract

The application provides a short-wave infrared camera fast automatic exposure method and system based on FPGA, relates to the fields of photoelectric imaging and high-speed data flow processing technology, and the automatic exposure method is constructed in strict synchronization with pixel data flow in the FPGA, multi-dimensional brightness statistics and feature extraction are completed in parallel in a single frame period, including global average gray, median gray, brightness distribution entropy and the like, the overall weighted fusion of the multiple statistics is carried out, and a comprehensive brightness evaluation index which is robust to short-wave infrared sensor overexposed pixel inversion and bad point noise is constructed.The application makes full use of the parallel computing and pipeline processing advantages of the FPGA, realizes high-speed automatic exposure control without software participation, and significantly improves the response speed and imaging stability of the short-wave infrared imaging system under strong dynamic light scenes.

Description

Technical Field

[0001] This invention relates to the field of optoelectronic imaging and high-speed data stream processing technology, and more specifically, to a method and system for rapid automatic exposure of a short-wave infrared camera based on FPGA. Background Technology

[0002] In high-speed imaging and machine vision systems, sensors typically output data as a continuous pixel data stream. Existing software solutions based on general-purpose processors require statistical analysis and decision-making after image acquisition, making it difficult to update control parameters within a single frame period. This results in system lag and difficulty adapting to application scenarios with rapidly changing lighting conditions. Short-wave infrared cameras often employ automatic exposure schemes based on CPUs or general-purpose image signal processors. These schemes typically require brightness statistics and exposure decisions after the entire frame of image is acquired, leading to significant processing delays and delayed exposure parameter activation. Furthermore, traditional fixed-step or linear control algorithms converge slowly when faced with large changes in lighting, easily causing exposure oscillations. Additionally, short-wave infrared sensors exhibit nonlinear characteristics such as pixel inversion under strong light conditions, further reducing the reliability of traditional brightness statistics methods. Moreover, existing automatic exposure schemes typically perform brightness statistics and exposure decisions on the entire frame of image after acquisition, with exposure parameters only taking effect in subsequent frames. This results in significant exposure lag in rapidly changing lighting or high-speed motion scenarios, preventing timely convergence of image brightness and severely impacting the imaging quality of short-wave infrared cameras in high-speed applications.

[0003] In addition, some existing hardware acceleration solutions adjust parameters based on only a single statistical feature, which can easily cause control oscillations when there are abnormal pixels or brightness reversal phenomena, affecting system stability.

[0004] Therefore, there is an urgent need for a hardware-level fast automatic exposure method and system that can complete multi-dimensional brightness perception, exposure decision and parameter update within a single frame period, and is strictly coupled with the timing of the short-wave infrared sensor, so as to meet the comprehensive requirements of short-wave infrared imaging for response speed, stability and robustness in high-speed dynamic scenes. Summary of the Invention

[0005] To address the problems in the background art, the present invention provides a method and system for rapid automatic exposure of a shortwave infrared camera based on FPGA, thereby solving the defects existing in the automatic exposure system of the prior art.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A fast automatic exposure method for a shortwave infrared camera based on FPGA includes the following steps: Step S1: Construct a parallel pipelined statistical architecture inside the FPGA that runs synchronously with the pixel data stream of the shortwave infrared sensor. During the single-frame image acquisition process, perform real-time brightness statistics on the input pixel data. Step S2: Based on the pixel data, the average gray value Gavg, the median gray value Gmed, and the luminance distribution entropy H, which characterizes the complexity of the gray value distribution of the current frame image, are calculated in parallel within the FPGA. Step S3: Based on the average gray value, the median gray value, and the brightness distribution entropy, construct a comprehensive brightness index Gsyn, wherein the comprehensive brightness index satisfies the following relationship: Gsyn=α*Gavg+(1-α)*Gmed +β(H)*(Gavg-Gmed); Where α is the basic fusion weight, 0 < α < 1, and β(H) is the modulation function that varies with the luminance distribution entropy; Step S4: Compare the comprehensive brightness index Gsyn with the preset target brightness value Gtar, and calculate the brightness deviation ΔG: ΔG = Gsyn - Gtar; Step S5: Generate an exposure adjustment step size ΔT based on the magnitude of the brightness deviation ΔG. The exposure adjustment step size is simultaneously constrained by both the brightness deviation ΔG and the brightness distribution entropy H, such that the exposure adjustment step size increases as the absolute value of the brightness deviation increases and decreases when the brightness distribution entropy is lower than a preset threshold, in order to suppress oscillations during the exposure adjustment process. Step S6: During the vertical blanking period after the current frame image transmission ends, the exposure parameters are updated according to the exposure adjustment step size, and the updated exposure parameters are written into the short-wave infrared sensor before the next frame exposure begins, so that the exposure adjustment takes effect immediately in the next frame.

[0007] Preferably, the modulation function β(H) is a function that monotonically changes with the luminance distribution entropy. When the luminance distribution entropy is high, the influence weight of the distribution difference term on the comprehensive luminance index is increased, and when the luminance distribution entropy is low, the influence weight is decreased.

[0008] Preferably, the average gray value, median gray value, and brightness distribution entropy are all calculated in real time during pixel data transmission using the parallel pipeline logic inside the FPGA.

[0009] A fast automatic exposure system for a short-wave infrared camera based on FPGA, comprising: A short-wave infrared image sensor is used to output pixel data streams and frame synchronization signals; an FPGA processing module is connected to the short-wave infrared image sensor and is used to execute the above-mentioned fast automatic exposure method. An exposure parameter update interface is used to write the updated exposure parameters into the shortwave infrared image sensor under frame synchronization timing control. The FPGA processing module completes brightness statistics, comprehensive brightness determination, and exposure parameter calculation within a single frame image acquisition cycle, and updates the exposure parameters within the vertical blanking period.

[0010] Preferably, the FPGA processing module includes a parallel pipelined statistics unit that runs synchronously with the pixel data stream, used to complete brightness statistics and exposure decisions without introducing additional frame delay.

[0011] The beneficial effects of this invention are as follows: (1) This invention constructs a parallel pipelined processing architecture that runs synchronously with the pixel data stream of the shortwave infrared sensor within the FPGA. It integrates the statistical calculation of average gray level, median gray level and brightness distribution entropy, the generation of comprehensive brightness index and exposure adjustment decision into the same frame processing cycle, and uses the vertical blanking period to complete the writing of exposure parameters, so that automatic exposure forms a tightly coupled frame-level closed-loop control, thereby significantly shortening the exposure response delay and avoiding the brightness distortion caused by traditional multi-frame lag. By implementing multi-dimensional brightness statistics, entropy calculation, comprehensive brightness determination and dynamic step size decision in a hardware pipeline manner, this invention completes the real-time operation of complex automatic exposure algorithms without introducing software processing delay, so that the highly robust and stable exposure control can directly act on the high-speed shortwave infrared image data stream, breaking through the constraint between the complexity and real-time performance of automatic exposure algorithms under traditional processor architecture.

[0012] (2) In the process of exposure adjustment, the present invention introduces a dual constraint mechanism of brightness deviation and brightness distribution entropy, so that the exposure adjustment step size is rapidly amplified when it deviates from the target brightness to achieve leap-like convergence, while the adjustment amplitude is automatically suppressed when the brightness distribution tends to be uniform or close to steady state. Thus, the balance between response speed and stability is achieved in the overall automatic exposure process, avoiding over-adjustment or oscillation problems caused by a single control strategy.

[0013] (3) Through the overall coordination of algorithm structure design, hardware parallel implementation and sensor timing control, this invention enables automatic exposure to maintain frame-level instantaneous effect and stable output under drastic lighting conditions, thereby improving the imaging consistency and system reliability of shortwave infrared cameras in application scenarios such as autonomous driving, all-weather perception and high-speed machine vision. Attached Figure Description

[0014] To facilitate understanding of the invention, it will be described in more detail with reference to the specific embodiments shown in the accompanying drawings. These drawings depict only typical embodiments of the invention and should not be considered as limiting the scope of protection of the invention.

[0015] Figure 1 This is a flowchart illustrating the fast automatic exposure method for a shortwave infrared camera based on FPGA provided in an embodiment of the present invention. Figure 2 This is the average grayscale convergence curve during the automatic exposure adjustment process provided in this embodiment of the invention. Detailed Implementation

[0016] Embodiments of the present invention are described below with reference to the accompanying drawings to enable those skilled in the art to better understand and implement the invention. However, the listed embodiments are not intended to limit the invention. Unless otherwise specified, the following embodiments and their technical features can be combined with each other, wherein identical components are denoted by the same reference numerals. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0017] This invention proposes a rapid automatic exposure method and system for a short-wave infrared camera based on FPGA. Please refer to [link / reference]. Figure 1 as well as Figure 2 , Figure 1 This is a flowchart illustrating the fast automatic exposure method for a shortwave infrared camera based on FPGA provided in an embodiment of the present invention. Figure 2 This is the average grayscale convergence curve during the automatic exposure adjustment process provided in this embodiment of the invention.

[0018] See Figure 1 In this embodiment of the invention, the fast automatic exposure method for a shortwave infrared camera based on FPGA includes the following steps: Step S1: Construct a parallel pipelined statistical architecture inside the FPGA that runs synchronously with the pixel data stream of the shortwave infrared sensor. During the single-frame image acquisition process, perform real-time brightness statistics on the input pixel data. Step S2: Based on pixel data, the average gray value Gavg, the median gray value Gmed, and the luminance distribution entropy H, which characterizes the complexity of the gray value distribution of the current frame image, are calculated in parallel inside the FPGA. Step S3: Based on the average gray value, median gray value, and brightness distribution entropy, construct a comprehensive brightness index Gsyn, where the comprehensive brightness index satisfies the following relationship: Gsyn=α*Gavg+(1-α)*Gmed +β(H)*(Gavg-Gmed); Where α is the basic fusion weight, 0 < α < 1, and β(H) is the modulation function that varies with the luminance distribution entropy; Step S4: Compare the comprehensive brightness index Gsyn with the preset target brightness value Gtar, and calculate the brightness deviation ΔG: ΔG = Gsyn - Gtar; Step S5: Generate exposure adjustment step size ΔT based on the magnitude of brightness deviation ΔG. The exposure adjustment step size is constrained by both brightness deviation ΔG and brightness distribution entropy H, so that the exposure adjustment step size increases with the increase of the absolute value of brightness deviation and decreases when the brightness distribution entropy is lower than a preset threshold, so as to suppress oscillations in the exposure adjustment process. Step S6: During the vertical blanking period after the current frame image transmission ends, the exposure parameters are updated according to the exposure adjustment step size, and the updated exposure parameters are written to the short-wave infrared sensor before the next frame exposure begins, so that the exposure adjustment takes effect immediately in the next frame.

[0019] The modulation function β(H) is a function that monotonically changes with the luminance distribution entropy. When the luminance distribution entropy is high, the weight of the distribution difference term on the overall luminance index is increased; when the luminance distribution entropy is low, the weight of this influence is decreased. , Let L be the normalized pixel probability of the i-th gray level, and L be the number of gray levels.

[0020] The average gray value, median gray value, and brightness distribution entropy are all calculated in real time during pixel data transmission using the parallel pipeline logic inside the FPGA.

[0021] In another embodiment of the present invention, a rapid automatic exposure system for a short-wave infrared camera based on an FPGA is also provided, comprising: A short-wave infrared image sensor is used to output pixel data streams and frame synchronization signals; an FPGA processing module is connected to the short-wave infrared image sensor to execute the above-mentioned fast automatic exposure method. The exposure parameter update interface is used to write the updated exposure parameters into the shortwave infrared image sensor under frame synchronization timing control. The FPGA processing module completes brightness statistics, comprehensive brightness determination, and exposure parameter calculation within a single frame image acquisition cycle, and updates the exposure parameters within the vertical blanking period.

[0022] Preferably, the FPGA processing module includes a parallel pipelined statistics unit that runs synchronously with the pixel data stream, used to perform brightness statistics and exposure decisions without introducing additional frame delay.

[0023] It is worth noting that existing automatic exposure algorithms typically adjust exposure based solely on the magnitude of brightness deviation, ignoring the concentration and complexity of the image's grayscale distribution. In scenes with uniform brightness distribution or low contrast, the exposure adjustment step size is difficult to converge reasonably, easily causing oscillations near the target brightness and affecting the consistency of consecutive image frames. Furthermore, real-time collaborative processing is difficult to achieve at the hardware level. Although multi-feature or complex criteria are introduced to improve exposure, these algorithms usually rely on software or multi-frame cumulative processing, making it difficult to run synchronously with high-speed pixel data streams. They cannot complete statistics, judgment, and parameter updates within a single frame period, limiting their practical application value in high-speed short-wave infrared imaging systems. A frame-level automatic exposure closed loop strongly coupled with the pixel data stream is constructed to achieve single-frame collaborative completion of brightness statistics, comprehensive judgment, and exposure updates. This invention constructs a parallel pipelined processing architecture within an FPGA that runs synchronously with the pixel data stream of a short-wave infrared sensor. It integrates the statistical calculation of average grayscale, median grayscale, and brightness distribution entropy, the generation of a comprehensive brightness index, and exposure adjustment decisions into a single frame processing cycle. Exposure parameters are written using the vertical blanking period, creating a tightly coupled frame-level closed-loop control for automatic exposure, from statistical analysis to decision updates. This significantly shortens exposure response latency and avoids brightness distortion caused by traditional multi-frame lag. This invention improves the accuracy of brightness perception in complex scenes through a comprehensive brightness determination mechanism that integrates multiple statistical parameters. Within a single frame, this invention extracts average grayscale, median grayscale, and brightness distribution entropy in parallel. It then uses an entropy-modulated comprehensive brightness index to uniformly model brightness location features, distribution bar features, and distribution difference information. This ensures that the brightness determination result considers both global energy levels and local distribution patterns, effectively reducing the risk of misjudgment of brightness caused by bad pixels, high-brightness reversal, or extreme distributions in short-wave infrared imaging.

[0024] This invention achieves synergistic optimization of rapid convergence and stability based on the joint constraints of brightness deviation and distribution entropy. During exposure adjustment, a dual constraint mechanism of brightness deviation and brightness distribution entropy is introduced. This allows the exposure adjustment step size to rapidly amplify when the brightness deviates significantly from the target value, achieving leapfrog convergence, while automatically suppressing the adjustment amplitude when the brightness distribution tends to be uniform or close to a steady state. This balance between response speed and stability is achieved simultaneously in the overall automatic exposure process, avoiding over-adjustment or oscillation problems caused by a single control strategy. Furthermore, the parallel and pipelined characteristics of FPGAs are fully utilized to achieve real-time hardware execution of highly complex automatic exposure algorithms. By implementing multi-dimensional brightness statistics, entropy calculation, comprehensive brightness determination, and dynamic step size decision in a hardware pipeline, this invention completes the real-time operation of complex automatic exposure algorithms without introducing software processing latency. This enables highly robust and stable exposure control to directly apply to high-speed shortwave infrared image data streams, breaking through the constraints between complexity and real-time performance in automatic exposure algorithms under traditional processor architectures.

[0025] In this invention, it is not a simple superposition of multiple statistics; the meanings of each variable in the comprehensive brightness index formula are as follows: Gavg: The average gray value of the current frame image, used to characterize the overall energy level of the image; H: The brightness distribution entropy of the current frame image, used to reflect the degree of concentration or complexity of pixel grayscale distribution; α: The basic fusion weight between the average gray level and the median gray level, satisfying 0 < α < 1; β(H): A distribution modulation factor that varies with the brightness distribution entropy; its design satisfies the following constraints: when the brightness distribution entropy is high, it indicates that the image grayscale distribution is complex and rich in information. In this case, the distribution difference term is allowed to participate in the brightness determination to improve the sensitivity to the actual brightness changes; when the brightness distribution entropy is low, it indicates that the image brightness distribution is concentrated or uniform. In this case, the influence of the distribution difference term on the overall brightness is suppressed to avoid misjudgment caused by a small number of abnormal pixels; through this entropy modulation mechanism, the overall brightness index can adaptively adjust the determination strategy according to the current image distribution state.

[0026] (Gavg-Gmed): The distribution difference term reflects the degree of skewness in the brightness distribution, that is, the asymmetry and skewness of the brightness distribution in the current frame; when there are a few abnormally bright pixels or inverted pixels in the image, the average gray level will deviate from the median; when the brightness distribution of the image is close to symmetrical or concentrated, the difference between the two approaches zero.

[0027] This invention introduces a distribution difference term modulated by the luminance distribution entropy into the comprehensive luminance index, enabling exposure determination to no longer be based solely on a single statistic or a fixed weight combination. Instead, it adaptively adjusts the determination logic according to the image luminance distribution state, thereby significantly improving the accuracy of exposure determination in complex scenes. Both the comprehensive luminance index and the entropy modulation mechanism are based on parallel-computable statistics, suitable for real-time implementation in an FPGA hardware pipeline, and work in conjunction with a frame-synchronized exposure update mechanism to achieve highly robust automatic exposure control without increasing system latency.

[0028] Figure 2This is an example of the average grayscale convergence curve during the automatic exposure adjustment process provided in this invention embodiment. Curves with hollow dots represent overexposure convergence test data, and curves with solid dots represent underexposure convergence test data. The FPGA-based fast automatic exposure method proposed in this invention has extremely high response speed and stability. Regardless of whether the initial image is in an extreme state of low or high grayscale, the system can rapidly adjust and converge the average grayscale to the target brightness range (grayscale value between 110 and 120) within a very short time (approximately 50ms-60ms, i.e., 5 to 6 frame cycles). Furthermore, the curve is smooth and stable after convergence, without any continuous brightness oscillation, verifying that the exposure control algorithm based on dual constraints of brightness deviation and distribution entropy can achieve real-time response and precise control in high-speed dynamic scenes.

[0029] The various embodiments in this invention are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0030] The embodiments described above are merely preferred embodiments of the present invention. The terms "in one embodiment," "in another embodiment," "in yet another embodiment," or "in still another embodiment" used in this specification all refer to one or more of the same or different embodiments according to this disclosure. Ordinary variations and substitutions made by those skilled in the art within the scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A rapid automatic exposure method for a shortwave infrared camera based on FPGA, characterized in that, Includes the following steps: Step S1: Construct a parallel pipelined statistical architecture inside the FPGA that runs synchronously with the pixel data stream of the shortwave infrared sensor. During the single-frame image acquisition process, perform real-time brightness statistics on the input pixel data. Step S2: Based on the pixel data, the average gray value Gavg, the median gray value Gmed, and the luminance distribution entropy H, which characterizes the complexity of the gray value distribution of the current frame image, are calculated in parallel within the FPGA. Step S3: Based on the average gray value, the median gray value, and the brightness distribution entropy, construct a comprehensive brightness index Gsyn, wherein the comprehensive brightness index satisfies the following relationship: Gsyn=α*Gavg+(1-α)*Gmed +β(H)*(Gavg-Gmed); Where α is the basic fusion weight, 0 < α < 1, and β(H) is the modulation function that varies with the luminance distribution entropy; Step S4: Compare the comprehensive brightness index Gsyn with the preset target brightness value Gtar, and calculate the brightness deviation ΔG: ΔG = Gsyn - Gtar; Step S5: Generate an exposure adjustment step size ΔT based on the magnitude of the brightness deviation ΔG. The exposure adjustment step size is simultaneously constrained by both the brightness deviation ΔG and the brightness distribution entropy H, such that the exposure adjustment step size increases as the absolute value of the brightness deviation increases and decreases when the brightness distribution entropy is lower than a preset threshold, in order to suppress oscillations during the exposure adjustment process. Step S6: During the vertical blanking period after the current frame image transmission ends, the exposure parameters are updated according to the exposure adjustment step size, and the updated exposure parameters are written into the short-wave infrared sensor before the next frame exposure begins, so that the exposure adjustment takes effect immediately in the next frame.

2. The fast automatic exposure method for a shortwave infrared camera based on FPGA according to claim 1, characterized in that, The modulation function β(H) is a function that monotonically changes with the luminance distribution entropy. When the luminance distribution entropy is high, the influence weight of the distribution difference term on the comprehensive luminance index is increased; when the luminance distribution entropy is low, the influence weight is decreased.

3. The FPGA-based rapid automatic exposure method for short-wave infrared cameras according to claim 1 or 2, characterized in that, The average gray value, median gray value, and brightness distribution entropy are all calculated in real time during pixel data transmission using the parallel pipeline logic inside the FPGA.

4. A rapid automatic exposure system for a shortwave infrared camera based on FPGA, characterized in that, include: A short-wave infrared image sensor is used to output pixel data streams and frame synchronization signals; An FPGA processing module, connected to the shortwave infrared image sensor, is used to execute the rapid automatic exposure method as described in any one of claims 1 to 3; An exposure parameter update interface is used to write the updated exposure parameters into the shortwave infrared image sensor under frame synchronization timing control. The FPGA processing module completes brightness statistics, comprehensive brightness determination, and exposure parameter calculation within a single frame image acquisition cycle, and updates the exposure parameters within the vertical blanking period.

5. The FPGA-based shortwave infrared camera rapid automatic exposure system according to claim 4, characterized in that, The FPGA processing module includes a parallel pipelined statistics unit that runs synchronously with the pixel data stream, used to perform brightness statistics and exposure decisions without introducing additional frame delay.

Images