Real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics

Abstract

This invention provides a real-time hardware high dynamic range (HDR) synthesis method and camera with dual-gain grayscale characteristics, relating to the field of optical imaging technology. The method acquires dual-gain image data using a deployed sensor with high and low gain channels; performs noise suppression processing on the acquired dual-gain image data and non-uniformity correction to obtain preprocessed image data; fuses the high-gain and low-gain images into a HDR image through image fusion, and enhances the image signal-to-noise ratio and readout speed by merging adjacent pixels; verifies the dynamic range, image quality, and processing speed of the synthesized HDR image through actual shooting tests. This invention provides a dual-gain channel foundation for HDR synthesis by simultaneously outputting high-brightness and low-brightness images through the sensor's multi-gain channels. By synthesizing images with different gains, the dynamic range of the image can be significantly improved while maintaining image detail and quality.

Description

Technical Field

[0001] This invention relates to the field of optical imaging technology, specifically to a real-time hardware high dynamic range synthesis method and camera for dual-gain grayscale characteristics. Background Technology

[0002] In modern imaging technology, image sensors are key components for acquiring image information. With technological advancements, the application areas of image sensors are constantly expanding, including but not limited to astronomical observation, medical imaging, industrial inspection, and security monitoring. In these applications, image sensors need to capture scenes with high dynamic range, meaning they can simultaneously record details in both very bright and very dark areas. However, traditional image sensors, such as CCD (Charge-Coupled Device) and early CMOS (Complementary Metal-Oxide-Semiconductor) sensors, often struggle to capture high dynamic range scenes in a single exposure due to limitations in their physical and electronic properties.

[0003] Traditional image sensors, under single exposure, have limited dynamic range, making it impossible to simultaneously capture details in both high-brightness and low-brightness areas. This results in either overexposure in bright areas or loss of detail in dark areas in high-contrast scenes. In low-light conditions, traditional sensors exhibit high noise levels, leading to degraded image quality and insufficient sensitivity. Increasing exposure time is necessary to improve sensitivity, but this can cause motion blur or overexposure. Consequently, traditional image processing methods are slow when synthesizing high dynamic range images, requiring complex circuit designs and significant computational resources, making them unsuitable for real-time processing. Summary of the Invention

[0004] In view of this, in order to overcome the above limitations, this invention proposes a real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics. The aim is to achieve fast and efficient image synthesis through hardware to improve the dynamic range and quality of images. By simultaneously outputting high-brightness and low-brightness images through the multi-gain channels of an sCMOS (Scientific Complementary Metal-Oxide-Semiconductor) sensor, a dual-gain channel foundation is provided for high dynamic range synthesis. By synthesizing images with different gains, the dynamic range of the image can be significantly improved while maintaining image detail and quality. This effectively solves the limitations of traditional image sensors in high dynamic range imaging and meets the demands of modern imaging technology for high dynamic range, high sensitivity, and real-time processing.

[0005] This invention is achieved using the following technical solution:

[0006] In a first aspect, the present invention provides a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics, the method comprising the following steps:

[0007] Dual-gain image data is acquired by deploying sensors with high and low gain channels;

[0008] The acquired dual-gain image data is subjected to noise suppression processing, and the image is corrected for non-uniformity to obtain preprocessed image data.

[0009] Image fusion combines high-gain and low-gain images into a high dynamic range image, and enhances the image signal-to-noise ratio and readout speed by merging adjacent pixels.

[0010] Through actual shooting tests, the dynamic range, image quality, and processing speed of the synthesized high dynamic range images were verified.

[0011] As a further aspect of the present invention, the deployed sensor is an sCMOS image sensor that has high and low gain channels and simultaneously outputs two images with different brightness levels.

[0012] As a further aspect of the present invention, the sCMOS image sensor integrates an A / D converter. Each column of pixels has a 12-bit high-speed A / D converter. A 32-channel LVDS high-speed serial transmitter sends high-speed image data out of the sCMOS image sensor. The top 16 channels and the bottom 16 channels are used to transmit low-gain images and high-gain images simultaneously, respectively.

[0013] As a further aspect of the present invention, the acquisition of dual-gain image data is controlled by an FPGA, and image data is read from both the high-gain and low-gain channels of the sensor simultaneously.

[0014] As a further aspect of the present invention, when acquiring dual-gain image data, the logic controller is responsible for generating control signals for the image sensor and providing a reference clock for the image sensor interface for timing control and data flow read / write control.

[0015] As a further aspect of the present invention, a sensor with high and low gain channels is deployed on a high dynamic range cooled sCMOS camera. The sCMOS camera is integrated with an FPGA, a sensor unit, and a main signal unit to form a high dynamic range camera system for capturing detailed scenes in high contrast environments.

[0016] As a further embodiment of the present invention, the sensor unit includes an FPGA processor, a DDR cache, a gigabit Ethernet port, a USB 3.0 high-speed interface, a TEC cooling module, a heating module, and a power management module; the sensor unit and the main signal unit are connected by two sets of 51-pin FPC flexible flat cables.

[0017] As a further embodiment of the present invention, the sCMOS camera has a maximum envelope of 44mm, the sensor unit is a circular circuit board with a diameter of 37.5mm, and the main signal board is a slender strip circuit board with a width of 39.5mm.

[0018] In a second aspect, the present invention also provides a camera for performing a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics, the camera comprising:

[0019] The sensor unit uses an sCMOS image sensor, has high and low gain channels, and is used to simultaneously output two images with different brightness levels.

[0020] The electronic architecture uses a XILINX logic controller to generate control signals for the image sensor and provide a reference clock for the image sensor interface;

[0021] The camera structure includes a sensor unit and a main signal unit, which are connected via an FPC flexible flat cable;

[0022] The cooling system employs an integrated heat dissipation technology based on a dual-layer TEC wind-powered cooling housing to provide effective cooling for the sCMOS camera.

[0023] The non-uniformity correction module performs non-uniformity correction on the high-gain channel to eliminate response differences between sensor pixels.

[0024] The dual-gain characteristic analysis module analyzes the dual-gain characteristics of sCMOS sensors.

[0025] High dynamic range image synthesis module, used for image synthesis;

[0026] The FPGA implementation module is used to program the image synthesis and image processing steps into the FPGA. By integrating the FPGA with the sensor unit and the main signal unit, a complete high dynamic range camera system is formed and its performance is tested.

[0027] The present invention also includes a computer device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics.

[0028] The present invention also includes a computer-readable storage medium storing computer instructions for causing the computer to execute the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics.

[0029] Compared with existing technologies, the real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics provided by this invention have the following advantages:

[0030] 1. Improved image dynamic range: This invention utilizes the high and low gain channels of the sCMOS sensor to simultaneously output images of different brightness levels. Through real-time hardware synthesis, the dynamic range of the image is significantly improved, enabling the camera to capture more details in high-contrast scenes, whether in high-brightness or low-brightness areas.

[0031] 2. Enhanced Image Quality: This invention eliminates response differences between sensor pixels through non-uniformity correction technology, improving image uniformity and quality. The use of back-end binning technology merges adjacent pixels, improving the signal-to-noise ratio and resulting in clearer images.

[0032] 3. Improved real-time processing capabilities and optimized hardware design: This invention implements the image synthesis algorithm in an FPGA, utilizing its high-speed parallel processing capabilities to achieve real-time image synthesis, meeting the needs of rapid imaging. It also optimizes the use of internal FPGA resources, including logic units, RAM blocks, and I / O ports, improving processing efficiency and reducing power consumption, making the camera system more efficient and stable.

[0033] 4. Compact Camera Design and Improved Sensitivity and Image Quality Through Effective Cooling: The camera of this invention features a compact design, including a sensor unit and a main signal unit, connected via an FPC flexible cable. This results in a small, lightweight camera that is easy to carry and integrate into various application systems. Furthermore, the use of a dual-layer TEC wind-cooled integrated housing heat dissipation technology provides an effective cooling solution for the sCMOS camera, reducing dark current and improving sensor sensitivity and image quality. The Zynq-7000 series FPGA processor and programmable resources enable hardware and software co-design, enhancing system flexibility and scalability.

[0034] In summary, the real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics of the present invention not only improve the dynamic range and quality of images, but also achieve real-time processing, featuring a compact design and an effective cooling solution. The camera system of the present invention is suitable for a variety of imaging applications requiring high dynamic range and high sensitivity, such as astronomical observation, security monitoring, and industrial inspection, and has broad market application prospects. Furthermore, through system integration and testing, the performance and stability of the camera system are ensured, enabling the camera to work reliably under various environmental conditions, reducing the manufacturing and operating costs of the system, and making high dynamic range imaging technology more economical and affordable, with significant beneficial effects.

[0035] These or other aspects of the invention will become more apparent from the following description of embodiments. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the accompanying drawings used in the description of the exemplary embodiments or related technologies will be briefly introduced below. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation thereof. In the drawings:

[0037] Figure 1 This invention relates to a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics and an sCMOS camera electronics architecture diagram.

[0038] Figure 2 This is a schematic diagram of the power tree of the sensor unit in the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics according to an embodiment of the present invention.

[0039] Figure 3 This is a block diagram of the main signal unit in the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics according to an embodiment of the present invention.

[0040] Figure 4 This is a schematic diagram of the power tree of the main signal unit in the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics according to an embodiment of the present invention.

[0041] Figure 5 This is a spectral response curve of an sCMOS image sensor in the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics according to an embodiment of the present invention.

[0042] Figure 6 This is a curve of dark current of an sCMOS image sensor versus temperature in the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics according to an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0045] The technical solutions in the exemplary embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Traditional image sensors have limited dynamic range under single exposure, making it impossible to capture details in both high-brightness and low-brightness areas simultaneously. This results in either overexposure in bright areas or loss of detail in dark areas in high-contrast scenes. Under low-light conditions, traditional sensors exhibit high noise levels, leading to degraded image quality and insufficient sensitivity. To improve sensitivity, exposure time needs to be increased, but this may result in motion blur or overexposure. Consequently, traditional image processing methods are slow when synthesizing high dynamic range images, requiring complex circuit designs and substantial computational resources, which cannot meet the demands of real-time processing.

[0047] In view of this, the present invention provides a real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics, aiming to achieve fast and efficient image synthesis through hardware to improve the dynamic range and quality of images. By simultaneously outputting high-brightness and low-brightness images through the multi-gain channels of the sCMOS sensor, a dual-gain channel basis is provided for high dynamic range synthesis. By synthesizing images with different gains, the dynamic range of the image can be significantly improved while maintaining the detail and quality of the image. This effectively solves the limitations of traditional image sensors in high dynamic range imaging and meets the needs of modern imaging technology for high dynamic range, high sensitivity, and real-time processing.

[0048] The technical solution of the present invention will be further described below with reference to specific embodiments:

[0049] See Figure 1 As shown, Figure 1A flowchart illustrating a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics provided by the present invention. One embodiment of the present invention provides a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics, comprising the following steps:

[0050] Step S10: Acquire dual-gain image data using a deployed sensor with high and low gain channels.

[0051] Step S20: Perform noise suppression processing on the acquired dual-gain image data and perform non-uniformity correction on the image to obtain preprocessed image data.

[0052] Step S30: The high-gain and low-gain images are fused into a high dynamic range image through image fusion, and the image signal-to-noise ratio and reading speed are enhanced by merging adjacent pixels.

[0053] Step S40: Verify the dynamic range, image quality, and processing speed of the synthesized high dynamic range image through actual shooting tests.

[0054] This embodiment of the real-time hardware high dynamic range synthesis method and camera for dual-gain grayscale characteristics first selects the sensor type and performs the corresponding electronic architecture design according to the task requirements; then, the camera structure and cooling design are carried out; then, non-uniformity correction is performed on the high-gain channel; then, the dual-gain characteristics of the sCMOS sensor are analyzed; then, the large dynamic range image synthesis method is studied; finally, the above algorithms are implemented in the FPGA and tested.

[0055] In this embodiment, the deployed sensor is an sCMOS image sensor with high and low gain channels, simultaneously outputting two images with different brightness levels. The sCMOS image sensor integrates an A / D converter; each column of pixels has a 12-bit high-speed A / D converter. A 32-channel LVDS high-speed serial transmitter sends high-speed image data out of the sCMOS image sensor. The top 16 channels and the bottom 16 channels are used to simultaneously transmit the low-gain image and the high-gain image, respectively. The performance parameters of this sCMOS image sensor are shown in Table 1 below, and its spectral response characteristics are as follows: Figure 5 As shown; the relationship between dark current and temperature is as follows: Figure 6 As shown, the evaluation indicates that the sCMOS sensor needs to be cooled to around 0°C.

[0056] Table 1. CMOS Image Sensor Parameter Values

[0057]

[0058]

[0059] In this embodiment, the camera uses Xilinx's XC7Z035 series FPGA (Field-Programmable Logic Device) as the logic controller to generate control signals for the image sensor and provide a reference clock for the image sensor interface. That is, the timing control and data flow read / write control of the entire system are all performed by the FPGA. See also Figure 1 As shown, Figure 1 This is a schematic diagram of the electronic architecture of an sCMOS camera in an image acquisition system.

[0060] The acquisition of dual-gain image data is controlled by an FPGA. Image data is read from both the high-gain and low-gain channels of the sensor. When acquiring dual-gain image data, the logic controller is responsible for generating control signals for the image sensor and providing a reference clock for the image sensor interface for timing control and data flow read / write control.

[0061] In this embodiment, see Figure 1 As shown, a sensor with high and low gain channels is deployed on a high dynamic range cooled sCMOS camera. The sCMOS camera is integrated with an FPGA, a sensor unit, and a main signal unit to form a high dynamic range camera system, which is used to capture detailed scenes in high contrast environments.

[0062] The sensor unit includes an FPGA processor, a DDR cache, a gigabit Ethernet port, a USB 3.0 high-speed interface, a TEC cooling module, a heating module, and a power management module; the sensor unit and the main signal unit are connected by two sets of 51-pin FPC flexible flat cables.

[0063] In this embodiment, the maximum envelope of the sCMOS camera is 44mm, the sensor unit is a circular circuit board with a diameter of 37.5mm, and the main signal board is a slender strip circuit board with a width of 39.5mm.

[0064] Among them, see Figure 2As shown, the power supply tree for the sensor unit uses an LDO ADM7172 and a linear amplifier LT6221, while the main signal board uses a digital signal processing board based on a ZYNQ 7FPGA XC7Z045. The LDO ADM7172, as a CMOS low-dropout linear regulator (LDO), operates from 2.3V to 6.5V and can provide up to 2A of output current. As a high-output-current LDO, the LDO ADM7172 is suitable for regulating high-performance analog and mixed-signal circuits operating on power rails from 6V to as low as 1.2V, providing high power supply rejection ratio and low noise, achieving excellent voltage and load transient response using only a single 4.7μF ceramic output capacitor. For a load step of 1mA to 1.5A, the load transient response is typically 1.5μs. The LDO ADM7172 regulator has an output noise of 5μV rms, which is independent of the output voltage. In this embodiment, the ADM7172 is packaged in an 8-pin 3mm×3mm LFCSP package, which is compact and can provide excellent thermal performance for applications with up to 2A output current. It has a flat shape and a small footprint.

[0065] In this embodiment, the linear amplifier LT6221, as a dual-channel, low-power, high-speed, rail-to-rail input and output operational amplifier, exhibits superior DC performance. Compared to other devices with similar bandwidth, the LT6221 offers reduced supply current, lower input offset voltage, lower input bias current, and higher DC gain. The LT6221 features an input offset voltage below 100 μV, an input bias current less than 15 nA, and an open-loop gain of 100 V / mV. It has an input range encompassing both supply rails and an output that swings within 10 mV of either rail, maximizing the signal dynamic range in low-supply-voltage applications. The LT6221 maintains its performance across a 2.2 V to 12.6 V supply range, and its input can be driven beyond the supply range without damage or output inversion.

[0066] In this embodiment, the LM27761 low-noise regulated switched-capacitor voltage inverter can also be used to provide an adjustable, ultra-low noise output for input voltages ranging from 2.7V to 5.5V. Using four low-cost capacitors in the application solution, up to 250mA of output current can be provided. The device's regulated output is adjustable from -5V to -1.5V. The LM27761 operates at a switching frequency of 2MHz (typical) to reduce output resistance and voltage ripple. With an operating current of only 370μA (charge pump power efficiency greater than 80% for most loads) and a typical shutdown current of 7μA, the LM27761 provides ideal performance when driving power amplifiers, DAC bias rails, and other high-current, low-noise voltage applications.

[0067] In this embodiment, the main signal board is a slender strip circuit board with a width of 39.5mm, and it adopts a digital signal processing board based on the ZYNQ 7FPGA XC7Z045. Specifically, the FPGA selected is the XQ7Z045-1RF676Q from Xilinx's Zynq-7000 series of Field Programmable Gate Arrays (FPGAs) for industrial, commercial, aerospace, defense, communications, and security applications. This series of FPGAs mainly consists of two parts: a processing system (PS) and programmable logic (PL). The PL uses a 28nm process; the PS is based on two Cortex A9 ARM cores and also includes on-chip memory, an external memory interface (DDR), and a series of peripheral interfaces. The Zynq-7000 series integrates the ARM CPU and peripherals into a single chip, giving it both processor and FPGA characteristics, making it particularly suitable for hardware and software co-design. The logic resources of the PL mainly include:

[0068] a) Logic unit: 350K;

[0069] b) 36Kb block RAM: 545 blocks;

[0070] c) The system's maximum operating clock speed is 667MHz;

[0071] d) I / O voltage 1.2V~3.3V, core voltage +1.0V;

[0072] e) User-available IO: 388.

[0073] See Figure 3As shown, 1GB DDR3 memory chips are connected to both the PS and PL ends of the FPGA. The program memory is 128MB of Nor FLASH. A Gigabit Ethernet connection is established on the PS end of the FPGA. USB 3.0 uses Cypress's dedicated USB 3.0 interface protocol chip, CYUSB3014. The power tree of the main signal board is shown below. Figure 4 As shown, the LTM4616 is a complete dual-channel, two-phase, 8A output per channel, switch-mode DC / DC power regulator system in a 15mm x 15mm surface-mount LGA or BGA package. The package integrates the switching controller, power FET, inductor, and all supporting components. The LTM4616 operates from 2.7V to 5.5V and supports two outputs ranging from 0.6V to 5V (each set by a single external resistor). This high-efficiency design provides up to 8A of continuous current (10A peak) per output. Only large-capacity input and output capacitors are required (depending on ripple requirements).

[0074] In this embodiment, the MAX1978 / MAX1979 is the smallest, safest, and most accurate fully monolithic temperature controller for Peltier thermoelectric cooler (TEC) modules. On-chip FETs and thermal control loop circuitry provide high efficiency while minimizing external components. Selectable switching frequencies of 500kHz / 1MHz and unique ripple cancellation circuitry optimize component size and efficiency while reducing noise. The switching speed of the built-in MOSFETs is optimized to reduce noise and EMI. An ultra-low drift chopper amplifier maintains temperature stability of ±0.001°C. Direct control of output current, rather than voltage, eliminates inrush current. Independent heating and cooling current and voltage limits provide the highest level of TEC protection.

[0075] This invention discloses a real-time hardware high dynamic range (HDR) synthesis method based on dual-gain grayscale characteristics. It utilizes the high and low gain channels of an sCMOS sensor to simultaneously output images at different brightness levels. Through real-time hardware synthesis, the dynamic range of the image is significantly improved, enabling the camera to capture more detail in high-contrast scenes, regardless of whether the image is in high or low brightness areas. Non-uniformity correction technology eliminates response differences between sensor pixels, improving image uniformity and quality. Back-end binning technology merges adjacent pixels, improving the signal-to-noise ratio and resulting in a clearer image.

[0076] This invention implements the image synthesis algorithm within an FPGA, leveraging its high-speed parallel processing capabilities to achieve real-time image synthesis and meet the demands of rapid imaging. It also optimizes the use of internal FPGA resources, including logic units, RAM blocks, and I / O ports, improving processing efficiency and reducing power consumption, resulting in a more efficient and stable camera system. The camera of this invention features a compact design, including a sensor unit and a main signal unit, connected via an FPC flexible cable, making the camera small, lightweight, portable, and easily integrated into various application systems. Furthermore, it employs a dual-layer TEC wind-cooled integrated heat dissipation technology, providing an effective cooling solution for the sCMOS camera, reducing dark current, and improving sensor sensitivity and image quality. Finally, it utilizes the processor and programmable resources of the Zynq-7000 series FPGA to achieve hardware-software co-design, enhancing system flexibility and scalability.

[0077] It should be understood that although the above description follows a certain order, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, some steps in this embodiment may include multiple steps or multiple stages, which are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0078] In one embodiment, the present invention provides a camera for performing the above-described real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics, the camera comprising:

[0079] The sensor unit uses an sCMOS image sensor, has high and low gain channels, and is used to simultaneously output two images with different brightness levels.

[0080] The electronic architecture uses a XILINX logic controller to generate control signals for the image sensor and provide a reference clock for the image sensor interface;

[0081] The camera structure includes a sensor unit and a main signal unit, which are connected via an FPC flexible flat cable;

[0082] The cooling system employs an integrated heat dissipation technology based on a dual-layer TEC wind-powered cooling housing to provide effective cooling for the sCMOS camera.

[0083] The non-uniformity correction module performs non-uniformity correction on the high-gain channel to eliminate response differences between sensor pixels.

[0084] The dual-gain characteristic analysis module analyzes the dual-gain characteristics of sCMOS sensors.

[0085] High dynamic range image synthesis module, used for image synthesis;

[0086] The FPGA implementation module is used to program the image synthesis and image processing steps into the FPGA. By integrating the FPGA with the sensor unit and the main signal unit, a complete high dynamic range camera system is formed and its performance is tested.

[0087] This embodiment utilizes the characteristic of domestically produced sCMOS sensors that can simultaneously output two images with different brightness levels through high and low gain channels. It studies a real-time hardware high dynamic range synthesis method and back-end binning for dual-gain grayscale characteristics to improve the dynamic range and sensitivity of the images. Based on the integrated heat dissipation technology of a dual-layer TEC wind-powered cooling housing, a miniaturized and compact readout circuit for a cooling sCMOS camera is designed.

[0088] In this embodiment, the camera employs the steps of a real-time hardware high dynamic range synthesis method with dual-gain grayscale characteristics as described above. Therefore, the camera's operation process will not be described in detail in this embodiment.

[0089] In summary, the real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics of the present invention not only improve the dynamic range and quality of images, but also achieve real-time processing, featuring a compact design and an effective cooling solution. The camera system of the present invention is suitable for a variety of imaging applications requiring high dynamic range and high sensitivity, such as astronomical observation, security monitoring, and industrial inspection, and has broad market application prospects. Furthermore, through system integration and testing, the performance and stability of the camera system are ensured, enabling the camera to work reliably under various environmental conditions, reducing the manufacturing and operating costs of the system, and making high dynamic range imaging technology more economical and affordable, with significant beneficial effects.

[0090] In one embodiment, a computer device is also provided in the present invention, including at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the steps of the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics.

[0091] In one embodiment, the present invention also provides a computer-readable storage medium storing computer instructions for causing the computer to perform the steps of the real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics.

[0092] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program, characterized by computer instructions, instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Furthermore, any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0093] Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as static random access memory or dynamic random access memory.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time hardware high dynamic range synthesis method of dual-gain gray scale characteristics, characterized in that, The method comprises the following steps: Obtaining double-gain image data by deploying a sensor with high and low gain channels, wherein the deployed sensor is an sCMOS image sensor with high and low gain channels, which simultaneously outputs two images with different brightness levels, and the sCMOS image sensor is integrated with an A / D converter, each column of pixels has a 12-bit high-speed A / D converter, and high-speed image data is sent out of the sCMOS image sensor by a 32-channel LVDS high-speed serial transmitter, the top 16 channels and the bottom 16 channels are respectively used to simultaneously transmit low-gain images and high-gain images; Performing noise suppression processing on the obtained double-gain image data, and performing non-uniformity correction on the image to obtain preprocessed image data; Fusing the high-gain and low-gain images into a high dynamic range image through image fusion, and enhancing the image signal-to-noise ratio and reading speed by merging adjacent pixels; Verifying the dynamic range, image quality and processing speed of the synthesized high dynamic range image through actual shooting tests; The sensor with high and low gain channels is deployed on a high dynamic range refrigeration sCMOS camera, the sCMOS camera is integrated with a sensor unit and a main signal unit by an FPGA, and a high dynamic range camera system is formed, which is used to capture scenes with details in a high-contrast environment; the sensor unit comprises an FPGA processor, a DDR buffer, a gigabit network port, a USB 3.0 high-speed interface, a TEC refrigeration module, a heating module and a power management module; the sensor unit and the main signal unit are connected through two groups of 51-core FPC flexible flat cables.

2. The real-time hardware high dynamic range synthesis method of dual-gain gray scale characteristics according to claim 1, characterized in that, The FPGA controls the reading of image data from the high-gain and low-gain channels of the sensor when obtaining double-gain image data.

3. The real-time hardware high dynamic range synthesis method of dual-gain gray scale characteristics according to claim 2, characterized in that, When obtaining double-gain image data, the logic controller is responsible for generating control signals of the image sensor and providing reference clock of the image sensor interface, which is used for timing control and read-write control of data flow.

4. The real-time hardware high dynamic range synthesis method of dual-gain gray scale characteristics according to claim 3, characterized in that, The maximum envelope of the sCMOS camera is 44mm, the sensor unit is a circular circuit board with a diameter of 37.5mm, and the main signal board is an elongated strip-shaped circuit board with a width of 39.5mm.

5. A camera characterized by, A real-time hardware high dynamic range synthesis method for performing the double-gain gray scale characteristics of any one of claims 1-4, the camera comprising: Sensor unit, using sCMOS image sensor, with high and low gain channels, and for outputting two images with different brightness levels simultaneously, sCMOS image sensor integrated with A / D converter, each column of pixels with a 12-bit high-speed A / D converter, high-speed image data sent out of sCMOS image sensor by 32-way LVDS high-speed serial transmitter, top 16-way and bottom 16-way channels used for transmitting low gain image and high gain image simultaneously, sCMOS camera integrated with FPGA and sensor unit, main signal unit, to form high dynamic range camera system, for capturing scenes with details in high contrast environment; the sensor unit includes FPGA processor, DDR buffer, gigabit network port, USB 3.0 high-speed interface, TEC refrigeration module, heating module, power management module; the sensor unit and the main signal unit are connected through two groups of 51-core FPC flexible flat cable; Electronic framework, using logic controller to generate control signals of image sensor and provide reference clock of image sensor interface; Camera structure, including sensor unit and main signal unit, connected through FPC flexible flat cable; Refrigeration system, using double-layer TEC wind-driven refrigeration machine shell integrated forming heat dissipation technology, to provide effective refrigeration for sCMOS camera; Non-uniformity correction module, for non-uniformity correction of high gain channel, to eliminate response difference between sensor pixels; Dual-gain characteristic analysis module, for analyzing dual-gain characteristics of sCMOS sensor; High dynamic range image synthesis module, for image synthesis; FPGA implementation module, for programming image synthesis and image processing steps in FPGA, through integrating FPGA with sensor unit and main signal unit, to form complete high dynamic range camera system, and to perform performance test.

Images