Real-time hardware high dynamic range synthesis method with double-gain gray scale characteristic and camera

Abstract

The invention provides a real-time hardware high dynamic range synthesis method with double-gain gray characteristics and a camera, and relates to the technical field of optical imaging, and the method comprises the steps: obtaining double-gain image data through a deployed sensor with a high-gain channel and a low-gain channel; performing noise suppression processing on the obtained double-gain image data, and performing non-uniformity correction on the image to obtain preprocessed image data; the high-gain image and the low-gain image are fused into a high-dynamic-range image through image fusion, and the signal-to-noise ratio and the reading speed of the image are enhanced by combining adjacent pixels; and verifying the dynamic range, the image quality and the processing speed of the synthesized high dynamic range image through an actual shooting test. High-brightness and low-brightness images are output at the same time through the multiple gain channels of the sensor, a double-gain channel basis is provided for high-dynamic-range synthesis, the dynamic range of the images can be remarkably widened by synthesizing the images with different gains, and meanwhile details and quality of the images are kept.

Description

Technical Field

[0001] The present invention relates to the field of optical imaging technology, and in particular to a real-time hardware high dynamic range synthesis method and a camera with dual-gain grayscale characteristics. Background Art

[0002] In modern imaging technology, image sensors are key components for acquiring image information. With the development of science and technology, the application areas of image sensors continue to expand, including but not limited to astronomical observation, medical imaging, industrial detection, security monitoring, etc. In these applications, image sensors need to capture scenes with a high dynamic range, that is, they can record details of very bright and very dark areas at the same time. However, traditional image sensors, such as CCD (Charge-Coupled Device) and early CMOS (Complementary Metal-Oxide-Semiconductor) sensors, often have difficulty capturing high dynamic range scenes in a single exposure due to the limitations of their physical and electronic properties.

[0003] Among them, traditional image sensors have limited dynamic range under single exposure and cannot capture details in both high-brightness and low-brightness areas at the same time. This results in either overexposure of bright areas or loss of details in dark areas in high-contrast scenes. In low-light conditions, traditional sensors have high noise levels, resulting in reduced image quality and insufficient sensitivity. In order to increase sensitivity, the exposure time needs to be increased, but this may cause motion blur or overexposure, which in turn causes traditional image processing methods to have slow processing speeds when synthesizing high dynamic range images, requiring complex circuit design and a large amount of computing resources, and cannot meet the needs of real-time processing. Summary of the invention

[0004] In view of this, in order to overcome the above limitations, the present invention proposes a real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics, aiming to achieve fast and efficient image synthesis through hardware to improve the dynamic range and quality of the image. The multi-gain channels of the sCMOS (Scientific Complementary Metal-Oxide-Semiconductor) sensor simultaneously output high-brightness and low-brightness images, providing a dual-gain channel basis for high dynamic range synthesis. By synthesizing images with different gains, the dynamic range of the image can be significantly improved while maintaining the details and quality of the image, which can effectively solve the limitations of traditional image sensors in high dynamic range imaging and meet the requirements of modern imaging technology for high dynamic range, high sensitivity and real-time processing.

[0005] The present invention is implemented by the following technical solutions:

[0006] In a first aspect, the present invention provides a real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics, the method comprising the following steps:

[0007] Obtain dual-gain image data through deployed sensors with high and low gain channels;

[0008] Performing noise suppression processing on the acquired dual-gain image data and performing non-uniformity correction on the image to obtain pre-processed image data;

[0009] The high-gain and low-gain images are fused into a high dynamic range image through image fusion, and the image signal-to-noise ratio and reading speed are enhanced by merging adjacent pixels;

[0010] Through actual shooting tests, the dynamic range, image quality and processing speed of the synthesized high dynamic range images are verified.

[0011] As a further solution of the present invention, the deployed sensor is a sCMOS image sensor having high and low gain channels and outputting two images of different brightness levels at the same time.

[0012] As a further solution of the present invention, the sCMOS image sensor is integrated with an A / D converter, and each column of pixels has a 12-bit high-speed A / D converter. A 32-channel LVDS high-speed serial transmitter sends high-speed image data out of the sCMOS image sensor, and the top 16 channels and the bottom 16 channels are respectively used to simultaneously transmit low-gain images and high-gain images.

[0013] As a further solution of the present invention, the acquisition of dual-gain image data is controlled by FPGA, and image data is read from the high-gain and low-gain channels of the sensor at the same time.

[0014] As a further solution of the present invention, when acquiring dual-gain image data, the logic controller is responsible for generating a control signal of the image sensor and providing a reference clock of the image sensor interface for timing control and read and write control of the data stream.

[0015] As a further solution of the present invention, a sensor with high and low gain channels is deployed on a high dynamic range cooled sCMOS camera. The sCMOS camera is integrated with an FPGA, a sensor unit, and a main signal unit to form a high dynamic range camera system for capturing detailed scenes in a high contrast environment.

[0016] As a further solution of the present invention, the sensor unit includes an FPGA processor, a DDR buffer, a gigabit network port, a USB3.0 high-speed interface, a TEC cooling module, a heating module, and a power management module; the sensor unit and the main signal unit are connected by two sets of 51-core FPC soft cables.

[0017] As a further solution of the present invention, the maximum envelope of the sCMOS camera is 44 mm, the sensor unit is a circular circuit board with a diameter of 37.5 mm, and the main signal board is a slender strip circuit board with a width of 39.5 mm.

[0018] In a second aspect, the present invention further provides a camera for executing a real-time hardware high dynamic range synthesis method of a dual-gain grayscale characteristic, the camera comprising:

[0019] The sensor unit adopts an sCMOS image sensor, has high and low gain channels, and is used to simultaneously output two images with different brightness levels;

[0020] Electronic architecture, using XILINX's logic controller to generate image sensor control signals and provide reference clock for image sensor interface;

[0021] The camera structure, including the sensor unit and the main signal unit, is connected via an FPC cable;

[0022] The cooling system uses a double-layer TEC wind cooling shell integrated cooling technology to provide effective cooling for the sCMOS camera;

[0023] A non-uniformity correction module performs non-uniformity correction on the high-gain channel to eliminate the response differences between sensor pixels;

[0024] Dual-gain characteristic analysis module, which analyzes the dual-gain characteristics of sCMOS sensors;

[0025] High dynamic range image synthesis module, used for image synthesis;

[0026] The FPGA implementation module is used to program the image synthesis and image processing steps into the FPGA. By integrating the FPGA with the sensor unit and the main signal unit, a complete high dynamic range camera system is constructed and performance testing is performed.

[0027] The present invention also includes a computer device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the real-time hardware high dynamic range synthesis method of the dual-gain grayscale characteristics.

[0028] The present invention also includes a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the real-time hardware high dynamic range synthesis method of the dual-gain grayscale characteristic.

[0029] Compared with the prior art, the real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics provided by the present invention have the following beneficial effects:

[0030] 1. Improved image dynamic range: The present invention utilizes the high and low gain channels of the sCMOS sensor to simultaneously output images of different brightness levels. Through a real-time hardware synthesis method, the dynamic range of the image is significantly improved, enabling the camera to capture more details in high-contrast scenes, whether in high-brightness or low-brightness areas.

[0031] 2. Enhanced image quality: The present invention eliminates the response differences between sensor pixels through non-uniformity correction technology, improves the uniformity and quality of the image. The back-end binning technology is used to merge adjacent pixels, improve the signal-to-noise ratio of the image, and make the image clearer.

[0032] 3. Improved real-time processing capability and optimized hardware design: The present invention implements the image synthesis algorithm in FPGA and uses its high-speed parallel processing capability to achieve real-time image synthesis and meet the needs of fast imaging. It also optimizes the use of FPGA internal resources, including logic units, RAM blocks, and IO ports, improves processing efficiency, reduces power consumption, and makes the camera system more efficient and stable.

[0033] 4. The camera is compact in design and improves the sensitivity and image quality of the camera through effective cooling: The camera of the present invention adopts a compact design, including a sensor unit and a main signal unit, which are connected by FPC flexible cables, making the camera small in size and light in weight, easy to carry and integrate into various application systems. In addition, the integrated molding heat dissipation technology based on the double-layer TEC wind-powered cooling shell is adopted to provide an effective cooling solution for the sCMOS camera, reduce the dark current, and improve the sensitivity and image quality of the sensor. And by using the processor and programmable resources of the Zynq-7000 series FPGA, the hardware and software collaborative design is realized, and the flexibility and scalability of the system are improved.

[0034] In summary, the real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics of the present invention not only improve the dynamic range and quality of the image, but also realize real-time processing, and have a compact design and an effective cooling solution. The camera system of the present invention is suitable for a variety of imaging applications requiring high dynamic range and high sensitivity, such as astronomical observation, security monitoring, industrial detection, etc., and has broad market application prospects. Through system integration and testing, the performance and stability of the camera system are ensured, so that the camera can work reliably under various environmental conditions, reducing the manufacturing and operating costs of the system, making high dynamic range imaging technology more economical and affordable, and having significant beneficial effects.

[0035] These and other aspects of the present invention will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the exemplary embodiments or related technical descriptions. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0037] Figure 1 A real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics and an electronic architecture diagram of an sCMOS camera in a camera according to an embodiment of the present invention.

[0038] Figure 2 A schematic diagram of a power tree for a sensor unit in a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics according to an embodiment of the present invention.

[0039] Figure 3 This is a block diagram of a main signal unit in a real-time hardware high dynamic range synthesis method with dual-gain grayscale characteristics according to an embodiment of the present invention.

[0040] Figure 4 The diagram is a schematic diagram of a power supply tree of a main signal unit in a real-time hardware high dynamic range synthesis method with dual-gain grayscale characteristics according to an embodiment of the present invention.

[0041] Figure 5 Spectral response curve of the sCMOS image sensor in the real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics according to an embodiment of the present invention.

[0042] Figure 6 This is a curve diagram of dark current of an sCMOS image sensor versus temperature in a real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0045] The following will be combined with the drawings in the exemplary embodiments of the present invention to clearly and completely describe the technical solutions in the exemplary embodiments of the present invention. Obviously, the exemplary embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] Because traditional image sensors have limited dynamic range under single exposure, they cannot capture details in both high-brightness and low-brightness areas at the same time. This results in either overexposure of bright areas or loss of details in dark areas in high-contrast scenes. Under low-light conditions, traditional sensors have high noise levels, resulting in reduced image quality and insufficient sensitivity. In order to improve sensitivity, the exposure time needs to be increased, but this may cause motion blur or overexposure, which in turn causes traditional image processing methods to have slow processing speeds when synthesizing high dynamic range images, requiring complex circuit design and a large amount of computing resources, and unable to meet the needs of real-time processing.

[0047] In view of this, the present invention provides a real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics, aiming to achieve fast and efficient image synthesis through hardware to improve the dynamic range and quality of the image. High-brightness and low-brightness images are output simultaneously through the multi-gain channels of the sCMOS sensor, providing a dual-gain channel basis for high dynamic range synthesis. By synthesizing images with different gains, the dynamic range of the image can be significantly improved while maintaining the details and quality of the image. The limitations of traditional image sensors in high dynamic range imaging can be effectively resolved, and the requirements of modern imaging technology for high dynamic range, high sensitivity and real-time processing can be met.

[0048] The technical solution of the present invention is further described below in conjunction with specific embodiments:

[0049] See also Figure 1 As shown, Figure 1A flow chart of a real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics provided by the present invention. A real-time hardware high dynamic range synthesis method for dual-gain grayscale characteristics provided in one embodiment of the present invention comprises the following steps:

[0050] Step S10: acquiring dual-gain image data by deploying a sensor with high and low gain channels.

[0051] Step S20: performing noise suppression processing on the acquired dual-gain image data and performing non-uniformity correction on the image to obtain pre-processed image data.

[0052] Step S30: fuse the high-gain image and the low-gain image into a high dynamic range image by image fusion, and enhance the image signal-to-noise ratio and reading speed by merging adjacent pixels.

[0053] Step S40: verifying the dynamic range, image quality and processing speed of the synthesized high dynamic range image through actual shooting test.

[0054] The real-time hardware high dynamic range synthesis method and camera of dual-gain grayscale characteristics of this embodiment first select the sensor selection and perform the corresponding electronic architecture design according to the task requirements; then perform the structure and cooling design of the camera; then perform non-uniformity correction on the high gain channel; then analyze the dual gain characteristics of the sCMOS sensor; then study the large dynamic range image synthesis method; finally, implement the above algorithm inside the FPGA and finally test it.

[0055] In this embodiment, the deployed sensor is a sCMOS image sensor with high and low gain channels, which outputs two images of different brightness levels at the same time. The sCMOS image sensor is integrated with an A / D converter. Each column of pixels has a 12-bit high-speed A / D converter. The 32-channel LVDS high-speed serial transmitter sends the high-speed image data out of the sCMOS image sensor. The top 16 channels and the bottom 16 channels are used to transmit low-gain images and high-gain images at the same time, respectively. The performance parameters of the sCMOS image sensor are shown in Table 1 below. The spectral response characteristics of the sensor are shown in Table 1 below. Figure 5 As shown; the relationship between dark current and temperature is shown in Figure 6 As shown, after evaluation, the sCMOS sensor needs to be cooled to around 0°C.

[0056] Table 1 sCMOS image sensor parameter values

[0057]

[0058]

[0059] In this embodiment, the camera uses the FPGA (field programmable logic device) XC7Z035 series of XILINX as a logic controller to generate control signals for the image sensor and provide a reference clock for the image sensor interface. That is, the timing control of the entire system and the read and write control of the data flow are all completed by the FPGA. Figure 1 As shown, Figure 1 Schematic diagram of the electronic architecture of the sCMOS camera in the image acquisition system.

[0060] Among them, the acquisition of dual-gain image data is controlled by FPGA, and image data is read from the high-gain and low-gain channels of the sensor at the same time. When acquiring dual-gain image data, the logic controller is responsible for generating control signals of the image sensor and providing a reference clock for the image sensor interface, which is used for timing control and read and write control of data flow.

[0061] In this embodiment, see Figure 1 As shown, the sensor with high and low gain channels is deployed on a high dynamic range cooled sCMOS camera. The sCMOS camera is integrated with an FPGA, a sensor unit, and a main signal unit to form a high dynamic range camera system for capturing detailed scenes in high contrast environments.

[0062] The sensor unit includes an FPGA processor, a DDR buffer, a Gigabit network port, a USB3.0 high-speed interface, a TEC cooling module, a heating module, and a power management module; the sensor unit and the main signal unit are connected through two sets of 51-core FPC flexible cables.

[0063] In this embodiment, the maximum envelope of the sCMOS camera is 44 mm, the sensor unit is a circular circuit board with a diameter of 37.5 mm, and the main signal board is a slender strip circuit board with a width of 39.5 mm.

[0064] Among them, see Figure 2As shown in the figure, the power tree of the sensor unit uses LDO ADM7172 and linear amplifier LT6221, and the main signal board uses a digital signal processing board based on ZYNQ 7FPGA XC7Z045. Among them, LDO ADM7172 is a CMOS low-dropout linear regulator (LDO) with an operating voltage range of 2.3V to 6.5V and can provide an output current of up to 2A. As a high output current LDO, LDOADM7172 is suitable for stabilizing high-performance analog and mixed-signal circuits that operate with power rails from 6V to as low as 1.2V. It can provide high power supply rejection ratio and low noise, and achieve excellent voltage and load transient response with only a small 4.7μF ceramic output capacitor. For a 1mA to 1.5A load step, the load transient response is typically 1.5μs. The LDO ADM7172 regulator has an output noise of 5 μV rms, which is independent of the output voltage. In this embodiment, the ADM7172 is packaged in an 8-lead 3 mm × 3 mm LFCSP package, which is compact and can provide excellent thermal performance for applications with up to 2 A output current, a low profile, and a small footprint.

[0065] In this embodiment, the linear amplifier LT6221 is a dual-channel, low-power, high-speed, rail-to-rail input and output operational amplifier with excellent DC performance. Compared with other devices with similar bandwidth, the linear amplifier LT6221 has reduced supply current, lower input offset voltage, lower input bias current and higher DC gain. The LT6221 has an input offset voltage of less than 100μV, an input bias current of less than 15nA and an open-loop gain of 100V / mV, an input range including two power rails, and an output swinging within 10mV of either power rail, which can maximize the signal dynamic range in low power supply voltage applications. The LT6221 can maintain its performance within the power supply range of 2.2V to 12.6V, and the input can be driven beyond the power supply range without causing damage or output inversion.

[0066] In this embodiment, the LM27761 low noise regulated switched capacitor voltage inverter can also be used to provide an adjustable ultra-low noise output for an input voltage range of 2.7V to 5.5V. Using four low-cost capacitors in the application solution, an output current of up to 250mA can be provided. The regulated output of the device can be adjusted from -5V to -1.5V. The LM27761 runs at a 2MHz (typical) switching frequency to reduce output resistance and voltage ripple. With an operating current of only 370μA (charge pump power efficiency is above 80% for most loads) and a typical shutdown current of 7μA, the LM27761 provides ideal performance when driving power amplifiers, DAC bias rails, and other high current, low noise voltage applications.

[0067] In this embodiment, the main signal board is a slender strip circuit board with a width of 39.5 mm, and the main signal board adopts a digital signal processing board based on ZYNQ 7FPGA XC7Z045. Specifically, the FPGA selected is XQ7Z045-1RF676Q from the Zynq-7000 series field programmable gate array (FPGA) series products launched by Xilinx for industrial, commercial, aviation, defense, communication and security applications. This series of FPGAs mainly includes two parts: processing system PS (processing system) and programmable resources PL (Programmable Logic). PL adopts 28nm process; PS is based on 2 CortexA9 ARM cores, and also includes on-chip memory, off-chip memory interface (DDR) and a series of peripheral interfaces. The Zynq-7000 series integrates ARM CPU and peripherals into one chip, so that the Zynq-7000 series has both processor and FPGA characteristics, which is particularly suitable for hardware and software collaborative design. The logic resources of the PL part mainly include:

[0068] a)Logic unit: 350K;

[0069] b) 36Kb block RAM: 545;

[0070] c) The system's maximum operating clock is 667MHz;

[0071] d) IO voltage 1.2V~3.3V, core voltage +1.0V;

[0072] e) User available IO: 388.

[0073] See also Figure 3As shown in the figure, 1GB DDR3 memory chips are mounted on the PS and PL ends of the FPGA. The program memory is 128MB Nor FLASH. Gigabit Ethernet is connected to the PS end of the FPGA. USB3.0 uses Cypress's dedicated USB3.0 interface protocol chip CYUSB3014. The power tree of the main signal board is as follows: Figure 4 As shown, the LTM4616 is a complete dual-channel, two-phase, 8A output per channel, switch mode DC / DC power regulator system in a 15mmx15mm surface mount LGA or BGA package. The switch controller, power FETs, inductor and all supporting components are built into the package. The LTM4616 operates from an input voltage range of 2.7V to 5.5V and supports two outputs in the range of 0.6V to 5V (each set by a single external resistor). The high efficiency design provides up to 8A of continuous current (10A peak) for each output. Only large-capacity input and output capacitors are required (depending on the ripple requirements).

[0074] In this embodiment, the MAX1978 / MAX1979 are the smallest, safest, and most accurate fully monolithic temperature controllers for Peltier thermoelectric cooler (TEC) modules. On-chip FETs and thermal control loop circuitry minimize external components while providing high efficiency. 500kHz / 1MHz selectable switching frequency and unique ripple cancellation circuitry optimize component size and efficiency while reducing noise. The switching speed of the built-in MOSFET is optimized to reduce noise and EMI. An ultra-low drift chopper amplifier maintains ±0.001°C temperature stability. Output current is controlled directly, rather than voltage, to eliminate inrush current. Independent heating and cooling current and voltage limits provide the highest level of TEC protection.

[0075] The present invention discloses a real-time hardware high dynamic range synthesis method with dual-gain grayscale characteristics. The high and low gain channels of the sCMOS sensor are used to simultaneously output images of different brightness levels. The real-time hardware synthesis method significantly improves the dynamic range of the image, so that the camera can capture more details in high-contrast scenes, whether in high-brightness or low-brightness areas. The non-uniformity correction technology eliminates the response differences between sensor pixels, thereby improving the uniformity and quality of the image. The back-end binning technology is used to merge adjacent pixels, thereby improving the signal-to-noise ratio of the image and making the image clearer.

[0076] The present invention implements the image synthesis algorithm in FPGA, and utilizes its high-speed parallel processing capability to achieve real-time image synthesis and meet the needs of fast imaging. It also optimizes the use of FPGA internal resources, including logic units, RAM blocks and IO ports, improves processing efficiency, reduces power consumption, and makes the camera system more efficient and stable. The camera of the present invention adopts a compact design, including a sensor unit and a main signal unit, which are connected by an FPC flexible cable, so that the camera is small in size and light in weight, easy to carry and integrate into various application systems. Moreover, the integrated molding heat dissipation technology based on the double-layer TEC wind-powered cooling housing is adopted to provide an effective cooling solution for sCMOS cameras, reduce dark current, and improve the sensitivity and image quality of the sensor. And by utilizing the processor and programmable resources of the Zynq-7000 series FPGA, the collaborative design of software and hardware is realized, and the flexibility and scalability of the system are improved.

[0077] It should be understood that, although described in a certain order, these steps are not necessarily performed in sequence in the above order. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, a part of the steps of the present embodiment may include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of the steps or stages in other steps or other steps.

[0078] In one embodiment, the present invention provides a camera for executing the above-mentioned real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics, the camera comprising:

[0079] The sensor unit adopts an sCMOS image sensor, has high and low gain channels, and is used to simultaneously output two images with different brightness levels;

[0080] Electronic architecture, using XILINX's logic controller to generate image sensor control signals and provide reference clock for image sensor interface;

[0081] The camera structure, including the sensor unit and the main signal unit, is connected via an FPC cable;

[0082] The cooling system uses a double-layer TEC wind cooling shell integrated cooling technology to provide effective cooling for the sCMOS camera;

[0083] A non-uniformity correction module performs non-uniformity correction on the high-gain channel to eliminate the response differences between sensor pixels;

[0084] Dual-gain characteristic analysis module, which analyzes the dual-gain characteristics of sCMOS sensors;

[0085] High dynamic range image synthesis module, used for image synthesis;

[0086] The FPGA implementation module is used to program the image synthesis and image processing steps into the FPGA. By integrating the FPGA with the sensor unit and the main signal unit, a complete high dynamic range camera system is constructed and performance testing is performed.

[0087] In this embodiment, based on the characteristic that the high and low gain channels of the domestic sCMOS sensor can simultaneously output two images of different brightness, a real-time hardware high dynamic range synthesis method and back-end binning of the dual-gain grayscale characteristics are studied to achieve an improvement in the image dynamic range and sensitivity; based on the integrated molding heat dissipation technology of the double-layer TEC wind-powered cooling casing, a miniaturized and compact cooling sCMOS camera readout circuit implementation method is designed.

[0088] In this embodiment, the camera adopts the steps of the real-time hardware high dynamic range synthesis method with dual-gain grayscale characteristics as described above during execution. Therefore, the operation process of the camera in this embodiment will not be described in detail.

[0089] In summary, the real-time hardware high dynamic range synthesis method and camera with dual-gain grayscale characteristics of the present invention not only improve the dynamic range and quality of the image, but also realize real-time processing, and have a compact design and an effective cooling solution. The camera system of the present invention is suitable for a variety of imaging applications requiring high dynamic range and high sensitivity, such as astronomical observation, security monitoring, industrial detection, etc., and has broad market application prospects. Through system integration and testing, the performance and stability of the camera system are ensured, so that the camera can work reliably under various environmental conditions, reducing the manufacturing and operating costs of the system, making high dynamic range imaging technology more economical and affordable, and having significant beneficial effects.

[0090] In one embodiment, a computer device is also provided in an embodiment of the present invention, comprising at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics.

[0091] In one embodiment, the present invention further provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the steps of the real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics.

[0092] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be realized by instructing the relevant hardware through a computer program represented by computer instructions, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0093] Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory or optical storage, etc. Volatile memory may include random access memory or external cache memory. As an illustration and not limitation, RAM may be in various forms, such as static random access memory or dynamic random access memory, etc.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time hardware high dynamic range synthesis method with dual-gain grayscale characteristics, characterized in that: The method comprises the following steps: Obtain dual-gain image data through deployed sensors with high and low gain channels; Performing noise suppression processing on the acquired dual-gain image data and performing non-uniformity correction on the image to obtain pre-processed image data; The high-gain and low-gain images are fused into a high dynamic range image through image fusion, and the image signal-to-noise ratio and reading speed are enhanced by merging adjacent pixels; Through actual shooting tests, the dynamic range, image quality and processing speed of the synthesized high dynamic range images are verified.

2. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 1, characterized in that: The deployed sensor is an sCMOS image sensor with high and low gain channels that simultaneously outputs two images with different brightness levels.

3. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 2, characterized in that: The sCMOS image sensor is integrated with an A / D converter. Each column of pixels has a 12-bit high-speed A / D converter. A 32-channel LVDS high-speed serial transmitter sends high-speed image data out of the sCMOS image sensor. The top 16 channels and the bottom 16 channels are used to simultaneously transmit low-gain images and high-gain images, respectively.

4. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 3, characterized in that: The acquisition of dual-gain image data is controlled by the FPGA, and image data is read from both the high-gain and low-gain channels of the sensor.

5. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 4, characterized in that: When acquiring dual-gain image data, the logic controller is responsible for generating control signals for the image sensor and providing a reference clock for the image sensor interface for timing control and read and write control of the data stream.

6. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 5, characterized in that: The sensor with high and low gain channels is deployed on a high dynamic range cooled sCMOS camera. The sCMOS camera is integrated with an FPGA, a sensor unit, and a main signal unit to form a high dynamic range camera system for capturing detailed scenes in high contrast environments.

7. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 6, characterized in that: The sensor unit includes an FPGA processor, a DDR buffer, a Gigabit network port, a USB3.0 high-speed interface, a TEC cooling module, a heating module, and a power management module; the sensor unit and the main signal unit are connected via two sets of 51-core FPC flexible cables.

8. The real-time hardware high dynamic range synthesis method of dual-gain grayscale characteristics as claimed in claim 7, characterized in that: The maximum envelope of the sCMOS camera is 44 mm, the sensor unit is a circular circuit board with a diameter of 37.5 mm, and the main signal board is a slender strip circuit board with a width of 39.5 mm.

9. A camera, characterized in that: A real-time hardware high dynamic range synthesis method for executing the dual-gain grayscale characteristics of any one of claims 1 to 8, the camera comprising: The sensor unit adopts an sCMOS image sensor, has high and low gain channels, and is used to simultaneously output two images with different brightness levels; An electronics architecture, using a logic controller to generate control signals for the image sensor and provide a reference clock for the image sensor interface; The camera structure, including the sensor unit and the main signal unit, is connected via an FPC cable; The cooling system uses a double-layer TEC wind cooling shell integrated cooling technology to provide effective cooling for the sCMOS camera; A non-uniformity correction module performs non-uniformity correction on the high-gain channel to eliminate the response differences between sensor pixels; Dual-gain characteristic analysis module, which analyzes the dual-gain characteristics of sCMOS sensors; High dynamic range image synthesis module, used for image synthesis; The FPGA implementation module is used to program the image synthesis and image processing steps into the FPGA. By integrating the FPGA with the sensor unit and the main signal unit, a complete high dynamic range camera system is constructed and performance testing is performed.

Images