High-dynamic-range imaging method, and imaging system

Abstract

Provided in the present application are a high-dynamic-range imaging method, and an imaging system. The method comprises: collecting a plurality of first images of a photographed region; determining a target region on the basis of the plurality of first images, wherein the target region is a part of the photographed region, and the brightness of the target region reaches a threshold value; and collecting a second image of the photographed region, wherein when the second image is imaged, the light intensity of a light signal from the target region is weakened. The imaging system comprises an image sensor, a processing module, a modulation module and a lens group. By means of the method and the imaging system in the present application, single-frame high-dynamic-range imaging can be effectively performed.

Description

High dynamic range imaging method and imaging system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on November 27, 2023, with application number 202311614303.8 and invention name “Method and imaging system for high dynamic range imaging”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of machine vision, and more particularly, to a high dynamic range imaging method and imaging system. Background Art

[0003] High dynamic range (HDR) imaging is a technology used to capture moving scenes with a wide range of brightness. Due to the limited dynamic range of a camera's sensor, traditional imaging methods often cannot capture both bright and dark details simultaneously. HDR imaging, on the other hand, can capture a wider range of brightness in a single exposure, resulting in greater detail and richer colors.

[0004] When multi-frame exposure fusion is applied to moving scenes, motion blur and camera shake will make it difficult to align multiple images, resulting in artifacts in the high dynamic range reconstruction of the moving scene, reduced performance, and the inability to clearly image or identify moving objects.

[0005] When using neuromorphic cameras for HDR imaging of moving scenes, they need to be fused with regular digital images captured by traditional frame-based cameras. Due to factors such as camera lens distortion and the different fields of view of the two sensors, in some cases, neuromorphic cameras produce inferior imaging results compared to multi-frame fusion algorithms and are more expensive. Summary of the Invention

[0006] The present application provides a high dynamic range imaging method and an imaging system, which can effectively perform high dynamic range imaging.

[0007] In a first aspect, an embodiment of the present application provides a method for high dynamic range imaging, including: acquiring multiple first images of a shooting area; determining a target area based on the multiple first images, where the target area is a partial area in the shooting area, wherein the brightness of the target area reaches a threshold; acquiring a second image of the shooting area, wherein when the second image is imaged, the light signal from the target area is weakened in intensity.

[0008] The high dynamic range imaging method of the embodiment of the present application weakens the light intensity according to the target area, so that the single-frame image acquired in a single time has high clarity and moderate brightness, thereby effectively performing high dynamic range imaging, which is suitable for high-speed imaging scenarios of moving objects, etc.

[0009] In a possible implementation manner of the first aspect, the target area includes at least a portion of the moving object, and the brightness of at least the portion of the moving object reaches a threshold.

[0010] In a possible implementation of the first aspect, capturing a second image of the shooting area includes: determining an optical mask based on pixels corresponding to the shooting area and pixels corresponding to the target area; modulating a light signal from the shooting area according to the optical mask; and imaging the modulated light signal to obtain the second image.

[0011] An optical mask is used to modulate the light signal from the shooting area so that the imaging of the shooting area has appropriate brightness and the implementation complexity is low.

[0012] In a possible implementation of the first aspect, modulating the light signal from the shooting area according to the optical mask includes: controlling a spatial light modulator to modulate the light signal from the target area according to the optical mask to weaken the light intensity of the light signal from the target area.

[0013] In a possible implementation of the first aspect, the optical mask is a binary bitmap, and the values ​​of pixels of the optical mask represent whether the light intensity is weakened.

[0014] The optical mask in the form of a binary bitmap can keep the brightness ratio between pixels corresponding to the target area unchanged, which is convenient for subsequent processing.

[0015] In a possible implementation of the first aspect, the optical mask is a grayscale bitmap, and the values ​​of pixels of the optical mask represent the degree to which light intensity is weakened.

[0016] The optical mask in the form of a grayscale bitmap can make the pixels corresponding to the target area have relatively close brightness, which is convenient for subsequent processing.

[0017] In a possible implementation of the first aspect, a target area is determined based on multiple first images, including: predicting a moving object area corresponding to the moving object based on the multiple first images; and determining the target area based on an overexposed area in the moving object area, wherein the brightness of the overexposed area reaches a threshold.

[0018] In a possible implementation of the first aspect, predicting a moving object area corresponding to a moving object based on multiple first images includes: determining features of the moving object based on images of the moving object in the multiple first images; and determining the moving object area based on the features of the moving object.

[0019] In a possible implementation of the first aspect, the multiple first images are multiple frames of images captured continuously.

[0020] The multiple consecutive first images can more accurately reflect the motion trajectory of the moving object in the shooting area.

[0021] In a possible implementation of the first aspect, the second image is captured as a next frame after the capture of multiple first images is completed.

[0022] In this case, the region where the moving object is located has a high degree of overlap with the region of the moving object in the predicted image, and the obtained second image has high definition and moderate brightness.

[0023] In a possible implementation manner of the first aspect, the method further includes: performing image recognition based on the second image.

[0024] The second image obtained by the method of the embodiment of the present application has higher quality, and thus can improve the accuracy of image recognition.

[0025] In a second aspect, an embodiment of the present application provides an imaging system, comprising: an image sensor for capturing multiple first images of a shooting area; a processing module for determining a target area based on the multiple first images, where the target area is a partial area in the shooting area, wherein the brightness of the target area reaches a threshold; a modulation module for modulating a light signal from the shooting area to weaken the light intensity of the light signal from the target area; the image sensor is also used to capture a second image of the shooting area based on the modulated light signal.

[0026] In a possible implementation manner of the second aspect, the target area includes at least a portion of the moving object, and the brightness of at least a portion of the moving object reaches a threshold.

[0027] In a possible implementation manner of the second aspect, the modulation module includes at least one of a digital micromirror array, a liquid crystal light modulator, or an acousto-optic modulator.

[0028] In a possible implementation of the second aspect, the processing module is used to determine an optical mask based on pixels corresponding to the shooting area and pixels corresponding to the target area; and the modulation module is used to modulate the light signal from the shooting area according to the optical mask.

[0029] In a possible implementation of the second aspect, the optical mask is a binary bitmap, and the values ​​of pixels of the optical mask indicate whether the light intensity is weakened.

[0030] In a possible implementation of the second aspect, the optical mask is a grayscale bitmap, and the values ​​of pixels of the optical mask represent the degree to which light intensity is weakened.

[0031] In a possible implementation of the second aspect, the processing module is used to: predict a moving object area corresponding to the moving object based on multiple first images; determine a target area based on an overexposed area in the moving object area, wherein the brightness of the overexposed area reaches a threshold.

[0032] In a possible implementation of the second aspect, the processing module is configured to: determine features of the moving object based on images of the moving object in a plurality of first images; and determine a moving object region based on the features of the moving object.

[0033] In a possible implementation of the second aspect, the multiple first images are multiple frames of images captured continuously.

[0034] In a possible implementation of the second aspect, the second image is captured as a next frame after the capture of multiple first images is completed.

[0035] In a possible implementation of the second aspect, the imaging system further includes: a lens group, configured to receive a light signal from a shooting area, and transmit the light signal from the shooting area to the image sensor via a modulation module.

[0036] In a possible implementation manner of the second aspect, the processing module is further configured to perform image recognition based on the second image.

[0037] In a third aspect, an embodiment of the present application provides a camera, comprising an imaging system as implemented in any one of the second aspects.

[0038] In a fourth aspect, an embodiment of the present application provides a terminal comprising an imaging system according to any implementation of the second aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a vehicle comprising an imaging system according to any implementation of the second aspect.

[0040] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium comprising computer program instructions. When the computer program instructions are executed by an imaging system, the imaging system executes a method as described in any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a schematic block diagram of an imaging system according to an embodiment of the present application.

[0042] FIG2 is a schematic flow chart of a method for high dynamic range imaging according to an embodiment of the present application.

[0043] FIG3 is a schematic diagram of an imaging system according to an embodiment of the present application.

[0044] FIG4 is a schematic block diagram of an imaging system according to an embodiment of the present application.

[0045] FIG5 is a schematic block diagram of a controller according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] This application will present various aspects, embodiments, or features around systems including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. Furthermore, combinations of these aspects may also be used.

[0047] Additionally, in the embodiments of this application, words such as "exemplary" and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0048] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field will know that with the emergence of new hardware functional modules, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0049] The following is a brief introduction to the commonly used technical terms in this field.

[0050] PredNet is a deep learning-based neural network model for video sequence prediction and processing.

[0051] The Automatic License Plate Recognition Network (ALPR Net) is a deep learning network used to automatically identify and extract license plate information. It can accurately detect and identify license plate numbers from vehicle images through image processing and pattern recognition techniques.

[0052] In a typical multi-frame exposure fusion high dynamic range (HDR) imaging method, a camera or video camera is first used to capture a series of images with different exposure settings such as overexposure, normal exposure, and underexposure, so that these images have different brightness settings to capture details in different brightness areas; then the images are aligned; followed by exposure fusion, using common image processing algorithms to extract the best brightness details in each image; finally, tone mapping is performed, and adjustments are made according to different tone mapping algorithms to maintain image details and contrast.

[0053] When the traditional multi-frame exposure fusion HDR imaging method is applied to motion scenes, artifacts, blurring and other phenomena are more serious, resulting in limited application.

[0054] In view of this, an embodiment of the present application provides a method for high dynamic range imaging, wherein a single-frame target digital image obtained by the method has a high dynamic range, high clarity and moderate brightness.

[0055] In one embodiment, the above method is implemented by an imaging system. In the imaging system 100 shown in Figure 1, the imaging system 100 may include a lens group 110, a spatial light modulator (SLM) 120 and an image sensor 130. The working process of the imaging system 100 is as follows: first, the lens group 110 collects light signals from the shooting area multiple times, and then the spatial light modulator 120 modulates the light signals, so that the modulated light signals are converted into digital images through the image sensor 130, and then the digital images are processed. The data processing can input optimized modulation parameters to the spatial light modulator 120 to improve the imaging effect of the image sensor 130. The imaging system 100 can also determine the output image based on the subsequently captured digital images. The image sensor 130 can be a complementary metal-oxide-semiconductor (CMOS) camera, a charge-coupled device (CCD) camera or other types of image sensors, and the embodiments of the present application are not limited to this.

[0056] The spatial light modulator 120 is an optical device used to modulate the phase, amplitude, or polarization state of a light wave, thereby achieving precise control of the light field. It is typically composed of an array of adjustable optical elements, each element corresponding to a pixel of the light wave. The state of these elements can be adjusted as needed, thereby changing the properties of the light wave. For example, the spatial light modulator 120 can use a digital micromirror array (DMD). A DMD is composed of many tiny mirrors, each of which is called a micromirror. These micromirrors are arranged in a two-dimensional array, and each micromirror is typically a few microns in size. A light source is irradiated onto the DMD. Each micromirror can independently switch between two states: typically, reflection toward the optical axis and reflection away from the optical axis. These two states correspond to the digital signal 0 and 1, respectively. A digital control circuit is responsible for controlling the state of each micromirror. This control process is high-speed and precise, typically completed in microseconds. The DMD can perform phase modulation. When the micromirrors are adjusted to reflect toward the optical axis, the phase of the light wave changes. When the micromirrors reflect away from the optical axis, the phase remains unchanged. By forming a specific reflection pattern on the micromirror array, the phase of the light wave can be modulated. Amplitude modulation can also be achieved by similarly adjusting the reflection angle of the micromirrors. It can also perform light wave modulation, combining phase and amplitude modulation to modulate the input light wave into an output light wave with specific spatial characteristics. The reflection angles of the two micromirrors, the positional relationship between the DMD and the optical axis, and other factors can be set as needed.

[0057] Another common spatial light modulator is the liquid crystal light modulator (LCSM). Liquid crystal is a material that can change the polarization state of light by adjusting its molecular orientation. An LCSM typically consists of a liquid crystal layer, transparent electrodes, and control circuitry. The control circuitry controls the state of each pixel. These circuits can rapidly change the state of the liquid crystals as needed, achieving real-time modulation of the light field. LCSMs can perform phase modulation, precisely adjusting the phase of light waves as they pass through the LC layer. This can be achieved by changing the orientation of the LC molecules or by introducing an electric field into the LC layer. They can also perform amplitude modulation, modulating the transparency of the LC to change the amplitude of the light wave. This is typically achieved by adjusting the orientation of the LC molecules to alter the degree of light transmission. They can also perform polarization modulation, changing the polarization of light by manipulating the orientation of the LC molecules.

[0058] Alternatively, an acousto-optic modulator can be used to change the refractive index of the medium by introducing sound waves into the optical path, thereby affecting the amplitude of the light.

[0059] In addition, there are other types of spatial light modulators. In any type of spatial light modulator, an adjustable optical filter can be introduced to achieve amplitude modulation of the light wave by adjusting the intensity of the transmitted light. This application does not limit the type of spatial light modulator.

[0060] The modulation parameters shown in Figure 1 can be those of various modulation methods, including phase modulation, amplitude modulation, polarization modulation, and wavelength modulation. For example, in a liquid crystal spatial light modulator (LCSLM), applying an electric field can adjust the orientation of the LC molecules, thereby changing the phase of the light wave as it passes through the LC layer. The magnitude and form of the voltage can serve as control parameters; increasing the voltage can cause the LC molecules to realign, shifting the phase. The amplitude of the light wave can also be adjusted by varying the electric field; the magnitude and form of the voltage can also serve as control parameters. For a DMD, the input phase modulation parameter is typically a digital pattern, where the values ​​of the pattern correspond to different phase or amplitude states. By controlling the state of each micromirror, corresponding phase modulation can be introduced into the output light wave. Controlling the state of each micromirror can also adjust the intensity of the reflected light, achieving digital amplitude modulation. Both the electric field and the digital pattern described above can be referred to as optical masks. Unless otherwise specified, the digital pattern will be referred to as the optical mask below.

[0061] The technical solutions of the embodiments of this application can be applied to various complex lighting scenarios, such as nighttime license plate recognition, military reconnaissance, underground search and rescue, and mineral exploration. The following description uses imaging methods for an imaging system in the field of traffic monitoring, but this application is not limited to this method. This imaging system in the field of traffic monitoring is used to identify license plate information of moving vehicles under strong headlights at night.

[0062] Figure 2 is a schematic flow chart of a method 200 for high dynamic range imaging according to an embodiment of the present application. The following embodiments all use the method for high dynamic range imaging of moving objects as an example, but the method is also effective for high dynamic range imaging of stationary objects, and this embodiment is not limited thereto.

[0063] 210 , capturing multiple first images of the shooting area.

[0064] When there are moving objects within the capture area, the moving objects can be identified by capturing multiple images. For example, when an imaging system in the field of traffic monitoring performs license plate recognition on a vehicle within a fixed capture area, the vehicle is in motion within the capture area. The imaging system captures multiple optical image signals of the capture area, each of which is converted into a digital image by an image sensor. These multiple digital images are multiple first images.

[0065] Optionally, the multiple first images may be multiple frames captured continuously. The multiple consecutive first images can more accurately reflect the motion trajectory of the moving object in the captured area. In one possible embodiment, the operating frequency of each component in the imaging system is F, the interval between any two moments is 1 / F seconds, and N+1 images are captured continuously from any time T to time T+N as the multiple first images.

[0066] 220 , determining a target area based on the multiple first images, where the target area is a partial area in the shooting area, wherein the brightness of the target area reaches a threshold.

[0067] Taking the method of high dynamic range imaging of a moving object as an example, the target area includes at least a part of the moving object and the brightness of at least a part of the moving object reaches a threshold. The specific threshold depends on the image sensor of the imaging system, and this application does not limit this. The analog electrical signal collected is converted into a digital image represented by brightness through an analog-to-digital conversion circuit. For an 8-bit digital image, the value range of each pixel is 0 to 255. If the pixel value is 255, it means that the light signal intensity of the corresponding area has reached saturation. Therefore, the threshold of the brightness of the target area can be 255, or it can be 245, 250, etc. Similarly, for a 12-bit digital image, the threshold of the brightness of the target area can be 4095, or 4085, 4090, etc., and this embodiment does not limit this.

[0068] First, based on the multiple first images, the moving object area corresponding to the moving object is predicted. Specifically, a video prediction algorithm can be executed. For example, a pre-trained PredNet neural network model can be used, which can recognize the edges, shapes, colors, motion states and other features of objects in the image, and predict the next frame of image based on the object features of multiple frames of images. In this embodiment, the characteristics of the moving object can be determined based on the images of the moving object in the multiple first images. For example, the objects can be traffic signs, roadside trees, moving vehicles, etc., and the moving objects mainly include moving vehicles. The images of the moving vehicles in the multiple first images can be analyzed to obtain characteristics such as the motion state of the moving vehicle.

[0069] It should be noted that only a portion of a moving vehicle may be within the imaging system's capture area. This portion of the vehicle within the imaging system's capture area can be further divided into a portion with a brightness that reaches a threshold and a portion with a brightness that is less than the threshold. In this case, the brightness of at least a portion of the moving object reaches the threshold.

[0070] Furthermore, the moving object region is determined based on the characteristics of the moving object. For example, a predicted image can be calculated based on the characteristics of the moving object. In this embodiment, the predicted image is calculated based on characteristics such as the motion state of the moving vehicle. The predicted image includes an image of the moving object, and the region corresponding to the image of the moving object is referred to as the moving object region.

[0071] In this embodiment, the target area can be determined based on the overexposed area in the moving object area. Specifically, the area corresponding to the pixels whose brightness reaches the threshold in the predicted image is called the overexposed area. For example, the target area can be the rear of a vehicle illuminated by strong light from the headlights. In some possible cases, the overexposed area is the part of the rear of the vehicle excluding the rear windshield, that is, the overexposed area is part of the moving object area; in other possible cases, some traffic signs in the shooting area are also illuminated by the headlights, and the overexposed area includes the area where the traffic signs are located. In this case, the overexposed area in the moving object area refers to the intersection of the overexposed area and the moving object area.

[0072] Furthermore, in some image processing methods, to facilitate subsequent processing, the pixels corresponding to overexposed areas, moving object areas, and target areas are expanded into a rectangular pixel array to which these pixels belong. Taking the overexposed area as an example, the overexposed area should be determined by this rectangular pixel array, and the actual three-dimensional shape of the overexposed area is not fixed. This embodiment does not impose any restrictions on the size and shape of the overexposed area, moving object area, and target area.

[0073] 230 , capturing a second image of the shooting area, wherein when the second image is formed, the light signal from the target area is weakened.

[0074] Specifically, an optical mask can be determined based on the pixels corresponding to the captured area and the pixels corresponding to the target area; the optical signal from the captured area can be modulated based on the optical mask; and the modulated optical signal can be used to generate a second image. Taking the imaging system 300 shown in FIG3 as an example, the imaging system 300 can be used to implement the methods described in 210 and 220, and can also implement the method described in 230 in the following manner.

[0075] The optical mask is used to ensure the image of the captured area has appropriate brightness. It can be loaded onto the digital micromirror array 320 to modulate the optical signal. For example, in this embodiment, the optical mask can be used to control the state of each micromirror in the digital micromirror array 320, adjusting the intensity of the reflected light to achieve digital amplitude modulation and reduce the intensity of the optical signal in the target area. This approach has low implementation complexity and manageable costs.

[0076] In one embodiment, each micromirror of the digital micromirror array 320 has two states and can be regarded as a binary bitmap. Therefore, when the optical mask is a binary bitmap, the arrangement of the pixels of the optical mask can be the same as the arrangement of the micromirrors of the digital micromirror array 320, where the value of the pixel of the optical mask indicates whether the light intensity at that position is reduced.

[0077] For example, the arrangement of the micromirrors of the digital micromirror array 320 is 30720×17280, the number of pixels of the optical mask is 30720×17280, and each pixel has a value of 0 or 1. A pixel value of 1 in the optical mask indicates that the micromirror reflects away from the optical axis, reducing the light intensity at that location. A pixel value of 0 indicates that the micromirror reflects toward the optical axis, not reducing the light intensity at that location. The specific degree of light intensity reduction depends on the specific positional relationship between the optical axis and the digital micromirror array 320, which is not limited in this embodiment. In one embodiment, the pixel corresponding to the target area in the optical mask has a value of 1, and the remaining pixels have a value of 0.

[0078] The optical mask in the form of a binary bitmap can keep the brightness ratio between pixels corresponding to the target area unchanged, which is convenient for subsequent processing.

[0079] For other types of spatial light modulators, tunable optical filters can be introduced to achieve amplitude modulation of light waves by adjusting the intensity of the transmitted light. For example, a liquid crystal optical filter can be used as a tunable optical filter. The molecular orientation of the liquid crystal in the liquid crystal optical filter can be controlled by an external voltage or electric field to adjust the transmitted spectral range or transmittance. The intensity distribution of the external electric field can be set based on the ratio between the brightness of the pixels corresponding to the target area and the threshold.

[0080] In another embodiment, the optical mask can also be a grayscale bitmap. Specifically, each micromirror on the digital micromirror array 320 can be divided into several small regions, with the arrangement of these small regions being identical to the arrangement of pixels in the resulting digital image. Grayscale modulation of each pixel can then be achieved by individually controlling the reflective state of these small regions. In this case, the multiple micromirrors of the digital micromirror array 320 form a microarray, corresponding to a pixel in the resulting digital image. The required arrangement of the pixels in the optical mask is identical to the arrangement of the pixels in the digital image obtained by imaging with a CMOS camera. The value of a pixel in the optical mask represents the degree to which the light intensity at that location is reduced.

[0081] For example, the digital image captured by the CMOS camera in Figure 3 has a pixel count of 1920×1080, and the micromirrors of the digital micromirror array 320 are arranged in a 30720×17280 array. The micromirrors of the digital micromirror array 320 can be grouped in 16×16 groups to form a microarray. The optical mask has 1920×1080 pixels, and each pixel has a value range of 0 to 255, with each pixel corresponding to a microarray. A pixel value of 0 indicates that all micromirrors in the microarray reflect toward the optical axis, without reducing the light intensity at that pixel. A pixel value of 127 indicates that 127 of the 256 micromirrors in the corresponding microarray reflect away from the optical axis. A pixel value of 255 indicates that 255 of the 256 micromirrors in the corresponding microarray reflect away from the optical axis, with only one micromirror reflecting toward the optical axis. The pixel values ​​of the optical mask can be set based on the ratio between the brightness of the pixel corresponding to the target area and a threshold.

[0082] The optical mask in the form of a grayscale bitmap can make the pixels corresponding to the target area have relatively close brightness, which is convenient for subsequent processing.

[0083] Alternatively, the second image can be captured in the next frame after the multiple first images are captured. In this case, the region where the moving object resides has a high degree of overlap with the region of the moving object in the predicted image, resulting in a second image with high clarity and moderate brightness. In one possible embodiment, the operating frequency of each component in imaging system 300 is F, the interval between any two moments is 1 / F seconds, and the multiple first images are N+1 images captured continuously from any time T to time T+N. In this embodiment, an image is captured at time T+N+1 as the second image.

[0084] In this embodiment, the second image can be input into the license plate recognition network, and image recognition can be performed based on the second image to obtain the license plate number in the second image. This application does not limit the use of the second image. The second image obtained by the method of the embodiment of the application has higher quality, thereby improving the accuracy of image recognition.

[0085] The high dynamic range imaging method of the embodiment of the present application weakens the light intensity according to the target area, so that the single-frame image acquired in a single time has high clarity and moderate brightness, thereby effectively performing high dynamic range imaging, which is suitable for high-speed imaging scenarios of moving objects, etc.

[0086] The present application also provides an imaging system that can execute the method of the aforementioned embodiment of the present application. FIG4 is a schematic diagram of the structure of an imaging system 600 provided in one embodiment of the present application. The imaging system 600 includes:

[0087] The image sensor 630 is used to capture multiple first images of the shooting area; and is also used to capture a second image of the shooting area according to the modulated light signal.

[0088] The processing module 640 is used to determine a target area based on multiple first images, where the target area is a portion of the shooting area, wherein the brightness of the target area reaches a threshold; and is also used to determine an optical mask based on pixels corresponding to the shooting area and pixels corresponding to the target area.

[0089] The modulation module 620 is used to modulate the light signal from the shooting area to reduce the light intensity of the light signal from the target area.

[0090] The imaging system 600 may further include a lens group 610 for receiving light signals from the shooting area and transmitting the light signals to the image sensor 630 via the modulation module 620 .

[0091] The lens group 610, modulation module 620, and image sensor 630 may be the lens group 110, spatial light modulator 120, and image sensor 130 in FIG1 , respectively, or the lens group 310, digital micromirror array 320, and CMOS camera 330 in FIG3 . The modulation module 620 may also be a liquid crystal spatial light modulator, a grating modulator, an acousto-optic modulator, etc., and the image sensor 630 may also be a CCD camera.

[0092] In an application scenario of this embodiment, the processing module 640 can also be used to perform image recognition based on the second image. The second image obtained by the method of the embodiment of the present application has high quality, thereby improving the accuracy of image recognition.

[0093] The specific implementation of the functions performed by each module has been described in the method embodiment and will not be repeated here.

[0094] The term "module" can be implemented in software and / or hardware, and is not specifically limited to this. For example, a "module" can be a software program, a hardware circuit, or a combination of the two that implements the above-mentioned functions, and can include code running on a computing instance. Exemplarily, the processing module can be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The modulation module can be a programmable hardware functional module such as a digital micromirror array, and the image sensor can be a programmable hardware functional module such as a CCD camera or a CMOS camera.

[0095] Therefore, the modules of each example described in the embodiments of this application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] The present application also provides a controller 700. As shown in Figure 5, the controller 700 includes: a processor 704 and a communication interface 708. Furthermore, the controller 700 may also include a bus 702 and a memory 706. It should be understood that the bus 702 and the memory 706 are optional. The processor 704, the memory 706 and the communication interface 708 communicate with each other via the bus 702. Exemplarily, the controller 700 can be a computing device or a system in a computing device for implementing the method of the embodiment of the present application. The controller 700 can be various terminal devices. It should be understood that the present application does not limit the number of processors and memories in the controller 700.

[0097] Bus 702 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG5 shows a single line, but this does not imply a single bus or type of bus. Bus 704 may include a path for transmitting information between various components of controller 700 (e.g., memory 706, processor 704, and communication interface 708).

[0098] The processor 704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0099] The memory 706 may include volatile memory, such as random access memory (RAM), or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0100] The memory 706 stores executable program codes, and the processor 704 executes the executable program codes to implement the functions of the aforementioned processing modules, thereby generating a high dynamic range image. In other words, the memory 706 stores instructions for a method for high dynamic range imaging.

[0101] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the controller 700 and other devices or communication networks (such as user terminals of intelligent agents). The communication interface can also be called an interface circuit.

[0102] In one possible implementation, the controller 700 is used to implement a hardware functional module in an intelligent agent.

[0103] An embodiment of the present application further provides a camera, which includes the hardware functional modules of the above-mentioned imaging system and is capable of executing the above-mentioned high dynamic range imaging method. The selection of specific hardware functional modules has been illustrated in the above-mentioned embodiment.

[0104] The present application also provides a terminal comprising the hardware functional modules of the imaging system described above, capable of executing the high dynamic range imaging method described above. The terminal may be a smartphone, smart tablet, IoT device, industrial control device, or other device with a camera function.

[0105] The present application also provides a vehicle comprising the hardware functional modules of the imaging system described above, capable of executing the high dynamic range imaging method described above. The vehicle may be a vehicle with assisted driving functions, an unmanned vehicle, or other vehicle capable of interacting with electronic devices.

[0106] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method of the present application.

[0107] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the method in the embodiment of the present application, or instruct the computing device to execute the method in the embodiment of the present application.

[0108] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0109] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0112] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for high dynamic range imaging, characterized in that: include: Acquiring a plurality of first images of a shooting area; Determine a target area according to the plurality of first images, the target area being a partial area in the shooting area, wherein the brightness of the target area reaches a threshold; A second image of the shooting area is acquired, wherein when the second image is formed, the light signal from the target area is weakened in intensity.

2. The method according to claim 1, characterized in that The target area includes at least a part of a moving object, and the brightness of at least a part of the moving object reaches the threshold.

3. The method according to claim 1 or 2, characterized in that: The acquiring the second image of the shooting area includes: Determining an optical mask according to pixels corresponding to the shooting area and pixels corresponding to the target area; modulating a light signal from the photographing area according to the optical mask; The second image is obtained by imaging the modulated light signal.

4. The method according to claim 3, characterized in that: The step of modulating the optical signal from the shooting area according to the optical mask comprises: The spatial light modulator is controlled to modulate the light signal of the target area according to the optical mask, so as to reduce the light intensity of the light signal from the target area.

5. The method according to claim 3 or 4, characterized in that: The optical mask is a binary bitmap, and the value of a pixel of the optical mask indicates whether the light intensity is weakened.

6. The method according to claim 3 or 4, characterized in that: The optical mask is a grayscale bitmap, and the values ​​of the pixels of the optical mask represent the degree to which the light intensity is weakened.

7. The method according to any one of claims 2 to 6, characterized in that The determining the target area according to the plurality of first images comprises: predicting a moving object region corresponding to the moving object according to the plurality of first images; The target area is determined according to an overexposed area in the moving object area, wherein the brightness of the overexposed area reaches the threshold.

8. The method according to claim 7, characterized in that The predicting, based on the plurality of first images, a moving object region corresponding to the moving object comprises: determining a feature of the moving object according to the image of the moving object in the plurality of first images; The moving object region is determined according to the characteristics of the moving object.

9. The method according to any one of claims 1 to 8, characterized in that The multiple first images are multiple frames of images captured continuously.

10. The method according to any one of claims 1 to 9, characterized in that The second image is acquired in the next frame after the acquisition of the plurality of first images is completed.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Image recognition is performed based on the second image.

12. An imaging system, characterized in that: include: An image sensor, used for acquiring a plurality of first images of a shooting area; A processing module, configured to determine a target area according to the plurality of first images, wherein the target area is a partial area in the shooting area, wherein the brightness of the target area reaches a threshold; A modulation module, used for modulating the light signal from the shooting area to weaken the light intensity of the light signal from the target area; The image sensor is further configured to capture a second image of the shooting area according to the modulated light signal.

13. The imaging system according to claim 12, characterized in that: The target area includes at least a part of a moving object, and the brightness of at least a part of the moving object reaches the threshold.

14. The imaging system according to claim 12 or 13, characterized in that: The modulation module includes at least one of a digital micromirror array, a liquid crystal light modulator or an acousto-optic modulator.

15. The imaging system according to any one of claims 12 to 14, characterized in that: The processing module is used to determine an optical mask according to pixels corresponding to the shooting area and pixels corresponding to the target area; The modulation module is used to modulate the light signal from the shooting area according to the optical mask.

16. The imaging system according to claim 15, characterized in that The optical mask is a binary bitmap, and the value of a pixel of the optical mask indicates whether the light intensity is weakened.

17. The imaging system according to claim 15, characterized in that The optical mask is a grayscale bitmap, and the values ​​of the pixels of the optical mask represent the degree to which the light intensity is weakened.

18. The imaging system according to any one of claims 13 to 17, characterized in that The processing module is used for: predicting a moving object region corresponding to the moving object according to the plurality of first images; The target area is determined according to an overexposed area in the moving object area, wherein the brightness of the overexposed area reaches the threshold.

19. The imaging system according to claim 18, characterized in that The processing module is used for: determining a feature of the moving object according to the image of the moving object in the plurality of first images; The moving object region is determined according to the characteristics of the moving object.

20. The imaging system according to any one of claims 12 to 19, characterized in that The multiple first images are multiple frames of images captured continuously.

21. The imaging system according to any one of claims 12 to 20, characterized in that The second image is acquired in the next frame after the acquisition of the plurality of first images is completed.

22. The imaging system according to any one of claims 12 to 21, characterized in that Also includes: The lens group is used to receive the light signal from the shooting area and transmit the light signal from the shooting area to the image sensor via the modulation module.

23. The imaging system according to any one of claims 12 to 22, characterized in that The processing module is further configured to perform image recognition according to the second image.

24. A camera, characterized in that: Comprising an imaging system as claimed in any one of claims 12 to 23.

25. A terminal, characterized in that: Comprising an imaging system as claimed in any one of claims 12 to 23.

26. A vehicle, characterized in that: Comprising an imaging system as claimed in any one of claims 12 to 23.

Images