Method and system for high dynamic range imaging

Abstract

The invention provides a high dynamic range imaging method and an imaging system. The method comprises the following steps: collecting a plurality of first images of a shooting area; a target area is determined according to the multiple first images, the target area is a partial area in the shooting area, and the brightness of the target area reaches a threshold value; a second image of the shooting area is collected, and when the second image is imaged, the light intensity of the light signal from the target area is weakened. The imaging system comprises an image sensor, a processing module, a modulation module and a lens group. According to the method and the imaging system, single-frame high-dynamic-range imaging can be effectively carried out.

Description

Technical Field

[0001] This application relates to the field of machine vision, and more specifically, to a method and an imaging system for high dynamic range imaging. Background Art

[0002] High Dynamic Range (HDR) imaging is a technique for capturing moving scenes with a wide range of brightness. Due to the limited dynamic range of the camera's photosensitive element, traditional imaging methods often cannot capture both bright and dark details simultaneously. HDR imaging can present more details and richer colors by capturing a wider range of brightness in a single exposure.

[0003] When multi-frame exposure fusion is applied to a moving scene, motion blur and camera shake will make it difficult to align multiple images, resulting in artifacts in the high dynamic range reconstruction of the moving scene, degraded performance, and inability to clearly image or identify moving objects.

[0004] When a moving scene uses a neuromorphic camera for HDR imaging, it needs to be fused with ordinary digital images captured by a traditional frame-based camera. Due to reasons such as camera lens distortion and different fields of view of the two sensors, in some cases, the neuromorphic camera has a worse imaging effect and higher cost than the multi-frame fusion algorithm. Summary of the Invention

[0005] This application provides a method and an imaging system for high dynamic range imaging, which can effectively perform high dynamic range imaging.

[0006] In a first aspect, an embodiment of this application provides a method for high dynamic range imaging, including: collecting multiple first images of a shooting area; determining a target area according to the multiple first images, where the target area is a partial area in the shooting area, and the brightness of the target area reaches a threshold; collecting a second image of the shooting area, where when the second image is taken, the optical signal from the target area is attenuated in light intensity.

[0007] The method for high dynamic range imaging according to the embodiment of this application can attenuate the light intensity according to the target area, so that a single-frame image collected once has high clarity and moderate brightness, thereby being able to effectively perform high dynamic range imaging and being applicable to high-speed imaging scenarios of moving objects, etc.

[0008] In a possible implementation manner of the first aspect, the target area includes at least a part of a moving object, and the brightness of at least a part of the moving object reaches the threshold.

[0009] In a possible implementation of the first aspect, acquiring a second image of the shooting area includes: determining an optical mask according to the pixels corresponding to the shooting area and the pixels corresponding to the target area; modulating the optical signal from the shooting area according to the optical mask; and imaging the modulated optical signal to obtain the second image.

[0010] Using an optical mask to modulate the optical signal from the shooting area makes the imaging of the shooting area have appropriate brightness and has low implementation complexity.

[0011] In a possible implementation of the first aspect, modulating the optical signal from the shooting area according to the optical mask includes: controlling a spatial light modulator according to the optical mask to modulate the optical signal of the target area so as to attenuate the optical intensity of the optical signal from the target area.

[0012] In a possible implementation of the first aspect, the optical mask is a binary bitmap, and the value of the pixels of the optical mask represents whether the optical intensity is attenuated.

[0013] The optical mask in the form of a binary bitmap can keep the brightness ratio between the pixels corresponding to the target area unchanged, which is convenient for subsequent processing.

[0014] In a possible implementation of the first aspect, the optical mask is a grayscale bitmap, and the value of the pixels of the optical mask represents the degree of attenuation of the optical intensity.

[0015] The optical mask in the form of a grayscale bitmap can make the pixels corresponding to the target area have relatively close brightness, which is convenient for subsequent processing.

[0016] In a possible implementation of the first aspect, determining the target area according to multiple first images includes: predicting the moving object area corresponding to the moving object according to multiple first images; and determining the target area according to the overexposed area in the moving object area, where the brightness of the overexposed area reaches a threshold.

[0017] In a possible implementation of the first aspect, predicting the moving object area corresponding to the moving object according to multiple first images includes: determining the characteristics of the moving object according to the images of the moving object in multiple first images; and determining the moving object area according to the characteristics of the moving object.

[0018] In a possible implementation of the first aspect, the multiple first images are multiple consecutive frames of images collected.

[0019] Multiple consecutive first images can more accurately reflect the movement trajectory of the moving object in the shooting area.

[0020] In a possible implementation of the first aspect, the second image is collected in the next frame after the collection of multiple first images is completed.

[0021] In this case, the overlap degree between the region where the moving object is located and the region of the moving object in the predicted image is high, and the obtained second image has high clarity and moderate brightness.

[0022] In a possible implementation manner of the first aspect, the method further includes: performing image recognition according to the second image.

[0023] The second image obtained by the method of the embodiments of the present application has high quality, so the accuracy of image recognition can be improved.

[0024] In a second aspect, an imaging system provided by the embodiments of the present application includes: an image sensor for collecting multiple first images of a shooting area; a processing module for determining a target area according to the multiple first images, where the target area is a partial area in the shooting area, and the brightness of the target area reaches a threshold; a modulation module for modulating an optical signal from the shooting area to attenuate the optical intensity of the optical signal from the target area; and the image sensor is further used for collecting a second image of the shooting area according to the modulated optical signal.

[0025] In a possible implementation manner of the second aspect, the target area includes at least a part of the moving object, and the brightness of at least a part of the moving object reaches the threshold.

[0026] In a possible implementation manner of the second aspect, the modulation module includes at least one of a digital micromirror array, a liquid crystal light modulator, or an acousto-optic modulator.

[0027] In a possible implementation manner of the second aspect, the processing module is used for determining an optical mask according to the pixels corresponding to the shooting area and the pixels corresponding to the target area; and the modulation module is used for modulating the optical signal from the shooting area according to the optical mask.

[0028] In a possible implementation manner of the second aspect, the optical mask is a binary bitmap, and the value of the pixel of the optical mask represents whether the optical intensity is attenuated.

[0029] In a possible implementation manner of the second aspect, the optical mask is a grayscale bitmap, and the value of the pixel of the optical mask represents the degree of attenuation of the optical intensity.

[0030] In a possible implementation manner of the second aspect, the processing module is used for: predicting a moving object area corresponding to the moving object according to the multiple first images; and determining the target area according to the overexposed area in the moving object area, where the brightness of the overexposed area reaches the threshold.

[0031] In a possible implementation manner of the second aspect, the processing module is used for: determining the features of the moving object according to the images of the moving object in the multiple first images; and determining the moving object area according to the features of the moving object.

[0032] In a possible implementation of the second aspect, the multiple first images are multiple consecutive frames of images acquired.

[0033] In a possible implementation of the second aspect, the second image is acquired in the next frame after the acquisition of the multiple first images is completed.

[0034] In a possible implementation of the second aspect, the imaging system further includes: a lens group, configured to receive optical signals from the shooting area and transmit the optical signals from the shooting area to the image sensor via a modulation module.

[0035] In a possible implementation of the second aspect, the processing module is further configured to perform image recognition based on the second image.

[0036] In a third aspect, an embodiment of the present application provides a camera, including the imaging system according to any one of the implementations of the second aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a terminal, including the imaging system according to any one of the implementations of the second aspect.

[0038] In a fifth aspect, an embodiment of the present application provides a vehicle, including the imaging system according to any one of the implementations of the second aspect.

[0039] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by the imaging system, the imaging system executes the method according to any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic block diagram of an imaging system according to an embodiment of the present application.

[0041] Figure 2 is a schematic flowchart of a method for high dynamic range imaging according to an embodiment of the present application.

[0042] Figure 3 is a schematic diagram of an imaging system according to an embodiment of the present application.

[0043] Figure 4 is a schematic block diagram of an imaging system according to an embodiment of the present application.

[0044] Figure 5 is a schematic block diagram of a controller according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] Aspects, embodiments or features of the present application will be presented in the context of a system including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all the devices, components, modules, etc. discussed in connection with the figures. In addition, combinations of these solutions may also be used.

[0046] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" in the present application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of the word "exemplary" is intended to present concepts in a specific manner.

[0047] The business scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art will know that with the emergence of new hardware functional modules, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0048] A brief introduction to commonly used technical terms in this field is given below.

[0049] PredNet is a neural network model based on deep learning, used for video sequence prediction and processing.

[0050] The Automatic License Plate Recognition Network (ALPR Net) is a deep learning network for automatically recognizing and extracting license plate information. It can accurately detect and recognize license plate numbers from vehicle images through image processing and pattern recognition techniques.

[0051] In a typical multi-frame exposure fusion HDR imaging method, first, a series of images are captured using a camera or a video camera with different exposure settings such as overexposure, normal exposure, and underexposure, so that these images have differences in brightness to capture details in different brightness regions; then image alignment is performed; then exposure fusion is carried out, and common image processing algorithms are used to extract the best brightness details in each image; finally, tone mapping is performed and adjusted according to different tone mapping algorithms to maintain the details and contrast of the image.

[0052] When traditional multi-frame exposure fusion HDR imaging methods are applied to motion scenes, phenomena such as artifacts and blurring are relatively serious, resulting in limited applications.

[0053] In view of this, the embodiments of the present application provide a method for high dynamic range imaging. The single-frame target digital image obtained by this method has a high dynamic range, relatively high clarity, and moderate brightness.

[0054] In one embodiment, the above method is implemented by an imaging system. In the imaging system 100 as shown in Figure 1 Figure 100, the imaging system 100 may include a lens group 110, a Spatial Light Modulator (SLM) 120, and an image sensor 130. The working process of the imaging system 100 is as follows: First, the lens group 110 collects optical signals from the shooting area multiple times, and then the spatial light modulator 120 modulates the optical signals, so that the modulated optical signals are converted into digital images by the image sensor 130. Then, data processing is performed on the digital images. The data processing can input optimized modulation parameters to the spatial light modulator 120 to improve the imaging effect of the image sensor 130. The imaging system 100 can also determine the output image according to the digital images captured subsequently. The image sensor 130 can be a Complementary Metal-Oxide-Semiconductor (CMOS) camera, a Charge-Coupled Device (CCD) camera, or other types of image sensors, and the embodiments of the present application do not limit this.

[0055] The spatial light modulator 120 is an optical device used to modulate the phase, amplitude, or polarization state of light waves, so as to achieve precise control of the light field. It is usually composed of an adjustable optical element array, and each element corresponds to a pixel of the light wave. The states of these elements can be adjusted according to requirements, thereby changing the properties of the light wave. For example, the spatial light modulator 120 can adopt a Digital Micromirror Device (DMD). The DMD consists of many tiny mirrors, and each mirror is called a micromirror. These micromirrors are arranged in a two-dimensional array, and the size of each micromirror is usually about a few micrometers. The light source irradiates on the DMD. Each micromirror can independently switch between two states, usually reflecting towards the optical axis and reflecting away from the optical axis. These two states correspond to 0 and 1 of the digital signal respectively. The digital control circuit is responsible for controlling the state of each micromirror, and this control process is fast and precise, usually completed at the microsecond level. The DMD can perform phase modulation. When the micromirror is adjusted to reflect towards the optical axis, the phase of the light wave changes; when the micromirror reflects away from the optical axis, the phase remains unchanged. By forming a specific reflection pattern on the micromirror array, modulation of the phase of the light wave can be achieved; amplitude modulation can also be achieved by adjusting the reflection angle of the micromirror in a similar way; optical wave modulation can also be performed, comprehensively considering phase and amplitude modulation, and modulating the input optical wave into an output optical wave with specific spatial characteristics. Among them, the reflection angles of the two states of the micromirror, the positional relationship between the DMD and the optical axis, etc. can all be set according to needs.

[0056] Another common spatial light modulator is a liquid crystal light modulator. Liquid crystal is a material that can change the polarization state of light by adjusting the orientation of its molecules. A liquid crystal light modulator usually consists of a liquid crystal layer, transparent electrodes, and a control circuit. The control circuit is used to control the state of each pixel. These circuits can change the state of the liquid crystal at a very high speed as needed to achieve real-time modulation of the light field. A liquid crystal light modulator can perform phase modulation. By changing the phase of the light wave when it passes through the liquid crystal layer, precise adjustment of the phase of the light wave can be achieved. This can be realized by changing the orientation of the liquid crystal molecules or by introducing an electric field into the liquid crystal layer; it can also perform amplitude modulation by modulating the transparency of the liquid crystal, thereby changing the amplitude of the light wave, which is usually achieved by adjusting the orientation of the liquid crystal molecules to change the light transmittance; it can also perform polarization modulation by controlling the orientation of the liquid crystal molecules to change the polarization direction of the light.

[0057] In addition, an acousto-optic modulator can also be used, which changes the refractive index of the medium by introducing sound waves into the optical path, thereby affecting the amplitude of the light.

[0058] In addition, there are other types of spatial light modulators. In any type of spatial light modulator, a tunable optical filter can be introduced to achieve amplitude modulation of the light wave by adjusting the intensity of the transmitted light. The type of spatial light modulator is not limited in this application.

[0059] Figure 1 The modulation parameters shown can be modulation parameters of various modulation methods such as phase modulation, amplitude modulation, polarization modulation, wavelength modulation, etc. For example, for a liquid crystal spatial light modulator, the direction of the liquid crystal molecules can be adjusted by applying an electric field, thereby changing the phase of the light wave when it passes through the liquid crystal layer. The magnitude and form of the voltage can be used as control parameters. Increasing the voltage may cause the liquid crystal molecules to rearrange and change the phase. The amplitude of the light wave can also be adjusted by changing the electric field, and the magnitude and form of the voltage can be used as control parameters. For a DMD, the parameters for input phase modulation are usually a digital pattern, where the values of the pattern correspond to different phase or amplitude states respectively. By controlling the state of each micromirror, corresponding phase modulation can be introduced into the output light wave; controlling the state of each micromirror can also adjust the intensity of the reflected light to achieve digital amplitude modulation. The above-mentioned electric field and digital pattern can both be called optical masks. Unless otherwise specified below, the digital pattern is referred to as an optical mask.

[0060] The technical solutions of the embodiments of this application can be applied to various complex lighting scenarios. For example, license plate recognition at night, military reconnaissance, underground search and rescue, vein exploration, etc. The following describes the imaging method of an imaging system in the field of traffic road monitoring, but this application is not limited thereto. The imaging system in the field of traffic road monitoring is used to identify the license plate information of moving vehicles in the environment of strong headlight illumination at night.

[0061] Figure 2 FIG. 200 is a schematic flowchart of a method for high dynamic range imaging according to an embodiment of the present application. The following embodiments are all exemplified by the method for high dynamic range imaging of a moving object. However, the method is also effective for high dynamic range imaging of a stationary object, and the present embodiment does not limit this.

[0062] 210, acquire multiple first images of the shooting area.

[0063] In the case where there is a moving object in the shooting area, the moving object can be determined by acquiring multiple images. For example, in the process of license plate recognition of vehicles in a fixed shooting area by an imaging system in the field of traffic road monitoring, the vehicle is moving in the shooting area. The imaging system acquires multiple optical image signals of the shooting area, and each optical image signal is respectively converted into a digital image by an image sensor. The multiple digital images are the multiple first images.

[0064] Optionally, the multiple first images can be multiple continuously acquired frames. The multiple consecutive first images can more accurately reflect the movement trajectory of the moving object in the shooting area. In a possible embodiment, the operating frequency of each component in the imaging system is F, and the interval between any two moments is 1 / F seconds. N + 1 images are continuously captured from any moment T to moment T + N as the multiple first images.

[0065] 220, determine a target area according to the multiple first images. The target area is a partial area in the shooting area, where the brightness of the target area reaches a threshold.

[0066] Taking the method for high dynamic range imaging of a moving object as an example, the target area includes at least a part of the moving object and the brightness of at least a part of the moving object reaches the threshold. The specific threshold depends on the image sensor of the imaging system, and the present application does not limit this. The analog electrical signal collected is converted into a digital image represented by brightness through an analog-digital conversion circuit. For an 8-bit digital image, the value range of each pixel is 0 to 255. If the pixel value is 255, it means that the optical signal intensity of the corresponding area reaches saturation. Therefore, the threshold of the brightness of the target area can be 255, or 245, 250, etc. Similarly, for a 12-bit digital image, the threshold of the brightness of the target area can be 4095, or 4085, 4090, etc., and the present embodiment does not limit this.

[0067] First, based on multiple first images, predict the moving object area corresponding to the moving object. Specifically, a video prediction algorithm can be executed. For example, a pre-trained PredNet neural network model can be adopted, which can identify features such as the edges, shapes, colors, and motion states of objects in the image, and predict the next image based on the object features of multiple frames of images. In this embodiment, the features of the moving object can be determined according to the images of the moving object in multiple first images. For example, the object can be a traffic sign, a roadside tree, a moving vehicle, etc., and the moving object mainly includes a moving vehicle. The images of the moving vehicle in multiple first images can be analyzed to obtain features such as the motion state of the moving vehicle.

[0068] It should be noted that only a part of the moving vehicle may be located in the shooting area of the imaging system. The part of the vehicle located in the shooting area of the imaging system can be further divided into a part with brightness reaching the threshold and a part with brightness less than the threshold. In this case, at least a part of the moving object has a brightness reaching the threshold.

[0069] Furthermore, determine the moving object area according to the features of the moving object. For example, a predicted image can be calculated according to the features of the moving object. In this embodiment, a predicted image is calculated according to the features such as the motion state of the moving vehicle. The predicted image includes the image of the moving object, and the area corresponding to the image of the moving object is called the moving object area.

[0070] In this embodiment, the target area can be determined according to the overexposed area in the moving object area. Specifically, the area corresponding to the pixels with brightness reaching the threshold in the predicted image is called the overexposed area. For example, the target area can be the rear of the vehicle illuminated by the strong light of the vehicle lamp. In some possible cases, the overexposed area is the part of the vehicle rear except the rear windshield, that is, the overexposed area is a part of the moving object area; in some other possible cases, some traffic signs in the shooting area are also illuminated by the vehicle lamp, and the overexposed area includes the area where the traffic signs are located. At this time, the overexposed area in the moving object area refers to the intersection of the overexposed area and the moving object area.

[0071] In addition, in some image processing methods, for the convenience of subsequent processing, the pixels corresponding to the overexposed area, the moving object area, and the target area are expanded into a rectangular pixel array to which these pixels belong. Taking the overexposed area as an example, the overexposed area at this time should be determined by this rectangular pixel array, and the true three-dimensional shape of the overexposed area is not fixed. The sizes and shapes of the overexposed area, the moving object area, and the target area are not limited in this embodiment.

[0072] 230, collect a second image of the shooting area. When the second image is taken, the optical signal from the target area has been attenuated in light intensity.

[0073] Specifically, an optical mask can be determined based on the pixels corresponding to the shooting area and the pixels corresponding to the target area; the optical signal from the shooting area can be modulated according to the optical mask; and then the modulated optical signal can be imaged to obtain a second image. Taking the imaging system 300 shown in Figure 3 as an example, the imaging system 300 can be used to implement the methods described in 210 and 220, and can also implement the method described in 230 in the following manner.

[0074] The optical mask is used to make the imaging of the shooting area have appropriate brightness. It can be loaded onto the digital micromirror array 320 to achieve the modulation of the optical signal. For example, in this embodiment, according to the optical mask, the state of each micromirror of the digital micromirror array 320 can be controlled, and the intensity of the reflected light can be adjusted to achieve digital amplitude modulation, and the light intensity of the optical signal in the target area can be weakened. This method has low implementation complexity and controllable cost.

[0075] In one embodiment, each micromirror of the digital micromirror array 320 has two states and can be regarded as a binary bitmap. Therefore, when the optical mask is a binary bitmap, the arrangement of the pixels of the optical mask can be the same as the arrangement of the micromirrors of the digital micromirror array 320, where the value of the pixel of the optical mask represents whether the light intensity at that position is weakened.

[0076] For example, the arrangement of the micromirrors of the digital micromirror array 320 is 30720×17280, the number of pixels of the optical mask is 30720×17280, and the value of each pixel is 0 or 1. The value of 1 for the pixel of the optical mask means that the micromirror reflects away from the optical axis, weakening the light intensity at that position, and the value of 0 means that the micromirror reflects towards the optical axis, not weakening the light intensity at that position. The specific degree of light intensity weakening depends on the specific positional relationship between the optical axis and the digital micromirror array 320, which is not limited in this embodiment. In one embodiment, the value of the pixel corresponding to the target area in the optical mask is 1, and the values of the remaining pixels are 0.

[0077] The optical mask in the form of a binary bitmap can keep the brightness ratio between the pixels corresponding to the target area unchanged, which is convenient for subsequent processing.

[0078] For other types of spatial light modulators, an adjustable optical filter can be introduced to achieve amplitude modulation of light waves by adjusting the intensity of the transmitted light. For example, a liquid crystal optical filter is selected as the adjustable optical filter, and the molecular orientation of the liquid crystal of the liquid crystal optical filter is controlled by an external voltage or electric field to adjust the transmitted spectral range or transmittance. The field strength distribution of the external electric field can be set according to the ratio of the brightness of the pixels corresponding to the target area to the threshold.

[0079] In another embodiment, the optical mask may also be a grayscale bitmap. Specifically, each micromirror on the digital micromirror array 320 can be divided into several small regions, such that the arrangement of these small regions is the same as the arrangement of the pixels of the obtained digital image. Then, by controlling the reflection states of these small regions respectively, the grayscale modulation of each pixel can be achieved. In such a case, multiple micromirrors of the digital micromirror array 320 form a microarray, corresponding to one pixel of the obtained digital image. The arrangement of the pixels of the required optical mask is the same as the arrangement of the pixels of the digital image obtained by the CMOS camera imaging. Among them, the value of the pixel of the optical mask represents the degree to which the light intensity at that position is attenuated.

[0080] For example, Figure 3 the number of pixels of the digital image obtained by the CMOS camera imaging in [reference] is 1920×1080, and the arrangement of the micromirrors of the digital micromirror array 320 is 30720×17280. The micromirrors of the digital micromirror array 320 can be grouped in 16×16 to form a microarray. The number of pixels of the optical mask is 1920×1080, and the value range of each pixel is 0 to 255. Each pixel corresponds to a microarray respectively. The pixel value of 0 indicates that all the micromirrors in the microarray deflect and reflect along the optical axis, without attenuating the light intensity of this pixel. The pixel value of 127 indicates that 127 out of 256 micromirrors in the corresponding microarray deflect and reflect away from the optical axis. The pixel value of 255 indicates that 255 out of 256 micromirrors in the corresponding microarray deflect and reflect away from the optical axis, and only 1 micromirror deflects and reflects along the optical axis. The value of the pixel of the optical mask can be set according to the ratio of the brightness of the pixel corresponding to the target area to the threshold.

[0081] The optical mask in the form of a grayscale bitmap can make the pixels corresponding to the target area have relatively close brightness, which is convenient for subsequent processing.

[0082] Optionally, the second image may be acquired in the next frame after multiple first images are acquired. In this case, the overlap degree between the area where the moving object is located and the area of the moving object in the predicted image is high, and the obtained second image has high clarity and moderate brightness. In a possible embodiment, the operating frequency of each component in the imaging system 300 is F, the interval between any two moments is 1 / F seconds, and multiple first images are N + 1 consecutive images captured from an arbitrary moment T to T + N. Then, in this embodiment, an image is captured at the moment T + N + 1 as the second image.

[0083] In this embodiment, the second image can be input into the license plate recognition network, and image recognition can be performed according to the second image to obtain the license plate number in the second image. The application does not limit the use of the second image. The second image obtained by the method of the embodiments of the present application has high quality, and thus can improve the accuracy of image recognition.

[0084] The method for high-dynamic range imaging according to the embodiments of the present application weakens the light intensity according to the target area, enabling a single-frame image captured once to have high clarity and moderate brightness, thereby being able to effectively perform high-dynamic range imaging and being applicable to high-speed imaging scenarios of moving objects, etc.

[0085] The present application also provides an imaging system, which can execute the method of the foregoing embodiments of the present application. Figure 4 It is a schematic structural diagram of an imaging system 600 provided by an embodiment of the present application. The imaging system 600 includes:

[0086] An image sensor 630, configured to collect multiple first images of a shooting area; and also configured to collect a second image of the shooting area according to the modulated optical signal.

[0087] A processing module 640, configured to determine a target area according to multiple first images, where the target area is a partial area in the shooting area, and the brightness of the target area reaches a threshold; and is also configured to determine an optical mask according to the pixels corresponding to the shooting area and the pixels corresponding to the target area.

[0088] A modulation module 620, configured to modulate the optical signal from the shooting area to weaken the light intensity of the optical signal from the target area.

[0089] The imaging system 600 may further include a lens group 610, configured to receive the optical signal from the shooting area and transmit the optical signal to the image sensor 630 via the modulation module 620.

[0090] The lens group 610, the modulation module 620, and the image sensor 630 may be respectively Figure 1 the lens group 110, the spatial light modulator 120, and the image sensor 130 in Figure 3 or the lens group 310, the digital micromirror array 320, and the CMOS camera 330 in

[0091] In an application scenario of this embodiment, the processing module 640 may also be configured to perform image recognition according to the second image. The second image obtained by the method of the embodiments of the present application has high quality, so the accuracy of image recognition can be improved.

[0092] The specific implementation manners of the functions executed by each module have been described in the method embodiments and will not be elaborated here.

[0093] The above-mentioned term "module" can be implemented in the form of software and / or hardware, without specific limitation. For example, a "module" can be a software program, a hardware circuit, or a combination of both that implements the above functions, and can include code running on a computing instance. Exemplarily, a processing module can be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). Among them, the above PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. A modulation module can be a programmable hardware function module such as a digital micromirror array, and an image sensor can be a programmable hardware function module such as a CCD camera or a CMOS camera.

[0094] Therefore, the modules in the examples described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0095] The present application also provides a controller 700. As Figure 5 shown, the controller 700 includes: a processor 704 and a communication interface 708. Further, the controller 700 may also include a bus 702 and a memory 706. It should be understood that the bus 702 and the memory 706 are optional. The processor 704, the memory 706, and the communication interface 708 communicate with each other through the bus 702. Exemplarily, the controller 700 can be a computing device or a system in a computing device for implementing the method of the embodiments of the present application. The controller 700 can be various terminal devices. It should be understood that the present application does not limit the number of processors and memories in the controller 700.

[0096] The bus 702 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5It is represented by only one line in the figure, but it does not mean that there is only one bus or one type of bus. The bus 704 may include a path for transmitting information between various components of the controller 700 (for example, the memory 706, the processor 704, and the communication interface 708).

[0097] The processor 704 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0098] The memory 706 may include a volatile memory, such as a random access memory (RAM). The memory 706 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0099] The memory 706 stores executable program code, and the processor 704 executes the executable program code to implement the functions of the foregoing processing modules, thereby generating a high dynamic range image. That is, the memory 706 stores instructions for the method of high dynamic range imaging.

[0100] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the controller 700 and other devices or communication networks (for example, the user terminal of the agent). The communication interface may also be referred to as an interface circuit.

[0101] In a possible implementation, the controller 700 is used to implement the hardware function modules in the agent.

[0102] The embodiment of the present application further provides a camera, which includes the hardware function modules of the foregoing imaging system and can execute the method of high dynamic range imaging described above. The selection of specific hardware function modules has been illustrated in the foregoing embodiments.

[0103] The embodiment of the present application further provides a terminal, which includes the hardware function modules of the foregoing imaging system and can execute the method of high dynamic range imaging described above. The terminal may be a device with a photographing function such as a smart phone, a smart tablet, an Internet of Things device, or an industrial control device.

[0104] The embodiments of the present application further provide a vehicle, which includes the hardware function modules of the above imaging system and is capable of executing the method for high-dynamic-range imaging. The vehicle may be a vehicle with an assisted driving function, an unmanned vehicle, or other vehicles that can interact with electronic devices.

[0105] The embodiments of the present application further provide a computer program product containing instructions. The computer program product may be software or a program product containing instructions that can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, it causes at least one computing device to execute the method in the embodiments of the present application.

[0106] The embodiments of the present application further provide a computer-readable storage medium. The computer-readable storage medium may be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive), etc. The computer-readable storage medium includes instructions that direct the computing device to execute the method in the embodiments of the present application, or direct the computing device to execute the method in the embodiments of the present application.

[0107] It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the order of execution is prior or subsequent. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0108] Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0109] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0110] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or modules can be in electrical, mechanical, or other forms.

[0111] As described above, this is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for high dynamic range imaging, characterized in that, it includes: Collecting multiple first images of the shooting area; Determining a target area according to the multiple first images, where the target area is a partial area in the shooting area, and the brightness of the target area reaches a threshold; Collecting a second image of the shooting area, where when the second image is formed, the light signal from the target area is attenuated in light intensity.

2. The method according to claim 1, characterized in that, the target area includes at least a part of the moving object, and the brightness of at least a part of the moving object reaches the threshold.

3. The method according to claim 1 or 2, characterized in that, collecting the second image of the shooting area includes: Determining an optical mask according to the pixels corresponding to the shooting area and the pixels corresponding to the target area; Modulating the light signal from the shooting area according to the optical mask; Imaging according to the modulated light signal to obtain the second image.

4. The method according to claim 3, characterized in that, modulating the light signal from the shooting area according to the optical mask includes: Controlling a spatial light modulator according to the optical mask to modulate the light signal of the target area so as to attenuate the light intensity of the light signal from the target area.

5. The method according to claim 3 or 4, characterized in that, the optical mask is a binary bitmap, and the value of the pixels of the optical mask represents whether the light intensity is attenuated.

6. The method according to claim 3 or 4, characterized in that, the optical mask is a grayscale bitmap, and the value of the pixels of the optical mask represents the degree of attenuation of the light intensity.

7. The method according to any one of claims 2 to 6, characterized in that, determining the target area according to the multiple first images includes: Predicting the moving object area corresponding to the moving object according to the multiple first images; Determining the target area according to the overexposed area in the moving object area, where the brightness of the overexposed area reaches the threshold.

8. The method according to claim 7, characterized in that, predicting the moving object area corresponding to the moving object according to the multiple first images includes: Determining the characteristics of the moving object according to the image of the moving object in the multiple first images; Determining the moving object area according to the characteristics of the moving object.

9. The method according to any one of claims 1 to 8, characterized in that, the multiple first images are multiple consecutive frames of images collected.

10. The method according to any one of claims 1 to 9, characterized in that, the second image is collected in the next frame after the collection of the multiple first images is completed.

11. The method according to any one of claims 1 to 10, characterized in that, the method further includes: Performing image recognition according to the second image.

12. An imaging system, characterized in that, it includes: An image sensor for collecting multiple first images of the shooting area; A processing module, configured to determine a target area according to the multiple first images, where the target area is a partial area in the shooting area, and the brightness of the target area reaches a threshold value; A modulation module, configured to modulate the optical signal from the shooting area to attenuate the optical intensity of the optical signal from the target area; The image sensor is further configured to collect a second image of the shooting area according to the modulated optical signal.

13. The imaging system according to claim 12, wherein, the target area includes at least a part of a moving object, and the brightness of at least a part of the moving object reaches the threshold value.

14. The imaging system according to claim 12 or 13, wherein, the modulation module includes at least one of a digital micromirror array, a liquid crystal light modulator or an acousto-optic modulator.

15. The imaging system according to any one of claims 12 to 14, wherein: the processing module is configured to determine an optical mask according to the pixels corresponding to the shooting area and the pixels corresponding to the target area; the modulation module is configured to modulate the optical signal from the shooting area according to the optical mask.

16. The imaging system according to claim 15, wherein, the optical mask is a binary bitmap, and the value of the pixel of the optical mask represents whether the optical intensity is attenuated.

17. The imaging system according to claim 15, wherein, the optical mask is a grayscale bitmap, and the value of the pixel of the optical mask represents the degree of attenuation of the optical intensity.

18. The imaging system according to any one of claims 13 to 17, wherein, the processing module is configured to: predict a moving object area corresponding to the moving object according to the multiple first images; determine the target area according to the overexposed area in the moving object area, where the brightness of the overexposed area reaches the threshold value.

19. The imaging system according to claim 18, wherein, the processing module is configured to: determine the features of the moving object according to the image of the moving object in the multiple first images; determine the moving object area according to the features of the moving object.

20. The imaging system according to any one of claims 12 to 19, wherein, the multiple first images are multiple consecutive frames of images.

21. The imaging system according to any one of claims 12 to 20, wherein, the second image is acquired in the next frame after the acquisition of the multiple first images is completed.

22. The imaging system according to any one of claims 12 to 21, wherein, further includes: a lens group, configured to receive the optical signal from the shooting area and transmit the optical signal from the shooting area to the image sensor via the modulation module.

23. The imaging system according to any one of claims 12 to 22, wherein, the processing module is further configured to perform image recognition according to the second image.

24. A camera, wherein, it includes the imaging system according to any one of claims 12 to 23.

25. A terminal, It is characterized in that it includes an imaging system according to any one of claims 12 to 23.

26. A vehicle It is characterized in that it includes an imaging system according to any one of claims 12 to 23.

Images