A multi-dimensional regulation and control large dynamic range imaging device, method and storage medium

Abstract

The embodiment of the application discloses a multi-dimensional regulation and control large dynamic range imaging device and method and storage medium, the device comprises: an imaging lens group, which is used for focusing and aberration correction of a target scene and outputs a target scene light signal; a silicon-based planar light field regulation and control device, which is used for receiving the target scene light signal, performing spectral polarization modulation on the target scene light signal, and obtaining a four-state polarization image; and a multi-dimensional electric field regulation and control device, which is used for photoelectric conversion of the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image. The imaging lens group effectively focuses and corrects aberration of the target scene and outputs a light signal of the target scene. Only the silicon-based planar light field regulation and control device is used to regulate and control the output light signal of the target scene, without a large number of optical devices, and the device has high integration. Meanwhile, the multi-dimensional electric field regulation and control device accurately obtains polarization intensity information and event information of the four-state polarization image, has a larger dynamic range and richer information acquisition dimensions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photoelectric imaging, and in particular to a multi-dimensional regulation and control large dynamic range imaging device, method and storage medium. BACKGROUND

[0002] In the process of space exploration and natural environment imaging, the backlight scene often causes the camera to overexpose, resulting in a decrease in image quality, loss of foreground target information, and limitation of the application scene of the imaging system. In order to have a good imaging effect in a backlight scene, a large dynamic range imaging device is needed.

[0003] Polarization imaging obtains the full Stokes vector of the target scene by obtaining the polarization information of the target, describes the reflection and scattering characteristics of the object surface through polarization characteristic information, and can analyze the surface topography, stress, texture, roughness and other characteristics of the foreground target, thereby improving the image feature information. At the same time, the background light can be effectively suppressed due to the lack of polarization of the strong light. Existing polarization photoelectric imaging cameras can be roughly divided into time division type, amplitude division type, focal plane division type and aperture division type according to their imaging methods. The time division type has poor effect in a moving scene, and needs to adjust the imaging element frequently, which can cause the drift of the light beam and reduce the imaging accuracy. The amplitude division type is large and complex, and needs to respond consistently between each detector, which can easily cause errors. The focal plane division type has parallax between each division, which affects the later registration. The aperture division type has crosstalk between each division, which causes the image quality to decrease. SUMMARY

[0004] Therefore, it is necessary to propose a multi-dimensional regulation and control large dynamic range imaging device, method and storage medium in view of the above problems.

[0005] A multi-dimensional regulation and control large dynamic range imaging device, the device comprising:

[0006] An imaging lens group for focusing and aberration correction of a target scene, and outputting a target scene light signal.

[0007] A silicon-based planar light field regulation device for receiving the target scene light signal, performing spectral polarization modulation on the target scene light signal, and obtaining a four-state polarization image.

[0008] A multi-dimensional electric field regulation device for photoelectric conversion of the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image.

[0009] The silicon-based planar light field regulation device comprises a planar light field polarization regulation device and a filter film,

[0010] The light filtering film is used for receiving the target scene light signal, performing wavelength selection on the target scene light signal, and outputting a light signal of a preset wavelength.

[0011] The planar light field polarization modulation device comprises a plurality of metal nanocolumns arranged according to a matrix Fourier optical design, each of the metal nanocolumns has a size of 20-70 nm, and is used for performing spectral polarization modulation on the light signal of the preset wavelength to obtain a four-state polarization image.

[0012] The multi-dimensional electric field modulation device specifically comprises:

[0013] The intensity image readout module comprises an integral readout circuit, and is used for obtaining intensity information of the four-state polarization image containing a global Stokes vector description of a target scene according to global light intensity of the four-state polarization image.

[0014] The event information readout module comprises a threshold gating circuit, and is used for obtaining dynamic information of the four-state polarization image according to dynamic light intensity of the four-state polarization image at a pixel position, and obtaining an event stream according to the dynamic information.

[0015] The integral readout circuit and the threshold gating circuit are cascaded and share a photodiode at each pixel, and are used for continuously photoelectrically converting the four-state polarization image for a preset exposure period to obtain global light intensity of the four-state polarization image and dynamic light intensity of the four-state polarization image at the pixel position.

[0016] The device further comprises:

[0017] The field of view diaphragm is used for limiting an imaging range of the imaging lens group.

[0018] A multi-dimensional modulation large dynamic range imaging method, the method comprising:

[0019] Focusing and aberration correction are performed on a target scene to output a target scene light signal.

[0020] The target scene light signal is received, polarization modulation is performed on the target scene light signal, and a four-state polarization image is obtained.

[0021] Photoelectric conversion is performed on the four-state polarization image to obtain polarization intensity information and an event stream of the four-state polarization image.

[0022] The target scene light signal is received, polarization modulation is performed on the target scene light signal, and a four-state polarization image is obtained, specifically comprising:

[0023] Receiving the target scene light signal, and obtaining the arbitrary diffraction order related Fourier Jones matrix coefficients of the planar light field polarization modulation device according to the phase delay caused by each metal nanocolumn to the target scene light signal.

[0024] Determining the required four-state polarization diffraction order light intensity according to the arbitrary diffraction order related Fourier Jones matrix coefficients.

[0025] Diffracting the target scene light signal to the required four-state polarization diffraction order light intensity, and when each required four-state polarization diffraction order light intensity is uniformly distributed on the image plane of the planar light field polarization modulation device, the sum of each required four-state polarization diffraction order light intensity reaches a maximum value, and a four-state polarization image is obtained.

[0026] The receiving the target scene light signal, and obtaining the arbitrary diffraction order related Fourier Jones matrix coefficients of the planar light field polarization modulation device according to the phase delay caused by each metal nanocolumn to the target scene light signal, specifically includes:

[0027] The arbitrary diffraction order related Fourier Jones matrix coefficients of the planar light field polarization modulation device are determined according to the phase delay caused by each metal nanocolumn to the target scene light signal. Determining the phase delay caused by each metal nanocolumn to the target scene light signal, wherein (x, y) represents the local coordinate axis of the nanocolumn, f x The transmission coefficient of the nanocolumn along its transverse axis, f y The transmission coefficient of the nanocolumn along its longitudinal axis, and θ is the direction of the nanocolumn relative to the reference coordinate system.

[0028] The arbitrary diffraction order related Fourier Jones matrix coefficients of the planar light field polarization modulation device are determined according to the phase delay caused by each metal nanocolumn to the target scene light signal. L is the length of a single period of the planar light field polarization modulation device, and H is the width of a single period of the planar light field polarization modulation device, wherein,

[0029] The photoelectric conversion of the four-state polarization image outputs the polarization intensity information and event stream of the four-state polarization image.

[0030] Continuously photoelectric converting the four-state polarization image for a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0031] Determining the total Stokes vector of the target scene according to the global light intensity, and taking the total Stokes vector as the polarization intensity information of the four-state polarization image.

[0032] If the dynamic light intensity variation of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, the polarization modulated event information is obtained.

[0033] Determine each event occurrence according to the event information, and obtain an event stream based on the each event occurrence.

[0034] A computer readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0035] The embodiment of the present application has the following beneficial effects:

[0036] The imaging lens group effectively focuses and corrects aberration of the target scene, outputs the light signal of the target scene, and only uses the silicon-based planar light field modulation device to modulate the output light signal of the target scene, which can effectively suppress the background strong light information by using the non-selectivity of polarization modulation of the background natural strong light, highlight the foreground target, has a simple and compact structure, does not need a large number of optical devices, and has high integration. And, the multi-dimensional electric field modulation device is used to quickly and accurately obtain the polarization intensity information and event information of the four-state polarization image, has a larger dynamic range and more rich information acquisition dimension, has light weight, can more efficiently image in backlight scene, can be applied to many scenes with high weight requirement, and has important significance for subsequent image analysis, target recognition, event detection and other applications. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0038] Among them:

[0039] Figure 1 A structural schematic diagram of an embodiment of a multi-dimensional modulation large dynamic range imaging device provided by the present application;

[0040] Figure 2 A structural schematic diagram of another embodiment of a multi-dimensional modulation large dynamic range imaging device provided by the present application;

[0041] Figure 3 A structural schematic diagram of an embodiment of a multi-dimensional electric field modulation device provided by the present application;

[0042] Figure 4 A flowchart of an embodiment of a multi-dimensional modulation large dynamic range imaging method provided by the present application;

[0043] Figure 5A flowchart of another embodiment of the multi-dimensional regulation large dynamic range imaging method provided by the present application is shown in FIG. 2.

[0044] Figure 6 A structural diagram of an embodiment of the storage medium provided by the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of the present application.

[0046] As shown in FIG. 1, Figure 1 Figure 1 A structural diagram of an embodiment of the multi-dimensional regulation large dynamic range imaging device provided by the present application is shown in FIG. 1. The multi-dimensional regulation large dynamic range imaging device 10 includes:

[0047] An imaging lens group 11 is used for imaging a target scene and correcting aberration, and outputs a target scene light signal.

[0048] Exemplarily, Figure 2 A structural diagram of another embodiment of the multi-dimensional regulation large dynamic range imaging device provided by the present application is shown in FIG. 2. The multi-dimensional regulation large dynamic range imaging device further includes a field diaphragm 14. The imaging lens group is usually composed of multiple lenses. After the light information of the target scene is focused and aberration-corrected by the lenses, a target scene light signal is output, and focused on an image plane or a focal plane to form a clear image. The field diaphragm 14 is located between the imaging lens group 11 and the silicon-based planar light field regulation device 12, and is used to limit the imaging range of the imaging lens group, so that there is no crosstalk between the four-state polarization state images after passing through the silicon-based planar light field regulation device 12.

[0049] The silicon-based planar light field regulation device 12 includes a plurality of metal nanocolumns arranged according to matrix Fourier optical design. The size of each metal nanocolumn is between 20-70 nm, and is used for spectral polarization modulation of the light signal of the preset wavelength to obtain a four-state polarization image.

[0050] ​Exemplarily, the silicon-based planar light field regulation device 12 is accurately positioned at the image plane or focal plane of the imaging lens group 11 to modulate the light information of the target scene in spectral polarization. The silicon-based planar light field regulation device 12 is composed of a planar light field polarization regulation device and a light filtering film. The silicon-based planar light field regulation device is composed of TiO2 metal nanocolumns, each TiO2 metal nanocolumn has a size of 20-70 μm, and is arranged according to matrix Fourier optical design with a period of 1.9 μm. The light filtering film is a 480 nm central transmission wavelength light filtering film, which is coated on the planar light field polarization regulation device to limit the influence of the dispersion effect of the planar light field polarization regulation device on the image quality.

[0051] Specifically, the light filtering film receives the light signal of the target scene, selects the wavelength of the light signal of the target scene, and outputs the light signal with a wavelength of 480 nm. Further, the planar light field polarization regulation device modulates the light signal with a wavelength of 480 nm in spectral polarization to obtain a four-state polarization image.

[0052] The multi-dimensional electric field regulation device 13 is used to photoelectrically convert the four-state polarization image to obtain the polarization intensity information and event stream of the four-state polarization image.

[0053] Exemplarily, in combination with the above description, Figure 3 , Figure 3 The structure diagram of an embodiment of the multi-dimensional electric field regulation device provided by the present application is shown. The multi-dimensional electric field regulation device 13 is accurately positioned at the image plane of the silicon-based planar light field regulation device 12, and is used to photoelectrically convert and electrically regulate the light information of the target scene, and perform high dynamic range imaging. Specifically, the multi-dimensional electric field regulation device 13 includes an intensity image readout module and an event information readout module. The intensity image readout module is mainly composed of an integration readout circuit 131, which is used to obtain the intensity information of the four-state polarization image containing the global Stokes vector description of the target scene according to the global light intensity of the four-state polarization image. The event information readout module is mainly composed of a threshold gating circuit 132, which is used to obtain the dynamic information of the four-state polarization image according to the dynamic light intensity of the four-state polarization image at the pixel position, and obtain the event stream according to the dynamic information. The integration readout circuit 131 and the threshold gating circuit 132 are cascaded and share a photodiode 133 at each pixel, which is used to continuously photoelectrically convert the four-state polarization image with a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0054] As can be known from the above description, the application effectively focuses and corrects aberration of a target scene through an imaging lens group, outputs a light signal of the target scene, only uses a silicon-based planar light field regulation device to regulate the output light signal of the target scene, effectively suppresses background strong light information through non-selectivity of polarization modulation on the background natural strong light, highlights the foreground target, has a simple and compact structure, does not need a large number of optical devices, has high integration, and simultaneously uses a multi-dimensional electric field regulation device to quickly and accurately acquire polarization intensity information and event information of a four-state polarization image, has a larger dynamic range and a more abundant information acquisition dimension, has light weight, can more efficiently image in a backlight scene, can be applied in many scenes with high weight requirements, and has important significance for subsequent image analysis, target recognition, event detection and the like.

[0055] As shown in Figure 4 , Figure 4 is a flowchart of an embodiment of a multi-dimensional regulation large dynamic range imaging method provided by the application. The multi-dimensional regulation large dynamic range imaging method comprises the following steps.

[0056] S101: focusing and correcting aberration of a target scene, and outputting a light signal of the target scene.

[0057] Exemplarily, the imaging lens group 11 focuses and corrects aberration of light information of the target scene, and outputs a light signal of the target scene, and further focuses the light signal of the target scene to an image plane or a focal plane of the silicon-based planar light field regulation device 12, to form a clear image.

[0058] S102: receiving the light signal of the target scene, performing polarization modulation on the light signal of the target scene, and acquiring a four-state polarization image.

[0059] Exemplarily, the light signal of the target scene is received, arbitrary diffraction order related Fourier Jones matrix coefficients of the planar light field polarization regulation device are acquired according to phase delay caused by each metal nanometer column to the light signal of the target scene, required four-state polarization diffraction order light intensity is determined according to the arbitrary diffraction order related Fourier Jones matrix coefficients, the light signal of the target scene is diffracted to the required four-state polarization diffraction order light intensity, until the total sum of the required four-state polarization diffraction order light intensity reaches a maximum value in the case that the required four-state polarization diffraction order light intensity is uniformly distributed on the planar light field polarization regulation device, and the four-state polarization image is acquired.

[0060] S103: photoelectric conversion of the four-state polarization image to acquire polarization intensity information and an event stream of the four-state polarization image.

[0061] Exemplarily, the four-state polarization image is photoelectrically converted for a preset exposure period to obtain a global light intensity of the four-state polarization image and a dynamic light intensity of the four-state polarization image at a pixel position; a global Stokes vector of the target scene is determined according to the global light intensity, and the global Stokes vector is taken as polarization intensity information of the four-state polarization image; if a dynamic light intensity change of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, event information after polarization modulation is obtained; event occurrence conditions are determined according to the event information, and an event stream is obtained based on the event occurrence conditions.

[0062] As Figure 5 shown, Figure 5 is a flowchart of another embodiment of a multi-dimensional regulation large dynamic range imaging method provided by the present application. The multi-dimensional regulation large dynamic range imaging method comprises the following steps:

[0063] S201: clearly imaging a target scene and performing aberration and chromatic aberration correction to output a target scene light signal.

[0064] It should be noted that step S201 has been discussed in detail in the implementation scenario shown in Figure 3 and will not be repeated here.

[0065] S202: receiving the target scene light signal, and obtaining an arbitrary diffraction order related Fourier Jones matrix coefficient of a planar light field polarization regulation device according to a phase delay caused by each metal nanocolumn to the target scene light signal.

[0066] Exemplarily, the phase delay caused by each metal nanocolumn to the target scene light signal is determined according to the formula as follows:

[0067]

[0068] wherein (x, y) represents a local coordinate axis of the nanocolumn, f x is a transmission coefficient of the nanocolumn along its horizontal axis, f y is a transmission coefficient of the nanocolumn along its vertical axis, and θ is a direction in which the nanocolumn is rotated relative to a reference coordinate system.

[0069] Further, the arbitrary diffraction order related Fourier Jones matrix coefficient of the planar light field polarization regulation device is determined according to the formula as follows:

[0070]

[0071] wherein (l, h) is a diffraction order, L is a length of a single period of the planar light field polarization regulation device, H is a width of the single period of the planar light field polarization regulation device, respectively, are spatial frequencies of each nanocolumn in (x, y) directions.

[0072] S203: determining the required four-state polarization diffraction order light intensity according to any diffraction order related Fourier Jones matrix coefficient.

[0073] Exemplarily, the required four-state polarization diffraction order light intensity is determined according to the formula as shown in the following:

[0074]

[0075] wherein I (l,h) is the required four-state polarization diffraction order light intensity, δ is the polarization state, is the Hermitian conjugate of , is the diffraction order related Fourier Jones matrix coefficient.

[0076] S204: diffracting the target scene light signal to the required four-state polarization diffraction order light intensity, until the total sum of the required four-state polarization diffraction order light intensity reaches the maximum value when the required four-state polarization diffraction order light intensity is uniformly distributed on the image plane of the planar light field polarization control device, and obtaining the four-state polarization image.

[0077] Exemplarily, the target scene light signal is diffracted to the required four-state polarization diffraction order light intensity, and an optimization problem is constructed with the constraint that the remaining diffraction order light intensity is equal to zero, and the gradient descent algorithm is used for iterative solution until the total sum of the required four-state polarization diffraction order light intensity reaches the maximum value when the required four-state polarization diffraction order light intensity is uniformly distributed on the image plane of the planar light field polarization control device, and the four-state polarization image is obtained.

[0078] S205: photoelectrically converting the four-state polarization image for a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0079] Exemplarily, the photoelectric conversion of the four-state polarization image is performed by the photodiode at each pixel for an exposure period of 100 ms to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0080] S206: determining the global Stokes vector of the target scene according to the global light intensity, and taking the global Stokes vector as the polarization intensity information of the four-state polarization image.

[0081] Exemplarily, the global light intensity of the four-state polarization image is shown in the following formula:

[0082]

[0083] wherein, For the multi-dimensional electric field to regulate the global light intensity of the received four-state polarization image, I1, I2, I3, I4 represent the four-state polarization image intensity respectively, λ is the spectral modulation parameter introduced by the planar light field polarization modulation device, and the global Stokes vector of the target scene is which can be expressed as:

[0084]

[0085] wherein M is a system polarization parameter given by the system, which is not specifically limited here.

[0086] S207: If the dynamic light intensity change of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, the polarization modulated event information is obtained.

[0087] Exemplarily, the event information readout module and the intensity image readout module work simultaneously to capture the dynamic information after polarization modulation in the target scene. Specifically, at time ts, the dynamic information after polarization modulation is as follows:

[0088]

[0089] wherein L is event information, is the dynamic light intensity of the four-state polarization image at each pixel position, which can be specifically expressed as:

[0090]

[0091] wherein λ is the spectral modulation parameter introduced by the planar light field polarization modulation device, is the dynamic light intensity of the four-state polarization image at the corresponding pixel position (x i ,y i ).

[0092] Further, if the dynamic light intensity change of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, the polarization modulated event information is:

[0093]

[0094] wherein δ is an event polarity, represent that positive events and negative events occurred in four polarization states respectively, Δt s is the event interval between two events, and C is a preset event threshold.

[0095] S208: Determine the occurrence of each event according to the event information, and obtain the event stream based on the occurrence of each event.

[0096] Exemplarily, the occurrence of each event can be recorded as:

[0097]

[0098] wherein, is the i-th event occurrence.

[0099] Finally, the event stream output can be obtained, denoted as:

[0100]

[0101] wherein, is the event stream, i [1, n].

[0102] From the above description, it can be known that the application adopts the silicon-based planar light field regulation device to regulate the target light information, the full Stokes vector information of the target scene can be obtained by single exposure, the real-time performance is high, and the silicon-based planar light field regulation device and the multi-dimensional electric field regulation device are combined, the light-electric field multi-dimensional regulation is fused, the spectral polarization information and the event information can be acquired at the same time, the dynamic range is larger and the information acquisition dimension is more abundant, the quality is light, the imaging in the backlight scene can be more efficiently performed, and the application can be applied to many scenes with high weight requirements.

[0103] As Figure 6 shown, Figure 6 is a structural schematic diagram of an embodiment of the storage medium provided by the application. At least one computer program 31 is stored in the storage medium 30, the computer program 31 is executed by a processor to realize the method as shown in Figure 4 and Figure 5 the above description, which will not be repeated here. In an embodiment, the storage medium 30 can be a storage chip, a hard disk or a mobile hard disk or an optical disc, and other readable and writable storage tools, and can also be a server and the like.

[0104] The above describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily have to be implemented in the specific order shown or in a continuous order to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0105] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment mainly explains the difference from other embodiments. Especially, for the system and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0106] The system, storage medium and method provided in the embodiments of the present specification are corresponding, and therefore the system and storage medium also have similar beneficial technical effects to the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding system and storage medium will not be described here again.

[0107] The system, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an electronic mail device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0108] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, the embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] The present specification is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to the embodiments of the present specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the flow Figure 1 The function of one flow or multiple flows and / or blocks Figure 1 The function of one block or multiple blocks.

[0110] These computer program instructions can also be stored in a computer-readable memory (storage medium) that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the flow Figure 1 The function of one flow or multiple flows and / or blocks Figure 1 The function of one block or multiple blocks.

[0111] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1

[0112] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0113] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.

[0114] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition provided herein, a computer readable medium does not include transitory media, such as modulated data signals and carrier waves.

[0115] It is also noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0116] ​​The specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of steps and methods described in this specification is not the only manner in which the disclosed methods can be carried out. Likewise, the use of particular programming languages, protocols, and protocols is not intended to limit the scope of the disclosure. In fact, the methods are not necessarily limited to any particular programming language, protocol, or protocol.

[0117] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the system embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the part of the method embodiments.

[0118] The above disclosure is only the preferred embodiment of the present application, and of course cannot limit the scope of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. A multi-dimensionally modulated large dynamic range imaging device, characterized by, The device includes: The imaging lens group is used to focus on the target scene and correct aberrations, and output the light signal of the target scene; A silicon-based planar optical field modulation device is used to receive the target scene optical signal, perform spectral polarization modulation on the target scene optical signal, and acquire a four-state polarization image. Specifically, receiving the target scene optical signal, performing polarization modulation on the target scene optical signal, and acquiring the four-state polarization image includes: receiving the target scene optical signal; acquiring the arbitrary diffraction order correlation Fourier-Jones matrix coefficients of the planar optical field polarization modulation device based on the phase delay caused by each metal nanopillar to the target scene optical signal; determining the required four-state polarization diffraction order intensity based on the arbitrary diffraction order correlation Fourier-Jones matrix coefficients; diffracting the target scene optical signal onto the required four-state polarization diffraction order intensity until the required four-state polarization diffraction order intensity is uniformly distributed on the image plane of the planar optical field polarization modulation device, and the sum of the required four-state polarization diffraction order intensity reaches its maximum value, thereby acquiring a four-state polarization image. A multi-dimensional electric field modulation device is used to perform photoelectric conversion on the four-state polarization image to obtain polarization intensity information and event stream of the four-state polarization image. Specifically, the photoelectric conversion of the four-state polarization image to output the polarization intensity information and event stream includes: continuously performing photoelectric conversion on the four-state polarization image at a preset exposure cycle to obtain the global light intensity and the dynamic light intensity of the four-state polarization image at pixel positions; determining the full Stokes vector of the target scene based on the global light intensity, and using the full Stokes vector as the polarization intensity information of the four-state polarization image; determining that the dynamic light intensity change at pixel positions of the four-state polarization image is greater than or equal to a preset event threshold, and then obtaining polarization-modulated event information; determining the occurrence of each event based on the event information, and obtaining an event stream based on the occurrence of each event.

2. The multi-dimensionally modulated large dynamic range imaging device of claim 1, wherein, The silicon-based planar optical field modulation device includes a planar optical field polarization modulation device and a filter film. The filter film is used to receive the target scene light signal, perform wavelength selection on the target scene light signal, and output a light signal with a preset wavelength; The planar optical field polarization modulation device includes several metal nanopillars arranged according to matrix Fourier optical design. The size of each metal nanopillar is between 20-70 nm. It is used to perform spectral polarization modulation on the optical signal of the preset wavelength to obtain a four-state polarization image.

3. The multi-dimensionally controlled large dynamic range imaging device of claim 2, wherein, The multidimensional electric field control device specifically includes: An intensity image readout module, including an integral readout circuit, is used to obtain the intensity information of the four-state polarization image containing the full Stokes vector description of the target scene based on the global light intensity of the four-state polarization image; The event information reading module includes a threshold gating circuit, which is used to obtain the dynamic information of the four-state polarization image based on the dynamic light intensity at the pixel position of the four-state polarization image, and to obtain the event stream based on the dynamic information.

4. The multi-dimensionally controlled large dynamic range imaging device of claim 3, wherein, The cascaded integral readout circuit and threshold gating circuit share a photodiode at each pixel, which is used to continuously perform photoelectric conversion on the four-state polarization image at a preset exposure cycle to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

5. The multi-dimensionally controlled large dynamic range imaging device of claim 4, wherein, The device further includes: A field stop is used to limit the imaging range of the imaging lens group.

6. A multi-dimensional controlled large dynamic range imaging method, characterized in that, The method includes: Focus and correct aberrations of the target scene, and output the target scene light signal; Receiving the target scene optical signal, performing polarization modulation on the target scene optical signal, and acquiring a four-state polarization image specifically includes: receiving the target scene optical signal; obtaining the arbitrary diffraction order related Fourier-Jones matrix coefficients of the planar optical field polarization control device based on the phase delay caused by each metal nanopillar to the target scene optical signal; determining the required four-state polarization diffraction order light intensity based on the arbitrary diffraction order related Fourier-Jones matrix coefficients; diffracting the target scene optical signal onto the required four-state polarization diffraction order light intensity until the required four-state polarization diffraction order light intensities are uniformly distributed on the image plane of the planar optical field polarization control device, and the sum of the required four-state polarization diffraction order light intensities reaches its maximum value, thereby acquiring a four-state polarization image; The photoelectric conversion of the four-state polarization image is performed to obtain polarization intensity information and event stream of the four-state polarization image. Specifically, the photoelectric conversion of the four-state polarization image and the output of the polarization intensity information and event stream include: continuously performing photoelectric conversion on the four-state polarization image at a preset exposure cycle to obtain the global light intensity and the dynamic light intensity of the four-state polarization image at pixel positions; determining the full Stokes vector of the target scene based on the global light intensity, and using the full Stokes vector as the polarization intensity information of the four-state polarization image; determining that the dynamic light intensity change at pixel positions of the four-state polarization image is greater than or equal to a preset event threshold, and then obtaining polarization-modulated event information; determining the occurrence of each event based on the event information, and obtaining an event stream based on the occurrence of each event.

7. The multi-dimensional controlled large dynamic range imaging method according to claim 6, characterized in that, The step of receiving the target scene optical signal and obtaining the arbitrary diffraction order related Fourier-Jones matrix coefficients of the planar optical field polarization control device based on the phase delay caused by each of the metal nanopillars to the target scene optical signal specifically includes: According to the above The phase delay caused by each of the metal nanopillars to the optical signal of the target scene is determined, where (x, y) represents the local coordinate axes of the nanopillars. Let be the transmission coefficient of the nanopillar along its transverse axis. Let θ be the transmission coefficient of the nanopillar along its longitudinal axis, and θ be the direction of rotation of the nanopillar relative to the reference coordinate system. according to Determine the coefficients of the arbitrary diffraction order related Fourier-Jones matrix of the planar optical field polarization control device, where (l,h) is the diffraction order, L is the length of a single period of the planar optical field polarization control device, and H is the width of a single period of the planar optical field polarization control device. , , , , where are the spatial frequencies of each nanopillar in the (x, y) direction, respectively.

8. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 6 to 7.

Images