Multi-dimensional regulation and control large dynamic range imaging device and method and storage medium

Abstract

The embodiment of the invention discloses a multi-dimensional regulation and control large dynamic range imaging device and method and a storage medium, and the device comprises an imaging lens group which is used for carrying out the focusing and aberration correction of a target scene, and outputting a target scene optical signal; the silicon-based planar light field regulation and control device is used for receiving the target scene light signal and performing spectrum polarization modulation on the target scene light signal to obtain a four-state polarization image; and the multi-dimensional electric field regulation and control device is used for performing photoelectric conversion on the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image. The target scene is effectively focused and subjected to aberration correction through the imaging lens group, the optical signal of the target scene is output, the output optical signal of the target scene is regulated and controlled only by adopting the silicon-based planar optical field regulation and control device, a large number of optical devices are not needed, and the integration level is high. Meanwhile, polarization intensity information and event information of a four-state polarization image are accurately obtained by adopting a multi-dimensional electric field regulation and control device, and a larger dynamic range and rich information obtaining dimensions are achieved.

Description

Technical Field

[0001] The present invention relates to the field of optoelectronic imaging technology, and in particular to a multi-dimensionally controlled large dynamic range imaging device, method and storage medium. Background Art

[0002] In the process of space exploration and natural environment imaging, the common backlighting scene will cause the camera to overexpose, resulting in image quality degradation and loss of foreground target information, which limits the application scenarios of the imaging system. In order to achieve better imaging effects in backlighting scenes, an imaging device with a large dynamic range is required.

[0003] Polarization imaging obtains the full Stokes vector of the target scene by acquiring the polarization information of the target, and describes the reflection and scattering characteristics of the object surface through the polarization feature information. It can analyze the surface morphology, stress, texture, roughness and other characteristics of the foreground target, improve the image feature information, and at the same time, due to the lack of polarization of the background strong light, the background light can be effectively suppressed. Existing polarization photoelectric imaging cameras can be roughly divided into time-division type, amplitude-division type, focal plane-division type, aperture-division type, etc. according to their different imaging methods. The time-division type has poor effect in moving scenes, and the imaging element needs to be adjusted frequently. Mechanical movement will also cause the light beam to drift, resulting in reduced imaging accuracy. The amplitude-division type is large and complex, and the response between each detector needs to be consistent, which is easy to cause errors. There is parallax between the divisions of the focal plane type, which affects the later registration. There is crosstalk between the divisions of the aperture type, which causes the image quality to deteriorate. Summary of the invention

[0004] Based on this, it is necessary to propose a multi-dimensionally controlled large dynamic range imaging device, method and storage medium to address the above problems.

[0005] A multi-dimensionally controlled large dynamic range imaging device, the device comprising:

[0006] The imaging lens group is used to focus and correct the aberration of the target scene and output the target scene light signal.

[0007] The silicon-based planar light field control device is used to receive the target scene light signal, perform spectral polarization modulation on the target scene light signal, and obtain a four-state polarization image.

[0008] The multi-dimensional electric field control device is used to perform photoelectric conversion on the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image.

[0009] The silicon-based planar light field control device includes a planar light field polarization control device and a filter film.

[0010] The filter film is used to receive the target scene light signal, perform wavelength selection on the target scene light signal, and output a light signal with a preset wavelength.

[0011] The planar light field polarization control device includes a plurality of metal nanocolumns arranged according to a matrix Fourier optical design, each of which has a size between 20 and 70 nm, and is used to perform spectral polarization modulation on the light signal of the preset wavelength to obtain a four-state polarization image.

[0012] Wherein, the multi-dimensional electric field control device specifically includes:

[0013] The intensity image readout module comprises an integral readout circuit, which is used to obtain the intensity information of the four-state polarization image containing the full Stokes vector description of the target scene according to the global light intensity of the four-state polarization image.

[0014] The event information readout module includes a threshold gating circuit, which is used to obtain dynamic information of the four-state polarization image according to the dynamic light intensity of the four-state polarization image at the pixel position, and obtain the event flow according to the dynamic information.

[0015] The integral readout circuit and the threshold selection circuit are cascaded and share the photodiode at each pixel, and are used to continuously perform photoelectric conversion on the four-state polarization image with a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0016] Wherein, the device further comprises:

[0017] The field stop is used to limit the imaging range of the imaging lens assembly.

[0018] A multi-dimensionally controlled large dynamic range imaging method, the method comprising:

[0019] The target scene is focused and aberration corrected, and the target scene light signal is output.

[0020] The target scene light signal is received, polarization modulation is performed on the target scene light signal, and a four-state polarization image is acquired.

[0021] The four-state polarization image is subjected to photoelectric conversion to obtain polarization intensity information and event flow of the four-state polarization image.

[0022] The receiving of the target scene light signal, performing polarization modulation on the target scene light signal, and acquiring a four-state polarization image specifically includes:

[0023] The target scene light signal is received, and the Fourier Jones matrix coefficients related to any diffraction order of the planar light field polarization control device are obtained according to the phase delay caused by each of the metal nanorods to the target scene light signal.

[0024] The required four-state polarization diffraction order light intensities are determined according to the Fourier Jones matrix coefficients related to the arbitrary diffraction order.

[0025] The target scene light signal is diffracted to the required four-state polarization diffraction order light intensities until the required four-state polarization diffraction order light intensities are uniformly distributed on the image plane of the planar light field polarization control device, and the sum of the required four-state polarization diffraction order light intensities reaches a maximum value, thereby acquiring a four-state polarization image.

[0026] The receiving of the target scene light signal and obtaining Fourier Jones matrix coefficients related to any diffraction order of the planar light field polarization control device according to the phase delay caused by each of the metal nanorods to the target scene light signal specifically include:

[0027] According to the Determine the phase delay caused by each of the metal nanorods to the target scene light signal, where (x, y) represents the local coordinate axis of the nanorod, f x The transmission coefficient of the nanorod along its horizontal axis, f y is the transmission coefficient of the nanorod along its longitudinal axis, and θ is the direction of rotation of the nanorod relative to the reference coordinate system.

[0028] according to Determine the Fourier Jones matrix coefficients related to any diffraction order of the plane light field polarization control device, where L is the length of a single period of the plane light field polarization control device, and H is the width of a single period of the plane light field polarization control device, wherein:

[0029] The step of performing photoelectric conversion on the four-state polarization image to output polarization intensity information and event stream of the four-state polarization image specifically includes:

[0030] The photoelectric conversion of the four-state polarization image is continuously performed in a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0031] A full Stokes vector of the target scene is determined according to the global light intensity, and the full Stokes vector is used as polarization intensity information of the four-state polarization image.

[0032] It is determined that the dynamic light intensity variation of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, and then the event information after polarization modulation is obtained.

[0033] The occurrence of each event is determined according to the event information, and an event stream is acquired based on the occurrence of each event.

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method described above.

[0035] The embodiments of the present invention have the following beneficial effects:

[0036] The present invention effectively focuses and corrects the aberration of the target scene through the imaging lens group, outputs the light signal of the target scene, and only uses the silicon-based planar light field control device to control the output light signal of the target scene. By utilizing the non-selectivity of polarization modulation to the background natural strong light, the background strong light information can be effectively suppressed, and the foreground target can be highlighted. The structure is simple and compact, and a large number of optical devices are not required. The integration is high. In addition, a multi-dimensional electric field control device is used to quickly and accurately obtain the polarization intensity information and event information of the four-state polarization image, which has a larger dynamic range and richer information acquisition dimensions. It is light in weight and can more efficiently form images in backlit scenes. It can be applied to many scenes with high weight requirements, which is of great significance for subsequent image analysis, target recognition, event detection and other applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] in:

[0039] Figure 1 A schematic structural diagram of an embodiment of a multi-dimensionally controlled large dynamic range imaging device provided by the present invention;

[0040] Figure 2 A schematic structural diagram of another embodiment of a multi-dimensionally controlled large dynamic range imaging device provided by the present invention;

[0041] Figure 3 A schematic structural diagram of an embodiment of a multi-dimensional electric field control device provided by the present invention;

[0042] Figure 4 A schematic diagram of a flow chart of an embodiment of a multi-dimensionally controlled large dynamic range imaging method provided by the present invention;

[0043] Figure 5A schematic flow chart of another embodiment of a multi-dimensionally controlled large dynamic range imaging method provided by the present invention;

[0044] Figure 6 A schematic structural diagram of an embodiment of a storage medium provided by the present invention. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] like Figure 1 As shown, Figure 1 A schematic diagram of a structure of an embodiment of a multi-dimensionally controlled large dynamic range imaging device provided by the present invention. A multi-dimensionally controlled large dynamic range imaging device 10, the device comprising:

[0047] The imaging lens group 11 is used to image the target scene and perform aberration correction, and output a target scene light signal.

[0048] For example, Figure 2 A schematic diagram of the structure of another embodiment of a multi-dimensionally controlled large dynamic range imaging device provided by the present invention. A multi-dimensionally controlled large dynamic range imaging device also includes a field of view aperture 14. The imaging lens group is usually composed of multiple lenses. After the light information of the target scene is focused and aberration corrected by the lens, the target scene light signal is output and focused on the image plane or focal plane to form a clear image. The field of view aperture 14 is located between the imaging lens group 11 and the silicon-based planar light field control device 12, and is used to limit the imaging range of the imaging lens group so that crosstalk does not occur between the four-state polarization state images after passing through the silicon-based planar light field control device 12.

[0049] The silicon-based planar light field control device 12 includes a plurality of metal nanocolumns arranged according to a matrix Fourier optical design, each of which has a size between 20 and 70 nm, and is used to perform spectral polarization modulation on the optical signal of the preset wavelength to obtain a four-state polarization image.

[0050] Exemplarily, the silicon-based planar light field control device 12 is precisely located in the image plane or focal plane of the imaging lens group 11 to perform spectral polarization modulation on the light information of the target scene. The silicon-based planar light field control device 12 is composed of a planar light field polarization control device and a filter film. Among them, the silicon-based planar light field control device is composed of TiO2 metal nanocolumns, each of which has a size between 20-70μm and is arranged according to a matrix Fourier optical design with an arrangement period of 1.9μm. The filter film is a filter film with a central transmission wavelength of 480nm, which is coated on the planar light field polarization control device to limit the influence of the dispersion effect of the planar light field polarization control device on the image quality.

[0051] Specifically, the filter film receives the target scene light signal, performs wavelength selection on the target scene light signal, and outputs a light signal with a wavelength of 480nm. Furthermore, the planar light field polarization control device performs spectral polarization modulation on the 480nm light signal to obtain a four-state polarization image.

[0052] The multi-dimensional electric field control device 13 is used to perform photoelectric conversion on the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image.

[0053] For example, referring to Figure 3 , Figure 3 A schematic diagram of the structure of an embodiment of a multi-dimensional electric field control device provided by the present invention. The multi-dimensional electric field control device 13 is precisely located on the image plane of the silicon-based planar light field control device 12, and is used to perform photoelectric conversion and electric field control on the light information of the target scene, and perform high dynamic range imaging. Specifically, it includes an intensity image readout module and an event information readout module. The intensity image readout module is mainly composed of an integral readout circuit 131, which is used to obtain the intensity information of the four-state polarization image containing the full Stokes vector description of the target scene according to the global light intensity of the four-state polarization image; the event information readout module is mainly composed of a threshold selection circuit 132, which is used to obtain the dynamic information of the four-state polarization image according to the dynamic light intensity of the four-state polarization image at the pixel position, and obtain the event flow according to the dynamic information. The integral readout circuit 131 and the threshold selection circuit 132 are cascaded and share the photodiode 133 at each pixel, which is used to continuously perform photoelectric conversion on the four-state polarization image with a preset exposure period, and obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0054] It can be known from the above description that the present invention effectively focuses and corrects the aberration of the target scene through the imaging lens group, outputs the optical signal of the target scene, and only uses the silicon-based planar light field control device to control the output optical signal of the target scene. By utilizing the non-selectivity of polarization modulation to the background natural strong light, the background strong light information can be effectively suppressed, and the foreground target can be highlighted. The structure is simple and compact, and a large number of optical devices are not required, and the integration is high. In addition, a multi-dimensional electric field control device is used to quickly and accurately obtain the polarization intensity information and event information of the four-state polarization image, which has a larger dynamic range and richer information acquisition dimensions, is light in weight, and can more efficiently image in backlit scenes. It can be applied to many scenes with high weight requirements, which is of great significance for subsequent image analysis, target recognition, event detection and other applications.

[0055] like Figure 4 As shown, Figure 4 A schematic diagram of a flow chart of an embodiment of a multi-dimensionally controlled large dynamic range imaging method provided by the present invention. A multi-dimensionally controlled large dynamic range imaging method, the method comprising:

[0056] S101: focusing and aberration correction are performed on the target scene, and the target scene light signal is output.

[0057] Exemplarily, the light information of the target scene is focused and aberration corrected by the imaging lens group 11, and then the target scene light signal is output and further focused on the image plane or focal plane of the silicon-based planar light field control device 12 to form a clear image.

[0058] S102: Receive a target scene light signal, perform polarization modulation on the target scene light signal, and obtain a four-state polarization image.

[0059] Exemplarily, a target scene light signal is received, and the Fourier Jones matrix coefficients associated with any diffraction order of the plane light field polarization control device are obtained according to the phase delay caused by each metal nanocolumn to the target scene light signal; the required four-state polarization diffraction order light intensity is determined according to the Fourier Jones matrix coefficients associated with any diffraction order. The target scene light signal is diffracted to the required four-state polarization diffraction order light intensity until the required four-state polarization diffraction order light intensity is evenly distributed on the plane light field polarization control device, so that the sum of the required four-state polarization diffraction order light intensity reaches the maximum value, and a four-state polarization image is obtained.

[0060] S103: Perform photoelectric conversion on the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image.

[0061] Exemplarily, the four-state polarization image is continuously subjected to photoelectric conversion with a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position; the full Stokes vector of the target scene is determined according to the global light intensity, and the full Stokes vector is used as the polarization intensity information of the four-state polarization image; it is determined that the dynamic light intensity change of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, and then the event information after polarization modulation is obtained; the occurrence of each event is determined according to the event information, and the event stream is obtained based on the occurrence of each event.

[0062] like Figure 5 As shown, Figure 5 A schematic diagram of another embodiment of a multi-dimensionally controlled large dynamic range imaging method provided by the present invention. A multi-dimensionally controlled large dynamic range imaging method, the method comprising:

[0063] S201: Clearly image the target scene and perform aberration and chromatic aberration correction, and output the target scene light signal.

[0064] It should be noted that step S201 Figure 3 This has been discussed in detail in the implementation scenario shown and will not be repeated here.

[0065] S202: Receive a target scene light signal, and obtain Fourier Jones matrix coefficients related to any diffraction order of the planar light field polarization control device according to the phase delay caused by each metal nanorod to the target scene light signal.

[0066] Exemplarily, the phase delay caused by each metal nanorod to the target scene light signal is determined according to the following formula:

[0067]

[0068] Where (x, y) represents the local coordinate axis of the nanorod, and f x The transmission coefficient of the nanorod along its horizontal axis, f y is the transmission coefficient of the nanorod along its longitudinal axis, and θ is the direction of rotation of the nanorod relative to the reference coordinate system.

[0069] Furthermore, the Fourier Jones matrix coefficients related to any diffraction order of the planar light field polarization control device are determined according to the following formula:

[0070]

[0071] Where (l,h) is the diffraction order, L is the length of a single period of the plane light field polarization control device, and H is the width of a single period of the plane light field polarization control device. are the spatial frequencies of each nanorod in the (x, y) direction, respectively.

[0072] S203: Determine the required four-state polarization diffraction order light intensity according to the Fourier Jones matrix coefficients related to any diffraction order.

[0073] Exemplarily, the required four-state polarization diffraction order light intensity is as shown in the formula:

[0074]

[0075] Among them, I (l,h) is the required four-state polarization diffraction order intensity, δ is the polarization state, for The Hermitian conjugation of are the Fourier Jones matrix coefficients associated with any diffraction order. S204: diffract the target scene light signal to the required four-state polarization diffraction order light intensities until the required four-state polarization diffraction order light intensities are evenly distributed on the image plane of the planar light field polarization control device, and the sum of the required four-state polarization diffraction order light intensities reaches a maximum value, thereby obtaining a four-state polarization image.

[0076] Exemplarily, the target scene light signal is diffracted to the required four-state polarization diffraction order light intensities, and the remaining diffraction order light intensities are constrained to be zero, forming an optimization problem and solving iteratively using a gradient descent algorithm until the required four-state polarization diffraction order light intensities are uniformly distributed on the image plane of the planar light field polarization control device, so that the sum of the required four-state polarization diffraction order light intensities reaches a maximum value, thereby acquiring a four-state polarization image.

[0077] S205: Continuously perform photoelectric conversion on the four-state polarization image at a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0078] Exemplarily, the four-state polarization image is continuously photoelectrically converted through the photodiode at each pixel with an exposure period of 100 ms to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

[0079] S206: Determine the full Stokes vector of the target scene according to the global light intensity, and use the full Stokes vector as polarization intensity information of the four-state polarization image.

[0080] Exemplarily, the global light intensity of the four-state polarization state image is expressed by the following formula:

[0081]

[0082] in, is the global light intensity of the four-state polarization image received by the multi-dimensional electric field control device, I1, I2, I3, I4 represent the four-state polarization image intensity, λ is the spectral modulation parameter introduced by the plane light field polarization control device, then the full Stokes vector of the target scene It can be expressed as:

[0083]

[0084] Wherein, M is a system polarization parameter, which is given by the system and is not specifically limited here.

[0085] S207: Determine whether the dynamic light intensity change of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, and then obtain event information after polarization modulation.

[0086] Exemplarily, the event information readout module and the intensity image readout module work simultaneously to capture the dynamic information after polarization modulation in the target scene. Specifically, at time ts, the dynamic information after polarization modulation is as shown in the following formula:

[0087]

[0088] Among them, L is the event information, is the dynamic light intensity of the four-state polarization image at each pixel position, which can be specifically expressed as:

[0089]

[0090] Where λ is the spectral modulation parameter introduced by the plane light field polarization control device, is the pixel position at the corresponding pixel (x i ,y i ) is the dynamic light intensity of the four-state polarization image at .

[0091] Further, it is determined that the dynamic light intensity change of the four-state polarization image at the pixel position is greater than or equal to the preset event threshold, and the event information after polarization modulation is:

[0092]

[0093] Where δ is the event polarity, Respectively represent the positive and negative events that occurred in the four polarization states, Δt s is the event interval between two events, and C is the preset event threshold.

[0094] S208: Determine the occurrence of each event according to the event information, and obtain the event stream based on the occurrence of each event.

[0095] For example, each event occurrence can be recorded as:

[0096]

[0097] in, is the i-th event occurrence.

[0098] Finally, the event stream output can be obtained, represented as:

[0099]

[0100] in, is an event stream, i∈[1,n].

[0101] From the above description, it can be seen that the present invention adopts a silicon-based planar light field control device to control the target light information, and the full Stokes vector information of the target scene can be obtained in a single exposure, with high real-time performance. In addition, a combination of a silicon-based planar light field control device and a multi-dimensional electric field control device is adopted to integrate the multi-dimensional control of light-electric field, so that spectral polarization information and event information can be obtained simultaneously. It has a larger dynamic range and richer information acquisition dimensions, is light in weight, can more efficiently form images in backlit scenes, and can be applied to many scenes with high weight requirements.

[0102] like Figure 6 As shown, Figure 6 The storage medium 30 stores at least one computer program 31, which is executed by a processor to implement the following. Figure 4 and Figure 5 In one embodiment, the storage medium 30 may be a storage chip, a hard disk, a mobile hard disk, a USB flash drive, an optical disk, or other readable and writable storage tools, or a server, etc.

[0103] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0105] The system, storage medium and method provided in the embodiments of this specification correspond to each other. Therefore, the system and storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding system and storage medium will not be repeated here.

[0106] The systems, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0108] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory (storage medium) that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0112] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0113] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0116] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0117] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A multi-dimensionally controlled large dynamic range imaging device, characterized in that: The device comprises: An imaging lens group, used for focusing and aberration correction of the target scene, and outputting the target scene light signal; A silicon-based planar light field control device, used to receive the target scene light signal, perform spectral polarization modulation on the target scene light signal, and obtain a four-state polarization image; The multi-dimensional electric field control device is used to perform photoelectric conversion on the four-state polarization image to obtain polarization intensity information and event flow of the four-state polarization image.

2. The multi-dimensionally controlled large dynamic range imaging device according to claim 1, characterized in that: The silicon-based planar light field control device includes a planar light field polarization control device and a filter film. The filter film is used to receive the target scene light signal, perform wavelength selection on the target scene light signal, and output a light signal with a preset wavelength; The planar light field polarization control device includes a plurality of metal nanocolumns arranged according to a matrix Fourier optical design, each of which has a size between 20 and 70 nm, and is used to perform spectral polarization modulation on the light signal of the preset wavelength to obtain a four-state polarization image.

3. The multi-dimensionally controlled large dynamic range imaging device according to claim 2, characterized in that: The multi-dimensional electric field control device specifically includes: An intensity image readout module, comprising an integral readout circuit, for acquiring intensity information of the four-state polarization image including a full Stokes vector description of a target scene according to the global light intensity of the four-state polarization image; The event information readout module includes a threshold gating circuit, which is used to obtain dynamic information of the four-state polarization image according to the dynamic light intensity of the four-state polarization image at the pixel position, and obtain the event flow according to the dynamic information.

4. The multi-dimensionally controlled large dynamic range imaging device according to claim 3, characterized in that: The integral readout circuit and the threshold selection circuit are cascaded and share the photodiode at each pixel, and are used to continuously perform photoelectric conversion on the four-state polarization image with a preset exposure period to obtain the global light intensity of the four-state polarization image and the dynamic light intensity of the four-state polarization image at the pixel position.

5. The multi-dimensionally controlled large dynamic range imaging device according to claim 4, characterized in that: The device also includes: The field stop is used to limit the imaging range of the imaging lens assembly.

6. A multi-dimensionally controlled large dynamic range imaging method, characterized in that: The method comprises: Focusing and aberration correction on the target scene, and outputting the target scene light signal; Receiving the target scene light signal, performing polarization modulation on the target scene light signal, and acquiring a four-state polarization image; The four-state polarization image is subjected to photoelectric conversion to obtain polarization intensity information and event flow of the four-state polarization image.

7. The multi-dimensionally controlled large dynamic range imaging method according to claim 6, characterized in that: The receiving the target scene light signal, performing polarization modulation on the target scene light signal, and acquiring a four-state polarization image specifically includes: Receiving the target scene light signal, and acquiring Fourier Jones matrix coefficients related to any diffraction order of the planar light field polarization control device according to the phase delay caused by each of the metal nanorods to the target scene light signal; Determine the required four-state polarization diffraction order light intensity according to the Fourier Jones matrix coefficients related to the arbitrary diffraction order; The target scene light signal is diffracted to the required four-state polarization diffraction order light intensities until the required four-state polarization diffraction order light intensities are uniformly distributed on the image plane of the planar light field polarization control device, and the sum of the required four-state polarization diffraction order light intensities reaches a maximum value, thereby acquiring a four-state polarization image.

8. The multi-dimensionally controlled large dynamic range imaging method according to claim 7, characterized in that: The receiving of the target scene light signal and obtaining Fourier Jones matrix coefficients related to any diffraction order of the planar light field polarization control device according to the phase delay caused by each of the metal nanorods to the target scene light signal specifically includes: According to the Determine the phase delay caused by each of the metal nanorods to the target scene light signal, where (x, y) represents the local coordinate axis of the nanorod, f x The transmission coefficient of the nanorod along its horizontal axis, f y is the transmission coefficient of the nanorod along its longitudinal axis, θ is the direction of rotation of the nanorod relative to the reference coordinate system; according to Determine the Fourier Jones matrix coefficients related to any diffraction order of the plane light field polarization control device, where L is the length of a single period of the plane light field polarization control device, and H is the width of a single period of the plane light field polarization control device, wherein:

9. The multi-dimensional controlled large dynamic range imaging method according to claim 6, characterized in that: The performing photoelectric conversion on the four-state polarization image to output the polarization intensity information and event stream of the four-state polarization image specifically includes: Continuously performing photoelectric conversion on the four-state polarization image at a preset exposure period to obtain a global light intensity of the four-state polarization image and a dynamic light intensity of the four-state polarization image at a pixel position; Determine a full Stokes vector of the target scene according to the global light intensity, and use the full Stokes vector as polarization intensity information of the four-state polarization image; Determining that the dynamic light intensity variation of the four-state polarization image at the pixel position is greater than or equal to a preset event threshold, then acquiring event information after polarization modulation; The occurrence of each event is determined according to the event information, and an event stream is acquired based on the occurrence of each event.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 6 to 9.

Images