CIS system for meta-imaging

Abstract

The present application relates to the field of computational imaging technology, and in particular to a CIS system for elemental imaging, which includes: a main imaging system, a microlens array, an image acquisition unit, a first mask arranged before and after the microlens array, including on the microlens array, and a second mask arranged near the main imaging system, including on the main imaging lens and before the microlens array, wherein the main imaging system is used to obtain a first image of a target object; the first mask is used to amplitude modulate the first image, and the microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object; the first mask is used to mask modulate the second image using a preset first mask modulation function, introducing more uniform spectral aliasing, and obtaining light field information of the target object, so that the image acquisition unit generates a final image based on the light field information. Therefore, the light field acquisition method based on two microlens arrays with added masks improves sampling efficiency and light field imaging and reconstruction quality, and also solves the problems of high cost and large volume when a camera array adds a mask between the camera lens and the sensor to achieve light field acquisition.

Description

Technical Field

[0001] The present application relates to the field of computational imaging technology, and in particular to a CIS (CMOS Image Sensor) system for elemental imaging. Background Art

[0002] In recent years, with the advancement of computer computing power, computational light field imaging has rapidly developed. Related technologies primarily focus on the distribution of light as a function of position and angle. Thanks to the spatial-perspective dimensionality assumption and advances in hardware and software, light field imaging can now be implemented in portable commercial devices, even integrated into mobile phones.

[0003] The widespread use of light field cameras has enabled a host of new applications, initially enabling light field rendering based on photorealistic images, and now enabling computer vision applications that leverage light field-encoded information, such as 3D reconstruction, segmentation, saliency detection, object detection and recognition, tracking, and video stabilization. Given its enormous potential and the technical challenges yet to be overcome, light field technology has garnered widespread attention from research teams both domestically and internationally.

[0004] The acquisition of 4D light field information is mainly achieved by multiplexing the angular domain into the spatial (or frequency) domain, encoding the 4D light field onto the 2D sensor plane. The light field acquisition device is called a "light field chip", which is mainly implemented using a microlens array or a small lens array mounted on an image sensor. Light field cameras can obtain spatial and angular 4D light field information through a single exposure, but light field chips face an inherent contradiction between spatial resolution and angular resolution. On the one hand, light field acquisition devices based on the principle of microlens arrays (such as Lytro Illume and Raytrix) obtain light field images with high angular resolution by directly dispersing light from various angles onto the imaging unit, but are limited by the resolution of the imaging unit, and their spatial resolution is often much smaller than that of today's more common imaging devices (SLR cameras, mobile phones, etc.); on the other hand, such as Figure 1 As shown in the figure, although the camera array system can control the size of the light field spatial resolution by changing the resolution of each camera, it is limited by factors such as the physical space of the camera, system complexity, and overall cost. Its angular resolution is far from meeting the requirements of real-time light field rendering.

[0005] Signal sampling in related technologies follows the Nyquist sampling theorem. This sampling method collects a large amount of data and requires a large amount of transmission bandwidth and storage space. The scanning light field imaging method improves the spatial resolution of light field imaging while ensuring angular resolution by sacrificing temporal resolution. However, existing scanning imaging methods have many scans and low imaging efficiency. During the acquisition process, there is a problem of spectrum aliasing leading to information loss and reduced imaging quality. Under the premise that the sensor resolution remains unchanged, compressed sensing technology can greatly improve the low spatial resolution of the light field captured by non-compressed light field. Based on the compressed light field shooting technology, a random mask is added between the camera lens and the sensor to achieve compressed acquisition of the target light field. Combining compressed sensing theory with an overcomplete light field dictionary to perform nonlinear optimization of the compressed encoded image, the dense light field can be restored offline. One way to achieve compressed coding is to add mask modulation during the imaging process to obtain better sampling. Summary of the Invention

[0006] The present application provides a CIS system for elemental imaging, which introduces more uniform spectral aliasing through mask modulation to solve the problem of degraded light field reconstruction quality caused by information aliasing in existing microlens array light field acquisition systems.

[0007] The first embodiment of the present application provides a CIS system for elemental imaging, comprising: a main imaging system, a microlens array, an image acquisition unit, and a first mask disposed before and after the microlens array, including on the microlens array, wherein:

[0008] The main imaging system is used to obtain a first image of the target object;

[0009] The microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object;

[0010] The first mask is used to perform mask modulation on the second image through a preset first mask modulation function, introduce preset spectrum aliasing, and obtain light field information of the target object, so that the image acquisition unit generates a final image according to the light field information.

[0011] Optionally, the above-mentioned CIS system for meta-imaging further includes:

[0012] A second mask is provided on the main imaging lens in the main imaging system and in front of the microlens array, wherein the second mask is used to perform mask modulation on the first image using a preset second mask modulation function to obtain a first image for phase modulation.

[0013] Optionally, the wave function of the first image is:

[0014]

[0015] Where x, y, and z are the three-dimensional spatial coordinates of the object point, z represents the depth, λ represents the wavelength, r represents the radial distance of the aperture plane, J0(·) represents the first-order zero-order Bessel function, s represents the distance between the main imaging system and the microlens array, d represents the distance between the object and the main imaging system, and D(r,λ,z) represents the change in imaging response at a distance z from the main imaging system. The expression is as follows:

[0016]

[0017] Optionally, the preset phase modulation function is:

[0018]

[0019] Where x0 and y0 are the center coordinates of the microlens array, f is the focal length, n is the refractive index, rect(.) is the rectangular window function, i is the imaginary part, and exp(.) is the exponential function.

[0020] Optionally, the preset first mask modulation function is:

[0021]

[0022] Wherein, m2(x, y) is a mask modulation function set before and after the microlens array, including on the microlens array and before the image acquisition unit.

[0023] Optionally, the light field information is:

[0024] U′(ω x ,ω y )=F ω (U0(x,y,z,λ)m1(r)·t(x,y,x0,y0));

[0025] Among them, ω x ,ω y is the frequency domain sampling of space (x, y), F ω (.) is a Fourier transform operation, and m1(r) is a second mask modulation function arranged near the main imaging system, including on the main imaging lens and in front of the microlens array.

[0026] Optionally, generating a final image on the image acquisition unit using the light field information includes:

[0027] Based on the imaging response, the light field information is used to generate a final image on the image acquisition unit, wherein the imaging response is:

[0028]

[0029] Among them, ω u is the spatial frequency position corresponding to angle u, ω v is the spatial frequency position corresponding to the angle v, s(ω u ,ω v ) is the acquisition process of specific frequency components by camera pixels.

[0030] Optionally, the image acquisition unit is a CMOS image sensor (CIS).

[0031] According to the CIS system for elemental imaging in the embodiment of the present application, a main imaging system is used to obtain a first image of a target object. A microlens array is used to phase-modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object. A first mask is used to mask-modulate the second image using a preset first mask modulation function, introducing a preset spectral aliasing to obtain light field information of the target object, so that the image acquisition unit generates a final image based on the light field information. Thus, the light field acquisition method based on two microlens arrays and the addition of a mask improves sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problems of high cost and large size when adding a mask between the camera lens and the sensor to achieve light field acquisition in a camera array.

[0032] A second embodiment of the present application provides an imaging method of a CIS system for meta-imaging, using the CIS system for meta-imaging as described in the first embodiment. The method includes the following steps:

[0033] obtaining a first image of the target object through the main imaging system;

[0034] Using the microlens array, phase-modulating the optical path of the first image by the phase modulation function to obtain a second image of the target object;

[0035] The first mask modulation is performed on the second image through the first mask modulation function to introduce preset spectrum aliasing to obtain light field information of the target object, and the light field information is converted into a final image on the image acquisition unit.

[0036] Optionally, the imaging method of the CIS system for elemental imaging further includes:

[0037] The first image is mask-modulated by using the second mask with a preset second mask modulation function to obtain a first image for phase modulation.

[0038] According to the imaging method of the CIS system for elemental imaging in the embodiment of the present application, a main imaging system is used to obtain a first image of the target object, a microlens array is used to phase-modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object, and a first mask is used to mask-modulate the second image using a preset first mask modulation function to obtain light field information of the target object, so that the image acquisition unit generates a final image based on the light field information. Thus, the light field acquisition method based on two microlens arrays and the addition of masks improves sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problems of high cost and large size when adding a mask between the camera lens and the sensor to achieve light field acquisition in a camera array.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0041] Figure 1 A schematic diagram of a series of conventional camera array light field masks in related technologies;

[0042] Figure 2 Schematic diagram of a block diagram of an imaging system of a CIS system for elemental imaging according to an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a microlens array mask CMOS image sensor system design according to one embodiment of the present application;

[0044] Figure 4 A schematic diagram of a CMOS image sensor system design comprising a microlens array and an optical lens combination mask according to one embodiment of the present application;

[0045] Figure 5 Flowchart of an imaging method of a CIS system for elemental imaging according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0047] The following describes a CIS system for meta-imaging according to an embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the background technology above, such as the high cost and large volume when the camera array adds a mask between the camera lens and the sensor to realize light field acquisition, the present application provides a CIS system for meta-imaging, which is used to obtain a first image of the target object through a main imaging system, and the microlens array is used to phase-modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object. The first mask is used to mask-modulate the second image using a preset first mask modulation function, introduce a preset spectrum aliasing, and obtain the light field information of the target object, so that the image acquisition unit generates a final image based on the light field information. Therefore, the light field acquisition method based on the addition of masks to two microlens arrays improves the sampling efficiency and the quality of light field imaging and reconstruction, and also solves the problems of high cost and large volume when the camera array adds a mask between the camera lens and the sensor to realize light field acquisition.

[0048] Specifically, Figure 2 A block diagram of a CIS system for elemental imaging provided in an embodiment of the present application.

[0049] like Figure 2 As shown, the CIS system 10 for elementary imaging includes: a main imaging system 100 , a microlens array 200 , an image acquisition unit 400 , and a first mask 300 disposed before and after the microlens array 200 , including on the microlens array 200 .

[0050] The main imaging system 100 is used to obtain a first image of the target object; the microlens array 200 is used to phase-modulate the optical path of the first image using a preset phase modulation function, introducing a preset spectral aliasing, and obtaining a second image of the target object; the first mask 300 is used to mask-modulate the second image using a preset first mask modulation function, introducing a preset spectral aliasing, and obtaining light field information of the target object, so that the image acquisition unit 400 generates a final image based on the light field information. Preferably, to improve image quality and reliability, the image acquisition unit 400 of the embodiment of the present application can utilize a CMOS image sensor. To meet the actual acquisition needs of more images, other image acquisition devices with image acquisition functions can also be used, and this is not specifically limited here.

[0051] Furthermore, in some embodiments, the wave function of the first image is:

[0052]

[0053] Where x, y, and z are the three-dimensional spatial coordinates of the object point, z represents the depth, λ represents the wavelength, r represents the radial distance of the aperture plane, J0(·) represents the first-order zero-order Bessel function, s represents the distance between the main imaging system and the microlens array, d represents the distance between the object and the main imaging system, and D(r,λ,z) represents the change in imaging response at a distance z from the main imaging system. The expression is as follows:

[0054]

[0055] Furthermore, in some embodiments, the preset phase modulation function is:

[0056]

[0057] Where x0 and y0 are the center coordinates of the microlens array, f is the focal length, n is the refractive index, rect(.) is the rectangular window function, i is the imaginary part, and exp(.) is the exponential function.

[0058] Furthermore, in some embodiments, the preset first mask modulation function is:

[0059]

[0060] Wherein, m2(x, y) is a mask modulation function provided before and after the microlens array 200 , including on the microlens array 200 and before the image acquisition unit 400 .

[0061] Specifically, compared to the related art in which the resolution of each camera is changed to control the size of the light field spatial resolution by using a camera array light field mask, but there are limitations such as the physical space of the camera, system complexity, and overall cost, the embodiment of the present application adopts two light field acquisition methods based on the microlens array 200 to increase the mask to improve the sampling efficiency and light field imaging quality.

[0062] As a feasible way, Figure 3 As shown, by adding a mask, namely a first mask 300, before and after the microlens array 200, including on the microlens array 200 and before the image acquisition unit 400, spectrum information shifting is achieved, frequency domain response zero points are reduced, and light field imaging quality and imaging resolution are improved.

[0063] As another possible way to achieve this, Figure 4As shown, by adding a mask, namely a first mask 300, before and after the microlens array 200, including on the microlens array 200 and in front of the image acquisition unit 400, and adding a mask, namely a second mask, near the main imaging system 100, including on the main imaging lens and in front of the microlens array 200, comprehensive light source modulation information and angle domain modulation information are obtained, thereby optimizing the imaging system response and improving the light field imaging quality and imaging resolution.

[0064] Specifically, in the embodiment of the present application, first, a first image of the target object is obtained through the main imaging system; secondly, the microlens array 200 uses a preset phase modulation function to phase modulate the optical path of the first image, thereby obtaining a second image of the target object.

[0065] The wave function of the first image passing through the primary imaging system can be expressed as:

[0066]

[0067] Where x, y, and z are the three-dimensional spatial coordinates of the object point, z means depth, λ is the wavelength, r is the radial distance of the aperture plane, and J0(·) is the first-kind zero-order Bessel function.

[0068] In the above formula, D(r,λ,z) can be expressed as:

[0069]

[0070] Where s is the distance between the main imaging system and the microlens array, and d is the distance between the object and the main imaging system.

[0071] The preset phase modulation function can be expressed as:

[0072]

[0073] Where x0 and y0 are the center coordinates of the microlens array, f is the focal length, n is the refractive index, rect(.) is the rectangular window function, i is the imaginary part, and exp(.) is the exponential function.

[0074] The preset first mask modulation function is:

[0075]

[0076] Wherein, m2(x, y) is a mask modulation function set before and after the microlens array, including on the microlens array and before the image acquisition unit 400.

[0077] Furthermore, in some embodiments, the above-mentioned CIS system 10 for elemental imaging also includes: a second mask arranged near the main imaging system 100, including on the main imaging lens and in front of the microlens array 200, the second mask being used to perform mask modulation on the first image using a preset second mask modulation function to obtain a first image for phase modulation.

[0078] Furthermore, in some embodiments, the light field information is:

[0079] U′(ω x ,ω y )=F ω (U0(x,y,z,λ)m1(r)·t(x,y,x0,y0));

[0080] Among them, ω x ,ω y is the frequency domain sampling of space (x, y), F ω (.) is a Fourier transform operation, and m1(r) is a second mask modulation function disposed near the main imaging system 100 , including on the main imaging lens and in front of the microlens array 200 .

[0081] Furthermore, in some embodiments, generating a final image on the image acquisition unit 400 using the light field information includes: generating a final image on the image acquisition unit 400 using the light field information based on the imaging response, wherein the imaging response is:

[0082]

[0083] Among them, ω u is the spatial frequency position corresponding to angle u, ω v is the spatial frequency position corresponding to the angle v, s(ω u ,ω v ) is the acquisition process of specific frequency components by camera pixels.

[0084] Specifically, the embodiment of the present application first utilizes a first mask 300 disposed before and after the microlens array 200, including on the microlens array 200, to perform mask modulation on the second image of the target object obtained above through a preset first mask modulation function through correlation calculation, and introduces a more uniform spectrum aliasing to avoid the degradation of light field reconstruction quality caused by information aliasing in the light field acquisition system of the microlens array 200, thereby obtaining light field information of the target object. This light field information includes four-dimensional position and direction information of light radiation during propagation, and solves problems such as image defocus in special occasions through digital refocusing technology; secondly, based on the imaging response, a final image is generated on the image acquisition unit 400 using the obtained light field information.

[0085] The light field information can be expressed as:

[0086] U′(ω x ,ω y )=F ω (U0(x,y,z,λ)m1(r)·t(x,y,x0,y0)); (5)

[0087] Among them, ω x ,ω y is the frequency domain sampling of space (x, y), F ω (.) is a Fourier transform operation, and m1(r) is a second mask modulation function disposed near the main imaging system 100 , including on the main imaging lens and in front of the microlens array 200 .

[0088] The imaging response can be expressed as:

[0089]

[0090] Among them, ω u is the spatial frequency position corresponding to angle u, ω v is the spatial frequency position corresponding to the angle v, s(ω u ,ω v ) is the acquisition process of specific frequency components by camera pixels.

[0091] In summary, the two aforementioned light field acquisition methods based on adding a mask to a microlens array not only reduce frequency domain zeros, but also achieve an optimal imaging system, ultimately improving the flexibility and resolution of the imaging system. It should be noted that the aforementioned mask placement is merely exemplary, and those skilled in the art can adjust the mask placement based on actual circumstances. For example, the second mask can be placed to the left of the main imaging system and in close proximity to the main imaging system, or the first mask can be placed between the main imaging system and the microlens array and in close proximity to the microlens array.

[0092] According to the CIS system for elemental imaging in the embodiment of the present application, a main imaging system is used to obtain a first image of a target object. A microlens array is used to phase-modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object. A first mask is used to mask-modulate the second image using a preset first mask modulation function, introducing a preset spectral aliasing to obtain light field information of the target object, so that the image acquisition unit generates a final image based on the light field information. Thus, the light field acquisition method based on two microlens arrays and the addition of a mask improves sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problems of high cost and large size when adding a mask between the camera lens and the sensor to achieve light field acquisition in a camera array.

[0093] Next, the microlens array imaging method proposed in accordance with the embodiments of the present application will be described with reference to the accompanying drawings.

[0094] Figure 5 4 is a flow chart of an imaging method of a CIS system for elemental imaging according to an embodiment of the present application.

[0095] like Figure 5 As shown, the imaging method of the CIS system for elemental imaging adopts the CIS system for elemental imaging as described above, and the method includes the following steps:

[0096] In step S501, a first image of a target object is obtained by a primary imaging system;

[0097] In step S502, a microlens array is used to phase-modulate the optical path of the first image by a phase modulation function to obtain a second image of the target object;

[0098] In step S503, the first mask modulation is performed on the second image by using a first mask modulation function to introduce a preset spectrum aliasing to obtain light field information of the target object, and the light field information is converted into a final image on the image acquisition unit.

[0099] Furthermore, in some embodiments, the imaging method of the CIS system for elemental imaging further includes:

[0100] The first image is mask-modulated by using a second mask with a preset second mask modulation function to obtain a first image for phase modulation.

[0101] According to the imaging method of the CIS system for elemental imaging in the embodiment of the present application, a main imaging system is used to obtain a first image of the target object, a microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object, and a first mask is used to mask-modulate the second image using a preset first mask modulation function, introducing a preset spectral aliasing to obtain light field information of the target object, so that the image acquisition unit generates a final image based on the light field information. Thus, the light field acquisition method based on two microlens arrays and the addition of masks improves sampling efficiency and the quality of light field imaging and reconstruction. It also solves the problems of high cost and large size when adding a mask between the camera lens and the sensor to achieve light field acquisition in a camera array.

[0102] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0104] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0105] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0106] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0107] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0108] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0109] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A CIS system for meta-imaging, characterized in that: include: A main imaging system, a microlens array, an image acquisition unit, and a first mask arranged before and after the microlens array, including a first mask on the microlens array, wherein: The main imaging system is used to obtain a first image of the target object; The microlens array is used to phase modulate the optical path of the first image using a preset phase modulation function to obtain a second image of the target object; The first mask is used to perform mask modulation on the second image using a preset first mask modulation function, introducing a preset spectrum aliasing, and obtaining light field information of the target object, so that the image acquisition unit generates a final image according to the light field information; Also includes: a second mask provided on a main imaging lens in the main imaging system and in front of the microlens array, the second mask being used to perform mask modulation on the first image using a preset second mask modulation function to obtain a first image for phase modulation; The wave function of the first image is: Where x, y, and z are the three-dimensional spatial coordinates of the object point, z means depth, λ is the wavelength, r is the radial distance of the aperture plane, J0(·) is the first-kind zero-order Bessel function, s is the distance between the main imaging system and the microlens array, d is the distance between the target object and the main imaging system, and D(r, λ, z) is the imaging response change modeled at a distance z from the main imaging system, expressed as follows: The preset phase modulation function is: Where x0, y0 are the center coordinates of the microlens array, f is the focal length, n is the refractive index, rect(.) is the rectangular window function, i is the imaginary part, and exp(.) is the exponential function; The preset first mask modulation function is: Wherein, m2(x,y) is the mask modulation function set before and after the microlens array, including on the microlens array and before the image acquisition unit; The light field information is: U′(ω x ,ω y )=F ω (U0(x,y,z,λ)m1(r)·t(x,y,x0,y0))! Among them, ω x ,ω y is the frequency domain sampling of space (x, y), F ω (.) is a Fourier transform operation, m1(r) is a second mask modulation function disposed near the main imaging system including on the main imaging lens and before the microlens array; Generating a final image on the image acquisition unit using the light field information includes: Based on the imaging response, the light field information is used to generate a final image on the image acquisition unit, wherein the imaging response is: Among them, ω u is the spatial frequency position corresponding to angle u, ω v is the spatial frequency position corresponding to the angle v, s(ω u ,ω v ) is the acquisition process of specific frequency components by camera pixels.

2. The system according to claim 1, wherein: The image acquisition unit is a CMOS image sensor.

3. An imaging method of a CIS system for elemental imaging, characterized in that: Using the CIS system for elemental imaging according to any one of claims 1 to 2, the method comprises the following steps: obtaining a first image of the target object through the main imaging system; Using the microlens array, phase-modulating the optical path of the first image by the phase modulation function to obtain a second image of the target object; The first mask modulation is performed on the second image through the first mask modulation function to introduce preset spectrum aliasing to obtain light field information of the target object, and the light field information is converted into a final image on the image acquisition unit.

4. The method according to claim 3, characterized in that Also includes: The first image is mask-modulated by using the second mask with a preset second mask modulation function to obtain a first image for phase modulation.

Images