Imaging apparatus, three-dimensional information acquisition method, and three-dimensional reconstruction method

Abstract

This invention provides an imaging device, a three-dimensional information acquisition method, and a three-dimensional reconstruction method. The imaging device includes a light source module, a light modulation system, and an imaging detector. The light source module emits broadband light within a target wavelength range towards a target object. The light modulation system includes a multifocal vector metalens, which is configured to separate the broadband light reflected from the target object into at least four target polarization states. The light modulation system separates the broadband light reflected from the target object into at least four target polarization states and into light of multiple target wavelengths, focusing them onto different imaging regions of the imaging detector to form multiple spectral atlases corresponding one-to-one with the target polarization states. Each spectral atlas includes multiple sub-images corresponding one-to-one with the target wavelengths. This improves the overall efficiency of the imaging device and also makes the imaging device more portable and has a higher degree of functional integration.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, and in particular to an imaging device, a three-dimensional information acquisition method, and a three-dimensional reconstruction method. Background Technology

[0002] Information related to the light field mainly includes intensity, wavelength, coherence, and polarization. Spectral imaging can measure the intensity of multiple wavelength bands and provide information about the composition of matter in a scene, thus obtaining spectral information. Spectral information is like a fingerprint of matter, effectively revealing the physical and chemical characteristics of a target object. Polarization imaging, on the other hand, can obtain the surface features of a target object by analyzing its polarization information. While spectral information may contain material properties, polarization information can better reveal the surface features of a target object, such as shape, shadows, and roughness. Because polarization information can provide information that has almost no physical correlation with spectral information, combining spectral and polarization information allows for better identification of target objects.

[0003] In recent years, spectral polarization imaging systems capable of acquiring four-dimensional (4D) information about target objects have attracted considerable interest. Combining spectral and polarization imaging, these systems are widely used in emerging fields such as remote sensing, materials analysis, scattering imaging, and medical diagnostics.

[0004] Traditional spectral polarization imaging systems record high-dimensional data cubes through sequential or parallel measurements. Sequential measurements, such as those performed by switching bandpass filter slices or polarizer slices or scanning the imaging scene to obtain two-dimensional (2D) subsets, require moving parts, resulting in low accuracy and poor real-time performance, thus limiting their application in dynamic scenes. Alternatively, parallel measurements can be achieved by simultaneously recording multiple subsets using beam splitters or specific detectors. For example, focal plane array (DoFP) sampling utilizes arrays with multi-channel spectral and polarization filters for detection; however, manufacturing large-scale spectral polarization arrays is challenging. Summary of the Invention

[0005] The purpose of this invention is to provide an imaging device, a three-dimensional information acquisition method, and a three-dimensional reconstruction method.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: an imaging device, comprising a light source module, a light modulation system, and an imaging detector;

[0007] The light source module is used to emit broadband light within the target wavelength range to the target object;

[0008] The optical modulation system includes a multifocal vector metalens, which is configured to separate broadband light reflected from the target object into at least four target polarization states. The optical modulation system separates the broadband light reflected from the target object into at least four target polarization states and into light of multiple target wavelengths, and focuses them onto different imaging regions of the imaging detector to form multiple spectral sets that correspond one-to-one with the target polarization states. Each spectral set includes multiple sub-maps that correspond one-to-one with the target wavelengths.

[0009] The focal plane of the imaging detector has an imaging region that corresponds one-to-one with the polarization state of the target.

[0010] As a further improved technical solution of the present invention, the multifocal vector meta-lens has a grating phase that realizes the target polarization state and a periodic arrangement that disperses broadband light into multiple target wavelengths.

[0011] As a further improved technical solution of the present invention, the multifocal vector meta-lens has a grating phase that realizes the target polarization state; the optical modulation system further includes a microlens array disposed on the side of the multifocal vector meta-lens facing the imaging detector and arranged sequentially along the optical axis, and a filter array corresponding to the microlens array.

[0012] As a further improvement of the present invention, the optical modulation system further includes a focusing lens disposed on the side of the multifocal vector meta-lens facing the imaging detector. The focusing lens is disposed at intervals with the multifocal vector meta-lens, and the focusing lens is used to focus the light after it has been split by the multifocal vector meta-lens onto different imaging areas of the imaging detector.

[0013] As a further improvement of the present invention, different imaging regions are phase-separated.

[0014] As a further improvement of the present invention, the target wavelength range includes the maximum target wavelength and the minimum target wavelength; the multi-focal vector meta-lens is designed such that the spectral maps corresponding to the maximum target wavelength and the minimum target wavelength are tangent to the boundaries of the corresponding imaging regions.

[0015] As a further improvement of the present invention, the diffraction angle, target wavelength, and period of the metasurface structure in the multifocal vector metalens satisfy the grating equation:

[0016] Where n0 is the refractive index of the medium in the incident direction; k0 is the incident wave vector 2π / λ0; θ0 is the incident angle in the x-direction; m is the diffraction order, which can be 0, ±1, ±2, etc.; P0 is the x-direction period of the metasurface structure in the multifocal vector metalens; nm Let θ be the refractive index of the medium in the direction of emission. m It is the diffraction angle.

[0017] As a further improvement of the present invention, the multifocal vector metalens is configured to separate the broadband light reflected from the target object into at least four target polarization states to obtain at least four polarization-sensitive beams, and the sub-image in the spectral set corresponding to the polarization-sensitive beams is a polarization sub-image; or, the multifocal vector metalens is configured to separate the broadband light reflected from the target object into at least four target polarization states to obtain at least four polarization-sensitive beams and at least one polarization-insensitive beam; the sub-image in the spectral set corresponding to the polarization-sensitive beams is a polarization sub-image, and the sub-image in the spectral set corresponding to the polarization-insensitive beams is an intensity sub-image.

[0018] To achieve the above-mentioned objectives, the present invention also provides a method for acquiring three-dimensional information based on the above-mentioned imaging device, comprising:

[0019] Acquire multiple spectral atlases formed on the imaging detector;

[0020] Multiple spectral sets are processed by spectral reconstruction algorithm to obtain an imaging map corresponding to each target wavelength to obtain the spectral information of broadband light reflected by the target object. Each imaging map includes a sub-map that corresponds one-to-one with the polarization state of the target, and the sub-map includes at least four polarization sub-maps with polarization information.

[0021] The polarization information corresponding to each target wavelength in the reflected broadband light is calculated based on at least four polarization sub-maps in each imaging image.

[0022] As a further improvement of the present invention, the three-dimensional information acquisition method further includes: calculating the depth information of the target object based on the parallax of at least two sub-images in any imaging image and / or the polarization information corresponding to light of any target wavelength.

[0023] As a further improvement of the present invention, the sub-image includes at least two intensity sub-images without polarization information. Specifically, "the depth information of the target object is calculated based on the disparity of at least two sub-images in any imaging image" means: the depth information of the target object is calculated based on the disparity of at least two intensity sub-images in any imaging image.

[0024] As a further improvement of the present invention, after calculating the depth information of the target object corresponding to all target wavelengths, the three-dimensional information acquisition method further includes the following steps: taking the average value of the depth information corresponding to all target wavelengths to obtain the final depth information of the target object.

[0025] As a further improvement of the present invention, before averaging the depth information corresponding to all target wavelengths, the three-dimensional information acquisition method further includes the following steps:

[0026] The depth information of the target object corresponding to all target wavelengths is obtained through comparison and calculation;

[0027] If the depth information corresponding to a certain target wavelength deviates too much from the depth information corresponding to other target wavelengths, then the depth information corresponding to the target wavelength with the excessive deviation is removed, and the average value of the remaining depth information corresponding to the target wavelengths is taken to obtain the final depth information of the target object; otherwise, the average value of the depth information corresponding to all target wavelengths is taken to obtain the final depth information of the target object.

[0028] To achieve the above-mentioned objectives, the present invention also provides a three-dimensional reconstruction method, which is based on multiple spectral atlases acquired by the imaging device described above; or the three-dimensional reconstruction method is based on the spectral information, polarization information, and / or depth information of the broadband light reflected by the target object obtained by the three-dimensional information acquisition method described above.

[0029] The beneficial effects of this invention are as follows: The imaging device of this invention, by setting a multi-focal vector metalens in the optical modulation system, at least achieves multi-polarization state beam splitting and polarization detection functions. That is, by using a multi-focal vector metalens to replace the optical elements in the traditional spectral polarization imaging system, the loss of energy in the polarization part of broadband light can be avoided at least, thereby improving the overall efficiency of the imaging device. At the same time, it also makes the imaging device more portable, has a high degree of functional integration, and has high light field control efficiency, which is conducive to the development of compact spectral polarization imaging devices. In addition, the spectral atlas with polarization information can be obtained in real time through a single imaging, thereby obtaining the spectral information and polarization information of the broadband light reflected by the target object, improving the efficiency of the imaging device, and enabling better identification of the target object. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the imaging device according to the first embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of multiple spectral atlases formed on an imaging detector in a specific embodiment of the present invention;

[0032] Figure 3 This is a photograph of the actual object in one specific implementation method;

[0033] Figure 4 This is a schematic diagram showing how a multidimensional dataset f is projected into a two-dimensional discrete dataset g through the system matrix H in an iterative algorithm.

[0034] Figure 5 To use spectral reconstruction algorithms for Figure 3 The image corresponding to each target wavelength is obtained by performing spectral separation processing on multiple spectral atlases. Figure 5 (a) is an image at a wavelength of 940 nm. Figure 5 (b) is the image at a wavelength of 950 nm. Figure 5 (c) is the image at a wavelength of 960nm;

[0035] Figure 6 According to Figure 5 Stokes parametric image calculated from the polariton diagram in (b);

[0036] Figure 7 According to Figure 5 (b) The polarization angle image and polarization degree image calculated from the polariton diagram;

[0037] Figure 8 yes Figure 5 A line graph containing polarization and spectral information corresponding to the facial and eye positions in the image;

[0038] Figure 9 This is a schematic diagram of the imaging device according to the second embodiment of the present invention;

[0039] Figure 10 It is a schematic diagram based on the binocular parallax principle of the first and second cameras;

[0040] Figure 11 This is a schematic diagram illustrating the principle of depth information calculation based on sub-graph disparity in this invention.

[0041] Figure 12 It is an image at a certain wavelength after spectral separation in a specific implementation method;

[0042] Figure 13 yes Figure 12 The intensity submap and the corresponding disparity map in the image, where, Figure 13 (a) is Figure 12 The original image containing the left and right intensity subgraphs. Figure 13 (b) is the pre-treated product. Figure 12 The left and right intensity subgraphs in the middle, Figure 13 (c) is based on Figure 13 The disparity map calculated from the left and right intensity submaps in (a) Figure 13 (d) is based on Figure 13 (b) shows the disparity map calculated from the preprocessed left intensity submap and the right intensity submap;

[0043] Figure 14This is a schematic diagram of the imaging device according to the third embodiment of the present invention;

[0044] Figure 15 This is a schematic diagram of the imaging device according to the fourth embodiment of the present invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. Please refer to the accompanying drawings for further details. Figures 1 to 15 The figures shown represent preferred embodiments of the present invention. However, it should be noted that these embodiments are not intended to limit the present invention. Any functional or structural equivalent modifications or substitutions made by those skilled in the art based on these embodiments are within the scope of protection of the present invention.

[0046] It should be understood that terms such as “comprising” or “having” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof, nor do they exclude the presence or addition of one or more other steps or combinations thereof.

[0047] Furthermore, it should be understood that although the terms first, second, third, fourth, etc., may be used herein to describe various elements or structures, the objects described should not be limited by these terms. These terms are used only to distinguish similar objects and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0048] Please refer to Figure 1 As shown, the present invention provides an imaging device 10, which includes a light source module 20, a light modulation system, an imaging detector 3, and a data processing module that is communicatively connected to the imaging detector 3. The light emitted from the light source module 20 irradiates the surface of the target object 30, and the reflected light generated is incident on the light modulation system. The reflected light is modulated by the light modulation system and then imaged on the imaging detector 3. The data processing module is built into a computer.

[0049] The light source module 20 is used to emit broadband light within the target wavelength range to the target object 30. The light source module 20 includes, but is not limited to, LEDs, semiconductor lasers (including edge-emitting EELs and vertical-emitting VCSELs), fiber lasers, etc. The broadband light emitted by the light source module 20 is reflected by the surface of the target object 30 to form broadband light incident on the optical modulation system.

[0050] The optical modulation system is configured to separate broadband light reflected by the target object into at least four target polarization states and into multiple target wavelengths, and focus them onto different imaging regions of the imaging detector to form multiple spectral sets 31 corresponding one-to-one with the target polarization states. Each spectral set 31 includes multiple sub-maps 311 corresponding one-to-one with the target wavelengths.

[0051] Combination Figure 2 The diagram shows a schematic of multiple spectral atlases 31 formed on an imaging detector 3 in a specific embodiment. Specifically, the target wavelengths include a first target wavelength of 940 nm, a second target wavelength of 950 nm, and a third target wavelength of 960 nm. The target polarization states are four linear polarization states: ψ = 0°, 45°, 90°, 135°; χ = 0°, 0°, 0°, 0°. Thus, four spectral atlases 31 are formed, each corresponding to one of the four linear polarization states. Each spectral atlas 31 includes three sub-images 311 corresponding to 940 nm, 950 nm, and 960 nm, respectively. The sub-image corresponding to the first target wavelength of 940 nm is defined as the first sub-image 311a, the sub-image corresponding to the second target wavelength of 950 nm as the second sub-image 311b, and the sub-image corresponding to the third target wavelength of 960 nm as the third sub-image 311c. The distribution of the three sub-images 311 in the imaging region is as follows: Figure 2 As shown in the figure, it can be seen that the second sub-figure 311b is located in the middle of the imaging region, the first sub-figure 311a is located near the center of the imaging detector 3 in the second sub-figure 311b, and the third sub-figure 311c is near the edge of the imaging detector 3.

[0052] Specifically, the optical modulation system includes at least a multifocal vector metalens 1, which is configured to separate the broadband light reflected from the target object into at least four target polarization states. The focal plane of the imaging detector 3 has an imaging region corresponding one-to-one with each of the target polarization states. In this invention, by introducing a multifocal vector metalens 1 with a metasurface structure, the optical modulation system achieves at least the function of multi-polarization state beam splitting and polarization detection. That is, by using a multifocal vector metalens 1 to replace the optical elements in the traditional spectral polarization imaging system, the loss of energy in the polarized part of the broadband light can be avoided, thereby improving the overall efficiency of the imaging device 10. At the same time, it also makes the imaging device 10 more portable, with higher functional integration and higher light field control efficiency, which is conducive to the development of compact spectral polarization imaging devices. In addition, the spectral atlas 31 with polarization information can be obtained in real time through a single imaging, thereby obtaining the spectral information and polarization information of the broadband light reflected by the target object 30, improving the efficiency of the imaging device 10, and enabling better identification of the target object 30.

[0053] At Figure 1As shown, in the first embodiment of the imaging device 10, the multifocal vector metalens 1 is configured to separate the broadband light reflected from the target object 30 into at least four target polarization states and multiple target wavelengths, and focus them onto different imaging regions of the imaging detector 3. That is, the multifocal vector metalens 1 integrates beam splitting and dispersion functions, multi-polarization state beam splitting and polarization analysis functions, and lens focusing functions. In this case, the optical modulation system can be configured to include only the multifocal vector metalens 1. In this embodiment, the multifocal vector metalens 1 replaces the phase-separated optical elements in a traditional spectral polarization imaging system, avoiding energy loss in the polarization and wavelength portions of the broadband light, thereby improving the overall efficiency of the imaging device 10. It has advantages such as high integration and high efficiency.

[0054] Specifically, the metasurface structure of the multifocal vector metalens 1 has a grating phase for realizing the target polarization state, a focusing phase for realizing lens focusing, and a periodic arrangement for dispersing broadband light into multiple target wavelengths. That is, the metasurface structure of the multifocal vector metalens 1 superimposes both the grating phase and the focusing phase.

[0055] In some optional embodiments, different imaging regions in the focal plane of the imaging detector are separated, so that there is no overlap between the spectral atlases 31 of different imaging regions, simplifying the algorithm steps for obtaining the three-dimensional information of the target object 30 through the spectral atlases 31.

[0056] In one specific embodiment, the target wavelength range includes a maximum target wavelength and a minimum target wavelength, for example, for Figure 2 In one specific embodiment shown, the maximum target wavelength is the third target wavelength of 960nm, and the minimum target wavelength is the first target wavelength of 940nm. The optical modulation system and the focal plane are configured such that the sub-images 311 corresponding to the maximum and minimum target wavelengths are tangent to the boundaries of the corresponding imaging regions. This ensures that each sub-image 311 is a complete image and allows the multi-focal vector meta-lens 1 to have a large field of view, improving image quality and enabling better identification of target objects. Of course, this is not a limitation. In other embodiments, the sub-images corresponding to the maximum and minimum target wavelengths can also be set to have a preset distance from the boundaries of the corresponding imaging regions. In this case, the corresponding field of view can be set to be smaller.

[0057] Specifically, the diffraction angle θ of the metasurface structure in the multifocal vector metalens 1 m The target wavelength and the period P0 of the metasurface structure in the multifocal vector metalens 1 satisfy the grating equation: Where n0 is the refractive index of the medium in the incident direction; k0 is the incident wave vector 2π / λ0; θ0 is the incident angle in the x-direction; m is the diffraction order, which can be 0, ±1, ±2, etc.; P0 is the x-direction period of the metasurface structure in the multifocal vector metalens 1; n m Let θ be the refractive index of the medium in the direction of emission. m Let θ be the diffraction angle. Therefore, the target wavelength, period P0, and diffraction angle θ are... m Once any two parameters are known, the other unknown parameter can be obtained according to the above grating equation.

[0058] It is known that the target wavelength range, the size of each imaging region in the focal plane, the aperture size corresponding to the optical modulation system, the focal length, the field of view, the total optical length (TTL), and the diffraction angle θ of the metasurface structure in the multifocal vector metalens 1 of the optical modulation system are all factors to consider. m Furthermore, the various parameters, such as the period P0, are interdependent. The remaining unknown parameters can be calculated based on some of the determined parameters to design the metasurface structure in the multifocal vector metalens 1, and / or the field of view, focal length, and focal plane size of the optical modulation system. For example, the field of view and total optical length (TTL) can be determined first, or the focal plane size can be determined first, and then the diffraction angle, target wavelength range, and image quality can be optimized; alternatively, the target wavelength range and diffraction angle can be determined first, and then the field of view can be optimized, and an imaging detector 3 with a focal plane of appropriate size can be selected.

[0059] In one specific embodiment, the known center target wavelength of the target wavelength range is 950nm, the maximum imaging area of ​​the focal plane is 1.44mm*2.04mm, and the period P0 of the metasurface structure on the designed multifocal vector metalens 1 is 5.5µm. Specifically, the pixel size of the focal plane in the imaging detector is 3µm, and the number of pixels is 480*680. Therefore, the effective area of ​​the entire focal plane is 1.44mm*2.04mm, that is, the maximum imaging area of ​​the focal plane is 1.44mm*2.04mm. In this embodiment, the multifocal vector metalens 1 is designed to separate the broadband light reflected by the target object 30 into four target polarization states. That is, the multifocal vector metalens 1 is designed to split the broadband light reflected by the target object 30 into four different diffraction orders. Thus, the actual imaging area on the focal plane is 1.44mm*1.44mm. The metasurface structure on the multifocal vector metalens 1 has a small period of 500 nm, and 11*11 metas form one metasurface unit, that is, the period P0 of the metasurface structure is 5.5 μm.

[0060] After determining the central target wavelength of the aforementioned target wavelength range to be 950 nm, the maximum imaging area of ​​the focal plane to be 1.44 mm * 2.04 mm, and the period P0 of the metasurface structure on the designed multifocal vector metalens 1 to be 5.5 μm, the first-order diffraction angle corresponding to the 950 nm wavelength is calculated to be 9.946° according to the aforementioned grating equation. It can be seen that the sub-image 311 corresponding to the central target wavelength is located in the middle of the corresponding imaging area. Therefore, the maximum field of view can be calculated based on the calculated first-order diffraction angle of 9.946°. At this maximum field of view, the sub-image 311 corresponding to the central target wavelength is tangent to the boundary of the imaging area. Then, an angle smaller than the maximum field of view is optimized as the target field of view. Based on this target field of view and the focal plane size, the minimum and maximum target wavelengths can be calculated, that is, the target wavelength range is obtained.

[0061] In another specific embodiment, given a target wavelength range of 900nm-1000nm, a central target wavelength of 950nm, and a metasurface structure period P0 of 5.5µm on the designed multifocal vector metalens 1, the diffraction angle can be calculated based on the aforementioned grating equation. Then, based on this diffraction angle, the aperture size, focal length, etc., can be optimized using Zemax optical design software to obtain the maximum field of view. After obtaining the maximum field of view, further detailed optimization is performed in Zemax optical design software or other optical design software to ensure that the spectral atlases 31 of adjacent imaging regions do not overlap and meet image quality requirements. The target field of view is then iteratively derived, ensuring that all sub-images 311 corresponding to the target wavelength within the target wavelength range do not exceed the imaging region, and that sub-images 311 of the same target wavelength do not overlap. Simultaneously, the optical design provides the binary phase of the multifocal vector metalens 1, such as the focusing phase.

[0062] After obtaining the period P0 of the metasurface structure on the multifocal vector metalens 1, and combining the binary plane phase and the grating phase calculated based on the target polarization state, the micro-nano structure is scanned using full analytical RCWA or other methods. This ensures that the metasurface structure on the multifocal vector metalens 1 satisfies the functions of multi-polarization state beam splitting and polarization analysis, focusing, and beam dispersion, while also meeting the basic requirements of high efficiency, no structural resonance, uniform phase distribution, and large range at all target wavelengths within the target wavelength range. Then, examples of structural parameters that simultaneously meet these conditions are listed, screened, and simulated to obtain the best solution, thus enabling the design of the metasurface structure.

[0063] In one specific embodiment, the grating phase configuration in the multifocal vector metalens 1 is such that the broadband light reflected from the target object 30 is separated into at least four target polarization states to obtain at least four polarization-sensitive beams. The sub-image 311 in the spectral atlas 31 corresponding to the polarization-sensitive beams is a polarization sub-image with polarization information. At this time, the spectral atlas 31 contains both spectral and polarization information. The data processing module has a built-in spectral reconstruction algorithm for obtaining the spectral information of the broadband light reflected from the target object 30 and a polarization reconstruction algorithm for obtaining the polarization information of the broadband light. After the data processing module obtains multiple spectral atlases on the imaging detector 3, it first performs spectral separation on each spectral atlas 31 using the spectral reconstruction algorithm to obtain an imaging image corresponding to each target wavelength. Each imaging image includes a sub-image 311 corresponding one-to-one with the target polarization state. In this embodiment, each imaging image includes a polarization sub-image corresponding one-to-one with the target polarization state. Then, the polarization reconstruction algorithm calculates the polarization information of each pixel of the reflected light from the target object 30 corresponding to that wavelength based on the imaging image at each wavelength.

[0064] In this embodiment, the multifocal vector metalens 1 is a monolithic multifocal vector metalens 1, including a substrate and a metasurface structure disposed on at least one side of the substrate. In one specific embodiment, the multifocal vector metalens 1 only includes the metasurface structure disposed on one side of the substrate, and the metasurface structure superimposes grating phase and focusing phase, etc. In this case, the metasurface structure can be disposed on the side of the substrate facing the target object 30, or it can be disposed on the side of the substrate away from the target object 30. In other embodiments, the multifocal vector metalens 1 may also include two metasurface structures disposed on opposite sides of the substrate. In this case, the phase, period, size, etc. of the two metasurface structures can be designed according to specific requirements.

[0065] Furthermore, the data processing module also has a depth reconstruction algorithm for acquiring the depth information of the target object. In this embodiment, the depth reconstruction algorithm is a depth reconstruction algorithm for acquiring the depth information of the target object 30 based on the polarization information of broadband light. That is, the depth information of the target object 30 is calculated based on the polarization information corresponding to any wavelength of light. Finally, by combining the spectral information, polarization information, and depth information, a more accurate three-dimensional reconstruction of the target object can be achieved, and the imaging system is simple.

[0066] The present invention also provides a three-dimensional information acquisition method based on the imaging device 10 described above, that is, the data processing module processes and calculates multiple spectral atlases 31 on the imaging detector 3 to obtain three-dimensional information, the three-dimensional information including but not limited to the spectral information of the broadband light reflected by the target object 30, the polarization information corresponding to each target wavelength in the reflected broadband light, and the depth information of the target object 30.

[0067] The three-dimensional information acquisition method includes the following steps:

[0068] S1: Acquire multiple spectral datasets 31 formed on the imaging detector 3;

[0069] S2: The multiple spectral sets 31 are subjected to spectral separation processing by a spectral reconstruction algorithm to obtain an imaging map corresponding to each target wavelength to obtain the spectral information of the broadband light reflected by the target object. Each imaging map includes a sub-map 311 that corresponds one-to-one with the polarization state of the target. The sub-map 311 includes at least four polarization sub-maps with polarization information.

[0070] S3: Calculate the polarization information of the target wavelength in the reflected broadband light based on at least four polarization sub-maps in each imaging image.

[0071] In one specific embodiment, the target wavelength is 940nm, 950nm, and 960nm, and the target polarization state is four linear polarization states: ψ = 0°, 45°, 90°, 135°; χ = 0°, 0°, 0°, 0°. Combined with... Figure 3 A photograph of the actual object is shown in a specific embodiment, along with... Figure 2 As shown, four spectral sets 31 are formed on the imaging detector 3. Each spectral set 31 corresponds to a target polarization state. Each spectral set 31 includes three sub-maps 311 corresponding to the target wavelengths of 940nm, 950nm, and 960nm, respectively.

[0072] In one specific embodiment, the spectral separation processing of multiple spectral sets 31 using the spectral reconstruction algorithm in step S2 is performed by using an iterative algorithm to perform spectral separation processing on each of the multiple spectral sets 31. Specifically, the spectral separation processing for one spectral set 31 is as follows:

[0073] Combination Figure 4 As shown, the multidimensional dataset f can be projected into a two-dimensional discrete dataset g through the system matrix H. Generally, the system is represented by g = Hf. After obtaining the spectral atlas 31 on the imaging detector 3, which is equivalent to obtaining the discrete dataset g, the actual multidimensional dataset f can be estimated using an iterative algorithm, such as the doubling algebraic reconstruction method (MART), thus obtaining multiple separated sub-maps 311.

[0074] The iterative formula for the doubling algebraic reconstruction method is as follows:

[0075] In the formula It is an estimate of the object, and iterative algorithms require a relatively long time to obtain a relatively high resolution.

[0076] Specifically, in optical imaging, the target object, represented by f, includes an N-dimensional information distribution. In the design of the multifocal vector metalens 1, the target object 30 has four-dimensional information, including two spatial coordinates (x, y), wavelength, and polarization information. Furthermore, we can use... The function is defined by (spatial vector) and Si (Stokes parameters, i = 0, 1, 2, 3). If the object is in a noise-free space and can be mapped to a two-dimensional image space through a linear process. Mathematically, g is a linear superposition of the values ​​of f, therefore it can be represented as follows:

[0077] in It is a function that includes the polarization and geometric mapping relationship between object space and image space, when the target object 30 is... When given, the functional representation of h comes from the multidimensional dataset mapped to the image space. The probability of n0 and n1 varies, and based on the polarization characteristics of the multifocal vector metalens 1, it has different polarization responses. Similarly, because of n0 and n2 in the above grating equations... m Both are 1, and the diffraction angle θ m The following is determined from the above grating equation: Assuming collimated light reaches the multifocal vector metalens 1, the diffraction displacement dm(λ) on the focal plane array (FPA) can be expressed by the diffraction angle θ. m (λ) yields: d m (λ)=L·tanθ m (λ), L, is the distance from the multifocal vector metalens to the focal plane; the diffraction displacement vector in imaging space can be represented by a unit vector. To describe, Regarding imaging detector 3, the measured power is the integral of the 4D dataset along a specific path, determined by the characteristics of the diffraction element. Considering space... The dimension of the spectrum (λ), and the path traversing the dataset, can be represented by the Dirac-δ function:

[0078] Based on the polarization characteristics of the multifocal vector metalens 1, the polarization sensitivity of different diffraction orders is different. There can be j types of combinations, and the polarization sensitivity of each diffraction order is as follows: Assuming the polarization of the diffraction order is linear polarization at 0° and 90°, then I ±1 =∝(S0±S1) / 2, I0=∝S0; Assuming the polarization of the diffraction order is linear polarization at 45° and 135°, then I ±1 =∝(S0±S2) / 2, I0=∝S0; Assuming the polarization of the diffraction order is left- or right-hand circular polarization, then |I ±1=∝(SO±S3) / 2, I0=∝S0, where I m It is the output energy of the mth order.

[0079] The transfer function of the Dirac-δ function with additional Stokes parameters (S0-S3) can be expressed as:

[0080]

[0081] A = (1 + λ(m)) / 2,

[0082] B = m / 2 (m = 0, ±1),

[0083] j corresponds to the polarization type; when the diffraction order is linearly polarized at 0° and 90°, j = 1. For simplicity, we assume all diffraction orders have a uniform diffraction efficiency. Therefore, the system function can now be described as:

[0084]

[0085] This continuous function precisely provides the analytical relationship between 4D and 2D space. Using this analytical relationship, the system matrix H can be ultimately represented after determining the spatial and spectral resolution of the system. The discrete dataset g from the discrete imaging detector 3 can be associated with discrete volume units in object space, and all this volume information can be represented by a multidimensional dataset vector f. The relationship between these f and g can be described by the system matrix H, which contains the position and polarization information of all these projections. An estimate of the multidimensional dataset f can be obtained using a reconstruction algorithm and by utilizing the inverse of the system matrix H and the discrete dataset g. g = Hf can be expressed as:

[0086] M: Number of pixels on the focal plane, N: Size of the cube. The cube is then iteratively calculated to obtain the cube dataset f.

[0087] Combination Figure 5 As shown, the above-mentioned spectral reconstruction algorithm is used to... Figure 3 Each spectral atlas 31 in the dataset is processed by spectral separation to obtain an imaging image corresponding to each target wavelength, wherein... Figure 5 (a) is an image at a wavelength of 940 nm. Figure 5 (b) is the image at a wavelength of 950 nm. Figure 5 (c) is an image at a wavelength of 960 nm. Specifically, step S3 is as follows:

[0088] S31: Establish a rectangular coordinate system in the imaging detector 3, specifically: establish a rectangular coordinate system with the center of the imaging detector 3 as the origin, the horizontal axis passing through the origin as the X-axis, and the vertical axis passing through the origin as the Y-axis;

[0089] S32: Perform pixel-level feature point registration on at least four polarization sub-maps in each imaging image;

[0090] S33: Based on the registered multiple polariton images, obtain the degree of polarization and polarization angle of each pixel at the target wavelength.

[0091] The following is a detailed explanation using the example of calculating the polarization information of the target wavelength based on the four polarization sub-maps in each imaging image:

[0092] The pixels of the four polarization sub-images are in one-to-one correspondence. For a pixel at a certain position (row i, column j), the intensity of the four polarization sub-images is denoted as... The four Stokes components at this location are respectively Based on the polarization states of the four polarizer diagrams, a 4x4 matrix A can be obtained. Each column of this matrix corresponds to a different polarization state, and any column can be represented as: [1, cos2χcos2ψ, cos2χsin2ψ, sin2χ] T The following relationship exists between them:

[0093] Based on this relationship, the four Stokes components at this location can be calculated as follows: The same principle applies to pixels at other locations, ultimately yielding the complete S0, S1, S2, S3 matrix, or image. Specifically, combining... Figure 6 As shown, it is based on Figure 5 (b) Stokes parametric images calculated from the four polariton images in the corresponding imaging plot.

[0094] According to the formula for calculating polarization degree image The degree of polarization is calculated separately for each pixel; based on the polarization angle image calculation formula. The polarization angle of each pixel is calculated separately to obtain the corresponding polarization information. Combined with... Figure 7 As shown, it is based on Figure 5 (b) The polarization angle image and polarization degree image calculated from the four polariton images in the corresponding imaging image.

[0095] After obtaining the polarization information of each pixel according to the above steps, the polarization information can be used to analyze and obtain the surface features of the target object, such as shape, shadow, roughness and other information.

[0096] Combination Figure 8 As shown, where Figure 8 (a) is to Figure 5 The line graph is obtained by calculating the polarization information of the face position in the image at each wavelength. Figure 8 (b) to make Figure 5 The line graphs are obtained by calculating the polarization information of the eye position in the imaging image at each wavelength. It can be seen that both line graphs simultaneously contain polarization and spectral information. Furthermore, it can be observed from the graphs that the intensity and trend of the Stokes parameter differ at different positions in the imaging image at different wavelengths, which can improve the resolution accuracy of the target object and expand the application range of the imaging device.

[0097] Furthermore, the three-dimensional information acquisition method also includes: calculating the depth information of the target object 30 based on the polarization information corresponding to light of any target wavelength. In this embodiment, the depth information of the target object 30 is directly obtained through polarization information, and the algorithm is relatively simple.

[0098] In one specific implementation, "calculating the depth information of the target object based on the polarization information corresponding to any target wavelength of light" specifically includes the following steps:

[0099] The normal vector of each pixel is calculated based on the polarization information corresponding to any target wavelength of light;

[0100] Based on the Lambertian reflection model, the coarse prior information of the target object depth is obtained, and the normal vector is corrected based on the coarse prior information of the target object depth to obtain the corrected normal vector.

[0101] Based on the corrected normal vector, the relative height of the surface of the target object 30 is obtained, that is, the depth information of the target object 30 is obtained.

[0102] The process of "calculating the normal vector of each pixel based on the polarization information corresponding to any target wavelength of light" specifically includes the following steps:

[0103] The zenith angle α(x, y) is obtained by solving for the relationship between the zenith angle α(x, y) and the degree of polarization ρ, where the relationship between the zenith angle α(x, y) and the degree of polarization ρ is as follows: n is the refractive index of the target object's surface. The degree of polarization ρ has a very weak dependence on n. The typical value of n for dielectrics is between 1.4 and 1.6, and is generally taken as 1.5. The azimuth angle... or It can be seen that the azimuth angle obtained using polarization information has an ambiguity of π radians;

[0104] After obtaining the zenith angle α(x, y) and azimuth angle Then, the normal vector is calculated according to the following formula:

[0105]

[0106] It can be seen that the azimuth angle obtained using polarization information has an ambiguity of π radians, and the corresponding normal vector obtained using polarization information also has an ambiguity of π radians.

[0107] "Obtaining rough prior depth information of the target object based on the Lambertian reflection model, and correcting the normal vector based on the rough prior depth information of the target object to obtain the corrected normal vector" specifically includes the following steps:

[0108] The reflectivity function of a Lambert surface is:

[0109] Where E(x, y) is the gray value at pixel (x, y).

[0110] τ is the azimuth angle of the illumination direction, and σ is the zenith angle of the illumination direction; p and q can be approximated by discrete difference of the surface height Z(x, y):

[0111]

[0112] make

[0113] f(x, y) = E(x, y) - R(p, q) = E(x, y) - R(Z(x, y), Z(x, y-1), Z(x-1, y)) = 0, which, after Taylor series expansion, gives...

[0114] n represents the number of iterations, Z n (x, y) represents the surface height in the nth iteration, and

[0115]

[0116] Assume Z 0 If (x, y) = 0, the final surface height Z can be obtained by iterating n times using the formula Zn(x, y). n (x, y); then based on the obtained Z n (x, y) and the formula Find p and q, and finally use the formula The surface reference normal vector normal_vector2 for each pixel is calculated;

[0117] Correcting pixel by pixel, for any pixel (x, y), selecting the vector that minimizes the Euclidean distance between normal_vector2 and normal_vector1. As the corrected azimuth angle

[0118] According to the corrected azimuth angle The corrected normal vector, normal_vector, is calculated, where,

[0119] Where p and q can be regarded as the rates of change of the target object's surface in the horizontal and vertical directions, reflecting the local curvature or shape of the target object's surface. s and q s It represents the rate of change of the direction of illumination, that is, the degree of influence of a small change in the direction of illumination on the illumination conditions. It can be used to describe the effect of horizontal and vertical changes in the direction of illumination on reflectivity.

[0120] "Obtaining the relative height of the target object's surface based on the corrected normal vector, i.e., obtaining the target object's depth information," specifically includes the following steps: Using the corrected normal vector, perform gradient integration to obtain the relative height of the target object's surface at 30°. The gradient integration formula is:

[0121]

[0122] It is known that the greater the relative height Z of the surface of the target object 30, the closer the surface of the target object 30 is to the multifocal vector metalens, and the more accurately the shape of the target object 30 can be reproduced.

[0123] Furthermore, after calculating the depth information of the target object 30 corresponding to each target wavelength using the depth information acquisition method described above, the three-dimensional information acquisition method further includes the following step: averaging the depth information corresponding to all target wavelengths to obtain the final depth information of the target object 30. This allows for obtaining more accurate depth information of the target object 30.

[0124] Furthermore, before averaging the depth information corresponding to all target wavelengths, the three-dimensional information acquisition method further includes the following steps:

[0125] The depth information of the target object 30 corresponding to all target wavelengths was obtained through comparison and calculation;

[0126] If any depth information corresponding to a certain target wavelength deviates significantly from the depth information corresponding to other target wavelengths, the depth information corresponding to the target wavelength with the significant deviation is removed, and the average value of the remaining depth information corresponding to the target wavelengths is taken to obtain the final depth information of the target object 30. If no such deviation exists, the average value of the depth information corresponding to all target wavelengths is taken to obtain the final depth information of the target object 30. Furthermore, this step allows for the acquisition of more accurate depth information for the target object 30.

[0127] Furthermore, the present invention also provides a three-dimensional reconstruction method, which is based on multiple spectral atlases 31 acquired by the imaging device 10 described above; or the three-dimensional reconstruction method is based on the spectral information, polarization information, and / or depth information of the broadband light reflected by the target object 30 obtained by the three-dimensional information acquisition method described above.

[0128] Specifically, after calculating the spectral information, polarization information, and depth information of the broadband light reflected by the target object 30 based on the multiple spectral atlases 31, the three-dimensional reconstruction method can obtain the material properties of each pixel of the target object 30 based on the spectral information, obtain the surface features of the target object 30 based on the polarization information, and then combine the depth information of the target object 30 to reconstruct the three-dimensional contour of the target object 30, thereby obtaining a three-dimensional image of the target object 30 with relatively comprehensive and accurate information.

[0129] At Figure 9 The image shown illustrates a second embodiment of the imaging device 10a. This second embodiment differs from the first embodiment in that the multifocal vector metalens 1a is configured to separate the broadband light reflected from the target object 30 into at least four target polarization states and into multiple target wavelengths. In this case, the multifocal vector metalens 1a has a grating phase to realize the target polarization states and a periodic arrangement to disperse the broadband light into multiple target wavelengths. That is, the multifocal vector metalens 1a integrates beam splitting and multi-polarization state beam splitting and polarization detection functions, but does not possess a lens focusing function. In this case, the optical modulation system further includes a focusing lens 2 disposed on the side of the multifocal vector metalens 1a facing the imaging detector 3. The focusing lens 2 is spaced apart from the multifocal vector metalens 1a, and is used to focus the light split by the multifocal vector metalens 1a onto different imaging areas of the imaging detector 3.

[0130] In this embodiment, the multifocal vector meta-lens 1a has no lens focusing function at all; the focusing function of the optical modulation system is entirely undertaken by the focusing lens 2, which improves the final image quality. In this case, when designing the metasurface structure of the multifocal vector meta-lens 1a, there is no need to superimpose the focusing phase. In other embodiments, the multifocal vector meta-lens 1a can also be configured to have partial lens focusing function, and the focusing lens 2 can have another partial lens focusing function. In this case, the multifocal vector meta-lens 1a is configured to also superimpose a portion of the focusing phase.

[0131] In one specific embodiment, the focusing lens 2 is a focusing lens 2 with a metasurface structure, which can further reduce the volume of the optical modulation system and improve the integration of the imaging device 10a.

[0132] In this embodiment, by additionally setting a focusing lens 2 spaced apart from the multifocal vector meta-lens 1a, the depth reconstruction algorithm in this embodiment can be set to obtain the depth information of the target object based on the polarization information of broadband light in the first embodiment, or it can be set to obtain the depth information of the target object 30 based on the principle of binocular parallax.

[0133] In the embodiment where the depth reconstruction algorithm is based on the principle of binocular parallax to obtain the depth information of the target object, the depth information acquisition step in the three-dimensional information acquisition method is specifically "to calculate the depth information of the target object 30 based on the parallax of at least two sub-images 311 in any imaging image".

[0134] In one specific embodiment, the multifocal vector meta-lens 1a is configured to separate the broadband light reflected from the target object 30 into at least four target polarization states to obtain polarization-sensitive beams that correspond one-to-one with the target polarization states. That is, when the sub-image 311 in the imaging image corresponding to each target wavelength is a polarization sub-image, "calculating the depth information of the target object 30 based on the parallax of at least two sub-images in any imaging image" specifically means "calculating the depth information of the target object 30 based on the parallax of at least two polarization sub-images in any imaging image".

[0135] In another specific embodiment, the multifocal vector metalens 1a is configured to separate the broadband light reflected from the target object 30 into at least four target polarization states to obtain at least four polarization-sensitive beams and at least one polarization-insensitive beam. Sub-image 311 in the spectral atlas 31 corresponding to the polarization-sensitive beam is a polarization sub-image, and sub-image 311 in the spectral atlas 31 corresponding to the polarization-insensitive beam is an intensity sub-image. That is, sub-image 311 in the imaging image corresponding to each target wavelength includes a polarization sub-image with polarization information and an intensity sub-image without polarization information. In this case, "calculating the depth information of the target object based on the parallax of at least two sub-images in any imaging image" specifically means: calculating the depth information of the target object 30 based on the parallax of at least two polarization sub-images in any imaging image, or calculating the depth information of the target object 30 based on the parallax of at least two intensity sub-images in any imaging image, or calculating the depth information of the target object 30 based on the parallax of at least one intensity sub-image and at least one polarization sub-image in any imaging image.

[0136] In some optional embodiments, "calculating the depth information of the target object based on the disparity of at least two sub-images in any imaging image" specifically means:

[0137] The at least two subgraphs were preprocessed using MATLAB.

[0138] The pixels of the at least two sub-images are matched to obtain the optimal pixel matching result;

[0139] Calculate the disparity value between matching pixels in the at least two sub-graphs;

[0140] Based on the principle of binocular parallax, the parallax value is converted into depth information.

[0141] Below, in conjunction with Figure 10 The diagram shown illustrates the principle of binocular parallax calculation, specifically explaining the calculation principle of deriving the depth information of the target object 30 based on the parallax of two images taken from different perspectives. Figure 10 To acquire two images of the target object 30 by simultaneously capturing images of the target object 30 using two cameras, namely, by simultaneously capturing images of the target object 30 using a first camera 5 and a second camera 6, the depth information of the target object 30 is calculated based on the parallax of the two acquired images.

[0142] Specifically, the depth information of the target object is calculated using the binocular parallax principle formula derived from the theory of triangle similarity:

[0143] Where B represents the baseline distance between the first camera 5 and the second camera 6, f is the focal length of the camera (mm), (x, y) represents the pixel coordinates in the Cartesian coordinate system of the imaging detector 3, pixel_size is the pixel size of the imaging detector (mm), and (x, y)*pixel_size is the disparity value disp.

[0144] According to the above binocular parallax principle formula, once we know the baseline distance B between the two cameras (5, 6) and the parallax value disp, we can calculate the distance between the target object 30 and the camera according to the above binocular parallax principle formula, and then obtain the depth value of the target object 30. Based on the depth value and the horizontal and vertical coordinates of the pixels in the original image, a three-dimensional point cloud model can be generated. The position of each point in the three-dimensional space corresponds to a point in the camera coordinate system. This three-dimensional point cloud model represents the three-dimensional structure of the observed scene, that is, the depth information of the target object 30.

[0145] Combination Figure 11 As shown, before calculating the disparity value between matching pixels in the at least two sub-images 311, it is necessary to first obtain the displacement of the disparity image. Calculate the overlapping area in sub-figure 311, based on this displacement. The multiple sub-images 311 in the corresponding imaging image are rearranged to separate the overlapping areas in the sub-images 311, resulting in multiple rearranged sub-images 311; then the disparity value is calculated based on at least two sub-images 311 in the multiple rearranged sub-images 311.

[0146] Specifically, taking the center of the focusing lens 2 as the origin O, as shown in the figure... Figure 11 The coordinate system shown has the target object's coordinates as (x0, y0, z0), where the labels represent virtual objects obtained by back-tracking diffracted rays. It can be seen that each sub-image corresponds to a virtual pinhole point O on the virtual pinhole plane. i Based on pinhole analysis, the imaging point I on the pickup plane corresponding to each sub-image can be obtained. The coordinates of imaging point I on the pickup plane are: Where m is the diffraction order, x mth and y mth These represent the x and y positions of the sub-map for the m-th order diffraction; z o z1: Distance from target object 30 to focusing lens 2; z2: Distance from focusing lens 2 to imaging detector 3; a is the grating constant, and m is the diffraction order.

[0147] After obtaining the coordinates of the imaging point I on the pickup plane, obtain the imaging point VI(x) on the imaging surface corresponding to the corresponding sub-image. mth y mth , z o ),in, d is the distance from the multifocal vector superlens 10a to the focusing lens 2. Where, I(x) mth y mth , z o ) and VI(x mth y mth , z o The shift Δx mapping between the two, i.e., the displacement of the disparity image:

[0148] Furthermore, the distance between the pinhole points corresponding to the two sub-figures 311 is the baseline distance B in the above binocular parallax principle formula. For example... Figure 11 As shown, assuming the two sub-images 311 include a first sub-image and a second sub-image, with the first sub-image corresponding to pinhole point O1 on the virtual pinhole plane and the second sub-image corresponding to pinhole point O2 on the virtual pinhole plane, then when calculating based on the binocular parallax principle using the first and second sub-images, the distance between pinhole point O1 and pinhole point O2 is the baseline distance B in the above binocular parallax principle formula. Simultaneously, the baseline distance B corresponding to the two sub-images can also be calculated using the following formula: B(baseline)=d*tanΔθ, where Δθ=|θ1-θ2|, and θ1 and θ2 represent the target diffraction angles corresponding to the two sub-images 311, respectively.

[0149] Specifically, in combination Figure 11 As shown, the imaging point on the pickup plane corresponding to the first sub-image is I1, and the imaging point on the pickup plane corresponding to the second sub-image is I2.

[0150] Specifically, similarity measures commonly used to calculate the disparity values ​​of two sub-images 311 include SSD (Sum of Squared Differences) or NCC (Normalized Cross-Correlation). Considering local information and the smoothness of the cost map, dynamic programming or similar methods are used to aggregate and optimize the cost near each pixel to obtain a more accurate and smoother disparity map. Based on the aggregated cost map, the matching pixel with the minimum cost at each pixel location is selected to obtain the final disparity map. Typically, the matching pixel corresponding to this minimum cost is considered the best match, representing the disparity value of the same object point in the two sub-images 311.

[0151] like Figure 12 The image shown is an image at a certain wavelength after spectral separation in a specific embodiment. It includes two intensity sub-images on the left and right, and four other polarization sub-images. In this embodiment, the depth information of the target object 30 is calculated based on the parallax of the two intensity sub-images on the left and right.

[0152] like Figure 13 (a) is shown as Figure 12 The original image of the left intensity subgraph and the right intensity subgraph, as shown in the figure. Figure 13 (b) shows the result after preprocessing. Figure 12 The left and right intensity subgraphs in the image, such as Figure 13 As shown in (c), it is based on Figure 13 The disparity map calculated from the left and right intensity submaps in (a) is as follows: Figure 13 As shown in (d), it is based on Figure 13 (b) shows the disparity map calculated from the preprocessed left and right intensity submaps. A comparison shows that the preprocessed disparity map better reflects the depth of the target object 30, and the overall result meets expectations.

[0153] After obtaining the disparity values ​​between the sub-images 311 and the baseline distance B mentioned above, the binocular disparity principle formula can be applied as follows: By converting the disparity values ​​of the two sub-images 311 into depth values, the resulting disparity map shows that the larger the disparity value, the closer the target object 30 is to the camera plane, and vice versa.

[0154] After obtaining the depth information of the target object 30 based on the principle of binocular parallax, the depth information can be used directly, or the depth information can be used to correct the normal vector of each pixel obtained based on the polarization information in the first embodiment to obtain the corrected normal vector. That is, the correction of the normal vector in this embodiment can be different from that in the first embodiment.

[0155] Specifically, correcting the normal vector of each pixel obtained based on polarization information in the first embodiment using this depth information includes the following steps:

[0156] The surface reference normal vector of each pixel is calculated based on the depth information of the target object;

[0157] The normal vector is corrected based on the surface reference normal vector to obtain the corrected normal vector.

[0158] Specifically, "calculating the surface reference normal vector of each pixel based on the depth information of the target object" means calculating the surface reference normal vector of each pixel according to the following formula:

[0159] Where p(x, y) and q(x, y) are the gradients of the object's surface with respect to the x and y directions, respectively.

[0160] p(x, y)=-(Depth_p(x, y)-Depth_p(x-1, y)),

[0161] q(x, y)=-(Depth_p(x, y)-Depth_p(x, y-1)).

[0162] "The normal vector is corrected based on the surface reference normal vector to obtain the corrected normal vector." This specifically includes the following steps:

[0163] The azimuth angle of the surface reference normal vector is calculated based on the surface reference normal vector.

[0164] Correction is performed pixel by pixel. For any pixel (x, y), the corresponding pixel is... and Choose the pixel closest to the corresponding pixel in π. The azimuth angle is used as the corrected azimuth angle. Obtain the corrected normal vector

[0165]

[0166] Specifically, "the azimuth angle of the surface reference normal vector is calculated based on the surface reference normal vector." "For example, the surface reference normal vector n_p(x, y) of each pixel mentioned above can also be represented as the zenith angle α." p(x, y) and azimuth angle Format:

[0167] Based on this formula and the formulas mentioned above

[0168] p(x, y)=-(Depth_p(x, y)-Depth_p(x-1, y)),

[0169] From q(x, y) = -(Depth_p(x, y) - Depth_p(x, y-1)), we can deduce:

[0170]

[0171] Therefore, the azimuth angle of the surface reference normal vector can be calculated.

[0172] It should be noted that the above... This refers to Azi, and That is, Azi+π.

[0173] The second embodiment of the present invention is the same as the first embodiment except for the differences mentioned above, and will not be described again here.

[0174] At Figure 14 As shown, this is a third embodiment of the imaging device 10b. The difference between this third embodiment and the first embodiment is that the multifocal vector meta-lens 1b is configured to separate the broadband light reflected from the target object 30 into at least four target polarization states and focus them. That is, in this embodiment, the multifocal vector meta-lens 1b only integrates the multi-polarization state beam splitting and polarization detection function and the lens focusing function, and does not have the beam dispersion function. At this time, the metasurface structure on the multifocal vector meta-lens 1b superimposes the grating phase that realizes the target polarization state and the focusing phase that realizes the lens focusing, which can avoid the loss of energy of the polarization part in the broadband light, thereby improving the overall efficiency of the imaging device 10b, and has the advantages of high integration and high efficiency.

[0175] Furthermore, in this embodiment, the optical modulation system further includes a microlens array 7 disposed on the side of the multifocal vector meta-lens 1b facing the imaging detector 3 and arranged sequentially along the optical axis, and a filter array 8 corresponding to the microlens array 7. The broadband light is separated into narrowband light of the target wavelength by the cooperating microlens array 7 and filter array 8.

[0176] Specifically, the microlens array 7 includes multiple coplanar microlenses arranged in a matrix, and the filter array 8 includes multiple coplanar filters arranged in a matrix and of different wavelengths, with the positions of the microlenses and the filters corresponding one-to-one.

[0177] In one specific embodiment, the filter array 8 and the imaging detector 3 are designed as an integrated unit, thereby improving the integration of the imaging device 10b.

[0178] In this embodiment, since the dispersion function is achieved through the filter array 8, that is, the filter array 8 has separated the images of different wavelengths, there is no need to perform the spectral reconstruction algorithm in the first embodiment. That is, the data processing module in this embodiment only needs to set the polarization reconstruction algorithm and the depth reconstruction algorithm to obtain the depth information of the target object 30 based on the polarization information of broadband light.

[0179] The third embodiment of the present invention is the same as the first embodiment except for the differences mentioned above, and will not be described again here.

[0180] At Figure 15 The image shown illustrates a fourth embodiment of the imaging device 10c. This fourth embodiment differs from the third embodiment in that the multifocal vector metalens 1c is only configured to separate the broadband light reflected from the target object 30 into at least four target polarization states. Specifically, the multifocal vector metalens 1c has a grating phase to realize the target polarization states. The multifocal vector metalens 1c only has multi-polarization state beam splitting and polarization detection functions and does not have a lens focusing function. In this case, the optical modulation system further includes a focusing lens 2 disposed on the side of the multifocal vector metalens 1c facing the imaging detector 3. Specifically, the focusing lens 2 is disposed between the multifocal vector metalens 1c and the microlens array 7, and the focusing lens 2 is spaced apart from the multifocal vector metalens 1c. The focusing lens 2 is used to focus the light split by the multifocal vector metalens 1c onto the microlens array 7.

[0181] In this embodiment, the multifocal vector meta-lens 1c has no lens focusing function at all; the focusing function of the optical modulation system is entirely undertaken by the focusing lens 2, which can improve image quality. In other embodiments, the multifocal vector meta-lens 1c can also be configured to have partial lens focusing function, and the focusing lens 2 can have another partial lens focusing function. In this case, the multifocal vector meta-lens 1c is also configured to superimpose the focusing phase.

[0182] Like the third embodiment, this implementation achieves the dispersion function through the filter array 8. That is, the filter array 8 has already separated images of different wavelengths. Therefore, the spectral reconstruction algorithms of the first and second embodiments are not required. In other words, the data processing module in this embodiment only needs to set up polarization reconstruction and depth reconstruction algorithms. Furthermore, since this embodiment has a focusing lens 2 spaced apart from the multifocal vector meta-lens 1c, the depth reconstruction algorithm in this embodiment can be either the depth reconstruction algorithm of the first embodiment based on the polarization information of broadband light to obtain the depth information of the target object 30, or the depth reconstruction algorithm of the second embodiment based on the principle of binocular parallax to obtain the depth information of the target object 30. Specific depth reconstruction algorithms can be found in the first and second embodiments, and will not be repeated here.

[0183] The fourth embodiment of the present invention is the same as the first, second and third embodiments except for the differences mentioned above, and will not be described again here.

[0184] Compared with existing technologies, the imaging devices 10, 10a, 10b, and 10c of this invention, by setting multi-focal vector metalenses 1, 1a, 1b, and 1c in the optical modulation system, at least achieve multi-polarization state beam splitting and polarization detection functions. That is, by using multi-focal vector metalenses 1, 1a, 1b, and 1c to replace optical elements in traditional spectral polarization imaging systems, the loss of energy in the polarization portion of broadband light can be avoided, thereby improving the overall efficiency of the imaging devices 10, 10a, 10b, and 10c. At the same time, the imaging devices 10, 10a, 10b, and 10c also have advantages such as being lighter overall, having higher functional integration, and higher light field control efficiency, which is conducive to the development of compact spectral polarization imaging devices. In addition, the spectral atlas 31 with polarization information can be obtained in real time through a single imaging, thereby obtaining the spectral and polarization information of the broadband light reflected by the target object 30, improving the efficiency of the imaging devices 10, 10a, 10b, and 10c, and enabling better identification of the target object 30.

[0185] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0186] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. An imaging device, comprising a light source module, a light modulation system, and an imaging detector; characterized in that: The light source module is used to emit broadband light within the target wavelength range to the target object; The optical modulation system includes a multifocal vector metalens, which is configured to separate broadband light reflected from the target object into at least four target polarization states. The optical modulation system separates the broadband light reflected from the target object into at least four target polarization states and into light of multiple target wavelengths, and focuses them onto different imaging regions of the imaging detector to form multiple spectral sets that correspond one-to-one with the target polarization states. Each spectral set includes multiple sub-maps that correspond one-to-one with the target wavelengths. The focal plane of the imaging detector has an imaging region that corresponds one-to-one with the polarization state of the target.

2. The imaging device of claim 1, wherein: The multifocal vector meta-lens has a grating phase that realizes the target polarization state and a periodic arrangement that disperses broadband light into multiple target wavelengths.

3. The imaging device of claim 1, wherein: The multifocal vector meta-lens has a grating phase that realizes the target polarization state; the optical modulation system further includes a microlens array disposed on the side of the multifocal vector meta-lens facing the imaging detector and arranged sequentially along the optical axis, and a filter array corresponding to the microlens array.

4. The imaging device of claim 2 or 3, wherein: The optical modulation system further includes a focusing lens disposed on the side of the multifocal vector meta-lens facing the imaging detector. The focusing lens is spaced apart from the multifocal vector meta-lens and is used to focus the light after it has been split by the multifocal vector meta-lens onto different imaging areas of the imaging detector.

5. The imaging device of claim 1, wherein: Different imaging regions are phase-separated.

6. The imaging device of claim 5, wherein: The target wavelength range includes the maximum target wavelength and the minimum target wavelength; the multi-focal vector meta-lens is designed such that the spectral maps corresponding to the maximum and minimum target wavelengths are tangent to the boundaries of the corresponding imaging regions.

7. The imaging device of claim 5, wherein: The diffraction angle, target wavelength, and period of the metasurface structure in the multifocal vector metalens satisfy the grating equation: where n0is the refractive index of the medium in the incident direction; k0is the incident wave vector 2π / λ0, θ0is the incident angle in the x direction, m is the diffraction order, which can be 0, +1, +2, etc.; P0is the x direction period of the super surface structure in the multi-focal vector metasurface lens, n m is the refractive index of the medium in the exit direction, θ m is the diffraction angle.

8. The imaging device of claim 1, wherein: The multifocal vector metalens is configured to separate broadband light reflected from the target object into at least four target polarization states to obtain at least four polarization-sensitive beams, and the sub-image in the spectral set corresponding to the polarization-sensitive beams is a polarization sub-image; or, the multifocal vector metalens is configured to separate broadband light reflected from the target object into at least four target polarization states to obtain at least four polarization-sensitive beams and at least one polarization-insensitive beam; the sub-image in the spectral set corresponding to the polarization-sensitive beams is a polarization sub-image, and the sub-image in the spectral set corresponding to the polarization-insensitive beams is an intensity sub-image.

9. A three-dimensional information acquisition method based on the imaging device according to any one of claims 1 to 8, characterized by: include: Acquire multiple spectral atlases formed on the imaging detector; Multiple spectral sets are processed by spectral reconstruction algorithm to obtain an imaging map corresponding to each target wavelength to obtain the spectral information of broadband light reflected by the target object. Each imaging map includes a sub-map that corresponds one-to-one with the polarization state of the target, and the sub-map includes at least four polarization sub-maps with polarization information. The polarization information corresponding to each target wavelength in the reflected broadband light is calculated based on at least four polarization sub-maps in each imaging image.

10. The three-dimensional information acquisition method according to Claim 9, wherein: The three-dimensional information acquisition method further includes: calculating the depth information of the target object based on the parallax of at least two sub-images in any imaging image and / or the polarization information corresponding to light of any target wavelength.

11. The three-dimensional information acquisition method according to Claim 10, wherein: The sub-image includes at least two intensity sub-images without polarization information. "Calculating the depth information of the target object based on the disparity of at least two sub-images in any imaging image" specifically means: calculating the depth information of the target object based on the disparity of at least two intensity sub-images in any imaging image.

12. The three-dimensional information acquisition method according to Claim 10, wherein: After calculating the depth information of the target object corresponding to all target wavelengths, the three-dimensional information acquisition method further includes the following steps: taking the average value of the depth information corresponding to all target wavelengths to obtain the final depth information of the target object.

13. The three-dimensional information acquisition method according to Claim 12, wherein: Before averaging the depth information corresponding to all target wavelengths, the three-dimensional information acquisition method further includes the following steps: The depth information of the target object corresponding to all target wavelengths is obtained through comparison and calculation; Determine if the depth information corresponding to a certain target wavelength deviates too much from the depth information corresponding to other target wavelengths. If so, remove the depth information corresponding to the target wavelength with excessive deviation, and take the average value of the remaining depth information corresponding to the target wavelengths to obtain the final depth information of the target object. If not, the final depth information of the target object is obtained by averaging the depth information corresponding to all target wavelengths.

14. A method of three-dimensional reconstruction, characterized by: The three-dimensional reconstruction method is based on multiple spectral atlases acquired by the imaging device according to any one of claims 1 to 8; or the three-dimensional reconstruction method is based on the spectral information, polarization information, and / or depth information of the broadband light reflected by the target object obtained by the three-dimensional information acquisition method according to any one of claims 9 to 13.

Images