Optical system, three-dimensional information acquisition method, and three-dimensional reconstruction method

Abstract

The invention provides an optical system, a three-dimensional information acquisition method and a three-dimensional reconstruction method.The optical system comprises a super-structure grating, a focusing lens and an imaging detector which are sequentially arranged along an optical axis, and the super-structure grating splits reflected light beams of a target object on at least four different diffraction levels to obtain at least four polarization sensitive light beams; the focusing lens is used for focusing the light beams split by the super-structure grating to different imaging areas of the imaging detector to form a plurality of sub-images, and the plurality of sub-images comprise polarization sub-images in one-to-one correspondence with the at least four polarization sensitive light beams; the image processing module is in communication connection with the imaging detector, and the image processing module calculates the depth information of the target object according to the parallax of the at least two sub-images and calculates the polarization information of the reflected light beam according to the at least four polarization sub-images; depth information and polarization information can be obtained through single shooting, and the optical system is simple and compact in size.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, and in particular to an optical system, a three-dimensional information acquisition method, and a three-dimensional reconstruction method. Background Technology

[0002] Compared with two-dimensional images, three-dimensional data of the surface of a target object can provide more comprehensive feature information. 3D reconstruction technology has been applied in many fields such as surveying and navigation, identity recognition, unmanned systems, and cultural relic protection.

[0003] Polarization-based 3D reconstruction technology offers advantages such as long-distance operation, high precision, low cost, and no need for light source modulation. Polarization-based 3D reconstruction primarily acquires the polarization state information of reflected light from the surface of a target object, then estimates the 3D depth data of the target object based on this information. Simultaneously, the polarization state information can also reflect the physicochemical characteristics of the target object, such as material and roughness, as well as geometric information such as surface properties and distribution. It has a particularly strong 3D reconstruction capability for the detailed texture information of the target object's surface. Specifically, polarization-based 3D reconstruction mainly obtains the surface normal vector of the target object based on the polarization state information to achieve 3D reconstruction of the target object's shape. However, the technology for obtaining the surface normal vector of the target object through polarization information is still immature. Directly obtaining the surface normal vector from polarization information will result in severe distortion, leading to blurring and flipping of the concavity and convexity of some reconstructed surface shapes.

[0004] The 3D reconstruction technology based on binocular cameras has good accuracy in the low-frequency part of the surface depth data it acquires, which complements the surface results of polarization 3D reconstruction. Moreover, obtaining depth data using binocular cameras is simpler than other technologies such as 3D scanning and multi-view reconstruction, and can be applied to more practical scenarios.

[0005] Based on the complementarity between polarization-based 3D reconstruction technology and binocular camera technology, some existing users, when performing 3D reconstruction, acquire polarization state information through a polarization imaging system and surface depth information of the target object through a binocular camera system. Then, they simultaneously perform 3D reconstruction based on both the polarization state information and the surface depth information acquired by the binocular system. This approach requires both a polarization imaging system and a binocular camera system, which results in a large optical system with relatively poor stability. Summary of the Invention

[0006] The purpose of this invention is to provide an optical system, a three-dimensional information acquisition method, and a three-dimensional reconstruction method.

[0007] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: an optical system, comprising:

[0008] A meta-grating, a focusing lens, and an imaging detector are arranged sequentially along the optical axis. The meta-grating is configured to split the reflected beam of the target object into at least four polarization-sensitive beams at at least four different diffraction orders. The focusing lens is spaced apart from the meta-grating and is used to focus the beams split by the meta-grating onto different imaging regions of the imaging detector to form multiple sub-images. The multiple sub-images include polarization sub-images that correspond one-to-one with the at least four polarization-sensitive beams.

[0009] An image processing module is communicatively connected to the imaging detector. The image processing module is configured to acquire the plurality of sub-images, calculate the depth information of the target object based on the parallax of at least two sub-images, and calculate the polarization information of the reflected beam based on at least four polarization sub-images.

[0010] As a further improvement of the present invention, the meta-grating is configured to split the reflected beam at at least five different diffraction orders to obtain at least four polarization-sensitive beams and at least one polarization-insensitive beam, wherein the plurality of sub-graphs include intensity sub-graphs that correspond one-to-one with the at least one polarization-insensitive beam.

[0011] As a further improvement of the present invention, the meta-grating is configured to split the reflected beam to obtain at least two polarization-insensitive beams; the image processing module calculates the depth information of the target object based on the parallax of at least two intensity sub-images.

[0012] As a further improvement of the present invention, the meta-grating is configured to split the reflected beam to obtain two polarization-insensitive beams; the two imaging regions on the imaging detector corresponding to the two intensity sub-maps are located on opposite sides of the center of the imaging detector in the horizontal direction; or the two imaging regions on the imaging detector corresponding to the two intensity sub-maps are located on opposite sides of the center of the imaging detector in the vertical direction.

[0013] As a further improvement of the present invention, the imaging region on the imaging detector is uniformly distributed around the center of the imaging detector.

[0014] As a further improvement of the present invention, the image processing module is configured to calculate the depth information of the target object based on the parallax of two sub-images located on opposite sides of the center of the imaging detector.

[0015] As a further improvement of the present invention, the optical system further includes a first lens disposed at a distance from the metagrating on the light-incident side of the metagrating, the first lens being used to expand the field of view.

[0016] As a further improvement of the present invention, the focusing lens is a focusing lens with a metasurface structure.

[0017] To achieve the above-mentioned objectives, the present invention also provides a method for acquiring three-dimensional information, comprising the following steps:

[0018] Multiple sub-images formed on the imaging detector are acquired. The multiple sub-images are formed by splitting the reflected beam of the target object into at least four different diffraction orders by a meta-grating and then focusing it onto different imaging areas of the imaging detector by a focusing lens. The multiple sub-images include at least four polarization sub-images.

[0019] The polarization information of the reflected beam is calculated based on at least four polariton diagrams;

[0020] The depth information of the target object is calculated based on the disparity of at least two sub-images.

[0021] As a further improvement of the present invention, the plurality of sub-images includes at least two intensity sub-images that do not contain polarization information; "calculating the depth information of the target object based on the disparity of at least two sub-images" specifically means: calculating the depth information of the target object based on the disparity of at least two intensity sub-images.

[0022] As a further improvement of the present invention, "calculating the depth information of the target object based on the disparity of at least two sub-images" specifically includes the following steps:

[0023] The at least two subgraphs were preprocessed using MATLAB.

[0024] The pixels of the at least two sub-images are matched to obtain the optimal pixel matching result;

[0025] Calculate the disparity value between matching pixels in the at least two sub-graphs;

[0026] Based on the principle of binocular parallax, the parallax value is converted into depth information.

[0027] To achieve the above-mentioned objectives, the present invention also provides a three-dimensional reconstruction method, wherein the three-dimensional reconstruction method is based on the depth information of the target object and the polarization information of the reflected beam obtained by the above-mentioned optical system; or the three-dimensional reconstruction method is based on the depth information of the target object and the polarization information of the reflected beam obtained by the above-mentioned three-dimensional information acquisition method.

[0028] The beneficial effects of this invention are as follows: The optical system of this invention can obtain depth and polarization information in a single shot using only the meta-grating and focusing lens. The optical system is simple, compact, and lower in cost. At the same time, by using a meta-grating with a metasurface structure to replace traditional optical elements, the optical system has advantages such as being lighter overall, having high functional integration, and high light field control efficiency. This is conducive to the development of compact polarization depth imaging modules, greatly reducing the size of the optical system and conforming to the current trend of miniaturization of optical systems. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the optical system in a specific embodiment of the present invention;

[0030] Figure 2 This is a simulation result of the meta-grating under natural light illumination in a specific embodiment of the present invention;

[0031] Figure 3 This is a simulation result of the meta-grating under polarized light incidence in a specific embodiment of the present invention;

[0032] Figure 4 The image is a real photograph taken using an optical system with a meta-grating as described in one specific embodiment.

[0033] Figure 5 This is a schematic diagram of the optical system in another specific embodiment of the present invention;

[0034] Figure 6 It is a schematic diagram based on the binocular parallax principle of the first and second cameras;

[0035] Figure 7 This is a schematic diagram of the depth information calculation principle of the optical system in this invention;

[0036] Figure 8 This is a real photograph of the intensity sub-map in a specific embodiment of the present invention;

[0037] Figure 9(a) is a real photograph of a specific embodiment;

[0038] Figure 9(b) shows the Stokes parameter image calculated from the polariton diagram in Figure 9(a);

[0039] Figure 9(c) shows the polarization angle and polarization degree images calculated from the polariton diagram in Figure 9(a). Detailed Implementation

[0040] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. Please refer to the accompanying drawings for further details. Figure 1Figure 9 illustrates a preferred embodiment of the present invention. However, it should be noted that these embodiments are not intended to limit the present invention. Any functional or structural equivalent modifications or substitutions made by those skilled in the art based on these embodiments are within the scope of protection of the present invention.

[0041] It should be understood that terms such as “comprising” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof, nor do they exclude the presence or addition of one or more other steps or combinations thereof.

[0042] Furthermore, it should be understood that although the terms first, second, third, fourth, etc., may be used herein to describe various elements or structures, the objects described should not be limited by these terms. These terms are used only to distinguish similar objects and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0043] Please refer to Figure 1 As shown, the present invention provides an optical system 10, which includes an image processing module (not shown) and a meta-grating 1, a focusing lens 2, and an imaging detector 3 arranged sequentially along the optical axis. The focusing lens 2 and the meta-grating 1 are spaced apart along the optical axis. A reflected beam generated by a light source 20 illuminating the surface of a target object 30 is incident on the meta-grating 1. The meta-grating 1 has a polarization-state beam splitting and polarization-detection function, configured to split the reflected beam from the target object 30 at at least four different diffraction orders to obtain at least four polarization-sensitive beams. The beams split by the meta-grating 1 are focused by the focusing lens 2 onto different imaging regions of the imaging detector 3 to form multiple sub-images, each sub-image including a polarization sub-image corresponding to one of the at least four polarization-sensitive beams. The image processing module is built into the computer and is communicatively connected to the imaging detector 3. The image processing module is configured to acquire the multiple sub-images, calculate the depth information of the target object 30 based on the parallax of at least two sub-images, and calculate the polarization information of the reflected beam based on at least four polarization sub-images. The optical system 10 of this invention can obtain depth and polarization information in a single shot using only the meta-grating 1 and the focusing lens 2. It features a simple and compact optical system 10, good imaging uniformity across fields of view, good timeliness, high spatial resolution, good anti-interference capability, and expandable functionality. Furthermore, by using the meta-grating 1 with a metasurface structure instead of traditional optical elements, the optical system 10 becomes lighter, more functionally integrated, and has higher light field control efficiency. This facilitates the development of compact polarization depth imaging modules, significantly reducing the size of the optical system 10 and conforming to the current trend of miniaturization in optical systems.

[0044] The light source 30 includes, but is not limited to, LEDs, semiconductor lasers (including edge-emitting EELs and vertical-emitting VCSELs), fiber lasers, and natural light. The reflected light beam from the surface of the target object is the incident light beam incident on the meta-grating.

[0045] In one specific embodiment, the focusing lens 2 and the imaging detector 3 can be configured as a single camera unit. Thus, the present invention can acquire the depth and polarization information of the target object 30 in a single shot using a meta-grating 1 and a single camera, resulting in a compact system size.

[0046] Specifically, the imaging detector 3 may include a CCD (Charge-coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0047] In one specific embodiment, the focusing lens 2 is a focusing lens 2 with a metasurface structure. This can further reduce the volume of the optical system 10 and improve the integration of the optical system 10.

[0048] Both the metasurface structures of the metagrating 1 and the focusing lens 2 can achieve their corresponding functions by designing the light shape, phase, and other parameters of each nanopillar in their metasurface structures. For example, the metasurface structure corresponding to the metagrating 1 can be designed with a preset grating phase, enabling the metagrating 1 to achieve multi-polarization state beam splitting and polarization analysis. Thus, the metagrating 1 can split the reflected beam at at least four different diffraction orders to obtain four polarization-sensitive beams and two polarization-insensitive beams. Similarly, the metasurface structure corresponding to the focusing lens 2 can be designed with a preset focusing phase, enabling the focusing lens 2 to focus each beam split by the metagrating 1 onto different imaging regions of the imaging detector 3 to form multiple sub-images. Of course, it is understandable that when designing metasurface structures, phases can be superimposed according to specific needs. For example, the grating phase can be superimposed with other phases that the subsequent focusing lens 2 needs to undertake to form a metagrating 1. This can change some functional phases such as image quality, optimize aberrations, and adjust brightness, thereby reducing the functions undertaken by the subsequent focusing lens 2, improving the focusing effect of the subsequent focusing lens 2, and thus improving the imaging quality.

[0049] Specifically, the design of the metasurface structure can be constructed based on a metasurface primitive structure parameter library. This library is a pre-constructed database containing relationships between metasurface primitive structures and phase distributions, or relationships between metasurface primitive structures and matrix information used to guide polarization behavior. Therefore, after determining the required phase distribution or matrix information for each metasurface structure, the corresponding metasurface primitive structure design scheme can be determined by traversing and searching the database.

[0050] After determining the metasurface structure design, micro / nano processes can be used to fabricate the metasurface structure, namely, to form a metagrating 1 and a metafocusing lens 2. In one specific embodiment, the metasurface structure is designed based on silicon material. Due to its low absorption loss and high refractive index in the near-infrared window, the design of the metasurface structure can be based on a wavelength of 940 nm. The micro / nano processes include, but are not limited to, film deposition, ultraviolet lithography, electron beam lithography (EBL), and RIE (reaction ion etching). The metasurface structure fabricated using the above processes can achieve a minimum linewidth of 30 nm and a maximum aspect ratio of 10:1, enabling high-precision vector diffraction.

[0051] Furthermore, the individual nanopillars in the metasurface structure can be formed into columnar shapes and possess subwavelength dimensions. The cross-sectional shape of the nanopillars can be extended from circular, circular, square, and square holes to free geometric shapes. The substrate in the metasurface structure can be a glass substrate or a silicon substrate, and the spaces between the nanopillars can include materials such as air, adhesive, silicon oxide, and silicon nitride as fillers.

[0052] In one optional embodiment, the meta-grating 1 is configured to split the reflected beam at at least five different diffraction orders to obtain at least four polarization-sensitive beams and at least one polarization-insensitive beam. After being focused by a focusing lens 2 onto an imaging detector 3, the polarization-sensitive beams can obtain a polarization sub-map with polarization information. The polarization-insensitive beams, after being focused by a focusing lens onto the imaging detector, can obtain an intensity sub-map without polarization information. That is, in this embodiment, the plurality of sub-maps include polarization sub-maps corresponding one-to-one with the at least four polarization-sensitive beams and intensity sub-maps corresponding one-to-one with the at least one polarization-insensitive beam. Therefore, the depth information of the target object can be calculated based on the parallax between the intensity sub-maps or the parallax between at least one intensity sub-map and at least one polarization sub-map. This reduces the influence of additional information from each sub-map on depth calculation. Of course, this is not the only option. In other alternative implementations, the meta-grating 1 can also be configured to split the reflected beam into polarization-sensitive beams, such as splitting the reflected beam into four polarization-sensitive beams at four different diffraction orders. Correspondingly, four polarization sub-maps are formed on the imaging detector 3. The image processing module calculates the depth information of the target object based on the parallax between at least two polarization sub-maps and calculates the polarization information of the reflected beam based on the four polarization sub-maps.

[0053] Combination Figure 2 As shown, in one specific embodiment, the meta-grating 1 is configured to split the reflected beam at six different diffraction orders to obtain four polarization-sensitive beams and two polarization-insensitive beams. Correspondingly, four polarization sub-maps 31 and two intensity sub-maps 32 are formed on the imaging detector 3. The image processing module calculates the depth information of the target object 30 based on the parallax of the two intensity sub-maps 32 according to the binocular parallax principle, and calculates the polarization information of the reflected beam based on the four polarization sub-maps 31. That is, in this embodiment, the intensity sub-map 32 for calculating depth information based on the binocular parallax principle and the polarization sub-map 31 for calculating polarization information can be obtained in a single shot using only the meta-grating 1 and the focusing lens 2. At the same time, the intensity sub-maps 32 and polarization sub-maps 31 are completely independent of each other, resulting in good imaging uniformity in each field of view, high spatial resolution, no influence from the external environment, good anti-interference ability, and good timeliness.

[0054] In one specific embodiment, the meta-grating 1 is configured to split the reflected beam to obtain four polarization states corresponding to four polarization-sensitive beams, which are four linear polarization states: ψ = 0°, 45°, 90°, 135°; χ = 0°, 0°, 0°, 0°. Of course, this is not a limitation; in other embodiments, the four polarization states can also be other polarization states on the Poincaré sphere, such as ellipsoidal polarization, left-handed circular polarization, right-handed circular polarization, etc.

[0055] Preferably, the imaging area on the imaging detector 3 is uniformly distributed around the center of the imaging detector. That is, the multiple sub-images formed are uniformly distributed around the center of the imaging detector. The image processing module is configured to calculate the depth information of the target object based on the parallax of two sub-images located on opposite sides of the center of the imaging detector, that is, to select the two farthest sub-images to calculate the depth information of the target object, which can make the obtained depth information more accurate.

[0056] Specifically, a rectangular coordinate system is established with the center of the imaging detector 3 as the origin, the horizontal axis passing through the origin as the X-axis, and the vertical axis passing through the origin as the Y-axis. In the embodiment where four polarization sub-maps 31 and two intensity sub-maps 32 are formed on the imaging detector 3, combined with... Figure 2 As shown, in one specific embodiment, the two imaging regions on the imaging detector 3 corresponding to the two intensity sub-maps 32 are located on opposite sides of the center of the imaging detector 3 in the horizontal direction, or on opposite sides of the center of the imaging detector 3 in the vertical direction. That is, the two intensity sub-maps 32 are located within two imaging regions corresponding to the X-axis or two imaging regions corresponding to the Y-axis. This is more conducive to the subsequent calculation of the disparity values ​​of the two intensity sub-maps 32 in the X-axis and Y-axis directions, thereby calculating the depth information of the target object 30 based on the disparity value and the binocular disparity principle.

[0057] Combination Figure 2 The image shows the simulation results of the meta-grating (four linear polarization states: ψ = 0°, 45°, 90°, 135°; χ = 0°, 0°, 0°, 0°) under natural light illumination in one specific embodiment described above. In the simulation, right-handed circularly polarized light and right-handed circularly polarized light are used to represent natural light. Figure 2 The left image (a) is a simulation diagram corresponding to right-handed circularly polarized light incident. Figure 2 Figure (b) on the right shows the simulation diagram corresponding to right-hand circularly polarized light incident. Combining Figures (a) and (b), it can be seen that the sub-patterns corresponding to the six diffraction orders are all bright and have good uniformity. This ensures better uniformity of the sub-patterns obtained when using the optical system subsequently, as the polarization sub-pattern corresponding to each polarization state can be seen, thus improving accuracy.

[0058] Combination Figure 3 The figure shows the simulation results of the meta-grating in one of the above specific embodiments under polarized light incident. Figure 3 Figure (a) shows the simulation results obtained after 0° polarized light is incident. Figure 3 Figure (b) shows the simulation results obtained after 45° polarized light incident. Figure 3 Figure (c) shows the simulation results obtained after 90° polarized light incident. Figure 3 Figure (d) shows the simulation results obtained after 135° polarized light is incident, which can demonstrate that the four polarization states designed are accurate and conform to the design.

[0059] Combination Figure 4 The images shown are actual photographs taken using an optical system with the meta-grating described in one of the specific embodiments above. Figure (a) is an actual photograph taken with incompletely polarized light incident, Figure (b) is an actual photograph taken with 0° polarized light incident, Figure (c) is an actual photograph taken with 45° polarized light incident, Figure (d) is an actual photograph taken with 90° polarized light incident, and Figure (e) is an actual photograph taken with 135° polarized light incident. Combined with... Figure 3 and Figure 4 It can be seen that, Figure 4 Real photos and Figure 3 The simulation results are basically consistent, indicating that the design and fabrication of the meta-grating 1 in this invention are without problems and conform to the design.

[0060] Combination Figure 5 As shown, in another specific embodiment of the present invention, the optical system 10a further includes a first lens 4 disposed at a distance from the meta-grating 1 on the light-incident side of the meta-grating 1, the first lens 4 being used to expand the field of view.

[0061] Furthermore, the present invention also provides a method for acquiring three-dimensional information, comprising the following steps:

[0062] Multiple sub-images formed on the imaging detector 3 are acquired. The multiple sub-images are formed by splitting the reflected beam of the target object into at least four different diffraction orders by the meta-grating 1 and then focusing it onto different imaging areas of the imaging detector 3 by the focusing lens 2. The multiple sub-images include at least four polarization sub-images.

[0063] The polarization information of the reflected beam is calculated based on at least four polariton diagrams;

[0064] The depth information of the target object 30 is calculated based on the disparity of at least two sub-images.

[0065] The three-dimensional information acquisition method described in this invention can obtain depth and polarization information in a single shot using only the meta-grating 1 and the focusing lens 2. This results in good imaging uniformity and high spatial resolution across all fields of view. It is not affected by the external environment, has good anti-interference capabilities, and good timeliness. This is beneficial for developing compact polarization depth imaging optical systems, greatly reducing the size of the optical system and conforming to the current trend of miniaturization of optical systems 10 and 10a.

[0066] Specifically, based on the aforementioned optical systems 10 and 10a, multiple sub-images required for calculating the depth and polarization information of the target object 30 can be formed on the imaging detector 3 in a single capture. The structures of the optical systems 10 and 10a are as described above and will not be repeated here.

[0067] In one specific embodiment, the plurality of sub-images includes four polarization sub-images with polarization information evenly distributed around the center of the imaging detector 3, and two intensity sub-images without polarization information. "Calculating the polarization information of the reflected beam based on at least four polarization sub-images" specifically means calculating the polarization information of the reflected beam based on the four polarization sub-images. "Calculating the depth information of the target object based on the parallax of at least two sub-images" specifically means calculating the depth information of the target object based on the parallax of the two intensity sub-images, thus avoiding mutual interference between the information of each sub-image on subsequent calculations. Of course, this is not a limitation. In other embodiments, the plurality of sub-images may also include only at least four polarization sub-images with polarization information, and the depth information of the target object can be calculated based on the same binocular parallax principle using the parallax between at least two polarization sub-images; or the plurality of sub-images may include six or even nine intensity sub-images, and the depth information of the target object can be calculated subsequently based on the six or nine intensity sub-images.

[0068] In a preferred embodiment, a rectangular coordinate system is established with the center of the imaging detector as the origin, the horizontal axis passing through the origin as the X-axis, and the vertical axis passing through the origin as the Y-axis. The two intensity sub-images are located within two imaging regions corresponding to the X-axis, which is more conducive to subsequent calculation of the disparity values ​​of the two intensity sub-images in the X-axis and Y-axis directions. Based on the disparity values ​​and the principle of binocular disparity, the depth information of the target object can then be calculated. Of course, this is not a limitation; in other embodiments, the two intensity sub-images can also be set to be located within two imaging regions corresponding to the Y-axis, which is also beneficial for subsequent calculation of the disparity values ​​of the two intensity sub-images in the X-axis and Y-axis directions.

[0069] In one specific implementation, "calculating the depth information of the target object based on the disparity of at least two sub-images" specifically means: calculating the depth information of the target object based on the disparity of two intensity sub-images. Of course, this is not a limitation.

[0070] Below, in conjunction with Figure 6 The diagram shown illustrates the principle of binocular parallax calculation, specifically explaining the calculation principle of deriving the depth information of the target object 30 based on the parallax of two images taken from different perspectives. Figure 6To acquire two images of the target object 30 by simultaneously capturing images of the target object 30 using two cameras, namely, by simultaneously capturing images of the target object 30 using a first camera 5 and a second camera 6, the depth information of the target object 30 is calculated based on the parallax of the two acquired images.

[0071] Specifically, the depth information of the target object is calculated using the binocular parallax principle formula derived from the theory of triangle similarity:

[0072] Where B represents the baseline distance between the first camera 5 and the second camera 6, f is the focal length of the camera (mm), D represents the maximum distance of the target object from the camera plane (m), pixel_x: x-direction resolution (px), zpixel_y: y-direction resolution (px), size_sensor: sensor size within the camera (mm); disp: disparity value. Considering that the resolution of the imaging detector used is the same in both the x and y directions, the above formula for the binocular disparity principle can also be simplified to:

[0073]

[0074] According to the above binocular parallax principle formula, once we know the baseline distance and parallax value disp between the two cameras (5 and 6), we can calculate the distance between the target object 30 and the camera according to the above binocular parallax principle formula, and then obtain the depth value of the target object 30. Based on the depth value and the horizontal and vertical coordinates of the pixels in the original image, a three-dimensional point cloud model can be generated. The position of each point in the three-dimensional space corresponds to a point in the camera coordinate system. This three-dimensional point cloud model represents the three-dimensional structure of the observed scene.

[0075] Specifically, "calculating the depth information of the target object based on the disparity of the two intensity sub-images" means:

[0076] Calculate the disparity value between matching pixels in the at least two intensity sub-maps;

[0077] Based on the principle of binocular parallax, the parallax value is converted into depth information.

[0078] Combination Figure 7 As shown, before calculating the disparity value between matching pixels in the at least two intensity sub-images, it is necessary to first obtain the displacement of the disparity image. Calculate the overlapping area in the subgraph, based on the displacement. The multiple sub-images formed on the imaging detector 3 are rearranged to separate the overlapping areas in the sub-images, resulting in multiple rearranged sub-images; then the disparity value is calculated based on at least two intensity sub-images in the multiple rearranged sub-images.

[0079] Specifically, taking the center of the focusing lens 2 as the origin O, as shown in the figure... Figure 7 In the coordinate system shown, the coordinates of the target object 30 are (x0, y0, z0), where label 30' represents the virtual object obtained by back-tracking the diffracted rays. It can be seen that each intensity sub-image corresponds to a virtual pinhole point O on the virtual pinhole plane. i Based on pinhole analysis, the imaging point I on the pickup plane corresponding to each intensity sub-image can be obtained. The coordinates of imaging point I on the pickup plane are: Where m is the diffraction order, x mth and y mth These represent the x and y positions of the sub-map for the m-th order diffraction; z o z1: Distance from target object 30 to focusing lens 2; z2: Distance from focusing lens 2 to imaging detector 3; a is the grating constant, and m is the diffraction order.

[0080] After obtaining the coordinates of the imaging point I on the pickup plane, the imaging point VI(x) corresponding to the intensity submap on the imaging surface is obtained. mth ,y mth ,z o ),in, d is the distance from the meta-grating 1 to the focusing lens 2. Where, I(x) mth ,y mth ,z o ) and VI(x mth ,y mth ,z o The shift Δx mapping between the two, i.e., the displacement of the disparity image:

[0081] In such Figure 2 In one specific embodiment shown, where the two imaging regions corresponding to the two intensity sub-images 32 are located on opposite sides of the center of the imaging detector 3 in the horizontal direction, after rearranging the multiple sub-images according to the displacement of the disparity image, the formula for calculating the disparity value based on the two rearranged intensity sub-images is as follows:

[0082] disp = x left -x right =x +m -x -m .

[0083] Furthermore, the distance between the pinhole points corresponding to the two intensity sub-maps is the baseline distance B in the above binocular parallax principle formula. For example... Figure 7As shown, assuming two intensity submaps include a first intensity submap and a second intensity submap, where the first intensity submap corresponds to pinhole point O1 on the virtual pinhole plane and the second intensity submap corresponds to pinhole point O2 on the virtual pinhole plane, then when calculating based on the binocular parallax principle using the first and second intensity submaps, the distance between pinhole point O1 and pinhole point O2 is the baseline distance B in the aforementioned binocular parallax principle formula. Simultaneously, the baseline distance B corresponding to the two intensity submaps can also be calculated using the following formula: B(baseline)=d*tanΔθ, where Δθ=|θ1-θ2|, and θ1 and θ2 represent...

[0084] The target diffraction angles corresponding to the two intensity sub-maps.

[0085] Specifically, in combination Figure 7 As shown, the imaging point on the pickup plane corresponding to the first intensity sub-image is I1, and the imaging point on the pickup plane corresponding to the second intensity sub-image is I2.

[0086] Specifically, similarity measures commonly used to calculate the disparity values ​​of two intensity sub-images include SSD (Sum of Squared Differences) or NCC (Normalized Cross-Correlation). Considering local information and the smoothness of the cost map, dynamic programming or similar methods are employed to aggregate and optimize the cost near each pixel to obtain a more accurate and smoother disparity map. Based on the aggregated cost map, the matching pixel with the minimum cost at each pixel location is selected to obtain the final disparity map. Typically, the matching pixel corresponding to this minimum cost is considered the best match, representing the disparity value of the same object point in both sub-images.

[0087] After obtaining the disparity values ​​of the two intensity sub-images and the baseline distance B mentioned above, the binocular disparity principle formula can be applied as follows: The disparity values ​​of the two intensity submaps are converted into depth values. The resulting disparity map shows that the larger the disparity value, the closer the object is to the camera plane, and vice versa. For example... Figure 8 The image shown is a photograph of the intensity sub-map in a specific implementation embodiment. Figure 8 Figure (a) in the figure is the left intensity subgraph; Figure 8 Figure (b) in the figure is the right intensity subgraph. Figure 8 Figure (c) in the figure is a disparity map calculated based on Figures (a) and (b). Figure 8 Figure (d) in the figure is a depth map calculated based on Figure (c) and the principle of binocular parallax.

[0088] After calculating the depth information of each pixel using the aforementioned binocular parallax principle, the 3D information acquisition method further includes the following steps: constructing a 3D point cloud image using the depth information of the matched pixels and the horizontal and vertical coordinate values ​​of each pixel in the original intensity sub-image. This 3D point cloud image represents the 3D structure of the observed scene.

[0089] Furthermore, before calculating the disparity value between matching pixels in the at least two intensity sub-maps, the three-dimensional information acquisition method further includes the following steps:

[0090] The at least two subgraphs were preprocessed using MATLAB.

[0091] The pixels of the at least two sub-images are matched to obtain the optimal pixel matching result.

[0092] Furthermore, "preprocessing the at least two sub-images using MATLAB" specifically includes spatial filtering, image rotation, brightness adjustment, and grayscale processing, so that the brightness and grayscale of all pixels in the two sub-images correspond one-to-one, which is beneficial for the subsequent calculation of disparity values ​​between corresponding pixels.

[0093] "Matching pixels from at least two sub-images to obtain the optimal pixel matching result" specifically means:

[0094] By using image classification and semantic segmentation, different regions in the sub-image are separated to better handle the detection and matching between feature points of the target object and eliminate interference from irrelevant information.

[0095] The disparity map (disparitySGM) algorithm is used to compute disparity maps between different sub-maps. It obtains information such as pixel, brightness, and orientation of feature points from pixels in different sub-maps, and matches pixels in different sub-maps, taking into account information such as brightness, pixel value, and corner points, in order to find the optimal matching result.

[0096] Obtaining the optimal pixel matching results can help us determine which pixels have similar positions in different sub-images, thus providing a basis for subsequent depth calculations.

[0097] Specifically, calculating the polarization information of the reflected beam based on at least four polariton diagrams includes the following steps:

[0098] A rectangular coordinate system is established on the imaging detector, specifically: the center of the imaging detector is taken as the origin, the horizontal axis passing through the origin is taken as the X-axis, and the vertical axis passing through the origin is taken as the Y-axis.

[0099] Pixel-level feature point registration is performed on the at least four polarization sub-images;

[0100] The polarization degree and polarization angle of each pixel of the incident beam are obtained from at least four registered polarimetric images, thus obtaining polarization information.

[0101] The following section uses four polarization sub-graphs as an example to illustrate the calculation steps for obtaining the polarization information of the reflected beam based on the four polarization sub-graphs.

[0102] The pixels of the four polarization sub-images are in one-to-one correspondence. For a pixel at a certain position (row i, column j), the intensity of the four polarization sub-images is denoted as... The four Stokes components at this location are respectively Based on the polarization states of the four polarization sub-diagrams, a 4x4 matrix A can be obtained. Each column of this matrix corresponds to a different polarization state, and any column can be represented as: [1, cos2χcos2ψ, cos2χsin2ψ, sin2χ] T The following relationship exists between them:

[0103] Based on this relationship, the four Stokes components at this location can be calculated as follows: The same principle applies to pixels at other locations, ultimately yielding the complete S0, S1, S2, S3 matrix, or image. Specifically, Figure 9(a) shows a real-world image from a specific implementation, and Figure 9(b) shows the Stokes parameter image calculated from the polariton diagram in Figure 9(a).

[0104] According to the formula for calculating polarization degree image The degree of polarization is calculated separately for each pixel; based on the polarization angle image calculation formula. The polarization angle of each pixel is calculated separately. As shown in Figure 9(c), the polarization angle image and polarization degree image are calculated based on the polariton image in Figure 9(a).

[0105] After obtaining the polarization information of each pixel of the reflected beam according to the above steps, the polarization information can be used to analyze and obtain information such as the material of the target object.

[0106] Furthermore, the present invention also provides a three-dimensional reconstruction method, which is based on the depth information of the target object and the polarization information of the reflected beam obtained by the above-mentioned optical system; or the three-dimensional reconstruction method is based on the depth information of the target object and the polarization information of the reflected beam obtained by the above-mentioned three-dimensional information acquisition method.

[0107] The three-dimensional reconstruction method includes the following steps:

[0108] Acquire the polarization information of the reflected beam and the depth information of the target object;

[0109] The normal vector of each pixel is calculated based on the polarization information of each pixel of the reflected beam;

[0110] The surface reference normal vector of each pixel is calculated based on the depth information of the target object;

[0111] The normal vector is corrected based on the surface reference normal vector to obtain the corrected normal vector;

[0112] Based on the corrected normal vector, the three-dimensional contour of the target object is reconstructed.

[0113] The step of "calculating the normal vector of each pixel based on the polarization information of each pixel of the reflected beam" specifically includes the following steps:

[0114] The zenith angle α(x,y) is obtained by solving for the relationship between the zenith angle α(x,y) and the degree of polarization ρ, where the relationship between the zenith angle α(x,y) and the degree of polarization ρ is as follows: n is the refractive index of the target object's surface. The degree of polarization ρ has a very weak dependence on n. The typical value of n for dielectrics is between 1.4 and 1.6, and is generally taken as 1.5. The azimuth angle... or It can be seen that the azimuth angle obtained using polarization information has an ambiguity of π radians;

[0115] After obtaining the zenith angle α(x,y) and azimuth angle Then, the normal vector is calculated according to the following formula:

[0116]

[0117] It can be seen that the azimuth angle obtained using polarization information has an ambiguity of π radians, and the corresponding normal vector obtained using polarization information also has an ambiguity of π radians.

[0118] The process of "calculating the surface reference normal vector for each pixel based on the depth information of the target object" specifically involves calculating the surface reference normal vector for each pixel using the following formula:

[0119] Where p(x,y) and q(x,y) are the gradients of the object's surface with respect to the x and y directions, respectively.

[0120] p(x,y)=-(Depth_p(x,y)-Depth_p(x-1,y)),

[0121] q(x, y)=-(Depth_p(x, y)-Depth_p(x, y-1)).

[0122] The process of "correcting the normal vector based on the surface reference normal vector to obtain the corrected normal vector" specifically includes the following steps:

[0123] The azimuth angle of the surface reference normal vector is calculated based on the surface reference normal vector.

[0124] Correction is performed pixel by pixel. For any pixel (x, y), the corresponding pixel is... Choose the pixel closest to the corresponding pixel in π. The azimuth angle is used as the corrected azimuth angle. Obtain the corrected normal vector

[0125]

[0126] Specifically, "the azimuth angle of the surface reference normal vector is calculated based on the surface reference normal vector." "For example, the surface reference normal vector n_p(x,y) of each pixel mentioned above can also be represented as the zenith angle αp(x,y) and azimuth angle." Format:

[0127] Based on this formula and the aforementioned formulas p(x,y)=-(Depth_p(x,y)-Depth_p(x-1,y)) and q(x,y)=-(Depth_p(x,y)-Depth_p(x,y-1)), we can conclude that: Therefore, the azimuth angle of the surface reference normal vector can be calculated.

[0128] It should be noted that the above... This refers to Azi, and That is, Azi+π.

[0129] "Based on the corrected normal vector, the 3D contour reconstruction of the target object is performed" specifically:

[0130] The relative surface height Z(x,y) of the target object is obtained by gradient integration using the corrected normal vector n_corr(x,y) for 3D contour reconstruction. The gradient integration formula is as follows:

[0131] It is known that the larger the relative height Z(x,y) of the target object's surface, the closer the object's surface is to the metasurface component.

[0132] Compared with existing technologies, the optical system of this invention can obtain depth and polarization information in a single shot using only the meta-grating and focusing lens. The optical system is simple, compact, and lower in cost. At the same time, by using a meta-grating with a metasurface structure to replace traditional optical elements, the optical system has advantages such as being lighter overall, having higher functional integration, and higher light field control efficiency. This is conducive to the development of compact polarization depth imaging modules, greatly reducing the size of the optical system and conforming to the current trend of miniaturization of optical systems.

[0133] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0134] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. An optical system, characterized in that: The optical system includes: A meta-grating, a focusing lens, and an imaging detector are arranged sequentially along the optical axis. The meta-grating is configured to split the reflected beam of the target object into at least four polarization-sensitive beams at at least four different diffraction orders. The focusing lens is spaced apart from the meta-grating and is used to focus the beams split by the meta-grating onto different imaging regions of the imaging detector to form multiple sub-images. The multiple sub-images include polarization sub-images that correspond one-to-one with the at least four polarization-sensitive beams. An image processing module is communicatively connected to the imaging detector. The image processing module is configured to acquire the plurality of sub-images, calculate the depth information of the target object based on the parallax of at least two sub-images, and calculate the polarization information of the reflected beam based on at least four polarization sub-images.

2. The optical system as described in claim 1, characterized in that: The meta-grating is configured to split the reflected beam at at least five different diffraction orders to obtain at least four polarization-sensitive beams and at least one polarization-insensitive beam, wherein the plurality of sub-maps include intensity sub-maps that correspond one-to-one with the at least one polarization-insensitive beam.

3. The optical system as described in claim 2, characterized in that: The meta-grating is configured to split the reflected beam to obtain at least two polarization-insensitive beams; the image processing module calculates the depth information of the target object based on the parallax of at least two intensity sub-images.

4. The optical system as described in claim 3, characterized in that: The meta-grating is configured to split the reflected beam to obtain two polarization-insensitive beams; the two imaging regions on the imaging detector corresponding to the two intensity sub-maps are located on opposite sides of the center of the imaging detector in the horizontal direction; or the two imaging regions on the imaging detector corresponding to the two intensity sub-maps are located on opposite sides of the center of the imaging detector in the vertical direction.

5. The optical system as described in claim 1, characterized in that: The imaging area on the imaging detector is uniformly distributed around the center of the imaging detector.

6. The optical system as described in claim 5, characterized in that: The image processing module is configured to calculate the depth information of the target object based on the parallax of two sub-images located on opposite sides of the center of the imaging detector.

7. The optical system as claimed in claim 1, characterized in that: The optical system further includes a first lens disposed at a distance from the metagrating on the light-incident side of the metagrating, the first lens being used to expand the field of view.

8. The optical system as claimed in claim 1, characterized in that: The focusing lens is a focusing lens with a metasurface structure.

9. A method for acquiring three-dimensional information, characterized in that: The three-dimensional information acquisition method includes the following steps: Multiple sub-images formed on the imaging detector are acquired. The multiple sub-images are formed by splitting the reflected beam of the target object into at least four different diffraction orders by a meta-grating and then focusing it onto different imaging areas of the imaging detector by a focusing lens. The multiple sub-images include at least four polarization sub-images. The polarization information of the reflected beam is calculated based on at least four polariton diagrams; The depth information of the target object is calculated based on the disparity of at least two sub-images.

10. The three-dimensional information acquisition method as described in claim 9, characterized in that: The plurality of sub-images includes at least two intensity sub-images that do not contain polarization information; "the depth information of the target object is calculated based on the disparity of at least two sub-images" specifically means: the depth information of the target object is calculated based on the disparity of at least two intensity sub-images.

11. The three-dimensional information acquisition method as described in claim 9, characterized in that: "Calculating the depth information of the target object based on the disparity of at least two sub-images" specifically includes the following steps: The at least two subgraphs were preprocessed using MATLAB. The pixels of the at least two sub-images are matched to obtain the optimal pixel matching result; Calculate the disparity value between matching pixels in the at least two sub-graphs; Based on the principle of binocular parallax, the parallax value is converted into depth information.

12. A three-dimensional reconstruction method, characterized in that: The three-dimensional reconstruction method is based on the depth information of the target object and the polarization information of the reflected beam obtained by the optical system according to any one of claims 1 to 8; or the three-dimensional reconstruction method is based on the depth information of the target object and the polarization information of the reflected beam obtained by the three-dimensional information acquisition method according to any one of claims 9 to 11.

Images