Full Stokes camera based on sub-aperture polarization imaging and use method

Abstract

The invention relates to a full Stokes camera based on aperture-division polarization imaging and a use method, and relates to the technical field of machine vision, the camera comprises a polarization grating, a micro-lens array and an image sensor, the polarization grating is arranged between an objective lens front group and an objective lens rear group; the micromirror array is arranged at the real image plane of the objective lens front group and the objective lens rear group; the image sensor is arranged on one side, far away from the objective lens rear group, of the microlens array and is close to the focus of the microlens array; the microlens array is provided with a plurality of microlenses, each microlens corresponds to a 2 * 2 pixel area on the image sensor, and the image sensor can collect quadruple polarization information of an image point of each microlens. The diaphragm formed by matching the QuadBayerCoding 2 * 2On-ChipLens technology with the optical filters with different polarization characteristics has the advantages of small system size, full pixel utilization, low cost and good compatibility, and is beneficial to deploying the polarization visual system in a narrow space with lower cost.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a full-Stokes camera based on aperture-divided polarization imaging and a method of using the camera. Background Art

[0002] Polarization is one of the important physical properties of light. When an object emits, reflects, or scatters light, it will also carry or change the polarization state of the light. The specific polarization state information is closely related to the surface shape, surface roughness, texture direction, and physical and chemical properties of the object. This gives polarization imaging unique advantages in three-dimensional measurement of highly reflective and texture-free objects.

[0003] The polarization state of light is often described by the Stokes vector. The Stokes vector contains four parameters, including light intensity, polarization degree, polarization angle, and circular polarization component. An imaging system that can measure all four parameters is called a full Stokes vector polarization imaging system.

[0004] The scheme to achieve full Stokes polarization imaging mainly relies on time-sharing, amplitude-sharing, multi-aperture, aperture-sharing, and polarization sensors. The time-sharing system uses multiple shots by rotating and replacing polarizers to achieve the measurement of four parameters. The time resolution is low and it is suitable for shooting static or slow-changing objects; the amplitude-sharing system uses selective polarization reflectors before imaging to separate polarized light in different directions in turn, and uses multiple cameras to capture the target; the multi-aperture system uses four cameras to shoot at the same time, the system is large and the parallax is obvious when shooting at close range; the aperture-sharing system uses a special imaging system to image images of different sub-apertures in different areas of the sensor, and detects with filters, and the required imaging system is relatively complex; the focal plane polarization sensor has the characteristics of high time resolution and small size, but the extinction is relatively low after the polarizer is miniaturized, and some sensors can only detect linear polarization components, and cannot obtain full Stokes parameters with high quality.

[0005] Light field cameras based on microlens arrays can output images of different sub-apertures. They are small in size and can be combined with general optical systems. By installing filters of different properties at the aperture, polarization images that support full Stokes vector calculations can be collected. However, light field cameras have low pixel utilization and only support objectives with the same F value as the microlens array, thus limiting their scope of application.

[0006] The QuadBayerCoding 2×2On-ChipLens technology (QBC 2×2OCL solution) based on 2×2 on-chip microlenses has been widely used in mobile phone camera module sensors in recent years to achieve fast and robust focusing functions. Its structure is similar to that of a light field camera, but each microlens corresponds to only 2×2 sub-pixels, and can capture images of 4 sub-apertures. At the same time, since the 2×2 sub-pixel structure can be regarded as a fan-shaped division around the center, even if combined with objective lenses with different aperture values, the four pixels can be normally sensitive to light, avoiding the pixel waste problem caused by the different F values ​​of the objective lens and the microlens in the light field camera. Therefore, the use of QuadBayerCoding 2×2On-ChipLens technology (QBC 2×2OCL solution) with an aperture composed of filters with different polarization characteristics can support the acquisition of the full Stokes vector, and has the advantages of small system size, full pixel utilization, low cost, and good compatibility, which is conducive to the deployment of polarization vision systems in small spaces at a lower cost. Summary of the invention

[0007] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present application is to provide a full-Stokes camera based on aperture-divided polarization imaging and a method of use, which has the characteristics of small size, light weight and good compatibility.

[0008] To achieve the above-mentioned invention object, the present application provides a full-Stokes camera based on aperture-divided polarization imaging, comprising:

[0009] A polarization grating is arranged between the front group and the rear group of the objective lens;

[0010] A microlens array, wherein the microlens array is arranged at the real image plane of the front group of the objective lens and the rear group of the objective lens;

[0011] An image sensor is disposed on a side of the microlens array away from the rear group of the objective lens and close to the focus of the microlens array;

[0012] The microlens array has a plurality of microlenses, each of which corresponds to a 2×2 pixel area on the image sensor, and the image sensor can collect quadruple polarization information of each microlens image point.

[0013] Furthermore, the polarization aperture includes four polarization filters with different filtering characteristics, and the four polarization filters are evenly installed in the polarization aperture frame.

[0014] Furthermore, the four polarization filters are all fan-shaped structures;

[0015] The polarizing aperture lens frame comprises an annular lens frame body, in which a cross partition frame is arranged, and the cross partition frame divides the annular frame body into four sector-shaped grooves;

[0016] The four polarization filters are installed in the corresponding four fan-shaped grooves.

[0017] Furthermore, the polarizing filter frame is mounted on a filter switcher, and the filter switcher has a plurality of filters, and the plurality of filters and the polarizing filter frame are arranged in a ring shape along the center of the filter switcher.

[0018] Furthermore, the filter switcher, the microlens array and the image sensor are all arranged in the camera body.

[0019] Furthermore, the focal length of the microlens array is 4 μm.

[0020] Furthermore, the gaps between adjacent microlenses are filled with light absorbing material, and the F value of the microlenses is 1.

[0021] The present application provides a method for using a full-Stokes camera based on split-aperture polarization imaging, comprising the following steps:

[0022] S1. Place the polarization diaphragm between the front and rear objective lenses, and adjust the angle so that the cross-shaped partition frame is aligned with the long and wide sides of the image sensor.

[0023] S2, adjust the shutter speed and image sensor gain parameters on the camera body to ensure normal exposure of the image;

[0024] S3, moving the camera body to focus and obtain a clear sub-aperture preview molten pool image;

[0025] S4, take a molten pool image or start recording a video;

[0026] S5, when the acquisition is completed, the four pixels under the microlens can be extracted respectively and rearranged in the order of the microlenses to form four images, that is, four groups of sub-aperture images;

[0027] S6. By performing calculations on each frame of the four groups of images or videos, the corresponding Stokes vector of each image point can be obtained, thus completing a shooting process.

[0028] Compared with the prior art, this application has the following beneficial effects:

[0029] 1. The size and mass of the system in this application are comparable to those of traditional machine vision systems, and can be easily deployed in smaller spaces, such as above the molten pool of additive manufacturing or welding equipment.

[0030] 2. This application is compatible with the imaging lenses of existing machine vision systems. Compared with aperture-based and amplitude-based systems, it greatly reduces the cost of acquiring imaging lenses and provides more options.

[0031] 3. This application does not require adding a new optical system behind the lens, has low requirements on the optical back focus of the objective lens, supports the design of a short back focus system, and is conducive to the design of an objective lens with high imaging quality.

[0032] 4. This application is compatible with a wider variety and grades of polarization filter elements. Compared with some existing split-focus plane polarization sensors, it can obtain polarization images with higher signal-to-noise ratio when using high-quality filters, and is more flexible in polarization element configuration to meet different image acquisition requirements.

[0033] 5. This application not only supports the acquisition of black and white images, but also supports the acquisition of color images, and can provide more object information compared to a simple black and white polarization sensor.

[0034] 6. Compared with the polarization imaging system based on light field camera, this application has the advantage of high pixel utilization and can provide images with high spatial resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the implementation methods are briefly introduced below. Obviously, the drawings described below are only some implementation methods of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A schematic diagram schematically shows the internal structure decomposition of a full-Stokes camera based on split-aperture polarization imaging of the present application;

[0037] Figure 2 Schematic representation Figure 1 Front view of the middle polarizing aperture;

[0038] Figure 3 Schematic representation Figure 1 Schematic diagram of a single pixel of the omnidirectional focus sensor.

[0039] In the figure:

[0040] 10. Polarization aperture; 11. Polarization filter; 12. Polarization aperture frame; 13. Ring frame body; 14. Cross divider;

[0041] 20. Objective lens front group;

[0042] 30. Objective lens rear group;

[0043] 40. microlens array; 41. microlens; 42. pixel;

[0044] 50. Image sensor;

[0045] 60. Filter switcher; 61. Filter. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] A full-Stokes camera based on split-aperture polarization imaging described in an embodiment of the present application includes:

[0048] A polarization grating 10, wherein the polarization grating 10 is disposed between the objective lens front group 20 and the objective lens rear group 30;

[0049] A microlens array 40, wherein the microlens array 40 is disposed at the real image plane of the objective lens front group 20 and the objective lens rear group 30, the objective lens front group 20 and the objective lens rear group 30 are optical lenses, and the optical system formed by the two lenses has a specification of 6mmF1.5;

[0050] An image sensor 50, wherein the image sensor 50 is disposed on a side of the microlens array 40 away from the rear lens group 30 and close to the focus of the microlens array 40;

[0051] The microlens array 40 has a plurality of microlenses 41 , each of the microlenses 41 corresponds to a 2×2 pixel area on the image sensor 50 , and the image sensor 50 can collect quadruple polarization information of an image point of each microlens 41 .

[0052] The polarization aperture 10 includes four polarization filters 11 with different filtering characteristics, and the four polarization filters 11 are evenly installed in the polarization aperture frame 12. The four polarization filters 11 are all fan-shaped structures, and the polarization aperture frame 12 includes an annular frame body 13, and a cross partition frame 14 is arranged in the annular frame body 13, and the cross partition frame 14 divides the annular frame body 13 into four fan-shaped grooves; the four polarization filters 11 are respectively installed in the corresponding four fan-shaped grooves, and the four polarization filters 11 are distributed in four quadrants, and the four polarization filters 11 can respectively select 0° linear polarization plate, 45° linear polarization plate, 90° linear polarization plate, 135° linear polarization plate, or be combined with blank, 1 / 4 wave plate, and 1 / 2 wave plate.

[0053] In specific operation, light passing through the front lens group 20 will be included in the polarization aperture 10, and will be polarized by the four polarization filters 11 distributed in the four quadrants, and form a real image under the action of the rear lens group 30. The cross-shaped partition frame 14 of the polarization filter frame 12 is consistent with the pixel arrangement direction of the image sensor 50, and can have a certain width to prevent a certain light beam from passing through two polarization filters 11 at the same time, and the center can be shielded to prevent the four quadrants from interfering with each other.

[0054] Each microlens 41 corresponds to an image point, and projects the light constituting the image point onto the pixel 42 under the microlens behind it in a certain direction. Each pixel under the microlens 41 includes 2×2 sub-pixels and can collect light from four polarization filters 11. The pixel under the microlens 41 is usually covered with a Bayer filter that provides color.

[0055] The original image collected by the image sensor 50 contains the quadruple polarization information of each image point. The pixels with the same coordinates are taken out from the microlens sub-pixels 42 behind each microlens 41 and rearranged in the order of the microlens array 40 to obtain four polarization filter 11 sub-aperture images of the photographed object. The sub-images are divided into four groups according to the coordinates, corresponding to the four polarization states. Then, the Stokes component of each image point can be obtained by calculation.

[0056] The microlens array 40 and the image sensor 50 of the present application can be assembled separately, and the two constitute an omnidirectional focusing sensor.

[0057] The polarizing filter frame 12 is mounted on a filter switcher 60, and the filter switcher 60 has a plurality of filters 61. The plurality of filters 61 and the polarizing filter frame 12 are arranged in a ring shape along the center of the filter switcher 60. The filter switcher 60, the microlens array 40 and the image sensor 50 are all arranged in the camera body.

[0058] By rotating the filter switch 60, different filters or polarization apertures 10 can be located between the objective lens front group 20 and the objective lens rear group 30, thereby realizing switching between polarization imaging and general imaging modes, or switching other filters.

[0059] The present application provides a full-Stokes camera based on aperture polarization imaging, wherein four polarization filters 11 are respectively a 50% grayscale filter, a 0° linear polarization filter, a 45° linear polarization filter, and a right-hand circular polarization filter combination, and the polarization diaphragm 10 can be rotated out of the objective lens diaphragm position through a gear mechanism; the microlens array 40 adopts a 6000×4000 convex lens array with a focal length of 4μm and an aperture of 4μm, and the microlens F value is 1; the gaps between adjacent microlenses are filled with light-absorbing materials; each microlens corresponds to an area of ​​2×2 pixels on an image sensor, and the specific method of use includes the following steps:

[0060] S1, place the polarization stop 10 between the objective lens front group 20 and the objective lens rear group 30, and adjust the angle so that the cross-type partition frame 14 and the image sensor 50 have the same length and width; a neutral density filter and a narrow-band filter can be added before the adjusted imaging system;

[0061] S2, adjusting the shutter speed and the image sensor 50 gain parameters on the camera body to ensure normal exposure of the image;

[0062] S3, moving the camera body to focus and obtain a clear sub-aperture preview molten pool image;

[0063] S4, take a molten pool image or start recording a video;

[0064] S5, when the acquisition is completed, the four pixels under the microlens can be extracted respectively and rearranged in the order of the microlenses to form four images, that is, four groups of sub-aperture images;

[0065] S6. By performing calculations on each frame of the four groups of images or videos, the corresponding Stokes vector of each image point can be obtained, thus completing a shooting process.

[0066] The calculation method is: let the pixel intensity values ​​corresponding to the four polarization filters 11 under a certain microlens 41 be I0, I1, I2, and I3 respectively. Then the calculation method of each component of the Stokes vector (S0, S1, S2, S3) of the image point is:

[0067] S0=2I0;

[0068] S1=2I1-2I0;

[0069] S2=2I2-I0;

[0070] S3=2I3-2I0.

[0071] The size and mass of the system in this application are comparable to those of traditional machine vision systems, and it is convenient to deploy in smaller spaces, such as above the molten pool of additive manufacturing or welding equipment; pixels are fully utilized and the spatial resolution is high. This application is compatible with the imaging lenses of existing machine vision systems and can be used without or with minimal modification; compared with aperture-based and amplitude-based systems, the cost of acquiring imaging lenses is greatly reduced and more options are provided. This application does not require the addition of a new optical system behind the lens, has low requirements for the optical back intercept of the objective lens, supports the design of a short back intercept system, and is conducive to the design of an objective lens with high imaging quality.

[0072] This application is compatible with a wide variety and grades of polarization filter elements. Compared with some existing split-focus plane polarization sensors, it can obtain polarization images with higher signal-to-noise ratio when using high-quality filters, and is more flexible in polarization element configuration to meet different image acquisition needs. This application supports the acquisition of color images and can provide more object information than a simple black and white polarization sensor. Compared with the polarization imaging system based on the light field camera, this application has the advantage of high pixel utilization and can provide images with high spatial resolution.

[0073] The present application is compatible with general photographic objective lens systems; supports combinations of photographic objective lenses with different aperture values; and facilitates switching between polarized imaging and general imaging modes.

[0074] In addition, it should be noted that the present application can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0075] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0078] Finally, it should be noted that the above is a preferred embodiment of the present application. It should be pointed out that although the preferred embodiment of the present application has been described, once the basic creative concept of the present application is known, a number of improvements and modifications can be made by a person skilled in the art without departing from the principles described in the present application, and these improvements and modifications should also be regarded as the protection scope of the present application. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present application.

Claims

1. A full-Stokes camera based on aperture polarization imaging, characterized in that: A polarization diaphragm (10), wherein the polarization diaphragm (10) is arranged between the objective lens front group (20) and the objective lens rear group (30); A microlens array (40), wherein the microlens array (40) is arranged at the real image plane of the objective lens front group (20) and the objective lens rear group (30); An image sensor (50), the image sensor (50) being arranged on a side of the microlens array (40) away from the rear group of objective lenses (30) and close to a focal point of the microlens array (40); The microlens array (40) has a plurality of microlenses (41), each of the microlenses (41) corresponds to a 2×2 pixel area on an image sensor (50), and the image sensor (50) is capable of collecting quadruple polarization information of an image point of each microlens (41).

2. A full-Stokes camera based on split-aperture polarization imaging according to claim 1, characterized in that: The polarization aperture (10) comprises four polarization filters (11) with different filtering characteristics, and the four polarization filters (11) are evenly installed in the polarization aperture frame (12).

3. A full-Stokes camera based on split-aperture polarization imaging according to claim 2, characterized in that: The four polarization filters (11) are all fan-shaped structures; The polarizing aperture lens frame (12) comprises an annular lens frame body (13), a cross partition frame (14) is arranged inside the annular lens frame body (13), and the cross partition frame (14) divides the annular frame body (13) into four sector-shaped grooves; The four polarization filters (11) are respectively installed in the corresponding four sector-shaped grooves.

4. The full-Stokes camera based on split-aperture polarization imaging according to claim 2, characterized in that: The polarized aperture frame (12) is mounted on a filter switcher (60), and the filter switcher (60) has a plurality of filters (61). The plurality of filters (61) and the polarized aperture frame (12) are arranged in a ring shape along the center of the filter switcher (60).

5. A full-Stokes camera based on split-aperture polarization imaging according to claim 4, characterized in that: The filter switch (60), the microlens array (40) and the image sensor (50) are all arranged in a camera body.

6. The full-Stokes camera based on split-aperture polarization imaging according to claim 1, characterized in that: The focal length of the microlens array (40) is 4 μm.

7. The full-Stokes camera based on split-aperture polarization imaging according to claim 1, characterized in that: The gaps between adjacent microlenses (41) are filled with light absorbing material, and the F value of the microlenses (41) is 1.

8. A method for using a full-Stokes camera based on split-aperture polarization imaging, comprising the following steps: S1, placing the polarization diaphragm (10) between the front group (20) and the rear group (30) of the objective lens, and adjusting the angle so that the cross-type partition frame (14) and the image sensor (50) have the same length and width; S2, adjusting the shutter speed and the gain parameters of the image sensor (50) on the camera body to ensure that the image is exposed normally; S3, moving the camera body to focus and obtain a clear sub-aperture preview molten pool image; S4, take a molten pool image or start recording a video; S5, when the acquisition is completed, the four pixels under the microlens can be extracted respectively and rearranged in the order of the microlenses to form four images, that is, four groups of sub-aperture images; S6. By performing calculations on each frame of the four groups of images or videos, the corresponding Stokes vector of each image point can be obtained, thus completing a shooting process.

Images