Incoherent holographic imaging device

The incoherent hologram imaging device addresses the issue of deteriorating resolution by dynamically adjusting optical parameters to match the subject's depth position, ensuring high-definition stereoscopic imaging.

JP7591424B2Active Publication Date: 2024-11-28NIPPON HOSO KYOKAI
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
JP2021027856
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-02-24
Publication Date
2024-11-28
Estimated Expiration
2041-02-24

AI Technical Summary

Technical Problem

Conventional incoherent holography technologies struggle with deteriorating resolution and image magnification when the depth position of the subject changes, and they lack the ability to adjust these parameters based on the imaging situation and subject position, especially when capturing videos of moving subjects.

Method used

An incoherent hologram imaging device that splits incoherent light into two paths with different spherical phases, using a movement stage to adjust the focal length, spacing, wavelength, and distance between optical components, and an image characteristic control section to optimize resolution and image magnification based on the subject's depth position.

Benefits of technology

The device can maintain high-definition stereoscopic imaging by adjusting resolution and image magnification to desired values, even when the subject's depth position changes, improving image quality and expanding the range of image expression.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591424000011
    Figure 0007591424000011
  • Figure 0007591424000012
    Figure 0007591424000012
  • Figure 0007591424000013
    Figure 0007591424000013
Patent Text Reader

Abstract

To provide an incoherent hologram imaging device with which it is possible to adjust the resolution and image magnification of a stereoscopic image even when an interval between a subject and the reference position of the incoherent hologram imaging device changes.SOLUTION: An incoherent hologram imaging device, provided with movement stages 6a, b capable of moving a lens system 3 and an imaging element 4 in a hologram optical system 30, includes an image characteristic control unit 8 for indicating that the lens system 3 on movement stages 6a, b and the imaging element 4 be moved in an optical axis direction L so as to be able to adjust the image characteristic pertaining to a hologram image of a subject 1 to a desired value, on the basis of at least one of information pieces: each focal distance regarding optical portions 2, 3 constituting a hologram optical system 30; a disposed position interval zl of the lens systems 2, 3; the wavelength of incoherent light; the wavelength width of incoherent light; and an interval zs between the subject 1 and the reference position of a hologram imaging device 50.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an incoherent hologram imaging device that splits incoherent light from a subject into two paths and then combines the paths to image a hologram. [Background technology]

[0002] Incoherent holography (also known as incoherent digital holography) is a promising passive stereoscopic imaging technique because it can record holograms of objects using low-coherence light sources such as sunlight, LEDs, and fluorescent light.

[0003] In incoherent holography, when imaging, incoherent light propagating from a subject is split into two beams within a hologram imaging device (hereinafter, these two beams are referred to as the first split beam and the second split beam). Different phase distributions are imparted to the first split beam and the second split beam using a concave mirror, a plane mirror, a lens, a spatial light modulator, or the like. As a phase distribution, a spherical phase or a planar phase (which corresponds to a spherical phase with an infinite radius of curvature, and is therefore broadly included in the spherical phase) that is equivalent to a lens, is often used.

[0004] Then, a wavelength filter is used to expand the coherence length, and these two light waves are made to interfere on the image sensor surface, forming a hologram that reflects the three-dimensional information of the subject. A propagation calculation is applied to this hologram to reconstruct the three-dimensional information of the subject, thereby obtaining a reconstructed image of the subject (see, for example, Patent Document 1 and Non-Patent Document 1).

[0005] Incidentally, the optical systems used in incoherent holography to date have been constructed based on the configuration of the optical system of a normal camera. In other words, in the optical system of a normal camera, the positional relationship between the subject and the camera lens is the most important parameter that determines the resolution, and the configuration of the lens system inside the camera can be designed so that an image is formed within the focal depth based on geometric optics, and the effects of the wavelength width of the light source can be ignored.

[0006] Conventional hologram imaging devices based on incoherent holography also incorporate the optical system configuration of a normal camera described above, and therefore the optical system is constructed with three fixed elements: (1) the focal length of each lens system, (2) the light propagation distance between each optical element, and (3) the wavelength width of the light source.

[0007] However, in the case of incoherent holography, the principle of forming an image is different from the "imaging" phenomenon used by normal cameras, and instead utilizes the principle of "self-interference", so the above factors (1), (2), and (3) have a significant effect on the resolution and multiplication factor. In particular, it is difficult to consider this in terms of geometric optics, as is the case with normal cameras, and it is necessary to find conditions that can improve the resolution based on the theory of wave optics. As a result of investigation based on the theory of wave optics, in incoherent holography, the resolution and image magnification of the reconstructed image change in a complex manner depending on (1) the focal length of the lens system constituting the holographic imaging device, (2) the light propagation distance between those elements, (3) the wavelength width of the light source, and further, (4) the distance between the subject and the holographic imaging device. This has already been disclosed by the inventors of the present application (see Patent Document 2 and Non-Patent Document 2). Note that the term "focal length" used in the incoherent holographic imaging device here is the reciprocal of the refractive power of the lens, and does not mean the distance between the imaging element and the lens when the subject is focused, as in the case of a normal camera lens. That is, the resolution and image magnification of a stereoscopic image also change significantly depending on (4) the distance (depth distance) between the subject and the holographic imaging device.

[0008] In particular, when it comes to resolution, it is desirable for it to be as high as possible when capturing a three-dimensional image of a subject with higher resolution; however, to maximize the resolution of incoherent holography, (4) the distance between the subject and the hologram imaging device is limited to a single predetermined value.

[0009] Specifically, assuming that the light source has a single wavelength, the best resolution is achieved when the diameters of the first and second divided beams are equal on the image sensor surface. This resolution is known to be higher than that of conventional cameras and imaging technologies. However, if the depth position of the subject deviates from this position, the diameters of the spots of the first and second divided lights will not match, resulting in a deterioration of the resolution, which may result in a situation where the resolution is inferior to that of a normal camera.

[0010] The reason why the problems described above have not been critical drawbacks for conventional incoherent holography technology is that most conventional incoherent holography technology was designed for use with fluorescence microscopes. In a microscope system, the depth position of the subject does not fluctuate significantly, so the degradation of resolution caused by such fluctuations is minor. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] Special Publication No. 2016-533542 [Patent Document 2] Patent Application No. 2018-192541 [Non-patent literature]

[0012] [Non-Patent Document 1] MK Kim, “Full color natural light holographic camera”, Optics Express, (2013), vol. 21, pp. 9636-9642. [Non-Patent Document 2] T. Nobukawa, Y. Katano, T. Muroi, N. Kinoshita, and N. Ishii, “Sampling requirements and adaptive spatial averaging for incoherent digital holography,” Optics Express, (2019), vol. 27, pp. 33634-33651. Summary of the Invention [Problem to be solved by the invention]

[0013] However, in recent years, the need for incoherent holography has arisen in the form of capturing images of people, animals, etc., to obtain moving images, as with a normal video camera. Taking into consideration this situation, the depth position of the subject varies greatly, unlike in the case of a microscope, and the degradation of resolution becomes evident. Furthermore, even when capturing still images, it is entirely conceivable that, depending on the imaging conditions, it may be difficult to adjust the distance from the subject, and so there has been a demand for incoherent holography technology that can be applied in such cases.

[0014] Furthermore, in the conventional incoherent holography technology, the resolution is discussed assuming that the wavelength of the light source is a single wavelength, but in reality, the wavelength of the light source is not a single wavelength, but generally has a width of several nm to several tens of nm. When the wavelength of the light source is not a single wavelength, the surface at which the resolution is best varies in a more complicated manner (see Patent Document 2 and Non-Patent Document 2, disclosed by the present inventor). If a wavelength filter with a very narrow wavelength width is used, the wavelength of the light source can be regarded as a single wavelength, but this is not a practical solution because the loss of light energy becomes large, the signal-to-noise ratio decreases, and as a result, the image quality is significantly reduced. Therefore, it is essential to consider the change in the resolution of incoherent holography taking into account the influence of the wavelength width and the influence of the subject position. In addition to the resolution, the image magnification of the stereoscopic image also changes depending on the depth position of the subject. If the photographer could easily adjust the resolution and image magnification according to the imaging situation and the subject without moving the holographic imaging device itself, it would be possible to easily capture higher-definition stereoscopic images flexibly and according to the photographer's intention. However, there is no holographic imaging device that allows adjustment of the resolution and image magnification of the stereoscopic image.

[0015] In this way, in a conventional holographic imaging device based on incoherent holography, the resolution of a three-dimensional image decreases when the depth position of the subject changes. Also, in a conventional holographic imaging device based on incoherent holography, the imaging device is designed to maximize the resolution assuming that the light source has a single wavelength, but the wider the wavelength width, the more complex the conditions that can maximize the resolution change. Furthermore, in conventional holographic imaging devices, the configuration of the imaging device was fixed, and it was not possible to adjust the resolution and image magnification according to the subject position. In addition, there were no design guidelines for imaging devices that could improve resolution when the subject position changed.

[0016] The present invention aims to provide an incoherent hologram imaging device that can adjust the resolution and image magnification of a three-dimensional image to desired values ​​even if the distance between the subject and the reference position of the incoherent hologram imaging device changes. [Means for solving the problem]

[0017] The present invention Noi The incoherent holographic imaging device An incoherent hologram imaging device includes a hologram optical system that divides incoherent light from a subject into a first divided light and a second divided light, imparts different spherical phases to the complex amplitude distributions of the two divided lights, and forms a hologram by causing the two divided lights to interfere with each other, and an imaging element that images the formed hologram, a moving stage capable of relatively moving at least one lens portion in the hologram optical system and the imaging element; a focal length for each lens portion constituting the holographic optical system, a distance between the lens portions, a wavelength of the incoherent light, a wavelength width of the incoherent light, and a distance between the subject and a reference position of the incoherent holographic imaging device. all The present invention is characterized in that it is equipped with an image characteristic control unit that instructs the movement of the moving stage so that the at least one lens portion and the imaging element move relatively in the optical axis direction so that the image characteristics related to the hologram image of the subject can be adjusted to a desired value based on the information. It should be noted that the "reference position" mentioned above and below can be any position, such as the position where the optical axis of the lens surface disposed closest to the subject intersects with the reference position.

[0019] Above E In a programmable imaging device, the "spherical phase" also includes a "planar phase" in which the radius of the sphere becomes infinite.

[0020] Ma As mentioned above Ta Incoherent hologram imaging device Place It is preferable that the incoherent hologram imaging device further comprises a calculation means for calculating the distance between the subject and a reference position of the incoherent hologram imaging device, based on a hologram imaged by the incoherent hologram imaging device or a complex amplitude distribution extracted from the hologram, and is configured so that, based on the information on the distance obtained by the calculation means, it is possible to image an subject set at a position corresponding to the distance with a desired resolution and image magnification.

[0021] In addition, Ta Incoherent hologram imaging device Placea signal processing means for varying a propagation distance or a reconstruction distance stepwise and continuously or based on an optimization algorithm for searching for an extreme value, based on a hologram captured by a hologram capture device or a complex amplitude distribution extracted from the hologram, evaluating the sharpness of the reconstructed image each time the propagation distance or the reconstruction distance is varied, and acquiring information relating to the distance between the subject and a reference position of the incoherent hologram imaging device according to the propagation distance at which the sharpness is highest, It is preferable that the information relating to the interval obtained by the signal processing means is converted into information relating to the interval in real space, and then, based on this converted value, an image of a subject set at the interval can be captured with a desired resolution and image magnification. Effect of the Invention

[0022] The incoherent hologram imaging device of the present invention is equipped with an image characteristic control unit that can adjust the image characteristics related to the hologram image of the subject to desired values ​​based on at least one of the following pieces of information: the focal length of each lens portion that constitutes the hologram optical system, the positional spacing between the lens portions, the wavelength of the incoherent light, the wavelength width of the incoherent light, and the spacing between the subject and a reference position of the incoherent hologram imaging device.

[0023] This image characteristic control section ,above Based on the at least one piece of information, the moving stage is instructed to move so that at least one lens portion and the imaging element move relatively in the optical axis direction so that image characteristics related to the hologram image of the subject can be adjusted to a desired value. be .

[0024] the above In the image characteristic control section teethSince (1) the focal length of each lens portion constituting the holographic optical system and (2) the distance between the lens portions can be controlled to desired values, the resolution can be improved regardless of the distance between the subject and the reference position of the holographic imaging device, and a high-definition 3D image can be captured. Furthermore, since the resolution and image magnification can be changed to desired values, the appearance of the 3D image can be changed to desired values, and the range of image expression can be expanded. [Brief description of the drawings]

[0025] [Figure 1] 1 is a schematic diagram illustrating an optical system of a holographic imaging device according to a first embodiment of the present invention. [Diagram 2] 10 is a schematic diagram showing an optical system of a holographic imaging device according to a second embodiment of the present invention. FIG. [Diagram 3] 3 is a schematic diagram showing a configuration of a light propagation distance scanning module of the holographic imaging device according to the second embodiment shown in FIG. 2. FIG. [Figure 4] FIG. 11 is a schematic diagram illustrating an optical system of a holographic imaging device according to a third embodiment of the present invention. [Diagram 5] FIG. 11 is a schematic diagram showing the configuration of a holographic imaging device according to a fourth embodiment of the present invention, which is equipped with a common-path interferometer type optical system and which incorporates the configuration of the main parts according to each embodiment of the present invention. [Figure 6] 6 is a schematic diagram showing the configuration of a holographic imaging device according to Modification 1, which is a partial modification of the embodiment shown in FIG. 5 (using a beam splitter and a reflective spatial light modulator). FIG. [Figure 7] 6 is a schematic diagram showing a configuration of a holographic imaging device according to Modification 2, which is a partial modification of the embodiment shown in FIG. 5 (a reflective optical modulator is used in the optical propagation distance scanning module). FIG. [Figure 8] 6 is a schematic diagram showing the configuration of a holographic imaging device according to Modification 3, which is a partial modification of the embodiment shown in FIG. 5 (the optical system has a Michelson interferometer type configuration). FIG. [Figure 9] 11 is a graph showing a change in the evaluation value of the evaluation index F with respect to the propagation distance zr in the embodiment. [Figure 10] 11 is a graph showing (a) a change in image magnification with respect to the depth position zs of a subject, and (b) a change in resolution with respect to the depth position zs of a subject, in an embodiment. [Figure 11] 1 shows a test chart (test target) used in the hologram imaging device according to the embodiment. [Figure 12] 1 is a reconstructed image obtained by imaging a test chart (test target) used in the holographic imaging device according to the embodiment under conditions that provide the highest resolution. [Figure 13] This is a reconstructed image obtained when the subject is shifted in the depth direction from the reconstructed image (state of FIG. 12) captured under conditions that provide the highest resolution for a test chart (test target) used in the hologram imaging device of the embodiment. [Figure 14] 13 is a graph showing the change in resolution with respect to the arrangement distance zh of the imaging element under the imaging conditions under which the reconstructed image in FIG. 12 was obtained. [Figure 15] This is a reconstructed image captured by changing the parameters of the hologram imaging device so as to improve the resolution compared to the reconstructed image in FIG. [Figure 16] 14 is a graph showing the change in the evaluation value of the evaluation index F with respect to the propagation distance zr (the blur amount of the reconstructed image) under the imaging conditions under which the reconstructed image of FIG. 13 was obtained. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, an incoherent holographic imaging device according to an embodiment of the present invention will be described with reference to the drawings. First, the configuration of the holographic imaging device according to the first to third embodiments will be described with reference to FIGS.

[0027] (Embodiment 1) A holographic imaging device 50 (holographic optical system 30) according to the first embodiment converts incoherent light from a subject 1 into parallel light in a lens system 2, and splits the light into a first divided light 15 and a second divided light 16 in a bifocal lens system 3. These two divided lights 15 and 16 are irradiated onto an imaging element 4 via a wavelength filter 5. The two divided lights interfere with each other on the imaging element 4, and the interference fringe image is captured as a hologram.

[0028] The two split lights 15, 16 split by the bifocal lens system 3 are configured to have different spherical phases in their complex amplitude distributions, and movable stages 6a, b are provided for moving the bifocal lens system 3 and the image sensor 4 (wavelength filter 5 is arranged on the subject side: the same applies below) along the optical axis L so that the distance between them can be changed by moving each of them.

[0029] Furthermore, the focal lengths of the desired optical members (lens system 2, bifocal lens system 3) constituting the hologram optical system 30 and the distance between the positions of the optical members (lens system 2, bifocal lens system 3, and image sensor 4) (here, the distance z between the lens system 2 and the bifocal lens system 3) are calculated. l , and the distance z between the bifocal lens system 3 and the image sensor 4 h ), the wavelength λ of the incoherent light, the wavelength width Δλ of the incoherent light, and the distance between the subject 1 and the reference position (here, the lens system 2) of the hologram imaging device 50 (hereinafter referred to as the depth position) z s Based on the above information and the above information, an image characteristics control unit 8 is provided which instructs the moving stages 6a, b to move optical members (here, the bifocal lens system 3 and the image sensor 4) placed on the moving stages 6a, b in the direction of the optical axis L so that the image characteristics related to the hologram image of the subject 1 can be adjusted to desired values. The image characteristics control unit 8 is disposed within the signal processing unit 7 which performs various signal processing.

[0030] The above-mentioned moving stages 6a and 6b will be further described with reference to Fig. 1. As described above, in the holographic imaging device 50 according to the first embodiment, the bifocal lens system 3 and the imaging element 4 are configured to be movable along the optical axis L, and thus the relative distance z between the bifocal lens system 3 and the imaging element 4 can be adjusted. h can be set to a desired value.

[0031] The moving stages 6a, b are provided for each of the members 3, 4 to be moved, and are connected to a signal processing unit 7. Movement information calculated by an image characteristics control unit 8 disposed in the signal processing unit 7 is input to each of the moving stages 6a, b, and based on this movement information, each of the members 3, 4 is moved to a desired position on the optical axis L. In addition, current position information of the moving stages 6a, b and movement distance information of the moving stages 6a, b are fed back to the image characteristics control unit 8 in the signal processing unit 7 from each of the moving stages 6a, b.

[0032] As a result, the focal lengths of all the lens systems (lens system 2, bifocal lens system 3) constituting the holographic imaging device 50 and the distance between the positions of the optical members (lens system 2, bifocal lens system 3, and imaging element 4) (here, the distance z between the lens system 2 and the bifocal lens system 3) are l , and the distance z between the bifocal lens system 3 and the image sensor 4 h ), the wavelength λ of the light source, its wavelength width Δλ, and the depth position z r A predetermined calculation is performed based on each piece of information, and the moving stages 6a and 6b are moved to the desired positions, thereby moving the members 3 and 4.

[0033] Such adjustments are possible because the image characteristic control unit 8 in the signal processing unit 7 determines the distance between the subject and the imaging device as unknown and the depth position z rIf there is a depth position information, that information is input from an external input unit (such as a keyboard) to a signal processing unit 7 (image characteristics control unit 8), and information calculated by the image characteristics control unit 8 is sent to each moving stage 6a, b connected to the signal processing unit 7, thereby adjusting the position of each optical component of the hologram optical system 30 and achieving the desired resolution and image magnification.

[0034] In addition, when it is desired to change the resolution or image magnification by focusing on a specific object whose depth position information is unknown, a distance measuring technology (hereinafter referred to as an autofocus-like technology) having a function equivalent to the autofocus technology of a camera is introduced to measure the depth position z r Measure. In the holographic imaging device according to the first embodiment, the greater the physical movement distance, the greater the degree of freedom in controlling the resolution or image magnification.

[0035] (Embodiment 2) As shown in FIG. 2, the hologram imaging device 150 (hologram optical system 130) of this embodiment 2 also converts incoherent light from the subject 101 into parallel light in the lens system 102, and splits it into a first divided light 115 and a second divided light 116 in the bifocal lens system 103. These two divided lights 115, 116 are irradiated onto the imaging element 104 via the wavelength filter 105 to image a hologram of the subject 101.

[0036] This embodiment differs from the above-mentioned first embodiment in that an optical propagation distance scanning module 109 is introduced on the optical path. Here, the configuration of the light propagation distance scanning module 109 will be described with reference to Fig. 3. This module 109 is configured by disposing a variable-focus lens system 113 consisting of one curvature modulation element between two sets of lens systems 112, 114. The two sets of lens systems 112, 114 are arranged so as to form a 4f optical system, and the variable-focus lens system 113 is disposed on the spatial frequency plane thereof. By changing the focal length of the variable-focus lens system 113 within this module 109, it is possible to generate light equivalent to light that has propagated an arbitrary distance.

[0037] Therefore, this module 109 has the same function as the above-mentioned embodiment 1 in which the optical components are physically moved, but instead of physically moving the optical components as in the above-mentioned embodiment 1, the focal length of the variable-focus lens system 113 in the light propagation distance scanning module 109 is electrically controlled to modulate the propagation distance of light between the optical components.

[0038] Furthermore, the variable-focus lens system 113 is connected to an image characteristics control unit 108 in the signal processing unit 107, and is configured so that the curvature of the variable-focus lens system 113 is instructed to change and information on the focal length of the variable-focus lens system 113 is fed back to the image characteristics control unit 108. The method of controlling the image characteristics in the image characteristics control unit 108 in the signal processing unit 107 is basically the same as that in the first embodiment.

[0039] (Embodiment 3) As shown in FIG. 4, the hologram imaging device 250 (hologram optical system 230) of this embodiment 3 also converts incoherent light from the subject 201 into parallel light in a variable-focus lens system 210, and splits it into a first divided light 215 and a second divided light 216 in a bifocal variable-focus lens system 211. These two divided lights 215, 216 are irradiated onto an imaging element 204 via a wavelength filter 205 to image a hologram of the subject 201.

[0040] The difference between this embodiment and the above-mentioned embodiment 2 is that, instead of the light propagation distance scanning module 109, a variable focus lens system 210 and a bifocal variable focus lens system 211, in which both the lens system and the bifocal lens system are variable types, are arranged on the optical path. It is also possible for only one of the lens system and the bifocal lens system to be of the variable type.

[0041] In this embodiment, by providing a variable focus lens system 210 and a dual focus variable focus lens system 211, it has the same functionality as the optical propagation distance scanning module 109 in the above-mentioned embodiment 2, but has a simpler configuration than the above-mentioned embodiment 2. The method of controlling the image characteristics in the image characteristics control section 208 in the signal processing section 207 is basically the same as that in the first and second embodiments.

[0042] (Autofocus-like technology) As mentioned above, when the depth position information of the subjects 1, 101, and 201 is unknown, if it is desired to change the resolution or image magnification, an autofocus-like technique is introduced to change the depth position z s It is also useful to ask for When using an autofocus-like technology, no additional distance measurement technology is required, and the depth position z of subjects 1, 101, and 201 can be determined. s can be automatically measured, and the measured depth position z s Based on the value of , the parameters of the holographic imaging devices 50, 150, 250 are modified.

[0043] The autofocus-like technology described here does not involve measuring by changing the focal length of the holographic optical system 30, 130, 230, but is a procedure performed using numerical calculations within the signal processing unit 7, 107, 207, and is specifically as follows.

[0044] That is, the propagation distance or reconstruction distance is changed stepwise and continuously for the captured hologram or the complex amplitude distribution extracted from the hologram to obtain a reconstructed image, and the sharpness of the reconstructed image is evaluated each time to find the propagation distance that maximizes the sharpness. When the propagation distance is converted into a depth distance in real space, the value corresponds to the depth position between the subjects 1, 101, and 201 and the image sensors 4, 104, and 204, so the depth position z of the subjects 1, 101, and 201 can be calculated using this value. sIn this embodiment, such a distance measurement technique is referred to as an autofocus-like technique in the sense that the signal processing unit 7, 107, or 207 searches for a condition under which the sharpness of the reconstructed image of the subject is maximized.

[0045] (Combination of embodiments) Although the holographic imaging devices 50, 150, and 250 shown in the first to third embodiments described above have different parameters, when improving the resolution, the depth position z s By correcting the deviation in the center position and diameter of the light beam of the first divided light 15, 115, 215 and the second divided light 16, 116, 216 caused by the change in the optical axis, parameters such as the intervals between the components in the holographic imaging device 250 are changed so that the cross sections of these two light beams match. It is also possible to combine the holographic imaging devices 50, 150, 250 shown in the first to third embodiments. By combining them, the degree of freedom of adjustable parameters in the image pickup elements 4, 104, 204 increases, and the degree of freedom of adjustment methods for the resolution and image magnification increases significantly. Therefore, hereinafter, a specific operation will be described using the fourth embodiment shown in FIG. 5, which combines the hologram imaging devices 50, 150, and 250 shown in the first to third embodiments.

[0046] (Embodiment 4) Holographic imaging device 450 (holographic optical system 430) according to this embodiment is configured as a holographic imaging device including all of the characteristic functions of the first to third embodiments, as described above. Incoherent light from a subject 401 is modulated by a liquid crystal lens (variable-focus lens system) 412a. Next, the phase of the incoherent light is modulated by a bifocal variable-focus lens system 411 consisting of a polarizer 413a, a transmissive spatial light modulator 414, and a polarizer 413b. Linearly polarized light is obtained by the polarizer 413a on the light incident side, and the linearly polarized light is modulated by the transmissive spatial light modulator 414.

[0047] A liquid crystal type is used as the transmissive spatial light modulator 414, but other spatial light modulators can also be used. Due to the elliptical shape of the liquid crystal molecules, when modulating the phase with the transmissive spatial light modulator 414, polarization selectivity is obtained, and the phase distribution of only one of two orthogonal linearly polarized lights can be modulated. The angle of the transmission axis of the polarizer 413a on the light incident side is arranged to be inclined at 45 degrees with respect to the long axis or short axis of the liquid crystal molecules, so that only the phase of the horizontal or vertical linearly polarized component of the linearly polarized component is modulated. As a result, the incoherent light is split into a first split light 415 and a second split light 416.

[0048] Then, the first and second split lights are transmitted through the polarizer 413b on the light exit side to change the polarization states of the first and second split lights to linearly polarized lights in the same direction, so that the two split lights can interfere with each other on the imaging surface of the imaging element 404. Note that a birefringent lens or a metalens may be used instead of the transmissive spatial light modulator 414. In short, it is essential to use a bifocal lens that can change two orthogonal polarization states to different polarization states and can make the focal lengths of the two lights different from each other.

[0049] After that, the phase is modulated by the light propagation distance scanning module 409 consisting of two lens systems 417a and 417b and a liquid crystal lens 412b. The two lens systems 417a and 417b are arranged in a 4f optical system (see FIG. 3), and the liquid crystal lens 412b is disposed on the spatial frequency plane (the position where the rear focal plane of the lens system 417a and the front focal plane of the lens system 417b overlap). Note that it is also possible to use a variable focal lens such as a moire lens, an Alvarez lens, or a liquid lens instead of the liquid crystal lens 412b.

[0050] Finally, the two split incoherent lights are made incident on a wavelength filter 405 to improve temporal coherence, and then irradiated onto the imaging surface of the imaging element 404 so that the light spots overlap well with each other, forming a hologram. The imaging element 404 captures the hologram as a digital image and transfers it to the signal processing unit 407. In addition, in order to obtain a high-quality three-dimensional image, a plurality of holograms having different phase shift amounts may be captured by the transmissive spatial light modulator 414 as necessary. In addition, as shown in Japanese Patent No. 6245551 and Japanese Patent Publication No. 2019-144520, a method of capturing a plurality of holograms in one capture may be introduced.

[0051] In the signal processing unit 407, the complex amplitude distribution obtained by applying the calculation of the phase shift method or the off-axis method to the captured hologram or hologram group is denoted as O(x, y). By applying the propagation calculation of the following equation (1), r We can reconstruct the image of the object at z r When the condition defined in the following formula (10) is satisfied, the depth position z s However, conceptually, it can be called "a virtual distance from the image pickup device surface at the time of image reconstruction to the reconstructed image surface."

number

[0052] Any distance z r Regarding the setting of z, the photographer (operator) may input an arbitrary value, or the depth position z s The propagation calculation itself is a calculation method that is generally used in holographic imaging devices for incoherent holography. The phase shift method and off-axis method are also calculation methods for removing noise from the conjugate image, direct image, and imaging element before reconstructing the image, and are common techniques in the field of holography.

[0053]

number

[0054]

number

[0055]

number

[0056]

number

[0057] When the effect of the wavelength width of light on the diameter of the hologram cannot be ignored, the diameter of the hologram is defined by the following formula (6) instead of the above formula (4).

number

[0058] In this way, D c is the diameter of the coherence region determined by the wavelength width, and is a parameter that has not been taken into consideration in conventional incoherent holography imaging devices, and is given, for example, by the following equation (7).

number

[0059] The above formula (7) is an example of a formula for taking the wavelength width into consideration, and the form of the formula changes depending on how precisely the optical system is modeled. In other words, the form of the formula is not important, but it is important to consider the relationship that an increase in the wavelength width reduces the resolution, and conversely, a decrease in the wavelength width improves the resolution. All of the parameters described above and all of the parameters shown below are stored in the memory of the signal processing unit 407.

[0060] In the hologram optical system 430 shown in FIG. 5, a transmission type spatial light modulator 414 and an image sensor 404 are disposed on a scanning stage. l and z h In addition, by using the liquid crystal lens 412a closest to the subject 401 and the transmissive spatial light modulator 414, the focal length f o , the focal length f of the first split light 415 of the bifocal variable-focus lens system 411 d1 , and the focal length f of the second split light 416 of the bifocal variable-focus lens system 411. d2 can each be electrically changed.

[0061] In addition, the optical propagation distance scanning module 409, which is composed of two lens systems 417a and 417b and a liquid crystal lens 412b, detects z h Specifically, in the optical propagation distance scanning module 409, the focal length f a Lens system 112 - focal length f v Variable focus lens system with focal length f113 b Since the lens system 114 is a 4f optical system, based on the following formula (8), v By changing z h It is possible to change the

[0062]

number

[0063] Here, the actual distance from the transmissive spatial light modulator 414 to the image sensor 404 is z h1 +z h2 +2f a +2f b However, by changing the focal length of the liquid crystal lens 412b in the light propagation distance scanning module 409 based on the above formula (8), the "light propagation distance" from the transmissive spatial light modulator 414 to the image sensor 404 can be changed. In this case, the image magnification is M T f b / fa It becomes. l In order to change the above, the light propagation distance scanning module 409 may be disposed between the main plane of the liquid crystal lens 412 a and the main plane of the transmissive spatial light modulator 414 .

[0064] The advantage of using the optical propagation distance scanning module 409 is that, as previously mentioned, no mechanical drives are required for the motion stages 406a, 406b, etc. As is clear from the above formulas (2) and (3), the resolution and image magnification depend on the depth position z s On the other hand, it also changes depending on the focal length of each lens system constituting the holographic imaging device 450 and the light propagation distance between each optical member. Therefore, by changing these elements based on the above formulas (2) and (3), it is possible to change the resolution and image magnification to a desired value according to the photographer's intention. In addition, s Even if the resolution deteriorates due to misalignment, this can be easily improved by using holographic imaging device 450 of this embodiment.

[0065] <Changes> In the following, modified versions of this embodiment will be described. For components having similar functions to those in embodiment 4, the reference numbers used for the components in embodiment 4 will be increased by 100 for modified version 1, 200 for modified version 2, and 300 for modified version 3, and redundant descriptions will be omitted. (Modification 1) In the holographic imaging device 450 shown in Fig. 5, the transmission type spatial light modulator 414 is used in the bifocal variable-focus lens system 411, but as shown in Fig. 6, in the holographic imaging device 550 of this embodiment, a reflection type spatial light modulator 519 can also be used as the bifocal variable-focus lens system 511. However, when the reflection type spatial light modulator 519 is used, it is necessary to insert a beam splitter 518 between the liquid crystal lens 512a and the reflection type spatial light modulator 519. Note that a polarizing beam splitter may be disposed instead of the beam splitter 518 and the two polarizers 513a and 513b.

[0066] (Modification 2) 6, a transmission type liquid crystal lens 512b constituting the light propagation distance scanning module 509 is used, whereas a reflection type reflective spatial light modulator (liquid crystal lens) 619 constituting the light propagation distance scanning module 609 is used in the holographic imaging device 550 shown in FIG. 7. The two split lights 615 and 616 output from the reflection type spatial light modulator 619 are reflected by a concave mirror 620 and emitted in the direction of the imaging element 604.

[0067] (Modification 3) Holographic imaging devices 450, 550, 650 shown in Fig. 5 to 7 are all of a common-path interferometer type in which first divided light 415, 515, 615 and second divided light 416, 516, 616 propagate coaxially, but may be of an equal-path-length Michelson interferometer type as shown in Fig. 8. Note that other equal-path-length types such as a Mach-Zehnder interferometer type and even a detour-type Fizeau interferometer type may also be applied. In a holographic imaging device 750 shown in FIG. 8, two liquid crystal lenses 712a and 712b with different focal lengths are used instead of a bifocal lens. Furthermore, although plane mirrors 721a and 721b that reflect both light beams and have a focal length equivalent to infinity are used, concave mirrors may be used instead.

[0068] (Other changes) Various types of imaging elements 4, etc., may be used in each of the above-mentioned holographic imaging devices 50, etc. For example, monochrome elements, elements capable of acquiring color information such as a Bayer array, and even elements equipped with a polarizer array may be used. Furthermore, the holographic imaging devices 450, 550, 650, and 750 according to the embodiment 4 and modified forms 1 to 3 described above are configured to change three parameters of the holographic imaging devices 50, 150, and 250, while the holographic imaging devices 50, 150, and 250 according to the above-described embodiments 1 to 3 are configured to change only one parameter as shown in Figures 1, 2, and 4, but may also be configured to change two of the above parameters.

[0069] (Method of calculating depth value) Depth position z of subject 1, etc. s When the depth position z of the object 1 is not quantitatively known, the autofocus technique described above can be used to estimate the depth position z of the object 1. s Alternatively, the distance may be measured using the following distance measurement technique.

[0070] First, a hologram is captured by any of the above-mentioned hologram imaging devices 50, etc. The captured hologram data is analyzed in the signal processing unit 7, etc., and the placement parameters of the hologram imaging devices 50, etc. and the parameters of each optical member are stored in the signal processing unit 7, etc. A complex amplitude distribution is extracted from the captured hologram data in the signal processing unit 7, etc., and a three-dimensional image of the subject 1, etc. is reconstructed by light backpropagation calculation based on the parameters stored in the signal processing unit 7, etc. At this time, as shown in FIG. 9, the propagation distance z r The amount of blur in the reconstructed image changes depending on the setting of . s You can find out.

[0071] The propagation distance z when the blur is smallest r is the depth position z of the object 1, etc., as described later. s The propagation distance z with small blurring r In order to obtain this, for example, the index shown in the following formula (9) is used to evaluate the amount of blur of each reconstructed image.

number

[0072] The higher the evaluation value of this evaluation index of sharpness, the smaller the amount of blur. Therefore, the propagation distance z r (In Fig. 9, z r =z 3 ) is the depth position z of the subject s The above formula (9) is an example, and other evaluation indices of sharpness based on the first or second derivative can also be used. This evaluation is carried out for all propagation distances z r If you want to improve the calculation speed and efficiency of this evaluation process, you can apply algorithms such as the golden section method, gradient method, Newton method, or Runge-Kutta method to find the propagation distance z with the largest evaluation value. r Find the conditions.

[0073] In incoherent digital holography, the propagation distance z r and the depth position of the object z s do not necessarily coincide with each other, and the two are related by the following equation (10).

number

[0074] From the above equation (10), the depth position of the subject z s This depth position z s By referring to the information in and substituting it into the above formulas (2) and (3), it is possible to derive the focal length of the lens system in the holographic imaging device 50 or the like and the propagation distance of light between each optical member in order to realize the target resolution or image magnification. These derived conditions are fed back from the signal processing unit 7 or the like to the holographic imaging device 50 or the like, and the focal length of the lens system and the propagation distance of light between each optical member are changed.

[0075] In the above description, the mathematical expressions related to the resolution and image magnification of a stereoscopic image are shown, and the process of adjusting these is described. However, since it is clear that the resolution and image magnification also affect the image quality, field of view, depth of field, and depth resolution, it is easy to derive the mathematical expressions related to the image characteristics such as the image quality, field of view, depth of field, and depth resolution. Therefore, in order to arbitrarily change the other image characteristics described above instead of the resolution and image magnification, it is also possible to use the technique of the incoherent hologram imaging device according to this embodiment. The incoherent holographic imaging device of the present invention will now be described in further detail with reference to examples in which specific values ​​are set. EXAMPLES

[0076] The simulation results obtained by capturing and reconstructing a hologram using a hologram imaging device with an imaging element mounted on a moving stage are shown below. In this simulation, the diffraction of light waves is calculated using the angular spectrum method, which makes it possible to accurately calculate the propagation of light and the behavior of light after passing through a lens. Specifically, as the holographic imaging device of this embodiment, the holographic imaging device 50 of FIG. 1 is used, and a moving stage 6a related to the bifocal lens system 3 is not provided, and only a moving stage 6b carrying an imaging element 4 (integrally provided with a wavelength filter 5 on the subject side) is provided.

[0077] The wavelength of the light from the light source irradiated on the subject 1 is 633 nm, and the wavelength width is 1 nm. The focal length of the lens system 2 is f o is 250 mm, the first focal length (first divided light 15) of the bifocal lens system 3 is f d1 is 300 mm, the second focal length (second split light 16) f d2 is ∞ mm (corresponding to the distribution on a flat surface with no curvature), and the distance z between lens system 2 and bifocal lens system 3 l is 100 mm, and the distance z between the bifocal lens system 3 and the image sensor 4 hThe distance was set to 600 mm. The imaging element 4 used had a pixel count of 2048×2048, a pixel pitch of 6.5 μm, and a gradation level of 16 bits.

[0078] In the embodiment of the hologram imaging device 50 configured as above, the subject 1 is disposed at a position (depth position) z s The changes in image magnification and resolution with respect to the image magnification and resolution calculated based on the above formulas (2) and (3) are plotted in graphs shown in FIG. 10(a) and FIG. 10(b), respectively. As is clear from FIG. 10(a) and FIG. 10(b), the subject 1 is placed at a position z s In particular, as shown in FIG. 10(b), the image magnification and resolution change depending on the position z s It can be seen that the resolution is highest when z is around 250 mm. s However, when imaging a subject 1 located more than 250 mm away in the depth direction, the resolution deteriorates. This deterioration has been difficult to suppress with conventional holographic imaging devices.

[0079] In the following, in the imaging device according to the embodiment of the present invention, the subject 1 is disposed at a position z s This shows that the resolution can be improved even if the z s FIG. 12 shows the result (reconstructed image) of capturing an image of the subject (test chart) 1 shown in FIG. 11 placed at z = 250 mm. As is clear from FIG. 10(b), s = 250 mm) is the imaging result under the conditions that give the highest resolution. Next, if the subject 1 moves 200 mm in the depth direction, i.e., z s = 450 mm, the result of imaging (reconstruction image) is shown in FIG. 13. In many cases, the depth position z s is unknown information. In that case, as described above, autofocus (including autofocus-like technology) is applied in the signal processing unit 7 to obtain the depth position z of the subject 1. s Get information about.

[0080] Next, an overall picture of the flow of improving the resolution in this embodiment will be described, assuming that the amount of movement of the subject 1 in the depth direction is known. Comparing the states of the reconstructed images in Fig. 12 and Fig. 13 described above, it is clear that the fine structure of the test chart in Fig. 13 is blurred and the resolution is degraded. This degradation is as expected from Fig. 10(b). As described above, this result corresponds to imaging with a conventional holographic imaging device, and can be seen as a comparative example. On the other hand, the imaging device 50 according to this embodiment is configured so that the position of the imaging element 4 can be changed, and therefore the resolution can be improved if the position can be appropriately set.

[0081] Position z of subject 1 s When imaging a test chart with a lens diameter of 450 mm, the lens diameter is set to the position z s and the distance z between the bifocal lens system 3 and the image sensor 4. h All information related to the other parameters of the imaging device 50, except for h The relationship between the resolution and the optical axis is evaluated. The relationship is plotted in a graph in Figure 14. As is clear from FIG. 14, unlike the results shown in FIG. 10(b), z h = 260 mm, that is, when the position z s The highest resolution is achieved when the is changed from 450 mm to 260 mm.

[0082] In fact, in the hologram imaging device 50 according to the embodiment of the present invention, the imaging element 4 mounted on the moving stage 6b is moved to the z h = 260 mm, and a hologram of subject 1 was captured and reconstructed, as shown in Fig. 15. Compared with the results in Fig. 13, it is clear that the resolution has been improved in Fig. 15, which shows the results of this embodiment.

[0083] From the above results, in the holographic imaging device 50 according to the present embodiment, the resolution iss In the above example, the arrangement of the image sensor 4 is adjusted to maximize the resolution, but the resolution can be reduced based on FIG. 14 to obtain an image with strong blur. Furthermore, although the above has been described with respect to the case where the objective is to improve resolution, parameters such as image magnification, depth of field, field of view, focal length of the optical elements constituting the imaging device, and the position of each element are functions of resolution and image magnification, and therefore adjustments can be made in a similar manner to the above even when the objective is to adjust each image characteristic that depends on these parameters.

[0084] Furthermore, while the above examples show an aspect in which a predetermined optical element of holographic imaging device 50 is mechanically moved to move the arrangement position of the predetermined optical element, if light propagation distance scanning module 109 is used as in holographic imaging device 150 of embodiment 2 of the present invention, it is possible to eliminate the influence of vibrations and the like only by electronic scanning without relying on mechanical drive, and to achieve the same effect as holographic imaging device 50. Also, as in holographic imaging device 250 of embodiment 3 of the present invention, an aspect in which variable-focus lenses 210, 211 are introduced makes it possible to arbitrarily control the resolution, image magnification, and the like, as in holographic imaging devices 50, 150, and to achieve the same effect.

[0085] Propagation distance z from the hologram surface r is changed stepwise and continuously, a reconstructed image is formed each time, and the index of equation (9) above is calculated for the reconstructed image, thereby obtaining a graph in FIG. 16, which plots the evaluation results of the amount of blur and sharpness of the image. In FIG. 16, the position where this evaluation result is maximum corresponds to the position where the subject is placed, because the amount of blur is small and the sharpness is high. rIn order to shorten the calculation time, it is preferable to use an optimization algorithm based on the golden section method, gradient method, or Newton's method, as described above, to specify the propagation distance at which the sharpness is maximized with fewer propagation calculations. The above operations allow the z value at which the sharpness is maximized to be determined. r By identifying the distance between the object and the image sensor and performing the conversion based on the above equation (10), s is obtained. [Explanation of symbols]

[0086] 1, 101, 201, 401, 501, 601, 701 Subject 2, 102, 112, 114, 417a, b, 517a, b, 617, 717a, b Lens system 3. 103 Bifocal lens system 4, 104, 204, 404, 504, 604, 704 image sensor 5, 105, 205, 405, 505, 605, 705 Wavelength Filters 6a, b, 406a, b, 506a, b, 606a, b, 706a, b Moving stage 7, 107, 207, 407, 507, 607, 707 Signal processing section 8, 108, 208, 408, 508, 608, 708 Image characteristic control section 30, 130, 230, 430, 530, 630, 730 Holographic optical system 50, 150, 250, 450, 550, 650, 750 Holographic imaging device 109, 409, 509, 609, 709 Optical propagation distance scanning module 113, 210 Variable focus lens system 211, 411, 511, 611, 711 Bifocal variable focus lens system 412a, b, 512a, b, 612, 712a, b, c, d Liquid crystal lenses 413a, b, 513a, b, 613a, b polarizer 414, 614 Transmissive spatial light modulator 15, 115, 215, 415, 515, 615, 715 1st division light 16, 116, 216, 416, 516, 616, 716 2nd split light 518, 718 Beam splitter 519, 619 Reflective spatial light modulator 620 concave mirror 721a,b plane mirror L optical axis

Claims

1. An incoherent hologram imaging device includes a hologram optical system that divides incoherent light from a subject into a first divided light and a second divided light, imparts different spherical phases to complex amplitude distributions of the two divided light beams, and forms a hologram by causing the two divided light beams to interfere with each other, and an imaging element that images the formed hologram, a moving stage capable of relatively moving at least one lens portion in the hologram optical system and the imaging element; an image characteristic control unit that instructs movement of the moving stage so that at least one of the lens portions and the imaging element move relatively in the optical axis direction so as to adjust image characteristics related to a hologram image of a subject to desired values ​​based on all of the information regarding the focal length of each lens portion that constitutes the hologram optical system, the positional spacing between the lens portions, the wavelength of the incoherent light, the wavelength width of the incoherent light, and the spacing between the subject and a reference position of the incoherent hologram imaging device.

2. 13. An incoherent hologram imaging device comprising: a calculation means for calculating a distance between the object and a reference position of the incoherent hologram imaging device based on a hologram captured by the incoherent hologram imaging device according to claim 1 or a complex amplitude distribution extracted from the hologram, and configured to be able to image an object set at a position corresponding to the distance based on information relating to the distance obtained by the calculation means, with a desired resolution and image magnification.

3. a signal processing means for varying a propagation distance or a reconstruction distance stepwise and continuously or based on an optimization algorithm for searching for an extreme value, based on a hologram captured by the incoherent hologram imaging device according to claim 1 or a complex amplitude distribution extracted from the hologram, evaluating sharpness of the reconstructed image each time the propagation distance or the reconstruction distance is varied, and acquiring information relating to a distance between the subject and a reference position of the incoherent hologram imaging device according to the propagation distance at which the sharpness is highest, The incoherent hologram imaging device is configured to convert the information relating to the interval obtained by the signal processing means into information relating to the interval in real space, and then, based on this converted value, image the subject set at the interval with the desired resolution and image magnification.

Citation Information

Patent Citations

  • Non-coherent telescoping digital holographic imaging method and assorted device

    CN104793475A

  • Luminous flux splitting element

    CN105009002A

  • Incoherent fluorescence digital holographic microscopy using a transmission liquid crystal lens

    JP2016533542A

  • Autofocus system and method in digital holography

    JP2017538160A

  • Polishing device

    JP2018192541A