Digital hologram signal processing device and digital hologram imaging / reproducing device

The digital hologram signal processing device addresses blurred images in conventional holography by converting coherent responses to incoherent responses, enabling accurate depth perception and flexible blur control, thus improving image quality and compatibility with normal cameras.

JP7579196B2Active Publication Date: 2024-11-07NIPPON HOSO KYOKAI
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2021065331
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-04-07
Publication Date
2024-11-07
Estimated Expiration
2041-04-07

AI Technical Summary

Technical Problem

Conventional digital holography technologies using coherent and incoherent light sources face issues with blurred two-dimensional images due to coherent light responses, leading to inconsistent depth perception and inability to synthesize with normal camera images, and lack of control over image blur.

Method used

A digital hologram signal processing device that adjusts the blur amount by calculating focused and defocused images using coherent and incoherent light sources, converting coherent responses to incoherent responses through convolution integrals and point spread functions, allowing control over image blur.

Benefits of technology

Enables accurate depth perception and synthesis with normal camera images, providing a natural blur effect that can be arbitrarily controlled, enhancing image quality and flexibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579196000007
    Figure 0007579196000007
  • Figure 0007579196000008
    Figure 0007579196000008
  • Figure 0007579196000009
    Figure 0007579196000009
Patent Text Reader

Abstract

To correctly obtain a feeling of depth such as the depth position, depth order, etc., of a subject and to compose an image of the subject together with an image picked up with a normal camera.SOLUTION: A digital hologram signal processing device comprises: in-focus / depth position information acquisition means 18A which acquires an in-focus image of a subject and depth position information on the in-focus image from a recomposed image of a digital hologram; reference image acquisition means 18B which adds respective obtained in-focus images to acquire a reference image B through computation for specifying an in-region in-focus image set at a desired depth position in a depth-of-field region; defocus image group acquisition means 18C which acquires a defocus image group Dd through computation for specifying an out-region in-focus image arranged at the desired depth position outside the depth-of-field region; and reference image / defocus image group addition means 18D which adds the obtained reference image B and the obtained defocus image group Dd to each other to obtain an incoherent response image.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to a digital hologram signal processing device and a digital hologram imaging and reconstructing device, and more particularly to a digital hologram signal processing device and a digital hologram imaging and reconstructing device that are capable of creating a desired reconstructed image whether using coherent light or incoherent light. [Background technology]

[0002] Digital holography technology uses the interference of light to capture a hologram that reflects the three-dimensional information of a subject using optoelectronic devices such as image sensors and photodetectors, and stores this as a digital hologram in the memory of a signal processing unit. Displaying the digital hologram stored in this memory using a spatial light modulator makes it possible to reconstruct a three-dimensional image of the subject that can be observed with the naked eye. Furthermore, by performing diffraction propagation calculations on the digital hologram in the signal processing unit, a two-dimensional image at any depth can be reconstructed, and refocusing, which allows the focal position to be changed arbitrarily after the subject is captured, is possible (see Non-Patent Document 1 below).

[0003] Digital holography technology can be used to realize useful functions such as the above-mentioned three-dimensional imaging and display and refocusing, but most conventional digital holography technologies required the use of highly coherent laser light as a light source. However, advances in incoherent digital holography technology have alleviated the problem of spatial coherence required of light sources, making it possible to capture digital holograms using incoherent light such as sunlight, fluorescent light, and LEDs, and to realize the above-mentioned three-dimensional imaging and display and refocusing. This has steadily expanded the range of applications of digital holography (see Patent Documents 1 and 2 below).

[0004] Digital holography, which uses laser light (coherent light) as a light source (see Non-Patent Document 1 below), and incoherent digital holography, which uses incoherent light as a light source (see Patent Documents 1 and 2 below), differ in the coherence length of the light source used and the configuration of the interference optical system used to create digital holograms, but they share the same diffraction propagation calculations for reconstructing two-dimensional images; in this case, they commonly use the Fresnel diffraction integral and angular spectrum method (or plane wave expansion). These diffraction calculations implicitly assume that the light has large temporal and spatial coherence, and the blurring of the reconstructed image indicates the response of the coherent light. In incoherent digital holography, incoherent light is used during imaging, but the image reconstructed exhibits properties related to coherent light. This response of coherent light is essential for reconstructing three-dimensional information using the wavefront information of light, and does not pose a problem when reproducing three-dimensional images.

[0005] Therefore, the blurring of a 2D image reconstructed by digital holography is completely different from the blurring of an ordinary camera, such as a single-lens reflex camera or smartphone, which uses incoherent light. In other words, if a 2D image is output using the ordinary method described above using digital holography, a technology for capturing 3D images, users accustomed to using ordinary cameras will find the blurring of the 2D image in digital holography unnatural.

[0006] Furthermore, it is known that humans normally judge the depth of an object and the order of perspective in the depth direction based on the degree of blur (the degree of defocus) (see Non-Patent Documents 2 and 3 below), and conventional digital hologram imaging and playback devices are unable to provide an accurate sense of depth from a two-dimensional image. Furthermore, when video material is created by combining images captured by a digital hologram imaging and playback device with images captured by a regular camera, inconsistencies arise. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Special Publication No. 2016-533542 [Patent Document 2] Japanese Patent Application Publication No. 2020-060709 [Non-patent literature]

[0008] [Non-Patent Document 1] Georges Nehmetallah and Partha P. Banerjee, "Applications of digital and analog holography in three-dimensional imaging", Advances in Optics and Photonics, (2012), vol. 4, pp. 472-553. [Non-patent document 2] Alex Paul Pentland, "A new sense for depth of field," IEEE Transactions on pattern analysis and machine intelligence, (1987), vol. 4, pp. 523-531. [Non-patent document 3] George Mather and David RR Smith, “Blur discrimination and its relation to blur-mediated depth perception,” Perception, (2002), vol. 31, pp. 1211-1219. Summary of the Invention [Problem to be solved by the invention]

[0009] When reconstructing a two-dimensional image using diffraction propagation calculations in conventional digital holography, the blurred state of the light distribution on defocused planes that are away from the focus plane exhibits the response of coherent light, resulting in ringing and optical singularities, making it impossible to obtain the smooth blur that can be obtained when using a normal camera.

[0010] In this way, the sense of depth of the subject's depth position and depth order cannot be accurately obtained, and it cannot be combined with images captured by a normal camera. Furthermore, the amount of blur in a digital holographic image is determined by the distance between the image capture / reconstruction device and the subject.

[0011] The present invention has been made to solve these problems, and aims to provide a digital hologram signal processing device and a digital hologram imaging and reproducing device that can accurately obtain a sense of depth, such as the depth position and depth order of a subject, can be synthesized with an image captured by a normal camera, and can easily adjust the amount of image blur in digital holography. [Means for solving the problem]

[0012] The digital hologram signal processing device of the present invention comprises: A digital hologram signal processing device performs signal processing on a digital hologram imaging and reproducing device that captures three-dimensional information of a subject using interference of light, a focused image / depth position information acquisition means for acquiring a desired number of focused images of a subject and information on the depth positions of the focused images from the reconstructed image of a digital hologram obtained by the interference optical system of the digital hologram imaging and reconstructing device; a reference image acquisition means for setting elements of the depth of field, the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to the image plane in accordance with a desired depth position within the depth of field so as to specify an in-region in-focus image set at the desired depth position within the depth of field, and for acquiring a reference image by adding together a predetermined number of the obtained in-region in-focus images; a defocused image group acquiring means for setting each of the elements of the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane in accordance with a desired depth position so as to be able to specify an out-of-area in-focus image located at a desired depth position outside the depth of field, and for calculating a convolution integral of the intensity distribution of each of the obtained out-of-area in-focus images with the intensity distribution of a point spread function of an incoherent imaging system to acquire a defocused image group; a reference image / defocused image group adding means for adding the reference image obtained by the reference image obtaining means and the defocused image group obtained by the defocused image group obtaining means to each other to obtain an incoherent response image; The present invention is characterized by the following features.

[0013] Furthermore, it is preferable that the calculation process for acquiring the reference image in the reference image acquisition means is a calculation process related to the convolution integral of a combined intensity distribution obtained by adding together the intensity distributions of the in-region in-focus images obtained by changing the depth position, and the intensity distribution of the point spread function of the incoherent imaging system.

[0014] Furthermore, when performing the arithmetic processing related to the convolution integral of the combined intensity distribution and the intensity distribution of the point spread function of the incoherent imaging system, it is preferable that the parameters of the point spread function of the incoherent imaging system are set so as to satisfy the relationship of the following formula (A):

number

[0015] Furthermore, it is preferable that the process of acquiring a desired number of focused images of the subject and information on the depth positions of the focused images in the focused image / depth position information acquisition means is performed based on the evaluation results of the sharpness of the reconstructed image of the digital hologram.

[0016] Furthermore, a digital hologram imaging and reproducing device of the present invention is characterized by comprising an imaging element, an imaging optical system that causes two beams of light, at least one of which carries subject information, to interfere with each other to form a hologram image on the imaging element, and any of the digital hologram signal processing devices described above.

[0017] Furthermore, it is preferable that the above digital hologram imaging and reproducing device comprises a coherent light source and an interferometer optical system that obtains a digital hologram using coherent light from the light source, and that the digital hologram signal processing device comprises a memory that stores the depth resolution of the interferometer optical system; it is also preferable that the digital hologram signal processing device comprises a self-interferometer optical system that obtains a digital hologram when incoherent light from a subject is incident on it, and that the digital hologram signal processing device comprises a memory that stores the depth resolution of the self-interferometer optical system. [Effects of the Invention]

[0018] According to the digital hologram signal processing device and digital hologram imaging and reproduction device of the present invention, by adding together a reference image formed by adding together in-focus images obtained by adjusting reconstructed images located within the depth of field, and a group of defocused images formed by adjusting reconstructed images located outside the depth of field, the blur characteristics of the image of the subject located at the defocused position can be converted from a coherent response to an incoherent response, and the amount of blur can be controlled arbitrarily by further changing the parameters of the point spread function of the incoherent imaging system.

[0019] This makes it possible to reconstruct a two-dimensional image showing the blur conditions seen with a regular camera, even from data captured using digital holography technology. Furthermore, it is possible to combine this with images captured by a regular camera.

[0020] Furthermore, it is possible to impart different amounts of blur depending on the depth position of the subject, and to freely control the amount of blur, something that was not possible with ordinary cameras or conventional digital holography.By adding blur, which is a feature of imaging, to an arbitrarily focused image, it is possible to create a dramatic effect. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram for mainly explaining the flow of signal processing related to blur conversion by a digital hologram signal processing device according to an embodiment of the present invention. [Figure 2] 1 is a schematic diagram showing the configuration of an incoherent imaging system of a digital hologram imaging and reproducing apparatus according to an embodiment of the present invention. [Figure 3] 1A and 1B are schematic diagrams showing an example of the configuration of a digital hologram imaging and reproducing device according to an embodiment of the present invention ((a) is a case where a coherent light source is used as illumination light, and (b) is a case where ambient incoherent light is used as illumination light). [Figure 4] FIG. 2 is a block diagram illustrating an example of a digital hologram signal processing unit according to an embodiment of the present invention. [Figure 5] 3(a) shows a subject image when the optical system of the device shown in FIG. 3(a) is used, and FIG. 3(b) shows a digital hologram obtained by capturing the subject image. [Figure 6] This shows two-dimensional images reconstructed based on the digital hologram shown in Figure 5(b) (reconstructed images when the deviation from the focal plane position is -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e), respectively). [Figure 7] The results of converting blur using the technology of this embodiment are shown (the F-number (in-focus plane position) is 8.33, and the deviation from the in-focus plane position is -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e), respectively). [Figure 8]The results of converting blur using the technology of this embodiment are shown (the F-number (in-focus plane position) is 25, and the deviation from the in-focus plane position is -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e). [Figure 9] The results of converting blur using the technology of this embodiment are shown (the F-number (in-focus plane position) is 83.33, and the deviation from the in-focus plane position is -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e), respectively). [Figure 10] In an embodiment of the present invention, when imaging multiple objects arranged in the depth direction, (a) shows the relative positions of three objects and (b) shows the digital hologram obtained by imaging the three objects. [Figure 11] This shows the case where the reconstructed image based on the digital hologram shown in Figure 10(b) is focused (the case of the reconstructed image of object "1" (a), the case of the reconstructed image of object "2" (b), and the case of the reconstructed image of object "3" (c)). [Figure 12] Using the embodiment technology of the present invention, the results of blur conversion for the reconstructed image based on the digital hologram shown in Figure 11(a) are shown, with the depth position of object "1" as the focal plane ((a) is for D = 1.0 (F-number: 20), (b) is for D = 0.6 (F-number: 33.33), and (c) is for D = 0.2 (F-number: 100)). [Figure 13] Using the embodiment technology of the present invention, the results of blur conversion for the reconstructed image based on the digital hologram shown in Figure 11(b) are shown, with the depth position of object "2" as the focal plane ((a) is for D = 1.0 (F-number: 20), (b) is for D = 0.6 (F-number: 33.33), and (c) is for D = 0.2 (F-number: 100)). [Figure 14]Using the embodiment technology of the present invention, the results of blur conversion for the reconstructed image based on the digital hologram shown in Figure 11(c) are shown, with the depth position of object "3" as the focal plane ((a) is for D = 1.0 (F-number: 20), (b) is for D = 0.6 (F-number: 33.33), and (c) is for D = 0.2 (F-number: 100)). DETAILED DESCRIPTION OF THE INVENTION

[0022] A digital hologram signal processing device and a digital hologram imaging and reproducing device according to an embodiment of the present invention will be described below with reference to the drawings. The flow of signal processing according to the present invention is shown in Figure 1. Note that each step from step 12 (S12) onwards is performed in a digital hologram signal processing device (hereinafter also referred to as signal processing units 18 and 18': see Figures 3(a) and 3(b)), specifically by using a computer program, and is displayed on a monitor. First, a digital hologram is captured (S11) using the interference optical system (described later) of the digital hologram capture and playback device and stored. Based on the captured digital hologram data, depth position information and focused images of each subject (subjects 1 to 4) are acquired using the following procedure (S12: <1> indicates the focused image and depth position of each of subjects 1 to 4).

[0023] That is, in step 12 (S12), a diffraction propagation calculation is applied to the intensity distribution of the digital hologram or the complex amplitude distribution extracted from the digital hologram using the phase shift method or Fourier fringe analysis method. Methods such as Fresnel diffraction integral and angular spectrum method (plane wave expansion) can be used to calculate the diffraction propagation. When calculating the diffraction propagation, the propagation distance is changed stepwise and sequentially to obtain two-dimensional reconstructed images at each propagation distance. These two-dimensional reconstructed images become focused or blurred depending on the change in the depth position of the object, i.e., depending on the propagation distance. When a focused image is formed, the propagation distance at that time corresponds to the depth position of the object. Therefore, by searching for the most focused image from the two-dimensional reconstructed images at each propagation distance, the depth position of the object can be obtained. The most focused image can also be found by evaluating the sharpness of the two-dimensional reconstructed images. Sharpness can be evaluated by comparing the derivatives of each image or the magnitude of the spatial frequency components of each image. These techniques are known as autofocus techniques in the field of digital holography.

[0024] The sharpness of each of the obtained two-dimensional reconstructed images is evaluated, and the propagation distance at which the sharpness is highest is taken as the depth position of the subject. This depth position information and the in-focus image of the subject are acquired and stored in signal processing units 18, 18' (see FIG. 3). Note that the depth position of the subject may also be acquired by using a device such as a lidar for measurement.

[0025] Next, the signal processors 18, 18' (see FIG. 3) are provided with information on the depth position to be focused, information on the desired depth of field centered on that depth position, and parameters of the incoherent imaging system corresponding to the amount of blur to be achieved (S13). The depth of field is set to a value greater than the depth resolution of the optical system in the digital hologram imaging and reproduction device 19, 19'. This allows the response of the imaging system to be calculated physically correctly, eliminating unnecessary calculations.In addition, information on the depth resolution of the optical system is stored in the memory of the signal processing units 18 and 18', reflecting information on the configuration of the optical system.

[0026] 2, there are four parameters of the incoherent imaging system: focal length f of virtual lens 6, aperture diameter D, distance z1 from virtual lens (hereinafter simply referred to as lens) 6 to the focused images of subjects 1 to 4, and distance z2 from virtual lens 6 to image plane 7. Based on these four parameters, the point spread function of the incoherent imaging system is formed as shown in the following equation (1).

number

[0027] Using the parameters set as described above, a two-dimensional image with blur conversion is generated using the following procedure. As shown in FIG. 2, the in-focus images of subjects 2 and 3 that exist within the set depth position information and depth of field are added together to obtain an added image I ((S14): <2> shows the added image I of the in-focus images within the depth of field). Then, using the following equation (2), a reference image B is obtained by convolution of the added image I with the point spread function (PSF) of the incoherent imaging system ((S15): Reference image B is shown in <3>).

number

[0028] When performing the calculation process of the above equation (2), by setting the parameters of the point spread function of the above equation (1) so as to satisfy the relationship of the below equation (3), it is possible to obtain an image that looks like it is focused on subjects 2 and 3 with a normal camera.

number

[0029] On the other hand, the influence of P(x, y) in the above equation (1), that is, the limitation of the finite aperture diameter of the lens, also includes an effect of reducing the resolution. Therefore, if you want to suppress the reduction in resolution, you can perform the calculation using the PSF in the above equation (2) as a delta function, or you can omit this calculation altogether and set B = I. In addition, when calculating the above equations (1) and (2), a method using a Fourier transform algorithm is generally used. In this case, energy is normalized to prevent any increase or decrease in energy depending on the algorithm, and care must be taken to ensure that the intensity is constant.

[0030] Separately from the creation of the reference image B described above, a defocused image Dd of the subjects 1 and 4 outside the depth of field is created. When creating the defocused image Dd, the following equations (4) and (5) are calculated. That is, a defocused image Dd is obtained by convolving each of the in-focus images of the subjects 1 and 4 outside the depth of field with the point spread function of the imaging system ((S16): <4> shows the defocused image Dd outside the depth of field).

number

[0031] Finally, by adding the reference image B and all the defocused images (the group of defocused images Dd), a two-dimensional image with a converted blur state can be obtained ((S17): <5> shows the two-dimensional image obtained as a result of signal processing (by adding all the images together)).

[0032] In the above calculations, the values ​​of f and z2 are fixed in order to impart a blurring effect to all subjects 1 to 4 when a common lens (virtual lens) 6 is used, but either of these may be changed depending on the subjects 1 to 4. The depth position information of each of the subjects 1 to 4 may also be changed to any value. By performing the above operations, it is possible to obtain a two-dimensional image that is blurred in a way that cannot be captured with a normal camera, and furthermore, since the depth position information can be changed arbitrarily, the range of video expression can be expanded.

[0033] As a digital hologram imaging and reproducing device that realizes the above-mentioned signal processing for blur conversion, for example, the device shown in FIGS. 3(a) and 3(b) can be used. The optical system of the imaging and reproducing device shown in Figures 3(a) and (b) is a common configuration in the field of digital holography. However, what makes this imaging and reproducing device unique is that it is equipped with signal processing units 18 and 18' that perform signal processing for blur conversion, and in addition, information about the depth resolution of the optical system, which is necessary for highly efficient calculation of the signal processing for blur conversion, is stored and set in the signal processing units 18 and 18'.

[0034] Figure 3(a) shows a digital hologram imaging and reconstructing device 19 that uses coherent light as a light source. Laser light from laser light source 16 within digital hologram imaging and reconstructing device 19 is converted into a plane wave by spatial filter 15 and lens 11, and then split by beam splitter 13. One of the split beams is irradiated onto subject 12, while the other is reflected by plane mirror 14. The former light is used as object light and the latter light as reference light, and after being combined by beam splitter 13, the two are caused to interfere on the surface of image sensor 17. This is captured as a digital hologram and stored in the memory section (not shown) of signal processing section 18.

[0035] On the other hand, Figure 3(b) shows a digital hologram imaging and reconstructing device 19' that uses ambient incoherent light as the light source. Reflected light from an object 12' illuminated with ambient incoherent light is collected by lens 11' and split by beam splitter 13'. The split incoherent light is reflected by plane mirror 14' and concave mirror 20, returned to beam splitter 13', and combined, causing interference on the surface of image sensor 17'. This is captured by image sensor 17' as a digital hologram, which is then stored in the memory section (not shown) of signal processing section 18'.

[0036] 4 is a block diagram showing the internal configuration of the signal processing unit 18 (similarly to the signal processing unit 18'). That is, the signal processing unit 18 includes a focused image / depth position information acquiring means 18A, a reference image acquiring means 18B, a defocused image group acquiring means 18C, and a reference image / defocused image group adding means 18D.

[0037] First, the focused image / depth position information acquisition means 18A is configured to acquire the focused image of the subject 12 (for each of the subject images 1 to 4) and the depth position information of the focused image from the reconstructed image of the digital hologram obtained by the interference optical system of the digital hologram imaging and reproduction device 19, 19'.

[0038] Furthermore, the reference image acquisition means 18B sets the elements of the depth of field, the focal length f of the lens 6, the aperture diameter D of the lens 6, and the distance z2 from the lens 6 to the image plane 7 according to the depth position so as to specify an in-region in-focus image that is set at a desired depth position within the depth of field region, and acquires the reference image B by adding together the intensity distributions of a predetermined number of obtained in-region in-focus images.

[0039] Furthermore, it is desirable that the calculation process for acquiring the reference image B in the reference image acquisition means 18B be a calculation using a convolution integral of the intensity distribution of the point spread function of the incoherent imaging system and the combined intensity distribution obtained by adding together the intensity distributions of the in-region focused images (focused images of subjects 2 and 3) obtained by sequentially changing the depth position setting value.

[0040] On the other hand, the defocused image group acquisition means 18C sets the elements of the focal length f of the lens 6, the aperture diameter D of the lens 6, and the distance z2 from the lens 6 to the image plane according to the depth position so as to be able to specify an out-of-area focused image (in-focus image of subjects 1 and 4) that is located at a desired depth position outside the depth of field, and calculates the convolution integral of the intensity distribution of each of the obtained out-of-area focused images (in-focus image of subjects 1 and 4) and the intensity distribution of the point spread function of the incoherent imaging system to obtain a defocused image group Dd. In other words, this defocused image group acquisition means 18C converts the blur state of the image of the subject 12 present at the defocused position from a coherent response to an incoherent response, and further adjusts the amount of blur by changing the parameters of the point spread function of the incoherent imaging system.

[0041] Furthermore, the reference image / defocused image group addition means 18D adds the reference image B obtained by the reference image acquisition means 18B and the defocused image group Dd obtained by the defocused image group acquisition means 18C to each other to obtain an incoherent response image. Although not shown in FIG. 4, the signal processing unit 18 is provided with a memory unit that stores the depth resolution of the optical system of the interferometer and the like.

[0042] The optical systems of the digital hologram imaging and reproducing devices 19 and 19' shown in Figures 3(a) and 3(b) represent examples of the optical systems of the digital hologram imaging and reproducing devices according to the present invention, and various other configurations can be adopted as long as they are capable of imaging a digital hologram using the interference phenomenon and deriving depth position information of the optical system. [Example]

[0043] Below, we will explain the reconstruction results obtained by the embodiments of the present invention, but we will also explain the contents and results of a simulation related to the prior art as a comparative example. In the simulations according to this embodiment and the comparative example, the diffraction of light waves is calculated by the angular spectrum method, and the propagation of light can be calculated accurately. The digital hologram imaging and reproducing device used was one using coherent light as shown in Figure 3(a). The wavelength of the coherent light was 633 nm, and the subject 12 was represented as a two-dimensional image as shown in Figure 5(a), with the depth position of the subject 12 set to 100 mm. The reference light was a plane wave with a planar phase. The image sensor 17 used had 256 x 256 pixels, a pixel pitch of 10 μm, and 8-bit gradation. The hologram image acquired by the image sensor in the comparative example was similar to that shown in Figure 5(b).

[0044] Figure 6 shows the image reconstructed by extracting the complex amplitude distribution from this hologram obtained by the imaging element using the phase shift method. The propagation distance was changed in 10 mm intervals in an area within a range of ±20 mm centered on the focus position of the subject, and a 2D image was obtained at each position. That is, (c) shows the reconstruction result when the set propagation distance is 0 mm (the focus position of the subject), and (a), (b), (d), and (e) show the reconstruction results when the set propagation distances are -20 mm, -10 mm, 10 mm, and 20 mm, respectively. From the reconstruction results in Figure 6, it is clear that in the reconstruction calculations of ordinary digital holography, ringing occurs in the defocused image, and the blur state is different from that of an ordinary camera.

[0045] Next, the results of reconstruction using the technology of this embodiment are shown in Figure 7. The parameters of the incoherent imaging system and virtual lens 6 are z1 = 100 mm, z2 = 100 mm, f = 50 mm, D = 6 mm (F-number 8.33), and the length of the depth of field region is 8 mm. It is clear that by applying the technology of this embodiment, the defocused image is smoother than in Figure 6. Furthermore, the processing results when D = 2 mm (F-number 25) and D = 0.6 mm (F-number 83.33) are shown in Figures 8 and 9 (the propagation distances for (a) to (e) are the same as in Figure 6). As the F-number increases, the amount of blur in the defocused image is reduced, and it is clear that the amount of blur is well controlled.

[0046] Next, the application results when three subjects 12 are placed 60 mm apart in the depth direction as shown in Figure 10(a) will be described below. When these three subjects 12 are imaged, the digital hologram shown in Figure 10(b) is obtained. For convenience, the three subjects 12 will be referred to as "object 1," "object 2," and "object 3" in the depth direction in Figure 10(a).

[0047] The results of applying the phase shift method and applying diffraction propagation calculations to the digital hologram in Fig. 10(b) are shown in Fig. 11(a) to (c). When each subject 12 is focused and reconstructed, it is clear that a defocused image with the blurred state (having a striped pattern) characteristic of coherent light is obtained. The reconstruction results obtained by applying the technology of this embodiment to the reconstruction results of FIGS. 11(a) to 11(c) are shown in FIGS. 12, 13, and 14. FIG.

[0048] Figures 12, 13, and 14 show the reconstruction results when the positions of objects "1," "2," and "3" are set as the depth of focus positions and the depth of field is set to 10 mm. In addition, the amount of blur for each of Figures 12 to 14 (a) shows the results when the aperture diameter D is 1 (F-number 20), for each of Figures 12 to 14 (b) shows the results when D is 0.6 (F-number 33.33), and for each of Figures 12 to 14 (c) shows the results when D is 0.2 (F-number 100). As is clear from the reconstruction results described above, by using the technology of this embodiment, it is possible to convert the blur state from a state using coherent light to a state using incoherent light, and further it is clear that the amount of blur can be adjusted as desired.

[0049] The simulation results described above are for the case where a digital hologram imaging and reproducing device 19 using coherent light is used, but the same effects can be achieved in the case of a digital hologram imaging and reproducing device 19' using incoherent light as shown in Figure 3(b).

[0050] The digital hologram signal processing device and digital hologram imaging and reproducing device of the present invention are not limited to the above-described embodiments, and various other modifications are possible. [Explanation of symbols]

[0051] 1~4, 12, 12´ Subject 6 Virtual Lens 7 Image plane 11,11´ lens 13, 13´ Beam splitter 14, 14´ plane mirror 15 Spatial Filters 16 Laser light source 17, 17´ image sensor 18, 18´ Signal processing section 18A Focused image / depth position information acquisition means 18B Reference image acquisition means 18C Blur adjustment method 18D Reference image / defocused image group addition means 19, 19´ Digital hologram imaging and reconstructing device 20 concave mirror B Reference image Dd Defocused images I Addition image

Claims

1. A digital hologram signal processing device performs signal processing for a digital hologram imaging and reproducing device that captures three-dimensional information of a subject by utilizing interference of light, a focused image / depth position information acquisition means for acquiring a desired number of focused images of an object and information on the depth positions of the focused images from a reconstructed image of a digital hologram obtained by an interference optical system of the digital hologram imaging and reconstructing device; a reference image acquisition means for setting each element of the depth of field, the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane in accordance with a desired depth position within the depth of field so as to specify an in-area in-focus image set at the desired depth position within the depth of field, and acquiring a reference image by adding together a predetermined number of the in-area in-focus images thus obtained; a defocused image group acquiring means for acquiring a defocused image group by calculating a convolution integral between an intensity distribution of each of the obtained out-of-area focused images and an intensity distribution of a point spread function of an incoherent imaging system, the defocused image group acquiring means setting each element of the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane according to a desired depth position so as to specify an out-of-area focused image located at a desired depth position outside the depth of field, and the intensity distribution of each of the obtained out-of-area focused images and an intensity distribution of a point spread function of an incoherent imaging system; a reference image / defocused image group adding means for adding the reference image obtained by the reference image obtaining means and the defocused image group obtained by the defocused image group obtaining means to each other to obtain an incoherent response image; A digital hologram signal processing device comprising:

2. The digital hologram signal processing device according to claim 1, characterized in that the calculation process for acquiring the reference image in the reference image acquisition means is a calculation process related to a convolution integral of a combined intensity distribution acquired by adding up the intensity distributions of each of the in-region focused images obtained by changing the depth position and an intensity distribution of the point spread function of the incoherent imaging system.

3. The digital hologram signal processing device according to claim 2, characterized in that, when performing the arithmetic processing relating to the convolution integral of the combined intensity distribution and the intensity distribution of the point spread function of the incoherent imaging system, parameters of the point spread function of the incoherent imaging system are set so as to satisfy the relationship of the following equation (A). [0010] However, the distance from the lens to the focused image of the subject is z 1 , the distance from the lens to the image plane is z 2 , and the focal length of the lens is f.

4. A digital hologram signal processing device as described in any one of claims 1 to 3, characterized in that the process of acquiring a desired number of focused images of the subject and information on the depth positions of the focused images in the focused image / depth position information acquisition means is performed based on the evaluation results of the sharpness of the reconstructed image of the digital hologram.

5. A digital hologram imaging and reproducing device comprising: an imaging element; an imaging optical system for forming a hologram image on the imaging element by causing two systems of light, at least one of which carries subject information, to interfere with each other; and a digital hologram signal processing device according to any one of claims 1 to 4.

6. The digital hologram imaging and reproducing device according to claim 5, further comprising a coherent light source and an interferometer optical system for obtaining a digital hologram using the coherent light from the light source, and further comprising a memory within the digital hologram signal processing device for storing the depth resolution of the interferometer optical system.

7. The digital hologram imaging and reproducing device according to claim 5, further comprising a self-interferometer optical system that obtains a digital hologram when incoherent light from a subject is incident thereon, and further comprising a memory within the digital hologram signal processing device that stores the depth resolution of the self-interferometer optical system.

Citation Information

Patent Citations

  • Method and device for processing picture

    JP1995021365A

  • Image reconstruction method, apparatus and program

    JP2005253702A

  • Hologram recording apparatus and method of recording hologram

    JP2010249639A

  • Holographic reconstruction device

    JP2010534859A

  • Incoherent fluorescence digital holographic microscopy using a transmission liquid crystal lens

    JP2016533542A