Image system, three-dimensional element image acquisition method, recording medium, second lens array, three-dimensional display device, and three-dimensional element image signal transmission method

Abstract

An image system (2) is provided with: an aerial image forming optical system (6) which has retroreflective characteristics and forms an aerial image (16) representing a three-dimensional object (4) by imaging light from the three-dimensional object (4) in the air; a first lens array (8) disposed at or near the imaging position of the aerial image (16) and having a plurality of first lenses (34) disposed in a two-dimensional shape; and a camera (10) having an imaging lens (36) and an imaging element (38), and outputting a three-dimensional element image signal including light ray information relating to light rays from the aerial image (16) by imaging the light rays from the aerial image (16) that have passed through the first lens array (8).

Description

Technical Field

[0001] The present invention relates to an imaging system, a method for acquiring a three-dimensional element image, a recording medium, a second lens array, a three-dimensional display device, and a method for transmitting a three-dimensional element image signal. Background Art

[0002] Light field technology is a technology that acquires, processes, and reproduces light information related to light emitted from a three-dimensional object. In this light field technology, a three-dimensional image that can be stereoscopically observed with the naked eye can be displayed based on the acquired light information, and by reproducing the light emitted from the three-dimensional object in the three-dimensional image, the shape or texture of the surface of the three-dimensional object can be realistically reproduced.

[0003] As a first example of light field technology, the following technology is known: a real image of a three-dimensional object as a photography target is formed using an optical lens, and a camera photographs light that passes through a lens array arranged at the imaging position (for example, refer to Non-Patent Document 1).

[0004] As a second example of the light field technology, a technology is known in which a three-dimensional object as an imaging target is imaged using a dedicated camera having an imaging element and a lens array disposed near the imaging element (see, for example, Non-Patent Document 2).

[0005] As a third example of the light field technology, a multi-camera technology is known that uses 100 or more cameras to photograph a three-dimensional object as an imaging target (see, for example, Non-Patent Document 3).

[0006] Prior Art Literature

[0007] Non-patent literature

[0008] Non-patent document 1: J.Arai, et al., "Integral three-dimensional television with video system using pixel-offset method," Optics Express, Vol.21, No.3, pp.3474-3485 (2013)

[0009] Non-patent document 2: Ng R., et.al "Light Field Photography with a Hand-held Plenoptic Camera," Stanford University Computer Science Tech Report, pp. 1-11 (2005)

[0010] Non-patent document 3: H.Watanabe, et.al, "Aktina Vision: Full-parallax three-dimensional display with 100million light rays," Scientific Reports, Vol.9, Article number: 17688 (2019)

[0011] Non-patent document 4: Parity mirror (registered trademark), [retrieved on June 29, 2021], Internet (https: / / evort.jp / store / piq / product / paritymirror)

[0012] Non-patent document 5: Yusho Yamamoto, "Aerial Display Based on Aerial Image Regression (AIRR)", Journal of the Image Society of Japan, Vil. 56, No. 4 pp. 341-351 (2017)

[0013] Non-patent document 6: Tetsuro Kuwayama, "Aerial display technology 3D imaging to realize a contactless society", Vol. 33, No. 3, pp. 9-20 (May 2021) Summary of the invention

[0014] Problems to be solved by the invention

[0015] In the technology disclosed in non-patent document 1, in order to obtain light information with a wide viewing angle, an optical lens with a very large diameter (about 50 cm to 1 m) is required, but it is difficult to manufacture an optical lens of such a size. In addition, since the distance from the three-dimensional object to be photographed to the camera is about several meters, the overall scale of the system becomes large and lacks practicality.

[0016] Furthermore, in the technology disclosed in Non-Patent Document 2, the viewing angle of light from a three-dimensional object that can be captured by a camera is narrow (approximately 1°), and the light information that can be obtained is extremely small, so the reproducibility of the three-dimensional image is low.

[0017] In addition, in the technology disclosed in non-patent document 3, more than 100 cameras are used, so the cost is high and the scale of the entire system becomes large, which lacks versatility. Furthermore, due to the limitation of the size of the camera, the cameras cannot be arranged at close intervals, so in order to obtain dense light information, it is necessary to perform processing to generate light information by interpolation from the images captured by the camera, which makes the processing complicated. In addition, it takes a long time to perform calculation processing on a large number of image signals captured simultaneously by more than 100 cameras, which requires a lot of labor and time.

[0018] The present invention aims to solve the above-mentioned problems, and its purpose is to provide an imaging system, a three-dimensional element image acquisition method, a recording medium, a second lens array, a three-dimensional display device and a three-dimensional element image signal transmission method that can suppress the overall scale of the system to be compact and obtain light information with a wide viewing angle.

[0019] Means for solving problems

[0020] In order to achieve the above-mentioned purpose, the imaging system involved in the first mode of the present invention comprises: an aerial image forming optical system, which has a regressive reflection characteristic, and forms an aerial image representing the three-dimensional object by imaging the light from the three-dimensional object in the air; a first lens array, which is arranged at the imaging position of the aerial image or in the vicinity thereof, and has a plurality of first lenses arranged in a two-dimensional state; and a camera, which has an imaging lens and an imaging element, and outputs a three-dimensional element image signal including light information related to the light from the aerial image by photographing the light from the aerial image after passing through the first lens array.

[0021] According to this method, an aerial image forming optical system having a regressive reflective characteristic is used, so that even when the size of the aerial image forming optical system is small, the aerial image forming optical system can obtain light from a dimensional object at a wide viewing angle. Thus, the overall scale of the system can be suppressed to be compact, and light information at a wide viewing angle can be obtained.

[0022] For example, in the imaging system involved in the second embodiment of the present invention, in the first embodiment, the camera may output the three-dimensional element image signal (i) to an external terminal device, (ii) to an external image recording device, or (iii) to a three-dimensional element image display device.

[0023] According to this aspect, it is possible to appropriately process the three-dimensional element video signal output from the camera.

[0024] For example, in the imaging system involved in the third embodiment of the present invention, in the first embodiment or the second embodiment, the aerial image forming optical system may form the aerial image at the imaging position symmetrical to the three-dimensional object relative to the aerial image forming optical system by reflecting the light from the three-dimensional object at least twice.

[0025] According to this aspect, an aerial image can be easily formed by the aerial image forming optical system.

[0026] For example, in the imaging system according to the fourth aspect of the present invention, in any one of the first to third aspects, the plurality of first lenses may each be formed into a circular, rectangular or hexagonal shape when the first lens array is viewed in a plan view.

[0027] According to this aspect, it is possible to use the first lens array having appropriate optical characteristics according to the application and the like.

[0028] For example, in the imaging system according to the fifth aspect of the present invention, in any one of the first to fourth aspects, the surface shape of each of the plurality of first lenses may be a spherical surface or an aspherical surface.

[0029] According to this aspect, it is possible to use the first lens array having appropriate optical characteristics according to the application and the like.

[0030] For example, in the imaging system involved in the 6th mode of the present invention, in any one of the 1st to 5th modes, it can also be constructed that the curvature in the horizontal direction of each of the multiple first lenses is the same as the curvature in the vertical direction, or the curvature in the horizontal direction of each of the multiple first lenses is larger than the curvature in the vertical direction.

[0031] According to this aspect, it is possible to use the first lens array having appropriate optical characteristics according to the application and the like.

[0032] For example, in the imaging system involved in the 7th mode of the present invention, in any one of the 1st to 6th modes, the imaging system can also be constructed as follows: an opening array, which is arranged between the aerial image forming optical system and the first lens array, and has a plurality of openings respectively arranged at the positions of the focal lengths of the plurality of first lenses.

[0033] According to this aspect, stray light can be prevented from entering the first lens array by cutting off stray light with the aperture array. In addition, by adjusting the size of each of the plurality of apertures, light information with a wide depth reproduction range can be obtained.

[0034] For example, in the imaging system according to the eighth aspect of the present invention, in any one of the first to seventh aspects, the camera may include a single digital camera or a plurality of digital cameras.

[0035] According to this aspect, when the camera includes a single digital camera, the scale of the entire system can be suppressed to be compact. In addition, when the camera includes a plurality of digital cameras, a wide viewing angle and more light information can be obtained.

[0036] For example, in the imaging system involved in the 9th mode of the present invention, in any one of the 1st mode to the 8th mode, the imaging system can also be constructed as follows: a three-dimensional element image display device, which receives the three-dimensional element image signal output from the camera and has a display surface for displaying a three-dimensional element image based on the three-dimensional element image signal; and a second lens array, which has a shape substantially the same as that of the first lens array and is configured to be opposite to the display surface of the three-dimensional element image display device, and has a plurality of second lenses arranged in a two-dimensional shape, and the optical path length between the display surface of the three-dimensional element image display device and the second lens array is substantially equal to the focal length of each of the plurality of second lenses.

[0037] According to this aspect, the observer can stereoscopically observe the three-dimensional element images as three-dimensional images by observing the three-dimensional element images displayed on the display surface of the three-dimensional element image display device through the second lens array.

[0038] In the three-dimensional element image acquisition method involved in the 10th embodiment of the present invention, an aerial image forming optical system with a regressive reflection characteristic is used to form an image of light from a three-dimensional object in the air, thereby forming an aerial image representing the three-dimensional object; the light from the aerial image is transmitted through a first lens array which is arranged at or near the imaging position of the aerial image and has a plurality of first lenses arranged in a two-dimensional state; a camera having an imaging lens and an imaging element is used to photograph the light from the aerial image after passing through the first lens array, thereby outputting a three-dimensional element image signal including light information related to the light from the aerial image.

[0039] According to this aspect, it is possible to obtain light information over a wide viewing angle while keeping the scale of the entire system compact.

[0040] A recording medium according to an eleventh aspect of the present invention records a three-dimensional element video represented by a three-dimensional element video signal output from the video system according to any one of the first to ninth aspects.

[0041] The second lens array according to the twelfth aspect of the present invention is the second lens array in the imaging system according to the ninth aspect.

[0042] A three-dimensional display device according to a thirteenth aspect of the present invention includes the three-dimensional element image display device and the second lens array in the image system according to the ninth aspect.

[0043] For example, in the three-dimensional display device involved in the 14th mode of the present invention, in the 13th mode, it can also be configured that the second lens array can be disassembled relative to the display surface of the three-dimensional element image display device, and the three-dimensional display device further includes: a detection unit, which detects the disassembly and assembly of the second lens array relative to the display surface of the three-dimensional element image display device; and a display control unit, which controls the display content on the display surface of the three-dimensional element image display device based on the detection result of the detection unit, and the display control unit (i) causes the display surface to display the three-dimensional element image when the second lens array is installed on the display surface, and (ii) causes the display surface to display images other than the three-dimensional element image when the second lens array is removed from the display surface.

[0044] According to this aspect, the display contents on the display surface of the three-dimensional element image display device can be appropriately changed in conjunction with the attachment and detachment of the second lens array with respect to the display surface of the three-dimensional element image display device.

[0045] For example, in the three-dimensional display device involved in the 15th embodiment of the present invention, in the 13th embodiment, the second lens array may also have: a polarization direction switching element for the light emitted from the display surface of the three-dimensional element image display device to pass through, and (i) when the three-dimensional element image is displayed on the display surface, the light emitted from the display surface is polarized in a first polarization direction, and (ii) when an image other than the three-dimensional element image is displayed on the display surface, the light emitted from the display surface is polarized in a second polarization direction different from the first polarization direction; and a polarization-dependent lens array for the light in the first polarization direction or the second polarization direction after passing through the polarization direction switching element to be incident.

[0046] According to this method, when the second lens array is installed on the display surface of the three-dimensional element image display device, the observer can observe the three-dimensional element image stereoscopically as a three-dimensional image, or visually recognize other images other than the three-dimensional element image as a two-dimensional image, based on the display content on the display surface of the three-dimensional element image display device.

[0047] The method for transmitting a three-dimensional element image signal involved in the 16th mode of the present invention is a method for transmitting the three-dimensional element image signal output from the camera in the image system as described in claim 1, comprising: (a) a step of converting the three-dimensional element image signal output from the camera into a signal of a multi-view image group or a signal of a multi-view image group with a depth image; (b) a step of encoding the signal converted in (a) in a prescribed encoding method; (c) a step of transmitting the signal encoded in (b); and (d) a step of decoding the signal transmitted in (c) in a prescribed decoding method, wherein the viewing angle of each of the multiple multi-view images included in the multi-view image group is greater than 30° and less than 90°, and the viewing angle interval of the multiple element images included in the multi-view image is greater than 0.5° and less than 2°.

[0048] According to this aspect, a three-dimensional element video signal can be easily transmitted.

[0049] Effects of the Invention

[0050] According to an imaging system and the like according to one embodiment of the present invention, it is possible to obtain light information at a wide viewing angle while suppressing the scale of the entire system to be compact. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a diagram showing the configuration of a video system according to the first embodiment.

[0052] Figure 2 This is a diagram showing an aerial image forming optical system of the imaging system according to the first embodiment.

[0053] Figure 3 This is a diagram showing a first lens array of the imaging system according to the first embodiment.

[0054] Figure 4 This is a diagram showing the optical path of light rays that pass through the first lens array of the imaging system according to the first embodiment.

[0055] Figure 5 This is a diagram showing a three-dimensional element image display device of the image system according to the first embodiment.

[0056] Figure 6 The diagram shows an example of a three-dimensional element image.

[0057] Figure 7 It is a diagram showing the configuration of an imaging system according to a comparative example.

[0058] Fig. 8A This is a diagram showing a first lens array according to Modification 1 of Embodiment 1.

[0059] Figure 8B This is a diagram showing a first lens array according to a second modification of the first embodiment.

[0060] Figure 8C This is a diagram showing a first lens array according to a third modification of the first embodiment.

[0061] Fig.8D This is a diagram showing a first lens array according to Modification 4 of Embodiment 1.

[0062] Fig. 8E This is a diagram showing a first lens array according to a fifth modification of the first embodiment.

[0063] Fig. 9 This is a diagram showing a partial structure of an image system according to the second embodiment.

[0064] Fig.10 This is a diagram showing a partial structure of an image system according to the third embodiment.

[0065] Fig.11 This is a diagram showing a partial structure of an image system according to a fourth embodiment.

[0066] Fig.12 This is a diagram showing the structure of an image system according to the fifth embodiment.

[0067] Fig.13 This is a diagram showing the structure of an image system according to a sixth embodiment.

[0068] Fig.14 This is a block diagram showing the functional structure of the image system involved in embodiment 7.

[0069] Fig.15 This is a diagram for explaining the operation of the processing unit involved in Implementation Example 7. DETAILED DESCRIPTION

[0070] Hereinafter, the embodiments of the present invention will be described in detail using the accompanying drawings. In addition, the embodiments described below all show general or specific examples. The numerical values, shapes, materials, constituent elements, configuration positions and connection methods of the constituent elements shown in the following embodiments are examples, and their purpose is not to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements not described in the independent claims are described as arbitrary constituent elements.

[0071] (Implementation Method 1)

[0072] [1-1. Composition of the imaging system]

[0073] First, refer to Figure 1 to Figure 5The configuration of the imaging system 2 according to the first embodiment will be described. Figure 1 This is a diagram showing the configuration of the imaging system 2 according to the first embodiment. Figure 2 This is a diagram showing the aerial image forming optical system 6 of the imaging system 2 according to the first embodiment. Figure 3 This is a diagram showing the first lens array 8 of the imaging system 2 according to the first embodiment. Figure 4 This is a diagram showing the optical path of light rays that pass through the first lens array 8 of the imaging system 2 according to the first embodiment. Figure 5 This is a diagram showing the three-dimensional element image display device 12 of the image system 2 according to the first embodiment.

[0074] The imaging system 2 is a system for obtaining a three-dimensional element image signal based on light (light beam group) emitted from a three-dimensional object 4 using light field technology, and displaying a three-dimensional element image representing the three-dimensional object 4 based on the obtained three-dimensional element image signal. In the present embodiment, the three-dimensional object 4 is, for example, a human head.

[0075] In this specification, "three-dimensional element image" means an image that cannot be viewed in stereoscopic form because it is a two-dimensional image, but can be viewed in stereoscopic form with the naked eye as a three-dimensional image by observing the three-dimensional element image through a lens array. In addition, in this specification, "image" is a concept that includes not only dynamic images but also images (still images).

[0076] like Figure 1 As shown, the imaging system 2 includes an aerial image forming optical system 6, a first lens array 8, a camera 10, a three-dimensional element image display device 12, and a second lens array 14. In addition, the three-dimensional element image display device 12 is combined with the second lens array 14 to form a three-dimensional display device 15. The imaging system 2 is used, for example, for (a) commercial purposes such as broadcasting, industrial design, showrooms, digital signage, and communication between remote locations, (b) entertainment purposes such as stereoscopic image theaters and public viewing, and (c) art / science / educational purposes such as digital art, 3D simulators, digital museums, and digital archives.

[0077] The aerial image forming optical system 6 is a panel-shaped optical device having a regressive reflective property, and forms an aerial image 16 representing the three-dimensional object 4 by imaging the light emitted from the three-dimensional object 4 in the air. Figure 1 The size in the vertical direction) is about 30 cm, for example, and the horizontal size (in Figure 1The size in the direction perpendicular to the paper surface) is, for example, approximately 30 cm. The aerial image forming optical system 6 has: an incident surface 18 for the light emitted from the three-dimensional object 4 to be incident, and an exit surface 20 (the surface on the opposite side of the incident surface 18) for the light used to form the aerial image 16 to be emitted. Figure 1 In the example, the incident surface 18 of the aerial image forming optical system 6 is arranged relative to Figure 1 The paper plane is perpendicular to and opposite to the three-dimensional object 4.

[0078] Among the light rays emitted from the three-dimensional object 4, light rays in an angle range corresponding to the viewing angle θ (e.g., 30° to 90°) are incident on the incident surface 18. In addition, the "viewing angle" is an angle range in which the observer can visually recognize the three-dimensional image. Here, there is no center of the optical system in the aerial image forming optical system 6, so when the size of the three-dimensional object 4 is large (e.g., when the three-dimensional object 4 is the whole body of a person), by arranging (tiling) multiple aerial image forming optical systems 6 in a two-dimensional shape, an aerial image 16 of a size corresponding to the size of the three-dimensional object 4 can be formed.

[0079] The aerial image 16 is formed at an imaging position that is plane-symmetrical with respect to the three-dimensional object 4 with respect to the aerial image forming optical system 6. In addition, the depth of the aerial image 16 is reversed with respect to the depth of the three-dimensional object 4. For example, when the face of a person in the three-dimensional object 4 faces the incident surface 18 of the aerial image forming optical system 6, the face of the person in the aerial image 16 faces the exit surface 20 of the aerial image forming optical system 6.

[0080] The aerial image forming optical system 6 is composed of, for example, Figure 2 The dihedral corner reflector array shown in (a) is composed of. Figure 2 As shown in FIG. 2( a ), the aerial image forming optical system 6 includes a base 22 and a plurality of dihedral corner reflectors 24 .

[0081] The base 22 is formed in a flat plate shape. The base 22 has a first principal surface 22 a and a second principal surface 22 b arranged on the opposite side of the first principal surface 22 a. The second principal surface 22 b of the base 22 functions as the emission surface 20 of the aerial image forming optical system 6 .

[0082] The plurality of dihedral corner reflectors 24 are arranged in an array on the first main surface 22a of the base 22. Each of the plurality of dihedral corner reflectors 24 is a small reflector formed in a rectangular parallelepiped shape. The size of one side of the dihedral corner reflector 24 is, for example, 0.1 mm to 1 mm. Figure 2 As shown in (b), a first reflecting surface 26a and a second reflecting surface 26b are formed on two adjacent side surfaces 24a and 24b of each of the plurality of dihedral corner reflectors 24. The top surface 24c of each of the plurality of dihedral corner reflectors 24 functions as the incident surface 18 of the aerial image forming optical system 6.

[0083] like Figure 2 As shown by the arrow 28 in (b), among the light rays incident on the top surface 24c of the dihedral corner reflector 24, the light rays are reflected once on the first reflecting surface 26a and once on the second reflecting surface 26b, for a total of two times (total reflection), and are emitted from the bottom surface 24d of the dihedral corner reflector 24 to form an aerial image 16.

[0084] In addition, in the present embodiment, the aerial image forming optical system 6 is composed of a dihedral corner reflector array, but it is not limited thereto, and may be composed of, for example, a structure in which curved needle-shaped reflectors are orthogonally arranged up and down (for example, see non-patent document 4), or an optical system composed of a combination of a retro-reflective sheet and a half-reflective mirror (for example, see non-patent document 5), etc. As the above-mentioned retro-reflective sheet, for example, a bead array or a corner cube array can be used (for example, see non-patent document 6).

[0085] like Figure 1 As shown, the first lens array 8 is arranged at the imaging position of the aerial image 16 (or in its vicinity). Here, the vicinity of the aerial image 16 means a range within a few centimeters from the imaging position of the aerial image 16. The first lens array 8 has: an incident surface 30 for incident light emitted from the aerial image 16, and an exit surface 32 (the surface on the opposite side of the incident surface 30) for exiting light incident on the incident surface 30. Figure 1 In the embodiment, the incident surface 30 of the first lens array 8 is arranged relative to Figure 1 The paper surface is perpendicular to the paper surface and is opposite to the exit surface 20 of the aerial image forming optical system 6.

[0086] In addition, regarding the positional relationship between the first lens array 8 and the aerial image 16, the depth position of the aerial image 16 where the first lens array 8 is disposed is imaged with the highest spatial resolution. Therefore, the first lens array 8 may be disposed at a depth position where it is desired to image with the highest spatial resolution among the depth positions of the aerial image 16. For example, when the aerial image 16 is a person's head, the first lens array 8 may be disposed at a depth position of the face of the person in the aerial image 16.

[0087] like Figure 3 As shown, the first lens array 8 has a plurality of first lenses 34 arranged two-dimensionally along the incident surface 30. Figure 3, a portion of the first lens array 8 is enlarged and illustrated. When the first lens array 8 is observed in a plan view (i.e., when observed from a direction perpendicular to the incident surface 30), each of the plurality of first lenses 34 is formed, for example, into a hexagon. That is, the first lens array 8 has a honeycomb structure. In this way, by forming each of the plurality of first lenses 34 into a hexagon, the plurality of first lenses 34 can be arranged two-dimensionally without gaps, so that the non-lens portion can be eliminated in the first lens array 8, and the effective area can be expanded. In addition, each of the plurality of first lenses 34 is a microlens, and the size L of each of the plurality of first lenses 34 is preferably, for example, 300 μm to 2.5 mm, and more preferably 0.5 mm to 1.5 mm.

[0088] like Figure 4 As shown, when the light emitted from the aerial image 16 is incident on the incident surface 30 of the first lens array 8, light having a viewing angle θ is obtained in each of the plurality of first lenses 34. In addition, the light captured by the camera 10 among the light emitted from the exit surface 32 of the first lens array 8 becomes light that is roughly parallel to the camera 10 side.

[0089] The surface shape of each of the plurality of first lenses 34 may be either spherical or aspherical. In addition, by making the surface shape of each of the plurality of first lenses 34 aspherical, the influence of spherical aberration of peripheral light can be reduced, and light can be obtained with uniform accuracy within the viewing angle θ.

[0090] The curvature of each of the multiple first lenses 34 can also be set to a uniform curvature in all directions (that is, the curvature in the horizontal direction can be the same as the curvature in the vertical direction). Alternatively, the curvature in the horizontal direction and the curvature in the vertical direction of each of the multiple first lenses 34 can be different from each other. In this way, the light interval, light density, field of view angle, etc. that can be obtained by the first lens array 8 can be controlled. For example, by making the curvature in the horizontal direction of each of the multiple first lenses 34 larger than the curvature in the vertical direction, the field of view angle in the horizontal direction can be made larger than the field of view angle in the vertical direction. In this specification, the "horizontal direction" is a direction parallel to the ground, and the "vertical direction" is a direction perpendicular to the ground.

[0091] In addition, in the present embodiment, the first lens array 8 is composed of a refractive lens, but it is not limited to this, and it can also be composed of a diffractive lens. Alternatively, the first lens array 8 can also be composed of a biconvex lens. In this case, each of the plurality of first lenses 34 is composed of a so-called cylindrical lens having a curvature in the horizontal direction and no curvature in the vertical direction. Thus, although only the light information in the horizontal direction is obtained, it is effective for obtaining a three-dimensional image with a high resolution and only horizontal parallax that can allocate pixel information to resolution characteristics.

[0092] The camera 10 is a digital camera having an imaging lens 36 and an imaging element 38, and is configured to be opposite to the exit surface 32 of the first lens array 8. In the present embodiment, only one camera 10 is configured. The camera 10 photographs the light emitted from the exit surface 32 of the first lens array 8 (i.e., the light from the aerial image 16 after passing through each of the multiple first lenses 34 of the first lens array 8). Specifically, the light emitted from the exit surface 32 of the first lens array 8 is imaged on the imaging element 38 through the imaging lens 36 and is photographed by the imaging element 38. The camera 10 generates a three-dimensional element image signal including light information related to the traveling direction and color of the light emitted from each of the multiple first lenses 34, and outputs the generated three-dimensional element image signal to the three-dimensional element image display device 12. At this time, as shown in FIG. Figure 4 As shown, when the camera 10 is sufficiently far away from the first lens array 8, the camera 10 can also capture the light emitted from the emission surface 32 of the first lens array 8 in the panoramic focus state. Thus, a three-dimensional element image signal with high resolution in the depth range can be generated.

[0093] In addition, in order to capture the light rays emitted from each of the plurality of first lenses 34 with 100 to 400 pixels or more, the camera 10 is preferably composed of an ultra-high-definition digital camera of 8K or higher. Figure 1 As shown, the camera 10 outputs the three-dimensional element image signal to the three-dimensional element image display device 12, but is not limited to this, and can also be output to an external terminal device such as a personal computer, and can also be recorded in an external image recording device such as a cloud server. In the case of outputting the three-dimensional element image signal to an external terminal device, in the terminal device, for example, a three-dimensional shape is calculated, a three-dimensional model is generated, and a multi-viewpoint image is generated. Alternatively, the camera 10 can also distribute the three-dimensional element image represented by the three-dimensional element image signal to a remote location via a network. In addition, for example, the three-dimensional element image represented by the three-dimensional element image signal can also be recorded on a recording medium such as a floppy disk, a hard disk, a CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD, and a semiconductor memory.

[0094] like Figure 1 As shown, the three-dimensional element image display device 12 is formed of, for example, a liquid crystal display, and receives a three-dimensional element image signal from the camera 10. A display surface 40 is formed on the front surface of the three-dimensional element image display device 12. On the display surface 40 of the three-dimensional element image display device 12, a three-dimensional element image 42 is displayed based on the three-dimensional element image signal from the camera 10. As described above, the three-dimensional element image 42 is a two-dimensional image that cannot be observed stereoscopically by itself. Figure 1 In the embodiment, the display surface 40 of the three-dimensional element image display device 12 is configured relative to Figure 1The paper plane is perpendicular to the observer 50. The camera 10 captures the light and the 3D element image display device 12 displays the 3D element image 42 in real time. Thus, the 3D element image 42 becomes a moving 2D image that is synchronized with the movement of the 3D object 4 that moves in real time.

[0095] In addition, the number of the first lenses 34 of the first lens array 8 is equivalent to the resolution of the three-dimensional element image 42. Furthermore, the number of pixels for capturing an image in one first lens 34 determines the light density. In addition, the viewing angle that can be captured by the camera 10 is determined according to the numerical aperture of the first lens 34. Therefore, as the first lens array 8, a small and dense lens structure is preferably used.

[0096] The second lens array 14 is a lens array having substantially the same shape as the first lens array 8. The second lens array 14 is arranged to be opposite to the display surface 40 of the three-dimensional element image display device 12. The second lens array 14 has: an incident surface 44 for incident light from the display surface 40 of the three-dimensional element image display device 12, and an exit surface 46 (a surface on the opposite side of the incident surface 44) for exiting the light incident on the incident surface 44. The second lens array 14 has a plurality of second lenses 48 arranged in a two-dimensional shape along the incident surface 44, and each of the plurality of second lenses 48 has substantially the same shape as the plurality of first lenses 34 of the first lens array 8. The optical path length between the display surface 40 of the three-dimensional element image display device 12 and the second lens array 14 is substantially equal to the focal length of each of the plurality of second lenses 48.

[0097] The light from the three-dimensional element image 42 displayed on the display surface 40 of the three-dimensional element image display device 12 is incident on the incident surface 44 of the second lens array 14. At this time, the light from the three-dimensional element image 42 is converted into light (that is, light emitted from the aerial image 16) that is respectively incident on the multiple first lenses 34 of the first lens array 8 by the multiple second lenses 48 of the second lens array 14. This is because the second lens array 14 has a shape substantially the same as the first lens array 8. Thus, the observer 50 can observe the three-dimensional element image 42 displayed on the display surface 40 of the three-dimensional element image display device 12 in stereoscopic form with the naked eye as a three-dimensional image 52. That is, the three-dimensional image 52 is an image that reproduces (regenerates) the light emitted from the aerial image 16, and can also be said to be an image that represents the three-dimensional object 4. At this time, the observer 50 can observe the three-dimensional element image 42 displayed on the display surface 40 of the three-dimensional element image display device 12 at the above-mentioned viewing angle θ (refer to Figure 1 ) for stereoscopic observation of the three-dimensional image 52.

[0098] The depth of the three-dimensional image 52 is correct when viewed from the observer 50 . For example, when the face of the person in the three-dimensional object 4 faces the incident surface 18 of the aerial image forming optical system 6 , the face of the person in the three-dimensional image 52 faces the observer 50 .

[0099] Figure 6 4 is a diagram showing an example of a three-dimensional element image 42. More specifically, Figure 6 (a) is a diagram showing a three-dimensional element image 42. Figure 6 (b) is to Figure 6 FIG. 4 is an enlarged view of a portion of the three-dimensional element image 42 shown in (a). Figure 6 In the example shown, the three-dimensional object represented by the three-dimensional element image 42 is a stuffed bear. Figure 6 As shown in (b), in the three-dimensional element image 42, a plurality of hexagonal element images 54 are arranged without gaps. Each of the plurality of element images 54 is an image obtained by photographing the light rays emitted from each of the plurality of first lenses 34 of the first lens array 8. Light rays from each of the plurality of element images 54 pass through the plurality of second lenses 48 of the second lens array 14, thereby reproducing the light rays emitted from the aerial image 16.

[0100] [1-2. Effect]

[0101] Reference Figure 7 The configuration of an imaging system 100 according to a comparative example will be described. Figure 7 is a diagram showing the configuration of an imaging system 100 according to a comparative example. Figure 7 In the Figure 1 The same components are denoted by the same reference numerals, and description thereof will be omitted.

[0102] like Figure 7 As shown, the imaging system 100 includes an optical lens 102, a first lens array 8, a camera 10, a three-dimensional element image display device 12, and a second lens array 14. That is, in the imaging system 100, the optical lens 102 is provided instead of the aerial image forming optical system 6 of the imaging system 2 described above.

[0103] The optical lens 102 is composed of a convex lens and is disposed at a distance twice the focal length f from the three-dimensional object 4. The optical lens 102 forms a real image 104 of the three-dimensional object 4 at a distance twice the focal length f. In addition, the up-down direction of the real image 104 is reversed relative to the up-down direction of the three-dimensional object 4, but the depth of the real image 104 is not reversed relative to the depth of the three-dimensional object 4.

[0104] The first lens array 8 is arranged at the imaging position of the real image 104. The light radiated from the real image 104 passes through the first lens array 8. The camera 10 captures the light after passing through the first lens array 8 to generate a three-dimensional element image signal, and outputs the generated three-dimensional element image signal to the three-dimensional element image display device 12. As a result, the three-dimensional element image represented by the three-dimensional element image signal is displayed in the three-dimensional element image display device 12. The observer 50 can observe the three-dimensional element image displayed on the three-dimensional element image display device 12 in stereoscopic form as a three-dimensional image 106 with naked eyes by observing the three-dimensional element image displayed on the three-dimensional element image display device 12 through the second lens array 14.

[0105] However, in the above-mentioned imaging system 100, in order to obtain light information with a wide viewing angle, the diameter D of the optical lens 102 needs to be very large (for example, D = about 50 cm to 1 m), and it is difficult to manufacture such a large optical lens 102. In addition, the distance L from the three-dimensional object 4 to the camera 10 is about several meters, so the overall scale of the imaging system 100 becomes large and lacks practicality.

[0106] In addition, the depth of the three-dimensional image 106 becomes the reverse depth when observed from the observer 50. For example, when the face of the person in the three-dimensional object 4 is facing the optical lens 102, the face of the person in the three-dimensional image 106 is facing the opposite side of the observer 50. This is because the depth of the real image 104 formed by the optical lens 102 is not reversed relative to the depth of the three-dimensional object 4. Therefore, the observer 50 cannot stereoscopically observe the three-dimensional image 106 at the correct depth.

[0107] In contrast, in the present embodiment, an aerial image forming optical system 6 having a regressive reflection characteristic is used, so even when the size of the aerial image forming optical system 6 is small (e.g., about 30 cm×30 cm), the viewing angle θ of the light from the three-dimensional object 4 incident on the aerial image forming optical system 6 can be ensured to be, for example, about 30° to 90°. Thus, the overall scale of the imaging system 2 can be suppressed to be compact, and light information with a wider viewing angle θ can be obtained. In addition, even when only a single digital camera is used as the camera 10, light information with a wider viewing angle θ can be obtained compared to a conventional dedicated camera with a built-in imaging element and a lens array.

[0108] In addition, in the aerial image 16 formed by the aerial image forming optical system 6, since there is no optical aberration or optical distortion, it is possible to suppress image degradation. In addition, in the aerial image forming optical system 6, since there is no inherent focal length, it is possible to form the aerial image 16 at the same magnification even when the three-dimensional object 4 is located at a position at an arbitrary distance from the aerial image forming optical system 6. In addition, the depth of the aerial image 16 is reversed relative to the depth of the three-dimensional object 4, so that the depth of the three-dimensional image 52 can be made the correct depth when observed from the observer 50. Furthermore, as shown in FIG. Figure 5 As shown in (a) and (b), the distribution of light radiated in various directions from the three-dimensional image 52 is roughly the same as the distribution of light radiated in various directions from the three-dimensional object 4. Therefore, the texture of the surface of the three-dimensional object 4 (such as gloss, etc.) can be faithfully reproduced through the three-dimensional image 52.

[0109] [1-3. Various Modifications of the First Lens Array]

[0110] [1-3-1. Modification 1]

[0111] Reference Fig. 8A The configuration of the first lens array 8A according to Modification 1 of Embodiment 1 will be described. Fig. 8A This is a diagram showing a first lens array 8A according to Modification 1 of Embodiment 1.

[0112] like Fig. 8A As shown, the first lens array 8A has a plurality of first lenses 34A configured in a two-dimensional shape. When observing the first lens array 8A in a plan view, the plurality of first lenses 34A are each formed into a circle. In addition, the arrangement of the plurality of first lenses 34A is a stacked arrangement. In this case, a non-lens portion 56 is formed between two adjacent first lenses 34A.

[0113] In the present modification, the arrangement of the plurality of first lenses 34A is a stacked arrangement, but the present invention is not limited thereto, and for example, a square arrangement may be employed.

[0114] [1-3-2. Modification 2]

[0115] Reference Figure 8B The configuration of the first lens array 8B according to the second modification of the first embodiment will be described. Figure 8B This is a diagram showing a first lens array 8B according to a second modification of the first embodiment.

[0116] like Figure 8BAs shown, the first lens array 8B has a plurality of first lenses 34A arranged in a two-dimensional shape. The shapes and arrangements of the plurality of first lenses 34A are the same as those of the above-mentioned modification 1. In addition, a non-lens portion 56 formed between two adjacent first lenses 34A is provided with a light shielding portion 58. For example, the light shielding portion 58 is formed by applying a black paint having light shielding properties. By means of the light shielding portion 58, it is possible to suppress useless leakage light from penetrating the non-lens portion 56.

[0117] [1-3-3. Modification 3]

[0118] Reference Figure 8C The configuration of a first lens array 8C according to Modification 3 of Embodiment 1 will be described. Figure 8C This is a diagram showing a first lens array 8C according to a third modification of the first embodiment.

[0119] like Figure 8C As shown, the first lens array 8C has a plurality of first lenses 34C arranged in a two-dimensional shape. When the first lens array 8C is viewed in a plan view, each of the plurality of first lenses 34C is formed in a square shape (an example of a rectangle). In addition, the plurality of first lenses 34C are arranged in a square shape. Thus, it is possible to Figure 8C left and right directions) and vertical directions (in Figure 8C The light is obtained at the same viewing angle in both directions (up and down).

[0120] [1-3-4. Modification 4]

[0121] Reference Fig.8D The configuration of a first lens array 8D according to Modification 4 of Embodiment 1 will be described. Fig.8D This is a diagram showing a first lens array 8D according to a fourth modification of the first embodiment.

[0122] like Fig.8D As shown, the first lens array 8D has a plurality of first lenses 34D arranged in a two-dimensional shape. When the first lens array 8D is viewed in a plan view, each of the plurality of first lenses 34D is formed into a rectangular shape (an example of a rectangle). The long sides of each of the plurality of first lenses 34D are in the horizontal direction (in Fig.8D The short side extends in the vertical direction (in the left and right direction) and the short side extends in the vertical direction (in the Fig.8D In addition, the arrangement of the plurality of first lenses 34D is a square arrangement. Thus, the viewing angle that can be obtained in the horizontal direction and the vertical direction can be easily controlled by the aspect ratio of the first lens 34D.

[0123] [1-3-5. Modification 5]

[0124] Reference Fig. 8EThe configuration of a first lens array 8E according to a fifth modification of the first embodiment will be described. Fig. 8E This is a diagram showing a first lens array 8E according to a fifth modification of the first embodiment.

[0125] like Fig. 8E As shown in FIG. 1 , the first lens array 8E has a plurality of first lenses 34E arranged in a two-dimensional shape. When the first lens array 8E is viewed in a plan view, each of the plurality of first lenses 34E is formed into a rectangular shape (an example of a rectangle). In addition, each of the long sides of the plurality of first lenses 34E is oriented horizontally (in the horizontal direction). Fig. 8E The short side extends in the vertical direction (in the left and right direction) and the short side extends in the vertical direction (in the Fig. 8E In addition, the plurality of first lenses 34E are arranged in a staggered arrangement. Thus, the resolution characteristics in the horizontal direction can be improved.

[0126] (Implementation Method 2)

[0127] Reference Fig. 9 The configuration of an imaging system 2F according to the second embodiment will be described. Fig. 9 It is a figure which shows a part of the structure of the imaging system 2F which concerns on Embodiment 2. In each embodiment shown below, about the component same as Embodiment 1 mentioned above, the same code|symbol is attached|subjected and the description is abbreviate|omitted.

[0128] like Fig. 9 As shown, the imaging system 2F includes a focusing lens 60 in addition to the components described in the above-mentioned embodiment 1. The focusing lens 60 is a convex lens, and is arranged to be opposite to the exit surface 32 of the first lens array 8. In addition, a camera 10 is arranged at a position away from the focusing lens 60 by a focal length f. The focusing lens 60 focuses the approximately parallel light rays toward the camera 10 side among the light emitted from the exit surface 32 of the first lens array 8 to the position of the camera 10. As a result, compared with the above-mentioned embodiment 1, the camera 10 can be brought closer to the first lens array 8, and the overall scale of the imaging system 2F can be suppressed to be more compact.

[0129] (Implementation method 3)

[0130] Reference Fig.10 The configuration of the imaging system 2G according to the third embodiment will be described. Fig.10 This is a diagram showing a partial configuration of a video system 2G according to the third embodiment.

[0131] like Fig.10As shown, the imaging system 2G includes an aperture array 62 in addition to the components described in the first embodiment. The aperture array 62 is arranged to face the incident surface 30 of the first lens array 8. That is, the aperture array 62 is arranged in the aerial image forming optical system 6 (refer to Figure 1 ) and the first lens array 8. The opening array 62 has a plurality of openings 64. The plurality of openings 64 are each arranged to be opposite to the plurality of first lenses 34 of the first lens array 8, and are arranged at the position of the focal length of each of the plurality of first lenses 34.

[0132] Thus, the aperture array 62 blocks stray light, thereby preventing stray light from entering the first lens array 8. As a result, blurring of the image in the three-dimensional element image 42 can be suppressed. In addition, by adjusting the size of each of the plurality of apertures 64, light information with a wide depth reproduction range can be obtained.

[0133] (Implementation 4)

[0134] Reference Fig.11 The configuration of the imaging system 2H ​​according to the fourth embodiment will be described. Fig.11 This is a diagram showing a partial configuration of a video system 2H ​​according to a fourth embodiment.

[0135] like Fig.11 As shown, the imaging system 2H ​​includes a plurality of cameras 10. Each of the plurality of cameras 10 is configured in the same manner as the single camera 10 described in the above-mentioned embodiment 1. That is, each of the plurality of cameras 10 captures light emitted from the exit surface 32 of the first lens array 8 to generate a three-dimensional element image signal, and outputs the generated three-dimensional element image signal to a three-dimensional element image display device (not shown).

[0136] As a result, the plurality of cameras 10 can easily acquire more light information at a wide viewing angle, thereby improving the reproducibility of the three-dimensional image 52 .

[0137] (Implementation 5)

[0138] Reference Fig.12 The configuration of the imaging system 2J according to the fifth embodiment will be described. Fig.12 This is a diagram showing the structure of a video system 2J according to the fifth embodiment.

[0139] like Fig.12 As shown, the three-dimensional display device 15J of the imaging system 2J includes a detection unit 66 and a display control unit 68 in addition to the three-dimensional element image display device 12 and the second lens array 14 described in the first embodiment. In addition, the second lens array 14 can be detached from the display surface 40 of the three-dimensional element image display device 12.

[0140] The detection unit 66 is a sensor for detecting attachment or detachment of the second lens array 14 with respect to the display surface 40 of the three-dimensional element image display device 12. The detection unit 66 outputs the detection result to the display control unit 68.

[0141] The display control unit 68 controls the display content on the display surface 40 of the three-dimensional element image display device 12 based on the detection result of the detection unit 66 .

[0142] Specifically, when the second lens array 14 is mounted on the display surface 40 of the three-dimensional element image display device 12, the display control unit 68 causes the display surface 40 to display the three-dimensional element image. Thus, the observer 50 can observe the three-dimensional element image displayed on the display surface 40 of the three-dimensional element image display device 12 through the second lens array 14, and can perform stereoscopic observation as a three-dimensional image 52 with naked eyes.

[0143] On the other hand, when the second lens array 14 is removed from the display surface 40 of the three-dimensional element image display device 12, the display control unit 68 causes the display surface 40 to display other images other than the three-dimensional element image. In addition, "other images other than the three-dimensional element image" are two-dimensional images that cannot be stereoscopically observed by themselves and cannot be stereoscopically observed even if the second lens array 14 is used, such as images that reproduce normal image content. Thus, the observer 50 can visually recognize other images displayed on the display surface 40 of the three-dimensional element image display device 12 as two-dimensional images by observing them without passing through the second lens array 14.

[0144] In addition, the display control unit 68 may be arranged inside the 3D element image display device 12 or outside the 3D element image display device 12 .

[0145] Therefore, in this embodiment, the display content on the display surface 40 of the three-dimensional element image display device 12 can be appropriately changed in conjunction with the attachment and detachment of the second lens array 14 to the display surface 40 of the three-dimensional element image display device 12 .

[0146] (Implementation method 6)

[0147] Reference Fig.13 The structure of the imaging system 2K according to the sixth embodiment will be described. Fig.13 This is a diagram showing the structure of an imaging system 2K according to the sixth embodiment.

[0148] like Fig.13 As shown, in the three-dimensional display device 15K of the video system 2K, the second lens array 14K has a polarization direction switching element 70 and a polarization-dependent lens array 72 .

[0149] The three-dimensional element image display device 12 is a liquid crystal display that uses light polarized in one direction for display. Alternatively, in the case of an EL (Electro Luminescence) display or an LED (Light Emitting Diode) display that does not use light polarized in one direction for display, the three-dimensional element image display device 12 is provided with an optical element (polarizer) that aligns polarization in one direction.

[0150] The polarization direction switching element 70 is arranged to be opposite to the display surface 40 of the three-dimensional element image display device 12. Thus, the light emitted from the display surface 40 of the three-dimensional element image display device 12 passes through the polarization direction switching element 70. The polarization direction switching element 70 controls the polarization direction of the light emitted from the display surface 40 of the three-dimensional element image display device 12 (i.e., the light passing through the polarization direction switching element 70) according to the display content on the display surface 40 of the three-dimensional element image display device 12.

[0151] Specifically, when a three-dimensional element image is displayed on the display surface 40 of the three-dimensional element image display device 12, the polarization direction switching element 70 polarizes the light emitted from the display surface 40 in the first polarization direction. On the other hand, when an image other than the three-dimensional element image is displayed on the display surface 40 of the three-dimensional element image display device 12, the polarization direction switching element 70 polarizes the light emitted from the display surface 40 in the second polarization direction that is orthogonal to the first polarization direction (i.e., different from the first polarization direction).

[0152] The polarization-dependent lens array 72 is composed of, for example, a liquid crystal lens, and is disposed opposite to the exit surface (i.e., the surface on the side away from the three-dimensional element image display device 12) of the polarization direction switching element 70. Thus, the light in the first polarization direction or the second polarization direction after passing through the polarization direction switching element 70 is incident on the polarization-dependent lens array 72.

[0153] When the light in the first polarization direction is incident on the polarization-dependent lens array 72, the light in the first polarization direction is converted into light incident on the first lens array 8 by the polarization-dependent lens array 72. That is, in this case, the polarization-dependent lens array 72 performs the same function as the second lens array 14 described in the above-mentioned embodiment 1. Thus, the observer 50 can observe the three-dimensional element image displayed on the display surface 40 of the three-dimensional element image display device 12 in stereoscopic form as a three-dimensional image 52 with naked eyes by observing it through the second lens array 14K.

[0154] On the other hand, when the light in the second polarization direction is incident on the polarization-dependent lens array 72, the light in the second polarization direction is transmitted through the polarization-dependent lens array 72 as it is. That is, in this case, the polarization-dependent lens array 72 performs the same function as, for example, a transparent glass plate without a lens function. Thus, the observer 50 can visually recognize the images other than the three-dimensional element images displayed on the display surface 40 of the three-dimensional element image display device 12 as two-dimensional images by observing them through the second lens array 14K.

[0155] Therefore, in this embodiment, when the second lens array 14K is installed on the display surface 40 of the three-dimensional element image display device 12, based on the display content on the display surface 40 of the three-dimensional element image display device 12, the observer 50 can stereoscopically observe the three-dimensional element image as a three-dimensional image 52, or visually recognize other images other than the three-dimensional element image as two-dimensional images.

[0156] (Implementation 7)

[0157] Reference Fig.14 and Fig.15 The configuration of the imaging system 2L according to the seventh embodiment will be described. Fig.14 This is a block diagram showing the functional structure of a video system 2L according to the seventh embodiment. Fig.15 This is a diagram for explaining the operation of the processing unit 74 involved in the seventh embodiment.

[0158] like Fig.14 As shown, the video system 2L includes a processing unit 74 as a functional configuration for performing a predetermined process on the three-dimensional element video signal output from the camera 10. The processing unit 74 includes a conversion unit 76, an encoding unit 78, and a decoding unit 80.

[0159] like Fig.15 As shown in (a), the conversion unit 76 converts the three-dimensional element image signal output from the camera 10 into a signal of a multi-view image group including a plurality of multi-view images. Here, the viewing angle θ of the multi-view image is greater than 30° and less than 90°. In addition, the viewing angle interval of the plurality of element images included in the multi-view image is preferably greater than 0.5° and less than 2°, and more preferably approximately 1°. Alternatively, the conversion unit 76 may also convert the three-dimensional element image signal output from the camera 10 into a signal of a multi-view image group with a depth image.

[0160] The encoding unit 78 encodes the signal of the multi-view image group converted by the conversion unit 76 using a coding method specified by H.264 / MVC (Multiview Video Coding), for example. Alternatively, the encoding unit 78 encodes the signal of the multi-view image group with depth image converted by the conversion unit 76 using a coding method specified by 3D HEVC (High Efficiency Video Coding), for example. The encoding unit 78 transmits the encoded signal to the decoding unit 80.

[0161] The decoding unit 80 decodes the signal encoded by the encoding unit 78 in a predetermined decoding method to generate a 3D element video signal. Fig.15 As shown in (b), among the plurality of element images included in the three-dimensional element image represented by the three-dimensional element image signal, the second lens array 14 (refer to Figure 1 ) includes a multi-viewpoint image. That is, the number of viewpoints of the multi-viewpoint image is equal to the number of pixels of one element image of the three-dimensional element image, and the number of pixels of the multi-viewpoint image group is equal to the number of element images of the three-dimensional element image (the number of second lenses 48). In addition, the element images of the three-dimensional element image correspond one-to-one to the second lenses 48 of the second lens array 14. The decoding unit 80 outputs the generated three-dimensional element image signal to the three-dimensional element image display device 12.

[0162] Therefore, in this embodiment, the three-dimensional element video signal can be easily transmitted.

[0163] (Other modifications, etc.)

[0164] Although the imaging systems according to the first to seventh embodiments of the present invention have been described above, the present invention is not limited to these embodiments. For example, the above-described embodiments may be combined.

[0165] In the above-mentioned embodiments, the three-dimensional element image display device 12 is formed by a liquid crystal display, but is not limited to this. For example, the three-dimensional element image display device 12 may be formed by a head-mounted display, a glasses-type display, or a goggle-type display.

[0166] Industrial Applicability

[0167] The present invention can be applied as, for example, an imaging system for acquiring a three-dimensional element image signal using a light field technique.

[0168] Description of reference numerals:

[0169] 2, 2F, 2G, 2H, 2J, 2K, 2L, 100 imaging system

[0170] 4 Three-dimensional objects

[0171] 6 Aerial image forming optical system

[0172] 8, 8A, 8B, 8C, 8D, 8E 1st lens array

[0173] 10. Camera

[0174] 12 Three-dimensional element image display device

[0175] 14, 14K 2nd lens array

[0176] 15, 15J, 15K 3D display device

[0177] 16 Aerial Image

[0178] 18, 30, 44 Incident surface

[0179] 20, 32, 46 exit surface

[0180] 22 base

[0181] 22a 1st main side

[0182] 22b Second main surface

[0183] 24 Dihedral Corner Reflector

[0184] 24a, 24b Side

[0185] 24c Top

[0186] 24d bottom

[0187] 26a 1st reflecting surface

[0188] 26b Second reflection surface

[0189] 28 Arrow

[0190] 34, 34A, 34C, 34D, 34E 1st lens

[0191] 36 Imaging lens

[0192] 38 Camera Components

[0193] 40 display surface

[0194] 42 Three-dimensional element images

[0195] 48 2nd lens

[0196] 50 Observer

[0197] 52, 106 Three-dimensional imaging

[0198] 54 element image

[0199] 56 Non-lens part

[0200] 58 shading part

[0201] 60 Condenser lens

[0202] 62 Opening Array

[0203] 64 Opening

[0204] 66 Detection Department

[0205] 68 Display control unit

[0206] 70 Polarization direction switching element

[0207] 72 Polarization-dependent lens array

[0208] 74 Processing Department

[0209] 76 Conversion Department

[0210] 78 Coding Department

[0211] 80 Decoding Department

[0212] 102 Optical lens

[0213] 104 Real Image

Claims

1. An imaging system comprising: The aerial image forming optical system has a retro-reflective property and forms an aerial image representing the three-dimensional object by imaging light from the three-dimensional object in the air; a first lens array, arranged at or near the imaging position of the aerial image, and having a plurality of first lenses arranged two-dimensionally; and The camera has an imaging lens and an imaging element, and outputs a three-dimensional element image signal including light information related to the light from the aerial image by imaging the light from the aerial image after passing through the first lens array.

2. The imaging system according to claim 1, The camera (i) outputs the 3D element video signal to an external terminal device, (ii) records it to an external video recording device, or (iii) outputs it to a 3D element video display device.

3. The imaging system according to claim 1, The aerial image forming optical system forms the aerial image at the imaging position symmetrical to the three-dimensional object with respect to the aerial image forming optical system by reflecting the light from the three-dimensional object at least twice.

4. The imaging system according to claim 1, When the first lens array is viewed in a plan view, each of the plurality of first lenses is formed in a circular, rectangular or hexagonal shape.

5. The imaging system according to claim 1, The surface shape of each of the plurality of first lenses is spherical or aspherical.

6. The imaging system according to claim 1, Each of the plurality of first lenses is configured such that a curvature in a horizontal direction is the same as a curvature in a vertical direction, or a curvature in a horizontal direction is larger than a curvature in a vertical direction.

7. The imaging system according to claim 1, The imaging system also has: The aperture array is arranged between the aerial image forming optical system and the first lens array, and has a plurality of apertures arranged at positions corresponding to the focal lengths of the plurality of first lenses.

8. The imaging system according to claim 1, The camera includes a single digital camera or a plurality of digital cameras.

9. The imaging system according to any one of claims 1 to 8, The imaging system also has: a three-dimensional element image display device that receives the three-dimensional element image signal output from the camera and has a display surface that displays a three-dimensional element image based on the three-dimensional element image signal; and The second lens array has a shape substantially the same as that of the first lens array and is arranged to face the display surface of the three-dimensional element image display device, and has a plurality of second lenses arranged in a two-dimensional shape. The optical path length between the display surface of the three-dimensional element image display device and the second lens array is substantially equal to the focal length of each of the plurality of second lenses.

10. A method for obtaining a three-dimensional element image, Using an aerial image forming optical system having a regressive reflection characteristic, light from a three-dimensional object is imaged in the air, thereby forming an aerial image representing the three-dimensional object. The light from the aerial image is transmitted through a first lens array which is arranged at or near the imaging position of the aerial image and has a plurality of first lenses arranged in a two-dimensional manner. A camera having an imaging lens and an imaging element is used to capture light from the aerial image after passing through the first lens array, thereby outputting a three-dimensional element image signal including light information related to the light from the aerial image. 11 . A recording medium having recorded thereon a three-dimensional element video represented by a three-dimensional element video signal output from the video system according to claim 1 .

12. A second lens array, which is the second lens array in the imaging system according to claim 9. 13 . A three-dimensional display device comprising the three-dimensional element image display device and the second lens array in the image system according to claim 9 .

14. The three-dimensional display device according to claim 13, The second lens array is detachable from the display surface of the three-dimensional element image display device. The three-dimensional display device further comprises: a detection unit that detects attachment or detachment of the second lens array relative to the display surface of the three-dimensional element image display device; and A display control unit controls display content on the display surface of the three-dimensional element image display device based on the detection result of the detection unit, The display control unit (i) causes the display surface to display the three-dimensional element images when the second lens array is installed on the display surface, and (ii) causes the display surface to display images other than the three-dimensional element images when the second lens array is removed from the display surface.

15. The three-dimensional display device according to claim 13, The second lens array has: a polarization direction switching element through which light emitted from the display surface of the three-dimensional element image display device passes, and (i) when the three-dimensional element image is displayed on the display surface, polarizes the light emitted from the display surface in a first polarization direction, and (ii) when an image other than the three-dimensional element image is displayed on the display surface, polarizes the light emitted from the display surface in a second polarization direction different from the first polarization direction; and The polarization-dependent lens array is used for incident light of the first polarization direction or the second polarization direction after passing through the polarization direction switching element.

16. A method for transmitting a 3D element image signal, which is a method for transmitting the 3D element image signal output from the camera in the image system according to claim 1, comprising: (a) a step of converting the three-dimensional element image signal output from the camera into a signal of a multi-view image group or a signal of a multi-view image group with a depth image; (b) a step of encoding the signal converted in (a) in a prescribed encoding method; (c) the step of transmitting the signal encoded in (b); and (d) a step of decoding the signal transmitted in (c) using a predetermined decoding method, The viewing angle of each of the plurality of multi-viewpoint images included in the multi-viewpoint image group is greater than or equal to 30° and less than or equal to 90°, and the viewing angle interval of the plurality of element images included in the multi-viewpoint image is greater than or equal to 0.5° and less than or equal to 2°.

Images