Video system, 3D elemental video acquisition method, and 3D elemental video signal transmission method

Abstract

A video system (2) comprises: an aerial image formation optical system (6) having recursive reflection characteristics and forming an aerial image (16), representing a three-dimensional object (4), by causing light rays from the three-dimensional object (4) to form an image in the air; a first lens array (8) that is disposed at or near the position where the aerial image (16) is formed, and that has a plurality of first lenses (34) arranged two-dimensionally; and a camera (10) that has an image-forming lens (36) and an imaging element (38), and that images light rays from the aerial image (16) which have passed through the first lens array (8) so as to output a three-dimensional element video signal which includes light ray information related to the light rays from the aerial image (16).

Description

[Technical Field]

[0001] The present invention relates to a video system, a three-dimensional elemental video acquisition method, a recording medium, a second lens array, a three-dimensional display device, and a three-dimensional elemental video signal transmission method. [Background technology]

[0002] Light field technology is a technology that acquires, processes, and reproduces light ray information related to light rays emitted from three-dimensional objects. This light field technology can display three-dimensional images that can be viewed stereoscopically with the naked eye based on the acquired light ray information, and by reproducing the light rays emitted from the three-dimensional object in this 3D image, it is possible to realistically reproduce the shape and texture of the surface of the three-dimensional object.

[0003] As a first example of light field technology, a technology is known in which an optical lens is used to form a real image of a three-dimensional object to be imaged, and light rays that pass through a lens array placed at the image-forming position are captured by a camera (see, for example, Non-Patent Document 1).

[0004] As a second example of light field technology, a technology is known in which an image of a three-dimensional object is captured using a dedicated camera equipped with an image sensor and a lens array arranged near the image sensor (see, for example, Non-Patent Document 2).

[0005] As a third example of light field technology, a multi-camera technology is known that uses 100 or more cameras to capture an image of a three-dimensional object (see, for example, Non-Patent Document 3). [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] J.Arai, et.al, “Integral three-dimensional television with video system using pixel-offset method,” Optics Express, Vol. 21, No. 3, pp. 3474-3485 (2013) [Non-patent document 2] Ng R., et.al “Light Field Photography with a Hand-held Plenoptic Camera,” Stanford University Computer Science Tech Report, pp.1-11 (2005) [Non-patent document 3] H.Watanabe, et.al, “Aktina Vision: Full-parallax three-dimensional display with 100 million light rays,” Scientific Reports, Vol.9, Article number: 17688 (2019) [Non-patent document 4] Parity Mirror (registered trademark), [Retrieved June 29, 2022], Internet (https: / / evort.jp / store / piq / product / paritymirror) [Non-Patent Document 5] Yuto Yamamoto, "Aerial Display Using Aerial Images by Retroreflection (AIRR)", Journal of the Imaging Society of Japan, Vol. 56, No. 4 pp. 341-351 (2017) [Non-patent document 6] Tetsuro Kuwayama, "3D Images in the Air: A Contactless Society," Vol. 33, No. 3, pp. 9-20 (May 2021) Summary of the Invention [Problem to be solved by the invention]

[0007] The technology disclosed in Non-Patent Document 1 requires an optical lens with a very large diameter (approximately 50 cm to 1 m) to acquire light ray information over a wide viewing angle, but it is difficult to fabricate an optical lens of this size. Also, since the distance from the three-dimensional object to be imaged to the camera is approximately several meters, the overall system becomes large and is not practical.

[0008] Furthermore, in the technology disclosed in Non-Patent Document 2, the viewing angle of light rays from a three-dimensional object that can be captured by a camera is narrow (approximately 1°), and the amount of light ray information obtained is extremely small, resulting in low reproducibility of three-dimensional images.

[0009] Furthermore, the technology disclosed in Non-Patent Document 3 uses 100 or more cameras, which increases costs and increases the overall system size, resulting in a lack of versatility. Furthermore, due to limitations on the size of the cameras, the cameras cannot be arranged closely together. Therefore, in order to obtain detailed light ray information, a separate process is required to interpolate and generate light ray information from the images captured by the cameras, which makes the process complicated. Moreover, it is necessary to take a long time to process the many video signals captured simultaneously by 100 or more cameras, which requires a huge amount of labor and time.

[0010] The present invention seeks to solve the above-mentioned problems, and its purpose is to provide an imaging system, a 3D elemental image acquisition method, a recording medium, a second lens array, a 3D display device, and a 3D elemental image signal transmission method that can acquire light ray information over a wide viewing angle while keeping the overall system size compact. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, the video system according to the first aspect of the present invention comprises an aerial image forming optical system having retroreflective properties, which forms an aerial image representing a three-dimensional object by focusing light rays from the three-dimensional object in the air; a first lens array arranged at or near the focusing position of the aerial image and having a plurality of first lenses arranged in a two-dimensional pattern; and a camera having an imaging lens and an image sensor, which captures light rays from the aerial image that have passed through the first lens array, and outputs a three-dimensional element video signal including light ray information regarding the light rays from the aerial image.

[0012] According to this aspect, since an aerial image forming optical system having retroreflection properties is used, even if the size of the aerial image forming optical system is relatively small, the aerial image forming optical system can acquire light rays from a three-dimensional object with a wide viewing angle. This makes it possible to acquire light ray information over a wide viewing angle while keeping the overall system size compact.

[0013] For example, in the video system according to the second aspect of the present invention, in the first aspect, the camera may be configured to (i) output the 3D element video signal to an external terminal device, (ii) record it in an external video recording device, or (iii) output it to a 3D element video display device.

[0014] According to this aspect, the three-dimensional elemental video signal output from the camera can be appropriately processed.

[0015] For example, in the video system according to the third aspect of the present invention, in the first or second aspect, the aerial image forming optical system may be configured to form the aerial image at the imaging position symmetrical to the three-dimensional object relative to the aerial image forming optical system by reflecting light rays from the three-dimensional object at least twice.

[0016] According to this aspect, an aerial image can be easily formed by the aerial image forming optical system.

[0017] For example, in the video system according to the fourth aspect of the present invention, in any one of the first to third aspects, when the first lens array is viewed in a plane, each of the plurality of first lenses may be configured to be formed in a circular, rectangular or hexagonal shape.

[0018] According to this aspect, a first lens array having appropriate optical characteristics depending on the application can be used.

[0019] For example, in the video system according to the fifth aspect of the present invention, in any one of the first to fourth aspects, the surface shape of each of the plurality of first lenses may be configured to be spherical or aspherical.

[0020] According to this aspect, a first lens array having appropriate optical characteristics depending on the application can be used.

[0021] For example, in the video system according to the sixth aspect of the present invention, in any one of the first to fifth aspects, each of the plurality of first lenses may be configured so that its curvature in the horizontal direction is the same as its curvature in the vertical direction, or so that its curvature in the horizontal direction is greater than its curvature in the vertical direction.

[0022] According to this aspect, a first lens array having appropriate optical characteristics depending on the application can be used.

[0023] For example, in the video system according to the seventh aspect of the present invention, in any one of the first to sixth aspects, the video system may further be configured to include an aperture array arranged between the aerial image forming optical system and the first lens array, the aperture array having a plurality of apertures each arranged at a focal length position of each of the plurality of first lenses.

[0024] According to this aspect, the aperture array cuts out unwanted light, thereby preventing unwanted light from entering the first lens array. In addition, by adjusting the size of each of the multiple apertures, light ray information with a wide depth reproduction range can be acquired.

[0025] For example, in the video system according to the eighth aspect of the present invention, in any one of the first to seventh aspects, the camera may be configured to include a single digital camera or a plurality of digital cameras.

[0026] According to this aspect, when the camera includes a single digital camera, the overall system size can be kept compact, and when the camera includes multiple digital cameras, more light ray information can be acquired over a wide viewing angle.

[0027] For example, in a video system according to a ninth aspect of the present invention, in any one of the first to eighth aspects, the video system further comprises a three-dimensional elemental image display device having a display surface that receives the three-dimensional elemental image signal output from the camera and displays a three-dimensional elemental image based on the three-dimensional elemental image signal, and a second lens array having approximately the same shape as the first lens array, arranged opposite the display surface of the three-dimensional elemental image display device, and having a plurality of second lenses arranged two-dimensionally, and the optical path length between the display surface of the three-dimensional elemental image display device and the second lens array may be configured to be approximately equal to the focal length of each of the plurality of second lenses.

[0028] According to this aspect, a viewer can view the three-dimensional elemental images displayed on the display surface of the three-dimensional elemental image display device through the second lens array, thereby stereoscopically viewing the three-dimensional elemental images as three-dimensional images.

[0029] A three-dimensional elemental image acquisition method according to a tenth aspect of the present invention forms an aerial image representing a three-dimensional object by using an aerial image forming optical system having retroreflection properties to focus light rays from the three-dimensional object in the air, transmits light rays from the aerial image through a first lens array arranged at or near the imaging position of the aerial image, the first lens array having a plurality of first lenses arranged two-dimensionally, and uses a camera having an imaging lens and an image sensor to image the light rays from the aerial image that have passed through the first lens array, thereby outputting a three-dimensional elemental image signal including light ray information regarding the light rays from the aerial image.

[0030] According to this aspect, it is possible to obtain light ray information over a wide viewing angle while keeping the overall system size compact.

[0031] A recording medium according to an eleventh aspect of the present invention is a recording medium on which a 3D elemental video represented by a 3D elemental video signal output from a video system according to any one of the first to ninth aspects is recorded.

[0032] A second lens array according to a twelfth aspect of the present invention is the second lens array in the video system according to the ninth aspect.

[0033] A three-dimensional display device according to a thirteenth aspect of the present invention includes the three-dimensional elemental image display device in the image system according to the ninth aspect and the second lens array.

[0034] For example, in a three-dimensional display device according to a fourteenth aspect of the present invention, in the thirteenth aspect, the second lens array is detachable from the display surface of the three-dimensional elemental image display device, and the three-dimensional display device further comprises a detection unit that detects attachment or detachment of the second lens array to or from the display surface of the three-dimensional elemental image display device, and a display control unit that controls the display content on the display surface of the three-dimensional elemental image display device based on the detection result of the detection unit, and the display control unit may be configured to (i) display the three-dimensional elemental image on the display surface when the second lens array is attached to the display surface, and (ii) display an image other than the three-dimensional elemental image on the display surface when the second lens array is detached from the display surface.

[0035] According to this aspect, the display content on the display surface of the three-dimensional elemental image display device can be appropriately changed in conjunction with the attachment / detachment of the second lens array to / from the display surface of the three-dimensional elemental image display device.

[0036] For example, in the 3D display device according to the 15th aspect of the present invention, in the 13th aspect, the second lens array may be configured to include a polarization direction switching element through which light rays emitted from the display surface of the 3D elemental image display device pass, the polarization direction switching element (i) polarizing the light rays emitted from the display surface in a first polarization direction when the 3D elemental image is displayed on the display surface, and (ii) polarizing the light rays emitted from the display surface in a second polarization direction different from the first polarization direction when an image other than the 3D elemental image is displayed on the display surface, and a polarization-dependent lens array onto which light rays in the first polarization direction or the second polarization direction that have passed through the polarization direction switching element are incident.

[0037] According to this aspect, while the second lens array is attached to the display surface of the three-dimensional elemental image display device, the observer can view the three-dimensional elemental images as three-dimensional images in stereoscopic form, or can view images other than the three-dimensional elemental images as two-dimensional images, depending on the display content on the display surface of the three-dimensional elemental image display device.

[0038] A method for transmitting a 3D element video signal according to a 16th aspect of the present invention is a method for transmitting the 3D element video signal output from the camera in the video system described in claim 1, and includes the steps of: (a) converting the 3D element video signal output from the camera into a signal of a multi-view video group or a signal of a multi-view video group with a depth image; (b) encoding the signal converted in (a) using a predetermined encoding method; (c) transmitting the signal encoded in (b); and (d) decoding the signal transmitted in (c) using a predetermined decoding method, wherein the viewing angle of each of the multiple multi-view videos included in the multi-view video group is between 30° and 90°, and the viewing angle interval of the multiple element videos included in the multi-view video is between 0.5° and 2°.

[0039] According to this aspect, the three-dimensional elemental video signal can be easily transmitted. [Effects of the Invention]

[0040] According to an image system or the like according to one aspect of the present invention, it is possible to obtain light ray information over a wide viewing angle while keeping the overall system size compact. [Brief explanation of the drawings]

[0041] [Figure 1] 1 is a diagram showing a configuration of a video system according to a first embodiment. [Figure 2] 1 is a diagram showing an aerial image forming optical system of a video system according to a first embodiment. [Figure 3] FIG. 2 is a diagram showing a first lens array of the video system according to the first embodiment. [Figure 4] 3 is a diagram showing the optical paths of light rays passing through a first lens array of the video system according to the first embodiment. FIG. [Figure 5] 1 is a diagram showing a three-dimensional elemental image display device of a video system according to a first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a three-dimensional elemental image. [Figure 7] FIG. 1 is a diagram showing a configuration of a video system according to a comparative example. [Figure 8A] FIG. 10 is a diagram showing a first lens array according to a first modification of the first embodiment. [Figure 8B] FIG. 10 is a diagram showing a first lens array according to a second modification of the first embodiment. [Figure 8C] FIG. 10 is a diagram showing a first lens array according to a third modification of the first embodiment. [Figure 8D] FIG. 10 is a diagram showing a first lens array according to a fourth modification of the first embodiment. [Figure 8E] FIG. 10 is a diagram showing a first lens array according to a fifth modification of the first embodiment. [Figure 9] FIG. 10 is a diagram showing a part of the configuration of a video system according to a second embodiment. [Figure 10] FIG. 11 is a diagram showing a part of the configuration of a video system according to a third embodiment. [Figure 11] FIG. 10 is a diagram showing a part of the configuration of a video system according to a fourth embodiment. [Figure 12] FIG. 13 is a diagram showing the configuration of a video system according to a fifth embodiment. [Figure 13] FIG. 20 is a diagram showing the configuration of a video system according to a sixth embodiment. [Figure 14] FIG. 13 is a block diagram showing the functional configuration of a video system according to a seventh embodiment. [Figure 15] FIG. 13 is a diagram for explaining the operation of a processing unit according to the seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0042] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement positions, and connection forms shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0043] (Embodiment 1) [1-1. Video system configuration] First, the configuration of a video system 2 according to the first embodiment will be described with reference to Figs. 1 to 5. Fig. 1 is a diagram showing the configuration of the video system 2 according to the first embodiment. Fig. 2 is a diagram showing an aerial image forming optical system 6 of the video system 2 according to the first embodiment. Fig. 3 is a diagram showing a first lens array 8 of the video system 2 according to the first embodiment. Fig. 4 is a diagram showing the optical paths of light rays passing through the first lens array 8 of the video system 2 according to the first embodiment. Fig. 5 is a diagram showing a 3D elemental image display device 12 of the video system 2 according to the first embodiment.

[0044] The video system 2 is a system for using light field technology to acquire three-dimensional elemental video signals based on light rays (a group of light rays) emitted from a three-dimensional object 4, and for displaying three-dimensional elemental videos representing the three-dimensional object 4 based on the acquired three-dimensional elemental video signals. In this embodiment, the three-dimensional object 4 is, for example, a human head.

[0045] In this specification, "3D elemental image" means an image that cannot be viewed stereoscopically because it is a 2D image, but can be viewed stereoscopically as a 3D image with the naked eye by viewing the 3D elemental image through a lens array. Also, in this specification, "image" is a concept that includes not only moving images but also images (still images).

[0046] As shown in Fig. 1, the video system 2 includes an aerial image forming optical system 6, a first lens array 8, a camera 10, a 3D elemental image display device 12, and a second lens array 14. The 3D elemental image display device 12 and the second lens array 14 are combined to form a 3D display device 15. This video system 2 is used, for example, in (a) business applications such as broadcasting, industrial design, showrooms, digital signage, and communication between remote locations, (b) amusement applications such as 3D video theaters and public viewings, and (c) art, science, and education applications such as digital art, 3D simulators, digital museums, and digital archives.

[0047] The aerial image forming optical system 6 is a panel-shaped optical device with retroreflection properties, and forms an aerial image 16 representing the three-dimensional object 4 by focusing light rays emitted from the three-dimensional object 4 in the air. The vertical size of the aerial image forming optical system 6 (the size in the up-down direction in FIG. 1 ) is, for example, approximately 30 cm, and the horizontal size (the size in the direction perpendicular to the paper surface in FIG. 1 ) is, for example, approximately 30 cm. The aerial image forming optical system 6 has an incident surface 18 onto which the light rays emitted from the three-dimensional object 4 are incident, and an exit surface 20 (the surface opposite to the incident surface 18) from which the light rays that form the aerial image 16 exit. In FIG. 1 , the incident surface 18 of the aerial image forming optical system 6 is perpendicular to the paper surface in FIG. 1 and is positioned so as to face the three-dimensional object 4.

[0048] Of the light rays emitted from the three-dimensional object 4, those within an angular range corresponding to the viewing angle θ (e.g., 30° to 90°) are incident on the incident surface 18. The "viewing angle" refers to the angular range over which a viewer can view a three-dimensional image. Here, since the aerial image forming optical system 6 does not have a center, if the size of the three-dimensional object 4 is relatively large (e.g., if the three-dimensional object 4 is the entire body of a person), an aerial image 16 of a size corresponding to the size of the three-dimensional object 4 can be formed by arranging (tiling) multiple aerial image forming optical systems 6 in a two-dimensional manner.

[0049] The aerial image 16 is formed at an imaging position that is plane-symmetrical to the three-dimensional object 4 with respect to the aerial image forming optical system 6. The depth of the aerial image 16 is inverted with respect to the depth of the three-dimensional object 4. For example, if the face of the person in the three-dimensional object 4 faces the entrance surface 18 of the aerial image forming optical system 6, the face of the person in the aerial image 16 will face the exit surface 20 of the aerial image forming optical system 6.

[0050] The aerial image forming optical system 6 is configured with, for example, a dihedral corner reflector array as shown in Fig. 2(a). As shown in Fig. 2(a), the aerial image forming optical system 6 has a base portion 22 and a plurality of dihedral corner reflectors 24.

[0051] The base portion 22 is formed in a flat plate shape. The base portion 22 has a first principal surface 22a and a second principal surface 22b disposed on the opposite side of the first principal surface 22a. The second principal surface 22b of the base portion 22 functions as the exit surface 20 of the aerial image forming optical system 6.

[0052] The dihedral corner reflectors 24 are arranged in an array on the first main surface 22a of the base portion 22. Each of the dihedral corner reflectors 24 is a micromirror formed in a rectangular parallelepiped shape. The size of one side of the dihedral corner reflector 24 is, for example, 0.1 mm to 1 mm. As shown in FIG. 2(b), a first reflecting surface 26a and a second reflecting surface 26b are formed on two adjacent side surfaces 24a and 24b of each of the dihedral corner reflectors 24, respectively. The top surface 24c of each of the dihedral corner reflectors 24 functions as the incident surface 18 of the aerial image forming optical system 6.

[0053] As shown by arrow 28 in (b) of Figure 2, of the light rays incident on the top surface 24c of the dihedral corner reflector 24, the light rays that are reflected twice (total reflection), once by each of the first reflecting surface 26a and the second reflecting surface 26b, exit from the bottom surface 24d of the dihedral corner reflector 24 and contribute to the formation of the aerial image 16.

[0054] In this embodiment, the aerial image forming optical system 6 is configured with a dihedral corner reflector array, but is not limited to this and may be configured with, for example, a structure in which looper-shaped mirrors are arranged vertically at right angles (see, for example, Non-Patent Document 4), or an optical system in which a retroreflective sheet and a half mirror are combined (see, for example, Non-Patent Document 5).As the retroreflective sheet described above, for example, a bead array or a corner cube mirror array can be used (see, for example, Non-Patent Document 6).

[0055] As shown in Fig. 1, the first lens array 8 is disposed at (or near) the imaging position of the aerial image 16. Here, "near the aerial image 16" means within a range of several centimeters from the imaging position of the aerial image 16. The first lens array 8 has an incident surface 30 onto which light rays emitted from the aerial image 16 are incident, and an exit surface 32 (the surface opposite to the incident surface 30) from which the light rays incident on the incident surface 30 exit. In Fig. 1, the incident surface 30 of the first lens array 8 is perpendicular to the plane of the paper on which Fig. 1 is drawn, and is disposed so as to face the exit surface 20 of the aerial image forming optical system 6.

[0056] Regarding the positional relationship between the first lens array 8 and the aerial image 16, the depth position of the aerial image 16 where the first lens array 8 is placed is imaged with the highest spatial resolution. Therefore, the first lens array 8 may be placed at a depth position of the aerial image 16 that is desired to be imaged with the highest spatial resolution. For example, if the aerial image 16 is of a person's head, the first lens array 8 may be placed at a depth position of the person's face in the aerial image 16.

[0057] As shown in FIG. 3, the first lens array 8 has a plurality of first lenses 34 arranged two-dimensionally along the incident surface 30. Note that FIG. 3 illustrates an enlarged portion of the first lens array 8. When the first lens array 8 is viewed from above (i.e., when viewed perpendicularly to the incident surface 30), each of the plurality of first lenses 34 is formed, for example, in a hexagonal shape. That is, the first lens array 8 has a honeycomb structure. By forming each of the plurality of first lenses 34 in a hexagonal shape, the plurality of first lenses 34 can be arranged two-dimensionally without gaps, thereby eliminating non-lens portions in the first lens array 8 and increasing the effective area. Note that each of the plurality of first lenses 34 is a microlens, and the size L of each of the plurality of first lenses 34 is preferably, for example, 300 μm to 2.5 mm, and more preferably 0.5 mm to 1.5 mm.

[0058] 4, when light rays emitted from the aerial image 16 are incident on the entrance surface 30 of the first lens array 8, light rays with a viewing angle θ are acquired by each of the multiple first lenses 34. Of the light emitted from the exit surface 32 of the first lens array 8, the light rays that are captured by the camera 10 are those that travel almost parallel to the camera 10.

[0059] The surface shape of each of the plurality of first lenses 34 may be spherical or aspherical. By making the surface shape of each of the plurality of first lenses 34 aspherical, the influence of spherical aberration on peripheral light rays can be reduced, and light rays can be obtained with uniform precision within the viewing angle θ.

[0060] The curvature of each of the multiple first lenses 34 may be uniform in all directions (i.e., the horizontal curvature and the vertical curvature may be the same). Alternatively, the horizontal curvature and the vertical curvature of each of the multiple first lenses 34 may be different from each other. This makes it possible to control the ray spacing, ray density, viewing angle, and the like that can be obtained by the first lens array 8. For example, by making the horizontal curvature of each of the multiple first lenses 34 larger than the vertical curvature, it is possible to make the viewing angle in the horizontal direction larger than the viewing angle in the vertical direction. In this specification, the "horizontal direction" refers to the direction parallel to the floor surface, and the "vertical direction" refers to the direction perpendicular to the floor surface.

[0061] In this embodiment, the first lens array 8 is configured with a refractive lens, but is not limited to this and may be configured with a diffractive lens. Alternatively, the first lens array 8 may be configured with a lenticular lens. In this case, each of the multiple first lenses 34 is configured with a so-called cylindrical lens that has a curvature in the horizontal direction but not in the vertical direction. This allows light ray information to be acquired only in the horizontal direction, but is effective in obtaining a high-resolution 3D image with only horizontal parallax, in which pixel information can be allocated to resolution characteristics.

[0062] The camera 10 is a digital camera having an imaging lens 36 and an image sensor 38, and is disposed to face the exit surface 32 of the first lens array 8. In the present embodiment, only one camera 10 is disposed. The camera 10 captures light rays emitted from the exit surface 32 of the first lens array 8 (i.e., light rays from the aerial image 16 that have passed through each of the multiple first lenses 34 of the first lens array 8). Specifically, the light rays emitted from the exit surface 32 of the first lens array 8 are imaged on the image sensor 38 by the imaging lens 36, and then captured by the image sensor 38. The camera 10 generates a 3D elemental video signal including light ray information regarding the traveling direction, color, etc. of the light rays emitted from each of the multiple first lenses 34, and outputs the generated 3D elemental video signal to the 3D elemental video display device 12. 4, if the camera 10 is sufficiently far from the first lens array 8, the camera 10 may capture the light beams emitted from the emission surface 32 of the first lens array 8 in a pan-focus state. This makes it possible to generate a 3D elemental video signal with high resolution in the depth range.

[0063] In order to capture the light beams emitted from each of the multiple first lenses 34 at 100 to 400 pixels or more, the camera 10 is preferably configured as an ultra-high-definition digital camera of 8K or higher. In the present embodiment, as shown in FIG. 1 , the camera 10 outputs the 3D elemental video signal to the 3D elemental video display device 12. However, this is not limited to this. The signal may be output to an external terminal device such as a personal computer, or may be recorded on an external video recording device such as a cloud server. When the 3D elemental video signal is output to an external terminal device, the terminal device may perform, for example, calculation of a 3D shape, generation of a 3D model, generation of a multi-viewpoint video, etc. Alternatively, the camera 10 may distribute the 3D elemental video represented by the 3D elemental video signal to a remote location via a network. The 3D elemental video represented by the 3D elemental video signal may be recorded on a recording medium such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD, or a semiconductor memory.

[0064] As shown in FIG. 1, the 3D elemental image display device 12 is formed, for example, with a liquid crystal display or the like, and receives a 3D elemental image signal from the camera 10. A display surface 40 is formed on the front surface of the 3D elemental image display device 12. A 3D elemental image 42 is displayed on the display surface 40 of the 3D elemental image display device 12 based on the 3D elemental image signal from the camera 10. As described above, the 3D elemental image 42 is a two-dimensional image that cannot be viewed stereoscopically by itself. In FIG. 1, the display surface 40 of the 3D elemental image display device 12 is perpendicular to the plane of the paper in FIG. 1 and is disposed facing the observer 50. The camera 10 captures light rays, and the 3D elemental image 42 is displayed by the 3D elemental image display device 12 in real time. As a result, the 3D elemental image 42 becomes a dynamic two-dimensional image synchronized with the movement of the 3D object 4, which is moving in real time.

[0065] The number of first lenses 34 in the first lens array 8 corresponds to the resolution of the 3D elemental images 42. Furthermore, the number of pixels capturing an image within one first lens 34 defines the light density. The viewing angle that can be captured by the camera 10 is determined by the numerical aperture of the first lenses 34. Therefore, it is preferable that the first lens array 8 has a minute and dense lens structure.

[0066] The second lens array 14 is a lens array having substantially the same shape as the first lens array 8. The second lens array 14 is disposed to face the display surface 40 of the 3D elemental image display device 12. The second lens array 14 has an incident surface 44 onto which light rays from the display surface 40 of the 3D elemental image display device 12 are incident, and an exit surface 46 (the surface opposite the incident surface 44) from which the light rays incident on the incident surface 44 exit. The second lens array 14 has a plurality of second lenses 48 arranged two-dimensionally along the incident surface 44, and each of the plurality of second lenses 48 has substantially the same shape as the plurality of first lenses 34 of the first lens array 8. The optical path length between the display surface 40 of the 3D elemental image display device 12 and the second lens array 14 is substantially equal to the focal length of each of the plurality of second lenses 48.

[0067] Light rays from the 3D elemental image 42 displayed on the display surface 40 of the 3D elemental image display device 12 are incident on the incident surface 44 of the second lens array 14. At this time, the light rays from the 3D elemental image 42 are converted by the multiple second lenses 48 of the second lens array 14 into light rays (i.e., light rays emitted from the aerial image 16) that are incident on each of the multiple first lenses 34 of the first lens array 8. This is because the second lens array 14 has substantially the same shape as the first lens array 8. As a result, an observer 50 can view the 3D elemental image 42 displayed on the display surface 40 of the 3D elemental image display device 12 through the second lens array 14, and can thereby stereoscopically view it as a 3D image 52 with the naked eye. In other words, the 3D image 52 is an image that reproduces (reproduces) the light rays emitted from the aerial image 16, and can also be said to be an image representing the 3D object 4. At this time, the viewer 50 can view the three-dimensional image 52 stereoscopically at the viewing angle θ (see FIG. 1) described above.

[0068] The depth of the three-dimensional image 52 is the correct depth as seen by the observer 50. For example, if the face of the person in the three-dimensional object 4 faces the entrance surface 18 of the aerial image forming optical system 6, the face of the person in the three-dimensional image 52 will face the observer 50.

[0069] FIG. 6 is a diagram illustrating an example of a three-dimensional elemental image 42. More specifically, FIG. 6(a) is a diagram illustrating the three-dimensional elemental image 42, and FIG. 6(b) is a diagram illustrating an enlarged portion of the three-dimensional elemental image 42 illustrated in FIG. 6(a). In the example illustrated in FIG. 6, the three-dimensional object represented by the three-dimensional elemental image 42 is a teddy bear. As illustrated in FIG. 6(b), the three-dimensional elemental image 42 includes a plurality of hexagonal elemental images 54 arranged side by side without any gaps. Each of these elemental images 54 is an image obtained by capturing light rays emitted from each of the plurality of first lenses 34 of the first lens array 8. The light rays from each of the plurality of elemental images 54 pass through the plurality of second lenses 48 of the second lens array 14, thereby reproducing the light rays emitted from the aerial image 16.

[0070] [1-2.Effects] The configuration of a video system 100 according to a comparative example will be described with reference to Fig. 7. Fig. 7 is a diagram showing the configuration of a video system 100 according to a comparative example. Note that in Fig. 7, the same components as those in Fig. 1 are assigned the same reference numerals, and their description will be omitted.

[0071] 7, video system 100 includes optical lens 102, first lens array 8, camera 10, three-dimensional elemental image display device 12, and second lens array 14. That is, video system 100 is provided with optical lens 102 instead of aerial image forming optical system 6 of video system 2 described above.

[0072] The optical lens 102 is composed of a convex lens and is disposed at a position that is twice the focal length f away from the three-dimensional object 4. The optical lens 102 forms a real image 104 of the three-dimensional object 4 at a position that is twice the focal length f away. Note that the up-down orientation of the real image 104 is inverted relative to the up-down orientation of the three-dimensional object 4, but the depth of the real image 104 is not inverted relative to the depth of the three-dimensional object 4.

[0073] The first lens array 8 is disposed at the imaging position of the real image 104. Light rays emitted from the real image 104 pass through the first lens array 8. The camera 10 captures the light rays that have passed through the first lens array 8, generates a three-dimensional elemental video signal, and outputs the generated three-dimensional elemental video signal to the three-dimensional elemental video display device 12. As a result, the three-dimensional elemental video represented by the three-dimensional elemental video signal is displayed on the three-dimensional elemental video display device 12. The observer 50 can view the three-dimensional elemental video displayed on the three-dimensional elemental video display device 12 through the second lens array 14, thereby achieving a stereoscopic view as a three-dimensional video 106 with the naked eye.

[0074] However, in the above-described imaging system 100, in order to acquire light ray information over a wide viewing angle, the diameter D of the optical lens 102 needs to be very large (for example, D=approximately 50 cm to 1 m), and it is difficult to manufacture an optical lens 102 of such a size. Also, since the distance L from the three-dimensional object 4 to the camera 10 is approximately several meters, the overall scale of the imaging system 100 becomes large, making it impractical.

[0075] Furthermore, the depth of the 3D image 106 is the opposite depth as seen by the observer 50. For example, if the face of the person in the 3D object 4 faces the optical lens 102, the face of the person in the 3D image 106 will face away from the observer 50. This is because the depth of the real image 104 formed by the optical lens 102 is not inverted relative to the depth of the 3D object 4. Therefore, the observer 50 cannot stereoscopically view the 3D image 106 with the correct depth.

[0076] In contrast, in this embodiment, the aerial image forming optical system 6 having retroreflection properties is used, so even if the size of the aerial image forming optical system 6 is relatively small (for example, about 30 cm × 30 cm), the viewing angle θ of the light rays from the three-dimensional object 4 that enter the aerial image forming optical system 6 can be ensured to be wide, for example, about 30° to 90°. This makes it possible to acquire light ray information over a wide viewing angle θ while keeping the overall size of the video system 2 compact. Furthermore, even when only a single digital camera is used as the camera 10, it is possible to acquire light ray information over a wider viewing angle θ than with a conventional dedicated camera with a built-in image sensor and lens array.

[0077] Furthermore, the aerial image 16 formed by the aerial image forming optical system 6 is free from optical aberration and optical distortion, thereby suppressing image degradation. Furthermore, because the aerial image forming optical system 6 does not have an inherent focal length, the aerial image 16 can be formed at life-size even when the three-dimensional object 4 is located at an arbitrary distance from the aerial image forming optical system 6. Furthermore, because the depth of the aerial image 16 is inverted relative to the depth of the three-dimensional object 4, the depth of the three-dimensional image 52 can be made correct as seen by the observer 50. Furthermore, as shown in (a) and (b) of Figures 5A and 5B, the distribution of light rays emitted in various directions from the three-dimensional image 52 is substantially the same as the distribution of light rays emitted in various directions from the three-dimensional object 4, so the surface texture (e.g., gloss, etc.) of the three-dimensional object 4 can be faithfully reproduced by the three-dimensional image 52.

[0078] [1-3. Various Modifications of the First Lens Array] [1-3-1. Variation 1] The configuration of a first lens array 8A according to Modification 1 of Embodiment 1 will be described with reference to Fig. 8A. Fig. 8A is a diagram showing a first lens array 8A according to Modification 1 of Embodiment 1.

[0079] As shown in Fig. 8A, the first lens array 8A has a plurality of first lenses 34A arranged two-dimensionally. When the first lens array 8A is viewed from above, each of the plurality of first lenses 34A is formed in a circular shape. The plurality of first lenses 34A are arranged in a bale-stacked arrangement. In this case, a non-lens portion 56 is formed between two adjacent first lenses 34A.

[0080] In this modified example, the plurality of first lenses 34A are arranged in a bale-stacked arrangement, but the arrangement is not limited to this, and may be, for example, a square arrangement.

[0081] [1-3-2. Variation 2] The configuration of the first lens array 8B according to the second modification of the first embodiment will be described with reference to Fig. 8B. Fig. 8B is a diagram showing the first lens array 8B according to the second modification of the first embodiment.

[0082] As shown in FIG. 8B, the first lens array 8B has a plurality of first lenses 34A arranged two-dimensionally. The shape and arrangement of the plurality of first lenses 34A are the same as those in the first modification example. In addition, a light-shielding portion 58 is arranged in a non-lens portion 56 formed between two adjacent first lenses 34A. The light-shielding portion 58 is formed by applying a black paint having light-shielding properties, for example. The light-shielding portion 58 can prevent unnecessary leakage light from passing through the non-lens portion 56.

[0083] [1-3-3. Variation 3] The configuration of a first lens array 8C according to Modification 3 of Embodiment 1 will be described with reference to Fig. 8C. Fig. 8C is a diagram showing a first lens array 8C according to Modification 3 of Embodiment 1.

[0084] As shown in Fig. 8C, the first lens array 8C has a plurality of first lenses 34C arranged two-dimensionally. When the first lens array 8C is viewed from above, each of the plurality of first lenses 34C is formed in a square shape (an example of a rectangular shape). The plurality of first lenses 34C are arranged in a square array. This allows light rays to be obtained at the same viewing angle in both the horizontal direction (left-right direction in Fig. 8C) and the vertical direction (up-down direction in Fig. 8C).

[0085] [1-3-4. Variation 4] The configuration of a first lens array 8D according to Modification 4 of Embodiment 1 will be described with reference to Fig. 8D. Fig. 8D is a diagram showing the first lens array 8D according to Modification 4 of Embodiment 1.

[0086] As shown in FIG. 8D, the first lens array 8D has a plurality of first lenses 34D arranged two-dimensionally. When the first lens array 8D is viewed from above, each of the plurality of first lenses 34D is formed in a rectangular shape (an example of a rectangular shape). The long side of each of the plurality of first lenses 34D extends in the horizontal direction (the left-right direction in FIG. 8D) and the short side extends in the vertical direction (the up-down direction in FIG. 8D). The plurality of first lenses 34D are arranged in a square array. This makes it possible to easily control the viewing angle that can be obtained in the horizontal and vertical directions by adjusting the aspect ratio of the first lenses 34D.

[0087] [1-3-5. Variation 5] The configuration of a first lens array 8E according to Modification 5 of Embodiment 1 will be described with reference to Fig. 8E. Fig. 8E is a diagram showing a first lens array 8E according to Modification 5 of Embodiment 1.

[0088] As shown in FIG. 8E, the first lens array 8E has a plurality of first lenses 34E arranged two-dimensionally. When the first lens array 8E is viewed from above, each of the plurality of first lenses 34E is formed in a rectangular shape (an example of a rectangular shape). Furthermore, the long side of each of the plurality of first lenses 34E extends in the horizontal direction (the left-right direction in FIG. 8E), and the short side extends in the vertical direction (the up-down direction in FIG. 8E). Furthermore, the plurality of first lenses 34E are arranged in a staggered pattern. This can improve the resolution characteristics in the horizontal direction.

[0089] (Embodiment 2) The configuration of a video system 2F according to embodiment 2 will be described with reference to Fig. 9. Fig. 9 is a diagram showing a part of the configuration of a video system 2F according to embodiment 2. Note that in each embodiment shown below, the same components as those in embodiment 1 above are denoted by the same reference numerals, and their description will be omitted.

[0090] As shown in FIG. 9, video system 2F includes condenser lens 60 in addition to the components described in the first embodiment. Collector lens 60 is a convex lens, and is disposed opposite exit surface 32 of first lens array 8. Camera 10 is disposed at a position spaced a focal length f from collecting lens 60. Collector lens 60 collects, at the position of camera 10, substantially parallel light rays that are emitted from exit surface 32 of first lens array 8 and are directed toward camera 10. This allows camera 10 to be located closer to first lens array 8 than in the first embodiment, making it possible to make the overall size of video system 2F more compact.

[0091] (Embodiment 3) The configuration of a video system 2G according to the third embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing a part of the configuration of a video system 2G according to the third embodiment.

[0092] 10, the video system 2G includes an aperture array 62 in addition to the components described in the first embodiment. The aperture array 62 is arranged opposite the incident surface 30 of the first lens array 8. That is, the aperture array 62 is arranged between the aerial image forming optical system 6 (see FIG. 1) and the first lens array 8. The aperture array 62 has a plurality of apertures 64. The plurality of apertures 64 are arranged opposite the plurality of first lenses 34 of the first lens array 8, respectively, and are arranged at the focal length positions of the plurality of first lenses 34.

[0093] As a result, by cutting out unnecessary light with the aperture array 62, it is possible to prevent the unnecessary light from entering the first lens array 8. As a result, it is possible to prevent the image from becoming blurred in the 3D elemental images 42. Furthermore, by adjusting the size of each of the plurality of apertures 64, it is possible to obtain light ray information with a wide depth reproduction range.

[0094] (Fourth embodiment) The configuration of a video system 2H ​​according to the fourth embodiment will be described with reference to Fig. 11. Fig. 11 is a diagram showing a part of the configuration of a video system 2H ​​according to the fourth embodiment.

[0095] 11, video system 2H ​​includes a plurality of cameras 10. Each of the plurality of cameras 10 is configured similarly to the single camera 10 described in the first embodiment. That is, each of the plurality of cameras 10 captures an image of light emitted from exit surface 32 of first lens array 8, generates a three-dimensional elemental video signal, and outputs the generated three-dimensional elemental video signal to a three-dimensional elemental video display device (not shown).

[0096] This allows the multiple cameras 10 to easily obtain more light ray information over a wide viewing angle, thereby improving the reproducibility of the three-dimensional image 52.

[0097] (Embodiment 5) The configuration of a video system 2J according to the fifth embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing the configuration of a video system 2J according to the fifth embodiment.

[0098] 12, the three-dimensional display device 15J of the video system 2J includes a detection unit 66 and a display control unit 68 in addition to the three-dimensional elemental image display device 12 and the second lens array 14 described in the first embodiment. The second lens array 14 is detachable from the display surface 40 of the three-dimensional elemental image display device 12.

[0099] The detection unit 66 is a sensor for detecting attachment / detachment of the second lens array 14 to / from the display surface 40 of the three-dimensional elemental image display device 12. The detection unit 66 outputs the detection result to the display control unit 68.

[0100] The display control unit 68 controls the display content on the display surface 40 of the three-dimensional elemental image display device 12 based on the detection result of the detection unit 66.

[0101] Specifically, when the second lens array 14 is attached to the display surface 40 of the three-dimensional elemental image display device 12, the display control unit 68 causes the three-dimensional elemental images to be displayed on the display surface 40. This allows the observer 50 to view the three-dimensional elemental images displayed on the display surface 40 of the three-dimensional elemental image display device 12 through the second lens array 14, thereby achieving a stereoscopic view of the images as a three-dimensional image 52 with the naked eye.

[0102] On the other hand, when the second lens array 14 is detached from the display surface 40 of the 3D elemental image display device 12, the display control unit 68 displays images other than the 3D elemental images on the display surface 40. Note that the "images other than the 3D elemental images" are 2D images that cannot be viewed stereoscopically by themselves and cannot be viewed stereoscopically even when the second lens array 14 is used, such as images produced by playing back normal video content. This allows the viewer 50 to view the other images displayed on the display surface 40 of the 3D elemental image display device 12 without passing through the second lens array 14, and thus to recognize them as 2D images.

[0103] The display control unit 68 may be disposed inside the three-dimensional elemental image display device 12 or may be disposed outside the three-dimensional elemental image display device 12.

[0104] Therefore, in this embodiment, the display content on the display surface 40 of the three-dimensional elemental image display device 12 can be appropriately changed in conjunction with the attachment or detachment of the second lens array 14 to or from the display surface 40 of the three-dimensional elemental image display device 12.

[0105] (Embodiment 6) The configuration of a video system 2K according to the sixth embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing the configuration of a video system 2K according to the sixth embodiment.

[0106] As shown in FIG. 13, in a three-dimensional display device 15K of a video system 2K, a second lens array 14K has a polarization direction switching element 70 and a polarization-dependent lens array 72.

[0107] The three-dimensional elemental image display device 12 is a liquid crystal display that displays light polarized in one direction. Alternatively, if the three-dimensional elemental image display device 12 is an EL (Electro Luminescence) display or LED (Light Emitting Diode) display that does not display light polarized in one direction, it is assumed to be equipped with an optical element (polarizing plate) that aligns the polarization in one direction.

[0108] The polarization direction switching element 70 is disposed so as to face the display surface 40 of the three-dimensional elemental image display device 12. As a result, light rays emitted from the display surface 40 of the three-dimensional elemental image display device 12 pass through the polarization direction switching element 70. The polarization direction switching element 70 controls the polarization direction of the light rays emitted from the display surface 40 of the three-dimensional elemental image display device 12 (i.e., the light rays that have passed through the polarization direction switching element 70) in accordance with the display content on the display surface 40 of the three-dimensional elemental image display device 12.

[0109] Specifically, when a three-dimensional elemental image is displayed on the display surface 40 of the three-dimensional elemental image display device 12, the polarization direction switching element 70 polarizes the light beam emitted from the display surface 40 in a first polarization direction. On the other hand, when an image other than a three-dimensional elemental image is displayed on the display surface 40 of the three-dimensional elemental image display device 12, the polarization direction switching element 70 polarizes the light beam emitted from the display surface 40 in a second polarization direction that is orthogonal to the first polarization direction (i.e., different from the first polarization direction).

[0110] The polarization-dependent lens array 72 is configured, for example, by a liquid crystal lens, and is arranged to face the exit surface (i.e., the surface on the side farther from the 3D elemental image display device 12) of the polarization direction switching element 70. As a result, light rays having the first polarization direction or the second polarization direction that have passed through the polarization direction switching element 70 are incident on the polarization-dependent lens array 72.

[0111] When light rays in a first polarization direction are incident on the polarization-dependent lens array 72, the light rays in the first polarization direction are converted by the polarization-dependent lens array 72 into light rays that are incident on the first lens array 8. That is, in this case, the polarization-dependent lens array 72 functions in the same manner as the second lens array 14 described in the first embodiment above. As a result, the observer 50 can view the 3D elemental images displayed on the display surface 40 of the 3D elemental image display device 12 through the second lens array 14K, and thereby see the 3D image 52 stereoscopically with the naked eye.

[0112] On the other hand, when light rays having a second polarization direction are incident on the polarization-dependent lens array 72, the light rays having the second polarization direction are transmitted through the polarization-dependent lens array 72 as they are. That is, in this case, the polarization-dependent lens array 72 functions similarly to, for example, a transparent glass plate having no lens function. As a result, the viewer 50 can view an image other than the 3D elemental image displayed on the display surface 40 of the 3D elemental image display device 12 through the second lens array 14K, and thereby visually recognize the image as a 2D image.

[0113] Therefore, in this embodiment, with the second lens array 14K still attached to the display surface 40 of the three-dimensional elemental image display device 12, the observer 50 can stereoscopically view the three-dimensional elemental images as three-dimensional images 52, or view images other than the three-dimensional elemental images as two-dimensional images, depending on the display content on the display surface 40 of the three-dimensional elemental image display device 12.

[0114] (Embodiment 7) The configuration of a video system 2L according to the seventh embodiment will be described with reference to Fig. 14 and Fig. 15. Fig. 14 is a block diagram showing the functional configuration of the video system 2L according to the seventh embodiment. Fig. 15 is a diagram for explaining the operation of a processing unit 74 according to the seventh embodiment.

[0115] 14, the video system 2L includes, as its functional configuration, a processing unit 74 for performing predetermined processing on the 3D elemental video signal output from the camera 10. The processing unit 74 includes a conversion unit 76, an encoding unit 78, and a decoding unit 80.

[0116] As shown in (a) of FIG. 15, the converter 76 converts the 3D elemental video signal output from the camera 10 into a signal of a multi-viewpoint video group including a plurality of multi-viewpoint videos. Here, the viewing zone angle θ of the multi-viewpoint video is not less than 30° and not more than 90°. Furthermore, the viewing zone angle interval between the plurality of elemental videos included in the multi-viewpoint video is preferably not less than 0.5° and not more than 2°, and more preferably about 1°. Alternatively, the converter 76 may convert the 3D elemental video signal output from the camera 10 into a signal of a multi-viewpoint video group with a depth image.

[0117] The encoding unit 78 encodes the signal of the multi-view video group converted by the conversion unit 76 using a predetermined encoding method such as H.264 / MVC (Multiview Video Coding). Alternatively, the encoding unit 78 encodes the signal of the multi-view video group with depth images converted by the conversion unit 76 using a predetermined encoding method such as 3D HEVC (High Efficiency Video Coding). The encoding unit 78 transmits the encoded signal to the decoding unit 80.

[0118] The decoding unit 80 generates a 3D element video signal by decoding the signal encoded by the encoding unit 78 using a predetermined decoding method. As shown in FIG. 15(b), among the multiple element videos included in the 3D element video represented by the 3D element video signal, one element video corresponding to one second lens 48 of the second lens array 14 (see FIG. 1) includes a multi-view video. That is, the number of viewpoints of the multi-view video is equal to the number of pixels of one element video of the 3D element video, and the number of pixels of the multi-view video group is equal to the number of element videos of the 3D element video (the number of second lenses 48). Note that there is a one-to-one correspondence between the element videos of the 3D element video and the second lenses 48 of the second lens array 14. The decoding unit 80 outputs the generated 3D element video signal to the 3D element video display device 12.

[0119] Therefore, in this embodiment, the 3D elemental video signal can be easily transmitted.

[0120] (Other variations, etc.) Although the video systems according to the first to seventh embodiments of the present invention have been described above, the present invention is not limited to these embodiments. For example, the above embodiments may be combined with each other.

[0121] In each of the above embodiments, the three-dimensional elemental image display device 12 is configured as a liquid crystal display, but this is not limited to this, and the three-dimensional elemental image display device 12 may be configured as, for example, a head-mounted display, an eyeglass-type display, or a goggle-type display. [Industrial Applicability]

[0122] The present invention can be applied, for example, to an imaging system for acquiring three-dimensional elemental image signals using light field technology. [Explanation of symbols]

[0123] 2, 2F, 2G, 2H, 2J, 2K, 2L, 100 Video System 4 3D object 6 Aerial image forming optical system 8, 8A, 8B, 8C, 8D, 8E First lens array 10 Camera 12 3D elemental image display device 14,14K Second Lens Array 15,15J,15K 3D display device 16 Aerial image 18,30,44 Entrance plane 20, 32, 46 Exit surface 22 Base 22a First principal surface 22b Second main surface 24 Two-sided corner reflector 24a,24b side 24c top 24d bottom 26a First Reflecting Surface 26b Second Reflecting Surface 28 Arrow 34, 34A, 34C, 34D, 34E First lens 36 Imaging lens 38 Image sensor 40 Display surface 42 3D elemental images 48 Second Lens 50 Observer 52,106 3D images 54 element images 56 Non-lens part 58 Light blocking section 60 Condenser Lens 62 aperture array 64 Aperture 66 Detector 68 Display control unit 70 Polarization direction switching element 72 Polarization-dependent lens array 74 Processing section 76 Conversion unit 78 Encoding section 80 Decoding section 102 Optical Lenses 104 Real Image

Claims

1. an aerial image forming optical system having retroreflection characteristics, which forms an aerial image representing a three-dimensional object by imaging light rays from the three-dimensional object in the air; and a first lens array that is arranged at or near the imaging position of the aerial image and has a plurality of first lenses that are arranged two-dimensionally; a camera having an imaging lens and an image sensor, which captures light rays from the aerial image that have passed through the first lens array, and outputs a three-dimensional elemental video signal including light ray information related to the light rays from the aerial image. Video system.

2. The camera outputs the three-dimensional elemental image signal to (i) an external terminal device, (ii) an external image recording device, or (iii) a three-dimensional elemental image display device. The video system according to claim 1 .

3. The aerial image forming optical system forms the aerial image at the imaging position symmetrical to the three-dimensional object with respect to the aerial image forming optical system by reflecting light rays from the three-dimensional object at least twice. The video system according to claim 1 .

4. When the first lens array is viewed from above, each of the first lenses is formed in a circular, rectangular, or hexagonal shape. The video system according to claim 1 .

5. The surface shape of each of the plurality of first lenses is spherical or aspherical. The video system according to claim 1 .

6. Each of the plurality of first lenses is configured so that the curvature in the horizontal direction is the same as the curvature in the vertical direction, or so that the curvature in the horizontal direction is larger than the curvature in the vertical direction. The video system according to claim 1 .

7. The video system further comprises: an aperture array disposed between the aerial image forming optical system and the first lens array, the aperture array having a plurality of apertures disposed at positions corresponding to the focal lengths of the plurality of first lenses, respectively; The video system according to claim 1 .

8. The camera may include a single digital camera or multiple digital cameras. The video system according to claim 1 .

9. The video system further comprises: a three-dimensional elemental image display device having a display surface that receives the three-dimensional elemental image signal output from the camera and displays a three-dimensional elemental image based on the three-dimensional elemental image signal; a second lens array having substantially the same shape as the first lens array, arranged to face the display surface of the three-dimensional elemental image display device, and having a plurality of second lenses arranged two-dimensionally; The optical path length between the display surface of the three-dimensional elemental image display device and the second lens array is approximately equal to the focal length of each of the plurality of second lenses. The video system according to any one of claims 1 to 8.

10. the second lens array is detachable from the display surface of the three-dimensional elemental image display device; The video system further comprises: a detection unit that detects attachment / detachment of the second lens array to / from the display surface of the three-dimensional elemental image display device; a display control unit that controls display content on the display surface of the three-dimensional elemental image display device based on the detection result of the detection unit, The display control unit (i) displays the three-dimensional elemental images on the display surface when the second lens array is attached to the display surface, and (ii) displays an image other than the three-dimensional elemental images on the display surface when the second lens array is detached from the display surface.

10. The video system according to claim 9.

11. forming an aerial image representing a three-dimensional object by focusing light rays from the three-dimensional object in the air using an aerial image forming optical system having retroreflection characteristics; transmitting light rays from the aerial image through a first lens array arranged at or near an imaging position of the aerial image, the first lens array having a plurality of first lenses arranged two-dimensionally; A camera having an imaging lens and an image pickup element captures light rays from the aerial image that have passed through the first lens array, thereby outputting a three-dimensional elemental video signal including light ray information regarding the light rays from the aerial image. 3D element image acquisition method.

12. 2. A method for transmitting the three-dimensional elemental video signal output from the camera in the video system according to claim 1, comprising: (a) converting the three-dimensional elemental image signals output from the cameras into signals of a multi-viewpoint image group or signals of a multi-viewpoint image group with depth images; (b) encoding the signal converted in (a) using a predetermined encoding method; (c) transmitting the signal encoded in (b); (d) decoding the signal transmitted in (c) using a predetermined decoding method; The viewing angle of each of the plurality of multi-viewpoint images included in the multi-viewpoint image group is 30° or more and 90° or less, and the viewing angle interval of the plurality of elemental images included in the multi-viewpoint image is 0.5° or more and 2° or less. A method for transmitting three-dimensional elemental video signals.

Images