Video system, 3D elemental video acquisition method, recording medium, and 3D elemental video signal transmission method

Abstract

A video system (2) comprises: an aerial image formation optical system (6) having recursive reflection characteristics and forming an aerial image (16), representing a three-dimensional object (4), by causing light rays from the three-dimensional object (4) to form an image in the air; a first lens array (8) that is disposed at or near the position where the aerial image (16) is formed, and that has a plurality of first lenses (34) arranged two-dimensionally; and a camera (10) that has an image-forming lens (36) and an imaging element (38), and that images light rays from the aerial image (16) which have passed through the first lens array (8) so as to output a three-dimensional element video signal which includes light ray information related to the light rays from the aerial image (16).

Description

Image system, 3D elemental image acquisition method, recording medium, second lens array, 3D display device, and 3D elemental image signal transmission method

[0001] The present invention relates to a video system, a three-dimensional elemental video acquisition method, a recording medium, a second lens array, a three-dimensional display device, and a three-dimensional elemental video signal transmission method.

[0002] Light field technology is a technology that acquires, processes, and reproduces light ray information related to light rays emitted from a three-dimensional object. This light field technology can display three-dimensional images that can be viewed stereoscopically with the naked eye based on the acquired light ray information, and by reproducing the light rays emitted from the three-dimensional object in this three-dimensional image, it is possible to realistically reproduce the shape and texture of the surface of the three-dimensional object.

[0003] As a first example of light field technology, a technology is known in which an optical lens is used to form a real image of a three-dimensional object to be imaged, and light rays that pass through a lens array placed at the image-forming position are captured by a camera (see, for example, Non-Patent Document 1).

[0004] As a second example of light field technology, a technology is known in which an image of a three-dimensional object to be imaged is captured using a dedicated camera equipped with an image sensor and a lens array arranged near the image sensor (see, for example, Non-Patent Document 2).

[0005] As a third example of the light field technology, a multi-camera technology is known that uses 100 or more cameras to capture an image of a three-dimensional object (see, for example, Non-Patent Document 3).

[0006] J. Arai, et.al., “Integral three-dimensional television with a video system using pixel-offset method,” Optics Express, Vol. 21, No. 3, pp. 3474-3485 (2013); Ng R., et.al., “Light Field Photography with a Hand-held Plenoptic Camera,” Stanford University Computer Science Tech Report, pp. 1-11 (2005); H. Watanabe, et.al., “Aktina Vision: Full-parallax three-dimensional display with 100 million light rays,” Scientific Reports, Vol. 9, Article number: 17688 (2019); Parity Mirror (registered trademark), [Retrieved June 29, 2022], Internet (https: / / evort.jp / store / piq / product / paritymirror); Yamamoto Yutaka, “Aerial display using aerial imaging by retroreflection (AIRR),” Journal of the Imaging Society of Japan, Vol. 56, No. 4 pp.341-351 (2017) Tetsuro Kuwayama, "Aerial Display Technology 3D Images for a Contactless Society," Vol.33, No.3, pp.9-20 (May 2021)

[0007] The technology disclosed in Non-Patent Document 1 requires an optical lens with a very large diameter (approximately 50 cm to 1 m) to acquire light ray information over a wide viewing angle, but it is difficult to fabricate an optical lens of such a size. In addition, the distance from the three-dimensional object to be imaged to the camera is approximately several meters, which makes the overall system large and impractical.

[0008] Furthermore, in the technology disclosed in Non-Patent Document 2, the viewing angle of light rays from a three-dimensional object that can be captured by a camera is narrow (approximately 1°), and the amount of light ray information obtained is extremely small, resulting in low reproducibility of the three-dimensional image.

[0009] Furthermore, the technology disclosed in Non-Patent Document 3 uses 100 or more cameras, which increases costs and the overall system size, resulting in a lack of versatility. Furthermore, due to limitations on the size of the cameras, the cameras cannot be spaced closely together. Therefore, in order to obtain detailed light ray information, a separate process is required to generate light ray information by interpolation from the images captured by the cameras, which makes the process complicated. Moreover, the large number of video signals captured simultaneously by 100 or more cameras requires a long period of computational processing, which requires a huge amount of time and effort.

[0010] The present invention seeks to solve the above-mentioned problems, and its object is to provide an imaging system, a 3D elemental image acquisition method, a recording medium, a second lens array, a 3D display device, and a 3D elemental image signal transmission method that can acquire light ray information over a wide viewing angle while keeping the overall system size compact.

[0011] In order to achieve the above-mentioned object, the video system of a first aspect of the present invention comprises an aerial image forming optical system having retroreflective properties, which forms an aerial image representing a three-dimensional object by imaging light rays from the three-dimensional object in the air; a first lens array arranged at or near the imaging position of the aerial image and having a plurality of first lenses arranged two-dimensionally; and a camera having an imaging lens and an image sensor, which images light rays from the aerial image that have passed through the first lens array, and outputs a three-dimensional element video signal including light ray information regarding the light rays from the aerial image.

[0012] According to this aspect, since an aerial image forming optical system having retroreflection properties is used, even if the size of the aerial image forming optical system is relatively small, the aerial image forming optical system can acquire light rays from a three-dimensional object with a wide viewing angle. This makes it possible to acquire light ray information over a wide viewing angle while keeping the overall system size compact.

[0013] For example, in the video system according to the second aspect of the present invention, in the first aspect, the camera may be configured to (i) output the three-dimensional element video signal to an external terminal device, (ii) record it in an external video recording device, or (iii) output it to a three-dimensional element video display device.

[0014] According to this aspect, the three-dimensional elemental image signal output from the camera can be appropriately processed.

[0015] For example, in the video system according to the third aspect of the present invention, in the first or second aspect, the aerial image forming optical system may be configured to form the aerial image at the imaging position symmetrical to the three-dimensional object relative to the aerial image forming optical system by reflecting light rays from the three-dimensional object at least twice.

[0016] According to this aspect, an aerial image can be easily formed by the aerial image forming optical system.

[0017] For example, in the video system according to the fourth aspect of the present invention, in any one of the first to third aspects, when the first lens array is viewed in a plane, each of the plurality of first lenses may be configured to be formed in a circular, rectangular or hexagonal shape.

[0018] According to this aspect, it is possible to use a first lens array having optical characteristics appropriate for the intended use.

[0019] For example, in the video system according to the fifth aspect of the present invention, in any one of the first to fourth aspects, the surface shape of each of the plurality of first lenses may be configured to be spherical or aspherical.

[0020] According to this aspect, it is possible to use a first lens array having optical characteristics appropriate for the intended use.

[0021] For example, in the video system according to the sixth aspect of the present invention, in any one of the first to fifth aspects, each of the plurality of first lenses may be configured so that its curvature in the horizontal direction is the same as its curvature in the vertical direction, or so that its curvature in the horizontal direction is greater than its curvature in the vertical direction.

[0022] According to this aspect, it is possible to use a first lens array having optical characteristics appropriate for the intended use.

[0023] For example, in the video system according to the seventh aspect of the present invention, in any one of the first to sixth aspects, the video system may further be configured to include an aperture array arranged between the aerial image forming optical system and the first lens array, the aperture array having a plurality of apertures each arranged at the focal length position of each of the plurality of first lenses.

[0024] According to this aspect, by cutting out unnecessary light with the aperture array, it is possible to prevent the unnecessary light from entering the first lens array. Furthermore, by adjusting the size of each of the multiple apertures, it is possible to obtain light ray information with a wide depth reproduction range.

[0025] For example, in the video system according to the eighth aspect of the present invention, in any one of the first to seventh aspects, the camera may be configured to include a single digital camera or multiple digital cameras.

[0026] According to this aspect, when the camera includes a single digital camera, the overall system size can be kept compact, and when the camera includes multiple digital cameras, more light ray information can be acquired over a wide viewing angle.

[0027] For example, in a video system according to a ninth aspect of the present invention, in any one of the first to eighth aspects, the video system further comprises a three-dimensional element image display device having a display surface that receives the three-dimensional element image signal output from the camera and displays a three-dimensional element image based on the three-dimensional element image signal, and a second lens array having substantially the same shape as the first lens array, arranged opposite the display surface of the three-dimensional element image display device, and having a plurality of second lenses arranged two-dimensionally, and the optical path length between the display surface of the three-dimensional element image display device and the second lens array may be configured to be substantially equal to the focal length of each of the plurality of second lenses.

[0028] According to this aspect, an observer can view the three-dimensional elemental images displayed on the display surface of the three-dimensional elemental image display device through the second lens array, thereby viewing the three-dimensional elemental images as three-dimensional images.

[0029] A three-dimensional elemental image acquisition method according to a tenth aspect of the present invention forms an aerial image representing a three-dimensional object by using an aerial image forming optical system having retroreflection properties to focus light rays from the three-dimensional object in the air, transmits light rays from the aerial image through a first lens array arranged at or near the imaging position of the aerial image, the first lens array having a plurality of first lenses arranged two-dimensionally, and uses a camera having an imaging lens and an image sensor to image the light rays from the aerial image that have passed through the first lens array, thereby outputting a three-dimensional elemental image signal including light ray information regarding the light rays from the aerial image.

[0030] According to this aspect, it is possible to obtain light ray information over a wide viewing angle while keeping the overall system size compact.

[0031] A recording medium according to an eleventh aspect of the present invention is a recording medium on which three-dimensional elemental images represented by three-dimensional elemental image signals output from a video system according to any one of the first to ninth aspects are recorded.

[0032] A second lens array according to a twelfth aspect of the present invention is the second lens array in the video system according to the ninth aspect.

[0033] A three-dimensional display device according to a thirteenth aspect of the present invention includes the three-dimensional elemental image display device and the second lens array in the image system according to the ninth aspect.

[0034] For example, in a three-dimensional display device according to a fourteenth aspect of the present invention, in the thirteenth aspect, the second lens array is detachable from the display surface of the three-dimensional elemental image display device, and the three-dimensional display device further comprises a detection unit that detects attachment / detachment of the second lens array to / from the display surface of the three-dimensional elemental image display device, and a display control unit that controls the display content on the display surface of the three-dimensional elemental image display device based on the detection result of the detection unit, and the display control unit may be configured to (i) display the three-dimensional elemental image on the display surface when the second lens array is attached to the display surface, and (ii) display an image other than the three-dimensional elemental image on the display surface when the second lens array is detached from the display surface.

[0035] According to this aspect, the display content on the display surface of the three-dimensional elemental image display device can be appropriately changed in conjunction with the attachment / detachment of the second lens array to / from the display surface of the three-dimensional elemental image display device.

[0036] For example, in the three-dimensional display device according to the fifteenth aspect of the present invention, in the thirteenth aspect, the second lens array may be configured to include a polarization direction switching element through which light rays emitted from the display surface of the three-dimensional elemental image display device pass, the polarization direction switching element (i) polarizing the light rays emitted from the display surface in a first polarization direction when the three-dimensional elemental image is displayed on the display surface, and (ii) polarizing the light rays emitted from the display surface in a second polarization direction different from the first polarization direction when an image other than the three-dimensional elemental image is displayed on the display surface, and a polarization-dependent lens array onto which light rays in the first polarization direction or the second polarization direction that have passed through the polarization direction switching element are incident.

[0037] According to this aspect, while the second lens array is attached to the display surface of the three-dimensional element image display device, the observer can view the three-dimensional element image as a three-dimensional image in stereoscopic form, or view images other than the three-dimensional element image as a two-dimensional image, depending on the display content on the display surface of the three-dimensional element image display device.

[0038] A method for transmitting a three-dimensional element video signal according to a sixteenth aspect of the present invention is a method for transmitting the three-dimensional element video signal output from the camera in the video system described in claim 1, and includes the steps of: (a) converting the three-dimensional element video signal output from the camera into a signal of a multi-view video group or a signal of a multi-view video group with a depth image; (b) encoding the signal converted in (a) using a predetermined encoding method; (c) transmitting the signal encoded in (b); and (d) decoding the signal transmitted in (c) using a predetermined decoding method, wherein the viewing angle of each of the multiple multi-view videos included in the multi-view video group is between 30° and 90°, and the viewing angle interval of the multiple element videos included in the multi-view video is between 0.5° and 2°.

[0039] According to this aspect, the three-dimensional elemental video signal can be easily transmitted.

[0040] According to an image system or the like according to one aspect of the present invention, it is possible to obtain light ray information over a wide viewing angle while keeping the overall system size compact.

[0041] 1 is a diagram showing the configuration of a video system according to a first embodiment. 2 is a diagram showing an aerial image forming optical system of the video system according to the first embodiment. 3 is a diagram showing a first lens array of the video system according to the first embodiment. 4 is a diagram showing optical paths of light rays passing through the first lens array of the video system according to the first embodiment. 5 is a diagram showing a 3D elemental image display device of the video system according to the first embodiment. 6 is a diagram showing an example of a 3D elemental image. 7 is a diagram showing the configuration of a video system according to a comparative example. 8 is a diagram showing a first lens array according to a first modification of the first embodiment. 9 is a diagram showing a first lens array according to a second modification of the first embodiment. 10 is a diagram showing a first lens array according to a third modification of the first embodiment. 11 is a diagram showing a first lens array according to a fifth modification of the first embodiment. 12 is a diagram showing a part of the configuration of a video system according to the second embodiment. 13 is a diagram showing a part of the configuration of a video system according to the third embodiment. 14 is a diagram showing a part of the configuration of a video system according to the fourth embodiment. 15 is a diagram showing a configuration of a video system according to the fifth embodiment. 16 is a diagram showing a configuration of a video system according to the sixth embodiment. 17 is a block diagram showing the functional configuration of a video system according to the seventh embodiment. 18 is a diagram for explaining the operation of a processing unit according to the seventh embodiment.

[0042] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement positions, and connection forms shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0043] (Embodiment 1) [1-1. Configuration of Video System] First, the configuration of a video system 2 according to embodiment 1 will be described with reference to Figs. 1 to 5. Fig. 1 is a diagram showing the configuration of a video system 2 according to embodiment 1. Fig. 2 is a diagram showing an aerial image forming optical system 6 of the video system 2 according to embodiment 1. Fig. 3 is a diagram showing a first lens array 8 of the video system 2 according to embodiment 1. Fig. 4 is a diagram showing the optical paths of light rays passing through the first lens array 8 of the video system 2 according to embodiment 1. Fig. 5 is a diagram showing a three-dimensional elemental image display device 12 of the video system 2 according to embodiment 1.

[0044] The video system 2 is a system for using light field technology to acquire three-dimensional elemental video signals based on light rays (a group of light rays) emitted from a three-dimensional object 4, and for displaying three-dimensional elemental videos representing the three-dimensional object 4 based on the acquired three-dimensional elemental video signals. In this embodiment, the three-dimensional object 4 is, for example, a human head.

[0045] In this specification, the term "three-dimensional elemental image" refers to an image that cannot be viewed stereoscopically because it is a two-dimensional image, but can be viewed stereoscopically as a three-dimensional image with the naked eye by viewing the three-dimensional elemental image through a lens array. Furthermore, in this specification, the term "image" is a concept that includes not only moving images but also images (still images).

[0046] 1, the video system 2 includes an aerial image forming optical system 6, a first lens array 8, a camera 10, a three-dimensional elemental image display device 12, and a second lens array 14. The three-dimensional elemental image display device 12 and the second lens array 14 are combined to form a three-dimensional display device 15. This video system 2 is used, for example, in (a) business applications such as broadcasting, industrial design, showrooms, digital signage, and communication between remote locations, (b) amusement applications such as three-dimensional image theaters and public viewings, and (c) art, science, and education applications such as digital art, 3D simulators, digital museums, and digital archives.

[0047] The aerial image forming optical system 6 is a panel-shaped optical device having retroreflection properties, and forms an aerial image 16 representing the three-dimensional object 4 by focusing light rays emitted from the three-dimensional object 4 in the air. The vertical size of the aerial image forming optical system 6 (the size in the up-down direction in FIG. 1 ) is, for example, approximately 30 cm, and the horizontal size (the size in the direction perpendicular to the paper surface in FIG. 1 ) is, for example, approximately 30 cm. The aerial image forming optical system 6 has an incident surface 18 onto which light rays emitted from the three-dimensional object 4 are incident, and an exit surface 20 (the surface opposite to the incident surface 18) from which light rays for forming the aerial image 16 exit. In FIG. 1 , the incident surface 18 of the aerial image forming optical system 6 is perpendicular to the paper surface in FIG. 1 and is positioned so as to face the three-dimensional object 4.

[0048] Of the light rays emitted from the three-dimensional object 4, those within an angular range corresponding to the viewing angle θ (e.g., 30° to 90°) are incident on the incident surface 18. The "viewing angle" refers to the angular range in which an observer can view a three-dimensional image. Here, since the aerial image forming optical system 6 does not have a center, if the size of the three-dimensional object 4 is relatively large (e.g., if the three-dimensional object 4 is the entire body of a person), an aerial image 16 of a size corresponding to the size of the three-dimensional object 4 can be formed by arranging (tiling) multiple aerial image forming optical systems 6 in a two-dimensional manner.

[0049] The aerial image 16 is formed at an imaging position that is plane-symmetrical to the three-dimensional object 4 with respect to the aerial image forming optical system 6. The depth of the aerial image 16 is inverted with respect to the depth of the three-dimensional object 4. For example, if the face of the person in the three-dimensional object 4 faces the entrance surface 18 of the aerial image forming optical system 6, the face of the person in the aerial image 16 will face the exit surface 20 of the aerial image forming optical system 6.

[0050] The aerial image forming optical system 6 is configured with, for example, a dihedral corner reflector array as shown in Fig. 2A. As shown in Fig. 2A, the aerial image forming optical system 6 has a base portion 22 and a plurality of dihedral corner reflectors 24.

[0051] The base portion 22 is formed in a flat plate shape. The base portion 22 has a first principal surface 22 a and a second principal surface 22 b disposed on the opposite side of the first principal surface 22 a. The second principal surface 22 b of the base portion 22 functions as the exit surface 20 of the aerial image forming optical system 6.

[0052] The dihedral corner reflectors 24 are arranged in an array on the first main surface 22a of the base portion 22. Each of the dihedral corner reflectors 24 is a micromirror formed in a rectangular parallelepiped shape. The size of one side of the dihedral corner reflector 24 is, for example, 0.1 mm to 1 mm. As shown in FIG. 2B , a first reflecting surface 26a and a second reflecting surface 26b are formed on two adjacent side surfaces 24a and 24b of each of the dihedral corner reflectors 24, respectively. The top surface 24c of each of the dihedral corner reflectors 24 functions as the incident surface 18 of the aerial image forming optical system 6.

[0053] As shown by arrow 28 in Figure 2 (b), of the light rays incident on the top surface 24c of the dihedral corner reflector 24, the light rays are reflected twice (total reflection), once by the first reflecting surface 26a and once by the second reflecting surface 26b, and then exit from the bottom surface 24d of the dihedral corner reflector 24, contributing to the formation of the aerial image 16.

[0054] In this embodiment, the aerial image forming optical system 6 is configured with a dihedral corner reflector array, but is not limited to this and may be configured with, for example, a looper-shaped mirror with an orthogonal arrangement structure (see, for example, Non-Patent Document 4), or an optical system combining a retroreflective sheet and a half mirror (see, for example, Non-Patent Document 5). As the retroreflective sheet described above, for example, a bead array or a corner cube mirror array can be used (see, for example, Non-Patent Document 6).

[0055] As shown in Fig. 1, the first lens array 8 is disposed at (or near) the imaging position of the aerial image 16. Here, "near the aerial image 16" means within a range of several centimeters from the imaging position of the aerial image 16. The first lens array 8 has an incident surface 30 onto which light rays emitted from the aerial image 16 are incident, and an exit surface 32 (the surface opposite to the incident surface 30) from which the light rays incident on the incident surface 30 exit. In Fig. 1, the incident surface 30 of the first lens array 8 is perpendicular to the plane of the paper on which Fig. 1 is drawn, and is disposed so as to face the exit surface 20 of the aerial image forming optical system 6.

[0056] Regarding the positional relationship between the first lens array 8 and the aerial image 16, the depth position of the aerial image 16 where the first lens array 8 is disposed is captured with the highest spatial resolution. Therefore, the first lens array 8 may be disposed at a depth position of the aerial image 16 that is desired to be captured with the highest spatial resolution. For example, if the aerial image 16 is a person's head, the first lens array 8 may be disposed at a depth position of the person's face in the aerial image 16.

[0057] As shown in FIG. 3 , the first lens array 8 has a plurality of first lenses 34 arranged two-dimensionally along the incident surface 30. Note that FIG. 3 illustrates an enlarged portion of the first lens array 8. When the first lens array 8 is viewed in a plan view (i.e., when viewed perpendicularly to the incident surface 30), each of the plurality of first lenses 34 is formed, for example, in a hexagonal shape. That is, the first lens array 8 has a honeycomb structure. By forming each of the plurality of first lenses 34 in a hexagonal shape, the plurality of first lenses 34 can be arranged two-dimensionally without gaps, thereby eliminating non-lens portions in the first lens array 8 and increasing the effective area. Note that each of the plurality of first lenses 34 is a microlens, and the size L of each of the plurality of first lenses 34 is preferably, for example, 300 μm to 2.5 mm, and more preferably 0.5 mm to 1.5 mm.

[0058] 4, when light rays emitted from the aerial image 16 are incident on the incident surface 30 of the first lens array 8, light rays with a viewing angle θ are acquired by each of the multiple first lenses 34. Furthermore, of the light emitted from the exit surface 32 of the first lens array 8, the light rays that are captured by the camera 10 are light rays that travel substantially parallel to the camera 10.

[0059] The surface shape of each of the plurality of first lenses 34 may be spherical or aspherical. By making the surface shape of each of the plurality of first lenses 34 aspherical, the influence of spherical aberration on peripheral light rays can be reduced, and light rays can be obtained with uniform precision within the viewing angle θ.

[0060] The curvature of each of the multiple first lenses 34 may be uniform in all directions (i.e., the horizontal curvature and the vertical curvature may be the same). Alternatively, the horizontal curvature and the vertical curvature of each of the multiple first lenses 34 may be different from each other. This makes it possible to control the ray spacing, ray density, viewing angle, and the like that can be obtained by the first lens array 8. For example, by making the horizontal curvature of each of the multiple first lenses 34 larger than the vertical curvature, it is possible to make the viewing angle in the horizontal direction larger than the viewing angle in the vertical direction. In this specification, the "horizontal direction" refers to the direction parallel to the floor surface, and the "vertical direction" refers to the direction perpendicular to the floor surface.

[0061] In this embodiment, the first lens array 8 is configured with a refractive lens, but is not limited to this and may be configured with a diffractive lens. Alternatively, the first lens array 8 may be configured with a lenticular lens. In this case, each of the multiple first lenses 34 is configured as a so-called cylindrical lens that has a curvature in the horizontal direction but not in the vertical direction. This allows light ray information to be acquired only in the horizontal direction, but is effective in obtaining a high-resolution 3D image with only horizontal parallax, in which pixel information can be allocated to resolution characteristics.

[0062] The camera 10 is a digital camera having an imaging lens 36 and an image sensor 38, and is disposed to face the exit surface 32 of the first lens array 8. In the present embodiment, only one camera 10 is disposed. The camera 10 captures light rays emitted from the exit surface 32 of the first lens array 8 (i.e., light rays from the aerial image 16 that have passed through each of the multiple first lenses 34 of the first lens array 8). Specifically, the light rays emitted from the exit surface 32 of the first lens array 8 are imaged on the image sensor 38 by the imaging lens 36 and then captured by the image sensor 38. The camera 10 generates a three-dimensional elemental video signal including light ray information regarding the traveling direction, color, etc. of the light rays emitted from each of the multiple first lenses 34, and outputs the generated three-dimensional elemental video signal to the three-dimensional elemental video display device 12. 4, if the camera 10 is sufficiently far from the first lens array 8, the camera 10 may capture, in a pan-focus state, the light rays emitted from the emission surface 32 of the first lens array 8. This makes it possible to generate a three-dimensional elemental video signal with high resolution in the depth range.

[0063] In order to capture the light beams emitted from each of the multiple first lenses 34 at 100 to 400 pixels or more, the camera 10 is preferably configured as an ultra-high-definition digital camera of 8K or higher. In the present embodiment, as shown in FIG. 1 , the camera 10 outputs the 3D element video signal to the 3D element video display device 12. However, this is not limited to this. The signal may be output to an external terminal device such as a personal computer, or may be recorded on an external video recording device such as a cloud server. When the 3D element video signal is output to an external terminal device, the terminal device may perform, for example, calculation of a 3D shape, generation of a 3D model, generation of a multi-viewpoint video, etc. Alternatively, the camera 10 may distribute the 3D element video represented by the 3D element video signal to a remote location via a network. The 3D element video represented by the 3D element video signal may also be recorded on a recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD, or semiconductor memory.

[0064] As shown in FIG. 1 , the 3D elemental image display device 12 is formed, for example, with a liquid crystal display or the like, and receives a 3D elemental image signal from the camera 10. A display surface 40 is formed on the front surface of the 3D elemental image display device 12. A 3D elemental image 42 is displayed on the display surface 40 of the 3D elemental image display device 12 based on the 3D elemental image signal from the camera 10. As described above, the 3D elemental image 42 is a two-dimensional image that cannot be viewed stereoscopically by itself. In FIG. 1 , the display surface 40 of the 3D elemental image display device 12 is perpendicular to the plane of FIG. 1 and is positioned facing the observer 50. The camera 10 captures light rays, and the 3D elemental image 42 is displayed by the 3D elemental image display device 12 in real time. As a result, the 3D elemental image 42 becomes a dynamic two-dimensional image synchronized with the movement of the 3D object 4, which is moving in real time.

[0065] The number of first lenses 34 in the first lens array 8 corresponds to the resolution of the three-dimensional elemental images 42. Furthermore, the number of pixels capturing an image within one first lens 34 defines the light density. The viewing angle that can be captured by the camera 10 is determined by the numerical aperture of the first lenses 34. Therefore, it is preferable that the first lens array 8 has a minute and dense lens structure.

[0066] The second lens array 14 is a lens array having substantially the same shape as the first lens array 8. The second lens array 14 is arranged to face the display surface 40 of the three-dimensional elemental image display device 12. The second lens array 14 has an incident surface 44 onto which light rays from the display surface 40 of the three-dimensional elemental image display device 12 are incident, and an exit surface 46 (the surface opposite the incident surface 44) from which light rays incident on the incident surface 44 exit. The second lens array 14 has a plurality of second lenses 48 arranged two-dimensionally along the incident surface 44, and each of the plurality of second lenses 48 has substantially the same shape as the plurality of first lenses 34 of the first lens array 8. The optical path length between the display surface 40 of the three-dimensional elemental image display device 12 and the second lens array 14 is substantially equal to the focal length of each of the plurality of second lenses 48.

[0067] Light rays from the three-dimensional elemental images 42 displayed on the display surface 40 of the three-dimensional elemental image display device 12 are incident on the incident surface 44 of the second lens array 14. At this time, the light rays from the three-dimensional elemental images 42 are converted by the multiple second lenses 48 of the second lens array 14 into light rays (i.e., light rays emitted from the aerial image 16) that are incident on each of the multiple first lenses 34 of the first lens array 8. This is because the second lens array 14 has substantially the same shape as the first lens array 8. As a result, an observer 50 can view the three-dimensional elemental images 42 displayed on the display surface 40 of the three-dimensional elemental image display device 12 through the second lens array 14, thereby achieving stereoscopic vision as a three-dimensional image 52 with the naked eye. In other words, the three-dimensional image 52 is an image that reproduces (reproduces) the light rays emitted from the aerial image 16 and can also be said to be an image representing the three-dimensional object 4. At this time, the viewer 50 can view the three-dimensional image 52 stereoscopically at the viewing angle θ (see FIG. 1) described above.

[0068] The depth of the three-dimensional image 52 is the correct depth as seen by the observer 50. For example, if the face of the person in the three-dimensional object 4 faces the incident surface 18 of the aerial image forming optical system 6, the face of the person in the three-dimensional image 52 faces the observer 50.

[0069] FIG. 6 is a diagram showing an example of a three-dimensional elemental image 42. More specifically, FIG. 6A is a diagram showing the three-dimensional elemental image 42, and FIG. 6B is a diagram showing an enlarged portion of the three-dimensional elemental image 42 shown in FIG. 6A. In the example shown in FIG. 6, the three-dimensional object represented by the three-dimensional elemental image 42 is a teddy bear. As shown in FIG. 6B, the three-dimensional elemental image 42 includes a plurality of hexagonal elemental images 54 arranged side by side without any gaps. Each of these elemental images 54 is an image obtained by capturing light rays emitted from each of the plurality of first lenses 34 of the first lens array 8. The light rays from each of the plurality of elemental images 54 pass through the plurality of second lenses 48 of the second lens array 14, thereby reproducing the light rays emitted from the aerial image 16.

[0070] [1-2. Effects] The configuration of a video system 100 according to a comparative example will be described with reference to Fig. 7. Fig. 7 is a diagram showing the configuration of a video system 100 according to a comparative example. Note that in Fig. 7, the same components as those in Fig. 1 are denoted by the same reference numerals, and their description will be omitted.

[0071] 7, video system 100 includes optical lens 102, first lens array 8, camera 10, three-dimensional elemental image display device 12, and second lens array 14. That is, video system 100 is provided with optical lens 102 instead of aerial image forming optical system 6 of video system 2 described above.

[0072] The optical lens 102 is composed of a convex lens and is disposed at a position that is twice the focal length f away from the three-dimensional object 4. The optical lens 102 forms a real image 104 of the three-dimensional object 4 at a position that is twice the focal length f away. Note that the up-down orientation of the real image 104 is inverted relative to the up-down orientation of the three-dimensional object 4, but the depth of the real image 104 is not inverted relative to the depth of the three-dimensional object 4.

[0073] The first lens array 8 is disposed at the imaging position of the real image 104. Light rays emitted from the real image 104 pass through the first lens array 8. The camera 10 captures the light rays that have passed through the first lens array 8, generates a three-dimensional elemental video signal, and outputs the generated three-dimensional elemental video signal to the three-dimensional elemental video display device 12. As a result, the three-dimensional elemental video represented by the three-dimensional elemental video signal is displayed on the three-dimensional elemental video display device 12. The observer 50 can view the three-dimensional elemental video displayed on the three-dimensional elemental video display device 12 through the second lens array 14, thereby achieving a stereoscopic view as a three-dimensional video 106 with the naked eye.

[0074] However, in the above-described video system 100, in order to acquire light ray information over a wide viewing angle, the diameter D of the optical lens 102 needs to be very large (for example, D = approximately 50 cm to 1 m), and it is difficult to manufacture an optical lens 102 of such a size. Also, since the distance L from the three-dimensional object 4 to the camera 10 is approximately several meters, the overall size of the video system 100 becomes large, making it impractical.

[0075] Furthermore, the depth of the three-dimensional image 106 is the opposite depth as seen by the observer 50. For example, if the face of the person in the three-dimensional object 4 is facing the optical lens 102, the face of the person in the three-dimensional image 106 will be facing away from the observer 50. This is because the depth of the real image 104 formed by the optical lens 102 is not inverted with respect to the depth of the three-dimensional object 4. Therefore, the observer 50 cannot stereoscopically view the three-dimensional image 106 at the correct depth.

[0076] In contrast, in the present embodiment, an aerial image forming optical system 6 having retroreflection properties is used. Therefore, even if the size of the aerial image forming optical system 6 is relatively small (for example, approximately 30 cm x 30 cm), the viewing angle θ of the light rays from the three-dimensional object 4 that enter the aerial image forming optical system 6 can be ensured to be wide, for example, approximately 30° to 90°. This makes it possible to acquire light ray information over a wide viewing angle θ while keeping the overall size of the video system 2 compact. Furthermore, even when only a single digital camera is used as the camera 10, light ray information over a wider viewing angle θ can be acquired compared to a conventional dedicated camera with a built-in image sensor and lens array.

[0077] Furthermore, the aerial image 16 formed by the aerial image forming optical system 6 is free from optical aberration and optical distortion, thereby suppressing image degradation. Furthermore, because the aerial image forming optical system 6 does not have an inherent focal length, the aerial image 16 can be formed at life-size even when the three-dimensional object 4 is located at an arbitrary distance from the aerial image forming optical system 6. Furthermore, because the depth of the aerial image 16 is inverted relative to the depth of the three-dimensional object 4, the depth of the three-dimensional image 52 can be made correct as viewed by the observer 50. Furthermore, as shown in (a) and (b) of Figures 5A and 5B, the distribution of light rays emitted in various directions from the three-dimensional image 52 is substantially the same as the distribution of light rays emitted in various directions from the three-dimensional object 4, so the three-dimensional image 52 can faithfully reproduce the surface texture (e.g., gloss, etc.) of the three-dimensional object 4.

[0078] [1-3. Various Modifications of the First Lens Array] [1-3-1. Modification 1] The configuration of a first lens array 8A according to Modification 1 of Embodiment 1 will be described with reference to Fig. 8A. Fig. 8A is a diagram showing the first lens array 8A according to Modification 1 of Embodiment 1.

[0079] As shown in Fig. 8A, the first lens array 8A has a plurality of first lenses 34A arranged two-dimensionally. When the first lens array 8A is viewed from above, each of the plurality of first lenses 34A is formed in a circular shape. The plurality of first lenses 34A are arranged in a bale-stacked arrangement. In this case, a non-lens portion 56 is formed between two adjacent first lenses 34A.

[0080] In this modified example, the arrangement of the plurality of first lenses 34A is a bale-stacked arrangement, but this is not limitative and may be, for example, a square arrangement.

[0081] [1-3-2. Modification 2] The configuration of the first lens array 8B according to Modification 2 of Embodiment 1 will be described with reference to Fig. 8B. Fig. 8B is a diagram showing the first lens array 8B according to Modification 2 of Embodiment 1.

[0082] As shown in FIG. 8B , the first lens array 8B has a plurality of first lenses 34A arranged two-dimensionally. The shape and arrangement of the plurality of first lenses 34A are the same as those in the first modification example. Furthermore, a light-shielding portion 58 is arranged in a non-lens portion 56 formed between two adjacent first lenses 34A. The light-shielding portion 58 is formed, for example, by applying a black paint having light-shielding properties. The light-shielding portion 58 can prevent unnecessary leakage light from passing through the non-lens portion 56.

[0083] [1-3-3. Modification 3] The configuration of a first lens array 8C according to Modification 3 of Embodiment 1 will be described with reference to Fig. 8C. Fig. 8C is a diagram showing the first lens array 8C according to Modification 3 of Embodiment 1.

[0084] As shown in Figure 8C, the first lens array 8C has a plurality of first lenses 34C arranged two-dimensionally. When the first lens array 8C is viewed in a plan view, each of the plurality of first lenses 34C is formed in a square shape (an example of a rectangular shape). The arrangement of the plurality of first lenses 34C is a square array. This allows light rays to be obtained at the same viewing angle in both the horizontal direction (left-right direction in Figure 8C) and the vertical direction (up-down direction in Figure 8C).

[0085] 8D, the configuration of a first lens array 8D according to a fourth modification of embodiment 1 will be described. Fig. 8D is a diagram showing the first lens array 8D according to the fourth modification of embodiment 1.

[0086] As shown in FIG. 8D , the first lens array 8D has a plurality of first lenses 34D arranged two-dimensionally. When the first lens array 8D is viewed in a plan view, each of the plurality of first lenses 34D is formed in a rectangular shape (an example of a rectangular shape). The long side of each of the plurality of first lenses 34D extends horizontally (left-right in FIG. 8D ) and the short side extends vertically (up-down in FIG. 8D ). The arrangement of the plurality of first lenses 34D is a square array. This allows the viewing angle obtainable in the horizontal and vertical directions to be easily controlled by the aspect ratio of the first lenses 34D.

[0087] 8E, a configuration of a first lens array 8E according to a fifth modification of embodiment 1 will be described. Fig. 8E is a diagram showing the first lens array 8E according to the fifth modification of embodiment 1.

[0088] As shown in FIG. 8E , the first lens array 8E has a plurality of first lenses 34E arranged two-dimensionally. When the first lens array 8E is viewed from above, each of the plurality of first lenses 34E is formed in a rectangular shape (an example of a rectangular shape). Furthermore, the long side of each of the plurality of first lenses 34E extends in the horizontal direction (the left-right direction in FIG. 8E ), and the short side extends in the vertical direction (the up-down direction in FIG. 8E ). Furthermore, the plurality of first lenses 34E are arranged in a staggered pattern. This allows for improved resolution characteristics in the horizontal direction.

[0089] (Embodiment 2) The configuration of a video system 2F according to embodiment 2 will be described with reference to Fig. 9. Fig. 9 is a diagram showing a part of the configuration of a video system 2F according to embodiment 2. Note that in each embodiment described below, the same components as those in embodiment 1 above are denoted by the same reference numerals, and their description will be omitted.

[0090] As shown in Figure 9, the video system 2F includes a condenser lens 60 in addition to the components described in the first embodiment. The condenser lens 60 is a convex lens and is disposed opposite the exit surface 32 of the first lens array 8. The camera 10 is disposed at a position spaced a focal length f from the condenser lens 60. The condenser lens 60 condenses, at the position of the camera 10, the substantially parallel light rays that are emitted from the exit surface 32 of the first lens array 8 and are directed toward the camera 10. This allows the camera 10 to be located closer to the first lens array 8 than in the first embodiment, thereby making the overall size of the video system 2F more compact.

[0091] (Third embodiment) The configuration of a video system 2G according to a third embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing a part of the configuration of a video system 2G according to the third embodiment.

[0092] 10 , the video system 2G includes an aperture array 62 in addition to the components described in the first embodiment. The aperture array 62 is arranged opposite the incident surface 30 of the first lens array 8. That is, the aperture array 62 is arranged between the aerial image forming optical system 6 (see FIG. 1 ) and the first lens array 8. The aperture array 62 has a plurality of apertures 64. The plurality of apertures 64 are arranged opposite the plurality of first lenses 34 of the first lens array 8, respectively, and are arranged at the focal length positions of the plurality of first lenses 34.

[0093] As a result, by cutting out unnecessary light with the aperture array 62, it is possible to prevent the unnecessary light from entering the first lens array 8. As a result, it is possible to prevent blurring of the image in the three-dimensional elemental image 42. Furthermore, by adjusting the size of each of the plurality of apertures 64, it is possible to obtain light ray information with a wide depth reproduction range.

[0094] (Fourth embodiment) The configuration of a video system 2H ​​according to a fourth embodiment will be described with reference to Fig. 11. Fig. 11 is a diagram showing a part of the configuration of a video system 2H ​​according to the fourth embodiment.

[0095] 11 , the video system 2H ​​includes a plurality of cameras 10. Each of the plurality of cameras 10 is configured similarly to the single camera 10 described in the first embodiment. That is, each of the plurality of cameras 10 captures an image of light emitted from the exit surface 32 of the first lens array 8, generates a three-dimensional elemental video signal, and outputs the generated three-dimensional elemental video signal to a three-dimensional elemental video display device (not shown).

[0096] This allows the multiple cameras 10 to easily obtain more light ray information over a wide viewing angle, thereby improving the reproducibility of the three-dimensional image 52.

[0097] Fifth Embodiment The configuration of a video system 2J according to a fifth embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing the configuration of a video system 2J according to the fifth embodiment.

[0098] 12 , the three-dimensional display device 15J of the video system 2J includes a detection unit 66 and a display control unit 68 in addition to the three-dimensional elemental image display device 12 and the second lens array 14 described in the first embodiment. The second lens array 14 is detachable from the display surface 40 of the three-dimensional elemental image display device 12.

[0099] The detection unit 66 is a sensor for detecting attachment / detachment of the second lens array 14 to / from the display surface 40 of the three-dimensional elemental image display device 12. The detection unit 66 outputs the detection result to the display control unit 68.

[0100] The display control unit 68 controls the display content on the display surface 40 of the three-dimensional elemental image display device 12 based on the detection result of the detection unit 66 .

[0101] Specifically, when the second lens array 14 is attached to the display surface 40 of the three-dimensional elemental image display device 12, the display control unit 68 displays three-dimensional elemental images on the display surface 40. This allows the observer 50 to view the three-dimensional elemental images displayed on the display surface 40 of the three-dimensional elemental image display device 12 through the second lens array 14, thereby achieving stereoscopic vision as a three-dimensional image 52 with the naked eye.

[0102] On the other hand, when the second lens array 14 is detached from the display surface 40 of the 3D elemental image display device 12, the display control unit 68 displays images other than the 3D elemental images on the display surface 40. Note that "images other than the 3D elemental images" refer to two-dimensional images that cannot be viewed stereoscopically by themselves and cannot be viewed stereoscopically even using the second lens array 14, such as images produced by playing back normal video content. This allows the viewer 50 to view the other images displayed on the display surface 40 of the 3D elemental image display device 12 as two-dimensional images without passing through the second lens array 14.

[0103] The display control unit 68 may be disposed inside the three-dimensional elemental image display device 12 or may be disposed outside the three-dimensional elemental image display device 12 .

[0104] Therefore, in this embodiment, the display content on the display surface 40 of the three-dimensional elemental image display device 12 can be appropriately changed in conjunction with the attachment or detachment of the second lens array 14 to or from the display surface 40 of the three-dimensional elemental image display device 12.

[0105] Sixth Embodiment The configuration of a video system 2K according to a sixth embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing the configuration of a video system 2K according to the sixth embodiment.

[0106] As shown in FIG. 13, in a three-dimensional display device 15K of a video system 2K, a second lens array 14K has a polarization direction switching element 70 and a polarization-dependent lens array 72.

[0107] The three-dimensional elemental image display device 12 is a liquid crystal display that displays light polarized in one direction. Alternatively, if the three-dimensional elemental image display device 12 is an EL (Electro Luminescence) display or an LED (Light Emitting Diode) display that does not display light polarized in one direction, it is assumed to be equipped with an optical element (polarizing plate) that aligns polarization in one direction.

[0108] The polarization direction switching element 70 is disposed so as to face the display surface 40 of the three-dimensional elemental image display device 12. As a result, light rays emitted from the display surface 40 of the three-dimensional elemental image display device 12 pass through the polarization direction switching element 70. The polarization direction switching element 70 controls the polarization direction of the light rays emitted from the display surface 40 of the three-dimensional elemental image display device 12 (i.e., the light rays that have passed through the polarization direction switching element 70) in accordance with the display content on the display surface 40 of the three-dimensional elemental image display device 12.

[0109] Specifically, when a three-dimensional elemental image is displayed on the display surface 40 of the three-dimensional elemental image display device 12, the polarization direction switching element 70 polarizes the light beam emitted from the display surface 40 in a first polarization direction. On the other hand, when an image other than a three-dimensional elemental image is displayed on the display surface 40 of the three-dimensional elemental image display device 12, the polarization direction switching element 70 polarizes the light beam emitted from the display surface 40 in a second polarization direction that is orthogonal to the first polarization direction (i.e., different from the first polarization direction).

[0110] The polarization-dependent lens array 72 is formed of, for example, a liquid crystal lens, and is arranged to face the exit surface (i.e., the surface on the side farther from the three-dimensional elemental image display device 12) of the polarization direction switching element 70. As a result, light rays having the first polarization direction or the second polarization direction that have passed through the polarization direction switching element 70 are incident on the polarization-dependent lens array 72.

[0111] When light rays in a first polarization direction are incident on the polarization-dependent lens array 72, the light rays in the first polarization direction are converted by the polarization-dependent lens array 72 into light rays that are incident on the first lens array 8. That is, in this case, the polarization-dependent lens array 72 functions in the same manner as the second lens array 14 described in the first embodiment. As a result, the observer 50 can view the three-dimensional elemental images displayed on the display surface 40 of the three-dimensional elemental image display device 12 through the second lens array 14K, thereby achieving stereoscopic vision as a three-dimensional image 52 with the naked eye.

[0112] On the other hand, when light rays having a second polarization direction are incident on the polarization-dependent lens array 72, the light rays having the second polarization direction are transmitted through the polarization-dependent lens array 72 as they are. That is, in this case, the polarization-dependent lens array 72 functions similarly to, for example, a transparent glass plate having no lens function. As a result, the observer 50 can view an image other than the 3D elemental image displayed on the display surface 40 of the 3D elemental image display device 12 through the second lens array 14K, and thereby visually recognize the image as a 2D image.

[0113] Therefore, in this embodiment, with the second lens array 14K still attached to the display surface 40 of the three-dimensional element image display device 12, the observer 50 can stereoscopically view the three-dimensional element image as a three-dimensional image 52, or view images other than the three-dimensional element image as two-dimensional images, depending on the display content on the display surface 40 of the three-dimensional element image display device 12.

[0114] (Seventh embodiment) The configuration of a video system 2L according to a seventh embodiment will be described with reference to Fig. 14 and Fig. 15. Fig. 14 is a block diagram showing the functional configuration of a video system 2L according to the seventh embodiment. Fig. 15 is a diagram for explaining the operation of a processing unit 74 according to the seventh embodiment.

[0115] 14, the video system 2L has, as its functional configuration, a processing unit 74 for performing predetermined processing on the three-dimensional elemental video signal output from the camera 10. The processing unit 74 has a conversion unit 76, an encoding unit 78, and a decoding unit 80.

[0116] As shown in (a) of Figure 15, the converter 76 converts the 3D elemental video signal output from the camera 10 into a signal of a multi-viewpoint video group including a plurality of multi-viewpoint videos. Here, the viewing zone angle θ of the multi-viewpoint video is equal to or greater than 30° and equal to or less than 90°. Furthermore, the viewing zone angle interval between the plurality of elemental videos included in the multi-viewpoint video is preferably equal to or greater than 0.5° and equal to or less than 2°, and more preferably about 1°. Alternatively, the converter 76 may convert the 3D elemental video signal output from the camera 10 into a signal of a multi-viewpoint video group with a depth image.

[0117] The encoding unit 78 encodes the signals of the multi-view video group converted by the conversion unit 76 using a predetermined encoding method such as H.264 / MVC (Multiview Video Coding). Alternatively, the encoding unit 78 encodes the signals of the multi-view video group with depth images converted by the conversion unit 76 using a predetermined encoding method such as 3D HEVC (High Efficiency Video Coding). The encoding unit 78 transmits the encoded signals to the decoding unit 80.

[0118] The decoding unit 80 generates a 3D element video signal by decoding the signal encoded by the encoding unit 78 using a predetermined decoding method. As shown in (b) of FIG. 15 , among the multiple element videos included in the 3D element video represented by the 3D element video signal, one element video corresponding to one second lens 48 of the second lens array 14 (see FIG. 1 ) includes a multi-view video. That is, the number of viewpoints in the multi-view video is equal to the number of pixels in one element video of the 3D element video, and the number of pixels in the multi-view video group is equal to the number of element videos of the 3D element video (the number of second lenses 48). Note that there is a one-to-one correspondence between the element videos of the 3D element video and the second lenses 48 of the second lens array 14. The decoding unit 80 outputs the generated 3D element video signal to the 3D element video display device 12.

[0119] Therefore, in this embodiment, the three-dimensional elemental video signal can be easily transmitted.

[0120] (Other Modifications, etc.) While the video systems according to the first to seventh embodiments of the present invention have been described above, the present invention is not limited to these embodiments. For example, the above-described embodiments may be combined with each other.

[0121] In each of the above embodiments, the three-dimensional elemental image display device 12 is configured as a liquid crystal display, but this is not limited to this, and the three-dimensional elemental image display device 12 may be configured as, for example, a head-mounted display, an eyeglass-type display, or a goggle-type display.

[0122] The present invention can be applied to, for example, an imaging system for acquiring three-dimensional elemental image signals using light field technology.

[0123] 2, 2F, 2G, 2H, 2J, 2K, 2L, 100 Imaging system 4 Three-dimensional object 6 Aerial image forming optical system 8, 8A, 8B, 8C, 8D, 8E First lens array 10 Camera 12 Three-dimensional elemental image display device 14, 14K Second lens array 15, 15J, 15K Three-dimensional display device 16 Aerial image 18, 30, 44 Incident surface 20, 32, 46 Exit surface 22 Base portion 22a First main surface 22b Second main surface 24 Dihedral corner reflector 24a, 24b Side surface 24c Top surface 24d Bottom surface 26a First reflecting surface 26b Second reflecting surface 28 Arrow 34, 34A, 34C, 34D, 34E First lens 36 Imaging lens 38 Imaging element 40 Display surface 42 Three-dimensional elemental image 48 Second lens 50 Observer 52, 106 Three-dimensional image 54 Elemental image 56 Non-lens portion 58 Light-shielding portion 60 Condenser lens 62 Aperture array 64 Aperture 66 Detector 68 Display controller 70 Polarization direction switching element 72 Polarization-dependent lens array 74 Processing unit 76 Conversion unit 78 Encoder 80 Decoder 102 Optical lens 104 Real image

Claims

1. An aerial image forming optical system having a retroreflective characteristic, the aerial image forming optical system forming an aerial image representing a three-dimensional object by imaging a light beam from the three-dimensional object in the air; a first lens array that is arranged at or near an imaging position of the aerial image and has a plurality of first lenses that are arranged two-dimensionally; a camera having an imaging lens and an image sensor, which captures light rays from the aerial image that have passed through the first lens array, and outputs a three-dimensional elemental video signal including light ray information related to the light rays from the aerial image. Video system.

2. The camera (i) outputs the three-dimensional elemental image signal to an external terminal device, (ii) records the three-dimensional elemental image signal in an external image recording device, or (iii) outputs the three-dimensional elemental image signal to a three-dimensional elemental image display device. The video system according to claim 1 .

3. The aerial image forming optical system forms the aerial image at the imaging position symmetrical to the three-dimensional object with respect to the aerial image forming optical system by reflecting a light beam from the three-dimensional object at least twice. The video system according to claim 1 .

4. When the first lens array is viewed in a plan view, each of the first lenses is formed in a circular, rectangular or hexagonal shape. The video system according to claim 1 .

5. The surface shape of each of the plurality of first lenses is spherical or aspherical. The video system according to claim 1 .

6. Each of the first lenses is configured such that a curvature in the horizontal direction is equal to a curvature in the vertical direction, or the curvature in the horizontal direction is greater than the curvature in the vertical direction. The video system according to claim 1 .

7. The imaging system further comprises: an aperture array disposed between the aerial image forming optical system and the first lens array, the aperture array having a plurality of apertures disposed at positions corresponding to the focal lengths of the plurality of first lenses, respectively; The video system according to claim 1 .

8. The camera may include a single digital camera or multiple digital cameras. The video system according to claim 1 .

9. The imaging system further comprises: a three-dimensional element image display device having a display surface for receiving the three-dimensional element image signal output from the camera and displaying a three-dimensional element image based on the three-dimensional element image signal; a second lens array having substantially the same shape as the first lens array, the second lens array being arranged so as to face the display surface of the three-dimensional elemental image display device and having a plurality of second lenses arranged two-dimensionally; The optical path length between the display surface of the three-dimensional elemental image display device and the second lens array is approximately equal to the focal length of each of the plurality of second lenses. The video system according to any one of claims 1 to 8.

10. The second lens array is detachable from the display surface of the three-dimensional elemental image display device, The imaging system further comprises: a detection unit that detects attachment / detachment of the second lens array to / from the display surface of the three-dimensional elemental image display device; a display control unit that controls display content on the display surface of the three-dimensional elemental image display device based on a detection result of the detection unit, The display control unit (i) causes the display surface to display the three-dimensional elemental images when the second lens array is attached to the display surface, and (ii) causes the display surface to display images other than the three-dimensional elemental images when the second lens array is detached from the display surface.

10. The imaging system according to claim 9.

11. forming an aerial image representing a three-dimensional object by imaging light rays from the three-dimensional object in the air using an aerial image forming optical system having retroreflective properties; a first lens array disposed at or near an imaging position of the aerial image, the first lens array having a plurality of first lenses disposed two-dimensionally, the first lens array transmitting light rays from the aerial image; A camera having an imaging lens and an image sensor captures light rays from the aerial image that have passed through the first lens array, thereby outputting a three-dimensional elemental image signal including light ray information related to the light rays from the aerial image. A method for acquiring three-dimensional elemental images.

12. A three-dimensional elemental image represented by a three-dimensional elemental image signal output from the image system according to claim 1 is recorded. Recording medium.

13. A method for transmitting the three-dimensional elemental image signal output from the camera in the image system according to claim 1, comprising the steps of: (a) converting the three-dimensional elemental image signal output from the camera into a multi-viewpoint image group signal or a multi-viewpoint image group signal with a depth image; (b) encoding the signal converted in (a) using a predetermined encoding method; (c) transmitting the signal encoded in (b); (d) decoding the signal transmitted in (c) using a predetermined decoding method; The viewing angle of each of the multiple multi-viewpoint images included in the multi-viewpoint image group is 30° or more and 90° or less, and the viewing angle interval of the multiple elemental images included in the multi-viewpoint images is 0.5° or more and 2° or less. A method for transmitting three-dimensional elemental video signals.