Spatial floating image display device

Abstract

To provide a more suitable spatial-suspended image display device which contributes to "3. Good Health and Well-Being", "9. Industry, Innovation and Infrastructure", and "11. Sustainable Cities and Communities" of Sustainable Development Goals.SOLUTION: A spatial-suspended image display device includes a display part, and a retroreflection plate which reflects image light from the display part, and forms a spatial-suspended image in the air by the reflected light, wherein there is a region where an object is displayed in a display range of the spatial-suspended image, a black display region surrounding the region is arranged, a physical frame is arranged so as to surround the spatial-suspended image from its periphery, the physical frame forms an opening window having such a cover structure as to cover a storage part storing the display part and the retroreflection plate, and a light-shielding plate is arranged between the opening window and the storage part storing the display part and the retroreflection plate inside the cover structure, and extends toward the storage part storing the display part and the retroreflection plate from at least both the upper end and the lower end of the opening window.SELECTED DRAWING: Figure 46

Description

Technical Field

[0001] The present invention relates to a spatial floating image display device.

Background Art

[0002] As a spatial floating information display system, a video display device that directly displays an image toward the outside and a display method that is displayed as a spatial screen are already known. Also, a detection system for reducing false detection of operations on the operation surface of the displayed spatial image is disclosed in, for example, Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, a touch operation on a spatial floating image is not performed on a physical button, a touch panel, or the like. For this reason, there are cases where the user cannot recognize whether a touch operation has been performed.

[0005] An object of the present invention is to provide a more suitable spatial floating image display device.

Means for Solving the Problems

[0006] To solve the above problems, for example, the configuration described in the claims is adopted. Although this application includes a plurality of means for solving the above problems, as an example, the spatial floating image display device includes a display unit that displays an image, and a retroreflective plate that reflects the image light from the display unit and forms a spatial floating image in the air by the reflected light. In the display range of the spatial floating image, there is an area where an object is displayed, and a black display area surrounding the area where the object is displayed is arranged. It has a physical frame arranged to surround the spatial floating image from the surroundings. The physical frame forms an opening window of a cover structure that covers a storage unit for storing the display unit and the retroreflective plate. Inside the cover structure, there is a light shielding plate arranged between the opening window, the display unit, and the storage unit for storing the retroreflective plate, and the light shielding plate has a light shielding plate extending from at least both the upper end and the lower end of the opening window toward the storage unit for storing the display unit and the retroreflective plate.

Advantages of the Invention

[0007] According to the present invention, a more suitable spatial floating image display device can be realized. Other problems, configurations, and effects will be clarified in the following description of the embodiments.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42A

Figure 42B

Figure 43

Figure 44

Figure 45

Figure 46

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the description of the examples, and various changes and modifications can be made by those skilled in the art within the scope of the technical idea disclosed in this specification. Also, in all the drawings for explaining the present invention, those having the same function are given the same reference numerals, and the repeated description may be omitted. In the following description of the examples, an image floating in space is expressed by the term "space floating image". Instead of this term, it may be expressed as "aerial image", "space image", "aerial floating image", "space floating optical image of the display image", "aerial floating optical image of the display image", etc. The term "space floating image" mainly used in the description of the examples is used as a representative example of these terms.

[0010] The following examples relate to an image display device capable of transmitting an image by image light from an image light source through a transparent member that partitions a space such as glass and displaying it as a space floating image outside the transparent member.

[0011] According to the following embodiments, for example, a video display device suitable for use in bank ATMs, ticket vending machines at stations, digital signage, etc. can be realized. For example, currently, in bank ATMs, ticket vending machines at stations, etc., a touch panel is usually used. However, by using a transparent glass surface or a light-transmissive plate material, high-resolution video information can be displayed in a state of being spatially floating on this glass surface or light-transmissive plate material. At this time, by making the divergence angle of the emitted video light small, that is, an acute angle, and further aligning it with a specific polarization, only the regular reflected light is efficiently reflected to the retroreflective member. Therefore, the light utilization efficiency is high, and it is possible to suppress ghost images that occur in addition to the main spatially floating image, which has been a problem in the conventional retroreflective method, and a clear spatially floating video can be obtained. In addition, by means of the device including the light source of this embodiment, it is possible to provide a novel and highly usable spatially floating video display device (spatially floating video display system) capable of significantly reducing power consumption. Further, for example, it is possible to provide a spatially floating video display device for a vehicle that enables so-called one-directional spatially floating video display that can be visually recognized inside and / or outside the vehicle. In the following embodiments, in any case, a plate-shaped member may be used as the retroreflective member. In this case, it may be referred to as a retroreflective plate.

[0012] On the other hand, in the conventional technology, an organic EL panel or a liquid crystal panel is combined with a retroreflective member 151 as a high-resolution color display video source 150. In the conventional technology, since the video light diffuses at a wide angle, in addition to the reflected light regularly reflected by the retroreflective member 151, as shown in FIG. 24, ghost images 301 and 302 are generated by the video light incident obliquely on the retroreflective member 2a, deteriorating the image quality of the spatially floating video. Also, as shown in FIG. 23, in addition to the regular spatially floating video 300, a plurality of first ghost images 301, second ghost images 302, etc. are generated. For this reason, there has been a major security issue in that the same spatially floating video, which is a ghost image, is monitored not only by the monitor.

[0013] <Spatially Floating Video Display Device> FIG. 1 is a diagram showing an example of the usage form of a spatial floating image display device according to an embodiment of the present invention, and is a diagram showing the overall configuration of the spatial floating image display device according to the present embodiment. The specific configuration of the spatial floating image display device will be described in detail with reference to FIG. 2 and the like. Light of a specific polarization with a sandwiching angle directivity characteristic is emitted as an image light beam from the image display device 1, once enters the retroreflective member 2, is retroreflected, passes through a transparent member 100 (such as glass), and forms a real image, an aerial image (spatial floating image 3), outside the glass surface.

[0014] In addition, in a store or the like, a space is partitioned by a show window (also referred to as "window glass") 105 which is a translucent member such as glass. According to the spatial floating image display device of the present embodiment, it is possible to display a floating image in one direction with respect to the outside and / or the inside of the store (space) through such a transparent member.

[0015] In FIG. 1(A), the inside (inside the store) of the window glass 105 is shown in the depth direction, and the outside (for example, the sidewalk) is in the front. On the other hand, by providing means for reflecting a specific polarization wave on the window glass 105, it is also possible to form an aerial image at a desired position inside the store.

[0016] FIG. 1(B) is a schematic block diagram showing the configuration of the above-described display device 1. The display device 1 includes an image display unit that displays the original image of the aerial image, an image control unit that converts the input image according to the resolution of the panel, and an image signal receiving unit that receives an image signal. The image signal receiving unit supports wired input signals such as HDMI (High-Definition Multimedia Interface) input and wireless input signals such as Wi-Fi (Wireless Fidelity), and also functions alone as an image receiving and displaying device, and can also display video information from a tablet, a smartphone, or the like. Furthermore, if a stack PC or the like is connected, it is also possible to provide capabilities such as calculation processing and video analysis processing.

[0017] FIG. 2 is a diagram showing an example of the main configuration and the retroreflective unit configuration of a spatial floating image display device according to an embodiment of the present invention. Using FIG. 2, the configuration of the spatial floating image display device will be described more specifically. As shown in FIG. 2(A), in an oblique direction of a transparent member 100 such as glass, a display device 1 that diverges image light of a specific polarization at a sandwiching angle is provided. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light of a specific polarization having a sandwiching diffusion characteristic.

[0018] The image light of a specific polarization from the display device 1 is reflected by a polarization separation member 101 having a film that selectively reflects the image light of the specific polarization provided on the transparent member 100 (in the figure, the polarization separation member 101 is formed in a sheet shape and adhered to the transparent member 100), and enters the retroreflective member 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflective member 2. The image light is polarization-converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, when entering and exiting the retroreflective member 2. Here, since the polarization separation member 101 that selectively reflects the image light of the specific polarization has the property of transmitting the polarization of the other polarization that has been polarization-converted, the image light of the specific polarization after polarization conversion passes through the polarization separation member 101. The image light that has passed through the polarization separation member 101 forms a spatial floating image 3, which is a real image, outside the transparent member 100.

[0019] Note that the light forming the spatial floating image 3 is a set of light rays that converge from the retroreflective member 2 to the optical image of the spatial floating image 3, and these light rays continue to travel even after passing through the optical image of the spatial floating image 3. Therefore, the spatial floating image 3 is an image with high directivity, unlike the diffused image light formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2, when the user views from the direction of arrow A, the spatial floating image 3 is viewed as a bright image. However, when another person views from the direction of arrow B, the spatial floating image 3 cannot be viewed as an image at all. This characteristic is very suitable for use in a system that displays an image that requires high security or an image with high confidentiality that needs to be concealed from a person facing the user.

[0020] Depending on the performance of the retroreflective member 2, the polarization axes of the video light after reflection may become uneven. In this case, some of the video light with uneven polarization axes is reflected by the above-described polarization separation member 101 and returns to the display device 1. This light may be retroreflected on the video display surface of the liquid crystal display panel 11 constituting the display device 1, generating a ghost image and degrading the image quality of the floating image in space.

[0021] Therefore, in this embodiment, an absorption type polarizing plate 12 is provided on the video display surface of the display device 1. By allowing the video light emitted from the display device 1 to pass through the absorption type polarizing plate 12 and absorbing the reflected light returning from the polarization separation member 101 with the absorption type polarizing plate 12, the above-described retroreflection can be suppressed. Thereby, it is possible to prevent the degradation of the image quality due to the ghost image of the floating image in space.

[0022] The above-described polarization separation member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects a specific polarization wave.

[0023] Next, FIG. 2(B) shows the surface shape of the retroreflective member used in this study, which is a typical retroreflective member 2, manufactured by Nippon Carbide Kogyo Co., Ltd. The light rays incident inside the regularly arranged hexagonal prisms are reflected by the wall surfaces and bottom surfaces of the hexagonal prisms and exit as retroreflected light in the direction corresponding to the incident light, displaying a floating image in space that is a real image based on the video displayed on the display device 1.

[0024] The resolution of this floating image in space depends greatly on the outer shape D and pitch P of the retroreflective portion of the retroreflective member 2 shown in FIG. 2(B) in addition to the resolution of the liquid crystal display panel 11. For example, when using a 7-inch WUXGA (1920×1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is about 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch is 300 μm, for example, one pixel of the floating image in space corresponds to 300 μm. Therefore, the effective resolution of the floating video image is reduced to about 1 / 3.

[0025] Therefore, in order to make the resolution of the floating image in space equivalent to that of the display device 1, it is desirable to make the diameter and pitch of the retroreflective portion approach one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moiré due to the retroreflective member and the pixels of the liquid crystal display panel, it is advisable to design the pitch ratio of each to be outside an integer multiple of one pixel. Also, the shape should be arranged so that none of the sides of the retroreflective portion overlap any of the sides of one pixel of the liquid crystal display panel.

[0026] On the other hand, in order to manufacture the retroreflective member at low cost, it is advisable to use a roll press method for molding. Specifically, it is a method of aligning the retroreflective portions and shaping them on a film. The inverse shape of the shape to be shaped is formed on the roll surface, an ultraviolet curable resin is applied on the fixing base material, and the necessary shape is formed by passing it between the rolls, irradiated with ultraviolet rays and cured to obtain the retroreflective member 2 of the desired shape.

[0027] <<Installation Method of Floating Image Display Device in Space>> Next, the installation method of the floating image display device in space will be described. The floating image display device in space can freely change the installation method according to the usage form. Fig. 3A is a diagram showing an example of the installation method of the floating image display device in space. The floating image display device shown in Fig. 3A is installed horizontally with the surface on the side where the floating image 3 in space is formed facing upward. That is, in Fig. 3A, the floating image display device is installed with the transparent member 100 facing upward, and the floating image 3 in space is formed above the floating image display device.

[0028] Fig. 3B is a diagram showing another example of the installation method of the floating image display device in space. The floating image display device shown in Fig. 3B is installed vertically with the surface on the side where the floating image 3 in space is formed facing sideward (in the direction of the user 230). That is, in Fig. 3B, the floating image display device is installed with the transparent member 100 facing sideward, and the floating image 3 in space is formed sideward (in the direction of the user 230) of the floating image display device.

[0029] <<Configuration of Floating Image Display Device in Space>> Next, the configuration of the spatial floating image display device 1000 will be described. FIG. 3C is a block diagram showing an example of the internal configuration of the spatial floating image display device 1000.

[0030] The spatial floating image display device 1000 includes a retroreflective portion 1101, an image display portion 1102, a light guide 1104, a light source 1105, a power supply 1106, an operation input portion 1107, a nonvolatile memory 1108, a memory 1109, a control portion 1110, an image signal input portion 1131, an audio signal input portion 1133, a communication portion 1132, an air operation detection sensor 1351, an air operation detection portion 1350, an audio output portion 1140, an image control portion 1160, a storage portion 1170, an imaging portion 1180, and the like.

[0031] Each component of the spatial floating image display device 1000 is arranged in a housing 1190. Note that the imaging portion 1180 and the air operation detection sensor 1351 shown in FIG. 3C may be provided outside the housing 1190.

[0032] The retroreflective portion 1101 in FIG. 3C corresponds to the retroreflective member 2 in FIG. 2. The retroreflective portion 1101 retroreflects the light modulated by the image display portion 1102. Among the reflected light from the retroreflective portion 1101, a spatial floating image 3 is formed by the light output to the outside of the spatial floating image display device 1000.

[0033] The image display portion 1102 in FIG. 3C corresponds to the liquid crystal display panel 11 in FIG. 2. The light source 1105 in FIG. 3C corresponds to the light source device 13 in FIG. 2. Then, the image display portion 1102, the light guide 1104, and the light source 1105 in FIG. 3C correspond to the display device 1 in FIG. 2.

[0034] The video display unit 1102 is a display unit that generates a video by modulating the transmitted light based on the input video signal under the control of the video control unit 1160 described later. The video display unit 1102 corresponds to the liquid crystal display panel 11 in FIG. 2. As the video display unit 1102, for example, a transmissive liquid crystal panel is used. Also, as the video display unit 1102, for example, a reflective liquid crystal panel or a DMD (Digital Micromirror Device: registered trademark) panel that modulates the reflected light may be used.

[0035] The light source 1105 generates light for the video display unit 1102 and is a solid light source such as an LED light source or a laser light source. The power supply 1106 converts the externally input AC current into a DC current and supplies power to the light source 1105. Also, the power supply 1106 supplies the necessary DC current to each part within the spatial floating video display device 1000.

[0036] The light guide 1104 guides the light generated by the light source 1105 and irradiates the video display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be referred to as the backlight of the video display unit 1102. Various methods are conceivable for the combination of the light guide 1104 and the light source 1105. Specific configuration examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0037] The air operation detection sensor 1351 is a sensor that detects the operation of the spatial floating video 3 by the finger of the user 230. The air operation detection sensor 1351 senses, for example, a range that overlaps with the entire display range of the spatial floating video 3. Note that the air operation detection sensor 1351 may sense only a range that overlaps with at least a part of the display range of the spatial floating video 3.

[0038] Specific examples of the air operation detection sensor 1351 include distance sensors using invisible light such as infrared rays, invisible light lasers, ultrasonic waves, etc. Further, the air operation detection sensor 1351 may be configured by combining a plurality of sensors in a plurality of combinations so as to be able to detect coordinates in a two-dimensional plane. Further, the air operation detection sensor 1351 may be composed of a ToF (Time of Flight) type LiDAR (Light Detection and Ranging) or an image sensor.

[0039] The air operation detection sensor 1351 only needs to be able to perform sensing for detecting a touch operation or the like by the user's finger on the object displayed as the spatial floating image 3. Such sensing can be performed using existing technologies.

[0040] The air operation detection unit 1350 acquires a sensing signal from the air operation detection sensor 1351, and based on the sensing signal, determines the presence or absence of contact of the object of the spatial floating image 3 by the finger of the user 230, calculates the position (contact position) where the finger of the user 230 contacts the object, and so on. The air operation detection unit 1350 is composed of a circuit such as an FPGA (Field Programmable Gate Array), for example. Further, some functions of the air operation detection unit 1350 may be realized by software by a spatial operation detection program executed by the control unit 1110, for example.

[0041] The air operation detection sensor 1351 and the air operation detection unit 1350 may be incorporated in the spatial floating image display device 1000, but may be provided externally as a separate body from the spatial floating image display device 1000. When provided as a separate body from the spatial floating image display device 1000, the air operation detection sensor 1351 and the air operation detection unit 1350 are configured to be able to transmit information and signals to the spatial floating image display device 1000 via a wired or wireless communication connection path or a video signal transmission path.

[0042] Further, the air operation detection sensor 1351 and the air operation detection unit 1350 may be provided separately. Thereby, it is possible to construct a system in which a spatial floating video display device 1000 without an air operation detection function is used as a main body and only the air operation detection function can be added as an option. Also, only the air operation detection sensor 1351 may be separate, and the air operation detection unit 1350 may be built into the spatial floating video display device 1000. When it is desired to arrange the air operation detection sensor 1351 more freely with respect to the installation position of the spatial floating video display device 1000, etc., there is an advantage in a configuration in which only the air operation detection sensor 1351 is separate.

[0043] The imaging unit 1180 is a camera having an image sensor, and images a space near the spatial floating video 3 and / or the face, arm, finger, etc. of the user 230. A plurality of imaging units 1180 may be provided. By using a plurality of imaging units 1180 or by using an imaging unit with a depth sensor, the air operation detection unit 1350 can be assisted during the detection process of a touch operation on the spatial floating video 3 by the user 230. The imaging unit 1180 may be provided separately from the spatial floating video display device 1000. When the imaging unit 1180 is provided separately from the spatial floating video display device 1000, it may be configured to transmit an imaging signal to the spatial floating video display device 1000 via a wired or wireless communication connection path or the like.

[0044] For example, when the air operation detection sensor 1351 is configured as an object intrusion sensor that detects the presence or absence of an object's intrusion into a plane (intrusion detection plane) including the display surface of the spatial floating video 3, information such as how far an object (e.g., a user's finger) that has not intruded into the intrusion detection plane is from the intrusion detection plane or how close the object is to the intrusion detection plane may not be detectable by the air operation detection sensor 1351.

[0045] In such a case, by using information such as the depth calculation information of the object based on the captured images of the plurality of imaging units 1180 and the depth information of the object by the depth sensor, the distance between the object and the intrusion detection plane can be calculated. Then, these pieces of information and various information such as the distance between the object and the intrusion detection plane are used for various display controls for the spatial floating image 3.

[0046] Further, without using the air operation detection sensor 1351, the air operation detection unit 1350 may detect a touch operation on the spatial floating image 3 by the user 230 based on the captured image of the imaging unit 1180.

[0047] Further, the imaging unit 1180 may capture the face of the user 230 who operates the spatial floating image 3, and the control unit 1110 may perform identification processing of the user 230. Also, in order to determine whether there is someone standing around or behind the user 230 who operates the spatial floating image 3 and whether the person is peeping at the operation of the user 230 on the spatial floating image 3, the imaging unit 1180 may capture a range including the user 230 who operates the spatial floating image 3 and the peripheral area of the user 230.

[0048] The operation input unit 1107 is, for example, an operation button or a light receiving unit of a remote controller, and inputs a signal for an operation different from the air operation (touch operation) by the user 230. Separately from the above-mentioned user 230 who performs a touch operation on the spatial floating image 3, the operation input unit 1107 may be used, for example, by an administrator to operate the spatial floating image display device 1000.

[0049] The video signal input unit 1131 connects to an external video output device and inputs video data. The audio signal input unit 1133 connects to an external audio output device and inputs audio data. The audio output unit 1140 can perform audio output based on the audio data input to the audio signal input unit 1133. Also, the audio output unit 1140 may output built-in operation sounds or error warning sounds.

[0050] The non-volatile memory 1108 stores various data used in the spatial floating image display device 1000. The data stored in the non-volatile memory 1108 includes, for example, data for various operations to be displayed in the spatial floating image 3, display icons, data of objects for the user to operate, layout information, and the like. The memory 1109 stores video data to be displayed as the spatial floating image 3, control data of the device, and the like.

[0051] The control unit 1110 controls the operations of each connected unit. Further, the control unit 1110 may perform arithmetic processing based on information acquired from each unit within the spatial floating image display device 1000 in cooperation with the program stored in the memory 1109. The communication unit 1132 communicates with external devices, external servers, etc. via a wired or wireless interface. Through the communication via the communication unit 1132, various data such as video data, image data, and audio data are transmitted and received.

[0052] The storage unit 1170 is a storage device that records various data and various information such as video data, image data, and audio data. For example, various information such as various data such as video data, image data, and audio data may be recorded in the storage unit 1170 in advance when the product is shipped. Further, the storage unit 1170 may record various information such as various data such as video data, image data, and audio data acquired from external devices or external servers via the communication unit 1132.

[0053] The video data, image data, etc. recorded in the storage unit 1170 are output as the spatial floating image 3 via the video display unit 1102 and the retroreflective unit 1101. Video data, image data, etc. such as display icons and objects for the user to operate, which are displayed as the spatial floating image 3, are also recorded in the storage unit 1170.

[0054] Layout information such as display icons and objects displayed as the spatial floating image 3, various metadata information about the objects, etc. are also recorded in the storage unit 1170. The audio data recorded in the storage unit 1170 is output as audio from, for example, the audio output unit 1140.

[0055] The video control unit 1160 performs various controls on the video signal input to the video display unit 1102. The video control unit 1160 performs, for example, control of video switching such as which video signal among the video signals stored in the memory 1109 and the video signals (video data) input to the video signal input unit 1131 is to be input to the video display unit 1102.

[0056] Further, the video control unit 1160 may generate a superimposed video signal by superimposing the video signal stored in the memory 1109 and the video signal input from the video signal input unit 1131, and perform control to form a composite video as the spatial floating image 3 by inputting the superimposed video signal to the video display unit 1102.

[0057] Further, the video control unit 1160 may perform control to perform image processing on the video signal input from the video signal input unit 1131, the video signal stored in the memory 1109, etc. Examples of the image processing include scaling processing such as enlarging, reducing, and deforming an image, brightness adjustment processing for changing the brightness, contrast adjustment processing for changing the contrast curve of the image, and Retinex processing for decomposing an image into light components and changing the weighting for each component.

[0058] Further, the video control unit 1160 may perform special effect video processing or the like for assisting the in-air operation (touch operation) of the user 230 on the video signal input to the video display unit 1102. The special effect video processing is performed based on, for example, the detection result of the touch operation of the user 230 by the in-air operation detection unit 1350 and the captured image of the user 230 by the imaging unit 1180.

[0059] As described so far, the spatial floating image display device 1000 is equipped with various functions. However, the spatial floating image display device 1000 does not necessarily need to have all of these functions, and any configuration may be used as long as it has the function of forming the spatial floating image 3.

[0060] <Spatial floating image display device 2> FIG. 4 is a diagram showing another example of the main configuration of a spatial floating image display device according to an embodiment of the present invention. The display device 1 includes a liquid crystal display panel 11 which is an image display element, and a light source device 13 that generates light of a specific polarization having a diffusion characteristic with a sandwiching angle. The display device 1 is composed of, for example, a small liquid crystal display panel with a screen size of about 5 inches to a large liquid crystal display panel exceeding 80 inches. The folding mirror 22 has a transparent member 100 as a substrate. On the surface of the transparent member 100 on the side of the display device 1, a polarization separation member 101 that selectively reflects image light of a specific polarization such as a reflective polarizing plate is provided, and reflects the image light from the liquid crystal display panel 11 toward the retroreflector 2. Thereby, the folding mirror 22 has a function as a mirror. The image light of a specific polarization from the display device 1 is reflected by the polarization separation member 101 provided on the transparent member 100 (the sheet-like polarization separation member 101 is adhered in the figure) and enters the retroreflector 2. Instead of the polarization separation member 101, an optical film having polarization separation characteristics may be vapor-deposited on the surface of the transparent member 100.

[0061] A λ / 4 plate 21 is provided on the light incident surface of the retroreflector, and the polarization is converted by passing the image light twice, and a specific polarization is converted into the other polarization having a phase difference of 90°. Thereby, the image light after retroreflection is transmitted through the polarization separation member 101, and a spatial floating image 3 which is a real image is displayed outside the transparent member 100.

[0062] Here, in the above-described polarization separation member 101, the polarization axis becomes uneven due to retroreflection, so a part of the image light is reflected and returns to the display device 1. This light is reflected again on the image display surface of the liquid crystal display panel 11 constituting the display device 1, generating a ghost image and significantly degrading the image quality of the spatial floating image.

[0063] Therefore, in this embodiment, an absorption-type polarizing plate 12 may be provided on the video display surface of the display device 1. The video light emitted from the display device 1 is transmitted, and the reflected light from the above-described polarization separation member 101 is absorbed to prevent image quality degradation due to ghost images of the spatial floating image. Further, in order to reduce image quality degradation caused by sunlight or illumination light outside the set, it is preferable to provide an absorption-type polarizing plate 102 on the surface of the transparent member 100 on the video light transmission output side.

[0064] Next, as shown in FIG. 5, a plurality of sensors 44 having a TOF (Time of Fly) function are arranged in multiple layers so as to sense the relationship between the distance and position of the object and the sensor 44 with respect to the spatial floating video obtained by the above-described spatial floating video display device. In addition to the coordinates of the object in the plane direction, it is also possible to sense the coordinates in the depth direction, the moving direction of the object, and the moving speed. In order to read two-dimensional distance and position, a combination of an infrared light emitting part and a light receiving part is arranged linearly in a plurality, and the light from the light emitting point is irradiated onto the object and the reflected light is received by the light receiving part. The distance to the object is clarified by the product of the difference between the time of emission and the time of reception and the speed of light. Also, the coordinates on the plane can be read from the coordinates at the portion where the difference between the emission time and the reception time is the smallest by a plurality of light emitting parts and light receiving parts. As described above, three-dimensional coordinate information can also be obtained by combining the coordinates of the object in the plane (two-dimensional) and a plurality of the above-described sensors.

[0065] Furthermore, a method for obtaining a three-dimensional spatial floating video as the above-described spatial floating video display device will be described with reference to FIG. 6. FIG. 6 is an explanatory diagram of the principle of three-dimensional video display used in the spatial floating video display device. A horizontal lenticular lens is arranged in accordance with the pixels of the video display screen of the liquid crystal display panel 11 of the display device 1 shown in FIG. 4. As a result, as shown in FIG. 6, in order to display the motion parallax from three directions of the motion parallax P1, P2, P3 in the horizontal direction of the screen, the video from three directions is taken as one block for every three pixels, and the video information from three directions is displayed for each pixel. The emission direction of light is controlled by the action of the corresponding lenticular lens (indicated by vertical lines in FIG. 6) and separated and emitted in three directions. As a result, a three-parallax stereoscopic image can be displayed.

[0066] <Reflection-type polarizing plate> In the spatial floating image display device of this embodiment, the polarization separation member 101 is used to improve the contrast performance that determines the image quality of the image, compared with a general half mirror. As an example of the polarization separation member 101 of this embodiment, the characteristics of a reflection-type polarizing plate will be described. FIG. 7 is an explanatory diagram of a measurement system for evaluating the characteristics of a reflection-type polarizing plate. The transmittance characteristics and reflection characteristics with respect to the light incident angle from the direction perpendicular to the polarization axis of the reflection-type polarizing plate in FIG. 7 are shown in FIGS. 8 and 9 as V-AOI, respectively. Similarly, the transmittance characteristics and reflection characteristics with respect to the light incident angle from the horizontal direction with respect to the polarization axis of the reflection-type polarizing plate are shown in FIGS. 10 and 11 as H-AOI, respectively.

[0067] In the characteristic graphs of FIGS. 8 to 11 (each displayed in color), the values of the angles (deg) shown outside the right column are shown from top to bottom in the order of the highest values of the vertical axis, that is, the transmittance (%). For example, in FIG. 8, in the range where the horizontal axis indicates light with a wavelength of approximately 400 nm to 800 nm, the transmittance is the highest when the angle in the vertical (V) direction is 0 degrees (deg), and the transmittance decreases in the order of 10 degrees, 20 degrees, 30 degrees, and 40 degrees. Also, in FIG. 9, in the range where the horizontal axis indicates light with a wavelength of approximately 400 nm to 800 nm, the transmittance is the highest when the angle in the vertical (V) direction is 0 degrees (deg), and the transmittance decreases in the order of 10 degrees, 20 degrees, 30 degrees, and 40 degrees. Also, in FIG. 10, in the range where the horizontal axis indicates light with a wavelength of approximately 400 nm to 800 nm, the transmittance is the highest when the angle in the horizontal (H) direction is 0 degrees (deg), and the transmittance decreases in the order of 10 degrees and 20 degrees. Also, in FIG. 11, in the range where the horizontal axis indicates light with a wavelength of approximately 400 nm to 800 nm, the transmittance is the highest when the angle in the horizontal (H) direction is 0 degrees (deg), and the transmittance decreases in the order of 10 degrees and 20 degrees.

[0068] As shown in FIGS. 8 and 9, the reflective polarizing plate with a grid structure has reduced characteristics for light incident from a direction perpendicular to the polarization axis. For this reason, specifications along the polarization axis are desirable, and the light source of this embodiment that can emit the outgoing video light from the liquid crystal display panel at an included angle is an ideal light source. Similarly, for the characteristics in the horizontal direction, there is a reduction in characteristics for light incident obliquely. Considering the above characteristics, hereinafter, a configuration example of this embodiment will be described in which a light source that can emit the outgoing video light from the liquid crystal display panel at a larger included angle is used as the backlight of the liquid crystal display panel. Thereby, a high-contrast spatial floating image can be provided.

[0069] <Display device> Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes a light source device 13 that constitutes its light source together with a video display element 11 (liquid crystal display panel). In FIG. 12, the light source device 13 is shown as an exploded perspective view together with the liquid crystal display panel.

[0070] As shown by the arrow 30 in FIG. 12, this liquid crystal display panel (video display element 11) has a diffusing characteristic at an included angle with respect to the light from the light source device 13 which is a backlight device, that is, it receives an illumination light beam having characteristics similar to those of a laser beam with strong directivity (linear propagation) and with the polarization planes aligned in one direction. The liquid crystal display panel (video display element 11) modulates the received illumination light speed according to the input video signal. The modulated video light is reflected by the retroreflective member 2 and transmitted through the transparent member 100 to form a spatial floating image which is a real image (see FIG. 1).

[0071] In addition, in FIG. 12, the liquid crystal display panel 11 constituting the display device 1 further includes an optical direction conversion panel 54 for controlling the directivity characteristics of the emitted light beam from the light source device 13, and an included angle diffusion plate (not shown) as required. That is, polarizing plates are provided on both sides of the liquid crystal display panel 11, and video light of a specific polarization modulates the light intensity according to the video signal and is emitted (see arrow 30 in FIG. 12). Thus, a desired video is projected as light of a specific polarization with high directivity (linear propagation) through the optical direction conversion panel 54 toward the retroreflective member 2. After being reflected by the retroreflective member 2, it passes through and forms a floating video 3 in the eyes of a monitor outside the store (space). A protective cover 50 (see FIGS. 13 and 14) may be provided on the surface of the above-described optical direction conversion panel 54.

[0072] In this embodiment, in order to improve the utilization efficiency of the emitted light beam 30 from the light source device 13 and significantly reduce power consumption, in the display device 1 including the light source device 13 and the liquid crystal display panel 11, the light from the light source device 13 (see arrow 30 in FIG. 12) is projected toward the retroreflective member 2. After being reflected by the retroreflective member 2, the directivity can be controlled so that a floating video is formed at a desired position by a transparent sheet (not shown) provided on the surface of a transparent member 100 (such as a windshield 105). Specifically, this transparent sheet controls the imaging position of the floating video while imparting high directivity by optical components such as a Fresnel lens or a linear Fresnel lens. According to such a configuration, the video light from the display device 1 can efficiently reach an observer outside the show window 105 (for example, on the sidewalk) with high directivity (linear propagation) like laser light. As a result, it is possible to display a high-quality floating video with high resolution and significantly reduce the power consumption of the display device 1 including the LED element 201 of the light source device 13.

[0073] <Example 1 of Display Device> FIG. 13 shows an example of the specific configuration of the display device 1. In FIG. 13, a liquid crystal display panel 11 and a light direction conversion panel 54 are arranged on the light source device 13 of FIG. 12. This light source device 13 is formed of, for example, plastic on the case shown in FIG. 12, and houses an LED element 201 and a light guide 203 inside. On the end face of the light guide 203, as shown in FIG. 12 and the like, in order to convert the divergent light from each LED element 201 into a substantially parallel light beam, it has a shape in which the cross-sectional area gradually increases toward the light receiving part, and has an action such that the divergence angle gradually decreases by total internal reflection a plurality of times when propagating inside, and a lens shape is provided. On the upper surface of the display device 1, a liquid crystal display panel 11 constituting such a display device 1 is attached. Further, on one side surface (the left end face in this example) of the case of the light source device 13, an LED (Light Emitting Diode) element 201 which is a semiconductor light source and an LED substrate 202 on which its control circuit is mounted are attached, and a heat sink which is a member for cooling the heat generated by the LED element and the control circuit may be attached to the outer surface of the LED substrate 202.

[0074] In addition, a frame (not shown) of a liquid crystal display panel attached to the upper surface of the case of the light source device 13 is configured by attaching the liquid crystal display panel 11 attached to the frame, and further, an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11. That is, the liquid crystal display panel 11, which is an image display element, together with the LED element 201, which is a solid light source, modulates the intensity of transmitted light based on a control signal from a control circuit (not shown) that constitutes the electronic device to generate a display image. At this time, since the generated image light has a narrow diffusion angle and only a specific polarization component, a novel image display device close to a surface-emitting laser image source driven by an image signal can be obtained. At present, it is technically and safety-impossible to obtain a laser light beam of the same size as the image obtained by the above-described display device 1 using a laser device. Therefore, in this embodiment, for example, light close to the above-described surface-emitting laser image light is obtained from a light beam from a general light source including an LED element.

[0075] Subsequently, the configuration of the optical system housed in the case of the light source device 13 will be described in detail with reference to FIG. 14 together with FIG. 13.

[0076] Since FIGS. 13 and 14 are cross-sectional views, only one of the plurality of LED elements 201 constituting the light source is shown, and these are converted into substantially collimated light by the shape of the light receiving end face 203a of the light guide 203. For this reason, the light receiving part of the light guide end face and the LED element are attached while maintaining a predetermined positional relationship.

[0077] Incidentally, each of these light guides 203 is formed of a light-transmissive resin such as acrylic. Although not shown in FIGS. 13 and 14, the light-receiving surface of the light guide 203 on one end side has, for example, an outer peripheral surface in a conical convex shape obtained by rotating a parabolic cross section, and a concave portion having a convex portion (i.e., a convex lens surface) is formed in the central region on the top side of such an outer peripheral surface. Further, in the central region of the flat portion on the other end side of the light guide 203, there is a convex lens surface protruding outward (or a concave lens surface recessed inward may also be used). These configurations will be described later in the description of FIG. 16 and the like. Incidentally, the outer shape of the light-receiving portion of the light guide to which the LED element 201 is attached has a parabolic surface shape forming a conical outer peripheral surface, and is set within a range of an angle capable of totally reflecting the light emitted from the LED element in the peripheral direction inside thereof, or a reflecting surface is formed.

[0078] On the other hand, the LED elements 201 are respectively arranged at predetermined positions on the surface of the LED substrate 202, which is the circuit board thereof. This LED substrate 202 is arranged and fixed with respect to the LED collimator (light-receiving end face 203a) such that the LED elements on its surface are respectively positioned at the central portions of the above-described concave portions.

[0079] According to such a configuration, depending on the shape of the light-receiving end face 203a of the light guide 203, the light emitted from the LED element 201 can be taken out as substantially parallel light, and it becomes possible to improve the utilization efficiency of the generated light.

[0080] As described above, the light source device 13 is configured by attaching a light source unit in which a plurality of LED elements 201, which are light sources, are arranged on a light-receiving end face 203a, which is a light-receiving part provided on the end face of the light guide 203. Then, the light source device 13 makes the diverging light beam from the LED element 201 into substantially parallel light by the lens shape of the light-receiving end face 203a on the end face of the light guide, and guides it inside the light guide 203 (in a direction parallel to the drawing), as indicated by the arrow, and emits it toward the liquid crystal display panel 11 arranged substantially parallel to the light guide 203 (in a direction perpendicular to the front from the drawing) by the light beam direction conversion means 204. By optimizing the distribution (density) of the light beam direction conversion means 204 according to the shape inside or on the surface of the light guide, the uniformity of the light beam incident on the liquid crystal display panel 11 can be controlled.

[0081] The above-described light beam direction conversion means 204 emits the light beam propagating inside the light guide toward the liquid crystal display panel 11 arranged substantially parallel to the light guide 203 (in a direction perpendicular to the front from the drawing) by the shape of the surface of the light guide or by providing, for example, portions with different refractive indexes inside the light guide. At this time, if the relative luminance ratio when comparing the luminance of the center of the screen and the peripheral portion of the screen in a state where the liquid crystal display panel 11 is facing the center of the screen and the viewing point is placed at the same position as the diagonal dimension of the screen is 20% or more, there is no practical problem, and if it exceeds 30%, the characteristics are even more excellent.

[0082] Note that FIG. 13 is a cross-sectional layout view for explaining the configuration and operation of the light source of the present embodiment for polarization conversion in the light source device 13 including the above-described light guide 203 and LED element 201. In FIG. 13, the light source device 13 is composed of, for example, a light guide 203 provided with light beam direction conversion means 204 on its surface or inside formed of plastic or the like, an LED element 201 as a light source, a reflection sheet 205, a retardation plate 206, a lenticular lens, etc., and a liquid crystal display panel 11 equipped with polarizing plates on its upper surface on the light source light incident surface and the video light emission surface is attached.

[0083] Also, a film or sheet-like reflective polarizing plate 49 is provided on the light source light incident surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one-sided polarized wave (for example, P wave) 212 among the natural light beams 210 emitted from the LED element 201, reflects it with the reflection sheet 205 provided on one surface (the lower side in the figure) of the light guide 203, and then directs it back toward the liquid crystal display panel 11. Therefore, a retardation plate (λ / 4 plate) is provided between the reflection sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the light is reflected by the reflection sheet 205 and passed through twice to convert the reflected light beam from P polarization to S polarization, thereby improving the utilization efficiency of the light source light as video light. The video light beam whose light intensity is modulated by the video signal in the liquid crystal display panel 11 (arrow 213 in FIG. 13) enters the retroreflective member 2, and as shown in FIG. 1, after reflection, it passes through the windshield 105 to obtain a spatial floating image that is a real image inside or outside the store (space).

[0084] FIG. 14 is a cross-sectional layout diagram for explaining the configuration and operation of the light source of this embodiment for polarization conversion in the light source device 13 including the light guide 203 and the LED element 201, similar to FIG. 13. Similarly, the light source device 13 is composed of, for example, a light guide 203 provided with a light beam direction conversion means 204 on its surface or inside formed of plastic or the like, an LED element 201 as a light source, a reflection sheet 205, a retardation plate 206, a lenticular lens, and the like. A liquid crystal display panel 11 having polarizing plates on the light source light incident surface and the video light output surface is attached as a video display element on the upper surface of the light source device 13.

[0085] Further, a film or sheet-like reflective polarizing plate 49 is provided on the light source light incident surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and one-sided polarized wave (for example, S wave) 211 of the natural light beam 210 emitted from the LED light source 201 is selectively reflected, reflected by the reflection sheet 205 provided on one surface (the lower side in the figure) of the light guide 203, and then directed back toward the liquid crystal display panel 11. A retardation plate (λ / 4 plate) is provided between the reflection sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the light is reflected by the reflection sheet 205 and passed through twice to convert the reflected light beam from S polarization to P polarization, thereby improving the utilization efficiency of the light source light as image light. The video light beam whose light intensity is modulated by the video signal in the liquid crystal display panel 11 (arrow 214 in FIG. 14) enters the retroreflective member 2, and as shown in FIG. 1, after reflection, it passes through the windshield 105 to obtain a spatial floating image that is a real image inside or outside the store (space).

[0086] In the light source device shown in FIGS. 13 and 14, in addition to the function of the polarizing plate provided on the light incident surface of the corresponding liquid crystal display panel 11, since the reflective polarizing plate reflects one-sided polarization components, the theoretically obtainable contrast ratio is the reciprocal of the cross transmittance of the reflective polarizing plate multiplied by the reciprocal of the cross transmittance obtained by the two polarizing plates attached to the liquid crystal display panel. As a result, high contrast performance can be obtained. Actually, it was experimentally confirmed that the contrast performance of the displayed image was improved by more than 10 times. As a result, a high-quality video comparable to that of the self-emitting organic EL was obtained.

[0087] <Example 2 of the display device> FIG. 15 shows another example of the specific configuration of the display device 1. The light source device 13 in FIG. 15 is the same as the light source device in FIGS. 17 and the like. This light source device 13 is configured by housing an LED, a collimator, a synthetic diffusion block, a light guide, etc. in a case made of, for example, plastic, and a liquid crystal display panel 11 is attached to the upper surface thereof. Further, on one side surface of the case of the light source device 13, LED (Light Emitting Diode) elements 14a and 14b, which are semiconductor light sources, and an LED substrate on which the control circuit thereof is mounted are attached, and on the outer surface of the LED substrate, a heat sink 103, which is a member for cooling the heat generated by the LED elements and the control circuit, is attached (see also FIGS. 17 and 18).

[0088] Further, on the liquid crystal display panel frame attached to the upper surface of the case, the liquid crystal display panel 11 attached to the frame, and further, an FPC (Flexible Printed Circuits) 403 (see FIG. 7) electrically connected to the liquid crystal display panel 11 and the like are attached and configured. That is, the liquid crystal display panel 11, which is a liquid crystal display element, together with the LED elements 14a and 14b, which are solid light sources, modulates the intensity of transmitted light based on a control signal from a control circuit (not shown here) that constitutes an electronic device, thereby generating a display image.

[0089] <Example 1 of the light source device of Example 2 of the display device> Subsequently, the configuration of the optical system such as the light source device housed in the case will be described in detail with reference to FIGS. 18(a) and (b) together with FIG. 17.

[0090] Figs. 17 and 18 show LEDs 14a and 14b that constitute a light source, and these are attached to predetermined positions with respect to LED collimators 15. Note that each of these LED collimators 15 is formed of a light-transmissive resin such as acrylic, for example. And as shown also in Fig. 18(b), this LED collimator 15 has an outer peripheral surface 156 of a conical convex shape obtained by rotating a parabolic cross-section. Further, at the central portion of the top of the LED collimator 15 (the side facing the LED substrate 102), there is a recess 153 in which a convex portion (i.e., a convex lens surface) 157 is formed. Also, at the central portion of the flat surface portion of the LED collimator 15 (the side opposite to the above-mentioned top), there is a convex lens surface (or it may be a concave lens surface recessed inward) 154 that protrudes outward. Note that the parabolic surface 156 forming the conical outer peripheral surface of the LED collimator 15 is set within an angular range capable of total reflection of the light emitted from the LEDs 14a and 14b in the peripheral direction inside it, or a reflecting surface is formed.

[0091] Also, the LEDs 14a and 14b are respectively arranged at predetermined positions on the surface of an LED substrate 102, which is their circuit board. This LED substrate 102 is arranged and fixed with respect to the LED collimator 15 such that the LEDs 14a or 14b on its surface are respectively positioned at the central portions of the recesses 153.

[0092] According to such a configuration, among the light emitted from the LED 14a or 14b, particularly the light radiated upward (in the right direction in the figure) from the central portion thereof is condensed by the two convex lens surfaces 157 and 154 forming the outer shape of the LED collimator 15 to become parallel light. Also, the light emitted in the peripheral direction from other portions is reflected by the parabolic surface forming the conical outer peripheral surface of the LED collimator 15 and similarly condensed to become parallel light. In other words, according to the LED collimator 15 that constitutes a convex lens at its central portion and forms a parabolic surface at its peripheral portion, almost all of the light generated by the LED 14a or 14b can be taken out as parallel light, and it becomes possible to improve the utilization efficiency of the generated light.

[0093] Note that a polarization conversion element 21 is provided on the light-emitting side of the LED collimator 15. As is clear from FIG. 18, this polarization conversion element 21 combines a columnar light-transmitting member (hereinafter referred to as a parallelogram column) having a parallelogram cross section and a columnar light-transmitting member (hereinafter referred to as a triangular column) having a triangular cross section, and is configured by arranging a plurality of them in an array parallel to a plane orthogonal to the optical axis of the parallel light from the LED collimator 15. Further, a polarization beam splitter (hereinafter abbreviated as "PBS film") 211 and a reflection film 212 are alternately provided at the interfaces between adjacent light-transmitting members arranged in this array. In addition, a λ / 2 phase plate 213 is provided on the emission surface from which the light that has entered the polarization conversion element 21 and passed through the PBS film 211 is emitted.

[0094] A rectangular composite diffusion block 16 shown also in FIG. 18(a) is further provided on the emission surface of this polarization conversion element 21. That is, the light emitted from the LED 14a or 14b becomes parallel light by the action of the LED collimator 15, enters the composite diffusion block 16, is diffused by the texture 161 on the emission side, and then reaches the light guide 17.

[0095] The light guide 17 is a member formed in a rod shape having a substantially triangular cross section (see FIG. 18(b)) from a light-transmitting resin such as acrylic. As is clear from FIG. 17, the light guide 17 includes a light guide light incident portion (surface) 171 facing the emission surface of the composite diffusion block 16 via the first diffusion plate 18a, a light guide light reflection portion (surface) 172 forming an inclined surface, and a light guide light emission portion (surface) 173 facing the liquid crystal display panel 11, which is a liquid crystal display element, via the second diffusion plate 18b.

[0096] In the light guide body light reflection part (surface) 172 of this light guide body 17, as shown also in FIG. 17 which is a partial enlarged view thereof, a number of reflecting surfaces 172a and connecting surfaces 172b are alternately formed in a sawtooth shape. And the reflecting surface 172a (the line segment rising to the upper right in the figure) forms an angle αn (n is a natural number, and in this example, for example, it is 1 to 130) with the horizontal plane indicated by the dashed-dotted line in the figure. As an example, here, αn is set to 43 degrees or less (however, 0 degrees or more).

[0097] The light guide body light incident part (surface) 171 is formed in a curved convex shape inclined toward the light source side. According to this, the parallel light from the emission surface of the composite diffusion block 16 is diffused and incident through the first diffusion plate 18a, and as is clear from the figure, it reaches the light guide body light reflection part (surface) 172 while being slightly bent (deflected) upward by the light guide body light incident part (surface) 171, and is reflected here and reaches the liquid crystal display panel 11 provided on the upper emission surface of the figure.

[0098] According to the display device 1 described in detail above, while further improving the light utilization efficiency and its uniform illumination characteristics, it is possible to manufacture it in a small size and at low cost including the modularized light source device of the S polarized light wave. In the above description, the polarization conversion element 21 has been described as being attached after the LED collimator 15, but the present invention is not limited thereto, and the same operations and effects can also be obtained by providing it in the optical path reaching the liquid crystal display panel 11.

[0099] Note that in the light guide body light reflection part (surface) 172, a number of reflecting surfaces 172a and connecting surfaces 172b are alternately formed in a sawtooth shape, and the illumination light beam is totally reflected on each reflecting surface 172a and goes upward. Further, in the light guide body light emission part (surface) 173, a sandwiching angle diffusion plate is provided and it is incident on the light direction conversion panel 54 that controls the directivity characteristics as a substantially parallel diffused light beam, and is incident on the liquid crystal display panel 11 from an oblique direction. In this embodiment, the light direction conversion panel 54 is provided between the light guide body emission part (surface) 173 and the liquid crystal display panel 11, but the same effect can also be obtained by providing the light direction conversion panel 54 on the emission surface of the liquid crystal display panel 11.

[0100] <Example 2 of the light source device of Example 2 of the display device> Regarding the configuration of the optical system such as the light source device 13, another example is shown in FIG. 19. In the example shown in FIG. 19, similar to the example shown in FIG. 18, a plurality (two in this example) of LEDs 14a and 14b that constitute the light source are shown, and these are attached to the LED collimator 15 at predetermined positions. Note that each of these LED collimators 15 is formed of a translucent resin such as acrylic.

[0101] And, similar to the example shown in FIG. 18, the LED collimator 15 shown in FIG. 19 has an outer peripheral surface 156 having a conical convex shape obtained by rotating a parabolic cross-section. Also, at the central portion at the top (top side) of the LED collimator 15, there is a concave portion 153 (see FIG. 18(b)) in which a convex portion (i.e., a convex lens surface) 157 is formed.

[0102] Also, at the central portion of the flat surface portion of the LED collimator 15, there is a convex lens surface (or it may be a concave lens surface recessed inward) 154 (see FIG. 18(b)) that protrudes outward. Note that the parabolic surface 156 that forms the conical outer peripheral surface of the LED collimator 15 is set within an angle range that enables total internal reflection of the light emitted from the LED 14a in the peripheral direction inside it, or a reflective surface is formed.

[0103] Also, the LEDs 14a and 14b are respectively arranged at predetermined positions on the surface of the LED substrate 102, which is their circuit board. This LED substrate 102 is arranged and fixed with respect to the LED collimator 15 such that the LEDs 14a or 14b on its surface are respectively located at the central portions of the concave portions 153.

[0104] According to such a configuration, among the light emitted from LED 14a or 14b by the above-described LED collimator 15, in particular, the light emitted upward (in the right direction of the figure) from the central portion thereof is condensed by the two convex lens surfaces 157 and 154 forming the outer shape of the LED collimator 15 to become parallel light. Further, the light emitted from other portions in the peripheral direction is reflected by the parabolic surface forming the conical outer peripheral surface of the LED collimator 15, and similarly, is condensed to become parallel light. In other words, according to the LED collimator 15 having a convex lens formed in the central portion thereof and a parabolic surface formed in the peripheral portion thereof, almost all of the light generated by the LED 14a or 14b can be taken out as parallel light, and the utilization efficiency of the generated light can be improved.

[0105] Note that a light guide 170 is provided on the light emission side of the LED collimator 15 via a first diffusion plate 18a. The light guide 170 is a member formed in a rod shape having a substantially triangular cross section (see FIG. 19(a)) by a light-transmissive resin such as acrylic. As is clear from FIG. 19(a), the light guide 170 includes a light guide light incident portion (surface) 171 facing the light emission surface of the diffusion block 16 via the first diffusion plate 18a, a light guide light reflection portion (surface) 172 forming an inclined surface, and a light guide light emission portion (surface) 173 facing the liquid crystal display panel 11, which is a liquid crystal display element, via a reflective polarizing plate 200.

[0106] If an object having a characteristic of reflecting P-polarized light (transmitting S-polarized light) is selected for this reflective polarizing plate 200, for example, P-polarized light among the natural light emitted from the LED, which is a light source, is reflected, passes through the λ / 4 plate 202 provided on the light guide light reflection portion 172 shown in FIG. 19(b), is reflected by the reflection surface 201, and is converted into S-polarized light by passing through the λ / 4 plate 202 again, so that all the light beams incident on the liquid crystal display panel 11 are unified into S-polarized light.

[0107] Similarly, if an object with the property of reflecting S-polarized light (transmitting P-polarized light) is selected as the reflective polarizing plate 200, the S-polarized light among the natural light emitted from the LED serving as the light source is reflected, passes through the λ / 4 plate 202 provided on the light reflecting portion 172 of the light guide shown in FIG. 19(b), is reflected by the reflecting surface 201, and is converted into P-polarized light by passing through the λ / 4 plate 202 again. All the light beams incident on the liquid crystal display panel 52 are unified into P-polarized light. Polarization conversion can also be achieved with the configuration described above.

[0108] <Example 3 of the display device> Subsequently, another example (Example 3 of the display device) of the specific configuration of the display device 1 will be described with reference to FIG. 16. The light source device of this display device 1 converts the divergent light beam (mixed with P-polarized light and S-polarized light) from the LED into a substantially parallel light beam by the collimator 18, and reflects the converted light beam toward the liquid crystal display panel 11 by the reflecting surface of the reflective light guide 304. Such reflected light is incident on the reflective polarizing plate 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizing plate 49 transmits light of a specific polarization state (for example, P-polarized light), and the transmitted polarized light is incident on the liquid crystal display panel 11. Here, light of polarization states other than the specific polarization state (for example, S-polarized light) is reflected by the reflective polarizing plate 49 and heads back to the reflective light guide 304 again.

[0109] The reflective polarizing plate 49 is installed at an inclination with respect to the liquid crystal display panel 11 so as not to be perpendicular to the principal ray of the light from the reflecting surface of the reflective light guide 304. Then, the principal ray of the light reflected by the reflective polarizing plate 49 is incident on the transmitting surface of the reflective light guide 304. The light incident on the transmitting surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304, passes through the λ / 4 plate 270 which is a retardation plate, and is reflected by the reflector 271. The light reflected by the reflector 271 passes through the λ / 4 plate 270 again and passes through the transmitting surface of the reflective light guide 304. The light passing through the transmitting surface of the reflective light guide 304 is incident on the reflective polarizing plate 49 again.

[0110] At this time, since the light that re-enters the reflective polarizing plate 49 has passed through the λ / 4 plate 270 twice, the polarization is converted into a polarization (for example, P polarization) that passes through the reflective polarizing plate 49. Therefore, the light whose polarization has been converted passes through the reflective polarizing plate 49 and enters the liquid crystal display panel 11. Note that regarding the polarization design related to polarization conversion, the polarization directions may be configured in reverse (reversing S polarization and P polarization) from the above description.

[0111] As a result, the light from the LED is aligned to a specific polarization (for example, P polarization), enters the liquid crystal display panel 11, is luminance-modulated according to the video signal, and a video is displayed on the panel surface. A plurality of LEDs constituting the light source are shown as in the above example (however, only one is shown in FIG. 16 due to the longitudinal section), and these are attached to the collimator 18 at predetermined positions.

[0112] Note that each collimator 18 is formed of a light-transmissive resin such as acrylic or glass. And this collimator 18 may have an outer peripheral surface having a conical convex shape obtained by rotating a parabolic cross-section. At the top of the collimator 18, it may have a concave portion formed with a convex portion (that is, a convex lens surface) at the central portion thereof. Also, at the central portion of the flat surface portion, it has a convex lens surface protruding outward (or a concave lens surface recessed inward may also be acceptable). Note that the parabolic surface forming the conical outer peripheral surface of the collimator 18 is set within an angle range capable of total reflection of the light emitted from the LED in the peripheral direction inside it, or a reflective surface is formed.

[0113] Note that the LEDs are respectively arranged at predetermined positions on the surface of the LED substrate 102 which is their circuit board. This LED substrate 102 is arranged and fixed with respect to the collimator 18 such that the LEDs on its surface are respectively located at the central portion of the top of the conical convex shape (the concave portion if there is a concave portion at the top).

[0114] According to such a configuration, among the light emitted from the LED, particularly the light emitted from the central portion thereof, the light is condensed by the convex lens surface forming the outer shape of the collimator 18 to become parallel light. Further, the light emitted from other portions in the peripheral direction is reflected by the parabolic surface forming the conical outer peripheral surface of the collimator 18, and similarly, is condensed to become parallel light. In other words, according to the collimator 18 having a convex lens formed in the central portion and a parabolic surface formed in the peripheral portion, almost all of the light generated by the LED can be taken out as parallel light, and the utilization efficiency of the generated light can be improved.

[0115] The above configuration is the same as the configuration of the light source device of the video display device shown in FIGS. 17, 18, etc. Further, the light converted into substantially parallel light by the collimator 18 shown in FIG. 16 is reflected by the reflection type light guide 304. Among the light, the light of a specific polarization wave passes through the reflection type polarizing plate 49 due to the action of the reflection type polarizing plate 49, and the light of the other polarization wave reflected by the action of the reflection type polarizing plate 49 passes through the light guide 304 again. The light is reflected by a reflector 271 located at a position opposite to the liquid crystal display panel 11 with respect to the reflection type light guide 304. At this time, the light is polarization-converted by passing through the λ / 4 plate 270, which is a retardation plate, twice. The light reflected by the reflector 271 passes through the light guide 304 again and enters the reflection type polarizing plate 49 provided on the opposite surface. Since the incident light has been polarization-converted, it passes through the reflection type polarizing plate 49 and enters the liquid crystal display panel 11 with the polarization directions aligned. As a result, since all the light of the light source can be utilized, the geometric optical utilization efficiency of the light is doubled. Further, since the polarization degree (extinction ratio) of the reflection type polarizing plate is also multiplied by the extinction ratio of the entire system, the contrast ratio of the entire display device is significantly improved by using the light source device of this embodiment. Note that by adjusting the surface roughness of the reflection surface of the reflection type light guide 304 and the surface roughness of the reflector 271, the light reflection diffusion angle at each reflection surface can be adjusted. The surface roughness of the reflection surface of the reflection type light guide 304 and the surface roughness of the reflector 271 may be adjusted for each design so that the uniformity of the light incident on the liquid crystal display panel 11 becomes more suitable.

[0116] Note that the quarter-wave plate 270, which is the phase difference plate in Fig. 16, does not necessarily need to have a phase difference of λ / 4 with respect to the polarized light incident perpendicularly to the quarter-wave plate 270. In the configuration of Fig. 16, any phase difference plate that changes the phase by 90° (λ / 2) when the polarized light passes through it twice is acceptable. The thickness of the phase difference plate may be adjusted according to the incident angle distribution of the polarized light.

[0117] <Example 4 of display device> Furthermore, another example (Example 4 of the display device) of the configuration of the optical system such as the light source device of the display device will be described with reference to FIG. 25. This is a configuration example in the case where a diffusion sheet is used instead of the reflective light guide 304 in the light source device of Example 3 of the display device. Specifically, two optical sheets (optical sheet 207A and optical sheet 207B) for converting the diffusion characteristics in the vertical and horizontal directions (not shown in the front-back direction of the figure) of the drawing are used on the light-emitting side of the light from the collimator 18, and the light from the collimator 18 is made to enter between the two optical sheets (diffusion sheets). These optical sheets may be a single sheet instead of two sheets. In the case of a single-sheet configuration, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes of the front and back surfaces of the single optical sheet. Also, a plurality of diffusion sheets may be used to share the functions. Here, in the example of FIG. 25, regarding the reflection and diffusion characteristics due to the front surface shape and back surface shape of the optical sheet 207A and the optical sheet 207B, it is advisable to optimize the design with the number of LEDs, the divergence angle from the LED substrate (optical element) 102, and the optical specifications of the collimator 18 as design parameters so that the surface density of the light beam emitted from the liquid crystal display panel 11 becomes uniform. That is, the diffusion characteristics are adjusted by the surface shapes of a plurality of diffusion sheets instead of the light guide. In the example of FIG. 25, the polarization conversion is performed in the same manner as in Example 3 of the display device described above. That is, in the example of FIG. 25, the reflective polarizing plate 49 may be configured to have the characteristic of reflecting S-polarized light (transmitting P-polarized light). In that case, among the light emitted from the LED which is the light source, the P-polarized light is transmitted, and the transmitted light enters the liquid crystal display panel 11. Among the light emitted from the LED which is the light source, the S-polarized light is reflected, and the reflected light passes through the retardation plate 270 shown in FIG. 25. The light that has passed through the retardation plate 270 is reflected by the reflecting surface 271. The light reflected by the reflecting surface 271 is converted into P-polarized light by passing through the retardation plate 270 again. The polarization-converted light passes through the reflective conversion plate 49 and enters the liquid crystal display panel 11.

[0118] Note that the quarter-wave plate 270 in Fig. 25 does not necessarily need to have a phase difference of λ / 4 with respect to the polarized light incident perpendicularly to the quarter-wave plate 270. In the configuration of Fig. 25, any wave plate that changes the phase by 90° (λ / 2) when the polarized light passes through it twice is acceptable. The thickness of the wave plate may be adjusted according to the incident angle distribution of the polarized light. Also, in Fig. 25, regarding the polarization design related to polarization conversion, the polarization directions may be configured in reverse (switching S-polarization and P-polarization) based on the above description.

[0119] The light emitted from the liquid crystal display panel 11 has similar diffusion characteristics in both the horizontal direction of the screen (displayed on the X-axis in Fig. 22(a)) and the vertical direction of the screen (displayed on the Y-axis in Fig. 22(b)) in a general TV application device. In contrast, for the diffusion characteristics of the light beam emitted from the liquid crystal display panel of this embodiment, for example, as shown in Example 1 of Fig. 22, the viewing angle at which the luminance becomes 50% of the front view (angle 0 degrees) is set to 13 degrees, which is 1 / 5 of the conventional 62 degrees. Similarly, for the viewing angle in the vertical direction, the reflection angle of the reflective light guide and the area of the reflection surface are optimized so that the upper viewing angle is suppressed to about 1 / 3 of the lower viewing angle with unevenness in the up and down directions. As a result, the amount of video light directed towards the monitoring direction is significantly improved compared to conventional liquid crystal TVs, and the luminance becomes 50 times or more.

[0120] Furthermore, for the viewing angle characteristics shown in Example 2 of Fig. 22, the viewing angle at which the luminance becomes 50% of the front view (angle 0 degrees) is set to 5 degrees, which is 1 / 12 of the conventional 62 degrees. Similarly, for the viewing angle in the vertical direction, the reflection angle of the reflective light guide and the area of the reflection surface are optimized so that the viewing angle is suppressed to about 1 / 12 of the conventional value with evenness in the up and down directions. As a result, the amount of video light directed towards the monitoring direction is significantly improved compared to conventional liquid crystal TVs, and the luminance becomes 100 times or more. As described above, by using the viewing angle as the included angle, the amount of light beam directed towards the monitoring direction can be concentrated, so the light utilization efficiency is significantly improved. As a result, even when using a conventional liquid crystal display panel for TVs, it is possible to achieve a significant increase in luminance with the same power consumption by controlling the light diffusion characteristics of the light source device, and a video display device corresponding to an information display system for a bright outdoor environment can be obtained.

[0121] When using a large liquid crystal display panel, the light around the screen is directed inward so that when the viewer faces the center of the screen, it heads in the direction of the viewer, thereby improving the overall uniformity of the screen brightness. Figure 20 shows the convergence angles of the long and short sides of the panel with the distance L from the panel to the viewer and the panel size (screen aspect ratio 16:10) as parameters. When monitoring the screen in a portrait orientation, the convergence angle can be set according to the short side. For example, when using a 22" panel in portrait orientation and the monitoring distance is 0.8 m, setting the convergence angle to 10 degrees can effectively direct the video light from the four corners of the screen towards the viewer.

[0122] Similarly, when using a 15" panel in portrait orientation and the monitoring distance is 0.8 m, setting the convergence angle to 7 degrees can effectively direct the video light from the four corners of the screen towards the viewer. As described above, by directing the video light around the screen towards the viewer who is in the optimal position to monitor the center of the screen according to the size of the liquid crystal display panel and whether it is used in portrait or landscape orientation, the overall uniformity of the screen brightness can be improved.

[0123] As a basic configuration, as shown in Figure 16 etc. mentioned above, a light beam with a sandwiching angle directivity characteristic is incident on the liquid crystal display panel 11 by a light source device, and the luminance is modulated according to the video signal, so that the video information displayed on the screen of the liquid crystal display panel 11 and the spatially floating image obtained by reflecting with a retroreflective member are displayed outdoors or indoors through a transparent member 100.

[0124] <Lenticular lens> To control the diffusion distribution of the video light from the liquid crystal display panel 11, a lenticular lens can be provided between the light source device 13 and the liquid crystal display panel 11, or on the surface of the liquid crystal display panel 11, and the lens shape can be optimized to control the emission characteristic in one direction. Furthermore, by arranging a microlens array in a matrix, the emission characteristics of the video light beam from the display device 1 in the X-axis and Y-axis directions can be controlled, and as a result, a video display device with desired diffusion characteristics can be obtained.

[0125] The operation of the lenticular lens will be described. By optimizing the lens shape, the lenticular lens can efficiently obtain a spatially floating image by transmitting or reflecting the light emitted from the display device 1 described above through the transparent member 100. That is, for the video light from the display device 1, two lenticular lenses are combined, or a sheet for controlling the diffusion characteristics is provided by arranging a microlens array in a matrix, so that in the X-axis and Y-axis directions, the luminance (relative luminance) of the video light can be controlled according to its reflection angle (the vertical direction is 0 degrees). In this embodiment, such a lenticular lens makes the luminance characteristics in the vertical direction steeper as shown in FIG. 22(b) compared to the prior art, and by changing the balance of the directivity characteristics in the up and down (positive and negative directions of the Y-axis) directions, the luminance (relative luminance) of the light due to reflection and diffusion is increased, so that the video light has a narrow diffusion angle (high straightness) and is only a specific polarization component, like the video light from a surface-emitting laser video source. This can suppress the ghost images generated by the retroreflective member when using a conventional video display device, and can be controlled so that the spatially floating image due to retroreflection can efficiently reach the eyes of the viewer.

[0126] Also, by the light source device described above, a video display device that emits light of a specific polarization that emits a video light beam that is nearly parallel to a specific direction can be realized by making the directivity characteristics in both the X-axis direction and the Y-axis direction have a large included angle with respect to the light emission diffusion characteristics (denoted as conventional in the figure) from a general liquid crystal display panel shown in FIGS. 22(a) and (b).

[0127] FIG. 21 shows an example of the characteristics of the lenticular lens adopted in this embodiment. In this example, in particular, the characteristics in the X direction (vertical direction) are shown. The characteristic O shows a luminance characteristic that is symmetric up and down with the peak in the light emission direction at an angle of around 30 degrees upward from the vertical direction (0 degrees). Also, the characteristics A and B in FIG. 21 show examples of characteristics in which the video light above the peak luminance is condensed at around 30 degrees to increase the luminance (relative luminance). Therefore, in these characteristics A and B, at angles exceeding 30 degrees, the luminance (relative luminance) of the light rapidly decreases compared to the characteristic O.

[0128] That is, according to the optical system including the above-described lenticular lens, when the video light beam from the display device 1 is made incident on the retroreflective member 2, the emission angle and the viewing angle of the video light aligned at the included angle by the light source device 13 can be controlled, and the degree of freedom in the installation of the retroreflective sheet (retroreflective member 2) can be greatly improved. As a result, the degree of freedom in the relationship of the imaging position of the spatial floating image that is reflected or transmitted through the transparent member 100 and imaged at a desired position can be greatly improved. As a result, it becomes possible to efficiently reach the eyes of an outdoor or indoor viewer as light having a narrow diffusion angle (high straight-ahead property) and only a specific polarization component. According to this, even if the intensity (luminance) of the video light from the video display device is reduced, the viewer can accurately recognize the video light and obtain information. In other words, by reducing the output of the video display device, it becomes possible to realize a spatial floating video display device with low power consumption.

[0129] <Touch operation assistance function> Next, the touch operation assistance function for the user will be described. First, the touch operation when the assistance function is not provided will be described. Here, the case where the user selects and touches one of two buttons (objects) will be described as an example, but the following content is also preferably applicable to, for example, ATMs in banks, ticket vending machines at stations, digital signage, and the like.

[0130] FIG. 26 is a diagram for explaining a display example and a touch operation of the spatial floating image display device 1000. The spatial floating image 3 shown in FIG. 26 includes a first button BUT1 displayed as "YES" and a second button BUT2 displayed as "NO". The user moves the finger 210 toward the spatial floating image 3 and touches the first button BUT1 or the second button BUT2 to select "YES" or "NO". In the examples of FIGS. 26 and 27 to 29, it is assumed that the first button BUT1 and the second button BUT2 are displayed in different colors. Here, in the region of the spatial floating image 3 other than the first button BUT1 and the second button BUT2, the image may not be displayed and may be made transparent. In that case, only the regions of the buttons (the display region of the first button BUT1 and the display region of the second button BUT2) where the range of the effect of the virtual shadow described later reaches are displayed. Therefore, in the following description, as a more preferable example, in the region of the spatial floating image 3 other than the first button BUT1 and the second button BUT2, for a wider region including the display region of the first button BUT1 and the display region of the second button BUT2, an image with a color or brightness different from that of the first button BUT1 and the second button BUT2 is displayed.

[0131] In a general video display device with a touch panel that is not a spatial floating video display device, the buttons selected by the user are composed of video buttons displayed on the touch panel surface. Therefore, the user can recognize the distance between the object (e.g., button) displayed on the touch panel surface and their own finger by visually checking the touch panel surface. However, in a spatial floating video display device, since the spatial floating video 3 floats in the air, it may not be easy for the user to recognize the depth of the spatial floating video 3. Therefore, in the touch operation on the spatial floating video 3, it may not be easy for the user to recognize the distance between the button displayed on the spatial floating video 3 and their own finger. Also, in a general video display device with a touch panel that is not a spatial floating video display device, the user can easily determine whether they have touched a button by the feeling when touching. However, in the touch operation on the spatial floating video 3, since there is no feeling when touching an object (e.g., button), the user may not be able to determine whether they have touched the object. Considering the above situation, in this embodiment, an auxiliary function for the touch operation on the user is provided.

[0132] In the following description, processing based on the position of the user's finger will be described, but the specific detection method of the position of the user's finger will be described later.

[0133] <<Auxiliary for Touch Operation Using Virtual Shadow (1)>> Figs. 27 to 29 are diagrams for explaining an example of an auxiliary method for touch operations using a virtual shadow. In the example of Figs. 27 to 29, it is assumed that the user touches the first button BUT1 to select "YES". The spatial floating video display device 1000 of the present embodiment assists the user's touch operation by displaying a virtual shadow on the display video of the spatial floating video 3. Here, "displaying a virtual shadow on the display video of the spatial floating video 3" means a video display process in which, for the video displayed as the spatial floating video 3, the luminance of the video signal is reduced for a partial region of the shape imitating a finger, so as to make it look as if a shadow is projected on the video. Specifically, this process may be performed by the calculation of the video control unit 1160 or the control unit 1110. In the virtual shadow display process, the luminance of the video signal for a partial region of the shape imitating a finger may be completely set to 0. However, rather than completely setting the luminance of the video signal for a partial region of the shape imitating a finger to 0, it is more suitable that the video is displayed with reduced luminance in that region because it is more naturally recognized as a shadow. In this case, in the virtual shadow display process, not only the luminance of the video signal for a partial region of the shape imitating a finger may be reduced, but also the chroma of the video signal may be reduced.

[0134] The spatial floating video 3 exists in the air where there is no physical contact surface, and in a normal environment, a finger shadow is not projected. However, according to the virtual shadow display process of the present embodiment, even in the air where a finger shadow is not originally projected, by making it look as if there is a shadow in the spatial floating video 3, it is possible to improve the user's perception of the depth of the spatial floating video 3 and the sense of reality of the spatial floating video 3.

[0135] FIG. 27 shows a state at a first point in time when a user attempts a touch operation on the first button BUT1 on the display surface 3a of the spatial floating image 3 with the finger 210. FIG. 28 shows a state at a second point in time when the finger 210 is closer to the spatial floating image 3 than in FIG. 27. FIG. 29 shows a state at a third point in time when the finger 210 touches the first button BUT1 on the display surface 3a of the spatial floating image 3. Also, (A) in FIGS. 27 to 29 shows a state when the display surface 3a of the spatial floating image 3 is viewed from the front (the normal direction of the display surface 3a), and (B) in FIGS. 27 to 29 shows a state when the display surface 3a of the spatial floating image 3 is viewed from the side (a direction parallel to the display surface 3a). In FIGS. 27 to 29, the x-direction is the horizontal direction on the display surface 3a of the spatial floating image 3, the y-direction is the direction orthogonal to the x-axis within the display surface 3a of the spatial floating image 3, and the z-direction is the normal direction of the display surface 3a of the spatial floating image 3 (the height direction with respect to the display surface 3a). In the explanatory diagrams of FIGS. 27 to 33, the spatial floating image 3 is illustrated as having a thickness in the depth direction for ease of explanation, but in reality, if the image display surface of the display device 1 is flat, the spatial floating image 3 is also flat and has no thickness in the depth direction. In this case, the spatial floating image 3 and the display surface 3a are in the same plane. In the description of this embodiment, the display surface 3a means a surface on which the spatial floating image 3 can be displayed, and the spatial floating image 3 means a portion where the spatial floating image is actually displayed.

[0136] In FIGS. 27, 28, and 29, the detection process of the finger 210 is performed using, for example, the captured image generated by the imaging unit 1180 or the sensing signal of the air operation detection sensor 1351. In the detection process of the finger 210, for example, the position (x coordinate, y coordinate) of the tip of the finger 210 on the display surface 3a of the spatial floating image 3, the height position (z coordinate) of the tip of the finger 210 with respect to the display surface 3a, etc. are detected. Here, the position (x coordinate, y coordinate) of the tip of the finger 210 on the display surface 3a of the spatial floating image 3 is the position coordinates on the display surface 3a of the intersection point of the perpendicular line from the tip of the finger 210 to the display surface 3a of the spatial floating image 3. Note that the height position of the tip of the finger 210 with respect to the display surface 3a is also depth information representing the depth of the finger 210 with respect to the display surface 3a. The arrangement of the imaging unit 1180 and the air operation detection sensor 1351 that detect the finger 210, etc. will be described in detail later.

[0137] At the first time point shown in FIG. 27, it is assumed that the finger 210 is at the position farthest from the display surface 3a of the spatial floating image 3 compared to the second time point shown in FIG. 28 and the third time point shown in FIG. 28. Let the distance (height position) between the tip of the finger 210 and the display surface 3a of the spatial floating image 3 at this time be dz1. That is, the distance dz1 indicates the height of the finger 210 with respect to the display surface 3a of the spatial floating image 3 in the z direction.

[0138] Note that the distance dz1 shown in FIG. 27 and the distance dz2 shown in FIG. 28 described later, etc., have the user side as the positive side with respect to the display surface 3a of the spatial floating image 3, and the side opposite to the user with respect to the display surface 3a as the negative side. That is, if the finger 210 is on the user side with respect to the display surface 3a, the distances dz1 and dz2 are positive values, and if the finger 210 is on the side opposite to the user with respect to the display surface 3a, the distances dz1 and dz2 are negative values.

[0139] In this embodiment, it is assumed that there is a virtual light source 1500 on the user side with respect to the display surface 3a of the spatial floating image 3. Here, the setting of the installation direction of the virtual light source 1500 may be actually stored as information in the non-volatile memory 1108 or the memory 1109 of the spatial floating image display device 1000. Also, the setting of the installation direction of the virtual light source 1500 may be a parameter that exists only in the design. Even when the setting of the installation direction of the virtual light source 1500 is a parameter that exists only in the design, the design installation direction of the virtual light source 1500 is uniquely determined from the relationship between the position of the user's finger and the display position of the virtual shadow described later. Here, in the examples of FIGS. 27 to 29, the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the right side of the display surface 3a as viewed from the user. Then, a virtual shadow 1510 imitating the shadow of the finger 210 formed by the light irradiated from the virtual light source 1500 is displayed on the spatial floating image 3. In the examples of FIGS. 27 to 29, the virtual shadow 1510 is displayed on the left side of the finger 210. This virtual shadow 1510 assists the user in performing a touch operation.

[0140] In the state of FIG. 27(B), compared with the states of FIG. 28(B) and FIG. 29(B), the tip of the finger 210 is the farthest in the distance in the normal direction from the display surface 3a of the spatial floating image 3. For this reason, in FIG. 27(A), the tip of the virtual shadow 1510 is formed at the position that is the farthest in the horizontal direction from the first button BUT1 to be touched, compared with the states of FIG. 28(A) and FIG. 29(A). Therefore, in FIG. 27(A), the horizontal distance between the tip of the finger 210 and the tip of the virtual shadow 1510 when the display surface 3a of the spatial floating image 3 is viewed from the front is the largest compared with the states of FIG. 28(A) and FIG. 29(A). In FIG. 27(A), the distance between the tip of the finger 210 and the tip of the virtual shadow 1510 in the horizontal direction of the display surface 3a of the spatial floating image 3 is denoted as dx1.

[0141] And in FIG. 28(B), finger 210 is closer to the spatial floating image 3 than in FIG. 27(B). Therefore, in FIG. 28(B), the distance dz2 in the normal direction between the tip of finger 210 and the display surface 3a of the spatial floating image 3 is smaller than dz1. At this time, in FIG. 28(A), the virtual shadow 1510 is displayed at a position where the distance in the horizontal direction between the tip of finger 210 and the tip of the virtual shadow 1510 on the display surface 3a of the spatial floating image 3 is dx2, which is smaller than dx1. That is, in the example of FIG. 28, since the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the right side of the display surface 3a as viewed from the user, in conjunction with the distance in the normal direction between the tip of finger 210 and the display surface 3a of the spatial floating image 3, the distance in the horizontal direction between the tip of finger 210 and the tip of the virtual shadow 1510 when the display surface 3a of the spatial floating image 3 is viewed from the front changes.

[0142] And when the tip of finger 210 and the tip of the virtual shadow 1510 come into contact, as shown in FIG. 29, the distance in the normal direction between the tip of finger 210 and the display surface 3a of the spatial floating image 3 becomes zero. At this time, the virtual shadow 1510 is displayed such that the distance between finger 210 and the virtual shadow 1510 in the horizontal direction on the display surface 3a of the spatial floating image 3 becomes zero. Thereby, the user can recognize that finger 210 has touched the display surface 3a of the spatial floating image 3. At this time, if the tip of finger 210 touches the area of the first button BUT1, the user can recognize that the first button BUT1 has been touched. That is, also in the example of FIG. 29, since the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the right side of the display surface 3a as viewed from the user, in conjunction with the distance in the normal direction between the tip of finger 210 and the display surface 3a of the spatial floating image 3, the distance in the horizontal direction between the tip of finger 210 and the tip of the virtual shadow 1510 when the display surface 3a of the spatial floating image 3 is viewed from the front has changed. That is, the display position of the tip of the virtual shadow 1510 is a position specified by the positional relationship between the position of the virtual light source 1500 and the position of the tip of the user's finger 210, and changes in conjunction with changes in the position of the tip of the user's finger 210.

[0143] According to the configuration and processing of "Assistance for Touch Operations Using Virtual Shadows (1)" described above, during a touch operation, the user can more preferably recognize the distance (depth) in the normal direction between the finger 210 and the display surface 3a of the spatial floating image 3 from the horizontal positional relationship on the display surface 3a of the spatial floating image 3 between the finger 210 and the virtual shadow 1510. Also, when the finger 210 touches an object (e.g., a button) that is the spatial floating image 3, the user can recognize that the object has been touched. Thereby, it becomes possible to provide a more suitable spatial floating image display device.

[0144] <<Assistance for Touch Operations Using Virtual Shadows (2)>> Next, as another example of the method for assisting touch operations using virtual shadows, the case where the virtual light source 1500 is provided on the left side of the display surface 3a as viewed from the user will be described. FIGS. 30 to 32 are diagrams for explaining another example of the method for assisting touch operations using virtual shadows. FIG. 30 corresponds to FIG. 27 and shows the state at the first point in time when the user attempts a touch operation on the first button BUT1 on the display surface 3a of the spatial floating image 3 with the finger 210. FIG. 31 corresponds to FIG. 28 and shows the state at the second point in time when the finger 210 is closer to the spatial floating image 3 than in FIG. 30. FIG. 32 corresponds to FIG. 29 and shows the state when the finger 210 touches the spatial floating image 3. Note that in FIGS. 30 to 32(B), for convenience of explanation, the figures are shown as viewed from the direction opposite to that of FIGS. 27 to 29(B).

[0145] In FIGS. 30 to 32, the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the left side of the display surface 3a as viewed from the user. Then, a virtual shadow 1510 that mimics the shadow of the finger 210 formed by the light irradiated from the virtual light source 1500 is displayed on the spatial floating image 3. In FIGS. 30 to 32, the virtual shadow 1510 is displayed on the right side of the finger 210. The virtual shadow 1510 assists the user with the touch operation.

[0146] In the state of Fig. 30(B), compared with the states of Fig. 31(B) and Fig. 32(B), the tip of the finger 210 is farthest from the display surface 3a of the spatial floating image 3 in the normal direction distance. In Fig. 30(B), the normal direction distance between the tip of the finger 210 and the display surface 3a of the spatial floating image 3 at this time is dz10. Also, in Fig. 30(A), the distance dx10 between the tip of the finger 210 and the tip of the virtual shadow 1510 in the horizontal direction of the display surface 3a of the spatial floating image 3 at this time.

[0147] In Fig. 31(B), the finger 210 is closer to the spatial floating image 3 than in Fig. 27(B). Therefore, in Fig. 31(B), the normal direction distance dz20 between the tip of the finger 210 and the display surface 3a of the spatial floating image 3 is smaller than dz10. At this time, in Fig. 31(A), the virtual shadow 1510 is displayed at a position where the distance between the tip of the finger 210 and the tip of the virtual shadow 1510 in the horizontal direction of the display surface 3a of the spatial floating image 3 is dx20, which is smaller than dx10. That is, in the example of Fig. 31, since the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the left side of the display surface 3a as viewed from the user, in conjunction with the normal direction distance between the tip of the finger 210 and the display surface 3a of the spatial floating image 3, the horizontal direction distance between the tip of the finger 210 and the tip of the virtual shadow 1510 when the display surface 3a of the spatial floating image 3 is viewed from the front changes.

[0148] Then, when the tip of the finger 210 touches the tip of the virtual shadow 1510, as shown in FIG. 32, the distance in the normal direction between the tip of the finger 210 and the display surface 3a of the spatial floating image 3 becomes zero. At this time, the virtual shadow 1510 is displayed such that the distance between the finger 210 and the virtual shadow 1510 in the horizontal direction on the display surface 3a of the spatial floating image 3 becomes zero. Thereby, the user can recognize that the finger 210 has touched the display surface 3a of the spatial floating image 3. At this time, if the tip of the finger 210 touches the area of the first button BUT1, the user can recognize that the first button BUT1 has been touched. That is, also in the example of FIG. 32, since the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the left side of the display surface 3a as viewed from the user, in conjunction with the distance in the normal direction between the tip of the finger 210 and the display surface 3a of the spatial floating image 3, the distance in the horizontal direction between the tip of the finger 210 and the tip of the virtual shadow 1510 when the display surface 3a of the spatial floating image 3 is viewed from the front has changed.

[0149] Also in the configuration and processing of "Assistance for touch operation using virtual shadow (2)" described above, the same effects as the configuration of FIGS. 27 to 29 can be obtained.

[0150] Here, when implementing the above-described processing of "Assistance for touch operation using virtual shadow (1)" and / or the processing of "Assistance for touch operation using virtual shadow (2)" in the spatial floating image display device 1000, there can be the following multiple implementation examples.

[0151] As a first implementation example, there is a method of implementing only "Assistance for Touch Operations Using Virtual Shadows (1)" in the spatial floating image display device 1000. In this case, since the virtual light source 1500 is on the user side with respect to the display surface 3a and is provided on the right side of the display surface 3a as viewed from the user, the virtual shadow 1510 is displayed on the left side of the tip of the user's finger 210 as viewed from the user. Therefore, if the user's finger 210 is a finger of the right hand, the visibility of the display of the virtual shadow 1510 is preferably not blocked by the user's right hand or right arm. Therefore, considering the tendency that there are many right-handed users statistically, even if only "Assistance for Touch Operations Using Virtual Shadows (1)" is implemented in the spatial floating image display device 1000, the probability that the display of the virtual shadow 1510 can be clearly seen is sufficiently high and suitable.

[0152] Also, as a first implementation example, it is also possible to implement both the process of "Assistance for Touch Operations Using Virtual Shadows (1)" and the process of "Assistance for Touch Operations Using Virtual Shadows (2)", and switch which process to perform according to whether the user performs the touch operation with the right hand or the left hand. In this case, it is possible to further increase the probability that the display of the virtual shadow 1510 can be clearly seen, and the convenience of the user is improved.

[0153] Specifically, when the user is performing a touch operation with the right hand, the virtual shadow 1510 is displayed on the left side of the finger 210 using the configuration shown in FIGS. 27 to 29. In this case, the visibility of the display of the virtual shadow 1510 is preferably not blocked by the user's right hand or right arm. On the other hand, when the user is performing a touch operation with the left hand, the virtual shadow 1510 is displayed on the right side of the finger 210 using the configuration shown in FIGS. 30 to 32. In this case, the visibility of the display of the virtual shadow 1510 is preferably not blocked by the user's left hand or left arm. As a result, whether the user performs a touch operation with the right hand or the left hand, the virtual shadow 1510 is displayed at a position where the user can easily view it, and the convenience of the user is improved.

[0154] Here, the determination of whether the touch operation is being performed with the right hand or the left hand may be made, for example, based on the captured image generated by the imaging unit 1180. For example, the control unit 1110 performs image processing on the captured image and detects the user's face, arm, hand, and fingers from the captured image. Then, the imaging unit 1180 estimates the user's posture or movement from the detected positions of these (face, arm, hand, and fingers), and determines whether the user is performing the touch operation with the right hand or the left hand. Note that in this determination, imaging of the face is not necessarily required as long as the vicinity of the center in the left - right direction of the user's body can be determined from other parts. Also, the above determination may be made based only on the arrangement of the arms, only on the arrangement of the hands, or on a combination of the arrangement of the arms and the arrangement of the hands. Further, in making these determinations, the arrangement of the face may be combined for the determination.

[0155] Note that in FIGS. 27 to 29 and FIGS. 30 to 32, a virtual shadow 1510 extending at an angle corresponding to the extending direction of the actual finger 210 is shown. The extending direction of the actual finger 210 may be calculated by imaging the finger with any of the imaging units described above. Here, the virtual shadow 1510 with the extending direction fixed at a predetermined angle may be displayed without reflecting the angle corresponding to the extending direction of the finger 210. Thereby, the load on the video control unit 1160 or the control unit 1110 for performing the display control of the virtual shadow 1510 is reduced.

[0156] For example, if the finger 210 is a finger of the right hand, it is natural for the user to extend the arm from the front - right side of the display surface 3a of the floating - in - space video 3 and try to touch the display surface 3a of the floating - in - space video 3 with the finger 210 pointing diagonally up - left toward the display surface 3a. Therefore, when the finger 210 is a finger of the right hand, if the virtual shadow of the finger indicated by the virtual shadow 1510 is configured to be displayed in a predetermined direction indicating the upper - right direction toward the display surface 3a of the floating - in - space video 3, a natural display can be achieved without reflecting the angle corresponding to the finger 210.

[0157] Also, for example, if the finger 210 is a finger of the left hand, it is natural for the user to extend the arm from the front left side of the display surface 3a of the spatial floating image 3 and try to touch the display surface 3a of the spatial floating image 3 with the finger 210 pointing diagonally up to the right toward the display surface 3a. Therefore, when the finger 210 is a finger of the left hand, if it is configured such that the shadow of the finger indicated by the virtual shadow 1510 is displayed in a predetermined direction indicating the upper left direction toward the display surface 3a of the spatial floating image 3, a natural display can be achieved without reflecting the angle corresponding to the finger 210.

[0158] In addition, when the user's finger 210 is on the side opposite to the user with respect to the display surface 3a of the spatial floating image 3, a display that allows the user to recognize that the finger 210 is on the back side of the spatial floating image 3 and cannot be touched may be provided. For example, a message may be displayed on the spatial floating image 3 to inform the user that the finger 210 is on the back side of the spatial floating image 3 and cannot be touched. Alternatively, for example, the virtual shadow 1510 may be displayed in a color different from the normal color, such as red. This makes it possible to more preferably prompt the user to return the finger 210 to an appropriate position.

[0159] <<An Example of the Setting Conditions of the Virtual Light Source>> Here, a method for setting the virtual light source 1500 will be described. FIG. 33 is a diagram for explaining the method for setting the virtual light source. Although FIG. 33 shows a situation where the user performs a touch operation with the left hand, the content described below is also preferably applicable when the user performs a touch operation with the right hand.

[0160] FIG. 33 shows a normal line L1 of the display surface 3a extending from the center point C of the display surface 3a of the spatial floating image 3 toward the user side, a line L2 connecting the virtual light source 1500 and the point C where the normal line L1 intersects the display surface 3a, and a virtual light source installation angle α defined by the angle between the normal line L1 and the line L2. In FIG. 33, for simplicity of explanation, the moment when the tip of the user's finger 210 is on the line L2 is shown.

[0161] Here, for the sake of simplicity in explanation, from FIGS. 27 to 33, the virtual light source 1500 is illustrated as being arranged at a position not too far from the display surface 3a of the spatial floating image 3 or the user's finger 210. Although the virtual light source 1500 may be set at such a position, the most preferable setting example is as follows. That is, it is desirable to set the distance between the virtual light source 1500 and the central point C of the display surface 3a of the spatial floating image 3 to infinity. The reason is as follows. If there is an object plane having a contact surface in the same coordinate system as the display surface 3a of the spatial floating image 3 in FIGS. 27 to 32, and the light source is the sun instead of the virtual light source, since the distance of the sun can be approximated as being almost infinite, with respect to the change in the distance (z direction) between the tip of the user's finger and the object plane, the position of the tip of the shadow of the user's finger on the actual object plane in the horizontal direction (x direction) changes linearly. Therefore, also in the setting of the virtual light source 1500 shown in FIGS. 27 to 33 of the present embodiment, the distance between the virtual light source 1500 and the central point C of the display surface 3a of the spatial floating image 3 is set to infinity, and with respect to the change in the distance (z direction) between the tip of the user's finger 210 and the display surface 3a of the spatial floating image 3, if the position of the tip of the virtual shadow 1510 in the spatial floating image 3 in the horizontal direction (x direction) is configured to change linearly, a virtual shadow that can be recognized more naturally by the user can be expressed.

[0162] If the virtual light source 1500 is set to be arranged at a position not too far from the display surface 3a of the spatial floating image 3 or the user's finger 210, with respect to the change in the distance (z direction) between the tip of the user's finger 210 and the display surface 3a of the spatial floating image 3, the position of the tip of the virtual shadow 1510 in the spatial floating image 3 in the horizontal direction (x direction) changes non-linearly, and the calculation for calculating the position of the tip of the virtual shadow 1510 in the horizontal direction (x direction) becomes somewhat complicated. On the contrary, if the distance between the virtual light source 1500 and the central point C of the display surface 3a of the spatial floating image 3 is set to infinity, with respect to the change in the distance (z direction) between the tip of the user's finger 210 and the display surface 3a of the spatial floating image 3, the position of the tip of the virtual shadow 1510 in the spatial floating image 3 in the horizontal direction (x direction) changes linearly, so there is also an effect that the calculation for calculating the position of the tip of the virtual shadow 1510 in the horizontal direction (x direction) can be simplified.

[0163] When the virtual light source setting angle α is small, from the user's perspective, the angle between the line connecting the virtual light source 1500 and the finger 210 and the normal line L1 cannot be increased. Therefore, the distance between the tip of the finger 210 and the tip of the virtual shadow 1510 in the horizontal direction (x-direction) of the display surface 3a of the spatial floating image 3 becomes short. As a result, the change in the position of the virtual shadow 1510 when the tip of the finger 210 performs a touch operation becomes difficult for the user to visually recognize, and there is a risk that the effect of the user's depth perception in the touch operation will decrease. To avoid this, it is desirable that the virtual light source 1500 be installed such that the angle between the line L2 connecting the virtual light source 1500 and the point C and the normal line L1 is, for example, 20° or more.

[0164] On the other hand, when the angle between the line connecting the virtual light source 1500 and the finger 210 and the normal line L1 approaches 90°, the distance between the tip of the finger 210 and the tip of the virtual shadow 1510 becomes extremely long. Then, the probability that the display position of the virtual shadow 1510 is outside the range of the spatial floating image 3 increases, and the probability that the virtual shadow 1510 cannot be displayed in the spatial floating image 3 increases. For this reason, it is desirable that the installation angle α of the virtual light source 1500 be 70° or less so that the angle between the line L2 connecting the virtual light source 1500 and the point C and the normal line L1 does not approach 90° too much.

[0165] That is, it is desirable that the virtual light source 1500 be installed at a position that does not approach too close to the plane including the normal line passing through the finger 210 and does not approach too close to the plane including the display surface 3a of the spatial floating image 3.

[0166] The spatial floating image display device 1000 of this embodiment can display a virtual shadow as described above. This results in a more physically natural visual effect than when a predetermined mark is superimposed on the image for assisting the user's touch operation. Therefore, the touch operation assistance technique using the display of the virtual shadow in the spatial floating image display device 1000 of this embodiment can provide a situation where the user can more naturally recognize the depth in the touch operation.

[0167] <<Method for Detecting Finger Position>> Next, a method for detecting the position of the finger 210 will be described. Hereinafter, a configuration for detecting the position of the finger 210 of the user 230 will be specifically described.

[0168] <<<Method for Detecting Finger Position (1)>>> FIG. 34 is a configuration diagram showing an example of a method for detecting the position of a finger. In the example shown in FIG. 34, the position of the finger 210 is detected using one imaging unit 1180 and one air operation sensor 1351. Note that all imaging units in the embodiments of the present invention have imaging sensors.

[0169] The first imaging unit 1180a (1180) is installed on the side opposite to the user 230 with respect to the spatial floating image 3. The first imaging unit 1180a may be installed in the housing 1190 as shown in FIG. 34, or may be installed at a location away from the housing 1190.

[0170] The imaging area of the first imaging unit 1180a is set to include, for example, the display area of the spatial floating image 3, the finger, hand, arm, face, etc. of the user 230. The first imaging unit 1180a images the user 230 who performs a touch operation on the spatial floating image 3 and generates a first captured image. Note that even if the display area of the spatial floating image 3 is imaged from the first imaging unit 1180a, it is a shooting from the side opposite to the traveling direction of the directional light beam of the spatial floating image 3, so the spatial floating image 3 itself cannot be visually recognized as an image. Here, in the example of the method for detecting the finger position (1), the first imaging unit 1180a is not merely an imaging unit, but also incorporates a depth sensor in addition to the imaging sensor. The configuration and processing of the depth sensor may use existing technologies. The depth sensor of the first imaging unit 1180a detects the depth of each part (for example, the finger, hand, arm, face, etc. of the user) in the captured image of the first imaging unit 1180a and generates depth information.

[0171] The air operation sensor 1351 is installed at a position where it can sense the display surface 3a of the floating-in-air image 3 as the sensing target surface. In FIG. 34, the air operation sensor 1351 is installed below the display surface 3a of the floating-in-air image 3, but it may also be installed on the side or above the display surface 3a. The air operation sensor 1351 may be installed in the housing 1190 as shown in FIG. 34, or may be installed at a location away from the housing 1190.

[0172] The air operation detection sensor 1351 in FIG. 34 is a sensor that detects the position where the finger 210 contacts or overlaps with the display surface 3a of the floating-in-air image 3. That is, when the tip of the finger 210 approaches the display surface 3a of the floating-in-air image 3 from the user side of the display surface 3a of the floating-in-air image 3, the air operation detection sensor 1351 can detect the contact of the finger 210 with the display surface 3a of the floating-in-air image 3.

[0173] For example, the control unit 1110 shown in FIG. 3C reads out a program for performing image processing and a program for displaying the virtual shadow 1510 from the non-volatile memory 1108. The control unit 1110 performs first image processing on the first captured image generated by the imaging sensor of the first imaging unit 1180a, and detects the finger 210 and calculates the position (x coordinate, y coordinate) of the finger 210. The control unit 1110 calculates the position (z coordinate) of the tip of the finger 210 with respect to the floating-in-air image 3 based on the first captured image generated by the imaging sensor of the first imaging unit 1180a and the depth information generated by the depth sensor of the first imaging unit 1180a.

[0174] In the example of FIG. 34, a touch detection unit that detects the position of the user's finger and detects a touch on the object of the floating-in-air image 3 is configured by the imaging sensor and depth sensor of the first imaging unit 1180a, the air operation sensor 1351, the air operation detection unit 1350, and the control unit 1110. Thereby, the position (x coordinate, y coordinate, z coordinate) of the finger 210 is calculated. Also, the touch detection result is calculated based on the detection result of the air operation detection unit 1350 or a combination of the detection result of the air operation detection unit 1350 and the information generated by the first imaging unit 1180a.

[0175] Then, based on the position (x coordinate, y coordinate, z coordinate) of finger 210 and the position of virtual light source 1500, the control unit 1110 calculates the position (display position) where the virtual shadow 1510 is to be displayed, and generates video data of the virtual shadow 1510 based on the calculated display position.

[0176] Note that the calculation of the display position of the virtual shadow 1510 in the video data by the control unit 1110 may be performed each time the position of finger 210 is calculated. Instead of calculating the display position of the virtual shadow 1510 in the video data each time the position of finger 210 is calculated, data of a display position map in which the display positions of the virtual shadow 1510 corresponding to the respective positions of a plurality of positions of finger 210 are calculated in advance is stored in the non-volatile memory 1108. When the position of finger 210 is calculated, video data of the virtual shadow 1150 may be generated based on the data of the display position map stored in the non-volatile memory 1108. Also, the control unit 1110 calculates the tip of finger 210 and the extending direction of finger 210 by first image processing, calculates the display position of the tip of finger 210 and the extending direction of the virtual shadow 1510 corresponding to the extending direction, and based on these, generates video data of the virtual shadow 1510 adjusted to a display angle corresponding to the actual direction of finger 210.

[0177] The control unit 1110 outputs the generated video data of the virtual shadow 1510 to the video control unit 1160. The video control unit 1160 generates video data (superimposed video data) obtained by superimposing the video data of the virtual shadow 1510 and other video data such as an object, and outputs the superimposed video data including the video data of the virtual shadow 1510 to the video display unit 1102.

[0178] The video display unit 1102 displays a video based on the superimposed video data including the video data of the virtual shadow 1510, thereby displaying the spatial floating video 3 in which the virtual shadow 1510 and an object or the like are superimposed.

[0179] The detection of a touch on an object is executed as follows, for example. The air operation detection unit 1350 and the air operation detection sensor 1351 are configured as described with reference to FIG. 3. When a finger 210 touches or overlaps a plane including the display surface 3a of the floating image 3 in space, the position is detected, and touch position information indicating the position where the finger 210 touches or overlaps the display surface 3a is output to the control unit 1110. Then, when the touch position information is input, the control unit 1110 determines whether the position (x coordinate, y coordinate) of the finger 210 calculated by the first image processing is included in the display range of each object displayed on the display surface 3a of the floating image 3 in space. And when the position of the finger 210 is included in the display range of any object, the control unit 1110 determines that a touch on this object has been performed.

[0180] According to the detection method described above, it is possible to detect the position of the finger 210 and the touch operation with a simple configuration combining one imaging unit 1180 (first imaging unit 1180a) having an imaging sensor and a depth sensor and one air operation detection sensor 1351.

[0181] As a modification of the finger position detection method (1), without using the detection results of the air operation detection unit 1350 and the air operation detection sensor 1351, the control unit 1110 may detect a touch operation by the finger 210 based only on the first captured image generated by the imaging sensor of the first imaging unit 1180a and the depth information generated by the depth sensor of the first imaging unit 1180a. For example, during normal operation, it is configured to be in a mode where the touch operation by the finger 210 is detected by combining the captured image of the imaging sensor of the first imaging unit 1180a, the detection result of the depth sensor, and the detection result of the air operation detection sensor 1351. When there is some problem in the operation of the air operation detection sensor 1351 or the air operation detection unit 1350, without using the detection results of the air operation detection unit 1350 and the air operation detection sensor 1351, the control unit 1110 may switch to a mode where it detects a touch operation by the finger 210 based only on the first captured image generated by the imaging sensor of the first imaging unit 1180a and the depth information generated by the depth sensor of the first imaging unit 1180a.

[0182] <<Method for Detecting Finger Position (2)>> FIG. 35 is a configuration diagram showing another example of a method for detecting the position of a finger. In the example shown in FIG. 35, the position of finger 210 is detected using two imaging units. The second imaging unit 1180b (1180) and the third imaging unit 1180c (1180) are both provided on the side opposite to the user 230 with respect to the spatial floating image 3.

[0183] The second imaging unit 1180b is installed, for example, on the right side as viewed from the user 230. The imaging area of the second imaging unit 1180b is set to include, for example, the spatial floating image 3, the finger, hand, arm, face, etc. of the user 230. The second imaging unit 1180b captures the user 230 performing a touch operation on the spatial floating image 3 from the right side of the user 230 and generates a second captured image.

[0184] The third imaging unit 1180c is installed, for example, on the left side as viewed from the user 230. The imaging area of the third imaging unit 1180c is set to include, for example, the spatial floating image 3, the finger, hand, arm, face, etc. of the user 230. The third imaging unit 1180c captures the user 230 performing a touch operation on the spatial floating image 3 from the left side of the user 230 and generates a third captured image. Thus, in the example of FIG. 35, the second imaging unit 1180b and the third imaging unit 1180c constitute a so-called stereo camera.

[0185] The second imaging unit 1180b and the third imaging unit 1180c may be installed in the housing 1190 as shown in FIG. 35, or may be installed at a location away from the housing 1190. Also, one imaging unit may be installed in the housing 1190 and the other imaging unit may be installed at a position away from the housing 1190.

[0186] The control unit 1110 performs second image processing on the second captured image and third image processing on the third captured image, respectively. Then, the control unit 1110 calculates the position (x coordinate, y coordinate, z coordinate) of the finger 210 based on the result of the second image processing (second image processing result) and the result of the third image processing (third image processing result).

[0187] In the example of FIG. 35, a touch detection unit that detects the position of the user's finger and detects a touch on the object of the spatial floating image 3 is configured by the second imaging unit 1180b, the third imaging unit 1180c, and the control unit 1110. Then, the position (x coordinate, y coordinate, z coordinate) of the finger 210 is calculated as a position detection result or a touch detection result.

[0188] As described above, in the example of FIG. 35, the virtual shadow 1510 is generated based on the position of the finger 210 calculated based on the second image processing result and the third image processing result. Further, based on the position of the finger 210 calculated based on the second image processing result and the third image processing result, it is determined whether or not the object has been touched.

[0189] According to this configuration, it is not necessary to employ an imaging unit having a depth sensor. Further, according to this configuration, by using the second imaging unit 1180b and the third imaging unit 1180c as a stereo camera, it is possible to improve the detection accuracy of the position of the finger 210. In particular, compared with the example of FIG. 34, the detection accuracy of the x coordinate and the y coordinate can be improved. Therefore, it is possible to more accurately determine whether or not the object has been touched.

[0190] Further, as a modification of the finger position detection method (2), the detection of the position (x coordinate, y coordinate, z coordinate) of the user's finger is performed based on the second captured image by the second imaging unit 1180b and the third captured image by the third imaging unit 1180c as described above, and thereby the display of the virtual shadow 1510 is controlled. Whether or not the object of the spatial floating image 3 has been touched may be detected by the air operation detection unit 1350 or the control unit 1110 based on the detection result by the air operation detection sensor 1351. According to this modification, since the air operation sensor 1351 that senses the display surface 3a of the spatial floating image 3 as a sensing target surface is used, the detection of the contact of the user's finger 210 with the display surface 3a of the spatial floating image 3 can be detected with higher accuracy than the detection accuracy in the depth direction by the stereo camera including the second imaging unit 1180b and the third imaging unit 1180c.

[0191] <<<Detection Method of Finger Position (3)>>> FIG. 36 is a configuration diagram showing another example of a method for detecting the position of a finger. Also in the example shown in FIG. 36, the position of finger 210 is detected using two imaging units. Different from the example of FIG. 35, the example of FIG. 36 has a configuration in which the fourth imaging unit 1180d (1180), which is one of the imaging units, is arranged at a position where it images the display surface 3a of the spatial floating image 3 from the side. Also, as in the example of FIG. 34, the first imaging unit 1180a (1180) is installed on the side opposite to the user 230 with respect to the spatial floating image 3. In the example of FIG. 36, the first imaging unit 1180a (1180) does not need to be equipped with a depth sensor as long as it can perform imaging.

[0192] Therefore, the fourth imaging unit 1180d is installed around the display surface 3a of the spatial floating image 3. In FIG. 36, the fourth imaging unit 1180d is installed below the side surface of the display surface 3a of the spatial floating image 3, but it may be installed on the side or above the display surface 3a. The fourth imaging unit 1180d may be installed in the housing 1190 as shown in FIG. 36, or may be installed at a location away from the housing 1190.

[0193] The imaging area of the fourth imaging unit 1180d is set to include, for example, the spatial floating image 3, the finger, hand, arm, face, etc. of the user 230. The fourth imaging unit 1180d images the user 230 who performs a touch operation on the spatial floating image 3 from around the display surface 3a of the spatial floating image 3, and generates a fourth captured image.

[0194] The control unit 1110 performs fourth image processing on the fourth captured image, and calculates the distance (z coordinate) between the display surface 3a of the spatial floating image 3 and the tip of the finger 210. Then, based on the position (x coordinate, y coordinate) of the finger 210 calculated by the first image processing on the first captured image by the first imaging unit 1180a described above, and the position (z coordinate) of the finger 210 calculated by the fourth image processing, the control unit 1110 performs processing related to the virtual shadow 1510 and determines whether there is a touch on the object.

[0195] In the example of FIG. 36, a touch detection unit that detects the position of the user's finger and detects a touch on an object is configured by the first imaging unit 1180a, the fourth imaging unit 1180d, and the control unit 1110. Then, the position (x coordinate, y coordinate, z coordinate) of the finger 210 is calculated as a position detection result or a touch detection result.

[0196] According to this configuration, it is possible to improve the detection accuracy of the distance between the display surface 3a of the spatial floating image 3 and the tip of the finger 210, that is, the depth of the finger 210 with respect to the display surface 3a of the spatial floating image 3, as compared with the example of the stereo camera configuration in FIG. 35.

[0197] Also, as a modification of the finger position detection method (3), the detection (x coordinate, y coordinate, z coordinate) of the user's finger position is performed based on the first captured image by the first imaging unit 1180a and the fourth captured image by the fourth imaging unit 1180d as described above, and thereby the display of the virtual shadow 1510 is controlled. The presence or absence of a touch on the object of the spatial floating image 3 may be detected by the air operation detection unit 1350 or the control unit 1110 based on the detection result by the air operation detection sensor 1351. According to this modification, since the air operation sensor 1351 that senses the display surface 3a of the spatial floating image 3 as a sensing target surface is used, it is possible to detect the contact of the user's finger 210 with respect to the display surface 3a of the spatial floating image 3 with higher accuracy than the detection accuracy by the fourth captured image by the fourth imaging unit 1180d.

[0198] <<Method of Displaying Input Content and Assisting Touch Operations>> An example of assisting the user's touch operation by other methods will be described. For example, it is also possible to display the input content and assist the touch operation. FIG. 37 is a diagram for explaining a method of displaying the input content and assisting the touch operation. FIG. 37 shows a case where numbers are input by a touch operation.

[0199] In the spatial floating image 3 of FIG. 37, a key input UI (user interface) display area 1600 including a plurality of objects such as objects for inputting, for example, numbers, an object 1601 for erasing the input content, an object 1603 for determining the input content, etc., and an input content display area 1610 for displaying the input content are included.

[0200] In the input content display area 1610, the content input by the touch operation (for example, numbers) is sequentially displayed in the spatial floating image 3 from the left end toward the right direction. The user can confirm the content input by the touch operation while looking at the input content display area 1610. Then, when the user inputs all the desired numbers, the user touches the object 1603. Thereby, the input content displayed in the input content display area 1610 is registered. Different from the physical contact on the surface of the display device, the touch operation on the spatial floating image 3 cannot obtain the feeling of contact by the user. Therefore, by separately displaying the input content in the input content display area 1610, the user can proceed with the operation while confirming whether his / her touch operation has been effectively performed, which is preferable.

[0201] On the other hand, when the user touches the wrong object or inputs content different from the desired one, the user can erase the last input content (here, "9") by touching the object 1601. Then, the user continues to perform the touch operation on the object for inputting numbers or the like. When the user inputs all the desired numbers, the user touches the object 1603.

[0202] In this way, by displaying the input content in the input content display area 1610, the user can be made to confirm the input content, and the convenience can be improved. Also, when the user touches the wrong object, the input content can be corrected, and the convenience can be improved.

[0203] <<Method for highlighting input content to assist touch operation>> Next, it is also possible to highlight the input content to assist touch operations. FIG. 38 is a diagram for explaining a method of highlighting the input content to assist touch operations.

[0204] FIG. 38 shows an example in which the numbers input by touch operations are highlighted. Along with FIG. 38, when an object corresponding to the number "6" is touched, the touched object is erased, and the input number "6" is displayed in the area where this object was displayed.

[0205] In this way, by displaying the number corresponding to the touched object instead of the object, it becomes possible to make the user recognize that the object has been touched, and it becomes possible to improve convenience. The number corresponding to the touched object may be referred to as a replacement object that replaces the touched object.

[0206] As another method of highlighting the input content, for example, the object touched by the user may be brightly lit, or the object touched by the user may be blinked. Although not shown here, by recognizing the distance between the finger 210 and the display surface 3a described in the embodiments of FIGS. 27 to 28, as the finger approaches the display surface, the object about to be touched changes to be brighter than the surrounding objects, and finally, when the finger swings onto the display surface, the degree of highlighting can reach the highest level, or it can be further brightly lit or blinked. Even in such a configuration, it becomes possible to make the user recognize that the object has been touched, and it becomes possible to improve convenience.

[0207] <<Method of Assisting Touch Operations by Vibration (1)>> Next, a method for assisting touch operations by vibration will be described. FIG. 39 is a diagram for explaining an example of a method for performing touch operation assistance by vibration. In FIG. 39, a case is shown where a touch operation is performed using a touch pen (touch input device) 1700 instead of a finger 210. The touch pen 1700 is equipped with a communication unit for transmitting and receiving various types of information such as signals and data to and from a device such as a spatial floating image display device, and a vibration mechanism that vibrates based on the input signal.

[0208] Suppose the user operates the touch pen 1700 and touches an object displayed in the key input UI display area 1600 of the spatial floating image 3 with the touch pen 1700. At this time, for example, the control unit 1100 transmits a touch detection signal indicating that a touch on the object has been detected from the communication unit 1132. When the touch pen 1700 receives the touch detection signal, the vibration mechanism generates vibration based on the touch detection signal. As a result, the touch pen 1700 vibrates. Then, the vibration of the touch pen 1700 is transmitted to the user, and the user recognizes that they have touched the object. In this way, the touch operation is assisted by the vibration of the touch pen 1700.

[0209] According to this configuration, it is possible to make the user recognize that they have touched an object by vibration.

[0210] Here, the case where the touch pen 1700 receives the touch detection signal transmitted from the spatial floating image device has been described, but other configurations may also be used. For example, when a touch on an object is detected, the spatial floating image display device notifies the upper device that a touch on the object has been detected. Then, the upper device transmits a touch detection signal to the touch pen 1700.

[0211] Alternatively, the spatial floating image display device and the upper device may transmit the touch detection signal via a network. In this way, the touch pen 1700 may receive the touch detection signal indirectly from the spatial floating image display device.

[0212] <<Method for Assisting Touch Operation by Vibration (2)>> Next, another method for assisting touch operation by vibration will be described. Here, by vibrating the terminal owned by the user, the user is made to recognize that they have touched an object. FIG. 40 is a diagram for explaining another example of the method for assisting touch operation by vibration. In the example of FIG. 40, user 230 wearing a wristwatch-type wearable terminal 1800 performs a touch operation.

[0213] The wearable terminal 1800 is equipped with, for example, a communication unit for transmitting and receiving various information such as signals and data to and from a device such as a spatial floating image display device, and a vibration mechanism that vibrates based on the input signal.

[0214] Suppose the user performs a touch operation with finger 210 and touches an object displayed in the key input UI display area 1600 of the spatial floating image 3. At this time, for example, the control unit 1100 transmits a touch detection signal indicating that a touch on the object has been detected from the communication unit 1132. When the wearable terminal 1800 receives the touch detection signal, the vibration mechanism generates vibration based on the touch detection signal. As a result, the wearable terminal 1800 vibrates. Then, the vibration of the wearable terminal 1800 is transmitted to the user, and the user recognizes that they have touched the object. In this way, the touch operation is assisted by the vibration of the wearable terminal 1800. Here, a wristwatch-type wearable terminal has been taken as an example for explanation, but it may also be a smartphone or the like worn by the user.

[0215] Note that the wearable terminal 1800 may receive a touch detection signal from a higher-level device, similar to the aforementioned touch pen 1700. Note that the wearable terminal 1800 may receive a touch detection signal via a network. In addition to the wearable terminal 1800, it is also possible to assist the touch operation using, for example, an information processing terminal such as a smartphone owned by the user.

[0216] According to this configuration, it becomes possible to make the user recognize that they have touched an object via various terminals such as the wearable terminal 1800 owned by the user.

[0217] <<Method of Assisting Touch Operation by Vibration (3)>> Next, other methods of assisting touch operation by vibration will be described. FIG. 41 is a diagram for explaining other examples of the method of assisting touch operation by vibration. In the example of FIG. 41, the user 230 stands on the diaphragm 1900 and performs a touch operation. The diaphragm 1900 is installed at a predetermined position where the user 230 performs the touch operation. As an actual usage form, the diaphragm 1900 is disposed, for example, under a mat (not shown), and the user 230 stands on the diaphragm 1900 through the mat.

[0218] As shown in FIG. 41, the diaphragm 1900 is connected to the communication unit 1132 of the spatial floating image display device 1000 via a cable 1910, for example. When a touch on an object is detected, for example, the control unit 1110 causes the communication unit 1132 to supply an alternating voltage to the diaphragm 1900 for a predetermined time. The diaphragm 1900 vibrates while the alternating voltage is being supplied. That is, the alternating voltage is a control signal output from the communication unit 1132 for vibrating the diaphragm 1900. The vibration generated by the diaphragm 1900 is transmitted from the feet to the user 230, and the user 230 can recognize that they have touched the object. In this way, the touch operation is assisted by the vibration of the diaphragm 1900.

[0219] The frequency of the alternating voltage is set to a value within the range where the user 230 can feel the vibration. The frequency of vibration that a person can feel is approximately in the range of 0.1 Hz to 500 Hz. Therefore, it is desirable that the frequency of the alternating voltage be set within this range.

[0220] Also, the frequency of the alternating voltage is preferably changed as appropriate according to the characteristics of the diaphragm 1900. For example, when the diaphragm 1900 vibrates in the vertical direction, it is said that a person has the highest sensitivity to vibrations of about 410 Hz. Also, when the diaphragm 1900 vibrates in the horizontal direction, it is said that a person has the highest sensitivity to vibrations of about 12 Hz. Furthermore, at frequencies of 34 Hz or higher, it is said that a person has a higher sensitivity to the vertical direction than the horizontal direction.

[0221] Therefore, when the diaphragm 1900 vibrates in the vertical direction, the frequency of the alternating voltage is preferably set to a value within a range including, for example, 410 Hz. Also, when the diaphragm 1900 vibrates in the horizontal direction, the frequency of the alternating voltage is preferably set to a value within a range including, for example, 12 Hz. Note that the peak voltage and frequency of the alternating voltage may be adjusted as appropriate according to the performance of the diaphragm 1900.

[0222] According to this configuration, it becomes possible for the user 230 to recognize that a touch has been made to the object by the vibration from the feet. Also, in the case of this configuration, it is also possible to set so that the display of the spatial floating image 3 does not change when a touch is made to the object, and even when another person peeks at the touch operation, the possibility of the input content being known is reduced, and the security can be further improved.

[0223] <<Modification Example 1 of Object Display>> Another example of object display in the spatial floating image 3 by the spatial floating image display device 1000 will be described. The spatial floating image display device 1000 is configured to display a spatial floating image 3 that is an optical image of a rectangular image displayed by the display device 1. There is a correspondence between the rectangular image displayed by the display device 1 and the spatial floating image 3. Therefore, when an image having luminance is displayed over the entire display range of the display device 1, the spatial floating image 3 will have an image having luminance displayed over the entire display range. In this case, although a sense of floating in the air can be obtained for the entire spatial floating image 3 which is rectangular, there is a problem that it is difficult to obtain a sense of floating in the air for each object displayed within the spatial floating image 3 itself. On the other hand, there may be a method of displaying only the object portion of the spatial floating image 3 as an image having luminance. However, the method of displaying only the object portion as an image having luminance can suitably obtain a sense of floating of the object, but on the other hand, there is a problem that the depth of the object is difficult to recognize.

[0224] Therefore, in the display example of FIG. 42A according to the present embodiment, within the display range 4210 of the spatial floating image 3, two objects, namely, the first button BUT1 displayed as "YES" and the second button BUT2 displayed as "NO", are displayed. The two object regions of the first button BUT1 and the second button BUT2 displayed as "NO" are regions in the display device 1 that include an image having luminance. A black display region 4220 is arranged around the display regions of these two objects so as to surround the object display regions.

[0225] The black display region 4220 is a region where black is displayed on the display device 1. That is, the black display region 4220 is a region having video information without luminance on the display device 1. In other words, the black display region 4220 is a region without video information having luminance. The region where black is displayed on the display device 1 becomes a spatial region where nothing can be seen by the user in the spatial floating image 3 which is an optical image. Further, in the display example of FIG. 42A, within the display range 4210, a frame image display region 4250 is arranged so as to surround the black display region 4220.

[0226] In the display device 1, the frame image display area 4250 is an area for displaying a pseudo frame using an image having luminance. Here, the pseudo frame in the frame image display area 4250 may display a single color as the frame image. Alternatively, such a pseudo frame in the frame image display area 4250 may be a frame image displayed using a decorative image. Alternatively, the frame image display area 4250 may display a frame such as a dashed line.

[0227] By displaying the frame image of the frame image display area 4250 as described above, the user can more easily recognize the plane to which the two objects, the first button BUT1 and the second button BUT2, belong, and can more easily recognize the depth position of the two objects, the first button BUT1 and the second button BUT2. However, since there is a black display area 4220 that is invisible to the user around these objects, the floating-in-air feeling of the two objects, the first button BUT1 and the second button BUT2, can be emphasized. In the floating-in-air image 3, the frame image display area 4250 exists at the outermost periphery of the display range 4210, but in some cases, it does not have to be at the outermost periphery of the display range 4210.

[0228] As described above, according to the display example of FIG. 42A, it is possible to more preferably achieve both the floating-in-air feeling of the object displayed in the floating-in-air image 3 and the recognition of the depth position.

[0229] <<Modification Example 2 of Object Display>> FIG. 42B is a modification example of the object display in FIG. 42A. It is a display example in which a message indicating that "touch operation is possible" is displayed near an object that can be touched by the user, such as the first button BUT1 and the second button BUT2. Here, as shown in FIG. 42B, a mark such as an arrow indicating an object that can be touched by the user may be displayed. In this way, the user can easily recognize the object that can be touched.

[0230] Here, such message display and mark display can also obtain a sense of floating in the air by being displayed so as to be surrounded by the black display area 4220.

[0231] <<Modification Example of Spatial Floating Image Display Device>> Next, a modification example of the spatial floating image display device will be described with reference to FIG. 43. The spatial floating image display device of FIG. 43 is a modification example of the spatial floating image display device of FIG. 3A. The same components as those described in FIG. 3A are denoted by the same reference numerals. In the description of FIG. 43, the differences from the components described in FIG. 3A will be described, and for the components that are the same as those described in FIG. 3A, since they have already been described in FIG. 3A, repeated description will be omitted.

[0232] Here, similar to the spatial floating image display device of FIG. 3A, the spatial floating image display device of FIG. 43 converts the video light from the display device 1 into the spatial floating image 3 by passing through the polarization separation member 101, the λ / 4 plate 21, and the retroreflective member 2.

[0233] Unlike the spatial floating image display device of FIG. 3A, the spatial floating image display device of FIG. 43 is provided with a physical frame 4310 so as to surround the spatial floating image 3 from the periphery. Here, an opening window is provided along the outer periphery of the spatial floating image 3 in the physical frame 4310, and the user can visually recognize the spatial floating image 3 at the position of the opening window of the physical frame 4310. When the spatial floating image 3 is rectangular, the shape of the opening window of the physical frame 4310 is also rectangular.

[0234] In the example of FIG. 43, an air operation detection sensor 1351 is provided in a part of the opening window of the physical frame 4310. As already described in FIG. 3C, the air operation detection sensor 1351 can detect a touch operation by the user's finger on the object displayed in the spatial floating image 3.

[0235] In the example of FIG. 43, the physical frame 4310 has a cover structure that covers the polarization separation member 101 on the upper surface of the floating image display device. Note that what the cover structure covers is not limited to the polarization separation member 101, and it may be configured to cover the storage portions of the display device 1 and the retroreflective member 2. However, the physical frame 4310 in FIG. 43 is only an example of this embodiment and does not necessarily have to have a cover structure.

[0236] Here, FIG. 44 shows the physical frame 4310 and the opening window 4450 of the floating image display device of FIG. 43 when the floating image 3 is not being displayed. At this time, naturally, the user cannot visually recognize the floating image 3.

[0237] On the other hand, an example of the configuration of the opening window 4450 of the physical frame 4310 of the floating image display device of FIG. 43 in this embodiment and an example of the display of the floating image 3 are shown using FIG. 45. In the example of FIG. 45, the opening window 4450 is configured to substantially coincide with the display range 4210 of the floating image 3.

[0238] Furthermore, the display example of the floating image 3 in FIG. 45 performs object display similar to the example of FIG. 42A, for example. Specifically, objects that can be touched by the user, such as the first button BUT1 and the second button BUT2, are displayed. These objects that can be touched by the user are surrounded by the black display area 4220, and a floating feeling in space is preferably obtained.

[0239] A frame image display area 4470 is provided on the outer periphery surrounding the black display area 4220. The outer periphery of the frame image display area 4470 is the display range 4210, and the edge of the opening window 4450 of the floating image display device is arranged to substantially coincide with the display range 4210.

[0240] Here, in the display example of FIG. 45, the frame video of the frame video display area 4470 is displayed in a color that is the same color system as the color of the physical frame 4310 around the opening window 4450. For example, if the physical frame 4310 is white, the frame video of the frame video display area 4470 is also displayed in white. If the physical frame 4310 is gray, the frame video of the frame video display area 4470 is also displayed in gray. For example, if the physical frame 4310 is yellow, the frame video of the frame video display area 4470 is also displayed in yellow.

[0241] In this way, by displaying the frame video of the frame video display area 4470 in the same color system as the color of the physical frame 4310 around the opening window 4450, the spatial continuity between the physical frame 4310 and the frame video of the frame video display area 4470 can be emphasized and conveyed to the user.

[0242] Generally, a user can recognize space more preferably with respect to a physical configuration than a space-floating video. Therefore, by displaying the space-floating video so as to emphasize the spatial continuity of the physical frame as in the display example of FIG. 45, the user can more preferably recognize the depth of the space-floating video.

[0243] Furthermore, in the display example of FIG. 45, since the space-floating images of objects that can be touched by the user, such as the first button BUT1 and the second button BUT2, are imaged on the same plane as the frame video display area 4470, the user can more preferably recognize the depth of the first button BUT1 and the second button BUT2 based on the depth recognition of the physical frame 4310 and the frame video display area 4470.

[0244] That is, according to the display example of FIG. 45, it is possible to more preferably achieve both the floating feeling in the air and the recognition of the depth position of the object displayed in the space-floating video 3. And it becomes possible to more easily recognize the depth position of the object displayed in the space-floating video 3 than in the display example of FIG. 42A.

[0245] Also, in the display example of FIG. 45, similar to the display example of FIG. 42B, a mark such as an arrow indicating an object that can be touched by the user may be displayed.

[0246] Incidentally, as a modification of the configuration of the spatial floating image display device of FIG. 43, as shown in FIG. 46, a light shielding plate 4610 or a light shielding plate 4620 having a black surface with a low light reflectance may be provided inside the cover structure of the physical frame 4310. By providing the light shielding plate in this way, even if the user peeks into the inside of the spatial floating image display device through the opening window, it is possible to prevent the user from visually recognizing components and the like that have nothing to do with the spatial floating image 3. Thereby, it is possible to prevent a situation where a real object that has nothing to do with the spatial floating image 3 is visually recognized behind the black display area 4220 such as in FIG. 42A, making it difficult to visually recognize the spatial floating image 3. In addition, it is possible to prevent the generation of stray light based on the spatial floating image 3.

[0247] Here, the light shielding plate 4610 and the light shielding plate 4620 form a cylindrical quadrangular prism corresponding to the rectangle of the spatial floating image 3, and may be configured to extend from the vicinity of the opening window of the spatial floating image display device toward the storage portions of the display device 1 and the retroreflective member 2. Also, in consideration of ensuring the divergence angle of light and the freedom of the user's viewing point, the configuration may include a frustum of a square pyramid shape in which the opposing light shielding plates are not parallel, and may be configured to extend from the vicinity of the opening window of the spatial floating image display device toward the storage portions of the display device 1 and the retroreflective member 2. In this case, the frustum of the square pyramid shape is a shape that spreads from the vicinity of the opening window of the spatial floating image display device toward the storage portions of the display device 1 and the retroreflective member 2.

[0248] Incidentally, the cover structure of FIG. 46 and the light shielding plate may be used in a spatial floating image display device that performs displays other than the display example of FIG. 45. That is, it is not always necessary to display the frame image display area 4470. As long as the physical frame 4310 of the cover structure of the spatial floating image display device is arranged so as to surround the display range 4210 of the spatial floating image 3, even if there is no frame image display area 4470 in FIG. 45, it is possible to contribute to improving the recognition of the depth position of the displayed object.

[0249] Although the above-described various embodiments have been described in detail, the present invention is not limited to only the above-described embodiments and includes various modifications. For example, the above-described embodiments have described the entire system in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add, delete, or replace other configurations for a part of the configuration of each embodiment.

[0250] In the technology according to this embodiment, by displaying high-resolution and high-brightness video information in a spatially floating state, for example, it enables a user to operate without feeling anxiety about contact infection of infectious diseases. If the technology according to this embodiment is used in a system used by an unspecified number of users, it is possible to reduce the risk of contact infection of infectious diseases and provide a contactless user interface that can be used without feeling anxiety. Thereby, it contributes to "3 Good health and well-being for all" of the Sustainable Development Goals (SDGs) advocated by the United Nations.

[0251] Also, in the technology according to this embodiment, by reducing the divergence angle of the emitted video light and further aligning it to a specific polarization, only regular reflected light is efficiently reflected to the retroreflective member, so that the light utilization efficiency is high and it is possible to obtain a bright and clear spatially floating image. According to the technology according to this embodiment, it is possible to provide a contactless user interface with excellent usability that can significantly reduce power consumption. Thereby, it contributes to "9 Build the infrastructure for industry and technological innovation" and "11 Make cities and human settlements inclusive, safe, resilient and sustainable" of the Sustainable Development Goals (SDGs) advocated by the United Nations.

[0252] Furthermore, the technology according to this embodiment enables the formation of a spatially floating image using video light with high directivity (straight - advancing property). In the technology according to this embodiment, even when displaying images that require high security, such as those on bank ATMs or ticket vending machines at stations, or highly confidential images that need to be concealed from people facing the user directly, by displaying video light with high directivity, it is possible to provide a non - contact user interface with a low risk of the spatially floating image being spied on by people other than the user. This contributes to "11 Sustainable cities and communities" among the Sustainable Development Goals (SDGs) proposed by the United Nations.

Explanation of symbols

[0253] 1... Display device, 2... Retro - reflective member, 3... Spatial image (spatially floating image), 105... Windshield, 100... Transparent member, 101... Polarization separation member, 12... Absorptive polarizing plate, 13... Light source device, 54... Light direction conversion panel, 151... Retro - reflective member, 102, 202... LED substrate, 203... Light guide, 205, 271... Reflective sheet, 206, 270... Phase - difference plate, 300... Spatially floating image, 301... Ghost image of the spatially floating image, 302... Ghost image of the spatially floating image, 230... User, 1000... Spatially floating image display device, 1110... Control unit, 1160... Video control unit, 1180... Imaging unit, 1102... Video display unit, 1350... Air operation detection unit, 1351... Air operation detection sensor, 1500... Virtual light source, 1510... Virtual shadow, 1610... Input content display area, 1700... Touch pen, 1800... Wearable terminal, 1900... Diaphragm, 4220... Black display area, 4250... Frame video display area.

Claims

1. A display unit for displaying an image; A retroreflector that reflects the image light from the display unit and forms a spatial floating image in the air by the reflected light; Equipped with In the display range of the floating image in space, there is an area in which an object is displayed, a black display area is disposed surrounding the area in which the object is displayed; A physical frame is disposed so as to surround the floating image in space, The physical frame forms an opening window having a cover structure that covers the display unit and a storage unit that stores the retroreflector, A light-shielding plate is disposed inside the cover structure between the opening window and the storage section that stores the display unit and the retroreflector, and the light-shielding plate extends from at least both the upper end and the lower end of the opening window toward the storage section that stores the display unit and the retroreflector. A floating image display device.

2. 2. The space floating image display device according to claim 1, The black display area is an area in which there is no image information having luminance in the display image of the display unit corresponding to the spatial floating image. A floating image display device.

3. 2. The space floating image display device according to claim 1, a sensor for detecting a position of a user's finger performing a touch operation on the object; A floating image display device.

4. 4. The space floating image display device according to claim 3, displaying a message near the object, indicating that the object is a touch-operable object; A floating image display device.

5. 5. The space floating image display device according to claim 4, displaying a mark indicating the object in addition to the message; A floating image display device.

6. 2. The space floating image display device according to claim 1, The light blocking plate forms a cylindrical rectangular prism. A floating image display device.

7. 2. The space floating image display device according to claim 1, The light-shielding plate forms a quadrangular pyramid. A floating image display device.

8. 8. The space floating image display device according to claim 7, The shape of the truncated quadrangular pyramid is a shape that expands from the vicinity of the opening window toward a storage section that stores the display section and the retroreflector. A floating image display device.

Images