Space floating image information display system

Abstract

To provide necessary information by suitably displaying an image to a user of a kiosk terminal or the like. According to the present invention, it contributes to "health and welfare for all three people", "9 Build foundation for industry and technical innovation", and "11 Keep living", which are sustainable development targets.SOLUTION: A kiosk terminal 1500 as a space floating video information display system includes a video display device, a retroreflective member for forming a space floating video 3 in the air by retroreflecting video light from the video display device, a sensing system including a sensor for detecting an operation by a user to the space floating video 3, and a control device for executing predetermined processing on the basis of the detected operation. The sensing system detects a contact state of the finger of the user or the belonging, moves the finger of the user or the belonging in the contact state, and draws a line as the floating image in space at a planar contact position of the floating image in space 3.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to a space floating image information display system. [Background technology]

[0002] As a space-floating video information display system, a video display device and a display method that directly displays a video as a spatial image toward the outside are already known. Also, a detection system that reduces false detections of operations on the operation surface of the displayed spatial image is described, for example, in JP 2019-128722 A (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-128722 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent document 1 describes that the purpose of reducing false detection of operations on an image formed in the air is to achieve this by accepting the user's movement as an operation when the user approaches the image from a predetermined direction.

[0005] However, prior art examples such as Patent Document 1 do not take into consideration users who are unfamiliar with using a user interface displayed as a floating image in space, and it is particularly difficult for users using the system for the first time to intuitively understand how to operate it, posing issues in terms of usability and convenience.

[0006] Therefore, when using the floating images produced by the floating image information display system as a user interface or man-machine interface that can be operated by users, we would like to realize an interface that is easy to use and convenient. In particular, we would like to provide a suitable floating image information display system that allows users to intuitively understand how to operate it and reduces operational errors and incorrect inputs, even when an unspecified number of users, including those using the system for the first time or those who are unfamiliar with it, use the floating image interface. We would also like to provide a user interface that is easy to use and convenient using floating images, even in devices commonly known as "kiosks," for example.

[0007] Furthermore, the user interface formed by the floating image, such as the operation menu with buttons and the like, is a non-contact user interface, which has the advantage of minimizing the risk of contact infection compared to general contact interfaces such as physical buttons. We would like to realize a floating image information display system that makes the most of these advantages. Furthermore, we would like to realize a system that can improve or enhance the visibility of the user interface using floating images, such as by preventing or reducing ghost images related to floating images.

[0008] An object of the present invention is to provide a technology for a space floating image information display system that is easy to use and convenient, and that can provide users of kiosk terminals and the like with a suitable interface using space floating images and necessary information. [Means for solving the problem]

[0009] In order to solve the above problems, for example, the configuration described in the claims is adopted. The present invention includes multiple means for solving the above problems, but one example is as follows. A space-floating image information display system according to an embodiment is a space-floating image information display system for forming a space-floating image in the air, and includes an image display device that displays an image of at least one object, a retroreflective member for forming the space-floating image in the air by retroreflecting image light from the image display device, a sensing system including a sensor for detecting a user's operation on the space-floating image, and a control device that executes predetermined processing based on the detected operation, in which the sensing system detects a contact state of the user's finger or belongings, and moves the user's finger or belongings while in contact, and a line is drawn as a space-floating image at the contact position on the planar surface of the space-floating image. [Effects of the Invention]

[0010] According to a representative embodiment of the present disclosure, a space-floating image information display system is easy to use and convenient, and can provide users of kiosk terminals with a suitable interface and necessary information using space-floating images. Problems, configurations, effects, etc. other than those described above will be described in the description of the invention. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing an example of a usage form of a space floating image information display system according to an embodiment; [Figure 2] 1 is a diagram illustrating an example of an internal configuration of a space floating image information display system according to an embodiment; [Figure 3] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image information display system according to an embodiment; [Figure 4] 10A and 10B are diagrams illustrating another example of the configuration of the main part and the configuration of the retroreflection part of the space floating image information display system according to an embodiment. [Figure 5] FIG. 10 is a perspective view showing an example of the arrangement of a member that blocks extraordinary rays generated by retroreflection according to an embodiment. [Figure 6] 1 is a cross-sectional view showing an example of the arrangement of a member that blocks extraordinary rays generated by retroreflection in one embodiment. FIG. [Figure 7] 1 is an explanatory diagram of a first sensing technique used in a space floating image information display system according to an embodiment; [Figure 8] FIG. 10 is an explanatory diagram of a second sensing technology used in the space floating image information display system according to an embodiment. [Figure 9] 1 is an explanatory diagram of the operation and device of a sensing system used in a space floating image information display system according to an embodiment. FIG. [Figure 10] FIG. 1 is a diagram showing the characteristics of the spectral irradiance of sunlight. [Figure 11] FIG. 1 is a diagram showing the reflection characteristics of polarized light incident on a medium with a refractive index of 1.5 versus the angle of incidence of the light. [Figure 12] 1 shows an explanatory diagram of a technology for inputting and drawing characters and the like on a floating image in space according to one embodiment. [Figure 13] 1 is a diagram showing a configuration of a main part of a space floating image information display system according to an embodiment of the present invention; [Figure 14] FIG. 10 is a diagram showing the configuration of the main components of another space floating image information display system according to an embodiment of the present invention. [Figure 15] FIG. 1 illustrates an example of a kiosk terminal. [Figure 16] 1 is a diagram illustrating an example of the appearance of a kiosk terminal according to one example (first embodiment). [Figure 17] 1 is a diagram illustrating an example of a cross-sectional structure of a kiosk terminal according to a first embodiment. [Figure 18] FIG. 10 is a diagram illustrating an example of the appearance of a kiosk terminal according to an example (second embodiment). [Figure 19] FIG. 10 is a diagram illustrating an example of a cross-sectional structure of a kiosk terminal according to a second embodiment. [Figure 20] FIG. 10 is a diagram showing a display example of a kiosk terminal according to one example (third embodiment). [Figure 21] FIG. 11 is a diagram showing a first operation flow of the kiosk terminal according to the third embodiment. [Figure 22] FIG. 11 is a diagram showing a second operation flow of the kiosk terminal according to the third embodiment. [Figure 23] FIG. 10 is a diagram illustrating a cooperative operation between a kiosk terminal and a mobile terminal according to an example (fourth embodiment). [Figure 24] FIG. 10 is a diagram showing an example of operation and display of a sign at a kiosk terminal according to one example (embodiment 5). [Figure 25] FIG. 10 is a diagram showing an example of a display on a kiosk terminal according to a modified example of the third embodiment etc. [Figure 26] FIG. 2 is a cross-sectional view showing a specific example of the configuration of the light source device. [Figure 27] 1 is a structural diagram showing a specific example of the configuration of a light source device. [Figure 28] FIG. 10 is a diagram showing a specific example of the configuration of a light source device of another type. [Figure 29] 1 is a diagram showing an example of the appearance of a vending machine according to one embodiment (another embodiment). FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention (also referred to as examples) will be described in detail with reference to the drawings. In the drawings, the same parts are generally designated by the same reference numerals, and repeated explanations will be omitted. In the drawings, the representation of components may not represent their actual positions, sizes, shapes, ranges, etc., in order to facilitate understanding of the invention.

[0013] For the sake of explanation, when describing processing by a program, the program, functions, processing units, etc. may be described as the main components, but the main hardware components are the processor, or a controller, device, computer, system, etc. that is configured with the processor, etc. A computer executes processing according to a program read into memory using resources such as memory and communication interfaces as appropriate through the processor. This realizes predetermined functions, processing units, etc. A processor is configured, for example, with semiconductor devices such as a CPU (Central Processing Unit) or GPU. A processor is configured with devices or circuits that are capable of performing predetermined calculations. Processing is not limited to software program processing, and can also be implemented with dedicated circuits. Dedicated circuits such as FPGAs (field-programmable gate arrays), ASICs (application specific integrated circuits), and CPLDs (Complex Programmable Logic Devices) can be used.

[0014] The program may be pre-installed as data on the target computer, or may be distributed as data from a program source to the target computer. The program source may be a program distribution server on a communication network, or a non-transitory computer-readable storage medium (e.g., a memory card). The program may be composed of multiple modules. The computer system may be composed of multiple devices. The computer system may be composed of a client-server system, a cloud computing system, etc. Various data and information may be composed of structures such as tables and lists, for example, but are not limited to these. Expressions such as identification information, identifier, ID (identification), name, and number are interchangeable.

[0015] A floating-in-space image information display system (hereinafter sometimes simply referred to as the system) according to one embodiment is configured to improve visibility by eliminating ghost images that significantly reduce the visibility of floating-in-space images and increasing the brightness of the floating-in-space images. The system according to one embodiment is applied to a kiosk terminal or the like, and provides a user interface such as an operation menu on a screen using floating-in-space images. In addition, when a user approaches the housing of the system (kiosk terminal), the system according to one embodiment first displays a concierge (sometimes referred to as a concierge image) as a predetermined figure on the screen using floating-in-space images. The system uses the floating-in-space image concierge to greet, guide, explain, and so on to the user.

[0016] In one embodiment of the space-floating image information display system, when a user approaches a space-floating image or operates the space-floating image, the concierge image automatically transitions or changes to an operation menu screen with a predetermined operation menu, such as a plurality of number buttons or selection buttons that can be pressed. The system appropriately guides and explains the operation menu to the user through the space-floating image concierge. In particular, if the system determines that the user is a first-time user or an elderly user, the concierge will guide the user through the operation menu in detail. Furthermore, if the system determines that the user is not sure how to operate the space-floating image, the concierge will guide the user through the operation menu in detail.

[0017] Furthermore, the floating image information display system of one embodiment has a function to identify and specify users, for example, by facial recognition using a camera. The system refers to user attribute information, such as age and system usage history, for users identified by this function. The system controls the system to change the method and content of the guide provided by the floating image concierge, etc., depending on the user's attributes.

[0018] Furthermore, the floating-in-space image information display system of one embodiment also provides a function that allows a user to input any character or figure onto the surface of the floating-in-space image with their fingers. The system detects the state of contact of the fingers with the surface of the floating-in-space image, and draws the inputted line on the surface of the floating-in-space image. The system acquires this inputted line as, for example, the user's signature.

[0019] In the following description of the embodiments, an image floating in space or an image displayed in the air may be expressed by the term "space-floating image." Instead of this term, it may be expressed as "aerial image," "spatial image," "floating image," "space-floating optical image of displayed image," "floating optical image of displayed image," etc. The term "space-floating image," which is mainly used in the description of the embodiments, is used as a representative example of these terms.

[0020] <Spatial Floating Video Information Display System> The present disclosure relates to an information display system that can display an image generated by image light from a large-area image light source as a floating image inside or outside a store space by transmitting the image through a transparent member that divides the space, such as the glass of a shop window. The present disclosure also relates to a large-scale digital signage system configured using multiple such information display systems.

[0021] According to the following embodiments, for example, high-resolution video information can be displayed in a floating state on the glass surface of a shop window or a light-transmitting plate. At this time, by making the divergence angle of the emitted video light small, i.e., an acute angle, and further aligning it with a specific polarization, it is possible to efficiently reflect only the normal reflected light from the retroreflective member (retroreflective member) or retroreflector. Therefore, according to the embodiments, the light utilization efficiency is high, and it is possible to suppress the ghost image that occurs in addition to the main floating image in space, which was a problem with conventional retroreflection methods, and to obtain a clear floating image in space.

[0022] Furthermore, a device including the light source of the present disclosure can provide a novel and highly usable space-floating image information display system that can significantly reduce power consumption. Furthermore, the technology of the present disclosure can provide a space-floating image information display system for a vehicle that can display a so-called unidirectional space-floating image that can be viewed from outside the vehicle through shield glass such as the vehicle's windshield, rear window, or side window.

[0023] On the other hand, conventional floating-space image information display systems combine an organic electroluminescence (EL) panel or display panel (sometimes referred to as a liquid crystal display panel or LCD panel) as a high-resolution color display image source with a retroreflective member. In conventional floating-space image display devices, image light is diffused over a wide angle. Therefore, when using the polyhedral retroreflective member 2 of the first embodiment shown in Figure 3(B), in addition to the light reflected normally by the retroreflective member 2 (resulting in a normal floating-space image), ghost images are generated by image light incident obliquely on the retroreflective member 2 (retroreflective portion 2a) shown in Figure 3(C). This reduces the image quality of the floating-space image. Furthermore, in conventional floating-space image display devices, in addition to the normal floating-space image, multiple ghost images are generated depending on the number of reflective surfaces. This allows people other than the viewer to view the same floating-space image, which is a ghost image, posing a significant security issue.

[0024] <First Configuration Example of the Space Floating Image Information Display System> Fig. 1A shows an example of the usage of the space-floating image information display system of the embodiment, and is an explanatory diagram of the overall configuration of the space-floating image information display system. In Fig. 1A, for example, in a store or the like, a space is partitioned by a show window (also called window glass) 105, which is a translucent material (also described as a transparent material) such as glass. According to this space-floating information display system, it is possible to display a space-floating image in one direction to the outside of the store space through such a transparent material.

[0025] Specifically, according to this system, light with a narrow-angle directional characteristic and specific polarization is emitted from the image display device 10 as an image light beam. The emitted image light beam is first incident on the retroreflective member 2, retroreflected, and transmitted through the window glass 105 to form a real, floating image (aerial image) 3 outside the store space. In FIG. 1A, the store interior inside the transparent member (here, the window glass) 105 is shown as the depth direction, and the outside of the window glass 105 (e.g., the sidewalk) is shown as the foreground. Alternatively, a member that reflects specific polarization can be provided on the window glass 105, and the image light beam can be reflected by this member to form an aerial image at a desired position within the store.

[0026] 1(B) shows the internal configuration of video display device 10. Video display device 10 includes a video display unit 1102 that displays an original aerial image, a video control unit 1160 that converts input video to match the resolution of the panel, and a video / audio signal receiving unit 1130 that receives and inputs video / audio signals.

[0027] Of these, the video and audio signal receiving unit 1130 is responsible for handling wired input signals via an input interface such as HDMI (High-Definition Multimedia Interface) (registered trademark), and wireless input signals such as Wi-Fi (Wireless Fidelity) (registered trademark). The video and audio signal receiving unit 1130 can also function independently as a video receiving and display device. Furthermore, the video and audio signal receiving unit 1130 can also display and output video and audio information from a tablet terminal, smartphone, etc. Furthermore, the video and audio signal receiving unit 1130 can also be connected to a processor (arithmetic processing device) such as a stick PC as needed, in which case the video and audio signal receiving unit as a whole can be equipped with capabilities such as calculation processing and video analysis processing.

[0028] [Functional blocks of the space-floating video information display system] 2 shows a functional block diagram of the space floating video information display system 1. The video display unit 1102 generates an image by modulating light passing through the panel of the video display unit 1102 based on a video signal. The video display unit 1102 may use, for example, a transmissive liquid crystal display panel, or in some cases may be configured using a reflective liquid crystal display panel that modulates light reflected on the panel based on a video signal, a DMD panel (DMD: Digital Micromirror Device, registered trademark), or the like.

[0029] The retroreflector 1101 retroreflects light modulated by the image display unit 1102. Of the light reflected from the retroreflector 1101, the light outputted to the outside of the space-floating image information display system 1 forms the space-floating image 3. The light source 1105 generates light for the image display unit 1102. For example, a solid-state light source such as an LED light source or a laser light source is used as the light source 1105. The power source 1106 converts AC current inputted from the outside into DC current and supplies power to the light source 1105. Furthermore, the power source 1106 supplies the DC current required by each of the other components.

[0030] The light guide 1104 guides light generated by the light source 1105 and irradiates it onto the image display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be called a backlight for the image display unit 1102. Various methods are possible for combining the light guide 1104 and the light source 1105. Specific configuration examples will be described later. As shown in FIG. 2, the portion consisting of the three components of the image display unit 1102, the light guide 1104, and the light source 1105 is particularly referred to as the image display device 10.

[0031] The aerial operation detection sensor 1351 is a sensor that senses an area that overlaps at least a portion of the display area of ​​the floating in space image 3, or an area that overlaps the entire display area, in order to detect an operation (also referred to as an aerial operation) of the floating in space image 3 by the user's fingers. Specific sensor configurations of the aerial operation detection sensor 1351 include distance sensors that use invisible light such as infrared rays, invisible light lasers, ultrasonic waves, etc., or a combination of these sensors that can detect coordinates on a two-dimensional plane. Furthermore, the aerial operation detection sensor 1351 may be configured as a ToF (Time of Flight) LiDAR (Light Detection and Ranging) sensor, which will be described later.

[0032] The aerial operation detection unit 1350 acquires the sensing signal acquired by the aerial operation detection sensor 1351, and based on this, calculates whether or not the user's fingers have touched the floating in space image 3, and the position of said touch on the floating in space image 3. The aerial operation detection unit 1350 may be configured with a circuit such as an FPGA.

[0033] The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 (these may be referred to as a sensing system) may be configured to be built into the space-floating video information display system 1, or may be provided externally and separately from the space-floating video information display system 1. If provided separately, they may be configured so that information or signals can be transmitted to the space-floating video information display system 1 via a wired or wireless communication connection path or a video signal transmission path. Both the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be provided separately. In this case, it is possible to construct a system in which the space-floating video information display system 1 without the aerial operation detection function serves as the main body, and only the aerial operation detection function can be added as an option. Alternatively, only the aerial operation detection sensor 1351 may be provided separately, and the aerial operation detection unit 1350 may be built into the space-floating video information display system 1. When it is desired to more freely arrange the aerial operation detection sensor 1351 relative to the installation position of the space floating image information display system 1, there is an advantage to a configuration in which only the aerial operation detection sensor 1351 is a separate body.

[0034] The imaging unit 1180 is a so-called camera having an image sensor, and captures images of the space near the floating-in-space image 3 and / or the user's face, arms, fingers, etc. The imaging unit 1180 may use multiple cameras or a camera with a depth sensor depending on the application. The imaging unit 1180 may be provided separately from the floating-in-space image information display system 1. When using multiple cameras or a camera with a depth sensor, the imaging unit 1180 may assist the mid-air operation detection unit 1350 in detecting a user's touch operation on the floating-in-space image 3, in other words, an operation of contacting the surface of the floating-in-space image 3. For example, if the mid-air operation detection sensor 1351 is configured as an object intrusion sensor for the plane to which the floating-in-space image 3 belongs, the mid-air operation detection sensor 1351 alone may not be able to detect how close an object (e.g., a user's finger) that has not intruded into the plane is to the plane. In such a case, by using depth calculation information based on the imaging results of multiple cameras in the imaging unit 1180 and depth information from a depth sensor, it becomes possible to calculate the distance between the plane and an object (such as a user's finger) that has not entered the plane of the space floating image 3. This calculation information can be used for various display controls of the space floating image 3.

[0035] Alternatively, this system may be configured so that the aerial operation detection unit 1350 detects a touch operation on the floating-in-space image 3 by the user based on the imaging results of the imaging unit 1180, without using the aerial operation detection sensor 1351.

[0036] The imaging unit 1180 may be configured to capture an image of the face of the user operating the Space Floating Image 3, and the control unit 1110 may perform a process of identifying and specifying the user or a process of authenticating the user based on the captured image. Alternatively, the imaging unit 1180 may be configured to capture an image including the area around the user operating the Space Floating Image 3, in order to determine whether or not another person is standing around or behind the user operating the Space Floating Image 3 and peeking at the user's operation of the Space Floating Image 3.

[0037] The operation input unit 1107 is an operation button or a remote control light receiving unit, and inputs signals related to user operations that are different from mid-air operations on the space floating image 3. The operation input unit 1107 may be used by an administrator of the space floating image information display system 1 to operate this system, separate from the above-mentioned user who touches the space floating image 3.

[0038] The video signal input unit 1131 has a function of connecting to an external video output device and inputting video data. The audio signal input unit 1133 has a function of connecting to an external audio output device and inputting audio data. On the other hand, the audio signal output unit 1140 has a function of outputting an audio signal based on audio data input to the audio signal input unit 1133. The audio signal output unit 1140 may also output an audio signal based on audio data such as numbers or character strings, or other data such as operation sounds or error warning sounds, which are pre-recorded in the storage unit 1170. The video signal input unit 1131 and the audio signal input unit 1133 are collectively referred to as the video / audio signal input unit 1130. The video signal input unit 1131 and the audio signal input unit 1133 may be configured separately, or may be combined into one unit.

[0039] Audio signal output unit 1140 is connected to a speaker or superdirectional speaker 30. Audio signal output unit 1140 may be connected to a speaker that outputs audio in the normal audible band, but in particular in cases where high confidentiality and security considerations are required, such as in an embodiment of a kiosk terminal described below, audio signal output unit 1140 may be connected to a superdirectional speaker so that the audio cannot be heard by anyone other than the user. A superdirectional speaker is a speaker that has the characteristic that audio in the audible band can be heard only by the ears of people present in a specific, limited spatial region, but cannot be heard by the ears of people present outside that specific spatial region.

[0040] The superdirectional speaker 30 is constructed by arranging multiple ultrasonic output elements capable of generating ultrasonic signals of, for example, about 40 kHz on a plane. In this case, the more ultrasonic output elements used, the louder the volume of the sound obtained by the superdirectional speaker. The principle of the superdirectional speaker will be briefly explained. As is well known, ultrasonic waves have a higher degree of directionality compared to sounds in the audible range (e.g., human speech). Therefore, by using a 40 kHz ultrasonic signal as a carrier (carrier wave) and modulating the carrier with an audio signal in the audible range (e.g., AM modulation), it is possible to make the sound audible only in a specific, limited spatial region.

[0041] For example, by using multiple cameras as imaging unit 1180, the positions of the user's face and ears can be identified, and based on the identification results, audio can be heard only in the area near the user's ears through output from superdirectional speaker 30. Specifically, by controlling the phase (in other words, delay time) of the ultrasonic signal input to the ultrasonic output elements that make up superdirectional speaker 30, audio can be heard only in the identified limited spatial area. Also, by arranging multiple ultrasonic output elements not on a flat surface but on, for example, a concave surface, audio can be heard only in the specific limited spatial area.

[0042] The nonvolatile memory 1108 stores various data used in the space floating image information display system 1. The data stored in the nonvolatile memory 1108 may include data for various operations displayed as the space floating image 3, user interface image information such as icons and buttons, data of objects for user operation, and layout information. The memory 1109 stores image data to be displayed as the space floating image 3 and data for controlling the device.

[0043] The control unit 1110 corresponds to a controller (in other words, a control device) of the space-floating image information display system 1, and controls the operation of each connected unit. The control unit 1110 is equipped with a device such as a processor. The control unit 1110 executes processing in accordance with a program read from the non-volatile memory 1108 or the storage unit 1170 to the memory 1109 or an internal memory, thereby realizing various functions. The control unit 1110 may cooperate with the program stored in the memory 1109 to perform arithmetic processing based on information acquired from each connected unit. The control unit 1110 may be implemented using a microcomputer or the like within a housing constituting the space-floating image information display system 1, or may be connected and implemented outside the housing.

[0044] The communication unit 1132 communicates with external devices, external servers, etc. via a wired or wireless communication interface. The communication unit 1132 transmits and receives video, images, audio, and various types of data through this communication.

[0045] The storage unit 1170 records videos, images, audio, various data, etc. For example, videos, images, audio, various data, etc. may be recorded in advance in the storage unit 1170 at the time of product shipment. Videos, images, audio, various data, etc. acquired from an external device or external server via the communication unit 1132 may also be recorded in the storage unit 1170. The videos, images, various data, etc. recorded in the storage unit 1170 can be output as the floating-in-space video 3 via the video display unit 1102, the video display device 10, and the retroreflector 1101.

[0046] Data such as icons, buttons, and objects for user operation that are displayed as a user interface (including operation menus and concierge images, which will be described later) in Space Floating Image 3, as well as data that configures the concierge image, may also be included in the video and image data recorded in storage unit 1170. In addition, layout information for operation menus and concierges, such as icons, buttons, and objects, that are displayed as a user interface in Space Floating Image 3, as well as various metadata related to the operation menus and concierges, may also be included in the various data recorded in storage unit 1170. In addition, audio data that the concierge of Space Floating Image 3 outputs as audio may also be recorded in storage unit 1170. The audio data recorded in storage unit 1170 may be output as an audio signal from speaker or superdirectional speaker 30 via audio signal output unit 1140.

[0047] The control unit 1110, or the video control unit 1160 or audio signal output unit 1140 may appropriately create video data and audio data for displaying and outputting the operation menu or concierge based on various data for configuring the operation menu or concierge, which is stored in the storage unit 1170, non-volatile memory 1108, etc.

[0048] The video control unit 1160 performs various controls on the video signal input to the video display unit 1102. The video control unit 1160 may perform video switching control, such as determining which video, between the video stored in the memory 1109 and the video input by the video signal input unit 1131, is to be input to the video display unit 1102. Alternatively, the video control unit 1160 may perform control to superimpose the video stored in the memory 1109 and the video input by the video signal input unit 1131 to generate a composite video to be input to the video display unit 1102. The video control unit 1160 may also control image processing on the video data input by the video signal input unit 1131 and the video stored in the memory 1109. Examples of image processing include scaling processing that enlarges, reduces, or transforms an image, brightness adjustment processing that changes the brightness, contrast adjustment processing that changes the contrast curve of an image, and Retinex processing that decomposes an image into light components and changes the weighting of each component.

[0049] Furthermore, the video control unit 1160 may perform special effect video processing or the like to assist the user in performing an aerial operation on the video input to the video display unit 1102. The special effect video processing may be controlled based on the result of the detection of the user's operation by the aerial operation detection unit 1350 and the result of the user's image captured by the image capturing unit 1180.

[0050] As described above, the space-floating image information display system 1 can be equipped with various functions. However, the space-floating image information display system 1 does not necessarily have to have all of the above-mentioned configurations. The space-floating image information display system 1 can have any configuration as long as it has at least the function of generating the space-floating image 3.

[0051] [First method for creating floating images in space] FIG. 3 shows the main configuration of the space floating image information display system of the embodiment, and also shows an example (referred to as a first method) of forming the space floating image 3 and the configuration of the retroreflective member 2.

[0052] 3A, this floating-in-space image information display system includes an image display device 10 that diverges image light of a specific polarization at a narrow angle in an oblique direction relative to a transparent member 100, which is a light-transmitting plate such as glass. The image display device 10 includes a liquid crystal display panel 11 and a light source device 13 that generates light of a specific polarization with a narrow-angle diffusion characteristic.

[0053] The image light of a specific polarization emitted from the image display device 10 is reflected by a polarization separating member 101 provided on a transparent member 100 and having a film that selectively reflects the image light of a specific polarization, and the reflected light is incident on the retroreflective member 2. In FIG. 3, the polarization separating member 101 formed in a sheet shape is adhered to the transparent member 100.

[0054] A retroreflective member 2 is provided in the other diagonal direction relative to the transparent member 100. A λ / 4 plate 21 (in other words, a quarter-wave plate) is provided on the image light incident surface of the retroreflective member 2. The image light passes through the λ / 4 plate 21 twice, once when it enters the retroreflective member 2 and once when it leaves, thereby undergoing polarization conversion from a specific polarization (one polarization) to the other polarization.

[0055] Here, the polarization separation member 101, which selectively reflects image light of a specific polarization, has the property of transmitting the polarized light of the other polarization after polarization conversion. Therefore, the image light of the other polarization after polarization conversion passes through the polarization separation member 101. The image light that passes through the polarization separation member 101 forms a space-floating image 3, which is a real image, outside the transparent member 100, as shown in the figure.

[0056] The light that forms the floating image 3 is a collection of light rays that converge from the retroreflective member 2 onto the optical image of the floating image 3, and these light rays continue to travel in a straight line even after passing through the optical image of the floating image 3. Therefore, the floating image 3 is an image with high directionality, unlike the diffused image light formed on a screen by a general projector or the like.

[0057] 3, when viewed by a user from the direction of arrow A, the levitating image 3 appears as a bright image, but when viewed by another person from, for example, the direction of arrow B, the levitating image 3 appears completely invisible. These characteristics of the levitating image 3 are extremely suitable for use in systems that display images that require high security or highly confidential images that should be concealed from people directly facing the user.

[0058] Depending on the performance of the retroreflective member 2, the polarization axis of the reflected image light may become misaligned. In this case, a portion of the image light whose polarization axis has become misaligned is reflected by the polarization separating member 101 described above and returns to the image display device 10. This portion of the image light is reflected again by the image display surface of the liquid crystal display panel 11 that constitutes the image display device 10, generating a ghost image. This may be a factor that causes a deterioration in the image quality of the spatially floating image 3.

[0059] Therefore, in this embodiment, an absorptive polarizer 12 is provided on the image display surface of the image display device 10. The absorptive polarizer 12 transmits the image light emitted from the image display device 10 and absorbs the reflected light returning from the polarization separation member 101, thereby suppressing re-reflection. Therefore, according to this embodiment using the absorptive polarizer 12, it is possible to prevent or suppress degradation of image quality due to ghost images of the spatially floating image 3.

[0060] The polarization separation member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects specific polarized waves.

[0061] FIG. 3B shows an example of the configuration of the retroreflective member 2 used in the first method. FIG. 3B shows the surface shape of a typical retroreflective member 2 manufactured by Nippon Carbide Industries Co., Ltd., which was used in this study. This retroreflective member 2 has regularly arranged hexagonal prism retroreflective portions (retroreflective elements) 2a on its surface. Light rays entering the interior of the hexagonal prism are reflected by the walls and bottom of the hexagonal prism and are emitted as retroreflected light in a direction corresponding to the incident light, displaying a real image, a floating image 3, based on the image displayed on the image display device 10.

[0062] The resolution of the floating image 3 in space depends not only on the resolution of the LCD panel 11 but also on the outer diameter D and pitch P of the retroreflective portion 2a of the retroreflective member 2 shown in FIG. 3B. For example, when using a 7-inch WUXGA (1920 × 1200 pixels) LCD panel 11, even if one pixel (one triplet) is approximately 80 μm, if the diameter D of the retroreflective portion 2a is 240 μm and the pitch P is 300 μm, one pixel of the floating image 3 will be equivalent to 300 μm. As a result, the effective resolution of the floating image 3 is reduced to about one-third. Therefore, in order to make the resolution of the floating image 3 equivalent to that of the image display device 10, it is desirable to make the diameter D and pitch P of the retroreflective portion 2a closer to that of one pixel of the LCD panel 11. On the other hand, in order to suppress the occurrence of moire caused by the retroreflective member 2a and the pixels of the liquid crystal display panel 11, it is advisable to design the pitch ratio of each so that it is not an integral multiple of one pixel. Also, it is advisable to arrange the shape so that none of the sides of the retroreflective member 2a overlaps any of the sides of one pixel of the liquid crystal display panel 11.

[0063] On the other hand, to manufacture the retroreflective member 2 at low cost, it is preferable to use a roll press method. Specifically, this method involves aligning the retroreflective portions 2a and forming them on a film. In this method, the reverse shape of the shape to be formed is formed on the surface of a roll, a UV-curable resin is applied to a base material for fixing, and the required shape is formed by passing the roll through the rolls. The resin is then irradiated with UV light to harden the resin, and the desired shape of the retroreflective member 2 is obtained.

[0064] [Second method for creating floating images in space] Next, Fig. 4 shows another example (referred to as a second method) of the formation of the space-floating image 3 in the space-floating image information display system of this embodiment and the configuration of the retroreflective member. Fig. 4(A) shows an overview of the formation of the space-floating image 3 using the retroreflective member 330 in the second method. Light from an object P (corresponding point P) in one space (in this example, the space below in the Z direction) is incident on the retroreflective member 330 and retroreflected to form a space-floating image 331 (corresponding point Q) in the other space (in this example, the space above in the Z direction).

[0065] FIG. 4B shows the surface shape of a typical retroreflective member 330 manufactured by Asukanet Co., Ltd., used in this study, to explain the operating principle. The retroreflective member 330 has four-sided structures (tetrahedrons) 330A regularly arranged on its surface (the XY plane in the figure). Multiple structures 330A are arranged between sidewalls 330B. The four-sided structures 330A are, for example, micromirrors having a rectangular prism shape extending in the Z direction. For example, light from an object P (also referred to as object light) enters the four-sided structure 330A. The light beam entering the four-sided structure 330A is reflected by two of the wall surfaces of the four-sided structure 330A (e.g., reflecting surface RS1 and reflecting surface RS2). The reflected light beams (both the light beams emitted upward from reflecting surface RS1 and the light beams emitted upward from reflecting surface RS2) are indicated as reflected light R0. The reflected light R0 is emitted as retroreflected light in a direction corresponding to the incident light, and forms and displays a space floating image 331, which is a real image based on the object P, as shown in FIG. 4(A).

[0066] As with the first-type retroreflective member 2 shown in FIG. 3, the resolution of the space-floating image 331 also depends heavily on the outer diameter (DS) and pitch (PT) of the retroreflective member 330's retroreflective portion (four-sided structure 330A). For example, when using a 7-inch WUXGA (1920 × 1200 pixels) LCD panel, even if one pixel (one triplet) is approximately 80 μm, if the outer diameter (DS) of the retroreflective portion is 120 μm and the pitch (PT) is 150 μm, one pixel of the space-floating image 331 will be equivalent to 150 μm. As a result, the effective resolution of the space-floating image 331 is reduced by about half. Therefore, in order to make the resolution of the space-floating image 331 equivalent to that of the image display device 10, it is desirable to make the diameter (DS) and pitch (PT) of the retroreflective portion (structure 330A) closer to that of one pixel of the LCD panel. On the other hand, to prevent moire from occurring due to the retroreflective member 330 and the pixels of the liquid crystal display panel, it is preferable to design the pitch ratio of each to be a different integer multiple of one pixel, as described above. Also, it is preferable to arrange the shape of the retroreflective portion (structure 330A) so that none of its sides overlaps any of the sides of one pixel of the liquid crystal display panel.

[0067] The light that forms the floating image 331 is a collection of light rays that converge from the retroreflective member 330 to the optical image of the floating image 331, and these light rays continue to travel in a straight line even after passing through the optical image of the floating image 331. Therefore, the floating image 331 is an image with high directionality, unlike the diffused image light formed on a screen by a general projector or the like.

[0068] In the configuration of Fig. 4, when a user views the floating image 331 from the direction of arrow A, the floating image 331 is perceived as a bright image, but when viewed by another person from the direction of arrow B, the floating image 331 is not visible at all. Similar to the floating image using the first type retroreflective member 2 described above, the characteristics of this floating image 331 are extremely suitable for use in systems that display images that require high security or highly confidential images that should be kept secret from people facing the user.

[0069] In the second-type retroreflective member 330, as shown in FIG. 4B, light from an object P enters the retroreflective member 330 from one side (the lower side in the Z direction) and is reflected by two reflective surfaces (RS1, RS2) provided on the four walls of the retroreflective member 330. The reflected light R0 forms a floating image 331 on the other side (the upper side in the Z direction) at point Q. At this time, the two reflective surfaces (RS1, RS2) generate extraordinary light R1 and R2, which are reflected in a different direction from the reflected light R0. The extraordinary light R1 and R2 generated by the two reflective surfaces (RS1, RS2) generate ghost images 332 and 333, as shown in FIG. 4A. Therefore, the ghost images 332 and 333 can cause a deterioration in the image quality of the floating image 331.

[0070] As described above, the retroreflective member 2 of the first method generates ghost images depending on the number of reflective surfaces. In contrast, the retroreflective member 330 of the second method generates ghost images only in two specific directions depending on the angle of incidence of the object light. Therefore, the retroreflective member 330 of the second method is less affected by ghost images and enables high-quality spatial image display. Therefore, the following description of the space-floating image display device and space-floating image information display system will be limited to the case where the retroreflective member 330 of the second method is applied.

[0071] [Technical measures to reduce ghost images] To realize a spatial image display device capable of forming a high-quality spatial floating image with reduced ghost images, it is recommended to provide an image light control sheet on the exit surface of the liquid crystal display panel as the image display element in order to control the divergence angle of the image light from the liquid crystal display panel and bend it in the desired direction. Furthermore, it is recommended to provide an image light control sheet on the light exit surface, the light entrance surface, or both of the surfaces of the retroreflective member 330 to absorb extraordinary light R1 and R2 ((B) in Figure 4) that causes ghost images.

[0072] FIG. 5 shows a specific method and configuration example for applying the image light control sheet to a space-floating image display device. In FIG. 5, an image light control sheet 334 is provided on the emission surface of a liquid crystal display panel 335, which is an image display element. In FIG. 5, the emission surface of the liquid crystal display panel 335 is shown as a plane (XY plane) formed by the X-axis and Y-axis shown in the figure. The image light control sheet 334 has transmissive portions and light-absorbing portions on its main surface (XY plane). In this case, moiré may occur due to interference caused by the pitch between the pixels of the liquid crystal display panel 335 and the transmissive portions and light-absorbing portions of the image light control sheet 334. The following two methods are effective for reducing this moiré.

[0073] (1) As a first method, the vertical stripes (diagonal lines shown in the figure) generated by the light-transmitting and light-absorbing portions of the image light control sheet 334 are arranged at a predetermined angle (inclination) θ0 with respect to the pixel arrangement (X-axis and Y-axis) of the liquid crystal display panel 335.

[0074] (2) In the second method, when the pixel size of the liquid crystal display panel 335 is A and the pitch of the vertical stripes of the image light control sheet 334 is B, the ratio (B / A) of these is selected to be a value that is not an integer multiple. Since one pixel of the liquid crystal display panel 335 is made up of three RGB color sub-pixels arranged in parallel and is generally square, it is not possible to suppress the occurrence of the above-mentioned moire over the entire screen. For this reason, the tilt θ0 shown in the first method in (1) can be optimized within a range of 5 to 25 degrees so that the position where the moire occurs can be intentionally shifted to a location where the spatial floating image is not displayed.

[0075] Although the above moiré reduction has been described using a liquid crystal display panel and an image light control sheet 334 as examples, the same method and configuration can also be applied to moiré that occurs between the retroreflective member 330 and the image light control sheet 334 when the retroreflective member 330 is provided with the image light control sheet 334. Because the retroreflective member 330 and the image light control sheet 334 are both linear structures, it is sufficient to optimally tilt the image light control sheet 334 with attention to the X-axis and Y-axis of the retroreflective member 330. This makes it possible to reduce large moiré patterns with long wavelengths and low frequencies that are visible to the naked eye.

[0076] 6A shows a vertical cross-sectional view of an image display device 10 having an image light control sheet 334 arranged on an image light exit surface 3351 of a liquid crystal display panel 335. The image light control sheet 334 has light-transmitting portions 336 and light-absorbing portions 337 arranged alternately on its main surface, and is adhesively fixed to the image light exit surface 3351 of the liquid crystal display panel 335 by an adhesive layer 338.

[0077] Furthermore, as described above, when a 7-inch WUXGA (1920 × 1200 pixels) liquid crystal display panel is used as the image display device 10, even if one pixel (one triplet) (indicated by A in the figure) is approximately 80 μm, the ghost images 332 and 333 shown in FIG. 4A that appear on both sides of the spatial floating image 331 can be reduced with the following configuration. For example, the pitch B of the image light control sheet 334 is set to 340 μm, with the distance d2 of the transmissive portion 336 being 300 μm and the distance d1 of the light absorbing portion 337 being 40 μm. In this case, the image light control sheet 334 can reduce ghost images by controlling the sufficient transmission characteristics and the diffusion characteristics of the image light from the image display device 10 that causes abnormal light. In this case, if the thickness of the image control sheet 334 is set to at least two-thirds of the pitch B, the ghost reduction effect is significantly improved.

[0078] 6B shows a vertical cross-sectional view of a configuration in which an image light control sheet 334 is disposed on the image light emission surface of the retroreflective member 330 (FIG. 4). The image light control sheet 334 is configured with light-transmitting portions 336 and light-absorbing portions 337 alternately arranged, and is inclined with respect to the retroreflective member 330 at a predetermined inclination angle θ1 to match the emission direction of retroreflected light 3341. As a result, the image light control sheet 334 absorbs the extraordinary light R1 and R2 (FIG. 4B) generated by the aforementioned retroreflection, while allowing the regularly reflected light to transmit without loss as retroreflected light 3341.

[0079] The retroreflective member 330 has spaces 3301 arranged therein, each corresponding to a retroreflective portion formed by the four-sided structure 330A (FIG. 4) described above. The spaces 3301 corresponding to the retroreflective portions are partitioned by the surfaces of the sidewalls 330B. The spaces 3301 have, for example, reflective surfaces R1 and R2. Light a1 incident on the retroreflective member 330 from, for example, the lower side is reflected, for example, by the reflective surface R1 of the spaces 3301, and the reflected light a2 is further reflected, for example, by the reflective surface R2, and exits from the upper side of the retroreflective member 330. The exiting light is incident on the image light control sheet 334 and exits as retroreflected light 3341.

[0080] When a 7-inch WUXGA (1920 × 1200 pixels) liquid crystal display panel is used, even if one pixel (one triplet) is approximately 80 μm, the configuration shown in FIG. 4A can reduce ghost images 332 and 333 that appear on both sides of a floating image 331. As shown in FIG. 6B, for example, the pitch B of the image light control sheet 334 is 420 μm, with the distance d2 of the transmissive portion 336 of the retroreflective member 330 being 400 μm and the distance d1 of the light absorbing portion 337 being 20 μm. In this case, the image light control sheet 334 provides sufficient transmission characteristics and controls the diffusion characteristics of the image light from the image display device 10 that causes abnormal light to be generated by the retroreflective member 330, thereby reducing ghost images.

[0081] The image light control sheet 334 described above also prevents external light from entering the spatial floating image display device, thereby improving the reliability of the components. For example, a viewing angle control film (VCF) from Shin-Etsu Polymer Co., Ltd. is suitable as the image light control sheet 334. The VCF has a sandwich structure in which transparent silicon and black silicon are alternately arranged, with synthetic resin placed on the light entrance and exit surfaces. Therefore, when this VCF is used as the image light control sheet 334 of this embodiment, the above-mentioned effects can be expected.

[0082] [Technology for sensing operations on floating images in space] A user (sometimes referred to as a user, observer, operator, etc.) is bidirectionally connected to the system via the space-floating image 3 (FIG. 2, etc.) produced by the space-floating image information display system 1. In other words, the user uses the system's applications (for example, the ticket sales function of a kiosk terminal) by viewing and operating the space-floating image 3. To do this, a sensing technology is required that allows the user to simulate the operation of the space-floating image 3 and sense that operation. An example of this sensing technology will be described below using a specific example. The "sensing technology" referred to here includes the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 described using FIG. 2, and is particularly a technology for detecting user operations in three-dimensional space (in other words, aerial operations). The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be referred to as a sensing system.

[0083] FIG. 7A shows a principle diagram for explaining the first sensing technology. The space-floating image FI by the space-floating image information display system 1 is divided into a plurality of areas (in other words, regions). In this embodiment, the space-floating image FI is divided vertically and horizontally into 12 areas (3 x 4 = 12). In FIG. 7A, the plane of the space-floating image FI is shown as the xy plane, and the front-to-back direction relative to the plane is shown as the z direction. For example, the illustrated area A301 is one area in the upper left of the sensing plane a3.

[0084] In the first sensing technology, a first distance measuring device 340 is provided that incorporates a Time of Flight (TOF) system corresponding to each area of ​​the floating image FI. The first distance measuring device 340 is provided in place of the mid-air operation detection sensor 1351 in FIG. 2. The light emitting unit of the first distance measuring device 340 emits light from an LED (Light Emitting Diode) as a light source in synchronization with a system signal. An optical element for controlling the divergence angle is provided on the light emitting side of the LED, and a pair of highly sensitive avalanche diodes (APDs) with picosecond time resolution are arranged in four columns and three rows to correspond to the 12 areas. The LED as a light source emits light in synchronization with a signal from the system, and the light is reflected by the object to be measured (here, the tip of the user's finger UH) and shifts in phase (Δt in FIG. 9, described below) by the time it takes for the light to return to the light receiving unit.

[0085] The calculation unit of the sensing system shown in FIG. 9B receives a signal from the system and a signal generated by the avalanche diode, which is the light-receiving unit of the first distance measuring device 340. From these signals, the calculation unit calculates the distance to the object by calculating the phase shift. The distance is calculated for each TOF system (TOF1 to TOF12) associated with each area. In FIG. 7A, sensing planes a3, a2, and a1 (also referred to as the first sensing plane a3, the second sensing plane a2, and the third sensing plane a1) are shown in order of proximity to the object as measurement layers of the distance measuring device 340 on the side closer to the object (finger UH) in the z direction relative to the plane of the floating image FI. Furthermore, sensing plane a0 is shown as the measurement layer on the side farther from the floating image FI. Distance L1 indicates the distance to sensing plane a0, distance L2 indicates the distance to sensing plane a1, distance L3 indicates the distance to sensing plane a2, and distance L4 indicates the distance to sensing plane a1.

[0086] Next, the sensing system can recognize the direction of movement of the object (finger UH) by recognizing which of the 12 areas it passed through on each measurement layer (sensing surface a3 to a1) and calculating the movement time on each measurement layer using the method described above.

[0087] Figure 9 (A) shows the timing of light emission from the LED light source and the timing of light reception by the light receiving element for each of the 12 measurement areas. SU1 to SU12 indicate the timing of light emission and light reception for each sensing unit associated with each area and TOF. The sensing system standardizes the individual data by delaying the timing of LED light emission for each area.

[0088] In reality, suppose the user intentionally extends their fingers UH toward the floating image FI in order to connect to the system bidirectionally. In this case, the sensing system obtains a first sensing signal S1 sensed in, for example, area A301 on the sensing surface a3 farthest from the floating image FI, a second sensing signal S2 sensed in, for example, a specific area on the sensing surface a2, and a third sensing signal S3 sensed in, for example, a specific area on the third sensing surface a1. Using these sensing signals (S1-S3), the sensing system calculates and processes the contact point with the floating image FI from the movement direction of the fingers UH and the time difference between when they crossed each sensing surface.

[0089] In order to obtain even more accurate position information, a sensing plane a0 is set at a position further back from the floating image FI. Based on the sensing at the sensing plane a0, the sensing system detects the passage of the finger UH through the floating image FI as an end signal, and calculates the contact point with the floating image FI as a three-dimensional coordinate from the position coordinate of that detection and the two sensing signals mentioned above.

[0090] 7B shows the action of selecting a portion of the floating-in-space image FI with the user's finger UH (particularly the fingertip) and the action of the user's finger UH removing the portion of the floating-in-space image FI. As shown in FIG. 7B, in the first sensing technology, when the user touches the desired position coordinate of the floating-in-space image FI and then removes the finger UH, the following occurs. That is, the sensing system sequentially transmits the third sensing signal S3 detected on the third sensing surface a3, the second sensing signal S2 detected on the second sensing surface a2, and the first sensing signal S1 detected on the first sensing surface a1 to the sensing system's arithmetic unit for calculation processing. As a result, the system recognizes that the user's finger UH has removed the specific coordinate of the floating-in-space image FI.

[0091] Next, we will explain a more accurate sensing technology for artificially manipulating floating images in space.

[0092] FIG. 8A shows a principle diagram for explaining the second sensing technology. The second sensing technology differs from the first sensing technology shown in FIG. 7A in that a second ranging device 341 is provided in addition to the first ranging device 340, thereby achieving more accurate sensing. The first ranging device 340 and the second ranging device 341 may be provided side by side. In the second sensing technology, the second ranging device 341 (particularly a CMOS sensor) is used in combination with the first sensing system as a second sensing system. As shown in FIG. 8A, the second ranging device 341 senses the same range (sensing planes a1, a2, a3, a0) as the first ranging device 340.

[0093] As described above, the first distance measuring device 340 has a built-in TOF system corresponding to each of a plurality of areas, for example, divided into 12 areas, in the space-floating image FI (the first sensing system in FIG. 8B). On the other hand, the second distance measuring device 341 uses a two-dimensional image sensor, for example, a 1 / 4-inch CMOS sensor for use in a sensing camera. The aspect ratio of this CMOS sensor is generally 3:4. For this reason, in this embodiment, to match the aspect ratio of the CMOS sensor, the sensing area of ​​the TOF system of the first distance measuring device 340 is also divided vertically into 3 and horizontally into 4, for a total of 12 areas, as described above.

[0094] Furthermore, while a CMOS sensor's resolution of around 1 million pixels is sufficient, unlike conventional camera systems, it does not require an RGB color separation filter. Therefore, CMOS sensors not only achieve smaller size and higher sensitivity for the same number of pixels, but also have high sensitivity to near-infrared light. Therefore, in the second sensing technology, the object to be measured (the tip of the finger UH) is illuminated by the light source light of the TOF system of the first ranging device 340 at a timing determined for each area, significantly improving detection accuracy. While a detailed explanation is omitted, Figure 9(B) shows the above-described system as a functional block diagram.

[0095] FIG. 8B shows sensing surfaces a1, a2, and a3 of the first ranging device 340 and sensing surfaces b1, b2, and b3 of the second ranging device 341 corresponding to the sensing surfaces. FIG. 8B also shows the action of selecting a part of the floating-in-space image FI with a finger UH and moving away from the part on those sensing surfaces. As shown in FIG. 8B, in a floating-in-space image information display system using the second sensing technology, when a user intentionally extends their finger UH toward the floating-in-space image FI, the following occurs. In this case, in addition to the three-dimensional information obtained by the first ranging device 340 described above, three-dimensional information obtained by the second ranging device 341 is also obtained. The planar resolution of the sensing surface b3 of the second ranging device 341, which corresponds to the sensing surface a3 of the first ranging device 340 that is farthest from the floating-in-the-air image FI, can be made highly accurate to match the resolution of the CMOS sensor used. Similarly, sensing surface b2 corresponds to sensing surface a2, and sensing surface b1 corresponds to sensing surface a1, thereby realizing a sensing system with significantly improved resolution in the planar direction.

[0096] At this time, the contact point with the floating image FI is calculated based on the time difference between the movement direction of the target object (the tip of the user's finger UH) and the time it crosses the sensing planes of the first and second distance measuring devices 340 and 341. To obtain even more accurate position information, a sensing plane a0 is set further back from the floating image FI. The sensing system detects the finger UH's passage through the floating image FI as an end signal, and calculates the contact point with the floating image FI as a more precise 3D coordinate from the position coordinates on the sensing plane a0 and the two sensing signals described above. Furthermore, increasing the frame rate of the CMOS sensor from 1 / 20 to 1 / 30 or 1 / 120 seconds significantly improves resolution, not only by improving detection accuracy in the planar direction but also by increasing the amount of planar information captured per unit time. At this time, the detection information from the second sensing technology is synchronized with the position information from the first sensing technology using a synchronization signal provided by the system.

[0097] 8(B), when the user touches the desired position coordinates of the floating image FI and then removes the finger UH, the third sensing signal S3 sensed by the third sensing surface a3, the second sensing signal S2 sensed by the second sensing surface a2, and the first sensing signal S1 sensed by the first sensing surface a1 are sequentially transmitted to the arithmetic unit of the sensing system, as in the first sensing technology described above. Then, through calculation processing in the arithmetic unit, the system recognizes that the user's finger UH has left the specific coordinates of the floating image FI.

[0098] The LED light source used in the TOF sensor of the first ranging device 340 of the sensing system described above should prevent a decrease in accuracy of the ranging device due to external light such as sunlight, and should use near-infrared light with high light energy in the range beyond the visible light range (380nm to 780nm) that cannot be seen by the naked eye.

[0099] Figure 10 shows a characteristic diagram of the spectral irradiance of sunlight. As the wavelength of the light source light for the LED of the TOF sensor, it is recommended to use light with a wavelength λ1 of 920 nm, which has the least energy of the spectral irradiance of sunlight shown in Figure 10.

[0100] <Technology for inputting text and figures into floating images> Next, as an embodiment of the floating image information display system, we will explain a technology that allows users to input any image, such as text or figures, into the floating image. By using this technology, users can input, for example, their signature into the floating image.

[0101] By using the sensing technology using the mid-air operation detection sensor 1351 in Figure 2 mentioned above, or the sensing technology using the TOF system in Figures 7 and 8, it is possible to determine the position and movement direction of the user's fingertip as the target, the contact point with the floating image in space, the position where the finger leaves the floating image in space (also referred to as the separation position), etc. Therefore, based on the detection information of the contact point position and separation position, the floating image information display system in space can draw the line segment connecting these two points as an image on the surface of the floating image in space.

[0102] In this embodiment, by utilizing the above sensing technology, the movement of the user's fingers, in other words, the movement of the contact point, can be detected on the surface of the floating image in space, allowing lines such as characters to be input and displayed.

[0103] For example, a floating-in-space image information display system detects the contact point between the floating-in-space image and the user's fingertip, then moves the fingertip to another position within the surface, and detects when the fingertip leaves the floating-in-space image and the point of departure. This makes it possible to draw shapes such as lines and curves, or letters such as alphabets and numbers, on the surface of the floating-in-space image. This allows you to input shapes and letters on the surface of the floating-in-space image, just like drawing shapes and letters on the surface of a liquid crystal display with a touch panel.

[0104] Furthermore, although not shown, the space floating image information display system uses sensing technology to sample the movement of the user's fingertip relative to the space floating image at a predetermined time interval (for example, 100 milliseconds), and draws lines on the space floating image connecting each sampling point. This makes it possible to draw characters or figures that can be drawn in one stroke, such as the numbers 1, 2, and 3, or the letters L and M, as space floating images.

[0105] Furthermore, for letters or figures that cannot be drawn in one stroke, such as 4 or H, the system can handle them as follows. That is, when a user inputs the letter H, for example, the system first detects and draws a vertical line (first line) created by continuous contact of the finger with the surface of the floating image in space. Next, the system detects that the user has removed their finger from the surface and then touched it again in a different position, and detects and draws a horizontal line (second line). Finally, the system detects that the user has removed their finger from the surface and then touched it again in a different position, and detects and draws a vertical line (third line). In this way, the letter H can be drawn on the surface of the floating image in space.

[0106] FIG. 12 is an explanatory diagram of a technology for inputting and drawing images such as letters and figures onto the above-mentioned floating-in-space image with the user's fingers. FIG. 12 shows a schematic diagram of a floating-in-space image FI displayed by the floating-in-space image information display system 1 as viewed from the user's viewpoint. In this example, the floating-in-space image FI is disposed at an angle to the horizontal plane of the system's housing 1201, and the floating-in-space image FI is viewed slightly diagonally downward from the user's viewpoint. In this example, the floating-in-space image FI includes a display of a rectangular frame 1202 (note that this is not a frame-shaped device). Initially, the frame 1202 is displayed in a predetermined color (e.g., green), and only a predetermined background color is displayed within the frame 1202.

[0107] The example of FIG. 12 shows a state in which the user draws the letter L, as an example of a simple character, on the surface of the floating-in-space image FI with the fingertip UF of the user's hand. At this time, the user communicates their intention to draw a character to the floating-in-space image information display system 1 in some way. For example, in one embodiment, a character input button 1203 is provided as a physical push button at an arbitrary location on the floating-in-space image information display system 1, in this example, at the bottom right position on the housing 1201. When the user presses the character input button 1203, the system interprets this as the user's intention to input characters, etc., into the floating-in-space image FI, and transitions to a mode for inputting and drawing characters into the floating-in-space image FI (also referred to as a character input mode). It is also possible to communicate the intention to draw a character to the floating-in-space image information display system 1 by voice, rather than by a physical push button.

[0108] Not limited to this, in other embodiments, a character input button may be displayed as a user interface in a part of the space-floating image FI provided by the space-floating image information display system 1. When the user touches the character input button, the system similarly transitions to a mode for inputting and drawing to the space-floating image FI.

[0109] In the character input mode, the space-floating image information display system 1 changes the entire screen of the space-floating image FI to, for example, white. Furthermore, in the character input mode, the system changes the frame 1202 of the space-floating image FI to a predetermined color, for example, red. The system three-dimensionally senses the movement and contact position of the user's finger UF on the surface of the space-floating image FI as described above. Based on this sensing, the system then draws the contact position corresponding to the movement of the finger UF on the surface of the space-floating image FI in a predetermined color, such as black or blue, different from the white background. This allows the trajectory of the movement of the finger UF to be drawn as a line on the white background surface of the space-floating image FI. This allows the user to draw desired characters or shapes while looking at the line they have drawn on the space-floating image FI.

[0110] The system controls the display so that the display state of the floating-in-space image FI differs between a state where the finger UF is not in contact with the surface of the floating-in-space image FI (in other words, a state where the fingertip is in front of the surface) and a state where the finger UF is in contact with the surface of the floating-in-space image FI (in other words, a state where the fingertip is inserted deep into the surface). This allows the user to more easily recognize whether or not their finger UF is in contact with the surface of the floating-in-space image FI.

[0111] FIG. 12 explains the steps when a user draws the letter L on the surface of the floating-in-space image FI. First, in step S121, the user inserts his / her finger UF into an arbitrary position on the floating-in-space image FI in a direction toward the depth (in other words, maintains contact). Next, in step S122, the user moves the finger UF downward on the floating-in-space image FI while still in contact. The downward arrow indicates the direction of movement of the finger UF (downward along the surface). Upon detecting this movement, the system draws a vertical line (first line) 1211 of the letter L as part of the floating-in-space image FI.

[0112] Next, step S122 shows a state in which the finger UF has reached the lower end position of the first vertical line 1211. Up to this point, the first vertical line 1211 has been drawn. In this state, contact by the finger UF is maintained. Next, in step S123, the user moves the finger UF to the right from the point at which the first line 1211 has been drawn. Upon detecting this movement, the system draws the horizontal line (second line 1212) of the letter L. The position of the finger UF has reached the right end position of the second horizontal line 1212.

[0113] Next, in step S124, the user removes the finger UF at the right end position of the second line 1212 from the surface of the floating-in-space image FI toward the user. The system detects this removal of the finger UF and concludes that drawing of the second line 1212 has ended. By performing the above series of operations, the user can draw the letter L on the floating-in-space image FI in one stroke.

[0114] When the user presses the character input button 1203 to enter character input mode, not only does the entire screen (background surface) of the space floating image FI change to white, but the frame 1202 of the space floating image FI also changes from green to red, for example. This is preferable because it allows the user to easily recognize that the character input mode is in effect. Furthermore, when the user wants to end the character input mode, all the user has to do is press the character input button 1203 again. In this case, the system ends the character input mode and returns to the previous mode. In the above example, the entire screen of the space floating image FI changes to white, but this is not limiting and the color may be other than white, for example, black or another color.

[0115] In addition, after the character input mode is set, when the user touches the floating-in-space image FI (contact with the finger UF), the frame 1202 of the floating-in-space image FI may change to, for example, red. By changing the frame 1202 of the floating-in-space image FI to red, the user can recognize that their finger is touching the surface of the floating-in-space image FI and that they can draw characters.

[0116] By moving the user's fingertip through the series of movements described above, the user can input and draw desired characters, figures, etc. as images of the space-floating image FI. Note that the frame 1202 of the space-floating image FI may also be a color other than red, for example, blue or another color. Furthermore, in the above example, the area in which images such as figures and characters can be input and drawn is the entire screen of the space-floating image FI. However, the area in which images can be input and drawn may also be a predetermined part of the screen of the space-floating image FI.

[0117] The user can not only use the space-floating image FI as a user interface such as an operation menu, but also, in the character input mode, can input or draw any character or figure on the space-floating image FI with his / her finger (or a pen made of conductive material, etc.). This is advantageous as it can further expand the range of application of space-floating images and space-floating image display devices compared to conventional devices.

[0118] <Second example of the configuration of the space floating image information display system> FIG. 13 shows the main components of a space-floating image information display system 1 according to one embodiment. The space-floating image information display system 1 of FIG. 13 is a system suitable for a user (surveillance user) to observe the space-floating image 3 from diagonally above. In the coordinate system (X, Y, Z) in FIG. 13, the housing 350 of the space-floating image information display system 1 is placed on a horizontal plane (XY plane), and the space-floating image 3 is formed at a slight angle in the front-to-back direction (Y direction) relative to the vertical direction (Z direction). When the surface of the space-floating image 3 is viewed directly from the user's viewpoint E, the viewpoint E is positioned slightly diagonally above the surface of the space-floating image 3 in the Y direction, aligned with the optical axis J2, as shown in the figure. The user can view the space-floating image 3 from the viewpoint E with a slightly downward angle in the Y direction.

[0119] The image display device 10 and other components are arranged in a predetermined positional relationship inside the housing 350. The top surface (XY plane) of the housing 350 forms an opening, and a retroreflective member 330 is arranged at a predetermined angle α1. The optical axis J1 of the image display device 10 faces obliquely upward at a predetermined angle β1 with respect to the Y direction.

[0120] The image display device 10 includes a liquid crystal display panel 11 as an image display element and a light source device 13 that generates light of a specific polarization with a narrow-angle diffusion characteristic. The liquid crystal display panel 11 can be used with a screen size ranging from a small one with a screen size of approximately 5 inches to a large one with a screen size of over 80 inches, and the display device 10 is configured with a panel selected from these. Image light from the liquid crystal display panel 11 is emitted along an optical axis J1 toward a retroreflective member 330 (also referred to as a retroreflective portion or retroreflector). Light from a narrow divergence angle light source device 13, which will be described later, is incident on the liquid crystal display panel 11. This generates an image light beam φ1 with a narrow divergence angle. The image light beam φ1 with a narrow divergence angle is incident on the retroreflective member 330 from below in the Z direction along the optical axis J1. Due to retroreflection by the retroreflective member 330, an image light beam φ2 with a narrow divergence angle is generated in the direction of the optical axis J2, above the retroreflective member 330 in the Z direction, according to the principle described above with reference to FIG. 4. The image light flux φ2 provides a space floating image 3 (space floating image 331 in FIG. 4) at a predetermined position outside the housing 350. The optical axis J2 points obliquely upward at a predetermined angle β2 with respect to the Y direction.

[0121] The space-floating image 3 is formed at a symmetrical position on the image display device 10, with the retroreflective member 330 as the plane of symmetry. The surface of the image display device 10 and the surface of the space-floating image 3 are arranged in positions that are approximately symmetrical or symmetrical with respect to the surface of the obliquely arranged retroreflective member 330. On the surface of the space-floating image 3, r2 indicates the center position corresponding to the optical axis J2, r1 indicates the lower end position corresponding to the lower ray of the image light beam φ2, and r3 indicates the upper end position corresponding to the upper ray of the image light beam φ2.

[0122] In this configuration, in order to eliminate the ghost images 332 and 333 generated by the retroreflective member 330 as explained in Fig. 4 and obtain a high-quality spatial floating image 3, an image light control sheet 334 (more specifically, see Fig. 5 and Fig. 6(A) above) is provided on the exit side of the liquid crystal display panel 11. This controls the diffusion characteristics in unnecessary directions.

[0123] Furthermore, as shown in FIG. 11 , the image light from the liquid crystal display panel 11 can theoretically have a high reflectance on a reflective member such as a retroreflective member 330, so it is recommended to use S-polarized waves (electromagnetic waves whose electric field component is perpendicular to the plane of incidence; S stands for Senkrecht). However, if a user wears polarized sunglasses, the floating image 3 will be reflected or absorbed by the polarized sunglasses. To address this, it is recommended to use P-polarized waves (electromagnetic waves whose electric field component is parallel to the plane of incidence; P stands for parallel). To achieve this, a depolarization element 339 is provided as an element that optically converts a portion of the image light of a specific polarization into the other polarization and converts it into pseudo-natural light. For example, the depolarization element 339 is disposed on the output side of the image light control sheet 334. This allows the user to monitor the floating image 3 clearly even when wearing polarized sunglasses.

[0124] Commercially available depolarizing element 339 includes Cosmoshine SRF (manufactured by Toyobo Co., Ltd.) and depolarizing adhesive (manufactured by Nagase & Co., Ltd.). In the case of Cosmoshine SRF (manufactured by Toyobo Co., Ltd.), by laminating the adhesive on an image display device, it is possible to reduce reflection at the interface and improve brightness. In addition, in the case of depolarizing adhesive (manufactured by Nagase & Co., Ltd.), a colorless transparent plate and an image display device are laminated together via the depolarizing adhesive.

[0125] In this embodiment, an image light control sheet 334B (similar to the image light control sheet 334; see FIG. 6B for details) is also provided on the image exit surface of the retroreflective member 330. This eliminates ghost images 332 and 333 (FIG. 4) that appear on both sides of the normal image of the space floating image 3 due to unwanted light.

[0126] In this embodiment, the retroreflective member 330 is tilted at a predetermined angle α1 with respect to the horizontal axis (Y direction), and the floating image 3 is generated obliquely with respect to the horizontal axis (especially at an angle closer to the vertical plane than the horizontal plane). Not limited to this, by changing the arrangement of the components, it is possible to design the position and inclination of the floating image 3.

[0127] In this embodiment, a first distance measuring device 340 is attached to a predetermined position on the housing 350. That is, the same sensing technology as in FIG. 7 is implemented in this system. This allows the user to access and interact with the floating image 3. The first sensing system including the first distance measuring device 340 detects the state of operation of the user's fingers or the like on the floating image 3. Furthermore, similar to (B) of FIG. 8 or FIG. 9, a second sensing system including a second distance measuring device 341 may be added.

[0128] The mounting position and field of view α3 of the first distance measuring device 340 may be appropriately selected so as to adequately cover the size of the floating image 3. In this example, the first distance measuring device 340 is mounted at the illustrated position on the rear side of the housing 350 in the Y direction (the rear side relative to the user and the floating image 3), on the extension of the slope of the retroreflective member 330, and slightly away so as not to block the image luminous flux of the image light. In this example, the field of view α3 (ranging from the upper end A to the lower end B) of the first distance measuring device 340 is set to a sufficiently wide field of view so as to cover the entire floating image 3 and the area including the face of the user viewing it from viewpoint E at the reference position (directly facing it). The field of view α3 includes the field of view α2 that captures the entire floating image 3. The field of view α2 corresponds to, for example, sensing planes a0, a1, a2, and a3 in FIG. 7.

[0129] The TOF sensor of the first ranging device 340 uses a ranging system in which the sensing surface of the space floating image 3 is divided into multiple areas, as shown in Figure 7 (or Figure 8). This increases the resolution of each sensing area. Furthermore, when using a second sensing technology using a CMOS sensor as shown in Figure 8 and Figure 9 (B), the detection accuracy can be further improved.

[0130] In this embodiment, a light source that emits visible light with a narrow-angle directivity is used as the light source device 13, and the first distance measuring device 340 is disposed at a position outside the narrow-angle image light beam on the housing 350 side. Also, the second distance measuring device 341 may be disposed. This can eliminate adverse effects on the sensing accuracy of the image light that forms the floating image 3 in space.

[0131] <Third Configuration Example of the Space Floating Video Information Display System> FIG. 14 shows another embodiment of the space-floating image information display system. The space-floating image information display system 1 of FIG. 13 is a system suitable for a user to observe the space-floating image 3 from diagonally above. In the coordinate system (X, Y, Z) in FIG. 14, the housing 350 of the space-floating image information display system 1 is placed on a horizontal plane (XY plane), and the space-floating image 3 is formed at a slight angle in the front-to-back direction (Y direction) with respect to the vertical direction (Z direction). When the surface of the space-floating image 3 is viewed directly from the user's viewpoint E, the viewpoint E is positioned slightly diagonally above the surface of the space-floating image 3 in the Y direction, aligned with the optical axis J2, as shown in the figure. The user can view the space-floating image 3 from the viewpoint E with a slightly downward angle in the Y direction.

[0132] The image display device 10, mirror 360, etc. are arranged in a predetermined positional relationship inside the housing 350. In the opening of the housing 350, which in this example has a surface (XZ surface) that stands in a substantially vertical direction, a retroreflective member 330 is arranged at a predetermined angle γ1 (a slightly downward tilted angle) with respect to the Z direction. The mirror 360 is a flat mirror.

[0133] In this embodiment, the image light from the image display device 10 is reflected by the mirror 360 and then incident on the retroreflective member 330. The housing 350 has a portion that protrudes upward in the Z direction, and the image display device 10 is disposed within this portion. The optical axis J1 of the image display device 10 faces downward in the Z direction and toward the rear in the Y direction, and is directed obliquely downward at a predetermined angle δ1 with respect to the Z direction.

[0134] The image display device 10 includes a liquid crystal display panel 11 as an image display element and a light source device 13 that generates light of a specific polarization with a narrow-angle diffusion characteristic. The liquid crystal display panel 11 can be used with a screen size ranging from a small one of approximately 5 inches to a large one exceeding 80 inches, and a panel selected from these is configured. Image light from the liquid crystal display panel 11 is reflected along optical axis J1 by a mirror 360, which is an optical path folding mirror, and is emitted toward the retroreflector 330 on the reflected optical axis J1B. Light from a light source device 13 with a narrow divergence angle, which will be described later, is incident on the liquid crystal display panel 11. This generates an image light beam φ1 with a narrow divergence angle. The image light beam φ1 with a narrow divergence angle is reflected by the mirror 360 and becomes an image light beam φ1B. The image light beam φ1B with a narrow divergence angle is incident on the retroreflector 330 from the right side in the Y direction along the optical axis J1B. Due to retroreflection by this retroreflective member 330, an image light beam φ2 with a narrow divergence angle is generated in the direction of optical axis J2 to the left of the retroreflective member 330 in the Y direction, according to the principle explained above in Figure 4. This image light beam φ2 produces a floating image 3 in space (a floating image 331 in Figure 4) at a predetermined position outside the opening of the housing 350. The optical axis J2 points obliquely upward at a predetermined angle δ2 with respect to the Y direction (an angle of (90 degrees - δ2) with respect to the Z direction).

[0135] The floating image 3 is formed at a position roughly symmetrical to the mirror 360, with the retroreflective member 330 as the plane of symmetry. In this embodiment, the mirror 360 is configured to fold the optical path, so the image display device 10 is positioned above the floating image 3 in the Z direction. As a result, a system can be realized in which the image light rays enter the retroreflective member 330 from obliquely above and exit obliquely above, forming the floating image 3 tilted obliquely as shown in the figure.

[0136] Furthermore, in order to form the image of the floating image 3 at an angle obliquely upward relative to the housing 350 (on the optical axis J2 shown in the figure), this can be achieved by arranging the retroreflective member 330 at a predetermined angle γ1 with respect to the vertical axis (Z direction) of the bottom surface of the housing 350 as shown in the figure. Furthermore, as a result of such a configuration in which the emission axis of the retroreflective member 330 is tilted slightly obliquely downward, it is possible to prevent degradation in the image quality of the floating image 3 that may occur when external light enters the retroreflective member 330 and enters the inside of the housing 350.

[0137] In order to eliminate ghost images (FIG. 4) that may occur in the space floating image 3 and obtain a space floating image 3 of higher image quality, in this embodiment as well, as in the second embodiment (FIG. 13), an image light control sheet 334 (FIGS. 5 and 6A) may be provided on the exit side of the liquid crystal display panel 11 to control the diffusion characteristics in unnecessary directions. In addition, an image light control sheet 334B (FIG. 6B) may also be provided on the image exit surface of the retroreflective member 330 to eliminate ghost images that occur on both sides of the normal image of the space floating image 3 due to unnecessary light.

[0138] By disposing the above-described structure inside the housing 350, it is possible to prevent external light from being incident on the retroreflective member 330 and prevent the occurrence of ghost images.

[0139] In this embodiment, the image light from the liquid crystal display panel 11 may be S-polarized as in FIG. 13, or, in the case of compatibility with polarized sunglasses, P-polarized light may be used and a depolarization element 339 may be provided.

[0140] In this embodiment, the retroreflective member 330 is tilted at a predetermined angle γ1 with respect to the vertical axis (Z direction), and the floating image 3 is generated at an angle with respect to the horizontal axis (especially at an angle closer to the vertical plane than the horizontal plane). Not limited to this, by changing the arrangement of the components, the position and inclination of the floating image 3 can be adjusted.

[0141] In this embodiment, a first distance measuring device 340 (FIG. 7) is attached to a predetermined position on the housing 350. That is, the same sensing technology as in FIG. 7 is implemented in this system. This allows the user to access and interact with the floating image 3. The first sensing system including the first distance measuring device 340 detects the state of operation of the user's fingers or the like on the floating image 3. Furthermore, similar to FIG. 8 and FIG. 9(B), a second sensing system including a second distance measuring device 341 may be added.

[0142] The mounting position and viewing angle γ3 of the first distance measuring device 340 may be selected appropriately so as to sufficiently cover the size of the floating in space image 3. In this example, the first distance measuring device 340 is mounted at the illustrated position on the bottom surface of the housing 350, near the front of the retroreflective member 330 in the Y direction, and slightly away so as not to block the image luminous flux of the image light. In this example, the viewing angle γ3 of the first distance measuring device 340 is set to a sufficiently wide viewing angle so as to cover the entire floating in space image 3 and an area including the face of the user viewing it from viewpoint E at the reference position. The viewing angle γ3 includes a viewing angle that captures the entire floating in space image 3.

[0143] In addition to the first distance measuring device 340, a second sensing technology using a CMOS sensor as shown in FIGS. 8 and 9B may also be used.

[0144] In this embodiment, a light source that emits visible light with narrow-angle directivity is used as the light source device 13, and the first distance measuring device 340 (and the second distance measuring device 341) is disposed at an outer position with respect to the narrow-angle image light beam on the housing 350 side. This can eliminate adverse effects on the sensing accuracy of the image light that forms the space floating image 3.

[0145] Furthermore, in this embodiment, a capacitive touch panel 361 may be disposed between the space floating image 3 and the retroreflective member 330, as shown in the figure, and fixed by a support member 362. The support member 362 may be, for example, frame-shaped, and supports the touch panel 361 inside. The support member 362 is fixed to, for example, the bottom surface of the housing 350. This touch panel 361 is made of a member that transmits image light for forming the space floating image 3 and light from the first distance measuring device 340.

[0146] This touch panel 361 is a capacitance type and detects the proximity of a user's fingers to the surface of the touch panel. Alternatively, this touch panel 361 detects the contact state of a user's fingers to the surface of the touch panel. By using the third sensing technology including this touch panel 361 in combination with the first sensing technology, etc., the detection accuracy can be further improved. Similarly, the size and installation position of this capacitance type touch panel 361 should be selected so that it can sufficiently cover the floating image 3 in space.

[0147] As a capacitive touch panel 361 capable of capturing highly accurate position information, for example, a projected capacitive touch panel can be employed. This type of touch panel is manufactured by patterning, for example, ITO transparent electrodes (Y-axis electrodes) with a fine line-to-line distance and thin copper films (X-axis electrodes) with a fine line-to-line distance on both sides of a transparent glass substrate by photolithographic etching. Therefore, when an object (e.g., a fingertip) approaches this transparent glass substrate, the change in capacitance is detected by each of the X-axis and Y-axis electrodes, and the relative coordinates of the object are obtained. Furthermore, this type of touch panel can achieve higher resolution as the line-to-line distance between the transparent electrodes becomes shorter, enabling multi-point detection. Therefore, this type of touch panel also allows simultaneous input by multiple fingers.

[0148] <First embodiment related to kiosk terminal> As one embodiment of the present invention, an example (referred to as embodiment 1) in which the space floating image information display system 1 described above is applied to a so-called kiosk terminal will be described below.

[0149] Kiosk terminals are information terminals that allow an unspecified number of people to access necessary information and use various services through a man-machine interface or user interface such as touch panel operation. Kiosk terminals are installed in public facilities, transportation facilities, entertainment facilities such as amusement parks, and more recently, inside convenience stores. Kiosk terminals are also used for selling various tickets and for providing government services (for example, issuing resident registration cards).

[0150] In the following description of the embodiments, an information terminal having a specific configuration is referred to as a "kiosk terminal." Instead of the term "kiosk terminal," it may be expressed as an "information terminal," an "information display device," an "information processing terminal," a "ticket issuing terminal," a "document issuing terminal," an "administrative terminal," a "service terminal," or the like. The term "kiosk terminal," which is primarily used in the description of the embodiments, is used as a representative example of these terms.

[0151] FIG. 15 shows an example of the appearance of a typical kiosk terminal based on conventional technology. This kiosk terminal 1400 has a metal housing 1450, for example, about 120 to 50 cm in height, and a liquid crystal display screen 1410 and input buttons 1420 are provided on the surface of the housing 1450 (the surface facing the user, particularly the inclined surface 1451). The liquid crystal display screen 1410 is part of a liquid crystal display device and is a screen with a touch panel that displays various information and accepts touch operations by the user. The input buttons 1420 are physical buttons for inputting a personal identification number or the like unique to the user, or touch buttons on a screen configured as a touch panel. In addition, an outlet 1430 is provided on a part of the surface of the housing 1450. The outlet 1430 is an outlet for taking out, for example, issued tickets or administrative documents as a result of operations on the kiosk terminal 1400.

[0152] FIG. 16 shows an example of the external configuration of a kiosk terminal as the space-floating image information display system of the first embodiment. FIG. 16 shows an overview of kiosk terminal 1500 as seen diagonally from the right. Kiosk terminal 1500 in FIG. 16 differs from kiosk terminal 1400 in FIG. 15 in the following ways. In kiosk terminal 1500 in FIG. 16, in addition to liquid crystal display screen 1510 using a liquid crystal display device on the surface of housing 1550 (particularly slope 1570), it is also provided with space-floating image display section 1520 below for displaying the above-mentioned space-floating image 3. In other words, this kiosk terminal 1500 has two screens, liquid crystal display screen 1510 and space-floating image display section 1520, displaying two types of images, and is configured such that slope 1570 separates the two display sections, liquid crystal display screen 1510 and space-floating image display section 1520.

[0153] In the configuration of Fig. 16, of the two screens, the screen (also referred to as the first screen) of the space-floating image display unit 1520 is used as the base. On this first screen, a concierge and operation menu are displayed as a user interface using the space-floating image 3. The first screen of the space-floating image display unit 1520 is basically an area of ​​a predetermined size in both the vertical and horizontal directions. In this example, the first screen has a slightly horizontally long size.

[0154] On the other hand, the liquid crystal display screen 1510 (also referred to as the second screen) can display any video, but is used for purposes such as advertising displays, similar to a typical kiosk terminal. The liquid crystal display screen 1510 is, for example, a liquid crystal touch panel screen equipped with a touch sensor.

[0155] In a modified example, the second screen, which is the liquid crystal display screen 1510, may be used as a user interface such as an operation menu in combination with the first screen of the space floating image display unit 1520.

[0156] As a modified example, a configuration in which the second screen, which is the liquid crystal display screen 1510, is not provided is also possible.

[0157] Also, as a modified example, both the concierge and the operation menu may be displayed as one space-floating image 3 on the first screen of the space-floating image display unit 1520 in Fig. 16. However, since the size of the first screen is limited, if both are displayed on the first screen, the displayed content may be small and detailed, making it difficult to see. Therefore, in the embodiment of Fig. 16, the display switching etc. is controlled so that one of the concierge and the operation menu is displayed as large as possible on the first screen.

[0158] Of course, the positional relationship between the liquid crystal display screen 1510 and the space-floating image display unit 1520 is not limited to the example configuration shown in FIG. 16 . For example, the top-bottom arrangement may be reversed. That is, on the slope 1570, the space-floating image display unit 1520 may be arranged at the top, and the liquid crystal display screen 1510 may be arranged at the bottom. Also, on the slope 1570, they may be arranged side by side on the left and right. However, as shown in FIG. 17 described below, in a configuration in which the kiosk terminal 1500 includes the space-floating image display unit 1520 in addition to the liquid crystal display screen 1510, it is more preferable in terms of the arrangement of components within the housing 1550 to arrange the liquid crystal display screen 1510 at the top and the space-floating image display unit 1520 at the bottom.

[0159] Furthermore, in the case of a configuration having two screens using two display units as shown in Fig. 16, to make it easier for the user to understand that the screens of the two display units are the liquid crystal display screen 1510 and the space-floating image display unit 1520, a message indicating this, such as "This is an LCD screen" or "This is a space-floating image," may be displayed on each screen. This improves usability for the user. Furthermore, instead of displaying this on the screen, it is also possible to physically display in advance in a position near the frame of each screen, such as "LCD screen" or "space-floating image."

[0160] In the example shown in Fig. 16, a concierge 1521 (also described as a person image or a concierge image) is displayed on the space-floating image display unit 1520 as the space-floating image 3. Details will be described later, but in the configuration shown in Fig. 16, the user can operate the screen (particularly the operation menu displayed on the space-floating image display unit 1520) by following the operation guidance by the concierge 1521 displayed on the space-floating image display unit 1520 in addition to the image or video information displayed on the liquid crystal display screen 1510. The concierge 1521 provides the user with operation guidance (also described as operation guide, operation explanation, etc.) by video and audio.

[0161] Therefore, the user feels as if there is a real person (concierge) on the kiosk terminal 1500. Moreover, the concierge carefully explains to the user how to operate the kiosk terminal 1500. Therefore, even a user who is using a kiosk terminal for the first time can operate the kiosk terminal 1500 more easily and receive the desired service without any confusion.

[0162] Here, the method for forming the space floating image 3 in the space floating image display unit 1520 in the embodiment of Fig. 16 may be the configuration of the retroreflective member 2 in Fig. 3 described above, or the configuration of the retroreflective member 330 in Fig. 13 or Fig. 14 described above. In addition to the above methods, other methods that can display a space floating image in the air may also be applied. Either method is applicable. In this embodiment, the case where the configuration in Fig. 14 is applied (Fig. 17) is particularly shown.

[0163] Furthermore, in the embodiment of Fig. 16, the detection method (sensing technology) for detecting user operations on the floating-in-space image 3 may be a method that combines the mid-air operation detection sensor 1351 and mid-air operation detection unit 1350 of Fig. 2 described above, or a method using the first or second sensing technology of Fig. 7 or Fig. 8 described above. In addition to the above methods, other methods that can detect mid-air operations on floating-in-space images may also be applied. Either method is applicable. In this embodiment, a case where the configurations of Fig. 7 and Fig. 14 are applied (Fig. 17) is particularly shown.

[0164] Furthermore, the kiosk terminal 1500 of FIG. 16 is equipped with two cameras 1541 and 1542, one on each side of the housing 1550 (particularly the inclined surface 1570). These cameras are stereo cameras corresponding to the imaging unit 1180 of FIG. 2. The cameras 1541 and 1542 capture an area including the face and iris of a user (e.g., a user operating the space-floating image 3) who is close to the front of the kiosk terminal 1500 (the near side in the Y direction in the figure). The control unit 1110 (FIG. 2) of the kiosk terminal 1500, which is the space-floating image information display system 1, is configured to perform processes such as detecting that a user has approached the kiosk terminal 1500, identifying and specifying the user, and authenticating the user, based on the images captured by the cameras 1541 and 1542. In this embodiment, two cameras 1541 and 1542 are provided, but one or more cameras may be provided. Furthermore, the camera may be placed inside the housing 1550, or the image information captured by an externally placed camera may be transmitted to the kiosk terminal 1500 via wire or wirelessly.

[0165] In addition, the two left and right cameras 1541, 1542 enable stereo photography, allowing the user's face and other objects to be captured as a three-dimensional image, and the distance and position to the face and other objects to be calculated. Therefore, this system can improve the accuracy of user face authentication compared to when a single camera captures a two-dimensional image of the user's face, and is also suitable from the perspective of preventing fraudulent use.

[0166] Furthermore, kiosk terminal 1500 in FIG. 16 is provided with two speakers, particularly two superdirectional speakers 1551 and 1552 (corresponding to speaker / superdirectional speaker 30 in FIG. 2), on the left and right sides of housing 1550. By providing the superdirectional speakers, it is possible to emit highly directional sound that can only be heard by the user who is operating kiosk terminal 1500. While kiosk terminal 1500 may be provided with speakers that output sound in the normal audible band, in the following cases, it is preferable to configure kiosk terminal 1500 with superdirectional speakers 1551 and 1552 in particular so that the sound cannot be heard by anyone other than the intended user. That is, when the sound emitted from kiosk terminal 1500 in response to user operation, for example, is to include the user's name, date of birth, or input number information, it is necessary to provide particularly high confidentiality and take security into consideration. In such cases, the use of superdirectional speakers is preferable.

[0167] 16, superdirectional speakers 1551 and 1552 are installed so as to protrude outside of housing 1550, but this is not limiting and they may be placed inside housing 1550. Superdirectional speakers 1551 and 1552 may output an audio signal that indicates that a button or the like on an operation menu of floating-in-space image 3 has been pressed so that only the user can hear it.

[0168] FIG. 17 is an explanatory diagram of the internal structure of the kiosk terminal 1500 of FIG. 16. FIG. 17 shows a YZ cross section of the inside of the housing 1550 of FIG. 16 seen from the right side. The upper part of the housing 1550 has a shape with a slope 1570. The upper part houses the image display device 10, mirror 360, retroreflective member 330, etc., which are components of the space floating image information display system 1 described in FIG. 14. This configuration is the same as that of FIG. 14, so a detailed description will be omitted. Note that the touch panel 361 of FIG. 14 is not applied to FIG. 17.

[0169] 2, a communication device incorporating the communication unit 1132, a power supply, and other components may be housed in the lower part of the housing 1550. In addition, a human presence sensor 1560 may be provided in the lower part (for example, the front) of the housing 1550 as shown in the figure. The human presence sensor 1560 detects when a person approaches the kiosk terminal 1500.

[0170] 17, similarly to FIG. 13, the direction of image light emitted from the image display device 10 in the housing 1550 is changed by a mirror 360 and made to enter the retroreflective member 330. Then, the retroreflective light by the retroreflective member 330 is emitted obliquely upward on the opposite side from the image display device 10 (the front side in the Y direction). As a result, a space-floating image 3 is generated at the position of the space-floating image display unit 1520.

[0171] The inclined surface 1570 of the housing 1550 has a predetermined angle ε1 with respect to the horizontal plane (Y direction). An opening (shown by a dotted line) of the space-floating image display unit 1520 is provided on the inclined surface 1570 at a position diagonally below the liquid crystal display screen 1510 above. A transparent member or the like may be provided in the opening. The image light from the retroreflective member 330 passes through the opening diagonally upward, forming a space-floating image 3, which is a real image, at a predetermined position outside the inclined surface 1570. This space-floating image 3 is formed tilted forward at a predetermined angle ε2 with respect to the inclined surface 1570. Therefore, to the user, the space-floating image 3 appears to be floating, jutting out in front of the inclined surface 1570. The user can easily view the space-floating image 3 displayed on the space-floating image display unit 1520 by gazing diagonally downward from viewpoint E diagonally above, as in FIG. 14 .

[0172] 14 is applied to the kiosk terminal 1500 in this way, the optical path is folded back by the mirror 360 inside the housing 1550, so a longer optical path can be ensured inside the housing 1550. This ensures a longer distance for the floating image 3 to project diagonally upward and forward relative to the retroreflective member 330. This also makes it possible to reduce the thickness dimension of the housing 1550 in the depth direction (Y direction).

[0173] 17, the first distance measuring device 340 (including the above-mentioned TOF sensor, etc.) senses a range covering the entirety of the Space Floating Image 3 through the opening of the Space Floating Image display unit 1520. Note that in FIG. 17, the bottom end B of the sensing range by the first distance measuring device 340 is diagonally upward with respect to the horizontal plane (Y direction) to match the opening, but this is not limiting and it may be closer to the horizontal direction (Y direction). The sensing range can be designed by changing the position of the opening of the housing 1550 or the arrangement position of the first distance measuring device 340.

[0174] Furthermore, a sensing system including the first distance measuring device 340 may be used to detect whether the user has come sufficiently close to the kiosk terminal 1500 (such as the slope 1570 of the housing 1550 or the floating image 3 in space), or may be used in conjunction with a camera to assist in this detection. Furthermore, when the first distance measuring device 340 and the second distance measuring device 341 as shown in Fig. 8 are provided in the configuration of Fig. 17, they may be placed, for example, on the left and right sides of the retroreflective member 330 in the X direction, with the optical axis of the distance measurement brought closer to the horizontal plane (Y direction).

[0175] The kiosk terminal 1500 uses a sensing system including the first distance measuring device 340 to sense the operation of the user's fingers on the space-floating image 3 on the space-floating image display unit 1520. The control unit 1110 of the kiosk terminal 1500 controls the display content of the space-floating image 3 on the space-floating image display unit 1520 (user interfaces such as a concierge 1521 and an operation menu described below) according to the detection information including that sensing.

[0176] The kiosk terminal, which is the space-floating image information display system of the first embodiment, is easy to use and convenient, and can provide users of the kiosk terminal with a suitable interface and necessary information using space-floating images. According to the system of the first embodiment, even if a user is operating a user interface such as an operation menu displayed as a space-floating image for the first time, is unfamiliar with operation, or is elderly, the user can be provided with operation guidance from a concierge displayed as a friendly human figure. This can avoid or reduce user input errors and operation errors on the space-floating image interface, enabling more reliable operation.

[0177] <Second embodiment related to kiosk terminal> FIG. 18 shows an example of the external configuration of another example (referred to as embodiment 2) related to a kiosk terminal. The kiosk terminal 1600 shown in FIG. 18 differs from the configuration of FIG. 16 in that it does not have a liquid crystal display screen 1510, but has a space-floating image display unit 1620 that covers roughly the entire surface of an inclined surface 1670 of a housing 1650. The size of one screen of this space-floating image display unit 1620 is larger than the size of the screen of the space-floating image display unit 1520 in FIG. 16. In the example of FIG. 18, the space-floating image display unit 1620 has one vertically long screen, and one space-floating image 3 is displayed. In this example, both a concierge and an operation menu are displayed side by side within the space-floating image 3 on this one screen, and the concierge provides guidance on operating the operation menu.

[0178] On the space-floating image display unit 1620, the space-floating image 3 is displayed on the near side of the slope 1670. In this example, the space-floating image 3 has a concierge 1621 displayed at the top and an operation menu 1622 displayed at the bottom. The operation menu 1622 includes, for example, operation buttons (indicated by circles and triangles). The concierge 1621 explains and guides the user on how to operate the operation menu 1622. In the case of the configuration in Fig. 18, as in the case of Fig. 16, the user can easily operate the kiosk terminal 1600, particularly the operation menu 1622, by following the guidance of the concierge 1621, and can receive the desired service.

[0179] 18 also includes cameras 1641 and 1642, super-directional speakers 1651 and 1652, and motion sensor 1660 in housing 1650, and is similar to the configuration in FIG.

[0180] Fig. 19 is an explanatory diagram of an example of the internal structure of the kiosk terminal 1600 of Fig. 18, showing a YZ cross section of the inside of the kiosk terminal 1600 seen from the right side. This kiosk terminal 1600 accommodates the components of the above-mentioned space-floating image information display system 1 of Fig. 13 inside a housing 1650. That is, the image display device 10, the retroreflective member 330, etc. are arranged at the top inside the housing 1650.

[0181] In the configuration of Fig. 19, the image display device 10 is disposed on the bottom surface of the upper part of the housing 1650 along a generally horizontal plane (more specifically, with the back side tilted slightly upward with respect to the Y direction), and the optical axis J1 of the image display device 10 faces generally vertically upward (Z direction). An opening (shown by a dotted line) for the space-floating image display unit 1620 is provided on an inclined surface 1670 on the upper part of the housing 1650. A retroreflective member 330 is disposed at a predetermined angle within the upper part of the housing 1650 near the opening. The inclined surface 1670 has a predetermined angle ε1, and the retroreflective member 330 is disposed at a predetermined angle ε3 with respect to the inclined surface 1670.

[0182] In the configuration of Fig. 19, similarly to the case of Fig. 13, image light from image display device 10 is incident on retroreflective member 330, and the retroreflected light is emitted in the opposite direction from image display device 10, i.e., toward the opening of slope 1670. As a result, a real image, a floating-in-space image 3, is generated at a predetermined position outside the opening of space-floating image display section 1620 on slope 1670. This floating-in-space image 3 is disposed obliquely at a predetermined angle ε2 with respect to slope 1670. A user can easily view this floating-in-space image 3 from viewpoint E diagonally above and with a line of sight diagonally below.

[0183] 19, a first distance measuring device 340 (including a TOF sensor, etc.) is installed at a predetermined position in the upper part of the housing 1650, in this example, at a position near the upper end of the retroreflective member 330. The optical axis of this first distance measuring device 340 is directed toward the space-floating image 3, and the viewing angle is set to be wide enough to cover the entire space-floating image 3. The sensing system implemented in this kiosk terminal 1600, including the first distance measuring device 340, senses the operation of the user's fingers UH on the space-floating image 3 on the space-floating image display unit 1620.

[0184] 18, both the figure of a concierge 1621 as a space-floating image 3 and an operation menu 1622 as a space-floating image 3 are simultaneously displayed on one space-floating image display unit 1620. The method of displaying a space-floating image on a kiosk terminal is not limited to this.

[0185] As a modified example, the display of the concierge and the display of the operation menu may be switched within one screen of the space floating image display unit 1620.

[0186] <Third embodiment related to kiosk terminal> Fig. 20 shows an example of the display of the space-floating image 3 on a kiosk terminal as an example (referred to as embodiment 3). The kiosk terminal in Fig. 20 shows a configuration based on the kiosk terminal 1500 in Fig. 16. The kiosk terminal in Fig. 20 is a kiosk terminal to which the space-floating image 3 formed by the space-floating image information display system 1 is applied as a user interface. Figs. 21 and 22 show the operation flow when a user operates the kiosk terminal in Fig. 20.

[0187] FIG. 21 shows a first operation flow related to the startup of the kiosk terminal of FIG. 20. The first operation flow shows basic operations. At the start of the flow in step S100, the kiosk terminal is in a standby state (in other words, a sleep state). In step S101, the kiosk terminal detects whether a user has approached the kiosk terminal. A user intending to use the kiosk terminal approaches the kiosk terminal, or the user stands at a predetermined position near the kiosk terminal. When the user approaches the kiosk terminal, the approach of the user to the kiosk terminal (particularly the front of the slope 1570) is detected based on images captured by cameras 1541 and 1542. Alternatively, instead of a camera, the approach of the user to the kiosk terminal 1500 may be detected by a human presence sensor 1560 using infrared rays as shown in FIG. 16 or the like. If the approach of the user to the kiosk terminal is detected (YES), the kiosk terminal transitions to a startup state in step S102. The kiosk terminal is then activated and available for use by the user.

[0188] In step S103, the kiosk terminal displays a concierge 1521 or an operation menu 1522 on the space-floating image display unit 1520, and receives an operation (in other words, an input) on the operation menu 1522 while guiding the user on the operation method, etc., and performs a predetermined process (application or service process) in response to the detection of the operation. In a specific example, initially, as shown in FIG. 20(A), the concierge 1521 is displayed on the space-floating image display unit 1520, and the concierge 1521 greets the user and provides operation guidance using video and audio. Then, after the greeting, etc., the display on the space-floating image display unit 1520 automatically transitions from the concierge 1521 to the operation menu 1522, as shown in FIG. 20(B). The user operates the operation menu 1522 with their fingers. The kiosk terminal detects the operation using sensing technology and performs a process in response to the detected operation.

[0189] The image of the concierge 1521 may be a video of an actual person filmed in advance, a human figure created using CG or the like, or an animation depicting a human figure. Furthermore, the image is not limited to a person, and may be an animal or a fictional character.

[0190] In step S104, it is detected whether the user has left the kiosk terminal. If the user has left the kiosk terminal, it is detected that the user has left the kiosk terminal (particularly the front slope 1570) based on images captured by cameras 1541 and 1542. Alternatively, instead of a camera, the user's departure from the kiosk terminal may be detected by a human sensor 1560 in FIG. 16 or the like. If it is detected that the user has left the kiosk terminal (Y), in step S105 the kiosk terminal transitions to a standby state.

[0191] Fig. 22 shows a second operation flow relating to the startup of the kiosk terminal in the third embodiment of Fig. 20. The second operation flow shows a more detailed example than the first operation flow. The second operation flow shows an operation flow for performing user authentication and the like after the kiosk terminal has entered a startup state.

[0192] In step S200, when the flow starts, the kiosk terminal is in a standby state. A user approaches the front of the kiosk terminal, or stands in a predetermined position near the kiosk terminal. In step S201, the kiosk terminal detects whether a user has approached the kiosk terminal (particularly the front of the slope 1570) by using stereo photography with two cameras, etc. If a user is detected (Y), in step S202, the kiosk terminal first displays a concierge 1521 on the floating-in-space image display unit 1520, as shown in FIG. 20(A), and greets the user with the image and audio of the concierge 1521, such as "Welcome, thank you for using our service," and also displays a greeting message.

[0193] Next, in step S203, the kiosk terminal authenticates the detected user. At this time, the kiosk terminal also displays a message indicating that authentication is in progress, such as "Authentication in progress," as floating image 3. The kiosk terminal may identify and specify the user by performing facial authentication of the approaching person (i.e., the detected user) based on images (especially facial images) captured by the two cameras 1551 and 1552. Alternatively, authentication may be performed based on an iris image of the user's eyes. Authentication here refers to determining whether the user is a legitimate user of this kiosk terminal who has been registered in advance in this system, based on stereo images obtained by capturing the face or iris of the user who is present within a predetermined range in front of the kiosk terminal using the two cameras.

[0194] Here, although authentication based on a user's image is possible using a single camera, using stereo images from two cameras 1551 and 1552 enables user authentication that is more accurate and less prone to fraud than authentication using a single camera image. For example, it becomes possible to detect fraudulent authentication attempts that use a face image printed on paper using an image of the face taken only from the front.

[0195] In addition to the above-mentioned facial recognition, other methods of user authentication can also be applied, such as the following: A method of authenticating a user by reading information from a user's personal identification card, such as a My Number card, or information from a commuter pass into a kiosk terminal is also applicable. Another method of authenticating a user is to exchange user information stored in a user's mobile terminal or the like via short-range wireless communication between the mobile terminal and the kiosk terminal.

[0196] Alternatively, a method can be applied in which code information such as a QR code (registered trademark) containing personal information (such as name, address, date of birth, and usage history of the kiosk terminal) that can identify the user is displayed on a mobile terminal such as a smartphone carried by the user, and the code information is presented to the space floating image display unit 1520 of the kiosk terminal (for example, held up to cameras 1551 and 1552).Then, the presented user information is read by the cameras 1551 and 1552 of the kiosk terminal, thereby performing user authentication.

[0197] In step S204, the kiosk terminal checks whether the authentication of the user was successful, and if successful, proceeds to step S206, and if not, proceeds to step S205. In step S205, the kiosk terminal prompts for another authentication method, and returns to step S203 to attempt authentication using the other authentication method. After going through the user authentication process, the kiosk terminal may present a predetermined operation menu screen (an operation menu compatible with an unspecified number of users) on the space-floating image display unit 1520, or may present a menu screen (a personalized operation menu) that is optimal for each individual user identified by authentication.

[0198] In a specific example, after successful authentication, as shown in (A) and (B) of FIG. 20, the display on the floating image display unit 1520 changes and transitions from a concierge 1521 to an operation menu 1522. The operation menu 1522 in (B) displays, for example, a message (text image) saying, "Please touch to select an item," and displays push buttons representing options and items, such as "1. Resident Certificate Issuance," "2. Ticket Purchase," "3. Registration Information," and "4. Other Menu." "1. Resident Certificate Issuance" is an item related to administrative procedures such as issuing a resident certificate. "2. Ticket Purchase" is an item related to purchasing or receiving tickets for concerts, movies, trains, etc. "3. Registration Information" is an item related to confirming or changing registration information related to the user. "4. Other Menu" is an item for transitioning to another menu. The user presses a button for the "2. Ticket Purchase" item, for example, depending on the purpose. The kiosk terminal transitions to the next operation menu display depending on the button pressed. A more detailed example is given below.

[0199] [Concierge guidance] In step S206 of Fig. 22, the space-floating image information display system 1 of the kiosk terminal accesses the kiosk server on the communication network via the communication unit 1132 (Fig. 2) and notifies the result information of the user authentication. The kiosk terminal also determines whether the user has used the kiosk terminal in the past by acquiring and referencing the user's attribute information from the user database in the kiosk server. Alternatively, the kiosk terminal may acquire and refer to the user's attribute information based on code information from the user's mobile terminal. The user's attribute information includes, for example, the user's gender, age, and past kiosk terminal usage history.

[0200] In step S206, the kiosk terminal may also acquire user information, including user attribute information, from the user's mobile terminal via wireless communication. The kiosk terminal may also acquire, in addition to the user attribute information, other predetermined data related to the user and the operation menu, such as ticket reservation information, previously entered and registered by the user, from an external device such as a kiosk server. In step S206, the kiosk terminal may also determine the user's age and other attributes based on images from cameras 1541 and 1542. The user attribute information acquired in step S206 is used in the determination of the next step, S207.

[0201] In the next step S207, the kiosk terminal determines whether the user is a "first-time user" or an "elderly user" based on the user attribute information obtained in step S206. If it is determined that the user meets such predetermined conditions (YES), the process proceeds to step S208, and if it is determined that the user does not meet the conditions (NO), the process proceeds to step S209.

[0202] The system branches into two responses depending on the determination result in step S207. Here, there are two main responses. One is the first processing flow (flow from step S209) intended for experienced users, and the other is the second processing flow (flow from step S208) intended for inexperienced users (first-time users or elderly users). In the first processing flow, the kiosk terminal omits operation guidance by a concierge or provides operation guidance that is simplified compared to the second processing flow. In the second processing flow, the kiosk terminal provides detailed operation guidance by a concierge.

[0203] In step S208, the kiosk terminal performs "audio-accompanied operation guidance by a concierge" on the space-floating image display unit 1520. Note that "concierge" means "guide," and for example, as shown in FIG. 20(A), a concierge 1521 (shown diagrammatically in the drawing) is displayed as a space-floating image 3, as an image of a person's upper body. The concierge 1521 uses voice and gestures to guide the user on how to operate the kiosk terminal, which uses space-floating image 3 as a user interface for the first time, or for elderly users who are unfamiliar with the operation. The concierge 1521 provides explanations and guidance on parts of the operation menu 1522, such as the one shown in (B), that the user does not understand.

[0204] More specifically, in step S208, first, as shown in (A) of FIG. 20, an image of a concierge 1521 appears on the screen of the space-floating image display unit 1520. The concierge 1521 outputs a text image and audio such as, "I will explain how to operate the kiosk terminal, so please operate the menu according to my explanation." The audio is provided by, for example, superdirectional speakers 1551 and 1552 of (A) of FIG. 20 in a way that only the user can hear. The concierge 1521 provides operation guidance by informing the user step by step how to use the kiosk terminal and the operation menu.

[0205] Next, in either step S208 or step S209, as shown in (A) to (B) of Fig. 20, the image of the concierge 1521 as the space-floating image 3 is switched to the image of the operation menu 1522 as the space-floating image 3. Here, in step S208, even after the screen of the space-floating image 3 is switched to the operation menu 1522, the guidance by the concierge 1521 continues. The kiosk terminal appropriately switches between the display of the concierge 1521 and the display of the operation menu 1522 on the space-floating image display unit 1520. The user operates the operation menu 1522 on the screen of the space-floating image display unit 1520 according to the operation guidance of the concierge 1521.

[0206] Next, in step S210, the user performs a predetermined series of operations on the kiosk terminal's operation menu 1522, such as entering a PIN number to issue a ticket. The kiosk terminal detects the operations, processes them, and issues a ticket. The user then collects the ticket from the ticket outlet 1530. If the user's desired operations and the corresponding kiosk terminal operation, such as ticket issuance, are completed (Y), the process proceeds to step S212.

[0207] In step S212, the kiosk terminal displays the operation menu 1622 again (for example, the first menu screen, or the next menu screen if there is a next menu, or the last menu screen, etc.) on the screen of the floating-in-space image display unit 1520. Alternatively, the kiosk terminal ends the display on the floating-in-space image display unit 1520. Furthermore, when the display ends, the kiosk terminal may finally display a concierge 1521 on the screen and output a message to the user indicating the end (such as "Thank you for using our service").

[0208] On the other hand, in step S209, the kiosk terminal displays a screen assuming that the user is not a first-time user but is somewhat familiar with the operation. The kiosk terminal switches the display on the screen of the space-floating image display unit 1520 to an operation menu 1522, as shown in (B) of Fig. 20. The user then performs desired operations such as selecting an item according to the operation menu 1622 on the screen. In step S211, similar to step S210, if a predetermined series of operations and the corresponding action (e.g., ticket issuance) are completed (Y), the process proceeds to step S212.

[0209] Here, even if the process proceeds to step S209, step S212 is provided in consideration of the possibility that the user's operation may not be successful. If the user's predetermined series of operations is not completed in step S211 (N), the process proceeds to step S212. In step S212, the kiosk terminal determines whether the user's operation is successful in the operation menu 1522. In a specific example, the kiosk terminal determines whether the user's operation has stopped (in other words, no input) in the operation menu 1522 for a period longer than a predetermined time (e.g., 30 seconds). The user may have stopped operation if, for example, they do not understand how to operate the operation menu 1522 as the floating image 3 in space. The kiosk terminal detects this long period of inactivity using a camera or sensing technology. If it is determined that the user has stopped operation for a long period of time (Y), the process proceeds to step S206; otherwise (N), the process proceeds to step S209.

[0210] In another example, in step S212, the kiosk terminal may determine whether the user is making an incorrect operation on the surface (operation menu 1522) of the floating image 3. For example, it may determine whether the user is trying to touch the button of an item at a position away from the button.

[0211] When the process proceeds from step S212 to step S208, the kiosk terminal displays a concierge 1521 on the space floating image display unit 1520 for the user, and the concierge 1521 provides operation guidance. At this time, the kiosk terminal may play back operation guidance with predefined content as operation guidance by the concierge 1521, but more preferably provides operation guidance for the part of the operation menu 1522 where the user stopped operating in step S212. For example, if the user's operation stopped on the operation menu after pressing the "2. Purchase tickets" button in (B) of Fig. 20, the kiosk terminal will provide guidance by the concierge 1521 on what operation to perform next in that operation menu.

[0212] The above-mentioned operation example can be similarly applied to the embodiment of Fig. 18, that is, the configuration example in which both the concierge 1621 and the operation menu 1622 are displayed side by side on the screen of one space floating image 3. In this case, it is possible for the concierge 1621 to provide operation guidance while pointing at the operation menu 1622 with, for example, his / her fingertip on the screen.

[0213] According to the third embodiment, when a user approaches the kiosk terminal, the kiosk terminal displays a concierge 1521 as floating image 3 and starts operation guidance by the concierge 1521. Therefore, even a user who is operating a kiosk terminal for the first time or an elderly user who is unfamiliar with operation can reliably perform predetermined operations by following the operation guidance by the concierge 1521 and the voice of the super-directional speaker (operation guidance that cannot be heard by others).

[0214] According to the third embodiment, the user is identified through authentication, and a concierge carefully explains the operation method to users who are operating the system for the first time, users who are unfamiliar with the operation, and elderly users based on the user's age and system usage history. Therefore, even such users can perform reliable key input operations on the operation menu as floating images. For experienced users, efficient service is also possible by omitting or simplifying the operation guide.

[0215] The following is also possible as a modification of the third embodiment. Fig. 25 shows a display example in this modification. The kiosk terminal displays an image of an operation menu 1522 as a base, as large as possible, within one screen of the Space Floating Image 3 displayed by the Space Floating Image Display Unit 1520. The kiosk terminal superimposes an image of a concierge 1521 on the operation menu 1522 in a relatively small size. The kiosk terminal then guides the user through the operation menu 1522 using the concierge 1521. Rather than a still image, it is preferable that the concierge 1521 be a moving image that includes, for example, gestures, hand movements, mouth and eye movements, etc. The kiosk terminal may appropriately change whether or not to display the concierge 1521 and the display position, etc., within the screen of the operation menu 1522.

[0216] <Fourth embodiment related to kiosk terminal> Fig. 23 shows another example (referred to as embodiment 4) in which a kiosk terminal and a mobile terminal carried by a user operate in cooperation with each other. The kiosk terminal 1700 in Fig. 23 is a kiosk terminal to which the space-floating image 3 formed by the space-floating image information display system 1 is applied as a user interface. When a user 1710 operates this kiosk terminal 1700, the kiosk terminal 1700 cooperates with a mobile terminal 1740 such as a smartphone carried by the user 1710. The kiosk terminal 1700 in Fig. 23 is based on the configuration of the kiosk terminal 1600 in Fig. 18.

[0217] As an example, a case will be described in which a user 1710 uses a kiosk terminal 1700 to carry out a procedure for purchasing a ticket (for example, a ticket to a concert). As in the above-described FIGS. 21 and 22 , when the user 1710 approaches the kiosk terminal 1700, the kiosk terminal 1700 authenticates the user 1710 and displays a concierge 1721 and an operation menu 1722 on the space-floating image display unit 1720. The user operates the operation menu 1722 following the guidance of the concierge 1721. For example, "2. Purchase tickets" is selected from the operation menu 1722 as shown in (B) of FIG. 20 . The kiosk terminal displays the operation menu 1722, which transitions from "2. Purchase tickets," on the screen of the space-floating image display unit 1720. The user 1720 performs detailed operations for purchasing tickets on the operation menu 1722.

[0218] The ticket is issued, for example, in a printed form on paper, and the user 1710 receives the paper ticket. Alternatively, the ticket can be obtained not only in a printed form but also in the form of a so-called electronic ticket (or "e-ticket"). In this case, instead of the printed form, the user 1710 receives the same information as the printed form ticket (i.e., the electronic ticket) on the mobile terminal 1740 owned by the user 1710. In this example, when operating the operation menu 1722, short-range wireless communication is performed between the mobile terminal 1740 and the kiosk terminal 1700 using a communication interface such as Bluetooth (registered trademark). The kiosk terminal 1700 directly transmits the electronic ticket to the user's mobile terminal 1740. At this time, the kiosk terminal uses a concierge 1721 to guide the user through the communication operation.

[0219] As another example, as shown in FIG. 23 , the kiosk terminal 1700 displays code information 1760 (in other words, a code image) such as a barcode or QR code (registered trademark) including the information content of the electronic ticket on the screen of the space-floating image display unit 1720 as space-floating image 3 (e.g., part of the operation menu 1722). The user 1710 reads the code information using his / her mobile terminal 1740. For example, the user 1710, following the guidance of the concierge 1721, performs an operation to read the code information 1760 displayed in the operation menu 1722 using a reader function such as a camera of the mobile terminal 1740. The concierge 1721 guides the user in the reading operation. The mobile terminal 1740 acquires the electronic ticket based on the read code information 1760 and stores it in a memory or the like within the mobile terminal 1740.

[0220] The items and information that the user 1710 can receive from the kiosk terminal 1700 are not limited to paper tickets and electronic tickets, but may also include, for example, coupons and point information that can be used for shopping, or even e-books. Tickets and coupons printed on paper may be lost, but electronic tickets and coupons are stored inside the mobile terminal 1740, reducing the possibility of loss. In addition to reducing the possibility of loss, storing electronic tickets and coupons inside the mobile terminal 1740 allows users to present the mobile terminal when actually using the ticket, or have the ticket issuer read the electronic ticket stored on the mobile terminal, compared to tickets and coupons printed on paper. This improves usability for users.

[0221] In response to a user's operation on the operation menu of the floating image 3, the kiosk terminal may transmit information related to the operation menu (for example, access history information, operation history information, etc.) to the user's mobile terminal via wireless communication.

[0222] <Fifth embodiment related to kiosk terminal> Fig. 24 shows, as an example (referred to as embodiment 5), a detailed example of the function (Fig. 12) that allows the user to input and draw any characters or figures to the above-mentioned floating image 3. For example, when the user receives a paper-based ticket or electronic ticket from a kiosk terminal through the series of procedures explained in Fig. 23 etc., the user "signs" to acknowledge that they have received the ticket.

[0223] The kiosk terminal 2000 in Fig. 24 is an example based on the configuration of the kiosk terminals in Fig. 18 and Fig. 23, and only the space-floating image display unit 2020 is shown. (A) shows a state in which a concierge 2021 is displayed as the space-floating image 3 on the space-floating image display unit 2020. (B) shows a state in which a user interface for signature input is displayed as the space-floating image 3. (C) shows a state in which the user has input and drawn a signature within the user interface. (D) shows a state in which a concierge 2021 is displayed as the space-floating image 3 upon completion of signature input.

[0224] After the kiosk terminal issues a ticket and the user receives it, the kiosk terminal displays a concierge 2021 as a space-floating image 3 on the space-floating image display unit 2020, as shown in (A) of FIG. 24. The concierge 2021 outputs a video and audio message to the user, such as "Please sign at the end to confirm that you have received your ticket." This prompts the user to sign the space-floating image 3 (in other words, sign as receipt).

[0225] Next, the kiosk terminal turns off the display of the concierge 2021, and displays a signature screen 2022 (in other words, a user interface for entering a signature) as a floating-in-space image 3 on the floating-in-space image display unit 2020, as shown in (B) of FIG. 20.

[0226] In the example of (B) in FIG. 24, the words "Please sign to receive the ticket" are displayed as text drawn as floating image 3 on the signing screen 2022. Below the words, a frame (signing frame area) 2023 indicating the area for signing is displayed. The frame 2021 is displayed, for example, on a predetermined background color (e.g., black), with the four sides of the frame in a predetermined color (e.g., green). When the fingertip is not inserted within the frame 2021, the frame 2021 is displayed in that predetermined color (e.g., green).

[0227] Next, as shown in FIG. 24(C), the user uses the fingertip of their finger UH or a predetermined conductive pen to draw their name, in this example, "Tom" in cursive, as a signature within frame 2023. At this time, the kiosk terminal, as described above (FIG. 12), uses sensing technology to detect the mid-air operation on the surface of the floating-in-space image 3, and draws a line segment in real time within the surface of the floating-in-space image 3 (within frame 2023) according to the detected contact position of the fingertip. Line segment 2024 is a single line corresponding to the input / drawn signature ("Tom"). Line segment 2024 is displayed in a predetermined color.

[0228] When the user has finished writing their name as a signature, they remove their fingertip from the signature frame area 2023 of the floating image 3. When the kiosk terminal determines and detects that a predetermined time (for example, 5 seconds) has passed since the fingertip was removed, it considers that the user's signature input is complete. The kiosk terminal then transitions to a screen 2025 indicating the completion of signature input, as shown in FIG. 24(D). On this screen 2025, for example, an image of the concierge 2021 is displayed, along with a message such as "Thank you for your autograph," and audio is output. This completes the series of operations for the user to "sign" to acknowledge that they have received the ticket.

[0229] Note that the technology shown in FIG. 24(B) in which a user uses their fingertip or the like to write their name as a signature within frame 2023 of the space-floating image 3 can be similarly applied to the technology described in FIG. 12 above. Also, in the description of FIG. 12, an example was described in which when a user inserts their fingertip deep into any position on the surface of the space-floating image 3, the frame of the entire space-floating image 3 changes to, for example, red. In contrast, in the example of FIG. 24(B), only a partial area (signature frame area 2023) within the space-floating image 3, rather than the frame of the entire space-floating image 3, is used as the area for inputting and drawing a signature. Then, when the kiosk terminal detects that a fingertip has been inserted into that area (in other words, that it has come into contact), it may change the frame 2023 of that area to, for example, red.

[0230] Furthermore, when the user places their fingertip at any position on the surface of the floating image 3 in space, that is, when there is contact, the frame line 2023 may be displayed in another predetermined color, such as blue, instead of red. Furthermore, when there is contact, the frame line 2023 may blink. Furthermore, when there is contact, the background area within the frame line 2023 may be changed to white, and the color of the lines drawn in that background area may be another predetermined color, such as black. Furthermore, the background area within the frame line 2023 may be black, and the color of the lines drawn in that background area may be white. In other words, the kiosk terminal controls the display in the sign frame area 2023 so that the user can easily recognize the lines of the characters and figures drawn in accordance with the movement of the user's fingertip in the sign frame area 2023. The kiosk terminal controls to change the display mode in the sign frame area 2023 depending on the state of the fingertip, such as whether or not the fingertip is in contact with the sign frame area 2023.

[0231] In conventional space-floating image display devices, the user mainly operates by selecting from buttons with options in an operation menu using space-floating images. In contrast, in the fifth embodiment, a function (sign in the air function) is provided that allows the user to input any character or the like as a sign by moving their finger on the space-floating image 3. Furthermore, with this function, the display of the frame line 2023 and the like changes depending on whether or not the user's fingertip is in contact with the surface of the space-floating image 3, making it easy for the user to understand whether or not the user's fingertip is in contact with the surface of the space-floating image 3, and to input a sign. In the above example, an example was shown in which input to the space-floating image 3 is used for signing, but this is not limited to this and it can also be used for other applications.

[0232] <First Configuration Example of Video Display Device> Next, a detailed configuration example of the space-floating image information display system 1 will be described using FIG. 26 and subsequent figures. FIG. 26 shows a more specific example of the configuration of the image display device 10 constituting the space-floating image information display system 1. The light source device 13 in FIG. 26 has a configuration similar to that of the light source device shown in FIG. 27 (described later). This light source device 13 is configured by housing LEDs, a collimator, a polarization conversion element, a composite diffusion block, a light guide, and the like in a case made of, for example, plastic. A liquid crystal display panel 11 is attached to the top surface of the light source device 13. Furthermore, an LED substrate 102 on which LED elements 102A, which are semiconductor light sources, and a control circuit for the LED elements are mounted are attached to one side of the case of the light source device 13. In addition, a heat sink (not shown), which is a member for cooling heat generated by the LED elements 102A and the control circuit, is attached to the outer surface of the LED substrate 102.

[0233] The liquid crystal display panel frame attached to the top surface of the case is configured to have attached thereto a liquid crystal display panel 11 attached to the frame, and further to have attached thereto a flexible printed circuit (FPC) and the like electrically connected to the liquid crystal display panel 11. That is, the liquid crystal display panel 11, which is a liquid crystal display element, generates a display image by modulating the intensity of transmitted light together with the LED elements 102A, which are solid-state light sources, based on a control signal from a control circuit constituting the electronic device.

[0234] <First configuration example of light source device> Next, a detailed description will be given of an example of the configuration of an optical system, such as a light source device, housed in a case in the first example of the image display device, with reference to Fig. 26 and Fig. 27. In Fig. 26, an LED 102A constituting a light source is attached at a predetermined position relative to a collimator 15. Note that Fig. 26 illustrates a cross section, so only one LED element and collimator are visible, but a plurality of LED elements are arranged on the LED substrate 102, and a plurality of collimators are arranged corresponding to the LED elements. Each collimator 15 is formed of a light-transmitting resin, such as acrylic. The collimator 15 has a conical convex outer circumferential surface 156 obtained by rotating a parabolic cross section, and a recess 153 with a convex portion (i.e., a convex lens surface) 157 formed in the center of its apex (the side in contact with the LED substrate 102).

[0235] Furthermore, the central part of the flat part (the side opposite to the apex) of the collimator 15 has a convex lens surface 154 that protrudes outward (or may be a concave lens surface that is recessed inward). The parabolic surface 156 that forms the outer peripheral surface of the cone shape of the collimator 15 is set within an angle range that allows total internal reflection of the light emitted from the LED element 102A in the peripheral direction, or a reflective surface is formed.

[0236] The LED elements 102A (LED elements 14a and 14b in FIG. 17A) are arranged at predetermined positions on the surface of the LED substrate 102. The LED substrate 102 is fixed to the collimator 15 so that the LED elements 102A on the surface are positioned in the centers of the recesses 153.

[0237] With this configuration, the collimator 15 described above focuses light emitted from the LED 102A, particularly light emitted upward from the central portion (toward the right in the drawing), by the two convex lens surfaces 157, 154 that form the outer shape of the collimator 15, to form approximately parallel light. Light emitted from other portions toward the periphery is reflected by the parabolic surface that forms the conical outer surface of the collimator 15, and similarly focused to form approximately parallel light. In other words, the collimator 15, which has a convex lens in the center and a parabolic surface in the periphery, makes it possible to extract almost all of the light generated by the LED element 102A as parallel light. This improves the utilization efficiency of the generated light.

[0238] A polarization conversion element 21 (in other words, a polarization conversion member) is provided on the light exit side of the collimator 15. The polarization conversion element 21 is an element that converts the polarization characteristics of incident light. As shown in FIG. 27A, the polarization conversion element 21 is configured by combining a columnar light-transmitting member having a parallelogram cross section (parallelogram prism) with a columnar light-transmitting member having a triangular cross section (triangular prism), and arranging a plurality of these elements in an array parallel to a plane perpendicular to the optical axis of the collimated light from the collimator 15. Furthermore, polarization beam splitters (PBS films) 211 and reflective films 212 are alternately provided at the interfaces between adjacent light-transmitting members arranged in the array. In addition, a λ / 2 phase plate 213 (in other words, a half-wave plate) is provided on the exit surface from which light that enters the polarization conversion element 21 and passes through the PBS film 211 exits.

[0239] 27(A), a rectangular synthetic diffusion block 16 is further provided on the exit surface of polarization conversion element 21. Light emitted from LED element 102A is converted into parallel light by the action of collimator 15, and after its polarization characteristics are converted through polarization conversion element 21, it enters synthetic diffusion block 16 and is diffused by texture 161 on the exit side before reaching light guide 17.

[0240] As shown in Fig. 22(B), light guide 17 is a rod-shaped member having a substantially triangular cross section, made of a translucent resin such as acrylic. As also shown in Fig. 26, light guide 17 includes light guide light incident portion 171 including an incident surface that faces the exit surface of synthetic diffusion block 16 via first diffuser plate 18a, light guide light reflecting portion 172 including a reflective surface that forms an inclined surface, and light guide light exit portion 173 including an exit surface that faces liquid crystal display panel 11 via second diffuser plate 18b.

[0241] 27(B), a large number of reflective surfaces 172a and connecting surfaces 172b are alternately formed in a sawtooth pattern on the light guide light reflecting portion 172 of the light guide 17. The reflective surfaces 172a (line segments sloping upward to the right in the drawing) form an angle αn (n is a natural number ranging from 1 to 130 in this example) with the horizontal plane. As an example, αn is set to 43 degrees or less (but 0 degree or more).

[0242] Light guide entrance portion 171 is formed in a curved convex shape inclined toward the light source. Accordingly, parallel light from the exit surface of synthetic diffusion block 16 is diffused and incident via first diffuser plate 18a. As is clear from the drawing, this incident light is slightly bent and deflected upward by light guide entrance portion 171, reaches light guide light reflecting portion 172, and is reflected there. This reflected light exits from the exit surface of light guide exit portion 173, which is located above in the drawing, and reaches liquid crystal display panel 11 provided opposite that exit surface.

[0243] According to the above-described image display device 10, it is possible to further improve light utilization efficiency and uniform illumination characteristics, and to manufacture the device in a compact size and at low cost, including the modularized S-polarized light source device 13. In the above description, the polarization conversion element 21 is attached after the collimator 15, but the invention is not limited to this and the polarization conversion element 21 may be provided in the optical path leading to the liquid crystal display panel 11.

[0244] The light guide light reflecting portion 172 has a number of alternating reflective surfaces 172a and connecting surfaces 172b formed in a sawtooth pattern, and the illumination light beam is totally reflected by each reflective surface 172a and directed upward. Furthermore, a narrow-angle diffuser plate is provided in the light guide light exiting portion 173, causing the light beam to be diffused as a substantially parallel beam and incident on the light redirecting panel 54, which adjusts the directivity, and then obliquely incident on the liquid crystal display panel 11. In this embodiment, the light redirecting panel 54 is provided between the exit surface of the light guide 17 and the liquid crystal display panel 11, but the same effect can be achieved by providing the light redirecting panel 54 on the exit surface of the liquid crystal display panel 11.

[0245] <Second Configuration Example of Video Display Device> Next, another example of the specific configuration of the image display device 10 will be described with reference to Fig. 28. In the light source device of the image display device 10 of Fig. 28, as in the previous example, a plurality of LED elements constituting a light source are provided on an LED substrate 102 (however, since this is a cross-sectional view, only one is shown). These LED elements are attached at predetermined positions relative to a collimator 15. The divergent beam of light from the LED elements (light containing a mixture of P-polarized and S-polarized light) is converted into a substantially parallel beam by the collimator 15 and reflected toward the liquid crystal display panel 11 by the reflective surface of the reflective light guide 304. The reflected light is incident on a reflective polarizing plate 49 arranged between the liquid crystal display panel 11 and the reflective light guide 304.

[0246] The reflective polarizing plate 49 transmits a specific polarized wave (for example, P-polarized light) and causes it to enter the liquid crystal display panel 11. The reflective polarizing plate 49 reflects the other polarized wave (for example, S-polarized light) and directs it back toward the reflective light guide 304. The reflective polarizing plate 49 is installed at an angle not perpendicular to the chief ray of the light from the reflective surface of the reflective light guide 304, and the chief ray of the light reflected by the reflective polarizing plate 49 enters the transmission surface of the reflective light guide 304.

[0247] The light incident on the transmission surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304, passes through the λ / 4 plate 270 which is a retardation plate, and is reflected by the reflector 271. The light reflected by the reflector 271 passes through the λ / 4 plate 270 again, and passes through the transmission surface of the reflective light guide 304. The light that has passed through the transmission surface of the reflective light guide 304 enters the reflective polarizer 49 again.

[0248] At this time, the light that re-enters the reflective polarizer 49 has passed through the λ / 4 plate 270 twice, and therefore has been converted into a polarized wave (for example, P-polarized wave) that is transmitted through the reflective polarizer 49. Therefore, the polarization-converted light transmits through the reflective polarizer 49 and enters the liquid crystal display panel 11. Note that, as explained above, the polarization design related to the polarization conversion may be configured to reverse the polarization of S-polarized wave and P-polarized wave.

[0249] As a result, the light from the LED elements serving as the light source is aligned to a specific polarization (for example, P polarization), enters the liquid crystal display panel 11, and is brightness-modulated in accordance with the video signal to display an image on the panel surface.

[0250] The collimators 15 in Fig. 28 are each made of, for example, a light-transmitting resin such as acrylic or glass. Similar to the collimator 15 in Fig. 27, the collimator 15 in Fig. 28 may have a cone-shaped convex outer surface obtained by rotating a parabolic cross section, and may have a concave portion at the top with a convex portion (i.e., a convex lens surface) formed in the center. Furthermore, the center of the flat portion of the collimator 15 may have a convex lens surface protruding outward (or a concave lens surface recessed inward). The parabolic surface forming the cone-shaped outer surface of the collimator 15 is set within an angle range that allows total internal reflection of the light emitted from the LED in the peripheral direction, or a reflective surface is formed.

[0251] 28, the LED elements are arranged at predetermined positions on the surface of the LED substrate 102. The LEDs on the surface of the LED substrate 102 are arranged and fixed so as to be positioned at the center of the apex of the convex cone shape (or in the concave portion if the apex has a concave portion) relative to the collimator 15.

[0252] With this configuration, the collimator 15 focuses the light emitted from the LED element, particularly the light emitted from the central portion, into parallel light by the convex lens surface that forms the outer shape of the collimator 15. Light emitted from other portions toward the periphery is reflected by the parabolic surface that forms the outer peripheral surface of the cone shape of the collimator 15, and is similarly focused into parallel light. In other words, the collimator 15, which has a convex lens in the center and a parabolic surface in the periphery, makes it possible to extract almost all of the light generated by the LED element as parallel light, thereby improving the utilization efficiency of the generated light.

[0253] The configuration of the light source device 13 and the like explained above can be applied as the light source device 13 of the image display device 10 constituting the space floating image information display system 1 shown in the above-mentioned FIGS.

[0254] Furthermore, the light converted into approximately parallel light by the collimator 15 shown in Fig. 28 is reflected by the reflective light guide 304. Of this reflected light, light of a specific polarized wave is transmitted through the reflective polarizing plate 49 due to the action of the reflective polarizing plate 49, and light of the other polarized wave reflected by the action of the reflective polarizing plate 49 is transmitted again through the light guide 304. This light is reflected by the reflector 271 located opposite the liquid crystal display panel 11 with respect to the reflective light guide 304. At this time, this light is polarized and converted by passing twice through the λ / 4 plate 270, which is a retardation plate.

[0255] The light reflected by the reflector 271 passes through the light guide 304 again and is incident on the reflective polarizer 49 provided on the opposite surface. Since this incident light has been polarization-converted, it passes through the reflective polarizer 49, aligns its polarization direction, and is incident on the liquid crystal display panel 11. As a result, all of the light from the light source can be used, and the geometrical optical utilization efficiency of light is doubled. Furthermore, since the degree of polarization (in other words, the extinction ratio) of the reflective polarizer is also included in the extinction ratio of the entire system, the use of the light source device 13 of this embodiment significantly improves the contrast ratio of the entire display device.

[0256] The angle of light reflection and diffusion at each reflective surface can be adjusted by adjusting the surface roughness of the reflective surface of reflective light guide 304 and the surface roughness of reflector 271. The surface roughness of the reflective surface of reflective light guide 304 and the surface roughness of reflector 271 can be adjusted for each design to optimize the uniformity of light incident on liquid crystal display panel 11.

[0257] <Embodiment relating to vending machine> Here, as another embodiment of the present invention, an example in which the space floating image information display system is applied to a vending machine will be described with reference to Fig. 29. Fig. 29 is a diagram showing the case in which the present invention is applied to a vending machine for drinks, for example.

[0258] 29, vending machine main body 2900 is equipped with space floating image display unit 2920. Vending machine 2900 also includes drink display unit 2980 that displays drinks sold by vending machine 2900, bill slot 2981 for inserting bills, coin slot 2982 for inserting coins, change outlet 2983 for removing change, and drink outlet 2984 for removing drinks purchased by the user.

[0259] The vending machine main body 2900 is equipped with a camera or a motion sensor, similar to a kiosk terminal, and when a user approaches, the space-floating image display unit 2920 is activated. Next, as shown in FIG. 29, a concierge appears on the space-floating image display unit 2920 and speaks to the user, for example, "Welcome. Thank you for your patronage. The screen will change to numeric buttons. Please select the product number you would like." After that, the concierge disappears from the space-floating image display unit 2920, and then the numeric buttons and an enter button are displayed. At this time, although not shown, a cancel button and a back button may be displayed in addition to the numeric buttons and the enter button.

[0260] The user selects a drink by operating the number buttons and enter button displayed on the floating-in-space image display unit 2920, and inserts a predetermined amount of money into the bill slot 2981 or coin slot 2982, whereupon the drink is dispensed in a form that can be taken out from the drink outlet 2984. After that, the number buttons and enter button disappear from the floating-in-space image display unit 2920, and the concierge appears again, uttering a voice such as, "Thank you very much. We look forward to seeing you again." Even in this case, as with the kiosk terminal, the voice may be emitted from a normal speaker, or may be emitted from a superdirectional speaker so that only the user can hear it.

[0261] By performing the above series of operations, the user can purchase the desired drink. Note that in the example of Fig. 29, an example of only the space-floating image display unit is shown, but the vending machine may also be equipped with both a liquid crystal display device and a space-floating image display unit, as in the example of the kiosk terminal, and may also be equipped with two or more space-floating image display units instead of one. Note that if two space-floating image display units are equipped, a concierge may be displayed on one of the space-floating image display units, and number buttons and a confirm button may be displayed on the other space-floating image display unit.

[0262] Alternatively, a plurality of different human figures or animated characters of different ages and genders may be displayed as concierges. Data for displaying the plurality of different human figures or animated characters of different ages and genders may be stored in non-volatile memory 1108 of FIG. 2, and one of the plurality of human figures or animated characters may be selected as appropriate and displayed on the floating-in-space image display unit as the concierge. In this case, which human figure or character to display may be determined depending on the user's attributes (such as age).

[0263] As described above, this embodiment is also equipped with a floating image display unit, just like the kiosk terminal, so that users can select and purchase products without contact. Furthermore, the floating image is displayed when the user approaches the vending machine, so a concierge figure or character can be displayed using a display method not found in conventional vending machines. As a result, compared to regular vending machines, the novelty of the image has the effect of attracting more users, and it is expected to contribute to increasing sales of products from the vending machine.

[0264] The present invention has been specifically described above based on the embodiments, but the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the spirit of the invention. In each embodiment, components can be added, deleted, or replaced, except for essential components. Unless otherwise specified, each component may be singular or plural. A combination of each embodiment is also possible.

[0265] The technology according to the embodiment displays high-resolution, high-brightness floating images in a floating state, allowing users to operate the system without worrying about contact infection. Applying the technology according to the embodiment to a system used by an unspecified number of users reduces the risk of contact infection and provides a contactless user interface that can be used without anxiety. The present invention, which provides such technology, contributes to the "Good Health and Well-Being" goal, one of the Sustainable Development Goals (SDGs) advocated by the United Nations.

[0266] Furthermore, the technology according to the embodiment reduces the divergence angle of the emitted image light and aligns it with a specific polarization, thereby efficiently reflecting only the normal reflected light from the retroreflective material, resulting in high light utilization efficiency and bright, clear floating images in space. The technology according to the embodiment can provide a highly usable non-contact user interface that can significantly reduce power consumption. The present invention, which provides such technology, contributes to the achievement of the United Nations' Sustainable Development Goals (SDGs): "Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation" and "Make cities and towns sustainable."

[0267] Furthermore, the technology according to the embodiment enables the formation of a floating image in space using highly directional (linear) video light. The technology according to the present embodiment makes it possible to provide a non-contact user interface with low risk of people other than the user viewing the floating image, even when displaying images that require high security, such as those on kiosk terminals, or highly confidential images that should be kept secret from people directly facing the user, by displaying highly directional video light. By providing the above-described technology, the present invention contributes to "Sustainable Cities and Communities," one of the Sustainable Development Goals (SDGs) advocated by the United Nations. [Explanation of symbols]

[0268] 1...space-floating image information display system, 2...retroreflective member, 3...space-floating image, 10...image display device, 11...liquid crystal display panel, 13...light source device, 1500...kiosk terminal, 1510...liquid crystal display screen, 1520...space-floating image display unit, 1521...concierge, 1522...operation menu, 1530...retrieval port, 1541, 1542...camera, 1550...casing, 1551, 1552...ultra-directional speaker, 1560...human presence sensor, 1570...slope.

Claims

1. A space floating image information display system that forms a space floating image in the air, an image display device disposed on the housing and configured to display an image of at least one object; a retroreflection unit disposed on the housing, for retroreflecting image light from the image display device to form the floating image in the air; a sensing system including a sensor for detecting a user's operation on the floating image in space; a control device that executes a predetermined process based on the user's operation detected by the sensing system, The sensing system detects a contact state of the user's finger or belongings, and moves the user's finger or belongings in the contact state, and a line is drawn as the floating image in space at the contact position on the plane of the floating image in space. A floating visual information display system.

2. 2. The space floating image information display system according to claim 1, A frame is displayed in a drawing area of ​​the entire or part of the surface of the floating image in space, and the control device controls the display color of the frame so that the color of the frame varies depending on whether the user's fingers or the belongings are in contact with the surface of the floating image in space. A floating visual information display system.

3. 2. The space floating image information display system according to claim 1, Displaying a frame on the entire or part of the drawing area of ​​the surface of the space floating image; The control device controls the background color of the area inside the frame so that the background color of the area inside the frame varies depending on whether or not the user's fingers or the belongings contact the area inside the frame of the surface of the floating image in space. A floating visual information display system.

4. 2. The space floating image information display system according to claim 1, Displaying a frame on the entire or part of the drawing area of ​​the surface of the space floating image; the control device switches between a mode in which the line is drawn and a mode in which the line is not drawn, and controls the display so that the color of the frame differs between the mode in which the line is drawn and the mode in which the line is not drawn. A floating visual information display system.

5. 2. The space floating image information display system according to claim 1, Displaying a frame on the entire or part of the drawing area of ​​the surface of the space floating image; The control device controls the display so that the frame blinks when the user's finger or the belongings are in contact with the surface of the floating image in space. A floating visual information display system.

6. 2. The space floating image information display system according to claim 1, a physical button at a predetermined location on the housing for transitioning to a mode in which the operation on the surface of the floating image in space is detected and the line is drawn; A floating visual information display system.

7. 2. The space floating image information display system according to claim 1, a button for transitioning to a mode in which the operation on the surface of the floating image in space is detected and the line is drawn is displayed as a part of the floating image in space; A floating visual information display system.

8. 2. The space floating image information display system according to claim 1, The control device acquires the lines drawn as characters or figures on the surface of the floating image in space as input information by the user. A floating visual information display system.

9. 9. The space floating image information display system according to claim 8, The control device acquires the lines drawn as characters or figures on the surface of the floating image in space as a signature by the user. A floating visual information display system.

10. 2. The space floating image information display system according to claim 1, The line drawn on the surface of the floating image in space is information about the user's name or attributes. A floating visual information display system.

Images