Detection device and spatial projection device

Abstract

To provide a detection device, a detection method and a spatial image projection device, which allow for detecting spatial manipulation using a simple configuration.SOLUTION: A detection device is provided, comprising: a light guide optical system 30 configured to spatially project an image by forming an image of projection light in a spatial imaging plane 40; and a sensor 70 provided in the same plane as the spatial imaging plane 40 and configured to detect user manipulation in a depth direction of the image by irradiating laser light with an amplitude in an out-of-plane direction of the spatial imaging plane 40.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a detection device, a detection method and a spatial projection device. [Background technology]

[0002] Conventionally, techniques for detecting operations on spatially projected images have been disclosed. For example, a stereoscopic two-dimensional image display device in Patent Document 1 includes a three-dimensional position detection sensor that is arranged in correspondence with a stereoscopic image display surface in space and outputs a signal corresponding to a position where an external physical action is received, and a speaker that outputs sound. The three-dimensional position detection sensor has an X-direction detection line sensor with a light-emitting portion and a light-receiving portion that face each other on four sides of a rectangular frame that defines a detection plane, and a Y-direction detection line sensor with a light-emitting portion and a light-receiving portion that face each other. In addition, the three-dimensional position detection sensor is capable of detecting the three-dimensional position of an object to be detected by arranging the X-direction detection line sensor and the Y-direction detection line sensor in multiple layers in the Z direction, which is the depth direction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-141102 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, a three-dimensional position detection sensor such as that disclosed in Patent Document 1 has a complicated configuration in which many line sensors are provided to surround the three-dimensional spatial region to be detected.

[0005] An object of the present invention is to provide a detection device, a detection method, and a spatial projection device that are capable of detecting a spatial operation with a simple configuration. [Means for solving the problem]

[0006] The detection device of the present invention includes a light guide optical system that projects an image into space by focusing a projection light on a spatial imaging plane, and a light guide optical system that projects an image into space on the same plane as the spatial imaging plane. and approximately at the center in the vertical direction of the spatial imaging plane. and a sensor that detects a user operation in the depth direction relative to the image by irradiating a laser light having an amplitude in a direction outside the plane of the spatial imaging plane.

[0008] The spatial projection device of the present invention comprises: The above-mentioned detection device The present invention is characterized by comprising: Effect of the Invention

[0009] According to the present invention, it is possible to provide a detection device, a detection method, and a spatial projection device that are capable of detecting spatial operations with a simple configuration. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing a spatial projection device according to an embodiment of the present invention as viewed from the planar side. [Diagram 2] 1 is a diagram showing a configuration of a projection device according to an embodiment of the present invention. [Diagram 3] 1 is a schematic diagram showing a spatial projection device according to an embodiment of the present invention as viewed from the front side; [Figure 4] 1 is a perspective view showing a positional relationship between a scanning range of a TOF lidar sensor according to an embodiment of the present invention and a detection area. [Diagram 5] 1 is a schematic diagram of a spatial projection device in a first modified example according to an embodiment of the present invention, as viewed from the planar side. [Figure 6] 10A and 10B are schematic diagrams showing response outputs of a spatial projection device in modified examples according to an embodiment of the present invention, where FIG. 10A shows modified example 2 and FIG. [Figure 7] 11A and 11B are schematic diagrams showing the response output of a spatial projection device in a fourth modified example of an embodiment of the present invention, in which (a) is a diagram showing a pointer image in the fourth modified example displayed in a spatial projection image, and (b) is a diagram showing how the color of the pointer image changes. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described. Fig. 1 is a schematic plan view of a spatial projection device 100 in a spatial projection system 1. The spatial projection device 100 includes a projection device 10 (projector), an imaging medium 20 onto which projection light P1 emitted from the projection device 10 is irradiated, a light guiding optical system 30 that guides the light projected onto the imaging medium 20, and a spatial imaging surface 40 (spatial imaging unit) on which the light guided by the light guiding optical system 30 is re-imaged. The spatial projection device 100 also includes a control device 60, a TOF lidar sensor 70, and a speaker 81 that is an output unit 80, which are connected to each other by wire or wirelessly. In the spatial projection device 100, a projection image 2a emitted from the projection device 10 and formed on an imaging medium 20 is diffused and transmitted from the imaging medium 20, enters the light-guiding optical system 30, and is re-imaged on a spatial imaging surface 40 by the light-guiding optical system 30, so that a spatial projection image 4a floating in the air can be visually recognized by a viewer 50. Furthermore, the spatial projection device 100 can detect an input operation by a finger 51 or the like on the spatial projection image 4a and execute various response processes.

[0012] First, the configuration of the projection device 10 will be described with reference to Fig. 2. The projection device 10 includes a storage unit 11, a processing unit 12, a projection unit 13, an operation unit 14, a communication unit 15, and an audio processing unit 16, which are connected to each other via an internal bus. The storage unit 11 is configured with, for example, an SSD (Solid State Drive) or an SRAM (Static Random Access Memory). The storage unit 11 stores data such as image data, video data, and control programs (not shown). The processing unit 12 is configured with a CPU, a microcomputer, etc., and reads out the control programs stored in the storage unit 11 to control the projection device 10.

[0013] The projection unit 13 forms an image from the image data sent from the processing unit 12 at a frame rate according to a preset image format, and outputs the image to the outside as projection light P1. The projection device 10 of this embodiment is a DLP (Digital Light Processing) type projection device. For example, the projection unit 13 can form color image light by irradiating blue wavelength band light, green wavelength band light, and red wavelength band light emitted by an internal light source device onto a DMD (Digital Micromirror Device) which is a display element, and the DMD reflects the blue wavelength band light, green wavelength band light, and red wavelength band light for each micromirror (or each pixel) in a time-division manner. The image light is output to the outside as projection light P1 of the projection device 10 via a projection lens in the projection unit 13. The projection light P1 (image light) output from the projection unit 13 is projected onto the imaging medium 20 of FIG. 1.

[0014] The operation unit 14 receives an operation signal from an operation key or the like provided on the housing of the projection device 10, and transmits the operation signal to the processing unit 12 via the bus. The processing unit 12 executes various functions such as projection processing in response to the operation signal from the operation unit 14.

[0015] The communication unit 15 receives an operation signal such as an infrared modulated signal from a remote controller (not shown) and transmits the operation signal to the processing unit 12. The communication unit 15 is also connected to external devices such as a control device 60, an output unit 80, and a TOF lidar sensor 70 (Time of Flight LiDAR (light detection and ranging) Sensor).

[0016] The sound processor 16 includes a sound source circuit such as a PCM sound source, and diffuses and emits sound by driving the speaker 17. When the image data to be projected includes a sound signal, the sound processor 16 converts the sound signal into an analog signal during the projection operation and outputs the sound via the speaker 17.

[0017] Returning to FIG. 1, the imaging medium 20 has any shape and size that includes the projection range of the projection light P1. Alternatively, the imaging medium 20 is disposed at any position that includes the projection range of the projection light P1. The imaging medium 20 of this embodiment is configured as a flat plate-like or film-like transmissive screen. The imaging medium 20 is a transmissive member that, when the projection light P1 (including light L1) emitted from the projection device 10 is irradiated onto the first surface 21 side of the imaging medium 20 on the projection device 10 side, diffuses and emits spatial projection light P2 (including light L2) from the second surface 22 side, which is the opposite surface of the first surface 21, toward the light-guiding optical system 30.

[0018] The light guide optical system 30 is provided on the second surface 22 side of the imaging medium 20, and includes a beam splitter 31 and a retroreflecting member 32 (retroreflecting mirror). The retroreflecting member 32 is arranged so as to be perpendicular to the placement surface S of the imaging medium 20 (a surface including the Y direction (front-back direction) and the Z direction (up-down direction) in FIG. 1). The beam splitter 31 is formed in a flat plate shape and arranged at an angle of 45 degrees with respect to the placement surface S of the imaging medium 20 and the retroreflecting member 32. The beam splitter 31 of this embodiment is a half mirror that reflects a part of the incident light and transmits the other part. The retroreflecting member 32 has a mirror surface that reflects the incident light in the opposite direction to the incident direction.

[0019] The spatial imaging surface 40 is a spatial region where the projection image 2a projected onto the imaging medium 20 is emitted as spatial projection light P2 (P3) and then re-imaged by the light-guiding optical system 30 to display the spatial projection image 4a.

[0020] The control device 60 can be configured with a personal computer, a smartphone, a PDA, or the like, and can transmit image data (including moving images and still images) and audio data stored in an internal or external storage unit (not shown) to the projection device 10. The control device 60 may transmit audio data to a speaker 81 provided outside the projection device 10. As the external storage unit, for example, an external storage device connected via a LAN or WAN can be used. The control device 60 detects an input from the TOF lidar sensor 70 and controls the operation of the projection device 10 and the output unit 80.

[0021] The TOF lidar sensor 70 has a light-projecting unit that emits (projects) light, which is a laser beam, and a light-receiving unit that receives the light that is emitted by the light-projecting unit and reflected by a detectable object related to a user operation. The control device 60 measures the time required for the TOF lidar sensor 70 to emit pulsed light in the infrared wavelength band from the light-projecting unit and receive the light by the light-receiving unit (specifically, the phase difference between the emitted light and the received light), and calculates the distance from the TOF lidar sensor 70 to the detectable object. The light emitted by the TOF lidar sensor 70 can be light of any wavelength, such as ultraviolet light, visible light, or near-infrared light.

[0022] The TOF lidar sensor 70 of this embodiment is disposed on the same plane as the spatial imaging plane 40, on the side (left side as viewed from the viewer 50), and at a position inside the spatial imaging plane 40 in the vertical direction (Z direction) (see FIGS. 1 and 3). The TOF lidar sensor 70 emits light from the light projecting unit while rotating a predetermined scanning range 71 so as to reciprocate or oscillate in an angular range θ1 (see FIG. 1) around an axis in the Z direction and an angular range θ2 (see FIG. 3) around an axis in the Y direction, and receives the light reflected by the detected object with the light receiving unit. The control device 60 can detect the distance from the TOF lidar sensor 70 to the finger 51 by projecting and receiving the light. The control device 60 can also detect the spatial position of the finger 51 from the distance to the finger 51 and the direction (solid angle) scanned by the TOF lidar sensor 70.

[0023] The spatial projection device 100 is configured so that the scanning range 71 of the TOF lidar sensor 70 includes the detection area 41, which is a spatial area in the front-rear direction (Y direction) as viewed from the viewer 50 side of the spatial projection image 4a. FIG. 4 is a perspective view showing the positional relationship between the scanning range 71 and the detection area 41, and the scanning range 71 is set in an angular range θ1, θ2 around the origin O where the TOF lidar sensor 70 is disposed. The detection area 41 in this embodiment is a spatial area in the shape of a rectangular thick plate. In this embodiment, an example is shown in which the planar area (area on the ZX plane) of the detection area 41 as viewed from the viewer 50 side coincides with the display range of the spatial projection image 4a, but it may be set narrower or wider than the display range of the spatial projection image 4a. That is, the detection area 41 can be set to include a part or all of the spatial projection image 4a. Also, the range of the detection area 41 in the depth direction (Y direction) as seen from the viewer 50 side is shown as an example including the front and back areas of the spatial projection image 4a, but may be set only to the front area or the back area of ​​the spatial projection image 4a. In this way, the TOF lidar sensor 70 can detect a user operation in the depth direction (Y direction) on the spatial projection image 4a by irradiating a laser light having an amplitude in the out-of-plane direction (Y direction) of the spatial imaging plane 40. Also, the TOF lidar sensor 70 of this embodiment is configured to be able to detect the position of the detection object in the detection area 41 in the three-dimensional space by scanning the space while rotating around two axes. The spatial projection device 100 can execute any response process according to the position where the detection object such as the finger 51 is detected in the detection area 41. Therefore, the detection area 41 also functions as an operation area where an input operation can be performed on the spatial projection image 4a.

[0024] The speaker 81 (output unit 80) is provided as an external speaker of the projection device 10 or the control device 60. In this embodiment, the speaker 81 can play back audio by linking an audio signal output from the communication unit 15 in accordance with the projection control of the projection device 10 with the projection control. The speaker 81 may also play back audio data output directly from the control device 60.

[0025] Next, a spatial projection method and a method for detecting a detected object in the spatial projection device 100 (spatial projection system 1) will be described. Light L1 emitted from a point light source (any point on a micromirror in a DMD (display element)) in the projection device 10 (in the projection unit 13) through a projection lens is imaged at an image point F1 on the imaging medium 20. Light imaged in the same manner as the light L1 is irradiated onto the imaging medium 20 over the irradiation range of the projection light P1, and a projection image 2a is projected.

[0026] Light from any point constituting the projected image 2a formed on the imaging medium 20 is diffused and transmitted at a predetermined diffusion angle from the second surface 22 and emitted. For example, the above-mentioned light L1 imaged at the imaging point F1 is diffused as light L2 at a predetermined diffusion angle and enters the beam splitter 31. A part of the light L2 is reflected by the beam splitter 31 toward the retroreflecting member 32. That is, the light L2 emitted from the point light source of the projected image 2a is guided as diffused light in the optical path from the imaging medium 20 to the retroreflecting member 32. Since the retroreflecting member 32 reflects the incident light in the opposite direction to the incident direction, the light L2 incident on the retroreflecting member 32 is reflected toward the beam splitter 31 as condensed light that is condensed at the same angle as the diffusion angle. A part of the light L3 reflected by the retroreflecting member 32 is transmitted by the beam splitter 31 and guided to the spatial imaging surface 40 side. Then, the light L3 is re-imaged at an image point F2 on the spatial image plane 40. The optical path length of the light L2 and the optical path length of the light L3 are substantially the same.

[0027] Then, the light L3 imaged at the image point F2 on the spatial imaging plane 40 is guided as light L4 having a collection angle of the light L3 and a diffusion angle similar to the diffusion angle of the light L2.

[0028] The light beams L1 to L4 from the point light sources in the display element in the projection device 10 as described above are guided over the effective area of ​​the optical path of the imaging medium 20 and the light guiding optical system 30. That is, the projection light P1, which is a collection of the light beams L1 from the point light sources emitted from the projection device 10, is irradiated from the first surface 21 side of the imaging medium 20 and is imaged in the imaging medium 20. The projection light P1 irradiated to the imaging medium 20 is emitted from the second surface 22 to the beam splitter 31 side as the spatial projection light P2, which is a collection of the light beams L2. A part of the spatial projection light P2 irradiated to the beam splitter 31 is reflected to the retroreflecting member 32 side. The retroreflecting member 32 reflects the spatial projection light P2 guided from the beam splitter 31 side as the spatial projection light P3 (a collection of the light beams L3). A portion of the spatial projection light P3 reflected by the retroreflection member 32 passes through the beam splitter 31 and is guided to the spatial imaging plane 40 side.

[0029] In this way, the light (a collection of point light sources) constituting the projected image 2a formed on the imaging medium 20 is re-imaged on the spatial imaging plane 40, which is a spatial projection plane, and is emitted toward the viewer 50. Therefore, the viewer 50 can visually recognize the spatial projection image 4a formed in the air on the spatial imaging plane 40. Furthermore, the viewer 50 can visually recognize the spatial projection image 4a even if the observation point is moved. For example, the light L4 emitted from the imaging point F2 can be visually recognized at a position within the diffusion angle range (emission angle range) of the light L4 shown in FIG. 1.

[0030] Furthermore, the up-down direction (Z direction) and left-right direction (X direction) of the spatial projection image 4a as viewed from direction A, which is the direction from the viewer 50 side toward the spatial imaging surface 40 side, are approximately the same as the up-down direction (Z direction) and front-rear direction (Y direction) of the projection image 2a as viewed from direction B. On the other hand, since the optical path lengths of the light L2 and the light L3 are approximately the same, the depth position of the spatial projection image 4a as viewed from direction A is in an inverse relationship to the depth position of the projection image 2a as viewed from direction B. When the flat imaging medium 20 of FIG. 1 is used, the spatial projection image 4a is also displayed as a flat planar image.

[0031] The spatial projection device 100 detects an input operation by a touch operation of an operator such as a viewer 50, including an operated image, etc. in the spatial projection image 4a, and can perform control according to the input operation. For example, a button, a slide bar, an icon, etc. can be displayed as the operated image. Therefore, the spatial projection device 100 also functions as a detection device that detects an operation on the spatial projection image 4a. When the operator brings the finger 51 close to the operated image, etc. displayed in the detection area 41, the control device 60 can cause the TOF lidar sensor 70 to detect the position of the finger 51, and can cause the control device 60 to acquire which part of the spatial projection image 4a and to what depth the finger 51 has pointed to.

[0032] At this time, the control device 60 is configured to be able to determine the penetration depth (pressing depth) of the finger 51 (detection object) in the detection area 41 detected by the TOF lidar sensor 70 in stages with respect to the spatial imaging plane 40, and to output a response with an intensity according to the penetration depth. In the example of FIG. 1, the control device 60 controls the speaker 81 to gradually increase the volume of the operation sound from a small state, or to gradually decrease the volume of the operation sound from a large state, as the penetration depth of the finger 51 deepens. Alternatively, the control device 60 can control the operation sound output from the speaker 81 to increase in pitch or decrease in pitch as the penetration depth of the finger 51 deepens. Furthermore, the control device 60 may control the tone output from the speaker 81 to change as the penetration depth of the finger 51 deepens.

[0033] When the penetration depth of the finger 51 reaches a predetermined depth, the control device 60 determines that an input operation has been performed on the spatial projection image 4a, and performs any response process. The response process can be, for example, switching the display content or setting content of the spatial projection image 4a, or outputting sound from the speaker 81 (or the speaker 17).

[0034] In this way, the TOF lidar sensor 70 detects a detectable object such as a finger 51 in the detection area 41 from the side of the spatial imaging surface 40, and an operation sound according to the penetration depth of the detectable object is output from the output unit 80, so that the operator can easily perceive to what extent the input operation on the spatial projection image 4a is recognized. Therefore, it is possible to configure a spatial projection device 100 (detection device) with a good operability.

[0035] Next, modified examples 1 to 4 will be described which are other response examples when a detectable substance is detected in detection area 41. In the description of modified examples 1 to 4, the same components as those shown in Figs. 1 to 4 will be denoted by the same reference numerals, and the description thereof will be omitted or simplified.

[0036] (Variation 1) 5 is a schematic diagram seen from the planar side of the spatial projection device 100 in Modification 1. In the spatial projection device 100 in Modification 1, a blower 82 (fan) is used as the output section 80 instead of the speaker 81.

[0037] The control device 60 in the spatial projection device 100 of the first modification is configured to determine the penetration depth of the finger 51 (detection object) in the detection area 41 detected by the TOF lidar sensor 70 in stages, and to blow air (response output) by the blower 82 with strength according to the penetration depth. The control device 60 controls the blower 82 to gradually change the blowing amount from weak to strong as the penetration depth of the finger 51 deepens, or to gradually change the blowing amount from strong to weak as the penetration depth deepens. The wind from the blower 82 may be directional, aiming at the tip of the finger 51, or may be aimed at the entire finger 51 with a predetermined width. The position of the blower 82 can be set arbitrarily, and is not limited to a lateral position with respect to the spatial imaging surface 40 as shown in FIG. 5, and may be configured to blow air toward the finger 51 from the far side beyond the spatial imaging surface 40 as viewed from the operator (viewer 50) side. Furthermore, the control device 60 may be configured to blow air using the blower 82 when it is determined that the finger 51 is located on the back side (upper side in FIG. 1) of the position of the spatial projection image 4a. In this way, by using the blower 82, the operator can receive tactile feedback and perceive the operational feeling of the input operation.

[0038] (Variation 2) Fig. 6(a) is a schematic diagram showing a response output of the spatial projection device 100 in Modification 2. In Modification 2, the control device 60 in Fig. 1 gradually changes the display mode of the pointer image 4a1 on the spatial imaging plane 40 according to the penetration depth of the finger 51 (detectable object) in the detection area 41 (see Fig. 1) detected by the TOF lidar sensor 70. Therefore, a projection unit that projects the spatial projection image 4a is applied as the output unit 80 that performs the response output (the same applies to Modifications 3 and 4).

[0039] The pointer image 4a1 in FIG. 6(a) is formed in a circular shape. The control device 60 changes the size of the circle to be smaller so that the pointer image 4a1 converges to the position of the finger 51 as the finger 51 penetrates deeper into the detection area 41 (in FIG. 6(a), it coincides with approximately the center of the spatial projection image 4a). Note that other images such as polygons, arrows, icons, etc. may be used as the pointer image 4a1. Also, the pointer image 4a1 is not limited to a closed figure such as the circular pointer image 4a1, and a partially open figure (for example, a dashed line, a chain line) may be used as the pointer image 4a1. The operator can visually recognize that the pointer image 4a1 has changed in size and determine that the input operation has been recognized by the spatial projection device 100.

[0040] (Variation 3) Fig. 6(b) is a schematic diagram showing a response output of the spatial projection device 100 in Modification 3. In Modification 3 as well, the control device 60 in Fig. 1 gradually changes the state of the pointer image 4a2 on the spatial imaging plane 40 according to the penetration depth of the finger 51 in the detection area 41 (see Fig. 1) detected by the TOF lidar sensor 70.

[0041] The pointer image 4a2 in Fig. 6(b) is composed of two L-shaped figures arranged with their inner corners facing each other. The pointer images 4a2 are located at the upper right and lower left as viewed from the operator's side, and the control device 60 changes the display mode so that the two pointer images 4a2 approach and converge to the position of the finger 51 (approximately the center of the spatial projection image 4a in Fig. 6(b)) as the finger 51 penetrates deeper into the detection area 41. That is, the overall shape of the two pointer images 4a2 changes from two spaced apart L-shaped figures to a closed rectangular shape. The operator can visually recognize that the two pointer images 4a2 have closed and changed to a rectangular shape and determine that the input operation has been recognized by the spatial projection device 100.

[0042] (Variation 4) 7(a) and 7(b) are schematic diagrams showing response outputs of the spatial projection device 100 in Modification 4. In Modification 4 as well, the control device 60 in Fig. 1 gradually changes the state of the pointer image 4a3 on the spatial imaging plane 40 according to the penetration depth of the finger 51 in the detection area 41 (see Fig. 1) detected by the TOF lidar sensor 70.

[0043] The pointer image 4a3 in Fig. 7(a) is formed in a circular shape. The control device 60 changes the display mode of the pointer image 4a3 so that the color of the pointer image 4a3 changes as the finger 51 penetrates deeper into the detection area 41. For example, the pointer image 4a3 changes so that the color becomes darker (the change shown in Fig. 7(b)) or lighter (the opposite change to Fig. 7(b)) as the finger 51 penetrates deeper into the detection area 41. In the fourth modification, the operator can visually recognize the color of the pointer image 4a3 and determine that the spatial projection device 100 has recognized the input operation.

[0044] Although examples of response output according to Modifications 1 to 4 have been described above, these configurations may be combined with other configurations shown in FIG. 1. For example, the spatial projection device 100 may be configured to include a speaker 81 and a blower 82 as the output unit 80. Furthermore, the display method of the pointer images 4a1 to 4a3 according to Modifications 2 to 4 may be selected or combined depending on the scene, position, and the like. For example, the display mode of the pointer image may be configured to gradually change the size and display position like the pointer image 4a1 and the pointer image 4a2, and may also be configured to change the color like the pointer image 4a3.

[0045] As described above, the spatial projection device 100 of this embodiment forms the projection image 2a to be projected as the spatial projection image 4a by the projection device 10. Therefore, it is possible to display the projection image 2a with higher brightness than when a display image or an actual object is used as the projection target, and the spatial projection image 4a can also be displayed with high brightness and clarity. In addition, the projection image 2a projected by the projection device 10 can easily be enlarged by enlarging the shape of the imaging medium 20 and appropriately spacing the distance between the projection device 10 and the imaging medium 20 to enlarge the spatial projection image 4a.

[0046] Further, in place of the beam splitter 31 and the retroreflecting member 32, the spatial projection device 100 may be provided with a light reflecting member that re-images each point light source (for example, light from the imaging point F1) imaged on the imaging medium 20 on the spatial imaging plane 40 (for example, the imaging point F2 that is the plane symmetrical position of the imaging point F1) at a plane symmetrical position. For example, the light reflecting member may be configured to have a plurality of fine prisms or micromirrors, to reflect light incident at a predetermined diffusion angle as light that is focused at a focusing angle substantially the same as the diffusion angle, and to emit the light incident from the point light source so as to be focused at a plane symmetrical position. This allows the spatial projection device 100 in FIG. 1 to be configured more simply with fewer parts, thereby making it possible to reduce the overall size.

[0047] In addition, the spatial projection device 100 described in this embodiment may be configured by arranging the projection device 10, imaging medium 20, light-guiding optical system 30, spatial imaging surface 40, control device 60, TOF lidar sensor 70 and output unit 80 in a single device, or may be configured by distributing them across multiple devices.

[0048] Furthermore, the projection device 10 is not limited to the DLP type and may be of another type. A projection device of an LCP (liquid crystal panel) type may be used as the projection device 10. In an LCP type projection device, the transmittance of blue wavelength band light, green wavelength band light, and red wavelength band light is controlled for each pixel by a liquid crystal filter (liquid crystal plate), and the blue wavelength band light, green wavelength band light, and red wavelength band light of each pixel that have passed through the liquid crystal filter are synthesized and output as projection light.

[0049] Furthermore, an LCoS (Liquid Crystal on Silicon) type projection device may be used as the projection device 10. In an LCoS type projection device, instead of the DMD in the DLP type projection device 10, a display element having a liquid crystal filter (liquid crystal layer) capable of varying the light transmittance (including light blocking) corresponding to each pixel is disposed on a reflective layer. Therefore, the projection device forms image light by reflecting the light source light irradiated to the display element while controlling the amount of light for each pixel, and can project this image light to the outside as projection light to project the projection image 2a onto the imaging medium 20.

[0050] In addition, in an example in which an LCP-type or LCoS-type projection device is used as the projection device 10, each light (blue wavelength band light, green wavelength band light, red wavelength band light) transmitted through the liquid crystal filter is polarized light having a predetermined polarization direction. Therefore, for example, in the first embodiment, a polarizing mirror that reflects one of S-polarized light and P-polarized light and transmits the other may be arranged as the beam splitter 31, and a quarter-wave plate may be arranged on the optical path between the beam splitter 31 and the retroreflecting member 32. As a result, the beam splitter 31 reflects light of a first polarization direction, which is one of S-polarized light and P-polarized light, among the light emitted from the imaging medium 20 toward the retroreflecting member 32, and after transmitting through the quarter-wave plate and being converted into circularly polarized light, it is reflected by the retroreflecting member 32. The circularly polarized light reflected by the retroreflecting member 32 transmits through the quarter-wave plate again and is converted into light of a second polarization direction perpendicular to the first polarization direction, and transmits through the beam splitter 31. In this way, when a polarizing mirror is used as the beam splitter 31, most of the light reflected by the beam splitter 31 can be transmitted through the beam splitter 31 after being reflected by the retroreflective member 32, thereby improving the light utilization efficiency compared to when a half mirror is used.

[0051] A laser scanning projection device may be used as the projection device 10. In the laser scanning projection device, a laser beam synthesized by combining blue, green and red wavelength band light into a desired color is irradiated onto a display element, and the display element reflects the laser beam while controlling the reflection angle in a time division manner, and irradiates the laser beam onto an image-forming medium 20, which is a projection target. At this time, the display element irradiates the laser beam so as to perform two-dimensional scanning in the vertical and horizontal directions onto the image-forming medium 20, and can project a projection image 2a onto the image-forming medium 20. In the laser scanning projection device, a projection lens for concentrating the light emitted from the display element can be omitted, so that the entire projection device can be made smaller. In addition, since this projection device can form an image using laser beams, it can project a clear intended projection image 2a even when a three-dimensional image-forming medium with large unevenness is used.

[0052] In addition, in this embodiment, a configuration has been described in which the TOF lidar sensor 70 can scan space while rotating around two axes to detect the position in three-dimensional space of the object to be detected in the detection area 41. However, the TOF lidar sensor 70 may also be configured to detect the object to be detected in the detection area 41 by scanning a specified section using any driving mechanism.

[0053] In addition, in this embodiment, a configuration has been described in which the control device 60 controls the operation of the TOF lidar sensor 70, the output unit 80, and the projection device 10, but a configuration in which a device having a projection unit 13 (corresponding to the projection device 10 in this embodiment) controls the TOF lidar sensor 70 and the output unit 80 may also be used.

[0054] In addition, instead of the flat imaging medium 20 shown in FIG. 1, etc., a three-dimensional imaging medium including a curved surface, unevenness, etc. may be used. As described above in FIG. 1, the optical path lengths of the light L2 and the light L3 guided by the light guide optical system 30 are approximately the same, and the depth position of the spatial projection image 4a seen from the A direction is inversely related to the depth position of the projection image 2a seen from the B direction. That is, as shown in FIG. 3, the imaging point located at the back side of the curved projection image 2a seen from the B direction corresponds to the imaging point located at the front side of the spatial projection image 4a seen from the A direction. Therefore, by forming the imaging medium into a three-dimensional surface, the three-dimensional spatial projection image 4a can be projected onto the spatial imaging surface 40.

[0055] Furthermore, although a plate-like or film-like configuration has been exemplified as the imaging medium 20, a fluid such as smoke or water may also be used.

[0056] The imaging medium 20 may be colored. This allows the color of the spatial projection image 4a to be changed or the color tone to be adjusted as desired. When a fluid is used as the imaging medium 20, the fluid may be colored. The color of the imaging medium 20 may be changed over time. This allows various dramatic effects to be expressed.

[0057] Furthermore, the light guide optical system 30 may be configured to re-image the light emitted from the imaging medium 20 onto the spatial imaging surface 40 using a Fresnel lens.

[0058] In addition, in this embodiment, a transmissive imaging medium has been described as an example in which the projection light P1 projected onto the imaging medium 20 enters from the first surface 21 side of the imaging medium 20 and exits from the opposite second surface 22 side of the first surface 21, and is guided to the light-guiding optical system 30 as spatial projection light P2. However, a reflective imaging medium (a projection medium such as a projector screen or a wall surface) in which the surface from which the projection light P1 exits and the surface from which the spatial projection light P2 exits are the same surface may also be used.

[0059] Furthermore, the projection image 2a formed on the imaging medium 20 by the projection light P1 is not limited to an image formed by the projection light emitted from the projection device 10, but may be an optical image formed by any light source light, illumination light, LED light, or laser light emitted as projection light from other light source devices. Furthermore, the projection image 2a may be formed by visible light using any light source, or may be formed by invisible light (for example, for applications such as electronic watermarking).

[0060] In addition, in this embodiment, a configuration using a TOF lidar sensor 70 as a sensor for detecting user operations has been shown, but it is not limited to the TOF lidar sensor 70. However, other sensors can be used that are arranged on the same plane as the spatial imaging surface 40 and detect user operations in the depth direction (Y direction) relative to the image by irradiating laser light having an amplitude in the out-of-plane direction (Y direction) of the spatial imaging surface 40.

[0061] The detection device (spatial projection device 100) described in this embodiment includes a light-guiding optical system 30 that projects an image into space by forming an image of projection light on a spatial imaging plane 40, and a sensor (70) that is provided on the same plane as the spatial imaging plane 40 and detects a user operation in the depth direction relative to the image by irradiating a laser light having an amplitude in a direction outside the plane of the spatial imaging plane 40. This allows the entire device including the sensor to be constructed inexpensively and simply, and makes it easy to detect spatial operations by the user.

[0062] In addition, a detection device that includes an imaging medium 20 on which the projection light P1 is imaged and in which a light-guiding optical system 30 guides the projection light P1 imaged on the imaging medium 20 as spatial projection light P2, P3 and re-images it on a spatial imaging surface 40 can display a high-brightness spatial projection image 4a.

[0063] In addition, a detection device in which the sensor (70) is configured to rotate around two axes while scanning and detect the position of a detectable object in three-dimensional space related to user operation can drive the sensor (70) with a small number of control axes and perform spatial scanning with a simple configuration.

[0064] Furthermore, a detection device in which the sensor (70) is a TOF lidar sensor 70 can be configured to have a small overall size.

[0065] Also, a configuration has been described that includes an output unit 80 capable of outputting a response with an intensity according to the penetration depth of the object to be detected with respect to the user operation detected by the sensor (70) with respect to the spatial imaging plane 40. This allows the operator to perceive the feeling of operation until the input operation on the spatial projection image 4a is recognized by the detection device based on the intensity of the response output.

[0066] Also, a configuration has been described in which the output unit 80 is a speaker or a blower, which allows the operator to perceive the operational feeling of the input operation by hearing or touching.

[0067] Also, a configuration has been described in which the display mode of the pointer images 4a1-4a3 on the spatial imaging plane 40 is changed stepwise in accordance with the penetration depth of the object related to the user operation detected by the sensor (70) into the spatial imaging plane 40. This allows the operator to visually recognize the change in the pointer images 4a1-4a3 and perceive the feeling of operation until the input operation on the spatial projection image 4a is recognized by the detection device.

[0068] In addition, by configuring the pointer images 4a1, 4a2 to change shape so as to converge to the position of the detectable object as the detectable object penetrates deeper, the operator can visually recognize the change in the pointer images 4a1, 4a2 and easily determine that the input operation has been recognized by the detection device.

[0069] In addition, by configuring the pointer image 4a3 to change color as the detectable object penetrates deeper, the operator can visually recognize the change in the pointer image 4a3 and easily determine that the input operation has been recognized by the detection device.

[0070] The above-described embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0071] The invention described in the first claim of this application is set forth below. [1] A light-guiding optical system that projects an image into space by focusing projection light on a spatial imaging plane; a sensor that is provided on the same plane as the spatial imaging plane and detects a user operation in a depth direction relative to the image by irradiating a laser beam having a swing width in a direction outside the plane of the spatial imaging plane; A detection device comprising: [2] An imaging medium on which the projection light is imaged, The light guiding optical system guides the projection light imaged by the imaging medium as spatial projection light and re-images the projection light on the spatial imaging surface. The detection device according to [1] above. [3] The detection device described in [1] or [2], characterized in that the sensor is configured to scan while rotating around two axes to detect the position of the object to be detected in three-dimensional space related to the user operation. [4] The detection device described in any one of [1] to [3], characterized in that the sensor is a TOF lidar sensor. [5] A detection device as described in any of [1] to [4], characterized in that it is provided with an output unit capable of responding with an intensity corresponding to the penetration depth of the detected object relative to the spatial imaging plane regarding the user operation detected by the sensor. [6] The detection device according to [5], wherein the output unit is a speaker or a blower. [7] A detection device as described in any of [1] to [4], characterized in that the display mode of a pointer image on the spatial imaging surface is gradually changed depending on the penetration depth of a detected object related to the user operation detected by the sensor into the spatial imaging surface. [8] The detection device described in [7], wherein the pointer image changes shape so as to converge to the position of the object to be detected as the object to be detected penetrates deeper. [9] The detection device according to [7] or [8], wherein the pointer image changes color as the object to be detected penetrates deeper.

[10] The image is projected into space by focusing the projection light onto a spatial imaging surface using a light guide optical system; a sensor provided on the same plane as the spatial imaging plane is used to irradiate the image with a laser beam having a swing width in an out-of-plane direction of the spatial imaging plane, thereby detecting a user operation in a depth direction with respect to the image; A detection method comprising:

[11] A light-guiding optical system that projects an image into space by focusing the projection light on a spatial imaging plane; a sensor that is provided on the same plane as the spatial imaging plane and detects a user operation in a depth direction relative to the image by irradiating a laser beam having a swing width in a direction outside the plane of the spatial imaging plane; A spatial projection device comprising: [Explanation of symbols]

[0072] 1 Spatial projection system 2a Projected image 4a Spatial projection image 4a1~4a3 Pointer image 10 Projection device 11 Storage unit 12 Processing section 13 Projection section 14 Operation unit 15 Communication unit 16 Audio processing unit 17 Speaker 20 Imaging medium 21 First surface 22 Second surface 30 Light guide optical system 31 Beam splitter 32 Retroreflective member 40 spatial image plane 41 detection area 50 viewers 51 fingers 60 Control device 70 TOF lidar sensor 71 Scanning range 80 Output section 81 Speaker 82 Blower 100 Space projection device F1,F2 image points L1~L4 light P1 Projection light P2,P3 Space projection light S Arrangement plane θ1,θ2 angle range

Claims

1. a light guide optical system that projects an image into space by focusing the projection light on a spatial imaging surface; a sensor that is provided on the same plane as the spatial imaging plane, at a side of the spatial imaging plane and at a substantially center of the spatial imaging plane in the up-down direction, and detects a user operation in the depth direction relative to the image by irradiating a laser beam having a swing width in a direction outside the spatial imaging plane; A detection device comprising:

2. an imaging medium onto which the projection light is imaged; the light guiding optical system guides the projection light formed by the imaging medium as a spatial projection light; re-imaging the image to an image point on the spatial image plane; 2. The detection device according to claim 1 .

3. The sensor is provided on the opposite side of the imaging point on the spatial imaging plane across the center of the spatial imaging plane, 3. The detection device according to claim 2.

4. 4. The detection device according to claim 1, wherein the sensor is configured to be capable of detecting a position in a three-dimensional space of a detection object related to the user's operation by scanning while rotating around two axes.

5. 5. The detection device according to claim 1, wherein the sensor is a Time-of-Flight Lidar sensor.

6. 6. The detection device according to claim 1, further comprising an output unit capable of outputting a response with an intensity according to a penetration depth of a detection object with respect to the spatial imaging plane, the detection object being detected by the sensor and related to the user operation.

7. The detection device according to claim 6 , wherein the output unit is a speaker or a blower.

8. The detection device according to any one of claims 1 to 5, characterized in that the display mode of a pointer image on the spatial imaging surface is gradually changed depending on the penetration depth of a detectable object related to the user operation detected by the sensor into the spatial imaging surface.

9. The detection device according to claim 8 , wherein the shape of the pointer image changes so as to converge to a position of the object to be detected as the object to be detected penetrates deeper.

10. 10. The detection device according to claim 8, wherein the pointer image changes color as the object to be detected penetrates deeper.

11. A spatial projection device comprising a detection device as described in any one of claims 1 to 10.

Images