Aerial display system and aerial imaging method

Abstract

To enable an observer to easily and clearly visually recognize a real image even in an environment where the observer has difficulty intuitively grasping the imaging position of the real image. [Solution] The aerial display system 100 comprises a display device 11 that displays an image, an aerial imaging member 12 that refracts the image light incident from the display device 11 to form an image in the air, and a wall 20 in which an opening 21 is formed within a range of 300 mm in front and behind the imaging plane P of the aerial imaging member 12, and is configured so that at least a portion of the image light passes through the opening 21 in the wall 20.

Description

[Technical field]

[0001] The present invention relates to a system and method for forming a real image of an image displayed by a display device in mid-air. [Background technology]

[0002] Conventionally, optical elements that form a real image of an image displayed by a display device in the air have been known. As an example of such an aerial imaging optical element, for example, Patent Document 1 discloses an optical element using a two-surface orthogonal reflector system. In this system, two orthogonally arranged reflecting surface arrays are used to sequentially reflect incident image light in directions orthogonal to each other, thereby forming a real image of the image in the air. It is said that by using such an aerial imaging optical element, it is possible to obtain a bright and clear real image with high light utilization efficiency. [Prior art documents] [Non-patent literature]

[0003] [Patent Document 1] Patent No. 5856357 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, the aerial imaging optical element of Patent Document 1 is assumed to be used in a non-contact input device, and is configured to form a real image of an icon image in the space near the user's hand and detect that the user has pointed to that real image with a finger, a touch pen, etc., thereby allowing a signal to be input without physically touching the display device. In this case, the user only needs to look at a relatively close range near the hand, making it easy for the user to focus their eyes on the imaging surface of the image, and as a result, the image is easily visible to the user.

[0005] On the other hand, when considering the use of such aerial imaging optical elements for relatively large video works or signage systems installed outdoors or in stores, it is necessary to form a real image of the video in a relatively large exhibition space. In this case, in order to allow the observer to clearly see the real image formed in the space, the observer is forced to walk close to the imaging surface in the large exhibition space and focus his / her eyes on the imaging surface. In other words, the real image will appear unclear to an observer standing at a position away from the imaging surface of the real image, and even if the observer stands near the imaging surface of the real image, the real image will also appear unclear if the imaging surface of the real image and the focus of the observer's eyes do not match. In this way, for example, when a real image of a video is formed in a relatively large exhibition space, in a situation where it is difficult for the observer to intuitively grasp the imaging position of the real image, there is a concern that the aesthetic effect of the video work and the advertising effect of the signage system will decrease.

[0006] Therefore, a main object of the present invention is to enable an observer to easily and clearly view a real image even in an environment in which it is difficult for the observer to intuitively grasp the imaging position of the real image. [Means for solving the problem]

[0007] Therefore, the inventor of the present invention has earnestly studied means for solving the problems of the above-mentioned conventional technology, and has found that by providing a physical wall near the imaging plane of a real image formed by an aerial imaging member and forming an opening in the wall near the imaging plane, the wall and the imaging plane are both within the viewer's field of vision at the same time, making it easier for the observer to intuitively grasp the imaging position of the real image. Based on this finding, the inventor has come to the realization that the problems of the conventional technology can be solved, and has completed the present invention. Specifically, the present invention has the following configuration or steps.

[0008] The first aspect of the present invention relates to an aerial display system. The aerial display system includes a display device, an aerial imaging member, and a wall. The display device displays an image. In this specification, the term "image" includes both or either of a moving image and a still image. The aerial imaging member refracts the image light incident from the display device to form an image in the air. As the aerial imaging member, a known type of optical element that can form a real image in the air can be used. As the aerial imaging member, in addition to the above-mentioned two-surface orthogonal reflector type (Patent Document 1), for example, a retroreflection type combining a half mirror and a retroreflection sheet, a concave mirror type combining a concave mirror and a half mirror, or a lens array type using a microlens array can be used. The wall has an opening formed within a range of 300 mm in front and behind the imaging surface of the aerial imaging member. This wall is configured to divide the exhibition space into the observer side and the display side. The range of 300 mm in front and behind the imaging surface is set in consideration of the human clear vision distance (the distance at which the human eye can easily focus). The range of 300 mm in front of and behind the imaging plane refers to a range of 600 mm in total length from a position 300 mm behind the imaging plane to a position 300 mm in front of the imaging plane, with respect to the optical axis of the image light perpendicular to the imaging plane. The aerial display system is configured so that at least a part of the image light emitted from the display device passes through the opening in the wall. The shape of the opening is not limited to a square shape, but can also be a circle, an ellipse, a triangle, a pentagon, or a polygon with more sides.

[0009] As in the above configuration, by providing a wall with an opening within a range of 300 mm in front of and behind the imaging plane of the aerial imaging component, the observer's eyes naturally focus on the wall or the opening. Then, because the observer looks at the aerial image through the opening while focusing on the opening, it becomes easier for the observer to intuitively grasp the position of the imaging plane, and as a result, the aerial image can be clearly seen. In addition, because the wall functions as a standard for dividing the exhibition space, the presence of the wall allows the observer to more accurately grasp the depth of the space, which also improves the visibility of the aerial image.

[0010] In the aerial display system according to the present invention, it is preferable that the wall has an upper wall portion and a lower wall portion, and an opening is formed between the upper wall portion and the lower wall portion. By forming an opening between the upper wall portion and the lower wall portion in this way, the aerial image appears to float between the upper and lower walls (the opening) from the observer, and the effect of improving the visibility of the aerial image caused by the presence of the wall can be more effectively demonstrated. In addition, the human visual field is narrower in the vertical direction (about 120 degrees) than in the horizontal direction (about 200 degrees), and the observer needs more clues for spatial recognition in the vertical direction. Therefore, by providing walls in the vertical direction, the observer can more accurately grasp the expanse of the space, and the presence of the aerial image located in that space can be further emphasized.

[0011] The aerial display system according to the present invention further includes a projection device. The projection device projects an image onto a portion of the wall other than the opening. The projection device may be arranged to project an image onto the front side of the wall (i.e., the observer side), or may be arranged to project an image onto the back side of the wall (i.e., the display device side). By projecting an image onto a portion of the wall other than the opening in this manner, an image expression that integrally connects the aerial image and the wall projected image becomes possible. For example, by projecting an image onto the wall so as to be continuous with the aerial image seen through the opening, a visual effect can be produced in which a part of the image appears to be floating out of the wall. In addition, by projecting an image related to the periphery of the aerial image seen through the opening, a visual effect can be produced in which the aerial image appears to be floating out of the wall. Furthermore, by linking the movement of the image between the aerial image and the wall projected image, a dynamic performance can be realized in which a part of the image appears to be floating out of the wall in a three-dimensional manner. In this way, by adding a projection device, it is possible to create a continuous image expression without making the boundary between the aerial image and the image projected onto the wall noticeable, greatly expanding the range of effects that can be produced by the aerial display system and making more effective use of the presence of walls.

[0012] In the aerial display system according to the present invention, the projection device is preferably arranged on the same side as the display device and the aerial imaging member, i.e., on the back side of the wall. In this case, at least a part of the wall where the image light is projected from the projection device is configured to at least partially transmit the image light. For example, the part of the wall where the image light is projected from the projection device may be transparent, semi-transparent, or mesh-like. In this way, by arranging the projection device on the same side as the display device and the aerial imaging member, the entire system can be installed integrally on the back side of the wall. This makes it possible to provide the observer with only a space for enjoying the image, and the devices do not enter the observer's eyes. In addition, by arranging the projection device on the back side of the wall and configuring a part of the wall to be transparent to the image light, the image projected from the projection device is displayed softly on the surface of the wall, and an integrated image expression without a sense of incongruity with the aerial image can be realized. Furthermore, by adjusting the transmittance of the wall, the balance between the aerial image and the wall-projected image can be optimized, and a more natural continuity of the image can be realized.

[0013] In the aerial display system according to the present invention, it is preferable that at least a part of the periphery of the opening of the wall is formed at an acute angle. By forming the periphery of the opening of the wall at an acute angle in this way, the boundary of the opening can be made less noticeable. In other words, compared to when the periphery of the opening is cut at a right angle with a thickness, the presence of the wall is reduced with an acute-angled periphery, and the boundary between the aerial image and the surrounding space becomes more natural. In particular, when an image is projected on the periphery of the opening, the image gradually becomes darker in the acute-angled part, so that it is possible to suppress the appearance of a step at the boundary between the aerial image and the wall surface. This further improves the visual continuity between the space in which the aerial image is floating and the wall surface, thereby enhancing the effect of integrated image expression.

[0014] The aerial display system according to the present invention preferably further comprises a detection device and a control device. The detection device detects the presence or absence or position of an object within a predetermined range in front of and behind the imaging surface of the aerial imaging member. Examples of the detection device are an infrared sensor, a three-dimensional sensor, a depth sensor, a stereo camera, a ToF camera, and an ultrasonic sensor. The control device controls the image displayed by the display device according to the detection result of the detection device. By providing the detection device and the control device in this way, the image can be changed interactively according to the observer's movement. For example, when the observer reaches out to the aerial image, the aerial image changes according to the movement, and the aerial image and the wall projection image change in conjunction with each other. Note that the predetermined range in front of and behind the imaging surface may be selected from a range within 500 mm in front of and behind the imaging surface. This detection range corresponds to the movement range in which a person naturally reaches out to touch the aerial image, so the observer can interact with the aerial image with an intuitive movement without difficulty. This allows the natural movement of the observer to be detected and the image to be controlled, thereby realizing a more attractive and interactive image expression.

[0015] In the aerial display system according to the present invention, the aerial imaging member is preferably a two-surface orthogonal reflector type. The aerial imaging member of this type includes a first group of reflecting surfaces and a second group of reflecting surfaces having a reflecting surface in a direction substantially orthogonal to the first group of reflecting surfaces. The angle substantially orthogonal is in the range of 85 to 95 degrees. The aerial imaging member is configured to form an aerial image by sequentially reflecting the image light incident from the display device by the first group of reflecting surfaces and the second group of reflecting surfaces. By adopting the two-surface orthogonal reflector type in this way, a bright and clear aerial image can be formed with high light utilization efficiency. In addition, this type can be realized with a compact configuration compared to other types (for example, a retroreflection type or a concave mirror type), so that the flexibility of installation of the entire system can be increased.

[0016] A second aspect of the present invention relates to an aerial imaging method. In the aerial imaging method according to the present invention, a display device displays an image (display step). Also, an aerial imaging member refracts image light incident from the display device to form an image in the air (imaging step). Here, a wall having an opening is provided within a range of 300 mm in front of and behind the imaging surface of the aerial imaging member, and at least a part of the image light passes through the opening in the wall. Effect of the Invention

[0017] According to the present invention, even in an environment where it is difficult for a viewer to intuitively grasp the imaging position of a real image, such as a relatively large exhibition space, the viewer can easily and clearly view the real image. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a side view showing a schematic diagram of an overall configuration of an aerial display system according to one embodiment of the present invention. [Diagram 2] This is a front view of the aerial display system shown in Figure 1, and shows a schematic diagram of image projection onto a wall surface and aerial image formation. [Diagram 3] A block diagram showing the functional configuration of a control system of an aerial display system according to one embodiment of the present invention. [Figure 4] 1A and 1B are diagrams showing how video content is divided, where (a) shows the entire video content, (b) shows an image output from a projection device, and (c) shows an image output from a display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes appropriate modifications of the embodiments described below within the scope obvious to those skilled in the art.

[0020] FIG. 1 shows the overall configuration of an aerial display system 100 according to an embodiment of the present invention. FIG. 2 shows a schematic front view of the aerial display system 100 according to this embodiment. Furthermore, FIG. 3 shows mainly the functional configuration of the control system of the aerial display system 100 according to this embodiment. As shown in FIG. 1, the aerial display system 100 according to this embodiment mainly includes an aerial display device 10, a wall 20, a projection device 30, and a detection device 40. Furthermore, as shown in FIG. 3, the aerial display system 100 further includes a control device 50, which controls the aerial display device 10, the projection device 30, and the detection device 40.

[0021] For example, as shown in FIG. 1 and FIG. 2, the aerial display system 100 according to the present embodiment is assumed to be implemented in an exhibition space in which the height from the floor to the ceiling exceeds the height of a general adult (1400 to 2000 mm). In such an exhibition space, the aerial display system 100 forms a real image of a video in the air in front of the line of sight of an observer in a standing posture, and allows the observer to view the real image formed in the air in this way. In addition, in this system, an image is projected onto a wall 20, and the observer views the projected image together with the real image formed in the air. The height from the floor to the ceiling is not particularly limited, but is preferably 2000 mm or more, and particularly preferably 2500 to 4000 mm. In addition, the width of the exhibition space, particularly the width of the wall 20, is preferably 1000 mm or more, and is preferably 1500 to 4500 mm. The aerial display system 100 according to the present embodiment can be suitably used for the exhibition of video works and advertisement displays in exhibition facilities such as museums and showrooms, or commercial facilities. From the observer's perspective, the image projected onto the wall 20 and the aerial image viewed through the opening 21 appear as one, so that the entire image can be expressed three-dimensionally with a sense of depth. Also, the observer can enjoy the image while moving freely in front of the wall 20, and multiple observers can observe at the same time. Note that the appropriate observation position for the observer is preferably within a range of 500 to 3000 mm in front of the wall 20.

[0022] The aerial display device 10 is a device for forming a real image of a video in the air within an exhibition space. The aerial display device 10 mainly comprises a display device 11, an aerial imaging member 12, and a support 13. The display device 11 outputs image light by displaying an image, and the aerial imaging member 12 forms a real image in the air using the image light. The support 13 holds the aerial imaging member 12 in a predetermined position and attitude. These components are arranged together on the back side of a wall 20, which will be described later.

[0023] The display device 11 may be a general display such as a liquid crystal display or an organic EL display. In this embodiment, the display device 11 is arranged along the floor surface and outputs image light upward (toward the ceiling). The arrangement of the display device 11 is not limited to this, and it is also possible to attach the display device 11 to the ceiling surface, or to lean it against a side wall by bending the optical path of the image light using an optical element such as a mirror. As described above, in this embodiment, it is assumed that an image is formed in a relatively large exhibition space, so it is preferable to adopt a large display for the display device 11. Since the size of the aerial image depends on the display size of the display device 11, for example, the diagonal length of the display screen of the display device 11 is preferably 40 inches or more, and particularly preferably 50 to 80 inches. In addition, the display device 11 is not limited to one display, and may be configured by connecting and installing multiple displays.

[0024] The aerial imaging member 12 is an optical member that refracts the image light incident from the display device 11 to form a real image in the air. As the aerial imaging member 12, any known type of optical element that can form a real image in the air can be used, such as a two-surface orthogonal reflector type, a retroreflection type, a concave mirror type, or a lens array type. The two-surface orthogonal reflector type includes a first reflecting surface group and a second reflecting surface group having a reflecting surface in a direction substantially orthogonal to the first reflecting surface group (substantially orthogonal means a range of 85 to 95 degrees). In this type, the image light incident from the display device 11 is sequentially reflected by the first reflecting surface group and the second reflecting surface group to form a real image in the air. Such a two-surface orthogonal reflector type has the advantage of being able to form a bright and clear real image with high light utilization efficiency. In addition, the retroreflection method has a configuration in which a half mirror and a retroreflection sheet are combined, and the image light reflected by the half mirror is returned to the incident direction by the retroreflection sheet to form a real image. This method has the advantage of being simple in structure and easy to manufacture. The concave mirror method has a configuration in which a concave mirror and a half mirror are combined, and the image light collected by the concave mirror forms a real image through the half mirror. This method has the advantage of being able to form a large real image. The lens array method has a configuration in which multiple microlenses are regularly arranged, and the image light is collected by each microlens to form a real image. This method has the advantage of being easy to make thin. In particular, the two-surface orthogonal reflector method has the advantage of being able to form a bright and clear real image with high light utilization efficiency, and can be realized with a compact configuration compared to other methods, so this method is preferably adopted in this embodiment.

[0025] Moreover, the aerial imaging member 12 is preferably a thin plate, and the thickness thereof can be, for example, about 10 to 50 mm. Moreover, the planar size of the aerial imaging member 12 is preferably equal to or larger than the display screen size of the display device 11. Specifically, it is determined based on the display screen size of the display device 11, the distance between the display device 11 and the aerial imaging member 12, and the inclination angle of the aerial imaging member 12. For example, if the display screen of the display device 11 is 50 inches, the aerial imaging member 12 preferably has a diagonal length of 60 inches or more. Moreover, in this embodiment, the aerial imaging member 12 is installed at an angle of about 45 degrees with respect to the floor surface by the support 13. This inclination angle is set so that the image light emitted from the display device 11 is imaged at an appropriate position, and can be appropriately adjusted within a range of 30 to 50 degrees. Note that, as shown in FIG. 1, the inclination angle of the aerial imaging member 12 is gradually inclined backward (toward the opposite side to the observer) as it rises from the floor surface.

[0026] As shown in FIG. 1, the image light output from the display device 11 is incident on the aerial imaging member 12 while being diffused. The aerial imaging member 12 refracts the image light incident from the display device 11 at a predetermined angle to form a real image in the air. Here, the incident angle and the outgoing angle of the image light with respect to the aerial imaging member 12 are equal. In the example shown in FIG. 1, the aerial imaging member 12 is arranged so that the refraction angle of the image light vertically emitted from the display device 11 is 90 degrees. That is, the image light traveling vertically from such a display device 11 is incident on the aerial imaging member 12 at an angle of 45 degrees and is outgoing at an angle of 45 degrees equal to the incident angle. In this way, all the image light emitted from the display device 11 while being diffused is refracted by the aerial imaging member 12 and then converges on the same plane to form a real image. The plane on which the image points of each image light are arranged is referred to as an "imaging plane" in this specification, and is indicated by a dotted line with a symbol P in FIG. 1. In addition, the distance from the position where the image light is incident on the aerial imaging member 12 to the display surface of the display device 11 (the incident optical path length L i ) and the distance from the position where the image light is emitted from the aerial imaging member 12 to the imaging surface P (the emission optical path length L o ) is equal. Thus, the incident optical path length Li and the output optical path length L o Since these are equal to each other, a real image having the same size as the display surface of display device 11 is formed on imaging plane P. The real image thus formed is visually recognized by the observer as if the display surface of display device 11 were located at the position of imaging plane P.

[0027] The aerial imaging member 12 is preferably made of a material with high light transmittance, and for example, acrylic resin or glass can be used. Acrylic resin is particularly suitable because it is easy to process and has sufficient light transmittance (90% or more). In the case of the aerial imaging member 12 of a two-surface orthogonal reflector type, the group of reflective surfaces included in the member can be formed by evaporating a thin metal film such as aluminum. The group of reflective surfaces needs to be formed with high precision so that the surfaces are orthogonal to each other (within an error of ±5 degrees).

[0028] The support 13 is a member for holding the aerial imaging member 12 at a predetermined position and attitude. The support 13 may have various adjustment mechanisms so that the relative position with respect to the aerial imaging member 12 can be accurately adjusted. For example, the support 13 can be provided with a height adjustment mechanism for adjusting the height of the aerial imaging member 12, a position adjustment mechanism for adjusting the front-to-back position, and an angle adjustment mechanism for adjusting the tilt angle. These adjustment mechanisms make it possible to finely adjust the position and attitude of the aerial imaging member 12 to an optimal state, for example, by using an alignment marker displayed on the display device 11. The support 13 has a fixing portion that can be fixed to the floor or side wall of the exhibition space, and is structured to stably hold the aerial imaging member 12.

[0029] The wall 20 is constructed to separate the exhibition space into an installation space of the aerial display device 10 and a space where observers can enter. In this embodiment, the wall 20 plays a role in making it easier for observers to view the real image formed by the aerial display device 10. In addition, the wall 20 also plays a role as a screen onto which an image is projected by the projection device 30. In this way, by combining the aerial image formed by the aerial display device 10 and the wall-projected image by the projection device 30, various image presentations can be realized.

[0030] As shown in FIG. 1 and FIG. 2, in this embodiment, the wall 20 is mainly composed of an upper wall portion 22 and a lower wall portion 23, and an opening 21 is formed between them. The image light output by the display device 11 of the aerial display device 10 is refracted by the aerial imaging member 12 and reaches the observer through the opening 21 of the wall 20. At this time, the arrangement of the aerial display device 10 and the wall 20 is adjusted so that the imaging plane P of the aerial imaging member 12 is formed near the opening 21 of the wall 20. Specifically, as shown in FIG. 1, the upper wall portion 22 and the lower wall portion 23 are configured to be aligned in a straight line perpendicular to the floor surface, and it is most preferable that the imaging plane P of the aerial imaging member 12 is formed to coincide with the straight line connecting the upper wall portion 22 and the lower wall portion 23. In other words, it is preferable that the imaging plane P is formed on an extension line of the upper wall portion 22 and the lower wall portion 23 within the opening 21 formed between the upper wall portion 22 and the lower wall portion 23. Furthermore, since the upper wall portion 22 and the lower wall portion 23 have thickness, the range of thicknesses of these upper and lower walls may be regarded as the thickness of the opening 21, and the design may be such that the imaging plane P is formed within the thickness range of the opening 21. In this way, by forming the imaging plane P within the opening 21 between the upper wall portion 22 and the lower wall portion 23, the distance to the wall 20 and the distance to the imaging plane P match as seen by the observer. In addition, the wall 20 and the imaging plane P are both included in the observer's field of vision at the same time. This makes it easier for the observer to intuitively grasp the imaging point P by relying on the wall 20, and therefore makes it easier for the observer to clearly view the real image of the image appearing on the imaging plane P.

[0031] As described above, it is most preferable that the imaging plane P is formed within the opening 21 of the wall 20 (on the extension of the upper wall portion 22 and the lower wall portion 23). However, since it is difficult to precisely adjust the position of the imaging plane P, a distance may exist between the imaging plane P and the opening 21 of the wall 20. Specifically, it is sufficient that the opening 21 (the extension of the upper wall portion 22 and the lower wall portion 23) is located within a range of 300 mm in front and behind the imaging plane P, based on the optical axis of the image light that advances perpendicularly to the imaging plane P. In other words, the range of 300 mm in front and behind the imaging plane P means a range of 600 mm in total length from a position 300 mm behind the imaging plane to a position 300 mm in front of the imaging plane P, relative to the optical axis of the image light that is perpendicular to the imaging plane P. In this way, it is also acceptable for the imaging plane P to be shifted within a range of 300 mm in front and behind the opening 21. This numerical range is determined in consideration of the visual characteristics of humans. That is, there is a distance called a clear vision distance for human eyes, at which the eyes are least tired and the object can be most easily focused. This clear vision distance is generally known to be about 250 mm to 300 mm, and in this distance range, the burden of focus adjustment is small and stereoscopic vision by binocular parallax also functions effectively. In addition, in this distance range, humans can easily intuitively grasp space. Therefore, by forming the opening 21 within a range of 300 mm in front and behind the imaging plane P, the focus of the observer's eyes looking at the wall 20 naturally matches the opening 21, and as a result, the observer can easily clearly view the real image formed on the imaging plane P in or near the opening 21. However, it is preferable that the distance between the imaging plane P and the opening 21 is as close as possible, more preferably 0 to 250 mm or 0 to 200 mm, and particularly preferably 0 to 150 mm or 0 to 100 mm.

[0032] In addition, the vertical height of the opening 21 in the wall (the linear distance from the lower end of the upper wall portion 22 to the upper end of the lower wall portion 23) can be appropriately set according to the size of the display device 11 (the size of the imaging surface P). For example, the height of the opening 21 may be set to match the vertical width of the display screen of the display device 11, or may be slightly smaller than the vertical width of the display screen. As a specific numerical value, the height of the opening 21 in the wall can be, for example, about 300 to 1000 mm. In addition, the width in the left-right direction (the length in the left-right direction) of the opening 21 can also be appropriately set according to the size of the display device 11 (the size of the aerial image). For example, the width of the opening 21 may be set to match the horizontal width of the display screen of the display device 11, or may be slightly smaller than the vertical width of the display screen. In this way, the opening 21 in the wall is designed to the size of the display device 11. The size of the opening 21 in the wall and the imaging surface P are substantially the same, and it is preferable that the opening 21 overlaps with the imaging surface P without excess or deficiency. However, for example, the imaging plane P may be larger than the opening 21 and may partially overlap the opening 21, or the imaging plane P may be smaller than the opening 21 and there may be a portion where a gap is formed between the imaging plane P and the periphery of the opening 21.

[0033] In addition, the peripheral portion of the opening 21 in the wall 20, that is, the lower end of the upper wall portion 22 and the upper end of the lower wall portion 23, are preferably formed at an acute angle θ as shown in the enlarged view of FIG. 1. Specifically, the angle θ of the peripheral portion of the opening 21 is preferably 80 degrees or less, and more preferably 60 degrees or less or 45 degrees or less. The lower limit of the angle θ is not particularly limited, but may be, for example, 10 degrees or more or 20 degrees or more. By forming the peripheral portion of the opening 21 at an acute angle in this manner, the boundary of the opening 21 can be made less noticeable. That is, compared to a case in which the peripheral portion of the opening 21 is cut at a right angle with a thickness, the presence of the wall 20 is reduced in the peripheral portion formed at an acute angle, and the boundary between the aerial image and the surrounding space becomes more natural. In addition, when an image is projected near the peripheral portion of the opening 21 as described later, the image gradually becomes darker in the portion formed at an acute angle, so that the boundary between the aerial image and the wall surface can be prevented from appearing like a step. This improves the visual continuity between the space in which the aerial image is floating and the wall surface, enhancing the effect of a unified image expression.

[0034] The wall 20 can be formed of a transparent or semi-transparent material such as an acrylic plate, a glass plate, or a resin plate. A preferred example of the resin plate is a milky semi-polycarbonate plate (a milky semi-transparent polycarbonate plate). In particular, the part where the image is projected from the projection device 30 needs to transmit and diffuse the image light appropriately, so it is preferable to perform a half-mirror process with a transmittance of about 30 to 70% or a light diffusion process with fine irregularities. The thickness of the plate material forming the wall 20 can be, for example, about 10 to 30 mm in order to ensure sufficient strength while enabling the peripheral edge of the opening 21 to be formed at an acute angle. The wall 20 is not limited to a single-layer structure, and may be a multi-layer structure in which multiple plate materials are laminated, and in this case, each layer is given different optical characteristics, thereby enabling more effective image expression. The wall 20 may be a mesh screen formed of metal, plastic, or the like. In the case of a mesh screen, the mesh pitch and opening ratio can be adjusted to obtain appropriate transmittance and projection characteristics. For example, the mesh pitch can be about 0.5 to 5.0 mm, and the aperture ratio can be about 30 to 70%. By using such a mesh screen, the image projected onto the wall 20 can be viewed while also partially viewing the space behind the wall 20, making it possible to present a more open space. In addition, the mesh screen has the advantage that the balance between the visibility of the projected image and the visibility of the aerial image can be adjusted by the mesh pitch and aperture ratio.

[0035] The projection device 30 projects an image onto a portion of the wall 20 other than the opening 21. As the projection device 30, a known projector such as a DLP (Digital Light Processing) type, a liquid crystal type, or a laser type can be used. As shown in FIG. 1, in this embodiment, the projection device 30 is disposed in a space on the opposite side of the wall 20 from the observer, similar to the aerial display device 10. By disposing the projection device 30 on the back side of the wall 20 in this way, the observer is less likely to directly see the projection device 30, so that a more immersive image presentation can be realized. In this embodiment, the projection device 30 is installed on the ceiling as shown in FIG. 1, but is not limited thereto, and it is also possible to install it on the back side of the wall 20, for example.

[0036] In this embodiment, the projection device 30 is disposed so as to project an image onto the rear surface of the wall 20, but as described above, if the wall 20 is transparent, semi-transparent, or mesh-like, the observer can view the image projected onto the wall 20 through the wall 20. Although not shown in the drawings, the projection device 30 can also be disposed so as to project an image onto the front surface (observer side) of the wall 20. In this case, the observer will view the image projected onto and reflected from the wall 20.

[0037] In addition, the projection device 30 must be selected so as to ensure sufficient brightness depending on the material of the wall 20 and the brightness of the ambient light. For example, it is preferable to use a projector having a brightness of about 5000 to 20000 lumens. In addition, the resolution of the projection device 30 is preferably full HD (1920 x 1080 pixels) or more, and more preferably 4K (3840 x 2160 pixels) or more. The projection distance from the projection device 30 to the wall 20 can be appropriately set depending on the focal length of the projection lens of the projection device 30, and can be, for example, about 1000 to 3000 mm. In addition, in this embodiment, one projection device 30 projects an image onto both the upper wall portion 22 and the lower wall portion 23 of the wall 20. Note that a plurality of projection devices 30 having such a configuration are arranged in the horizontal direction of the ceiling. However, separate projection devices 30 may be provided for the upper wall portion 22 and the lower wall portion 23.

[0038] The detection device 40 is a sensor for detecting the presence or absence or position of an object (mainly the observer's hand or finger) within a predetermined range of the imaging surface P of the aerial imaging member 12. For example, a known sensor such as an infrared sensor, a three-dimensional sensor, a depth sensor, a stereo camera, a ToF (Time of Flight) camera, or an ultrasonic sensor can be used as the detection device 40. As shown in FIG. 1, in this embodiment, the detection device 40 is arranged in a space on the same side as the observer, with the wall 20 as a boundary. The detection device 40 is installed on the ceiling on the front side of the wall 20, and can detect the presence or absence of an object close to the front of the wall 20 and its position information by irradiating detection light or the like along the front surface of the wall 20. By arranging the detection device 40 on the front side of the wall 20 in this way, it is possible to detect an observer approaching the wall 20 at an early stage. However, the detection device 40 may be arranged on the back side of the wall 20 (the same side as the aerial display device 10). In this case, the detection device 40 detects the observer's hand or finger that has entered the back side of the wall 20 through the opening 21. Furthermore, the detection device 40 may be installed at any position, such as on the floor or ceiling, as long as the position can sufficiently cover the space near the imaging plane P.

[0039] The detection range of the detection device 40 is set within a range of, for example, 500 mm in front of and behind the image plane P. This detection range corresponds to the range in which an observer naturally reaches out to touch the aerial image. In other words, it is set as a range that the observer can reach comfortably when reaching out from a standing position to interact with the aerial image. It is preferable that the detection device 40 has a position resolution of at least about 10 mm within this range, and that the detection response time can be detected at a frame rate of, for example, 30 frames / second or more so that the detection response time can follow human movements in almost real time. By satisfying such performance requirements, a natural response to the observer's intuitive movements can be realized.

[0040] The detection device 40 may be capable of detecting not only the presence or absence and position of an object in the space near the imaging surface P, but also the movement and gesture of the object. For example, the detection device 40 can detect the approach speed and movement direction of the observer's hand approaching the opening 21, tapping and swiping with the fingertips, and the like. It can also detect the movement of multiple fingers, such as pinch-in and pinch-out movements. In this way, the detection device 40 can detect various movements and gestures of the observer, and these detection results are reflected in the image control of the display device 11 and the projection device 30 via the control device 50. Specifically, interactive image expression can be realized according to the observer's movements, such as the aerial image retreating when the observer approaches the hand, changing the aerial image when the observer taps with a finger, and changing the entire image in accordance with the swipe movement.

[0041] As shown in FIG. 3, the control device 50 is connected to the display device 11, the projection device 30, and the detection device 40 of the above-mentioned aerial display device 10 via a wired or wireless LAN or the like, and controls the entire aerial display system 100. For example, the control device 50 generates images to be output by the display device 11 and the projection device 30, and executes control to cause each of these devices 11 and 30 to output the images. In addition, the control device 50 generates or changes images to be output by the display device 11 and the projection device 30 based on a detection signal by the detection device 40. The control device 50 may be configured by one computer, or the functions executed by the control device 50 may be distributed to multiple computers, and the control device 50 may be constructed by these multiple computers. For example, when the control device 50 has a distributed configuration, for example, an image generation server that generates images for the display device 11 and the projection device 30 and an information processing server that analyzes input information from the detection device 40 can be provided separately. In addition, each of these servers is connected to each other via a network and shares necessary information.

[0042] As shown in FIG. 3, the control device 50 has at least a control unit 51 and a storage unit 52. The control unit 51 is composed of, for example, a processor and a memory. Examples of the processor are a known CPU (Central Processing Unit), GPU (Graphics Processing Unit), and other control circuits. The processor performs a predetermined arithmetic processing according to a program stored in the memory, and executes various control processing while writing the arithmetic results in the working space of the memory. The memory is composed of, for example, a volatile memory such as a RAM (Random Access Memory), and is used for the arithmetic processing by the above-mentioned processor. The storage unit 52 is an element (storage) for mainly storing video content to be output to the display device 11 and the projection device 30, and information used for the arithmetic processing in the control unit 51. The storage unit 52 is composed of a non-volatile memory such as a ROM (Read Only Memory). The video content includes, for example, texture images for aerial imaging or wall projection, image sequences for animation, pattern images for generating various visual effects, and parameter information for video effects that change according to the detection results of the detection device 40. The storage unit 52 may also store tracking parameters for tracking the position and movement of an object detected by the detection device 40. The storage unit 52 may also store programs used for processing in the control unit 51. For example, a group of programs including an image processing program, a video generation program, and libraries for various types of arithmetic processing are stored in the storage unit 52. These programs are executed by the processor of the control unit 51, causing the control unit 51 to realize various functions.

[0043] FIG. 4 shows an example of video content generated by the control unit 51 of the control device 50 and output by the display device 11 and the projection device 30, respectively. FIG. 4(a) shows an example of the entire video content generated by the control device 50, which is a single video (moving or still image), and this video can be divided into three stages, parts A to C. Part A is a part that the projection device 30 projects onto the upper wall portion 22 of the wall 20, part B is a part that the display device 11 of the aerial display device 10 forms an aerial image on the imaging surface P provided at a position corresponding to the opening 21 of the wall 20 via the aerial imaging member 12, and part C is a part that the projection device 30 projects onto the lower wall portion 23 of the wall 20. In this way, in order to express the continuity or unity of the entire video, the control unit 51 may first generate a video in which each part is connected, and then divide it into each part. For example, the unity of the entire video can be enhanced by unifying the color tone between each part, adjusting the brightness balance, synchronizing the timing of the movements, etc.

[0044] On the other hand, FIG. 4(b) shows an example of an image output from the projection device 30 and projected onto the wall 20. As shown in FIG. 4(b), the projection device 30 outputs the A part projected onto the upper wall portion 22 of the wall 20 and the C part projected onto the lower wall portion 23 of the entire image generated by the control unit 51 as is, but the B part corresponding to the opening 21 of the wall 20 is not output as is, but is replaced with the B' part representing a blank (non-luminescent part) or black. This is to prevent double projection near the opening 21 and ensure the visibility of the aerial image. That is, in this embodiment, the projection device 30 projects a single image by combining the A part, the B' part, and the C part onto the wall 20. Also, FIG. 4(b) shows an example of an image output from the display device 11 and projected into an aerial image near the opening 21 of the wall 20. As shown in FIG. 4(c), the display device 11 extracts and displays only the B part from the entire image generated by the control unit 51. This part B is the part that is visible to the observer through the opening 21 in the wall 20, and is the central element that is most likely to attract attention in the entire video. For this reason, it is preferable to place a characteristic part of the video content in part B. An example of a characteristic part is a virtual object, character, or icon image that the observer wants to touch. In addition, since the video in part B appears to be floating in the air, a more effective video expression can be achieved by adding a three-dimensional depth expression or a movement that makes it appear to jump out in front.

[0045] In this way, it is preferable that the control unit 51 divides one video content into a part for wall projection (part A, part C) and a part for aerial imaging (part B), and outputs them to the projection device 30 and the display device 11, respectively. The observer, for example, observes the entire wall 20 as shown in FIG. 1 and FIG. 2, and therefore sees an image in which parts A to C are connected. However, since there is no substance (wall 20) in part B, which is the most reachable part of the entire image from the observer, the observer can have the illusion that the entire image is floating in the air. Also, when the observer reaches out to the opening 21 of the wall 20, the finger is detected by the detection device 40, and the image changes. For example, various interactive image expressions are possible, such as a virtual object in the image being deformed in response to the approach of the hand, a virtual object avoiding the hand, an aerial image being deformed in response to a tapping motion with a finger, and the entire image being scrolled in response to a swipe motion. Therefore, the observer can have the illusion that he or she touches an insubstantial image and that the image has changed due to the touch.

[0046] For example, the image generation process by the control device 50 is executed in the following procedure. First, when the detection device 40 detects the approach or movement of a real object (such as the observer's finger), the control device 50 predicts the real object position after a predetermined time based on the position, movement direction, and movement speed of the detected real object. Next, the control device 50 determines the relative relationship between the predicted real object position and the virtual object of the currently displayed image. Then, when the approach of the real object to the virtual object is predicted, a virtual object retreat or avoidance operation pattern is determined for a part or all of the image. At this time, the virtual object retreat or avoidance operation is executed smoothly over a certain period of time (for example, 0.5 seconds) taking into consideration the continuity of the image display. By repeatedly executing the above process, a natural image change can be realized. Note that as the virtual object retreat or avoidance operation pattern, an image retreating up, down, left, right, image deformation, image splitting or merging, etc. can be adopted.

[0047] 1 and 2 show an example in which an image is projected from a projection device 30 onto a wall 20 that separates a space where an observer enters from a space where an aerial display device 10 is installed. However, in addition to projecting an image onto this wall 20, it is also possible to project an image from another projection device onto both or either of the left and right side walls shown in FIG.

[0048] In the above, in order to express the contents of the present invention, the present specification has described the embodiments of the present invention with reference to the drawings. However, the present invention is not limited to the above-mentioned embodiments, and includes modifications and improvements that are obvious to a person skilled in the art based on the matters described in the present specification. [Explanation of symbols]

[0049] 10... Aerial display device 11... Display device 12... Aerial imaging member 13... Support 20...wall 21...opening 22...Top wall part 23...Bottom wall part 30...Projection device 40...Detection device 50: control device 51: control section 52...Memory unit 100...Aerial display system P...Image plane

Claims

1. A display device for displaying an image; an aerial imaging member that refracts a first image light incident from the display device and forms an image in the air; a wall having an opening formed within a range of 300 mm in front of and behind the imaging surface of the aerial imaging member; a projection device that projects a second image light onto a portion of the wall other than the opening, At least a portion of the first image light passes through an opening in the wall; the projection device is disposed on the same side as the display device and the aerial imaging member with respect to the wall; At least a part of the wall, where the second image light is projected, at least partially transmits the second image light. Aerial display system.

2. The wall has an upper wall portion and a lower wall portion, and the opening is formed between the upper wall portion and the lower wall portion. The aerial display system of claim 1 .

3. A display device for displaying an image; an aerial imaging member that refracts the image light incident from the display device and forms an image in the air; a wall having an opening formed within a range of 300 mm in front of and behind the imaging surface of the aerial imaging member; At least a portion of the image light passes through an opening in the wall; The wall is formed such that at least a part of the periphery of the opening has an acute angle when viewed in the thickness direction of the wall. Aerial display system.

4. a detection device for detecting the presence or absence or position of an object within a predetermined range in front of and behind the imaging surface of the aerial imaging member; a control device that controls an image displayed by the display device in accordance with a detection result of the detection device The aerial display system of claim 1 .

5. The aerial imaging member is a first group of reflecting surfaces; and a second group of reflecting surfaces having a reflecting surface in a direction substantially perpendicular to the first group of reflecting surfaces; The image light incident from the display device is successively reflected by the first reflecting surface group and the second reflecting surface group to form an aerial image. The aerial display system of claim 1 .

6. A display device displays an image; a step of an aerial imaging member refracting a first image light incident from the display device to form an image in the air; a wall having an opening formed therein is provided within a range of 300 mm in front of and behind an image forming surface of the aerial imaging member, and a projection device projects a second image light onto a portion of the wall other than the opening; At least a portion of the first image light passes through an opening in the wall; the projection device is disposed on the same side as the display device and the aerial imaging member with respect to the wall; At least a part of the wall, where the second image light is projected, at least partially transmits the second image light. Aerial imaging method.

7. A step of displaying an image by a display device; a step of an aerial imaging member refracting the image light incident from the display device to form an image in the air, a wall having an opening is provided within a range of 300 mm in front of and behind the imaging surface of the aerial imaging member; At least a portion of the image light passes through an opening in the wall; The wall is formed such that at least a part of the periphery of the opening has an acute angle when viewed in the thickness direction of the wall. Aerial imaging method.

Images