Aerial display system and aerial imaging method

Abstract

Even in environments where it is difficult for the observer to intuitively grasp the position where the real image is formed, this technology makes it easier for the observer to clearly perceive the real image. [Solution] The aerial display system 100 comprises a display device 11 that displays images, an aerial imaging member 12 that refracts the image light incident from the display device 11 to form an image in the air, and a wall 20 in which an opening 21 is formed within a range of 300 mm in front of and behind the imaging surface P of the aerial imaging member 12, and is configured so that at least a portion of the image light passes through the opening 21 of the wall 20.

Description

Technical Field

[0001] The present invention relates to a system and method for forming a real image of a video displayed by a display device in the air.

Background Art

[0002] Conventionally, an optical element for forming a real image of a video displayed by a display device in the air is known. As an example of such an aerial imaging optical element, for example, Patent Document 1 discloses an optical element of a two-plane orthogonal reflector type. In this method, by using two reflector arrays arranged orthogonally and sequentially reflecting the incident video light in directions orthogonal to each other, a real image of the video can be formed in the air. By using such an aerial imaging optical element, it is said that a bright and clear real image can be obtained with high light utilization efficiency.

Prior Art Documents

Non-Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, the aerial imaging optical element of Patent Document 1 is assumed to be used for a non-contact input device, and a real image of an icon image is formed in the space near the user's hand, and the user uses a finger, a touch pen, etc. to detect that the real image has been indicated, so that signal input can be performed without physically contacting the display device. In this case, since the user only needs to look in a relatively close range near the hand, it is easy to focus on the imaging surface of the image, and as a result, the image is easily and clearly visible to the user.

[0005] On the other hand, when considering the application of such aerial imaging optical elements to relatively large video works or signage systems installed outdoors or in stores, it becomes necessary to project the actual image of the video onto a relatively large exhibition space. In this case, in order for the observer to clearly perceive the projected image in space, the observer would be forced to walk closer to the image plane within the large exhibition space and focus their eyes on the image plane. In other words, the image would appear blurry to an observer standing far from the image plane, and even an observer standing near the image plane would still appear blurry if their eyes do not focus on the image plane. Thus, in situations where it is difficult for the observer to intuitively grasp the position of the projected image, such as when the actual image of a video is projected onto a relatively large exhibition space, there is a concern that the aesthetic effect of the video work and the advertising effect of the signage system will be reduced.

[0006] Therefore, the main objective of the present invention is to make it easier for observers to clearly perceive the real image, even in environments where it is difficult for observers to intuitively grasp the position where the real image is formed. [Means for solving the problem]

[0007] Therefore, the inventors of the present invention diligently considered means to solve the problems of the prior art described above, and as a result, they found that by providing a physical wall near the image-forming surface of the real image created by the aerial imaging member, and forming an opening in the wall near the image-forming surface, the wall and the image-forming surface come into the observer's field of view simultaneously, making it easier for the observer to intuitively grasp the position of the real image. Based on this finding, the inventors realized that the problems of the prior art could be solved, and thus completed the present invention. Specifically, the present invention has the following configuration or process.

[0008] The first aspect of the present invention relates to an aerial display system. The aerial display system comprises a display device, an aerial imaging member, and a wall. The display device displays an image. In this specification, "image" includes both moving images and still images, or either one. The aerial imaging member refracts the image light incident from the display device to form an image in the air. As the aerial imaging member, any known type of optical element capable of forming a real image in the air can be used. As the aerial imaging member, in addition to the two-plane orthogonal reflector method described above (Patent Document 1), for example, a retroreflective method combining a half mirror and a retroreflective sheet, a concave mirror method combining a concave mirror and a half mirror, or a lens array method using a microlens array can be used. The wall has an opening formed within a range of 300 mm in front of and behind the imaging surface of the aerial imaging member. This wall is configured to partition the exhibition space between the observer side and the display device side. The range of 300 mm in front of and behind the imaging surface is set considering the human clear vision distance (the distance at which the human eye can easily focus). The 300mm range in front of and behind the image plane refers to a total length of 600mm, from 300mm behind the image plane to 300mm in front of the image plane, with respect to the optical axis of the image light perpendicular to the image plane. The aerial display system is configured such that at least a portion of the image light emitted from the display device passes through an opening in the wall. The shape of the opening is not limited to a rectangle; it can also be a circle, ellipse, triangle, pentagon, or any polygon with more sides.

[0009] As described above, by providing a wall with an opening within a 300mm range in front of and behind the image-forming surface of the aerial imaging member, the observer's eye naturally focuses on the wall or its opening. As the observer focuses on the opening and views the aerial image through it, the position of the image-forming surface becomes easier to grasp intuitively, resulting in clearer visualization of the aerial image. Furthermore, since the wall functions as a reference point for dividing the exhibition space, the observer can more accurately perceive the depth of the space, which also improves the visibility of the aerial image.

[0010] In the aerial display system according to the present invention, it is preferable that the wall has an upper wall portion and a lower wall portion, with an opening formed between the upper wall portion and the lower wall portion. By forming an opening between the upper wall portion and the lower wall portion in this way, the aerial image appears to float to the observer between the upper and lower walls (opening), and the effect of improving the visibility of the aerial image caused by the presence of the wall can be more effectively exerted. In addition, the human field of view is narrower in the vertical direction (approximately 120 degrees) than in the horizontal direction (approximately 200 degrees), and spatial recognition requires more cues in the vertical direction. Therefore, by providing a wall in the vertical direction, the observer can grasp the extent of the space more accurately, and the presence of the aerial image located in that space can be further emphasized.

[0011] The aerial display system according to the present invention further comprises a projection device. The projection device projects an image onto a part of the wall other than the opening. The projection device may be positioned to project an image onto the surface side of the wall (i.e., the observer side) or onto the back side of the wall (i.e., the display device side). By projecting an image onto a part of the wall other than the opening in this way, it becomes possible to create an integrated image representation of the aerial image and the wall projection image. For example, by projecting an image onto the wall so as to be continuous with the aerial image visible through the opening, a visual effect can be created in which part of the image appears to be popping out of the wall and floating. Also, for example, by projecting related images around the aerial image visible through the opening, a visual effect can be created in which the aerial image appears to be floating up from the wall. Furthermore, by linking the movement of the images between the aerial image and the wall projection image, for example, a dynamic effect can be achieved in which part of the image appears to pop out three-dimensionally from the wall. In this way, by adding projection equipment, it becomes possible to create continuous video expressions that eliminate the boundary between aerial images and wall projections, greatly expanding the range of effects for the aerial display system and allowing for more effective use of the wall.

[0012] In the aerial display system according to the present invention, it is preferable that the projection device is positioned on the same side (i.e., the back side of the wall) as the display device and the aerial image forming member, with the wall as the boundary. In this case, at least a portion of the wall onto which the image light is projected from the projection device is configured to transmit the image light at least partially. For example, the portion of the wall onto which the image light is projected from the projection device may be transparent, semi-transparent, or mesh-like. By positioning the projection device on the same side as the display device and the aerial image forming member in this way, the entire system can be integrally installed on the back side of the wall. This provides the observer with only a space to enjoy the image, and the devices do not enter the observer's field of vision. Furthermore, by positioning the projection device on the back side of the wall and configuring a portion of the wall to allow image light to pass through, the image projected from the projection device is displayed softly on the surface of the wall, achieving an integrated image expression that does not feel out of place with the aerial image. In addition, by adjusting the transmittance of the wall, the balance between the aerial image and the wall projection image can be optimized, achieving a more natural continuity of image.

[0013] In the aerial display system according to the present invention, it is preferable that at least a portion of the periphery of the opening of the wall is formed at an acute angle. By forming the periphery of the opening of the wall at an acute angle in this way, the boundary of the opening can be made less conspicuous. That is, compared to the case where the periphery of the opening is thick and cut at a right angle, the presence of the wall is reduced when the periphery is formed at an acute angle, and the boundary between the aerial image and the surrounding space becomes more natural. In particular, when projecting an image onto the periphery of the opening, the image gradually darkens in the part formed at an acute angle, so it is possible to suppress the appearance of a step-like difference at the boundary between the aerial image and the wall surface. As a result, the visual continuity between the space in which the aerial image is floating and the wall surface is improved, and the effect of integrated image expression can be enhanced.

[0014] The aerial display system according to the present invention preferably further comprises a detection device and a control device. The detection device detects the presence or position of an object within a predetermined range in front of and behind the image-forming surface of the aerial imaging member. Examples of detection devices include infrared sensors, three-dimensional sensors, depth sensors, stereo cameras, ToF cameras, and ultrasonic sensors. The control device controls the image displayed by the display device according to the detection results of the detection device. By providing a detection device and a control device in this way, the image can be changed interactively in response to the observer's actions. For example, if an observer reaches out to the aerial image, the aerial image may change in response to that action, or the aerial image and the wall projection image may change in conjunction. The predetermined range in front of and behind the image-forming surface can be selected from, for example, a range of 500 mm in front of and behind the image-forming surface. This detection range corresponds to the range of motion in which a person would naturally reach out to touch the aerial image, allowing the observer to interact with the aerial image with effortless and intuitive movements. As a result, by detecting the observer's natural movements and controlling the image, a more attractive and interactive visual expression can be realized.

[0015] In the aerial display system according to the present invention, the aerial imaging member is preferably a two-plane orthogonal reflector type. An aerial imaging member of this type includes a first group of reflective surfaces and a second group of reflective surfaces having reflective surfaces in a direction substantially orthogonal to the first group of reflective surfaces. Substantially orthogonal means in the range of 85 to 95 degrees. The aerial imaging member is configured to form an aerial image by sequentially reflecting the image light incident from the display device with the first group of reflective surfaces and the second group of reflective surfaces. By adopting this two-plane orthogonal reflector type, a bright and clear aerial image can be formed with high light utilization efficiency. Furthermore, this type can be realized with a compact configuration compared to other types (e.g., retroreflection type or concave mirror type), thus increasing the flexibility of the overall system installation.

[0016] The second aspect of the present invention relates to a method for aerial imaging. In the aerial imaging method according to the present invention, a display device displays an image (display step). Further, an aerial imaging member refracts the image light incident from the display device to form an image in the air (imaging step). Here, a wall having an opening formed within a range of 300 mm before and after the imaging surface of the aerial imaging member is provided, and at least a part of the image light passes through the opening of the wall.

Effects of the Invention

[0017] According to the present invention, even in an environment where it is difficult for an observer to intuitively grasp the imaging position of a real image, such as a relatively large exhibition space, it is possible to make it easier for the observer to clearly visually recognize the real image.

Brief Description of the Drawings

[0018] [Figure 1] FIG. 1 is a side view schematically showing the overall configuration of an aerial display system according to an embodiment of the present invention. [Figure 2] It is a front view of the aerial display system shown in FIG. 1, and schematically shows the state of image projection onto a wall surface and aerial imaging. [Figure 3] It is a block diagram showing the functional configuration of a control system of an aerial display system according to an embodiment of the present invention. [Figure 4] It is a diagram showing the divided state of video content, where (a) shows the entire video content, (b) shows the video output from the projection device, and (c) shows the video output from the display device.

Modes for Carrying Out the Invention

[0019] Hereinafter, modes for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to the modes described below, and includes those appropriately modified by those skilled in the art within an obvious range from the following modes.

[0020] FIG. 1 shows the overall configuration of an aerial display system 100 according to an embodiment of the present invention. FIG. 2 schematically shows a front view of the aerial display system 100 according to this embodiment. Further, FIG. 3 shows the functional configuration of mainly the control system of the aerial display system 100 according to this embodiment. As shown in FIG. 1, the aerial display system 100 according to this embodiment mainly includes an aerial display device 10, a wall 20, a projection device 30, and a detection device 40. Further, as shown in FIG. 3, the aerial display system 100 further includes a control device 50, and the control device 50 controls the aerial display device 10, the projection device 30, and the detection device 40.

[0021] For example, as shown in FIGS. 1 and 2, the aerial display system 100 according to this embodiment is assumed to be installed in a display space where the height from the floor to the ceiling exceeds the height of an average adult (1400 - 2000 mm). In such a display space, the aerial display system 100 forms a real image of a video in the air at the front of the line of sight of an observer in a standing position, and allows the observer to visually recognize the real image thus formed in the air. Also, in this system, a video is projected onto the wall 20, and the observer is allowed to visually recognize the projected video together with the real image formed in the air. The height from the floor to the ceiling is not particularly limited, but for example, it is preferably 2000 mm or more, and particularly preferably 2500 - 4000 mm. Also, the width of the display space, particularly the width of the wall 20, is preferably 1000 mm or more, and preferably 1500 - 4500 mm. The aerial display system 100 according to this embodiment can be suitably used, for example, for displaying video works or advertising displays in exhibition facilities such as art museums and showrooms, or commercial facilities. When viewed from the observer, the video projected on the wall 20 and the aerial image visually recognized through the opening 21 appear to be integrated, so that a three-dimensional expression with a sense of depth can be realized for the entire video. Also, the observer can enjoy the video while freely moving in front of the wall 20, and multiple observers can observe simultaneously. Note that the appropriate observation position of the observer is preferably within the range of 500 - 3000 mm in front of the wall 20.

[0022] The aerial display device 10 is a device for forming a real image of a video in the air within the exhibition space. The aerial display device 10 mainly comprises a display device 11, an aerial image forming member 12, and a support 13. The display device 11 outputs video light by displaying a video, and the aerial image forming member 12 uses this video light to form a real image in the air. The support 13 holds the aerial image forming member 12 in a predetermined position and orientation. These components are arranged together on the back side of the wall 20, which will be described later.

[0023] The display device 11 can be a general-purpose display such as a liquid crystal display or an organic EL display. In this embodiment, the display device 11 is positioned along the floor and outputs image light upwards (towards the ceiling). The placement of the display device 11 is not limited to this; for example, it can be mounted on the ceiling or leaned against a side wall by bending the optical path of the image light using an optical element such as a mirror. As mentioned above, in this embodiment, it is assumed that an image will be projected onto a relatively large exhibition space, so it is preferable to use a large display for the display device 11. The size of the aerial image depends on the display size of the display device 11, so for example, it is preferable that the diagonal length of the display screen of the display device 11 is 40 inches or more, and particularly preferable that it is 50 to 80 inches. Also, the display device 11 is not limited to a single display, but may be configured by connecting multiple displays together.

[0024] The aerial imaging member 12 is an optical element that refracts the image light incident from the display device 11 to form a real image in the air. As the aerial imaging member 12, any known optical element capable of forming a real image in the air can be used, such as a two-plane orthogonal reflector system, a retroreflection system, a concave mirror system, or a lens array system. The two-plane orthogonal reflector system includes a first group of reflective surfaces and a second group of reflective surfaces having reflective surfaces in a direction substantially orthogonal to the first group of reflective surfaces (substantially orthogonal means in the range of 85 to 95 degrees). In this system, a real image is formed in the air by sequentially reflecting the image light incident from the display device 11 by the first group of reflective surfaces and the second group of reflective surfaces. Such a two-plane orthogonal reflector system has the advantage of being able to form a bright and clear real image with high light utilization efficiency. Furthermore, the retroreflective method has a configuration that combines a half-mirror and a retroreflective sheet, and a real image is formed by returning the image light reflected by the half-mirror back in the incident direction by the retroreflective sheet. This method has the advantage of having a simple structure and being easy to manufacture. The concave mirror method has a configuration that combines a concave mirror and a half-mirror, and the image light focused by the concave mirror forms a real image via the half-mirror. This method has the advantage of being able to form a large real image. The lens array method has a configuration in which multiple microlenses are arranged regularly, and a real image is formed by focusing the image light with each microlens. This method has the advantage of being easy to make thin. In particular, the two-plane orthogonal reflector method has the advantage of being able to form a bright and clear real image with high light utilization efficiency, and can be realized with a compact configuration compared to other methods, so it is preferable to adopt this method in this embodiment.

[0025] Furthermore, the aerial imaging member 12 is preferably a thin, plate-like structure, and its thickness can be, for example, about 10 to 50 mm. The planar size of the aerial imaging member 12 is preferably about the same as or greater than the display screen size of the display device 11. Specifically, this is determined based on the display screen size of the display device 11, the distance between the display device 11 and the aerial imaging member 12, and the inclination angle of the aerial imaging member 12. For example, if the display screen of the display device 11 is 50 inches, the diagonal length of the aerial imaging member 12 is preferably 60 inches or more. In this embodiment, the aerial imaging member 12 is installed by a support 13 at an angle of approximately 45 degrees to the floor. This inclination angle is set to image the image light emitted from the display device 11 at an appropriate position and can be adjusted as appropriate within the range of 30 to 50 degrees. As shown in Figure 1, the inclination angle of the aerial imaging member 12 is such that it gradually tilts backward (away from the observer) in the direction rising from the floor.

[0026] Furthermore, as shown in Figure 1, the image light output from the display device 11 diffuses as it enters the aerial imaging member 12. The aerial imaging member 12 then refracts the image light incident from the display device 11 at a predetermined angle, thereby forming a real image in the air. Here, the angle of incidence and the angle of emission of the image light with respect to the aerial imaging member 12 are equal. In the example shown in Figure 1, the aerial imaging member 12 is positioned such that the refraction angle of the image light emitted perpendicularly from the display device 11 is 90 degrees. That is, the image light traveling perpendicularly from such a display device 11 enters the aerial imaging member 12 at an angle of 45 degrees and exits at an angle of 45 degrees equal to the angle of incidence. In this way, all the image light emitted diffusely from the display device 11 is refracted by the aerial imaging member 12 and then converges on the same plane to form a real image. In this specification, the plane on which the imaging points of each image light are aligned is called the "imaging plane," and is shown in Figure 1 by the dotted line labeled P. Furthermore, the distance from the position where the image light enters the aerial imaging member 12 to the display surface of the display device 11 (incident light path length L) i ) and the distance from the position where the image light is emitted from the aerial imaging member 12 to the imaging plane P (the length of the emitted light path L). o ) become equal. Thus the incident optical path length Li and the output light path length L o As these two values ​​become equal, a real image the same size as the display surface of the display device 11 is formed on the imaging plane P. The real image thus formed is perceived by the observer as if the display surface of the display device 11 were located at the position of the imaging plane P.

[0027] The aerial imaging member 12 is preferably made of a material with high light transmittance, such as acrylic resin or glass. Acrylic resin is particularly suitable because it has excellent processability and sufficient transmittance (90% or more). In the case of a two-plane orthogonal reflector type aerial imaging member 12, the group of reflective surfaces included in the member can be formed by depositing a thin metal film such as aluminum. The group of reflective surfaces must be formed with high precision so that their surfaces are orthogonal to each other (within an error of ±5 degrees).

[0028] The support 13 is a member for holding the aerial imaging member 12 in a predetermined position and orientation. The support 13 may have various adjustment mechanisms to precisely adjust its relative position to the aerial imaging member 12. For example, the support 13 may be equipped with a height adjustment mechanism to adjust the height of the aerial imaging member 12, a position adjustment mechanism to adjust its front-to-back position, and an angle adjustment mechanism to adjust its tilt angle. These adjustment mechanisms allow for fine-tuning of the position and orientation of the aerial imaging member 12 to an optimal state, for example, using alignment markers displayed on the display device 11. The support 13 has a fixing part that can be fixed to the floor or side wall of the exhibition space, and is structured to stably hold the aerial imaging member 12.

[0029] The wall 20 is constructed to separate the exhibition space into the installation space for the aerial display device 10 and the space where observers can enter. In this embodiment, the wall 20 plays a role in making the real image formed by the aerial display device 10 easily visible to observers. In addition, the wall 20 also serves as a screen onto which images are projected by the projection device 30. By combining the aerial image formed by the aerial display device 10 and the wall projection image by the projection device 30 in this way, various video effects can be realized.

[0030] As shown in Figures 1 and 2, in this embodiment, the wall 20 is mainly composed of an upper wall portion 22 and a lower wall portion 23, with an opening 21 formed between them. The image light output by the display device 11 of the aerial display device 10 is refracted by the aerial imaging member 12 and reaches the observer through the opening 21 of the wall 20. At this time, the arrangement of the aerial display device 10 and the wall 20 is adjusted so that the imaging surface P of the aerial imaging member 12 is formed near the opening 21 of the wall 20. Specifically, as shown in Figure 1, the upper wall portion 22 and the lower wall portion 23 are configured to be aligned in a straight line perpendicular to the floor, and it is most preferable that the imaging surface P of the aerial imaging member 12 is formed to coincide with the straight line connecting the upper wall portion 22 and the lower wall portion 23. In other words, it is preferable that the imaging surface P is formed within the opening 21 formed between the upper wall portion 22 and the lower wall portion 23, on the extension of the upper wall portion 22 and the lower wall portion 23. Furthermore, since the upper wall portion 22 and the lower wall portion 23 have thickness, the thickness range of these upper and lower walls may be considered as the thickness of the opening 21, and the image-forming surface P may be designed to be formed within this thickness range of the opening 21. In this way, by forming the image-forming surface P within the opening 21 between the upper wall portion 22 and the lower wall portion 23, the distance to the wall 20 and the distance to the image-forming surface P will be the same from the observer's perspective. Also, both the wall 20 and the image-forming surface P will be in the observer's field of view at the same time. As a result, the observer can more easily intuitively grasp the image point P by relying on the wall 20, and thus more easily perceive the real image of the image appearing on the image-forming surface P clearly.

[0031] As described above, it is most preferable that the image plane P is formed within the opening 21 of the wall 20 (on the extension of the upper wall portion 22 and the lower wall portion 23). However, since it is difficult to precisely adjust the position of the image plane P, a distance may exist between the image plane P and the opening 21 of the wall 20. Specifically, the opening 21 (on the extension of the upper wall portion 22 and the lower wall portion 23) should be located within a range of 300 mm in front of and behind the image plane P, using the optical axis of the image light traveling perpendicular to the image plane P as a reference. In other words, the range of 300 mm in front of and behind the image plane P means a total length of 600 mm, from a position 300 mm behind the image plane to a position 300 mm in front of the image plane, with respect to the optical axis of the image light perpendicular to the image plane P. Thus, it is permissible for the image plane P to be shifted by 300 mm in front of and behind the opening 21. This numerical range was determined considering the characteristics of human vision. In other words, the human eye has a distance called the clear vision distance, at which eye strain is minimized and the ability to focus on an object is most comfortable. This clear vision distance is generally known to be approximately 250mm to 300mm, and within this distance range, the burden of focusing is low, and stereoscopic vision due to binocular parallax functions effectively. Furthermore, within this distance range, humans can intuitively grasp space. Therefore, by forming the aperture 21 within a range of 300mm in front of and behind the image plane P, the observer's eyes, when looking at the wall 20, naturally focus on the aperture 21, and as a result, the real image formed on the image plane P within or near the aperture 21 is also more easily seen clearly by the observer. However, it is preferable that the distance between the image plane P and the aperture 21 be as close as possible, more preferably 0 to 250mm or 0 to 200mm, and particularly preferably 0 to 150mm or 0 to 100mm.

[0032] Furthermore, the vertical height of the wall opening 21 (the straight-line distance from the lower end of the upper wall portion 22 to the upper end of the lower wall portion 23) can be appropriately set according to the size of the display device 11 (size of the image-forming surface P). For example, the height of the opening 21 may be set to match the vertical width of the display screen of the display device 11, or it may be set to be slightly smaller than the vertical width of the display screen. Specifically, the height of the wall opening 21 can be set to, for example, 300 to 1000 mm. Also, the horizontal width (horizontal length) of the opening 21 can be appropriately set according to the size of the display device 11 (size of the aerial image). For example, the width of the opening 21 may be set to match the horizontal width of the display screen of the display device 11, or it may be slightly smaller than the vertical width of the display screen. In this way, the wall opening 21 is designed to match the size of the display device 11. Preferably, the size of the wall opening 21 and the image-forming surface P are substantially the same, and the opening 21 and the image-forming surface P overlap without excess or deficiency. However, for example, the image plane P may be larger than the aperture 21 and partially overlap the image plane P with respect to the aperture 21, or the image plane P may be smaller than the aperture 21 and there may be a gap formed between the image plane P and the periphery of the aperture 21.

[0033] Furthermore, it is preferable that the periphery of the opening 21 in the wall 20, that is, the lower end of the upper wall portion 22 and the upper end of the lower wall portion 23, be formed at an acute angle θ, as shown in the enlarged view of Figure 1. Specifically, the angle θ of the periphery of the opening 21 is preferably 80 degrees or less, and preferably 60 degrees or less or 45 degrees or less. The lower limit of the angle θ is not particularly limited, but for example, it may be 10 degrees or more or 20 degrees or more. By forming the periphery of the opening 21 at an acute angle in this way, the boundary of the opening 21 can be made less conspicuous. That is, compared to the case where the periphery of the opening 21 is thick and cut at a right angle, the presence of the wall 20 is reduced in the case of a periphery formed at an acute angle, and the boundary between the aerial image and the surrounding space becomes more natural. Also, as will be described later, when projecting an image near the periphery of the opening 21, the image gradually darkens in the part formed at an acute angle, so it is possible to suppress the appearance of a step-like difference at the boundary between the aerial image and the wall surface. This improves the visual continuity between the space in which the aerial image is projected and the wall surface, thereby enhancing the effect of integrated visual expression.

[0034] Furthermore, the wall 20 can be formed from a transparent or translucent material such as an acrylic plate, a glass plate, or a resin plate. A preferred example of a resin plate is a milky white translucent polycarbonate plate. In particular, for the part onto which the image is projected from the projection device 30, it is necessary to transmit and diffuse the image light appropriately, so it is preferable to apply a half-mirror finish with a transmittance of about 30-70% or a light-diffusing finish with fine irregularities. The thickness of the plate material forming the wall 20 can be, for example, about 10-30 mm in order to ensure sufficient strength while enabling the formation of a sharp angle around the periphery of the opening 21. Note that the wall 20 is not limited to a single-layer structure, but may also be a multi-layer structure formed by laminating multiple plate materials. In this case, by giving each layer different optical properties, more effective image expression is possible. The wall 20 may also be a mesh screen made of metal or plastic. In the case of a mesh screen, appropriate transmittance and projection characteristics can be obtained by adjusting the mesh pitch and aperture ratio. For example, the mesh pitch can be set to approximately 0.5 to 5.0 mm, and the aperture ratio to approximately 30 to 70%. Using such a mesh screen allows for a more open spatial design, as the projected image on the wall 20 can be viewed while also partially viewing the space beyond the wall 20. Furthermore, mesh screens have the advantage of allowing adjustment of the balance between the visibility of the projected image and the visibility of the aerial image by adjusting the mesh pitch and aperture ratio.

[0035] The projection device 30 projects an image onto a part of the wall 20 other than the opening 21. The projection device 30 can be a known projector, such as a DLP (Digital Light Processing), liquid crystal, or laser projector. As shown in Figure 1, in this embodiment, the projection device 30, like the aerial display device 10, is positioned in the space opposite the observer, separated by the wall 20. By positioning the projection device 30 behind the wall 20 in this way, it becomes difficult for the observer to directly see the projection device 30, thus enabling a more immersive visual experience. In this embodiment, the projection device 30 is installed on the ceiling as shown in Figure 1, but it is not limited to this; it can also be installed on the back of the wall 20, for example.

[0036] Furthermore, in this embodiment, the projection device 30 is positioned to project an image onto the back surface of the wall 20. However, as mentioned above, if the wall 20 is transparent, semi-transparent, or mesh-like, the observer can see the image projected onto the wall 20 by passing through the wall 20. Although not shown in the illustration, it is also possible to position the projection device 30 to project an image onto the front surface (observer side) of the wall 20. In this case, the observer will see the image projected onto and reflected from the wall 20.

[0037] Furthermore, the projection device 30 must be selected to ensure sufficient brightness depending on the material of the wall 20 and the brightness of the ambient light. For example, it is preferable to use a projector with a brightness of about 5,000 to 20,000 lumens. The resolution of the projection device 30 is preferably Full HD (1920 x 1080 pixels) or higher, and more preferably 4K (3840 x 2160 pixels) or higher. The projection distance from the projection device 30 to the wall 20 can be appropriately set according to the focal length of the projection lens of the projection device 30, but can be, for example, about 1,000 to 3,000 mm. In this embodiment, one projection device 30 is used to project images onto both the upper wall portion 22 and the lower wall portion 23 of the wall 20. Multiple projection devices 30 with this configuration are arranged horizontally on the ceiling. However, separate projection devices 30 may be provided for the upper wall portion 22 and the lower wall portion 23.

[0038] The detection device 40 is a sensor for detecting the presence or position of an object (mainly the observer's hand or fingers) within a predetermined range of the imaging surface P of the aerial imaging member 12. As the detection device 40, known sensors such as an infrared sensor, a three-dimensional sensor, a depth sensor, a stereo camera, a ToF (Time of Flight) camera, or an ultrasonic sensor can be used. As shown in Figure 1, in this embodiment, the detection device 40 is positioned in the space on the same side as the observer, separated by the wall 20. This detection device 40 is installed on the ceiling on the front side of the wall 20, and by irradiating detection light along the front of the wall 20, it can detect the presence or position of an object approaching the front of the wall 20. By positioning the detection device 40 on the front side of the wall 20 in this way, it is possible to detect an observer approaching the wall 20 at an early stage. However, the detection device 40 may also be positioned on the back side of the wall 20 (the same side as the aerial display device 10). In this case, the detection device 40 will detect the observer's hand or fingers that have entered the back side of the wall 20 through the opening 21. Furthermore, the detection device 40 can be installed at any location, such as the floor or ceiling, as long as it can sufficiently cover the space near the imaging plane P.

[0039] Furthermore, the detection range of the detection device 40 is set to, for example, a range of 500 mm in front of and behind the image plane P. This detection range corresponds to the range of motion in which an observer would naturally reach out and try to touch the aerial image. In other words, it is set to a range that an observer can easily reach when reaching out from a stationary position to interact with the aerial image. Preferably, the detection device 40 has a positional resolution of at least 10 mm within this range, and the detection response time is preferably such that it can detect at a frame rate of, for example, 30 frames / second or more, so that it can track human movements in near real time. By meeting these performance requirements, a natural response to the observer's intuitive movements can be achieved.

[0040] Furthermore, the detection device 40 may be capable of detecting not only the presence and position of objects in the space near the image plane P, but also the movement and gestures of those objects. For example, it can detect the approach speed and direction of movement when the observer's hand approaches the opening 21, as well as tapping and swiping movements with the fingertips. It can also detect the movement of multiple fingers, such as pinch-in and pinch-out movements. In this way, the detection device 40 can detect various actions and gestures of the observer, and these detection results are reflected in the image control of the display device 11 and projection device 30 via the control device 50. Specifically, interactive image representation can be realized in response to the observer's actions, such as the aerial image receding when the observer brings their hand closer, the aerial image changing when the observer taps with their finger, or the entire image changing in accordance with a swipe movement.

[0041] As shown in Figure 3, the control device 50 is connected to the display device 11, projection device 30, and detection device 40 of the aerial display device 10 via a wired or wireless LAN, and is responsible for controlling the entire aerial display system 100. For example, the control device 50 generates the images to be output by the display device 11 and projection device 30, and controls these devices 11 and 30 to output the images. The control device 50 also generates or changes the images to be output by the display device 11 and projection device 30 based on detection signals from the detection device 40. The control device 50 may be configured as a single computer, or the functions performed by this control device 50 can be distributed among multiple computers, and the control device 50 can be constructed using these multiple computers. For example, in the case of a distributed configuration of the control device 50, a separate image generation server can be provided to generate images for the display device 11 and projection device 30, and an information processing server can be provided to analyze input information from the detection device 40. These servers are interconnected via a network to share necessary information.

[0042] Furthermore, as shown in Figure 3, the control device 50 has at least a control unit 51 and a storage unit 52. The control unit 51 is composed of, for example, a processor and memory. Examples of processors include a known CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and other control circuits. The processor performs predetermined arithmetic processing according to a program stored in memory, and executes various control processes while writing the calculation results to the memory's workspace. The memory is composed of volatile memory such as RAM (Random Access Memory) and is used for the arithmetic processing by the processor described above. The storage unit 52 is mainly an element (storage) for storing video content to be output to the display device 11 and the projection device 30, and information used for arithmetic processing in the control unit 51. The storage unit 52 is composed of non-volatile memory such as ROM (Read Only Memory). Video content includes, for example, texture images for aerial projection or wall projection, image sequences for animation, pattern images for generating various visual effects, and parameter information for video effects that change according to the detection results of the detection device 40. Furthermore, the storage unit 52 may store tracking parameters for tracking the position and movement of objects detected by the detection device 40. The storage unit 52 may also store programs used for processing in the control unit 51. For example, a group of programs including image processing programs, video generation programs, and libraries for various calculation processes may be stored. These programs are executed by the processor of the control unit 51, enabling the control unit 51 to perform various functions.

[0043] Figure 4 shows an example of video content generated by the control unit 51 of the control device 50 and output by the display device 11 and projection device 30, respectively. Figure 4(a) shows an example of the entire video content generated by the control device 50, which is a single video (moving image or still image) and can be divided into three parts, A to C. Part A is the part that the projection device 30 projects onto the upper wall portion 22 of the wall 20, Part B is the part that the display device 11 of the aerial display device 10 projects onto an image-forming surface P provided at a position corresponding to the opening 21 of the wall 20 via the aerial image-forming member 12, and Part C is the part that the projection device 30 projects onto the lower wall portion 23 of the wall 20. In this way, in order to express the continuity or unity of the entire video, it is preferable for the control unit 51 to first generate a video with each part linked together and then divide it into each part. For example, the sense of unity of the entire video can be enhanced by unifying the color tone, adjusting the brightness balance, and synchronizing the timing of movement between each part.

[0044] On the other hand, Figure 4(b) shows an example of an image output from the projection device 30 and projected onto the wall 20. As shown in Figure 4(b), the projection device 30 outputs part A, which is projected onto the upper wall portion 22 of the wall 20, and part C, which is projected onto the lower wall portion 23 of the wall 20, as they are. However, part B, which corresponds to the opening 21 of the wall 20, is not output as is, but is replaced with part B', which represents a blank space (non-emitting portion) or black. This is to prevent double projection near the opening 21 and to ensure the visibility of the aerial image. In other words, in this embodiment, the projection device 30 projects a single image onto the wall 20 by combining part A, part B', and part C. Also, Figure 4(b) shows an example of an image output from the display device 11 and projected onto the wall 20 near the opening 21. As shown in Figure 4(c), the display device 11 extracts and displays only part B from the entire image generated by the control unit 51. Part B is the portion visible to the observer through the opening 21 in the wall 20, and is the most attention-grabbing central element of the entire video. Therefore, it is preferable to place the characteristic parts of the video content in Part B. Characteristic parts include, for example, virtual objects, text, or icon images that the observer would want to touch. Furthermore, since the video in Part B appears to float in the air, adding three-dimensional depth or movement that makes it appear to pop out can result in a more effective video expression.

[0045] Thus, it is preferable for the control unit 51 to divide a single video content into parts for wall projection (parts A and C) and part for aerial imaging (part B), and output them to the projection device 30 and the display device 11, respectively. As the observer observes the entire wall 20, as shown in Figures 1 and 2, for example, they will see an image in which parts A to C are connected. However, since part B, which is the most easily accessible to the observer, does not contain a physical object (wall 20), the observer can be made to feel as if the entire image is floating in the air. Furthermore, when the observer reaches out their hand to the opening 21 in the wall 20, their fingers are detected by the detection device 40, and the image changes. For example, various interactive video expressions are possible, such as virtual objects in the image deforming in response to the approach of a hand, virtual objects avoiding the hand, aerial imaging deforming in response to tapping movements with fingers, or the entire image scrolling in response to swiping movements. Therefore, the observer can be made to feel as if they are touching an intangible image and that the image has changed as a result of that contact.

[0046] For example, the image generation process by the control device 50 is performed in the following procedure. First, when the detection device 40 detects the proximity or movement of a real object (such as the observer's fingers), the control device 50 predicts the position of the real object after a predetermined time based on the detected position, direction of movement, and speed of movement of the real object. Next, it determines the relative relationship between the predicted position of the real object and the virtual object in the currently displayed image. Then, if it is predicted that the real object will approach the virtual object, it determines an action pattern for the virtual object to move away or avoid it for part or all of the image. At this time, considering the continuity of the image display, the action of the virtual object to move away or avoid it is executed smoothly over a certain period of time (for example, 0.5 seconds). By repeatedly performing the above process, natural image changes can be achieved. The action patterns for the virtual object to move away or avoid it can include the image moving up, down, left, or right, the image deforming, or the image splitting or merging.

[0047] Figures 1 and 2 show an example in which an image is projected from the projection device 30 onto a wall 20 that separates the space where the observer enters from the space where the aerial display device 10 is installed. However, in addition to projecting an image onto this wall 20, it is also possible to project an image from another projection device onto, for example, both or either of the left and right side walls shown in Figure 2.

[0048] In this specification, embodiments of the present invention have been described with reference to the drawings in order to express the content of the present invention. However, the present invention is not limited to the above embodiments, and includes modifications and improvements that are obvious to those skilled in the art based on the matters described in this specification. [Explanation of Symbols]

[0049] 10...Aerial display device 11...Display device 12…Aerial imaging member 13…Support 20...Wall 21...Opening 22...Top wall part 23...Bottom wall part 30…Projection device 40…Detection device 50...Control device 51...Control unit 52...Memory Unit 100...Aerial Display System P...Image plane

Claims

1. A display device that displays images, An aerial imaging member that refracts the image light incident from the display device and forms an image in the air, The aerial imaging member is provided with a wall having an opening formed within a range of 300 mm in front of and behind the imaging surface, At least a portion of the aforementioned video light is configured to pass through the opening in the wall. Aerial display system.

2. The wall has an upper wall portion and a lower wall portion, and the opening is formed between the upper wall portion and the lower wall portion. The aerial display system according to claim 1.

3. The system further comprises a projection device for projecting images onto parts of the wall other than the opening. The aerial display system according to claim 1.

4. The projection device is positioned on the same side as the display device and the aerial image forming member, with respect to the wall. At least a portion of the wall onto which the image light is projected from the projection device is configured to transmit the image light at least partially. The aerial display system according to claim 3.

5. The wall is formed such that at least a portion of the periphery of the opening is acutely angled. The aerial display system according to claim 1.

6. A detection device for detecting the presence or position of an object within a predetermined range in front of and behind the imaging surface of the aerial imaging member, The system further includes a control device that controls the image displayed by the display device according to the detection result of the detection device. The aerial display system according to claim 1.

7. The aerial imaging member is, The first group of reflective surfaces, It includes a second group of reflective surfaces having reflective surfaces in a direction substantially perpendicular to the first group of reflective surfaces, The device is configured to form an aerial image by sequentially reflecting the incident image light from the display device onto the first group of reflective surfaces and the second group of reflective surfaces. The aerial display system according to claim 1.

8. The process of a display device displaying an image, The process includes the aerial imaging member refracting the image light incident from the display device to form an image in the air, A wall with an opening is provided within a range of 300 mm in front of and behind the image-forming surface of the aerial imaging member. At least a portion of the aforementioned video light passes through the opening in the wall. A method for forming an image in mid-air.

Images