Game program using aerial image display device

Abstract

The invention provides a game program capable of utilizing the characteristics of an aerial image display device to the greatest extent. A game program executed by an aerial image display device is represented. The game program is characterized by comprising: a step for controlling an image system that projects an image into the air, and projecting an object to be operated into the air; and changing the motion of the object to be operated projected by the image system in accordance with the motion of the user's hand detected by a motion sensor provided in the aerial image display device, and in the step of changing the motion of the object to be operated, reflecting the influence of gravity on the motion of the object to be operated.

Description

Technical Field

[0001] The present invention relates to a game program that can be executed using an aerial image display device capable of displaying aerial images. Background Art

[0002] In recent years, technology that can indicate aerial images formed in the air using special optical elements in a non-contact manner has attracted much attention. By using this technology, a completely non-contact interface can be realized with the same ease of use as existing touch panels. The non-contact interface minimizes the risk of infection with pathogens such as viruses and bacteria.

[0003] For example, in Patent Document 1, since there is a hygiene problem when a surgeon touches a pointing device such as a computer mouse during surgery, a mouse on a non-contact remote pointer control device displayed using an aerial imaging technology is operated. As such a special optical element, there is an optical imaging device described in Patent Document 2, for example.

[0004] Prior art literature Patent Literature Patent Document 1: Japanese Patent Application Publication No. 2018-147054 Patent Document 2: International Publication No. 2009 / 131128 However, the current application of aerial image display devices is mainly limited to practical fields. The appearance of an image floating in an empty space through aerial imaging technology is impressive in itself, which helps to create a futuristic atmosphere. In addition, if it is displayed as an aerial image, it will appear from the environment, so it can also be felt in three dimensions.

[0005] Therefore, it is considered that the invention is not limited to the practical field, but is also effective in entertainment such as games, etc. In particular, it has a good application prospect in entertainment that utilizes the unique characteristics of the aerial image display device.

[0006] Therefore, an object of the present invention is to provide a game program that utilizes the feature of an aerial image display device that allows the user to directly touch an image appearing in the air with the hand. Summary of the invention

[0007] In order to solve the above-mentioned problem, a game program according to one aspect of the present invention is a game program executed by an aerial image display device, wherein the aerial image display device comprises: a computer; an image system, which is controlled by the computer to project an image into the air; and a motion sensor, which detects the movement of a user's hand performed near the aerial image projected by the image system and sends the detection result to the computer. The game program is characterized in that the computer is caused to execute: a step of controlling the image system to project an operated object into the air as an aerial image; and a step of changing the movement of the operated object projected by the image system according to the movement of the user's hand relative to the operated object detected by the motion sensor, wherein in the step of changing the movement of the operated object, the influence of gravity applied to the operated object is reflected in the movement of the operated object.

[0008] In addition, in one embodiment, it is characterized in that mass, center of gravity and moment of inertia are assigned to the operated object, and an interaction corresponding to the interference between the aerial image of the operated object and the user's hand is further set, and in the step of changing the movement of the operated object, based on these mass, center of gravity and moment of inertia, the influence of gravity and the interaction is reflected in the movement of the operated object.

[0009] Furthermore, in one embodiment, it is characterized in that the imaging system comprises a display device for displaying the operated object and an optical plate whose incident surface is obliquely facing the display device, and the image displayed on the display device is recursively transmitted through the optical plate to be imaged as the operated object.

[0010] According to the game program involved in the present invention, it is possible to provide unprecedented entertainment by making full use of the characteristics of the aerial image display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a perspective view showing an aerial image display device 1 for executing an embodiment of the game program of the present invention.

[0012] Figure 2 The aerial image display device 1 is shown along the Figure 1 Cross-sectional view along line AA.

[0013] Figure 3 This is a diagram for explaining an embodiment of the game program of the present invention, showing a situation where an egg T displayed in the air as an operation object is placed on the hand and balanced.

[0014] Figure 4This is a diagram for explaining an embodiment of the game program of the present invention, showing a situation in which an egg T displayed in the air as an operation object is placed on the hand and balanced in a state in which the wind blows to the left.

[0015] Figure 5 This is a diagram for explaining an embodiment of the game program of the present invention, showing a situation in which an egg T displayed in the air as an operation object is placed on the hand and balanced in a state in which the wind is blowing to the right.

[0016] Figure 6 This is a diagram for explaining an embodiment of the game program of the present invention, showing a situation where, when the score reaches a certain value or more, the egg T breaks and a character emerges from it.

[0017] Figure 7 1 is a diagram for explaining an embodiment of the game program of the present invention, and shows an example in which the egg T performs unexpected movements.

[0018] Figure 8 It is a diagram for explaining an embodiment of the game program of the present invention, showing a situation in which an egg T is balanced using both hands. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the game program of the present invention will be described with reference to the drawings. Figure 1 It is a perspective view showing an aerial image display device 1 according to an embodiment for executing a game program. Figure 2 The aerial image display device 1 is shown along the Figure 1 Cross-sectional view along line AA.

[0020] like Figure 1 and Figure 2 As shown, the aerial image display device 1 includes a liquid crystal display 10 and an optical plate 20 as indispensable components. In addition, the aerial image display device 1 also includes an information processing device 30, a speaker 40, etc. for controlling the liquid crystal display 10. Here, the liquid crystal display 10, the speaker 40, and the information processing device 30 are accommodated in the lower housing 12 and are connected by signal lines (omitted in the figure).

[0021] The information processing device 30 is essentially a small computer, which is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a storage device for storing various programs and data, and an input / output interface. As an input / output interface, for example, a USB port, a wireless LAN such as WIFI, etc. are installed. In addition, the information processing device 30 outputs an image signal to the liquid crystal display 10 to display the source of the aerial image. In addition, the information processing device 30 outputs a voice signal to the speaker 40 to generate a guiding voice and effect sound. The information processing device 30 can also use a commercially available general-purpose small personal computer, a general-purpose tablet computer, etc.

[0022] The liquid crystal display 10 is housed in a lower housing 12 with an open upper portion and a rectangular shape when viewed from above, and is supported in a substantially horizontal manner with the display screen facing upward. In addition, the optical plate 20 is embedded in the upper housing 14 in a manner such that the incident surface 21 faces downward and is obliquely opposite to the display screen of the liquid crystal display 10. Here, the optical plate 20 and the liquid crystal display 10 are fixed at an angle of approximately 45 degrees.

[0023] As such an optical plate 20, for example, a recursive transmission type optical imaging element (double-sided orthogonal reflector) described in Japanese Patent Application Laid-Open No. 2011-175297 can be used. The optical imaging element is realized by arranging a plurality of mutually orthogonal planar light reflecting parts at a certain interval. In addition, a structure such as a double-sided corner reflector having a reflecting surface formed on the side of a quadrilateral hole described in Patent No. 4900618 can also be used.

[0024] In addition, the upper housing 14 constituting the aerial image display device 1 can be easily separated from the lower housing 12. If the upper housing 14 is separated, maintenance and adjustment of the interior of the lower housing 12 can be easily performed, and the height during transportation can be reduced.

[0025] like Figure 2 As shown, the light from the display screen of the liquid crystal display 10 is incident on the optical plate 20, reflected twice inside the optical plate 20, and emitted to the other side. As a result, an aerial image G as a real image is formed in the space on the other side with the optical plate 20 as the object surface. In this case, in order to realize a clearer aerial image G, it is preferable not to add external light to the light from the liquid crystal display 10.

[0026] Furthermore, a three-dimensional motion sensor 7 composed of an infrared LED 72 and a pair of infrared cameras 74 is provided on the near front side of the lower housing 12. The motion sensor 7 can accurately detect the movement of the user's hand in three dimensions. The detection range of the motion sensor 7 is defined by the emission angle of the infrared LED 72 and the field of view of the infrared camera 74. Such a motion sensor 7 can be used as a commercial product, such as the Leap Motion Controller 2 of Ultraleap.

[0027] In particular, the three-dimensional motion sensor 7 is installed so that its tilt angle can be adjusted. Figure 2 The storage component is rotated within a certain angle range (here ±20 degrees) with the direction perpendicular to the paper plane of the cross-sectional view as the rotation axis. Here, the center of the detection range of the motion sensor 7 is adjusted to be offset more toward the user side than the center of the aerial image G. By adjusting in this way, the effective operation detection range can be maximized.

[0028] Although not shown here, in order to be able to view the aerial image at eye level, the aerial image display device 1 may be placed on a table or the like at an appropriate height. In particular, it is appropriate to provide a lifting device that can freely adjust the height. As such a lifting device, there is, for example, the electric lifting device Mario N sold by Yamato Metal Manufacturing Co., Ltd.

[0029] Usually, such an aerial image display device is used as a transactional input device such as an acceptance system. That is, if the non-contact interface screen is displayed as an aerial image G, the non-contact interface screen can be operated by gestures such as touching with a finger. The operation is detected by the motion sensor 7 composed of an infrared LED 72 and an infrared camera 74, and the corresponding operation signal is sent to the information processing device 30 for prescribed processing. As an operation interface, for example, controls such as buttons, check boxes, and drop-down menus are included. Therefore, a completely non-contact interface can be realized with the same ease of use as an existing touch panel.

[0030] In the present invention, the aerial image display device is used for entertainment, particularly games. Here, an object as an object to be operated is displayed in the air, and the mass, center of gravity and moment of inertia of the object are set. The movement and rotation of the object are determined by simulation based on these mass, center of gravity and moment of inertia.

[0031] As an external force acting on an object, in addition to gravity, there is also interaction with the user's hand. This interaction is obtained by simulating the external force that may be received from the user's hand when the object exists in reality.

[0032] In the following examples, Figure 3As shown, a game is installed in which an egg T displayed in the air as an operated object is placed on the hand and balanced. First, an aerial image of an upright egg T is displayed. The user gently touches the bottom of the displayed egg T with his hand. Thus, the game starts, and the egg T falls due to the influence of gravity, but if there is a hand there, the egg T will be supported by the hand. In this case, the egg T on the hand will tilt with the contact point with the hand as a fulcrum due to the influence of gravity and fall from the hand. If it falls, it will break and the game will end. Therefore, by moving the hand left and right, the angle of the egg T is made vertical and balanced, so that the egg T will not fall (refer to Figure 4 ).

[0033] That is, the movement of the egg T is simulated as an inverted pendulum with the user's hand as a fulcrum. However, since the aerial image is a plane in this embodiment, the movement of the egg T is limited to the display plane of the aerial image, and the rotation of the egg T is limited to the direction perpendicular to the display plane of the aerial image as the rotation axis. Therefore, the degree of freedom is 3. As a result, the amount of calculation for the simulation is greatly reduced. In this case, the user achieves balance by moving his hand left and right. The simulation is performed based on a well-known mechanical model as an inverted pendulum.

[0034] Specifically, based on the set mass, center of gravity, and moment of inertia, the movement and rotation of the egg T generated by the interaction corresponding to the interference between the aerial image of the egg T and the user's hand are simulated. The interaction between the user's hand and the aerial image of the egg T also considers the friction coefficient between the user's hand and the egg T, and is defined in such a way that a force corresponding to the movement of the user's hand that contacts the aerial image of the egg T is applied to the egg T at the contact position.

[0035] Here, in order to increase the difficulty of the game, wind is simulated. For example, if the wind blows to the left, the user needs to balance the egg T slightly tilted to the right (see Figure 4 Likewise, if the wind is blowing to the right, the user needs to balance the egg tart slightly to the left (see Figure 5 ). If the wind stops, you need to find a balance in the middle. By changing the wind speed and the degree of change, you can try various levels of difficulty. In addition, in order to visually see the direction of the wind, you can add visual effects such as flowing fog (see Figure 4 and Figure 5 ).

[0036] When the difficulty is high, the wind speed is high and the wind direction changes randomly. The score is accumulated according to the set difficulty and the time it takes for the egg T to be balanced. The score is displayed in the middle of the egg T. Figure 4 and Figure 5 The score is 7. If the score is above a certain value, the egg T will break and the character will emerge from it (see Figure 6). This character is a unique character that varies depending on the score, and can be output from a card or a 3D printer.

[0037] You can also increase the difficulty by making the egg T do unexpected actions. For example, make the egg T suddenly jump (see Figure 7 ). The user must catch it well. The hand drops slightly in coordination with the falling of the egg tart, and if it is not caught well, it will break.

[0038] In addition, if you are a beginner and have difficulty balancing, you can use your left hand to assist. For example, if you are about to fall to the left, use your left hand to gently touch the left side of the egg tart to balance (see Figure 8 ).

[0039] Industrial Applicability According to the game program involved in the present invention, it is possible to make full use of the characteristics of the aerial image display device having an unprecedented interface.

[0040] The above describes the aerial image display device according to the present invention based on the implementation mode, but the present invention is not limited thereto and can be modified within the scope not departing from the main purpose of the present invention. If possible, the technologies described in each implementation mode or known technologies can be combined.

[0041] Here, the information processing device 30 is installed in the aerial image display device 1, but the present invention is not limited to this. For example, the information processing device 30 may be omitted, and an external input terminal for inputting an image signal to the liquid crystal display 10 and an external voice input terminal for inputting a voice signal to a speaker may be provided in the aerial image display device 1, and necessary image signals and voice signals may be supplied from an external computer or the like.

[0042] Furthermore, in the above-described embodiment, the operation detection unit is composed of the infrared LED 72 and the infrared camera 74, but the present invention is not limited thereto. For example, the operation detection unit may also be composed of the visible light LED 72 and a plurality of visible light cameras.

[0043] Furthermore, in the above-mentioned embodiment, an aerial image display device using a recursive transmission type optical imaging element is adopted. However, the present invention is not limited thereto, and for example, an aerial image display device using a recursive reflection type optical imaging element may also be adopted.

[0044] Furthermore, in the above-mentioned embodiment, a liquid crystal display is used, but the present invention is not limited to this, and an organic EL display or electronic paper with a backlight may also be used.

[0045] Furthermore, in the above-mentioned embodiment, a three-dimensional motion sensor composed of an infrared LED and an infrared camera is used, but the present invention is not limited to this, and a normal camera that detects visible light can also be used for tracking.

[0046] Description of Reference Numerals 1 Aerial image display device 7 Motion Sensor 10 LCD display 12 Lower housing 14 Upper shell 20 Optical Plate 21 Incident surface 30 Information processing device 40 Speakers 72 Infrared LEDs 74 Infrared Camera

Claims

1. A game program executed by an aerial image display device, the aerial image display device comprising: a computer; an image system controlled by the computer to project an image into the air; and a motion sensor to detect a hand movement of a user near the aerial image projected by the image system and to send the detection result to the computer, wherein the game program causes the computer to execute: The step of controlling the imaging system to project the operated object into the air as an aerial image; and The step of changing the motion of the operated object projected by the imaging system according to the motion of the user's hand relative to the operated object detected by the motion sensor, In the step of changing the movement of the operated object, the influence of gravity applied to the operated object is reflected in the movement of the operated object.

2. The game program according to claim 1, characterized in that: Mass, center of gravity and moment of inertia are assigned to the operated object, and an interaction corresponding to the interference between the aerial image of the operated object and the user's hand is further set. In the step of changing the movement of the operated object, the influence of gravity and the interaction is reflected in the movement of the operated object based on these mass, center of gravity and moment of inertia.

3. The game program according to claim 2, characterized in that: The image system includes a display device for displaying the operated object and an optical plate whose incident surface faces the display device at an angle. The image displayed on the display device is recursively transmitted through the optical plate to form an image of the operated object.

Images