Simulation Equipment

Abstract

To achieve both of improvement in accuracy and reduction in cost of a simulation device which synchronizes a motion of a virtual tool used in a virtual space to the motion of a tool used by a user in a real space.SOLUTION: The above-mentioned problem is solved by a simulation device comprising: tool position detection means which is provided at two spots of a tool used by a user in a real space to detect position information of the tool; tool posture calculation means which calculates posture information of the tool on the basis of the position information on the two spots of the tool; attachment object detection means which is provided in an attachment object attached to the head of the user to detect position information and posture information of a head of the user; locus information calculation means which calculates locus information of a virtual tool used by the user in a virtual space on the basis of the position information of the tool and the posture information of the tool; and three-dimensional image control means which causes a display unit to display a three-dimensional image including the virtual tool viewed from a virtual view point position of the virtual space corresponding to the position information and posture information of the head of the user on the basis of the locus information of the virtual tool.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a simulation device. [Background technology]

[0002] For example, as an example of a technique for analyzing a swing of a ball-hitting tool, a swing analysis system and a swing analysis method capable of displaying a swing trajectory with high accuracy in a virtual space are known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-187501 A Summary of the Invention [Problem to be solved by the invention]

[0004] The swing analysis system of Patent Document 1 includes a swing analysis device, a sensor device, an imaging system, a head mounted display (HMD), and a marker. The sensor device is attached to the grip end of the bat and detects acceleration data and angular velocity data of the grip end. The imaging system is an optical motion capture system, and captures an image of a marker attached to the head of the bat to obtain position information and posture information of the head.

[0005] In the conventional swing analysis system, it is desirable to detect the swing trajectory with high accuracy using a simple tracking system. However, in the swing analysis system using the simple tracking system, there are cases where the swing trajectory cannot be detected with high accuracy when the swing speed becomes high.

[0006] The above problem is not limited to bat swing analysis, but also occurs in a simulation device that synchronizes the movement of a virtual tool used by a user in a virtual space with the movement of the tool used by the user in a real space.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to achieve both improved accuracy and cost reduction in a simulation device that synchronizes the movement of a virtual tool used in a virtual space with the movement of a tool used by a user in real space. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the present invention provides a method for controlling a user's movement by using a tool in real space, the method comprising: and posture information The way to detect Tool inspection An output means, The tool detection means detects Position information of at least two locations of the tool A line connecting a tool posture calculation means for calculating posture information of the tool based on the above-mentioned, and a wearable device detection means for detecting position information and posture information of the user's head, the wearable device being provided on a wearable device worn on the user's head; The tool detection means detects Location information of the tool; The tool attitude calculation means calculates The virtual tool is characterized in that it comprises: trajectory information calculation means for calculating trajectory information of a virtual tool handled by the user in the virtual space based on the tool orientation information; and three-dimensional image control means for causing a display unit to display a three-dimensional image including the virtual tool viewed from a virtual viewpoint position in the virtual space corresponding to position information and orientation information of the user's head based on the trajectory information of the virtual tool. Effect of the Invention

[0009] According to the present invention, it is possible to improve the accuracy and reduce the cost of a simulation device that synchronizes the movement of a virtual tool used in a virtual space with the movement of a tool used by a user in a real space. [Brief description of the drawings]

[0010] [Figure 1]1 is a schematic diagram of an example of a batting simulator according to an embodiment of the present invention. [Diagram 2] 1 is a schematic diagram showing an example of the arrangement of a three-sided display unit and a projector in an example of a batting simulator according to the present embodiment. [Diagram 3] 1 is a schematic diagram showing an example of a stereoscopic image for a user at bat on the left side and a stereoscopic image for a user at bat on the right side. FIG. [Figure 4] 1 is a system configuration diagram showing an example of a batting simulator according to an embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 6] FIG. 2 is a functional configuration diagram of an example of a batting simulator according to the present embodiment. [Figure 7] 11 is a flowchart illustrating an example of a processing procedure of the batting simulator according to the present embodiment. [Figure 8] 13 is a flowchart of an example of a processing procedure for calculating trajectory information of a bat object in a virtual space based on position information and attitude information of a dedicated bat. [Figure 9] FIG. 1 is an image diagram of an example of a swing trajectory of a virtual bat displayed three-dimensionally on a three-sided display unit. [Figure 10] 1 is a schematic diagram of an example of a batting simulator according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Next, an embodiment of the present invention will be described in detail. In this embodiment, a batting simulator will be described as an example of a simulation device that synchronizes the movement of a tool handled by a user in a real space with the movement of a virtual tool handled in a virtual space. In the batting simulator, a user holds a dedicated bat and swings it in the real space, and can then swing a virtual bat in the virtual space that is synchronized with the movement of the dedicated bat.

[0012] [First embodiment] <Summary> Fig. 1 is a schematic diagram of an example of a batting simulator according to the present embodiment. Fig. 2 is a schematic diagram showing an example of the arrangement of a three-sided display unit and a projector in an example of a batting simulator according to the present embodiment. Fig. 2(A) shows a schematic diagram of the batting simulator as viewed from above. Fig. 2(B) shows a schematic diagram of the batting simulator as viewed from the front. Fig. 2(C) shows a schematic diagram of the batting simulator as viewed from the side.

[0013] 1 and 2, the batting simulator has a three-sided display unit 10, a floor projector 21, a side projector 22, and a front projector 23. A user P uses the batting simulator while wearing liquid crystal shutter glasses 41, a helmet (not shown) on his / her head, and holding a dedicated bat 42.

[0014] The three-surface display unit 10 has a floor display unit 11, wall display units (a first wall display unit 12 and a second wall display unit 13), and first to third curved units 14 to 16. The floor display unit 11 is, for example, square or rectangular in shape. The first wall display unit 12 and the second wall display unit 13 are, for example, trapezoidal in shape with an upper side longer than a lower side. The floor display unit 11, the first wall display unit 12, and the second wall display unit 13 each have a substantially flat surface.

[0015] 1 and 2, the first wall display unit 12 and the second wall display unit 13 form an angle of, for example, about 90 degrees. In addition, the angle formed by the floor display unit 11 and the first wall display unit 12, and the angle formed by the floor display unit 11 and the second wall display unit 13 are preferably obtuse angles, for example, about 94 degrees.

[0016] The floor display unit 11, the first wall display unit 12, and the second wall display unit 13 are connected to each other via a first curved portion 14, a second curved portion 15, and a third curved portion 16. For example, the floor display unit 11 and the first wall display unit 12 are smoothly connected to each other via a first curved portion 14 having a predetermined radius of curvature. The floor display unit 11 and the second wall display unit 13 are smoothly connected to each other via a second curved portion 15 having a predetermined radius of curvature. In addition, the first wall display unit 12 and the second wall display unit 13 are smoothly connected to each other via a third curved portion 16 having a predetermined radius of curvature.

[0017] The user P can stand at any position on the floor display unit 11. In this embodiment, the space on the floor display unit 11 is called the real space. The position and direction of the real space in this embodiment are expressed using Cartesian coordinates (XpYpZp coordinates) with the origin PO being a position near a corner of the floor display unit 11 (a corner facing the first wall display unit 12 and the second wall display unit 13).

[0018] When viewed from the viewpoint of user P on floor display unit 11, for example, floor display unit 11 is located downward (-Zp direction). First wall display unit 12 is located on the front right side (+Xp direction) of user P. Second wall display unit 13 is located on the front left side (+Yp direction) of user P.

[0019] A swing area SR is provided on the floor display unit 11. The swing area SR is spaced a predetermined distance from both the first wall display unit 12 and the second wall display unit 13 so that the user can safely swing the dedicated bat 42 without worrying about hitting the first wall display unit 12 and the second wall display unit 13.

[0020] The floor projector 21, the side projector 22, and the front projector 23 are devices for projecting a stereoscopic image (stereoscopic video) onto the three-screen display unit 10. In this embodiment, an example will be described in which a time-division method is used as a display method for stereoscopic images. The floor projector 21, the side projector 22, and the front projector 23 of the time-division method alternately project images for the right eye and the left eye that are generated separately onto the three-screen display unit 10 in a time-division manner. The floor projector 21, the side projector 22, and the front projector 23 of the time-division method are sometimes called 3D projectors of an active shutter method.

[0021] In the time-division method, liquid crystal shutter glasses 41 are used so that the user can alternately view with the right and left eyes the right-eye and left-eye images projected on the three-screen display unit 10 in a time-division manner by the floor projector 21, the side projector 22, and the front projector 23. The liquid crystal shutter glasses 41 alternately open and close the user's field of view so that the left eye's field of view is closed when the right-eye image is displayed on the three-screen display unit 10, and the right eye's field of view is closed when the left-eye image is displayed on the three-screen display unit 10. The display method for the stereoscopic images can use existing technology, and other methods such as a circular polarization method, a linear polarization method, and a frame sequential method may also be used.

[0022] The floor projector 21 is a projector for the floor display unit 11, and is disposed above the first wall display unit 12. An image from the floor projector 21 is projected onto the floor display unit 11, the first curved portion 14, and the second curved portion 15. The side projector 22 is a projector for the first wall display unit 12, and is disposed near the ceiling at a position sufficiently distant from the first wall display unit 12 in the X direction. An image from the side projector 22 is projected obliquely downward onto the first wall display unit 12, the first curved portion 14, and the third curved portion 16. The front projector 23 is a projector for the second wall display unit 13, and is disposed near the ceiling at a position sufficiently distant from the second wall display unit 13 in the Y direction. An image from the front projector 23 is projected obliquely downward onto the second wall display unit 13, the second curved portion 15, and the third curved portion 16.

[0023] The images projected on the three-sided display unit 10 by the floor projector 21, the side projector 22, and the front projector 23 are images for the right eye and the left eye that are alternately projected in a time-division manner. By viewing the images for the right eye and the left eye that are alternately projected in a time-division manner on the three-sided display unit 10 through the liquid crystal shutter glasses 41 that the user P wears, the user P on the floor display unit 11 can recognize the images as three-dimensional.

[0024] The batting simulator according to this embodiment can accommodate both a user P who uses the left batting box and a user P who uses the right batting box, as shown in Fig. 3. Fig. 3 is a schematic diagram showing an example of a stereoscopic image for a user who uses the left batting box and a stereoscopic image for a user who uses the right batting box.

[0025] Fig. 3(A) shows an example of a stereoscopic image displayed on the three-sided display unit 10 when user P bats in the left batter's box. The position of the left batter's box is set so that user P is safe when swinging his dedicated bat 42. For example, when expressed in coordinates, Fig. 3(A) shows that pitcher image E1 is displayed on the first wall display unit 12 located in the Xp direction, and ball image E2 moves along the Xp coordinate toward home plate image E5H.

[0026] Fig. 3(B) shows an example of a stereoscopic image displayed on the three-sided display unit 10 when the user P uses the right bat's box. The position of the right bat's box is set so that the user P is safe when swinging the dedicated bat 42. For example, when expressed in coordinates, Fig. 3(B) shows that the pitcher image E1 is displayed on the second wall display unit 13 located in the Yp direction, and the ball image E2 moves along the Yp coordinate toward the home plate image E5H. In Figs. 3(A) and 3(B), the home plate image E5H displayed on the floor display unit 11 is changed by 90 degrees in plan view.

[0027] Note that Figures 1 to 3 omit the configuration for user P to operate the batting simulator of this embodiment, the configuration for collecting the usage fee for the batting simulator from user P, the configuration for detecting user P's line of sight, the configuration for detecting the swing trajectory of the dedicated bat 42, the configuration for controlling the stereoscopic image displayed on the three-sided display unit 10, and the configuration for determining the batting result, but the configuration is as shown in Figure 4.

[0028] <System configuration> Fig. 4 is a system configuration diagram showing an example of a batting simulator according to this embodiment. The batting simulator 1 shown in Fig. 4 has a floor projector 21, a side projector 22, a front projector 23, liquid crystal shutter glasses 41, a dedicated bat 42, a helmet 43, an operation unit 100, a screen display unit 110, a coin insertion unit 120, an IC card unit 130, a computer 150, a transmitter 190 for liquid crystal shutter glasses, an infrared sensor 210, tracker communicators 220a to 220c, trackers 230a to 230b attached to the dedicated bat 42, and a tracker 230c attached to the helmet 43.

[0029] The batting simulator 1 includes a computer 150. The computer 150 is, for example, a PC (Personal Computer) or the like, and may be a single PC or a plurality of PCs that perform processing in cooperation with each other.

[0030] The computer 150 is connected to a floor projector 21, a side projector 22, a front projector 23, an operation unit 100, a screen display unit 110, a coin insertion unit 120, an IC card unit 130, a liquid crystal shutter glasses transmitter 190, an infrared sensor 210, and tracker communications devices 220a to 220c so as to be capable of data communication.

[0031] The operation unit 100 is one of the devices that functions as an interface for the user P to perform operations. The operation unit 100 has input devices such as buttons and a touch panel. The screen display unit 110 is one of the devices that functions as an interface for the user P to perform operations. The screen display unit 110 has a display device such as a liquid crystal monitor and a speaker that outputs sound.

[0032] Coin insertion unit 120 is an example of a fee collection device that collects a fee for using the batting simulator from user P. IC card unit 130 includes, for example, an IC card reader / writer that reads and writes information from an IC card, which is an information storage medium. IC card unit 130 can read identification information that can identify user P, such as a card ID, from user P's IC card. The identification information read from user P's IC card can be used, for example, for electronic payment or for storing the results of simulation by the batting simulator.

[0033] Floor projector 21, side projector 22, and front projector 23 display stereoscopic images on three-surface display unit 10 under the control of computer 150. Liquid crystal shutter glasses transmitter 190 is connected to liquid crystal shutter glasses 41 so as to be able to wirelessly communicate with them. Liquid crystal shutter glasses transmitter 190 opens and closes the left and right lenses of liquid crystal shutter glasses 41 alternately under the control of computer 150.

[0034] The left and right lenses of the liquid crystal shutter glasses 41 are controlled to open and close so as to close the field of view of the left eye of the user P when an image for the right eye is displayed on the three-sided display unit 10, and close the field of view of the right eye of the user P when an image for the left eye is displayed on the three-sided display unit 10.

[0035] A dedicated bat 42 used in the batting simulator 1 according to this embodiment has a tracker 230a attached to the tip and a tracker 230b attached to the grip end. Furthermore, a helmet 43 worn on the head of a user P has a tracker 230c attached to the apex. Note that the manner in which trackers 230a and 230b are attached to the dedicated bat 42 shown in the figure is one example. Furthermore, the manner in which tracker 230c is attached to the helmet 43 shown in the figure is also one example.

[0036] An infrared marker (not shown) is attached to each of trackers 230a to 230c. Infrared sensor 210 is installed in a position where it can capture an image of dedicated bat 42 and helmet 43 being used by user P. Infrared sensor 210 detects the position of the infrared marker in real space by analyzing the captured image, and notifies computer 150 of the detected position information. The position information of trackers 230a to 230c notified by infrared sensor 210 indicates the positions of the tip and grip end of dedicated bat 42, and the top part of helmet 43 in real space.

[0037] In addition, the tracker communicators 220a to 220c are installed at positions where they can wirelessly communicate with the dedicated bat 42 and helmet 43 being used by the user P. For example, the tracker communicator 220a receives, from the tracker 230a attached to the tip of the dedicated bat 42, position information and attitude information of the tip of the dedicated bat 42 detected by the tracker 230a, and notifies the computer 150. The tracker communicator 220b receives, from the tracker 230b attached to the grip end of the dedicated bat 42, position information and attitude information of the grip end of the dedicated bat 42 detected by the tracker 230b, and notifies the computer 150. The tracker communicator 220c receives, from the tracker 230c attached to the tip of the helmet 43, position information and attitude information of the apex of the helmet 43 detected by the tracker 230c, and notifies the computer 150.

[0038] The infrared sensor 210, the tracker communications devices 220a to 220c, and the trackers 230a to 230c may use a tracking system such as a base station, dongle, and tracker of HTC VIVE (registered trademark).

[0039] As described above, in the batting simulator 1 according to this embodiment, an existing tracking system is utilized to obtain position information of the dedicated bat 42 and position information of the helmet 43, so that the cost of a simulation device that synchronizes the movement of a tool such as the dedicated bat 42 that the user P uses in real space with the movement of a virtual bat or the like used in virtual space can be reduced.

[0040] The computer 150 in Fig. 4 is realized by, for example, a computer having the hardware configuration shown in Fig. 5. Fig. 5 is a diagram showing the hardware configuration of an example of a computer according to this embodiment.

[0041] 5 includes an input device 501, an output device 502, an external I / F 503, a RAM 504, a ROM 505, a CPU 506, a communication I / F 507, and a HDD 508, all of which are connected to each other via a bus B. The input device 501 and the output device 502 may be connected and used when necessary.

[0042] The input device 501 is a touch panel, operation keys or buttons, keyboard, mouse, etc. that the user uses to input various signals. The output device 502 is composed of a display such as a liquid crystal or organic electroluminescence display for displaying a screen, and a speaker for outputting sound data such as voice and music. The communication I / F 507 is an interface that connects the computer to a network. This allows the computer to perform data communication via the communication I / F 507.

[0043] The HDD 508 is an example of a non-volatile storage device that stores programs and data. The stored programs and data include an OS, which is basic software that controls the entire computer, and applications that provide various functions on the OS. Instead of the HDD 508, the computer may use a drive device that uses a flash memory as a storage medium (for example, a solid-state drive: SSD).

[0044] The external I / F 503 is an interface with an external device. The external device may include a recording medium 503a. This allows the computer 500 to read and / or write data from and to the recording medium 503a via the external I / F 503. The recording medium 503a may include a flexible disk, a CD, a DVD, an SD memory card, a USB memory, and the like.

[0045] The ROM 505 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 505 stores programs and data such as the BIOS, OS settings, and network settings that are executed when the computer starts up. The RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily retains programs and data.

[0046] The CPU 506 is a calculation device that realizes the control and functions of the entire computer by reading programs and data from storage devices such as the ROM 505 and the HDD 508 onto the RAM 504 and executing processing (including processing for generating images, sounds, etc. to be output to the output device 502).

[0047] The batting simulator 1 according to this embodiment can realize various functions, for example, as shown in FIG. 6, by executing a program on the computer 150 having the above-mentioned hardware configuration.

[0048] <Functional configuration> The batting simulator 1 according to this embodiment realizes, for example, the functional configuration of Fig. 6 by executing a program in a computer 150. Fig. 6 is a diagram showing an example of the functional configuration of the batting simulator according to this embodiment.

[0049] Batting simulator 1 has a virtual model setting unit 50, a turn at bat selection unit 52, a viewpoint position calculation unit 54, a line of sight direction setting unit 56, a stereoscopic image control unit 58, a batting result determination unit 60, a bat detection unit 62, and a helmet detection unit 64. Bat detection unit 62 has a bat position detection unit 70, a bat posture calculation unit 72, and a trajectory information calculation unit 74.

[0050] The virtual model setting unit 50 places various objects in a three-dimensional virtual space. The various objects to be placed in the three-dimensional virtual space include objects corresponding to various things such as people, objects, or facilities that exist in a real baseball stadium. These objects include objects that can be moved or operated (e.g., a pitcher object, a ball object, a bat object, etc.) and objects that cannot be moved or operated (e.g., a pitching plate object, left and right batter's box objects, a home plate object, an outfield fence object, etc.).

[0051] The batting simulator 1 according to this embodiment sets a virtual model that reproduces the appearance of a pitcher pitching in a virtual space by moving a pitcher object, which is an example of an object that can be moved. The virtual model setting unit 50 stores shape data for a number of objects. The shape data includes data on the three-dimensional shape of the object and data on its pattern and color (texture data).

[0052] Furthermore, motion data of objects that can be moved or operated is stored in the virtual model setting unit 50. For example, in the batting simulator 1 according to this embodiment, pitching motion data for operating a pitcher object and ball motion data for moving a ball object are stored as motion data.

[0053] For example, the pitcher motion data defines the posture of the pitcher object in a time series at each predetermined time from the start of the pitching motion of the pitcher object. The ball motion data defines the position coordinates of the ball object in the virtual space at each predetermined time from the start of the movement of the ball object. The ball motion data is created, for example, by converting a pitch (the path and speed of the ball) of a real pitcher into data. By moving the ball object in accordance with the ball motion data, the ball object moves in the virtual space along the same path and at the same speed as a ball pitched by a real pitcher.

[0054] The virtual model setting unit 50 places a pitcher object near the pitching plate object and causes the pitcher object to pitch in accordance with the pitcher motion data. The virtual model setting unit 50 also moves the ball object in accordance with the ball motion data. The virtual model setting unit 50 synchronizes the pitching motion of the pitcher object with the movement of the ball object, and causes the ball object to start moving at the timing when the pitcher object swings his / her hand down.

[0055] The turn to bat selection unit 52 selects the left or right turn to bat in response to the operation of the user P on the operation unit 100. The viewpoint position calculation unit 54 calculates the viewpoint position (left and right virtual viewpoint positions) of the user P in the virtual space based on the position information of the vertex part of the helmet 43 detected by the infrared sensor 210 and the position information and attitude information of the vertex part of the helmet 43 detected by the tracker communication device 220c. Note that the method of calculating the left and right virtual viewpoint positions can use existing technology.

[0056] The line of sight direction setting unit 56 sets the three line of sight directions of the user P required for generating a stereoscopic image according to the turn at bat selected by the turn at bat selection unit 52. Note that existing technology can be used as a method for setting the three line of sight directions of the user P required for generating a stereoscopic image.

[0057] The stereoscopic image control unit 58 generates six images by rendering images obtained when the virtual model is viewed in the three line-of-sight directions of the user P set by the line-of-sight direction setting unit 56 from the left and right virtual viewpoint positions calculated by the viewpoint position calculation unit 54. For example, the stereoscopic image control unit 58 generates three images obtained when the directions of the floor display unit 11, the first wall display unit 12, and the second wall display unit 13 are viewed from the left virtual viewpoint position as images for the left eye. The stereoscopic image control unit 58 also generates three images obtained when the directions of the floor display unit 11, the first wall display unit 12, and the second wall display unit 13 are viewed from the right virtual viewpoint position as images for the right eye.

[0058] The stereoscopic image control unit 58 controls the floor projector 21, the side projector 22, and the front projector 23 to alternately project the separately generated right-eye and left-eye images in a time-division manner. The stereoscopic image control unit 58 also performs opening and closing control so as to close the field of view of the left eye of the user P when the right-eye image is displayed on the three-sided display unit 10, and close the field of view of the right eye of the user P when the left-eye image is displayed on the three-sided display unit 10.

[0059] Therefore, by viewing the images for the right eye and the left eye that are alternately displayed in a time-division manner on the three-surface display unit 10 through the liquid crystal shutter glasses 41 that the user P is wearing, the user can recognize the images projected on the three-surface display unit 10 as three-dimensional.

[0060] 3, an image corresponding to a ball object is displayed three-dimensionally as a ball image E2. For example, the ball image E2 is displayed three-dimensionally in response to the movement of the ball object in the virtual space, moving from the position of the pitcher image E1 through a position where the user P can hit the ball with the dedicated bat 42 to the rear of the home plate image E5H. The user P can recognize that the ball image E2 is moving along a path corresponding to the path of movement of the ball object in the virtual space.

[0061] Therefore, the user P can swing the special bat 42 by matching the timing and position with the moving ball image E2, and can practice batting with the same feeling as batting practice using a real baseball.

[0062] When the user P swings the dedicated bat 42 at the incoming ball image E2, the hitting result determination unit 60 determines the hitting result based on the position (movement path) of the ball object in the virtual space and the position (movement path) of the bat object (virtual bat) in the virtual space detected by the bat detection unit 62 described below.

[0063] For example, if the distance when the ball object and the bat object are closest to each other in the virtual space is less than a certain value, the hitting result determination unit 60 determines that the ball object and the bat object have collided (collided), and if the distance is equal to or greater than the certain value, the hitting result determination unit 60 determines that the ball object and the bat object have collided. If it is determined that the ball object and the bat object have collided, the hitting result determination unit 60 determines the direction of the hit ball, base hit information, etc., based on the circumstances of the collision between the ball object and the bat object (ball speed, swing speed, collision angle, collision position of the bat object, rotation direction and rotation speed of the ball object, etc.). For example, the hitting result determined by the hitting result determination unit 60 is displayed on the three-sided display unit 10 under the control of the stereoscopic image control unit 58.

[0064] The bat detection unit 62 detects the movement of the dedicated bat 42 swung by the user P in real space. The bat position detection unit 70 detects position information of the tip of the dedicated bat 42 using the tracker 230a. In addition, the bat position detection unit 70 detects position information of the grip end of the dedicated bat 42 using the tracker 230b. Note that the bat position detection unit 70 may use position information of the infrared markers of the trackers 230a to 230b notified by the infrared sensor 210 to detect the positions of the tip and grip end of the dedicated bat 42.

[0065] The bat posture calculation unit 72 calculates posture information of the dedicated bat 42 based on position information of the tip and grip end of the dedicated bat 42 detected by the bat position detection unit 70. The bat posture calculation unit 72 calculates posture information of the dedicated bat 42 based on a line connecting the position of the tip and the position of the grip end of the dedicated bat 42 detected by the bat position detection unit 70.

[0066] In this way, the batting simulator 1 according to this embodiment does not use posture information of the dedicated bat 42 detected by the tracker 230a and tracker 230b attached to the dedicated bat 42, but instead uses posture information calculated from position information of the tip and grip end of the dedicated bat 42.

[0067] The reason for this is that data analysis has revealed that the accuracy of posture information of the dedicated bat 42 detected from the dedicated bat 42 with a high swing speed (e.g., 90 km / h or more) may be impaired. Note that the accuracy of the position information of the dedicated bat 42 detected from the dedicated bat 42 with a high swing speed (e.g., 90 km / h or more) was not impaired compared to the posture information.

[0068] Therefore, in the batting simulator 1 according to this embodiment, the posture information of the dedicated bat 42 detected by the tracker 230a and the tracker 230b attached to the dedicated bat 42 is not used, but posture information calculated from the position information of the tip and the grip end of the dedicated bat 42 is used.

[0069] The trajectory information calculation unit 74 calculates trajectory information of a bat object (a virtual bat handled in the virtual space by the user P) in the virtual space based on the position information of the tip and grip end of the dedicated bat 42 detected by the bat position detection unit 70 and the posture information of the dedicated bat 42 calculated by the bat posture calculation unit 72. The trajectory information calculation unit 74 smoothes the calculated trajectory information of the bat object using a Kalman filter. Smoothing using a Kalman filter is an example of a method of smoothing time-series measurement values.

[0070] The helmet detection unit 64 detects the movement of the helmet 43 worn on the head of the user P in real space. The helmet detection unit 64 detects position information and orientation information of the apex of the helmet 43 by using the tracker 230c.

[0071] <Batting simulator processing> 7 is a flowchart of an example of a processing procedure of the batting simulator according to this embodiment. In step S10, the batting simulator 1 accepts a selection of information necessary for the batting simulation based on an operation of the user P on the operation unit 100. For example, the user P selects an at bat turn (right or left), an opposing pitcher, and a difficulty level that is differentiated by the type of ball, ball speed, etc.

[0072] In step S12, the batting simulator 1 accepts a start instruction from the user P and starts playing. The user P puts on the liquid crystal shutter glasses 41, wears the helmet 43 on his / her head, and stands at bat with the dedicated bat 42 in hand.

[0073] In step S14, the batting simulator 1 displays a pitching video that the user P can recognize as a three-dimensional object on the three-sided display unit 10. For example, the batting simulator 1 displays a pitching image of a pitcher object performing a pitching motion on the three-sided display unit 10. The batting simulator 1 also displays on the three-sided display unit 10 a ball video corresponding to the ball object, timing with the pitching motion, moving from the position of the pitcher video, through a position where the user P can hit the ball with the dedicated bat 42, to the rear of the home plate video.

[0074] The user P swings the dedicated bat 42 in synchronization with the timing and position of the moving ball image. In step S16, the batting simulator 1 calculates trajectory information of a bat object in the virtual space (a virtual bat swung in the virtual space in synchronization with the swing of the dedicated bat 42 by the user P) based on the position information and posture information of the dedicated bat 42.

[0075] In step S18, the batting simulator 1 determines whether or not the ball object and the bat object collide in the virtual space based on the trajectory information representing the position (movement path) of the ball object and the position (movement path) of the bat object in the virtual space. Furthermore, when the batting simulator 1 determines that the ball object and the bat object collide in the virtual space, it determines the direction of the hit ball, base hit information, etc. based on the circumstances (ball speed, swing speed, collision angle, etc.) at which the ball object and the bat object collide.

[0076] In step S20, the batting simulator 1 determines whether or not a ball object, which is a virtual ball, has been hit with a bat object, which is a virtual bat. If it is determined that the bat object and the ball object have collided in the virtual space, the batting simulator 1 determines that the ball object, which is a virtual ball, has been hit with the bat object, which is a virtual bat.

[0077] If it is determined that the virtual ball has been hit with the virtual bat, batting simulator 1 proceeds to step S22 and displays a stereoscopic image of the ball hit with the virtual bat on tri-surface display unit 10 based on the direction of the hit ball and the base hit information of step S18. If it is determined that the virtual ball has not been hit with the virtual bat, batting simulator 1 determines that it has been a strikeout and skips the processing of step S22. Note that batting simulator 1 stereoscopically displays the swing trajectory of the virtual bat on tri-surface display unit 10, regardless of whether the ball has been hit or not.

[0078] According to the process of the flowchart in FIG. 7, by rendering a virtual model in which a ball object, a bat object, etc. are arranged, the result of hitting in the virtual space can be displayed three-dimensionally in the real space as a ball image and a bat image.

[0079] The process of step S16 in Fig. 7 is executed, for example, according to a process procedure as shown in Fig. 8. Fig. 8 is a flowchart of an example of a process procedure for calculating trajectory information of a bat object in a virtual space based on position information and attitude information of the dedicated bat.

[0080] In step S30, the tracker 230a attached to the tip of the dedicated bat 42 and the tracker 230b attached to the grip end detect position information. In step S32, the bat detection unit 62 calculates posture information of the dedicated bat 42 based on a line connecting the position information detected by the tracker 230a attached to the tip of the dedicated bat 42 and the position information detected by the tracker 230b attached to the grip end.

[0081] In step S34, the bat detection unit 62 calculates trajectory information of a bat object in virtual space (a virtual bat in virtual space) that synchronizes its movement with the dedicated bat 42 in real space, based on the position information of the tip and grip end of the dedicated bat 42 detected in step S30 and the posture information of the dedicated bat 42 calculated in step S32. Then, in step S36, the bat detection unit 62 smoothes the trajectory information of the virtual bat calculated in step S34 using a Kalman filter.

[0082] According to the process of the flowchart in Fig. 8, the trajectory information of the virtual bat calculated in step S34 is smoothed, and the trajectory information of the virtual bat can be displayed with high accuracy as shown in Fig. 9. Fig. 9 is an image diagram of an example of the swing trajectory of the virtual bat displayed three-dimensionally on a three-sided display unit. Fig. 9 is an example of the swing trajectory of the virtual bat synchronized with the swing trajectory of dedicated bat 42 displayed with high accuracy.

[0083] [Other embodiments] In the above embodiment, an example was described in which the three-sided display unit 10 has three display units in three directions as viewed from the user P. However, it is also possible to omit one of the three display units, for example, as shown in FIG. 10.

[0084] Fig. 10 is a schematic diagram of an example of a batting simulator according to this embodiment. The batting simulator in Fig. 10 is an example having a two-sided display unit having two display units in two directions.

[0085] 10, the batting simulator has a two-sided display unit, a floor projector 21, and a front projector 23. As in the first embodiment, a user P uses the batting simulator while wearing liquid crystal shutter glasses 41, a helmet (not shown) on his / her head, and holding a dedicated bat 42. The two-sided display unit has a floor display unit 11, a wall display unit (second wall display unit 13), and a second curved portion 15.

[0086] The images projected on the dual-display unit by the floor projector 21 and the front projector 23 are images for the right eye and the left eye that are alternately displayed in a time-division manner. By viewing the images for the right eye and the left eye that are alternately displayed on the dual-display unit in a time-division manner through the liquid crystal shutter glasses 41 that the user P wears on the floor display unit 11 can recognize the images as three-dimensional.

[0087] In the above embodiment, an example has been described in which a stereoscopic image is projected onto the three-sided display unit 10 or the like using a projection-type projector, but a stereoscopic image may also be displayed using a display device such as a liquid crystal display or a plasma display, or a head-mounted display.

[0088] In addition, the above embodiment has been described using the dedicated bat 42, which is an example of a tool that the user P uses in real space, but the present invention can also be applied to tools other than the dedicated bat 42. This embodiment can also be applied to tools such as a tennis racket, a table tennis racket, a badminton racket, an ice hockey stick, a golf club, and a bamboo sword used in kendo.

[0089] Furthermore, the batting simulator 1 according to this embodiment can be used for the purpose of practicing sports such as baseball, and can also be used for the purpose of games and play. For example, when used for games, it is not limited to sports games, and can also be applied to role-playing games and action games. For example, in a role-playing game, by applying it to sensing the movement of a sword, shield, etc. that a user P moves in real space, the movement of a sword or shield that the user P can move in the virtual space in the game can be synchronized with the movement of a sword or shield that the user P moves in real space.

[0090] (summary) According to this embodiment, it is possible to reduce the cost of a simulation device that synchronizes the movement of a tool such as a dedicated bat 42 that a user P uses in real space with the movement of a virtual bat or the like used in virtual space.

[0091] The present invention is not limited to the above specifically disclosed embodiments, and various modifications and variations are possible without departing from the scope of the claims. For example, the bat position detection unit 70 is an example of a tool position detection means. The bat posture calculation unit 72 is an example of a tool posture calculation means. The helmet detection unit 64 is an example of an attached item detection means. The trajectory information calculation unit 74 is an example of a trajectory information calculation means. The stereoscopic image control unit 58 is an example of a stereoscopic image control means. The stereoscopic image control unit 58 also functions as a movement information calculation means. The hitting result determination unit 60 is an example of a collision determination means. [Explanation of symbols]

[0092] 1. Batting Simulator 10 3-sided display 11 Floor display section 12 1st wall display section 13 2nd wall display section 21 Floor projector 22 Side projector 23 Front projector 41 Liquid crystal shutter glasses 42 Dedicated Bat 43 Helmet 50 Virtual model setting section 52 Batting Selection Section 54 Viewpoint position calculation unit 56 View direction setting section 58 Stereoscopic image control unit 60 Hitting result determination section 62 Bat detector 64 Helmet detector 70 Bat position detection unit 72 Bat posture calculation unit 74 Orbit information calculation section 100 Operation Unit 110 Screen display unit 120 Coin Insert Unit 130 IC card unit 150 Computers 190 Transmitter for Liquid Crystal Shutter Glasses 210 Infrared Sensor 220a~220c Tracker communication device 230a~230c Tracker

Claims

1. tool detection means provided at least at two locations of a tool handled by a user in real space, the tool detection means detecting position information and orientation information of the tool; a tool orientation calculation means for calculating orientation information of the tool based on a line connecting information on at least two positions of the tool detected by the tool detection means; a wearable device detection means provided in a wearable device worn on the user's head and configured to detect position information and posture information of the user's head; trajectory information calculation means for calculating trajectory information of a virtual tool used by the user in a virtual space, based on the position information of the tool detected by the tool detection means and the orientation information of the tool calculated by the tool orientation calculation means; a stereoscopic image control means for controlling a display unit to display a stereoscopic image including the virtual tool viewed from a virtual viewpoint position in the virtual space corresponding to position information and orientation information of a head of the user, based on trajectory information of the virtual tool; A simulation device having the above configuration.

2. a movement information calculation means for calculating movement information of an object moving in the virtual space; a collision determination unit that performs a collision determination between the virtual tool and the object based on trajectory information of the virtual tool and movement information of the object, The stereoscopic image control means causes the display unit to display a stereoscopic image including the virtual tool and the object based on a result of the collision determination.

2. The simulation device according to claim 1,

3. The trajectory information calculation means smoothes the calculated trajectory information of the virtual tool using a Kalman filter.

3. The simulation device according to claim 1 or 2,

4. The tool detection means and the attachment detection means detect the position information in the same manner.

4. The simulation device according to claim 1, wherein:

5. The stereoscopic image control means generates images for the right eye and the left eye so that a stereoscopic image including the virtual tool and the object viewed from the virtual viewpoint position is displayed on a floor display unit and a wall display unit located in at least two directions as viewed from the user, alternately displays the images for the right eye and the left eye on the floor display unit and the wall display unit, and controls alternately opening and closing the fields of view of the right eye and the left eye of the glasses worn by the user in synchronization with the images for the right eye and the left eye displayed on the floor display unit and the wall display unit.

3. The simulation device according to claim 2,

6. the tool detection means is attached to a grip end and a tip of a bat that the user uses in real space, The tool posture calculation means calculates posture information of the tool based on a line connecting a position detected by a first tool detection means attached to a grip end of the bat and a position detected by a second tool detection means attached to a tip of the bat.

6. The simulation device according to claim 1, wherein:

Images