Training device, method and program

Abstract

This training device comprises: a storage unit 1 which has stored therein video information that corresponds to settings of each video; a setting unit 2 which sets video settings information that is for specifying settings for a video; a video synthesis unit 3 which synthesizes a virtual video showing a target person by using exercise information of said target person together with video information that corresponds to the video settings information and is read from the storage unit 1; and a video presentation unit 4 which presents the synthesized virtual video to the target person. The video synthesis unit 3 delays the motion of the target person in the virtual video on the basis of gaze movement acquired from a sensor attached to the target person.

Description

[Technical field]

[0001] The present invention relates to a technique for improving the athletic ability of a subject. [Background technology]

[0002] The technology disclosed in Patent Document 1 is known as a technology for improving the athletic ability of a subject.

[0003] The training device described in Patent Document 1 includes a memory unit that stores motion information used when presenting the motion of an object as an image, and attention motion type information, which is information that associates changing motion type information that indicates the type of motion information that differs between motions with high and low evaluations with attention visual information that is information that identifies the attention area and attention time in the virtual image, a training condition setting unit that is given training target motion information that is the type of motion to be trained by the trainee and outputs attention visual information associated with the changing motion type information that corresponds to the training target motion information based on the attention motion type information, and a second image synthesis unit that uses the motion information to synthesize a virtual image that presents an environment that cannot be reproduced in the real environment, and the second image synthesis unit synthesizes a virtual image on which a visual effect that emphasizes the attention area identified by the attention visual information is superimposed near the attention time included in the attention visual information in the virtual image.

[0004] This makes it possible to train the subject to be able to handle perceptual information that contributes greatly to motor control. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-42219 A Summary of the Invention [Problem to be solved by the invention]

[0006] The training device of Patent Document 1 is limited in the types of training that can be performed.

[0007] An object of the present invention is to provide a training device, a training method, and a program that are capable of carrying out training that cannot be carried out by the training device of Patent Document 1. [Means for solving the problem]

[0008] In order to solve the above problems, a training device according to one aspect of the present invention includes a storage unit in which video information corresponding to each video setting is stored, a setting unit that sets video setting information which is information for specifying the video setting, an image synthesis unit that synthesizes a virtual image in which the subject appears using motion information of the subject and video information corresponding to the video setting information read from the storage unit, and an image presentation unit that presents the virtual image to the subject, and the image synthesis unit synthesizes a virtual image in which the subject appears using motion information of the subject and video information corresponding to the video setting information read from the storage unit, and If is greater than a predetermined threshold, Slow down the subject's movements in the virtual image. Effect of the Invention

[0009] It is possible to carry out training that cannot be carried out using the training device of Patent Document 1. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional configuration of a training device. [Diagram 2] FIG. 2 is a diagram showing an example of a processing procedure of the training method. [Diagram 3] FIG. 3 is a diagram illustrating an example of a functional configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described. Note that components having the same functions and steps performing the same processes are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0012] The training device includes a memory unit 1, a setting unit 2, an image synthesis unit 3, and an image presentation unit 4.

[0013] The training device may include a sensor 5 for acquiring eye movement of a training subject, and a subject camera 6 for acquiring motion information of the subject.

[0014] The training method is realized, for example, by each component of the training device performing the processes from step S2 to step S4 described below and shown in FIG.

[0015] The training device and method will be described below using baseball batting training as an example. However, the use of the training device and method is not limited to baseball batting training. The training device and method may be used for training in other sports. Examples of other sports are interpersonal ball games such as softball, tennis, and table tennis, and badminton, which have roles of serving (throwing) and receiving (hitting back).

[0016] Each component of the training device will now be described.

[0017] <Storage section 1> Image information corresponding to the settings of each image is stored in the storage unit 1. The image information may literally be information about an image, or may be information necessary for generating a virtual image, which will be described later.

[0018] An example of a video setting is the setting of the movement of objects that appear in the video.

[0019] For example, when training for baseball batting, the object motion setting is at least one of (1) the motion setting of the pitcher and (2) the motion setting of the ball.

[0020] (1) The settings for the pitcher's movements can also be said to be settings for the type of ball the pitcher will throw.

[0021] Video information on the movements of pitchers corresponding to each pitch type is stored in the storage unit 1. For example, video information on the movements of an overhand pitcher who throws a straight pitch, video information on the movements of a sidearm pitcher who throws a curveball, and video information on the movements of an underhand pitcher who throws a slider are stored in the storage unit 1.

[0022] (2) The setting of the ball's movement can also be considered the setting of the type of ball.

[0023] A series of ball position information corresponding to each pitch type is stored in the storage unit 1. For example, a series of straight ball position information, a series of curve ball position information, and a series of slider ball position information are stored in the storage unit 1.

[0024] The ball position information sequence is information indicating the position of the ball at each time t=(Tstart, Tstart+1, ..., Tstart+Tc) in the time range from the time Fstart when the ball leaves the pitcher's hand to the time Tc has elapsed. The time here may be a frame number.

[0025] The image settings may include image environment settings, such as sunny, rainy, bright, and dark.

[0026] <Settings section 2> The setting unit 2 sets video setting information, which is information for specifying video settings (step S2).

[0027] The set video setting information is output to the video synthesis unit 3 .

[0028] The setting unit 2 may set the video setting information randomly, or may set the video setting information based on a predetermined criterion.

[0029] Furthermore, the setting unit 2 may set video setting information input by a user of the training device and method using an input device such as a mouse, a keyboard, a touch pad, etc. An example of a user of the training device and method is a subject of training performed by the training device and method.

[0030] The video setting information may include two or more pieces of information for specifying the video setting. For example, the setting unit 2 may set the video setting information as (pitcher's movement, ball's movement)=(straight, straight).

[0031] The setting unit 2 may set video setting information in which the pitch type corresponding to the pitcher's movement is different from the pitch type corresponding to the ball's movement, such as (pitcher's movement, ball's movement)=(straight, curve).

[0032] As in this example, when the video setting information is (pitcher's movement, ball movement) = (straight, curve), the image presented by the video presentation unit 4 described below will be an image of a situation that would never occur in reality, in which the pitcher throws the ball with a form for throwing a straight ball, but the ball moves in a curved trajectory.

[0033] In this way, by making it possible to set different video setting information for the type of pitch corresponding to the pitcher's movement and the type of pitch corresponding to the ball's movement, it becomes possible to train to respond to situations that would never occur in reality.

[0034] <Video composition section 3> The video composition unit 3 receives the video setting information set by the setting unit 2 .

[0035] Moreover, motion information of the subject acquired by the subject camera 6 is input to the image synthesizing unit 3 .

[0036] Moreover, the image synthesizing unit 3 receives the movement of the target person's line of sight acquired by the sensor 5.

[0037] The image synthesizing unit 3 synthesizes a virtual image showing the subject, using the motion information of the subject and the image information corresponding to the image setting information read from the storage unit 1 (step S3).

[0038] At this time, the image synthesizer 3 delays the movement of the subject in the virtual image based on the movement of the line of sight acquired from a sensor 5 attached to the subject.

[0039] The synthesized virtual image is output to the image presentation unit 4.

[0040] The subject's motion information is, for example, the three-dimensional positions of markers attached to the subject's body, generated by optical motion capture using multiple cameras, the subject cameras 6. In this case, the subject's movements are represented by an avatar.

[0041] The subject's motion information may be image information of the subject captured by the subject camera 6. In this case, the virtual image is synthesized using the actual image information of the subject.

[0042] The video synthesizing unit 3 reads from the storage unit 1 video information corresponding to the video setting information set by the setting unit 2 .

[0043] For example, when the video setting information is set as (pitcher's movement, ball movement)=(straight, straight), the video synthesis unit 3 reads from the memory unit 1 video information of the movement of the overhand throw pitcher, which corresponds to the pitch type called a straight, and a series of positional information of the straight ball.

[0044] The sensor 5 may be attached directly to the subject or indirectly to the subject. The sensor 5 is attached to, for example, a head-mounted display worn by the subject.

[0045] The sensor 5 may be a gaze measuring device independent of the head mounted display and not built into the head mounted display.

[0046] The image synthesizer 3 delays the movement of the subject in the virtual image based on the movement of the subject's line of sight acquired from a sensor 5 attached to the subject.

[0047] Examples of eye movement are the eye path and the viewpoint path. The eye path is the path of the direction in which you are looking. The viewpoint path is the path of the position in which you are looking.

[0048] Below, an example of how to slow down the movement of a subject in a virtual image based on the subject's line of sight will be described.

[0049] First, the image synthesis unit 3 judges whether the movement of the subject's line of sight acquired by the sensor 5 has become larger than a predetermined threshold. The predetermined threshold is a value determined in advance so as to obtain a desired result.

[0050] The image synthesis unit 3 may determine that the movement of the gaze has become larger than a predetermined threshold when a predictive saccade occurs in which the subject tracks the thrown ball and shifts his or her gaze to the predicted impact point, or when the magnitude of the gaze shift due to the predictive saccade is larger than a predetermined magnitude.

[0051] In addition, the image synthesis unit 3 may determine that the movement of the gaze has become larger than a predetermined threshold when a catch-up saccade occurs, in which the gaze moves quickly to track the ball when it is thrown, or when the magnitude of the gaze shift due to the catch-up saccade is larger than a predetermined amount.

[0052] When the eye movement exceeds a predetermined threshold, the image synthesizer 3 slows down the movement of the subject in the virtual image. For example, when the subject is practicing batting, the rotation speed of the subject's bat swing in the virtual image is slowed down.

[0053] When the movement of the line of sight is greater than a predetermined threshold, the image synthesizer 3 may delay the movement of the subject in the virtual image to a greater extent as the predetermined threshold becomes smaller.

[0054] In the case of a skilled batting player, it takes a long time to judge the ball that has been thrown. This is because the skilled batting player delays the swing of the bat until the very last moment in order to judge the ball that has been thrown.

[0055] On the other hand, the time it takes an inexperienced batter to judge a thrown ball is shorter than the time it takes an experienced batter to judge a thrown ball.

[0056] For this reason, in order to have an inexperienced subject imitate the batting style of an experienced person, when the time to judge the thrown ball is short, it is advisable to show the subject a virtual image in which the subject's movements are slowed down, thereby making the subject understand that it is preferable to have a slower timing for swinging the bat, in other words, that it is okay to have more time to judge the thrown ball.

[0057] Here, the time to judge the thrown ball can be considered as the time it takes for the eye movement to exceed a certain threshold. Therefore, when the eye movement exceeds a certain threshold, the image synthesis unit 3 assumes that the subject has judged the thrown ball, and delays the subject's movement in the virtual image.

[0058] The image synthesizer 3 may change the amount of delay in the movement of the subject in the virtual image in accordance with at least one of the passage of time and the behavior of the subject.

[0059] For example, the image synthesis unit 3 may decrease the amount of delay in the movement of the subject in the virtual image over time, so that the amount of delay in the movement of the subject becomes zero at the time the bat comes into contact with the ball.

[0060] By setting the delay in the subject's movement to zero at the time the bat makes contact with the ball, it is possible to reduce any discomfort the subject may feel.

[0061] The virtual image may include a person other than the training target, such as a pitcher.

[0062] <Video display unit 4> The virtual image synthesized by the image synthesizer 3 is input to the image presenter 4.

[0063] The image presenting unit 4 presents the virtual image to the subject (step S4).

[0064] The image display unit 4 is, for example, a head-mounted display worn by the subject. The image display unit 4 may be a display such as a liquid crystal display, an organic EL display, a micro LED display, a video projector, a cathode ray tube, or a plasma display.

[0065] As described above, by presenting a virtual image in which the subject's movements are delayed based on the eye movement obtained from a sensor attached to the training subject, it is possible to conduct training that was previously impossible.

[0066] [Variations] Although the embodiments of the present invention have been described above, the specific configurations are not limited to these embodiments, and it goes without saying that appropriate design changes, etc., are included in the present invention as long as they do not deviate from the spirit of the present invention.

[0067] The various processes described in the embodiments may not only be executed in chronological order according to the order described, but may also be executed in parallel or individually depending on the processing capabilities of the devices executing the processes or as necessary.

[0068] For example, data exchange between components of the training device may be performed directly or via a storage unit (not shown).

[0069] Furthermore, the predetermined threshold used in the processing of the video synthesis unit 3 may be set by a user of the training device and method using an input device such as a mouse, keyboard, or touchpad. This allows the user of the training device and method to set the predetermined threshold to a desired value, thereby improving the convenience of the training device and method.

[0070] The image synthesizer 3 may speed up the movement of the subject in the virtual image based on the movement of the line of sight acquired from the sensor 5 attached to the subject.

[0071] [Programs, recording media] The processing of each unit of each of the above-mentioned devices may be realized by a computer. In this case, the processing contents of the functions that each device should have are described by a program. Then, this program is loaded into the storage unit 1020 of the computer 1000 shown in Fig. 3 and operated by the arithmetic processing unit 1010, the input unit 1030, the output unit 1040, etc., whereby various processing functions of each of the above-mentioned devices are realized on the computer. The computer 1000 may be equipped with a display unit 1060.

[0072] The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a non-transitory recording medium, specifically, a magnetic recording device, an optical disk, etc.

[0073] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0074] A computer that executes such a program, for example, first stores a program recorded on a portable recording medium or a program transferred from a server computer in the auxiliary recording unit 1050, which is a non-transient storage device of the computer. Then, when executing a process, the computer reads the program stored in the auxiliary recording unit 1050, which is a non-transient storage device of the computer, into the storage unit 1020, and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium into the storage unit 1020 and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct instruction to a computer but has a nature that specifies the processing of the computer, etc.).

[0075] In this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing may be realized by hardware. For example, the setting unit 2 and the video synthesis unit 3 may be configured by a processing circuit. Also, the storage unit 1 may be a memory.

[0076] Needless to say, other modifications can be made without departing from the spirit of the present invention.

[0077] [Additional Note] The training device may be any of the training devices described below.

[0078] The training device includes a memory in which image information corresponding to each image setting is stored, a processing circuit, and a display for presenting the virtual images to the subject.

[0079] The processing circuit is configured to (i) set image setting information, which is information for specifying the image settings, and (ii) synthesize a virtual image showing the subject using the subject's movement information and image information corresponding to the image setting information read from the memory.

[0080] The processing circuitry is also configured to slow movement of the subject in the virtual image based on gaze movement obtained from a sensor attached to the subject.

Claims

1. a storage unit in which video information corresponding to each video setting is stored; a setting unit that sets video setting information that is information for specifying video settings; an image synthesis unit that synthesizes a virtual image in which the subject is shown by using motion information of the subject and image information corresponding to the image setting information read from the storage unit; an image presenting unit that presents the virtual image to the subject; The image synthesis unit slows down the movement of the subject in the virtual image when the movement of the line of sight acquired from the sensor attached to the subject is greater than a predetermined threshold. training equipment.

2. 2. The training device of claim 1, When the movement of the line of sight is greater than a predetermined threshold, the image synthesis unit delays the movement of the subject in the virtual image to a greater extent as the predetermined threshold becomes smaller. training equipment.

3. 3. The training device of claim 2, the predetermined threshold is configurable by a user of the training device; training equipment.

4. 4. The training device of claim 1, further comprising: The image synthesis unit changes a delay amount of the movement of the subject in the virtual image according to the behavior of the subject. training equipment.

5. The storage unit stores video information corresponding to each video setting, A setting step in which a setting unit sets video setting information which is information for specifying a video setting; an image synthesis step in which an image synthesis unit synthesizes a virtual image in which the subject is shown by using motion information of the subject and image information corresponding to the image setting information read from the storage unit; An image presenting step in which an image presenting unit presents the virtual image to the subject, The image synthesis unit slows down the movement of the subject in the virtual image when the movement of the line of sight acquired from the sensor attached to the subject is greater than a predetermined threshold. Training method.

6. A program for causing a computer to function as each part of the training device according to any one of claims 1 to 4.

Images