EVALUATION APPARATUS, EVALUATION METHOD, AND PROGRAM

Abstract

Provided is an evaluation device or the like that evaluates a subject from a body motion of the subject and an object behavior associated with the body motion of the subject. The evaluation device evaluates a subject on the basis of an error between an object behavior estimated from a physical activity of the subject and an actual object behavior by using time-series data on the physical activities of the subject and time-series data on object behaviors associated with the physical activities.

Description

[Technical field]

[0001] The present invention relates to an evaluation device, an evaluation method, and a program for evaluating a subject. [Background technology]

[0002] Baseball pitchers throw balls of various speeds, types, and trajectories in order to strike out batters, while batters must accurately predict the behavior of the ball as it reaches their hand and then hit it.

[0003] Non-Patent Document 1 proposes a method for measuring, analyzing, evaluating, and providing feedback on the behavior of a ball thrown by a pitcher.

[0004] Non-Patent Document 2 suggests that information about the pitcher's pitching form has an effect on the prediction of the ball's behavior to the batter. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] "TRACKMAN FOR THOSE WHO KNOW HOW TO PRACTICE LIKE A PRO", [online], [retrieved November 15, 2021], Internet<URL:https: / / trackmanbaseball.com / > [Non-Patent Document 2] Kimura, T., Nasu, D., & Kashino, M., "Utilizing Virtual Reality to Understand Athletic Performance and Underlying Sensorimotor Processing", Proceedings 2018, 2(6), 299 Summary of the Invention [Problem to be solved by the invention]

[0006] Non-Patent Document 1 merely evaluates the behavior of the pitched ball, but as shown in Non-Patent Document 2, it is believed that batters use pitching form information to predict the behavior of the ball and then hit it. Therefore, it is necessary to evaluate pitchers based on both the ball's behavior and pitching form. However, such an evaluation method has not been proposed.

[0007] The present invention aims to provide an evaluation device, evaluation method, and program for evaluating a subject (e.g., a pitcher) based on the subject's physical movement (e.g., pitching form) and the behavior of an object (e.g., the behavior of a ball) accompanying the subject's physical movement. [Means for solving the problem]

[0008] In order to solve the above problems, according to one aspect of the present invention, an evaluation device uses time series data regarding the subject's physical movement and time series data regarding the behavior of an object accompanying the physical movement, and evaluates the subject based on the error between the behavior of the object estimated from the physical movement and the actual behavior of the object. Effect of the Invention

[0009] According to the present invention, it is possible to effectively evaluate a subject based on the subject's body movements and the behavior of an object accompanying the subject's body movements. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining background experimental results. [Diagram 2] FIG. 1 is a functional block diagram of an evaluation system according to a first embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of a processing flow of the evaluation system according to the first embodiment. [Figure 4] FIG. 11 is a functional block diagram of a learning system according to a second embodiment. [Diagram 5] FIG. 11 is a diagram showing an example of a processing flow of a learning system according to the second embodiment. [Figure 6]FIG. 13 is a diagram showing an example of the configuration of a computer to which the present technique is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described. In the drawings used in the following description, components having the same functions and steps performing the same processes are denoted by the same reference numerals, and duplicated explanations will be omitted. In the following description, the symbols " - " etc. should be written directly above the character immediately preceding it, but due to limitations in text notation, they are written immediately after the character in question. Within formulas, these symbols are written in their original positions. Furthermore, unless otherwise specified, processing performed on each element of a vector or matrix applies to all elements of that vector or matrix.

[0012] <Key Points of the First Embodiment> In this embodiment, a pitcher is evaluated using the error between the ball speed predicted from the pitching form and the actual ball speed.

[0013] It has been reported that baseball batters use not only the initial behavior of the ball but also pitching form information to predict the behavior (e.g., ball speed) of a pitched ball and hit it (e.g., Non-Patent Document 2, etc.). However, only the behavior of the ball is used in general evaluation indices for pitchers (Non-Patent Document 1), and there are no evaluation indices that use the relationship between the behavior of the ball and the pitching form.

[0014] In this embodiment, pitchers are evaluated using the error between the ball behavior predicted from the pitching form and the actual ball behavior. The larger the error, the more difficult it is to predict the ball behavior from the pitching form, which means that the batter's prediction is more likely to be wrong. In this case, the more difficult the prediction, the higher the pitcher's evaluation.

[0015] In this embodiment, the key point is to use footage taken from the batter's perspective to calculate the features of the pitching form, and to make predictions from footage seen from the batter's perspective. It is easy to imagine that there is a relationship between the pitching form and the behavior of the ball, but unless this relationship is conveyed to the batter as information, it will not affect his athletic performance. Therefore, by using footage from the batter's perspective, it is possible to obtain evaluation indices that are directly linked to athletic performance.

[0016] This embodiment is based on the discovery that there is a correlation between features obtained from pitching form video (motion video data) filmed from the batter's side and the pitched ball speed, and evaluates the difficulty of predicting a pitcher's ball speed from the error between the ball speed estimated from the motion video data (estimated ball speed) and the actual ball speed (actual ball speed).

[0017] First, the contents of the experiment that served as the basis for the discovery of the correlation between pitching form and ball speed, which is the background of this embodiment, and the results thereof will be described.

[0018] [Background Experiment] (i) Using a video camera, a video of a pitcher throwing a fastball multiple times (N times) is taken from the batter's side, and the ball speed at that time is measured using a speed gun or the like. In this embodiment, the subject is a fastball, but other types of pitches may be used. However, it is assumed that the pitches are of the same type. In addition, in this experiment, video taken from slightly behind the batter's box (from the batter's eye level) is used. However, video taken from the umpire's position or behind the backstop (spectator seats) may also be used. The point is to use video taken from the batter's side (not behind or to the side of the pitcher) in order to evaluate the difficulty of predicting the batter. In other words, time-series data related to the subject's (pitcher's) movements as seen from the subject's (batter's) opponent may be used.

[0019] (ii) Determine the temporal and spatial analysis ranges of the acquired video.

[0020] For example, the time range of analysis is set to a point (t1) a predetermined time (e.g., 300 ms) back from the point (t2) when the ball is released. Note that t1 can be set freely as long as it is the time until the ball is released (t2), and can be real time (e.g., 300 ms before release) as in this example, or the point when a motion event is detected (the point when the pitcher lifts his leg). In this experiment, it was set so that the correlation of the results would be maximized. Note that the time range of analysis is set to the point (t2) when the ball is released so that ball information is not included in the optical flow described later.

[0021] Also, for example, the width of the spatial analysis range is set to half the mound width (W) from the center of the plate, and the height of the spatial analysis range is set to half the mound width (H) from the bottom of the mound. In essence, the spatial analysis range should be the range that contains the entire pitching motion. In this experiment, the spatial analysis range was defined using the mound width, but the spatial analysis range may be limited to a part of the body (such as the pitching arm).

[0022] (iii) Calculate the optical flow feature of the pitching form from the captured video. Optical flow is a vector representation of the movement of an object in a time-series digital image (captured video). There are various conventional techniques for calculating optical flow, and any method may be used. The method for calculating the optical flow feature will be described below.

[0023] (1) Calculate the average value f(t) of the optical flow within the spatial analysis range at a certain time t using the following equation.

number

[0024] (2) Integrate f(t) over the time domain of analysis.

number

[0025] (3) Average OF for each pitcher over N innings - OF is calculated. In this experiment, the average value is used, but other representative values ​​may be used. For example, the median value may be used.

[0026] The results of the above experiment are shown in Figure 1. The numbers in the circles in Figure 1 represent the identification numbers of the target pitchers. These results show that there is a correlation between the optical flow features of the pitching form photographed from the batter's side and the ball speed.

[0027] First Embodiment FIG. 2 is a functional block diagram of the evaluation system according to the first embodiment, and FIG. 3 shows the processing flow thereof.

[0028] The evaluation system includes an evaluation device 100 , a motion video data acquisition unit 110 , and an object behavior data acquisition unit 140 .

[0029] The evaluation device 100 includes a feature extraction unit 120 , an estimation unit 130 , and an evaluation unit 150 .

[0030] The motion video data acquisition unit 110 captures the subject's body movements to acquire motion video data. In this example, the motion video data acquisition unit 110 captures the subject's motion (pitching) from the batter's side while pitching the ball to acquire the motion video data.

[0031] The object behavior data acquisition unit 140 acquires the behavior of an object accompanying the physical movement of the subject. In this example, the object behavior data acquisition unit 140 measures the pitched ball speed (actual ball speed) of the subject and acquires the object behavior data.

[0032] The evaluation device 100 uses time-series data on the subject's physical motion and information on the actual behavior of an object associated with the physical motion to evaluate the subject based on the error between the object's behavior estimated from the physical motion and the actual object's behavior, and outputs an evaluation value. Motion video data acquired by filming the subject's physical movements is used as time-series data on the subject's physical motion, and the pitched ball speed (actual ball speed) acquired by the object behavior data acquisition unit 140 is used as information on the actual behavior of an object associated with the physical motion. In this embodiment, the evaluation device 100 estimates the ball speed from the motion video data of the pitcher, calculates the difficulty of predicting the pitcher's ball speed using the error between the estimated ball speed and the actual ball speed, and evaluates the pitcher based on the difficulty of predicting the ball speed.

[0033] The evaluation device 100 and the learning device 200 described later are special devices configured by loading a special program into a publicly known or dedicated computer having, for example, a central processing unit (CPU), a main storage device (RAM), and the like. The evaluation device 100 and the learning device 200 execute each process under the control of, for example, the central processing unit. Data input to the evaluation device 100 and the learning device 200 and data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out to the central processing unit as necessary and used for other processes. At least a part of each processing unit of the evaluation device 100 and the learning device 200 may be configured by hardware such as an integrated circuit. Each storage unit provided in the evaluation device 100 and the learning device 200 can be configured by, for example, a main storage device such as a RAM (Random Access Memory), or middleware such as a relational database or a key-value store. However, each storage unit does not necessarily need to be provided inside the evaluation device 100 and the learning device 200, but may be configured as an auxiliary storage device made up of a hard disk, optical disk, or semiconductor memory element such as a flash memory, and may be configured to be provided outside the evaluation device 100 and the learning device 200.

[0034] Each part will be explained below.

[0035] <Exercise video data acquisition unit 110> The motion image data acquisition unit 110 captures the motion of the subject while pitching from the batter's side (S110) and outputs the motion image data. A commercially available video camera or the like may be used as the imaging device.

[0036] <Feature Extraction Unit 120> The feature extraction unit 120 receives motion video data as input, extracts motion features from the motion video data (S120), and outputs them. In this example, the optical flow (hereinafter also referred to as "OF") of the pitching form is found from the images included in the motion video data, and the feature of the OF is calculated. The method for extracting the feature is the same as the procedure described in [Background Experiment]. An evaluation may be performed for one pitch by a certain pitcher, or for N pitches. For example, the feature is extracted as follows:

[0037] First, the temporal and spatial analysis ranges of the acquired image are determined.

[0038] Next, the average value f(t) of the optical flow within the spatial analysis range at a certain time t is calculated using the following equation.

number

number

[0039] <Estimation part 130> The estimation unit 130 estimates (130) the behavior of an object (ball speed in this example) accompanying the subject's physical movement from the features using a model that has been trained in advance to input the features and output an estimation result of the behavior of an object accompanying the subject's physical movement from the subject's physical movement, and outputs the estimation result. In this example, the model has been trained in advance to input the features extracted from motion video data capturing a pitching form, and output an estimation result of the ball speed (estimated ball speed). The model training method will be described in the second embodiment.

[0040] <Object behavior data acquisition unit 140> The object behavior data acquisition unit 140 acquires the behavior of an object accompanying the subject's body movement (S140) and outputs the object behavior data. For example, the object behavior data acquisition unit 140 is a device (such as a speed gun or a radar-type / image-type ball trajectory measuring device) for measuring the subject's pitched ball speed (actual ball speed), and measures the actual ball speed as the object behavior data.

[0041] <Evaluation Section 150> The evaluation unit 150 receives the estimation result (estimated ball speed in this example) and the object behavior data (actual ball speed in this example) from the estimation unit 130, evaluates the subject based on the error between them (S150), and outputs an evaluation value. In other words, the evaluation unit 150 calculates a higher evaluation as the deviation between the estimation result from the estimation unit 130 and the object behavior data increases.

[0042] Here, we evaluate the magnitude of the absolute value of the error under the assumption that a typical batter predicts ball speed using pitching form information. The larger the absolute value of the error, the greater the difference between the ball speed predicted from the pitching form and the actual ball speed, meaning that prediction is more difficult. The more difficult the prediction, the harder it is for the batter to hit the ball, and the higher the pitcher's evaluation. In other words, the greater the discrepancy between the estimation result and the object behavior data, the more difficult the prediction is, and the higher the subject's evaluation.

[0043] <Effects> With the above configuration, the subject can be evaluated based on the subject's body movements and the behavior of an object accompanying the subject's body movements.

[0044] <Modification> In this embodiment, the subject is a pitcher and the physical movement is pitching, but other sports may be the subject. This embodiment is not limited to baseball, and can be applied to competitive ball games such as tennis, volleyball, table tennis, soccer, and basketball. The ball games referred to here are competitive games using a sphere (ball) or similar objects, and the balls used in ball games vary in size, material, and shape. Here, ball games include rugby, badminton (a shuttlecock), and ice hockey (a puck). Furthermore, this embodiment is applicable to any sport in which the subject's physical movement and the associated behavior of an object can be estimated. In this case, the subject is evaluated based on the error between the behavior of the object estimated from the physical movement and the actual behavior of the object.

[0045] In this embodiment, the time series data of the optical flow feature amount is used as the time series data of the subject's body movement, but other time series data of the body movement may be used. For example, the optical flow itself obtained from a video captured by a video camera may be used as the time series data of the subject's body movement, or the time series data of the body movement obtained by image analysis of the video may be used. In addition, the time series data of the body movement obtained by optical motion capture or the time series data of the body movement obtained by an acceleration sensor may be used.

[0046] In addition, in either measurement method, physical movement can include whole-body movement, movement of the throwing arm (hand, wrist, forearm, etc.), and movement of the throwing arm relative to the torso, and the movement in this case can be velocity, acceleration, or a combination of these.

[0047] Second Embodiment In this embodiment, a learning system that learns a model constituting the estimation unit 130 in the evaluation device 100 of the first embodiment will be described. Fig. 4 shows a functional block diagram of the learning system, and Fig. 5 shows an example of the processing flow thereof.

[0048] The learning system includes a learning device 200 , a motion video data acquisition unit 110 , and an object behavior data acquisition unit 140 .

[0049] The learning device 200 includes a feature extraction unit 220 and a learning unit 230.

[0050] The learning device 200 receives learning data as input, learns a regression model using the learning data, and outputs the learned regression model.

[0051] The processing of each part will be explained below.

[0052] In order to obtain learning data, a number of subjects perform pitching movements a number of times in advance.

[0053] <Movement video data acquisition unit 110 and object behavior data acquisition unit 140> The motion video data acquisition unit 110 captures an image of a subject performing a pitching motion to acquire motion video data for learning (S210), while the object behavior data acquisition unit 140 measures and outputs an actual ball speed for learning (S240). The configurations of the motion video data acquisition unit 110 and the object behavior data acquisition unit 140 are the same as those in the first embodiment.

[0054] <Feature Extraction Unit 220> The feature extraction unit 220 receives as input the motion video data for learning acquired by the motion video data acquisition unit 110, and similarly to the feature extraction unit 120, extracts and outputs features of the pitching form from the images included in the motion video data (S220). The method of extracting features is the same as that of the feature extraction unit 120. Then, a set of learning data consisting of pairs of the obtained features and actual ball speeds is created.

[0055] <Learning Section 230> The learning unit 230 receives a set of learning data as input, learns a model that estimates the behavior of an object accompanying the physical movement of a subject from time-series data on the subject's physical movement (S230), and outputs the model. In this embodiment, the model receives the feature values ​​extracted from the motion video data as input, and outputs the estimated ball speed (estimated ball speed). In this example, a simple regression model (see FIG. 1) is learned in which the feature values ​​in the learning data are X and the actual ball speed is Y.

[0056] <Effects> With this configuration, it is possible to learn the model used in the first embodiment, which takes feature values ​​as input and outputs estimated values ​​of object behavior.

[0057] <Modification> In this embodiment, a simple regression model is learned, but any model that estimates the behavior of an object accompanying a physical movement from time-series data related to the physical movement of a subject may be used. For example, the behavior to be estimated may be the type and course of a ball, the number of rotations of the ball, the inclination of the rotation axis, etc., in addition to the ball speed. The model may be a neural network model or the like other than the simple regression model. For example, the neural network model can be learned by machine learning or the like using learning data in which the actual behavior is the correct answer.

[0058] <Other Modifications> The present invention is not limited to the above-mentioned embodiment and modified examples. For example, the above-mentioned various processes may be executed not only in chronological order as described, but also in parallel or individually depending on the processing capacity of the device executing the processes or as necessary. In addition, appropriate modifications are possible within the scope of the present invention.

[0059] <Programs and recording media> The various processes described above can be implemented by loading a program for executing each step of the above method into the memory unit 2020 of the computer shown in FIG. 6 and operating the control unit 2010, input unit 2030, output unit 2040, etc.

[0060] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other type of recording medium.

[0061] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0062] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. The above-mentioned process may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result, without transferring a program from the server computer to the computer. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct command to the computer but has a nature that specifies the processing of the computer, etc.).

[0063] In addition, in this embodiment, the present device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

Claims

1. An estimation unit that estimates a behavior of an object associated with a physical movement of a subject from the time series data regarding the physical movement of the subject using a model that is trained to input time series data regarding the physical movement and output an estimation result of a behavior of the object associated with the physical movement from the physical movement; an evaluation unit that evaluates the subject based on an error between an estimated object behavior and an actual object behavior, the time series data relating to the subject's physical movement is time series data relating to the subject's movement as seen by an opponent of the subject; the evaluation unit calculates a higher evaluation as the deviation between the estimated behavior of the object and the actual behavior of the object increases; The exercise performed by the subject is a competitive ball game, and the object is a ball used in the ball game. Evaluation equipment.

2. An evaluation method using an evaluation device, an estimation step in which the evaluation device estimates a behavior of an object accompanying a physical movement of a subject from the time series data regarding the physical movement of the subject, using a model trained to input time series data regarding the physical movement and output an estimation result of a behavior of an object accompanying the physical movement from the physical movement; an evaluation step in which the evaluation device evaluates the subject based on an error between an estimated object behavior and an actual object behavior, the time series data relating to the subject's physical movement is time series data relating to the subject's movement as seen by an opponent of the subject; In the evaluation step, a higher evaluation is calculated as the deviation between the estimated behavior of the object and the actual behavior of the object increases, The exercise performed by the subject is a competitive ball game, and the object is a ball used in the ball game. Evaluation method.

3. A program for causing a computer to function as the evaluation device of claim 1.

Images