Information processing apparatus, information processing method, and information analysis system

Abstract

This information processing device is provided with a calculation unit that performs a process for generating action tendency information indicating a movement tendency with respect to a specific action that is a human action.

Description

Technical Field

[0001] The present technology relates to an information processing device, an information processing method, and an information analysis system, and to a technical field of performing analysis processing of specific actions in sports and the like. Background Art

[0002] In recent years, various types of analysis have been performed on the performance of sports players such as baseball, softball, football, and basketball, and the analysis has been used for practice and game strategies. For example, a GPS receiver is attached to a player's uniform, etc. to measure the distance the player runs during a game, thereby obtaining the player's exercise load.

[0003] In addition, the following patent document 1 discloses a technology by which a target dynamic image and a comparison dynamic image can be selected from a plurality of dynamic images obtained by capturing images of the movements of people playing a ball game, thereby making it easy to grasp the technical level of the movements, points for improvement in techniques, etc.

[0004] Citation List

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-145702A Summary of the invention

[0007] Problems to be solved by the present invention

[0008] For example, various actions of sports players have been video recorded, and the videos have been repeatedly viewed to analyze the state of the actions, but there are cases where it is difficult to recognize the actions of various parts of the body only through the recorded dynamic images. For example, in the pitching action of a baseball pitcher, it is difficult to understand the relationship between the pitching form and the ball type as a result of the pitching by only viewing the actual dynamic images.

[0009] It is therefore an object of the present disclosure to be able to provide useful information for predicting specific actions and their outcomes.

[0010] Solution to the problem

[0011] An information processing device according to the present technology includes a calculation unit that performs processing to generate action tendency information indicating an action tendency regarding a specific action as a human action.

[0012] A specific action is conceived as an action observed as a segment in a sport. For example, it is a pitching action in baseball, a penalty kick action in football, etc. Action tendency information indicating an action tendency is, for example, the action tendency of each person regarding a specific action, for example, information including the action of an object observed as an action habit, or information indicating the action tendency itself. For example, there is an artificial intelligence (AI) model that has learned the pitching action of a certain pitcher, and important movements in a specific action are determined from the AI ​​model, for example, position information of a specific part of the body at a certain timing, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is an explanatory diagram of an information analysis system according to an embodiment of the present technology.

[0014] Figure 2 is a block diagram of an information analysis system according to an embodiment.

[0015] Figure 3 is a block diagram of an information processing device constituting an information analysis system according to an embodiment.

[0016] Figure 4 It is an explanatory diagram of the tendency presentation screen according to the first embodiment.

[0017] Figure 5 It is an explanatory diagram of an important action model and a feature amount map according to the first embodiment.

[0018] Figure 6 is an explanatory diagram of another example of the important action model and the feature amount map according to the first embodiment.

[0019] Figure 7 It is an explanatory diagram of the prediction accuracy and the time series change diagram according to the first embodiment.

[0020] Figure 8 It is an explanatory diagram of a computer graphics (CG) model on a tendency presentation screen according to the first embodiment.

[0021] Fig. 9 It is an explanatory diagram of highlight display on the tendency presentation screen according to the first embodiment.

[0022] Fig.10 is a flowchart of the processing in the data accumulation phase according to the first embodiment.

[0023] Fig.11 is a flowchart of processing in the data display phase according to the first embodiment.

[0024] Fig.12 It is an explanatory diagram of a training screen according to the second embodiment.

[0025] Fig.13 It is an explanatory diagram of starting playback of a pitching video according to the second embodiment.

[0026] Fig.14 It is an explanatory diagram of the answer timing according to the second embodiment.

[0027] Fig.15 It is an explanatory diagram after answering according to the second embodiment.

[0028] Fig.16 It is an explanatory diagram of a training screen of a referee's viewpoint according to the second embodiment.

[0029] Fig.17 is a flowchart of processing in the data accumulation phase according to the second embodiment.

[0030] Fig.18 is a flowchart of processing in the data display phase according to the second embodiment. DETAILED DESCRIPTION

[0031] Hereinafter, the embodiments will be described in the following order.

[0032] <1. Configuration of information analysis system>

[0033] <2. First Embodiment: Processing of the Tendency Presentation Screen>

[0034] <3. Second Embodiment: Processing of Training Screen>

[0035] <4. Summary and Modifications>

[0036] Note that in this disclosure, "image" includes both dynamic images and still images. For example, assuming Figure 1 The imaging device 10 shown in the diagram mainly captures dynamic images, but for example, the image displayed on the terminal device 5 may be a dynamic image or a still image. In addition, "image" refers to an image actually displayed on the screen, and "image" in the signal processing process or transmission path before being displayed on the screen refers to image data.

[0037] <1. Configuration of information analysis system>

[0038] Figure 1 An overview of an information analysis system 1 according to the embodiment is illustrated.

[0039] Figure 1 The information analysis system 1 includes an imaging device 10, a server device 2, a score data server 3, a sensor 4, and a terminal device 5. These components are connected to each other via wired communication, wireless communication, network communication, and the like.

[0040] For example, a plurality of imaging devices 10 images an area of ​​a subject (eg, the situation of a game) in a stadium such as a baseball field from various positions. Note that although a plurality of imaging devices 10 are shown, it is sufficient that at least one imaging device 10 is provided.

[0041] In the present embodiment, an example of analyzing a pitch of a pitcher in baseball or softball will be described. Thus, the imaging device 10 is arranged to image at least the position of the pitcher (ie, the area including the pitcher's mound).

[0042] Then, in the information analysis system 1, the captured data of a subject such as a pitcher is extracted from the captured image of the imaging device 10, and information about the pitching action can be presented based on the captured data. The captured data is the captured data of the skeleton of the subject (skeleton captured data), the captured data of the expression of the subject (expression captured data), and the captured data of the items (bat, gloves, etc.) held by the subject (item captured data). Here, the item captured data indicates the position (ready position or set position) and angle of the bat held by the subject, the position and angle of the gloves worn by the subject, the degree of opening (the degree of opening of the little finger part and the thumb part), etc. That is, the information analysis system 1 can present information about the pitching action based on at least one of the skeleton captured data, the expression captured data, or the item captured data. In this embodiment, as an example, a case where information about the pitching action is presented based on the skeleton captured data will be described.

[0043] Over the years, with respect to games in sports such as football and basketball, the following technology has been called an Electronic Performance and Tracking System (EPTS): the posture and position of players and referees, the position and rotation of the ball, etc. are estimated from a designated field based on images captured by dedicated cameras and information obtained by sensors (accelerometers and GPS sensors) attached to people (players) and objects (ball) involved in the game.

[0044] Specifically, the imaging device 10 captures images for obtaining EPTS data including skeleton capture data of a player, tracking data indicating a trajectory of a ball, and the like.

[0045] Furthermore, the image captured by the imaging device 10 can also be used as a real-scene image of a game or the like.

[0046] The EPTS data generated based on the images captured by the imaging device 10 is transmitted to the server device 2. For example, in the case where an information processing device (not shown) is set on the side of a stadium such as a baseball field to record images captured by a plurality of imaging devices 10 and generate EPTS data, the EPTS data generated by the information processing device is transmitted to the server device 2.

[0047] Alternatively, a captured image obtained by the imaging device 10 may be transmitted to the server device 2, and the server device 2 may generate EPTS data.

[0048] Sensor 4 is a sensor that detects the movement of a player, a ball, etc. Specifically, a sensor attached to a player or a ball is envisioned, such as the above-mentioned acceleration sensor or GPS sensor. The player's movement information can also be obtained from the information detected by sensor 4. Alternatively, the information of sensor 4 can be used supplementarily when obtaining bone capture data or estimating posture from an image. The detection information of sensor 4 can also be used for the speed, rotation speed, trajectory, etc. of the ball.

[0049] The detection information of the sensor 4 may be transmitted to the server device 2, or may be input to an information processing device (not shown) that generates EPTS data on the side of a stadium such as a baseball stadium.

[0050] The score data server 3 is, for example, an information processing device of an association that generates and distributes score data SD of baseball. Here, the score data server 3 is configured as a server device capable of transmitting the score data SD of the game to the server device 2.

[0051] The score data server 3 may sequentially transmit the score data SD to the server device 2 during the game, or may transmit all the score data SD of the game together after the game, for example.

[0052] In the case of baseball, the score data SD includes the following various types of information on the game.

[0053] For example, the information includes information about the entire game, such as the score and player information for each inning of the game, and information about each pitch of the pitcher. For each pitch, the scoring data includes, for example, the following information and various types of other information: pitching time, pitcher name, batter name, whether the batter is right-handed or left-handed, whether there is a runner on first base, second base, or third base, baserunner name, strike / ball count, whether the pitch is a ball or a strike, out count, batting result (hits, swings, misses, foul balls, ground ball outs, fly ball outs, foul fly balls, etc.), pitch speed, type of pitch, rotation speed, rotation axis, pitch path.

[0054] The terminal device 5 is, for example, an information processing device such as a smartphone, a tablet terminal, or a personal computer, and as the terminal device 5, for example, a device used by a person related to a baseball team (such as a player or an analyst of the baseball team) is envisioned. Then, the terminal device 5 is a device that presents an image obtained by analyzing the movement of each player, an image for training, etc. to the player, the analyst, etc. The image provided by the server device 2 is displayed on the display unit 5a of the terminal device 5.

[0055] The server device 2 performs various processes for presenting analysis information in the terminal device 5. For example, analysis results of skeleton captured data of a subject generated based on an image captured from the imaging device 10 are displayed.

[0056] Specifically, the server device 2 causes the terminal device 5 to display the information on the tendency of the pitching action of the pitcher. The information on the tendency of the pitching action includes, for example, information indicating the tendency (difference) of the pitching action for each type of ball (such as a straight ball, a curve ball, and a fork ball) as a result of the pitching. The tendency of the pitching action includes what is generally called the habit of the pitching action of the pitcher. The process of displaying such information on the tendency of the pitching action will be described later as a first embodiment.

[0057] In addition, the server device 2 performs various processing for presenting images for training the user using the terminal device 5. For example, the processing is to generate a CG image of a pitching action based on the skeleton capture data of the subject generated from the image captured by the imaging device 10, and present it to the user to answer the ball type. The processing of displaying such an image for training will be described later as a second embodiment.

[0058] As the server device 2 , an information processing device that performs cloud computing, that is, a cloud server is assumed.

[0059] However, the processing for causing the terminal device 5 to display the image of the analysis result and the image for training can be performed by an information processing device other than the cloud server. For example, it is also conceivable that an information processing device such as a personal computer installed at the competition venue has the function of serving as the server device 2, and performs the acquisition of bone capture data, the processing of generating comparison images, etc. In addition, it is also conceivable that the terminal device 5 also has the function of serving as the server device 2, and performs the processing of acquiring bone capture data and generating images of the analysis result or images for training.

[0060] Figure 2 The diagram illustrates the above Figure 1 An example of a functional configuration of a server device 2 in the information analysis system 1 and an input / output system related to the server device 2.

[0061] The imaging device 10 is configured as a digital camera device including an imaging element such as a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor, and obtains a captured image as digital data. In this example, each imaging device 10 obtains a captured image as a dynamic image.

[0062] As reference Figure 1As described above, each imaging device 10 images the game conditions in the baseball field (particularly at least the pitching action of the pitcher) and is arranged at a predetermined position. The number of imaging devices 10 is one or more, and there is no particular regulation, but in order to generate accurate EPTS data, it is advantageous that the number is as large as possible.

[0063] Each imaging device 10 captures a dynamic image in a synchronized state, and outputs the captured image.

[0064] The recording unit 11 records each image captured by the plurality of imaging devices 10 , and supplies each captured image to the EPTS data generating unit 12 .

[0065] The EPTS data generating unit 12 performs analysis processing on one or more captured images to generate EPTS data individually, and integrates all individual EPTS data to generate overall EPTS data. The EPTS data includes, for example, the position of the player (pitcher, outfielder, batter or base runner) or the ball at each frame timing, the player's skeleton capture data, the player's posture based on the skeleton capture data, the ball's rotation speed and rotation direction information, etc.

[0066] Furthermore, the EPTS data generating unit 12 may also generate EPTS data by using not only captured images but also information obtained by the sensor 4, which may be information from, for example, an acceleration sensor embedded in the ball or a GPS sensor attached to the player's uniform.

[0067] The EPTS data generation unit 12 may generate information for determining a pitching motion or a speed or trajectory of a ball, for example, for each of all pitches of a pitcher during a game, as EPTS data for the entire game.

[0068] The EPTS data generating unit 12 may generate EPTS data from a plurality of captured images obtained by a plurality of imaging devices 10, and may also generate EPTS data from a plurality of captured images obtained by one imaging device 10. In addition, the EPTS data generating unit 12 may also generate EPTS data from a plurality of images and information from one or more sensors, or may also generate EPTS data from one captured image and information from one sensor.

[0069] The EPTS data generated by the EPTS data generating unit 12 is transmitted to the server device 2. The EPTS data includes skeleton capture data BD and tracking data indicating the trajectory, rotation, and the like of an item (ball, bat, racket, and the like) used in a competition.

[0070] Note that the image data captured by the imaging device 10 may also be transmitted to the server device 2 via, for example, the EPTS data generating unit 12 .

[0071] The EPTS data generating unit 12 may be provided in the server device 2. For example, in this case, the image captured by the imaging device 10 and the detection information of the sensor 4 only need to be transmitted to the EPTS data generating unit 12 in the server device 2 via network communication or the like.

[0072] In addition, as referenced Figure 1 As described above, the score data SD is sequentially transmitted from the score data server 3 to the server device 2 .

[0073] The server device 2 includes an information processing device such as a computer device, and is provided with functions as a calculation unit 21 , a presentation control unit 22 , and a storage control unit 23 by software, for example.

[0074] The calculation unit 21 performs processing to generate action tendency information indicating an action tendency regarding a specific action as a human action.

[0075] A specific action is conceived as an action observed as a segment in a sport, for example, it is a pitching action of a baseball pitcher, a penalty kick action in football, etc.

[0076] The action tendency information indicating the action tendency is, for example, action information including the action tendency (for example, action habits) of each person regarding a specific action, or information indicating the action tendency itself.

[0077] Examples of the action tendency information described in the embodiment include an AI model that has learned the habits of each pitcher obtained by learning using EPTS data (mainly skeleton capture data BD) and scoring data SD, feature quantities that contribute greatly to the pitch in the AI ​​model in prediction, and important movement information observed according to the pitch. The important action information is, for example, information indicating the difference of each pitch as the position or trajectory of a specific part of the body.

[0078] Furthermore, the calculation unit 21 performs processing for generating display information for user perception training, the display information including a dynamic image of a specific action and an operation member for answering an action result of the specific action indicated by the dynamic image.

[0079] Thus, the calculation unit 21 has functions as a drawing processing unit 21 a , a captured data processing unit 21 b , a score data processing unit 21 c , and an analysis processing unit 21 d .

[0080] The captured data processing unit 21 b performs processing for generating a CG image of a player or the like, time series information of changes in the skeleton according to the action, and the like based on the captured data such as the skeleton captured data BD.

[0081] The score data processing unit 21 c performs processing related to the score data SD acquired from the score data server 3 .

[0082] Specifically, processing such as management of score data SD for each individual pitch, extraction of score data SD for analysis of pitching motion, and extraction of score data SD for displaying an image is performed.

[0083] The graphics processing unit 21a performs processing for generating a skeletal animation or CG of a pitching form by combining the skeleton capture data BD and the score data SD.

[0084] The analysis processing unit 21d performs processing for generating an AI model for each pitcher, analyzing the action (pitching) habits using the AI ​​model, and generating display information for displaying the analysis results.

[0085] For example, the analysis processing unit 21d generates action tendency information based on the skeleton capture data. For example, the skeleton capture data of each pitch of a certain pitcher is learned to generate an AI model of the pitcher. Then, the analysis processing unit 21d determines the points (positions, parts, areas, etc.) on the body that are the action differences of each ball type according to the AI ​​model, and also determines the presence or absence of habits, important movements that are regarded as habits, etc. The analysis processing unit 21d then generates display information for displaying the analysis results.

[0086] Note that the analysis processing unit 21d generates an AI model for each pitcher. For example, when a certain pitcher is targeted, the AI ​​model is generated and updated by learning the EPTS data and score data SD of all pitches of the pitcher as input. Then, at each time point, the feature quantity in the AI ​​model is determined. For example, the feature quantity in this case can be said to be a variable suitable for predicting the difference in pitching actions corresponding to the type of ball.

[0087] The storage control unit 23 performs a process of storing the received skeleton capture data BD, score data SD, etc. in a storage medium. In addition, the storage control unit 23 performs a process of storing various types of data generated by the calculation unit 21 in a storage medium.

[0088] The presentation control unit 22 performs control to display various types of image information generated by the processing of the calculation unit 21 in a predetermined mode on the terminal device 5. Specifically, the presentation control unit 22 performs processing to display the bone image or CG generated by the calculation unit 21, an image indicating the analysis result, etc. on the terminal device 5.

[0089] Furthermore, when the presentation control unit 22 presents a user interface (UI) image, a process of receiving operation information of the user of the terminal device 5 on the UI image is also performed.

[0090] The above-mentioned calculation unit 21, presentation control unit 22, and storage control unit 23 may be provided in one information processing device, or may be separately provided in a plurality of information processing devices.

[0091] An image generated by the server device 2 or data for image generation is transmitted to the terminal device 5 and displayed on the display unit 5 a of the terminal device 5 .

[0092] Will describe Figure 1 and Figure 2 The configuration of the information processing device 70 used in the information analysis system 1. For example, Figure 2 The server device 2, the terminal device 5, the EPTS data generation unit 12, etc. can be Figure 3 It is implemented by the information processing device 70 shown in the diagram.

[0093] Furthermore, the information processing device 70 may be realized as, for example, a dedicated workstation, a general-purpose personal computer, a mobile terminal device, or the like.

[0094] Figure 3 The CPU 71 of the information processing device 70 illustrated in the figure performs various types of processing according to a program stored in a ROM 72 or a nonvolatile storage unit 74 such as an electrically erasable programmable read-only memory (EEP-ROM), or a program loaded into a RAM 73 from a storage unit 79. In addition, the RAM 73 also stores data and the like necessary for the CPU 71 to perform various types of processing as appropriate.

[0095] The image processing unit 85 is implemented as a processor that performs various types of image processing. For example, the processor is a processor that can perform any of image generation processing of CG images based on the skeleton capture data BD, image analysis processing of captured images, etc., generation processing of animation images and 3D images, database (DB) processing, image effect processing, EPTS data generation processing, etc.

[0096] The image processing unit 85 can be implemented by, for example, a CPU separate from the CPU 71, a graphics processing unit (GPU), a general purpose computing based on a graphics processing unit (GPGPU), an artificial intelligence (AI) processor, or the like.

[0097] Note that the image processing unit 85 may be provided as a function in the CPU 71 .

[0098] The CPU 71, the ROM 72, the RAM 73, the nonvolatile storage unit 74, and the image processing unit 85 are connected to one another via a bus 83. In addition, an input / output interface 75 is also connected to the bus 83.

[0099] An input unit 76 including operating members and operating devices is connected to the input / output interface 75. As the input unit 76, for example, one of various operating members and operating devices including a keyboard, mouse, key, dial, touch panel, touch pad, remote controller, etc. is contemplated.

[0100] The user operation is detected by the input unit 76 , and a signal corresponding to the input operation is interpreted by the CPU 71 .

[0101] Furthermore, a display unit 77 including a liquid crystal display (LCD), an organic electroluminescence (EL) panel, or the like, and an audio output unit 78 including a speaker or the like are connected to the input / output interface 75 integrally or separately.

[0102] The display unit 77 performs various types of displays as a user interface. The display unit 77 is realized as, for example, a display device provided in a housing of the information processing device 70, an independent display device connected to the information processing device 70, or the like.

[0103] The display unit 77 displays various images on the display screen based on instructions from the CPU 71. Furthermore, the display unit 77 displays various operation menus, icons, messages, etc. based on instructions from the CPU 71, that is, displays as a graphical user interface (GUI).

[0104] For example, in a case where the information processing device 70 is regarded as the terminal device 5, the display unit 77 displays a tendency presentation screen 30, a training screen 100, and the like described later.

[0105] In some cases, a storage unit 79 including a solid state drive (SSD), a hard disk drive (HDD), or the like, and a communication unit 80 including a modem or the like are connected to the input / output interface 75 .

[0106] For example, in a case where the information processing device 70 is regarded as the server device 2 , the storage unit 79 can be regarded as a storage medium used by the storage control unit 23 to store information.

[0107] The communication unit 80 performs communication processing via a transmission path such as the Internet, and performs wired / wireless communication and communication based on bus communication and the like with various devices.

[0108] Furthermore, as necessary, a drive 82 is connected to the input / output interface 75 , and a removable recording medium 81 such as a flash memory, a memory card, a magnetic disk, an optical disk, or a magneto-optical disk is appropriately mounted.

[0109] The drive 82 can read data files such as image files, various computer programs, etc. from the removable recording medium 81. The read data files are stored in the storage unit 79, and the images and sounds included in the data files are output by the display unit 77 and the audio output unit 78. In addition, the computer program and the like read from the removable recording medium 81 are installed in the storage unit 79 as needed.

[0110] In the information processing apparatus 70, the software can be installed through network communication via the communication unit 80 or the removable recording medium 81. Alternatively, the software may be stored in the ROM 72, the storage unit 79, or the like in advance.

[0111] For example, when the information processing device 70 is regarded as the server device 2, the functions of the above-mentioned calculation unit 21, presentation control unit 22 and storage control unit 23 are set by software, and these functions are processed by the CPU 71 and the image processing unit 85.

[0112] <2. First Embodiment: Processing of the Tendency Presentation Screen>

[0113] As a first embodiment, a method of causing the server device 2 to cause the terminal device 5 to display, for example, Figure 4 The processing of the tendency presentation screen 30 is shown in the diagram.

[0114] In the case of baseball, it is important for players to be able to predict the type of ball and pitching route of the next pitch based on the opponent's pitcher's pitching action, pitching trajectory and game situation.

[0115] Currently, the pitching action habits and pitching trajectory tendencies of the opposing pitcher are only analyzed in the brain based on the batter's experience and images when he is standing at the batting position, so there are great individual differences. Under this current situation, for example, if the pitching action tendencies are analyzed for each pitcher to generate action tendency information, and a display based on the action tendency information can be provided, the team can inform each player of the pitcher's habits, which is very effective as a game countermeasure.

[0116] Thus, the server device 2 causes the terminal device 5 to present Figure 4 The tendency to present screen 30 and provide information.

[0117] In the tendency presentation screen 30 , first, a pitcher selection unit 31 , a viewpoint selection unit 32 , and an action phase selection unit 33 are provided as settings for analysis processing.

[0118] The pitcher selection unit 31 , the viewpoint selection unit 32 , and the action phase selection unit 33 are interfaces for setting the range of input data for learning when an AI model is generated by learning data of a pitcher's pitching action.

[0119] The pitcher selection unit 31 is, for example, a field that can be selected by each pitcher who is an opponent, and the user selects a pitcher to be analyzed from, for example, a pull-down item.

[0120] The viewpoint selection unit 32 is a field for selecting the viewpoint position for the pitching action of the pitcher, and for example, the user selects batter (home plate), right-handed batter, left-handed batter, first base runner, second base runner, third base runner, umpire, etc. as the viewpoint position from the drop-down items. This is because the pitching action of the pitcher looks different depending on the viewpoint position.

[0121] The action phase selection unit 33 is a field for selecting a time period to be analyzed in a series of actions of a pitching action.

[0122] The action phase 34 is displayed near the action phase selection unit 33, and describes the content of the phase to be selected. For example, the time period from the start of pitching to the non-pivot foot reaching the highest position is phase 0, the subsequent time period until the non-pivot foot touches the ground is phase 1, the subsequent time period until the elbow of the dominant arm passes the ear is phase 2, and the subsequent time period until immediately after the ball is released is phase 3.

[0123] The user can select the conditions of the analysis time periods of "until stage 0", "until stage 1", "until stage 2" and "until stage 3" in the action stage selection unit 33. For example, for a batter, the habit of stage 2 or stage 3 is difficult to deal with, because even if the habit is discovered, the withdrawal of the swing has already begun, which is difficult to use as a reference. For a batter, it is desired to find the habit of stage 0. In the case of such a need, "until stage 0" can be an analysis target. In addition, the various stages of the action stage 34 are not limited to those described above. For example, a predetermined time period before the start of pitching (the time period before stage 0) can be included as a stage.

[0124] The calculation unit 21 of the server device 2 analyzes the EPTS data and the score data according to the conditions selected by the user using the pitcher selection unit 31, the viewpoint selection unit 32, and the action phase selection unit 33, and displays the analysis result.

[0125] As the analysis result, an example is shown in which the pitching motion display unit 40 on the left side of the screen and the tendency display unit 60 on the right side of the screen are roughly displayed.

[0126] The pitching motion display unit 40 is an area for intuitively and easily displaying how the pitching forms differ between a straight ball and a breaking ball, for example.

[0127] The tendency display unit 60 is an area for displaying how many pitching habits there are and important movements regarded as habits.

[0128] First, description will be given of the tendency display unit 60. The tendency display unit 60 displays prediction accuracy 61, a time series change graph 62, a message 63, an important action model 64, a feature amount graph 66, accuracy of each situation 67, and the like.

[0129] As described above, through the function of the analysis processing unit 21d, the calculation unit 21 learns the pitching action data of each pitch of the pitcher to be analyzed, specifically, the skeleton capture data BD of the pitching action, and generates the action tendency information of the pitching action of the pitcher. The action tendency information is an AI model generated by learning the skeleton capture data BD and the score data SD as input data, and information analyzed from the AI ​​model, for example, the importance of the feature quantity of the body part that differs for each ball type in the pitching action, etc.

[0130] In the tendency display unit 60 , the analysis result of the pitching action of the target pitcher using the AI ​​model is indicated by the important action model 64 and the feature amount map 66 . Figure 5 The important action model 64 and the feature quantity graph 66 are enlarged and shown in FIG.

[0131] The important action model 64 indicates the joints 68 and the bones 69 in the image of the pitcher imitating the pitching action. Then, the body position to be paid attention to is highlighted as the focus point 65. The highlighting as the focus point 65 emphasizes a specific part of the body that is related to the important movement of the body that is considered to be likely to have a habit through the analysis of the AI ​​model of the pitcher. The "important movement" mentioned here means a summary of the movements that are considered important when predicting straight balls and changeups in the AI ​​analysis process. For example, in the case where differences are observed in terms of position (coordinate value in three-dimensional coordinates), trajectory of position change, timing, etc. depending on the type of ball, the action in which the difference occurs is set as an important movement, and the body position related to the important movement is indicated as the focus point 65.

[0132] In this example, as important movements emphasized at the focus point 65, "the position of the left elbow", "the distance between the left knee and the elbow", "the position of the right wrist", "the angle of the right ribs", and "the position of the left wrist" are picked up.

[0133] Note that the focus point 65 is not limited to the information indicating the body position (position / angle of the part of the pitcher) related to the important movement as described above, and Figure 6As shown in the diagram in FIG. 6 , the timing, expression, and position of items related to important movements may be included. For example, “the position of the left elbow in stage 0,” “the shape of the glove in stage 1,” “the position of the right wrist in stage 0,” “the shape of the mouth in stage 0,” and “the position of the left knee in stage 1” may be important movements to be emphasized in the focus point 65.

[0134] exist Figure 5 and Figure 6 In the feature amount graph 66 illustrated in FIG. 6A , the vertical axis indicates the importance of the feature amount, and the horizontal axis indicates the importance of the feature amount of each important movement.

[0135] The importance is the degree of importance of determining the habit. For each pitch, the more the action tends to be different, the higher the importance. On the other hand, even if there is a difference in the action of each pitch, if it cannot be said that the difference is a difference for each pitch, the importance is low.

[0136] For example, in the case of an analysis for the purpose of ball type prediction, whether or not it is important is determined in terms of the difference in each ball type. Thus, in the case of predicting not only the ball type but also the ball type and the ball path, an action in which a difference is observed for each combination of the ball type and the ball path is an "important move".

[0137] The user can obtain a hint for discovering the pitcher's habits by viewing the important action model 64 and the feature quantity graph 66 .

[0138] For example, the user can recognize the possibility of the existence of a habit by referring to the focus point 65 of the important action model 64 and the importance indicated by the feature amount graph 66 with respect to the “important movement” indicated by the focus point 65 .

[0139] Note that Figure 5 and Figure 6 , the important action model 64 is shown from the batsman's viewpoint as a descriptive example, but in practice, it is desirable to display the important action model 64, the joints 68, and the skeleton 69 in a directionality from the viewpoint selected by the viewpoint selection unit 32.

[0140] Incidentally, some pitchers have easily observable habits for each pitch, while some pitchers have almost no habits. In other words, whether the action identified as an "important move" by analysis is actually a habit related to the pitch that a person can visually recognize varies from pitcher to pitcher.

[0141] Therefore, the tendency display unit 60 also displays the prediction accuracy 61 , the time series change graph 62 , the message 63 , and the accuracy 67 of each condition.

[0142] Prediction accuracy 61 indicates the prediction accuracy of the AI ​​model. The server device 2 calculates the prediction accuracy by comparing the actual pitching result (e.g., pitch type) and the pitching result (e.g., pitch type) predicted from the “important move” under the current analysis conditions (pitcher, viewpoint, and action phase), and displays the value as prediction accuracy 61.

[0143] If the value displayed with prediction accuracy 61 is high, the user can refer to the displayed "Important Movements" to determine that the possibility of a habit is high. Conversely, if the value displayed with prediction accuracy 61 is low, the user cannot determine that the possibility of a habit is high even if he pays attention to "Important Movements".

[0144] In addition, the time series change graph 62 shows, for example, the change in the prediction accuracy value every day. For example, assuming that new pitching action data is input for each game, and the AI ​​model and the analysis information based on the AI ​​model are updated, the prediction accuracy changes in each game. The state of the change is indicated by the time series change graph 62, and the state of the change is clearly indicated in the message 63.

[0145] Note that the vertical axis of the time series change graph 62 represents prediction accuracy, while the horizontal axis represents date and time.

[0146] exist Figure 4 In the example above, the forecast accuracy does not vary significantly with date.

[0147] On the other hand, the pitchers being analyzed may correct habits, and thus the prediction accuracy may change significantly. Figure 7 The diagram illustrates a state in which the value of the prediction accuracy 61 decreases from about 86% to 65%. In this case, it is conceivable that there is a possibility of displaying as a message 63 that the habit has been corrected.

[0148] In addition, the pitcher's habit may be easy to appear or difficult to appear, depending on the situation of the game. Therefore, the prediction accuracy of each game situation is also displayed by the accuracy 67 of each situation. For example, the server device 2 displays the accuracy in the situation where the runner is in the scoring position, the accuracy in the situation where there is a runner, the prediction accuracy in the situation where there is no runner, etc. Therefore, the user can adjust the degree of reference to the "important move" according to the situation.

[0149] Next, the display of the pitching motion display unit 40 will be described.

[0150] The pitching action display unit 40 displays a pitcher's name 41 , an image selection unit 42 , a highlight selection unit 43 , a play button 44 , a stop button 45 , a drag bar 46 , an action stage 49 , an action comparison display unit 50 , and the like.

[0151] As the pitcher name 41, the names of the team and the pitcher are displayed, clearly indicating who the target of analysis is.

[0152] In the image selection unit 42, "bone (bone model)" or "CG (CG model)" can be selected, and the display of the motion comparison display unit 50 is switched according to the user's operation. Figure 4 In FIG. 1 , the motion comparison display unit 50 shows an example of display of bone models 51 and 52 indicating skeletons. Figure 8 , the action comparison display unit 50 shows an example of display of CG models 53 and 54 as CG images of pitchers.

[0153] The user can select which of the display of the bone models 51 and 52 and the display of the CG models 53 and 54 is to be displayed on the action comparison display unit 50 by operating the image selection unit 42 .

[0154] Figure 4 The bone model 51 indicated by the solid line in the figure shows the pitching action of the pitcher to be analyzed when throwing a straight ball through the bone image, while the bone model 52 indicated by the dotted line shows the pitching action of the same pitcher when throwing a changeup through the bone image.

[0155] Figure 8 The CG model 53 indicated by the solid line in the figure shows the pitching action of the pitcher to be analyzed when throwing a straight ball through a CG image, while the CG model 54 indicated by the dotted line shows the pitching action of the same pitcher when throwing a changeup through a CG image.

[0156] The motion comparison display unit 50 displays the bone models 51 and 52 or the CG models 53 and 54 as dynamic images in a three-dimensional coordinate space. The calculation unit 21 generates images of the bone models 51 and 52 and the CG models 53 and 54 by the function of the drawing processing unit 21a.

[0157] The calculation unit 21 generates the bone model 51 and the CG model 53 as the average shape when the target pitcher throws a straight ball. That is, the calculation unit 21 generates the average shape of the pitching action observed in the skeleton capture data when the pitching result is a straight ball, and sets the average shape as the bone model 51, or generates the CG model 53 based on the bone model 51.

[0158] Similarly, the computing unit 21 generates the bone model 52 and the CG model 54 as the average shape when the target pitcher throws a breaking ball. That is, the computing unit 21 generates the average shape of the pitching action observed in the skeletal capture data when the pitching result is a breaking ball, and sets the average shape as the bone model 52, or generates the CG model 54 based on the bone model 52.

[0159] The user can easily and intuitively recognize the difference in morphology between a straight pitch and a breaking ball pitch by displaying the bone models 51 and 52 in a superimposed manner in the action comparison display unit 50. In addition, since the bone models 51 and 52 are used, it is easy to understand the difference in morphology.

[0160] Furthermore, the user can easily recognize the difference in morphology between a straight pitch and a breaking ball pitch by displaying the CG models 53 and 54 in an overlapping manner on the action comparison display unit 50. In the case of the CG models 53 and 54, the image is closer to the pitcher actually seen from the batting cage, etc., compared with the bone models 51 and 52, so it is valuable to the user in this respect.

[0161] That is, it is desirable for the user to switch between the bone models 51 and 52 and the CG models 53 and 54 while viewing.

[0162] Note that although bone models 51 and 52 and CG models 53 and 54 are shown from the viewpoint of the third base runner coach as an example of description in the figure, in reality, bone models 51, 52 and CG models 53, 54 are generated as the pitching action seen from the viewpoint selected by the viewpoint selection unit 32.

[0163] The highlight selection unit 43 is provided with a check box for allowing the user to select whether to perform highlighting on the display of the action comparison display unit 50. When the user checks the check box, a highlight 55 is added, such as Fig. 9 As shown in the diagram, highlight 55 emphasizes the different movements between the straight ball and the breaking ball.

[0164] The play button 44, the stop button 45, and the drag bar 46 are prepared for the playback operation of the dynamic image of the bone models 51 and 52 or the CG models 53 and 54 in the action comparison display unit 50. The user can start the playback of the dynamic image of the bone models 51 and 52 or the dynamic image of the CG models 53 and 54 by operating the play button 44, and can stop the playback by operating the stop button 45.

[0165] The dynamic image is a dynamic image in the time period from the start time of the pitching action to immediately after the ball is released. The slider 47 on the drag bar 46 indicates the current playback position.

[0166] Furthermore, since the action stage 49 is displayed along the slider 46, it is easy to understand at which stage the current playback section is.

[0167] Fig.10 and Fig.11 The diagram illustrates a processing example of the calculation unit 21 of the server device 2 for displaying the tendency presentation screen 30 as described above.

[0168] Fig.10 The diagram illustrates a process for accumulating data of pitching movements, for example, a process for accumulating data of pitching movements corresponding to a certain game. This process can be performed in real time during the game, or can be performed by aggregating input data after the game, etc.

[0169] In step S101, the calculation unit 21 checks whether there is the latest EPTS data. Here, the latest EPTS data is the data that the calculation unit 21 has not yet calculated. Fig.10 The processed EPTS data.

[0170] During a game, as the game progresses, data such as the pitcher's bone capture data BD, the trajectory of the ball, and the number of rotations are input for each pitch. If there is such input, there is unprocessed EPTS data. Alternatively, after the game, since EPTS data has been stored for all pitches of the target game, all EPTS data are unprocessed EPTS data.

[0171] Note that the EPTS data on all pitches may include not only the EPTS data on the pitches of the batter, but also the EPTS data on the pinned balls. For example, in order to display the habit of finding pitches and pinned balls from the viewpoint of the first base runner on the tendency presentation screen 30, the EPTS data on the pinned balls is also required.

[0172] When the latest EPTS data exists, the calculation unit 21 advances the process from step S102 to step S103. If the latest EPTS data does not exist, the calculation unit 21 proceeds to step S107, and if the game has not ended, the calculation unit 21 returns to step S101.

[0173] In step S103, the calculation unit 21 acquires the score data SD corresponding to the latest EPTS data. That is, as a result of pitching, data associated with the unprocessed EPTS data is acquired as the score data SD such as the type of ball, the number of outs, and the condition of the base runner.

[0174] In step S104, the calculation unit 21 performs a process of storing the combined data obtained by combining the score data SD and the current EPTS data in a storage medium. The storage medium is, for example, Figure 3 Storage unit 79 in.

[0175] In step S105, the calculation unit 21 performs analysis processing of the pitching action. That is, as a process through the function of the analysis processing unit 21d, the calculation unit 21 inputs the current EPTS data, generates or updates the AI ​​model of the pitcher who is pitching, and uses the AI ​​model to analyze important movements, etc. Then, the calculation unit 21 stores the information of the AI ​​model and the analysis results, that is, the model of the pitching form of the pitcher who has pitched, the information of important movements for ball type prediction, etc., as action tendency information in the storage medium.

[0176] In addition, the calculation unit 21 makes a prediction based on the "important moves" determined by the AI ​​model for past pitches, and also stores information on whether the prediction is correct. This is used to display the prediction accuracy 61 and the like.

[0177] Specifically, the action tendency information to be stored is, for example,

[0178] -Input data for AI model (pre-processed EPTS data and score data)

[0179] -Learned AI model

[0180] - AI model prediction results and true values

[0181] -Information on the characteristic quantities of important actions.

[0182] Note that the analysis process in step S105 is performed for each condition so that a result is obtained according to the condition selection on the tendency presentation screen 30 .

[0183] For example, in the case of generating an AI model of a pitching action of a certain pitcher based on the skeleton capture data BD, for the AI ​​model, the corresponding feature quantity is analyzed by the action that can be observed from the corresponding viewpoint (batter's viewpoint, first base viewpoint, second base viewpoint, etc.). In addition, for each action stage, the feature quantity in stage 0, the feature quantity up to stage 1, the feature quantity up to stage 2, and the feature quantity up to stage 3 are analyzed. Then, all the information is stored in the storage medium as information on the feature quantity of important movements related to a certain pitcher.

[0184] By executing the above-described process in step S107 until the end of the game, the action tendency information about the pitcher who plays in the current game is generated or updated.

[0185] By performing such processing in each game, action tendency information about a large number of pitchers is generated and updated.

[0186] Fig.11This is an example of processing of the calculation unit 21 when the user requests display of the tendency presentation screen 30. This processing is performed, for example, when the user activates an application for an analysis service provided by the server device 2 in the terminal device 5.

[0187] In step S201, the calculation unit 21 monitors the condition selection made by the user. For example, when the application is activated in the terminal device 5, the server device 2 first displays the pitcher selection unit 31, the viewpoint selection unit 32, and the action phase selection unit 33 on the tendency presentation screen 30, and requests the user to make a selection.

[0188] In a period in which no condition is selected, the calculation unit 21 proceeds to step S208, and returns to step S201 if the application is not ended.

[0189] After receiving the conditions selected from the terminal device 5 according to the user's operation (i.e., the conditions of the target pitcher, the viewpoint, and the action phase), the calculation unit 21 proceeds from step S201 to step S202 and searches for data corresponding to the conditions. The data to be searched here is Fig.10 The combined data of the EPTS data and the score data SD stored in the storage medium in step S104 is the combined data about the pitcher corresponding to the condition.

[0190] Note that even in the case where no condition is selected in step S201, in response to a search start instruction being input by a user operation, the calculation unit 21 proceeds from step S201 to step S202 and searches for data corresponding to a predetermined condition. The predetermined condition is, for example, a condition according to which a result with the highest prediction accuracy can be obtained, or a condition that the number of times the user has browsed in the past is large.

[0191] The situation where there is no combined data corresponding to the condition means that no pitching samples have been obtained for the current target pitcher. That is, no pitching samples have been obtained for the target pitcher this time. Fig.10 In this case, the calculation unit 21 proceeds from step S203 to step S204, instructs the terminal device 5 to display on the tendency presentation screen 30 that there is no corresponding data, and then proceeds to step S208.

[0192] Note that in step S203, it is not limited to the determination of whether there is a combination of data corresponding to the condition, and it can also be determined whether there is a predetermined number of combination of data corresponding to the condition. For example, if the number of combination of data that meets the condition is equal to or greater than 100, the process can proceed to step 205, and if the number of the combination of data is less than 100, the process can proceed to step S204.

[0193] On the other hand, in the case where there is combined data corresponding to the condition, the calculation unit 21 proceeds from step S203 to step S205 and performs display control of the tendency of the pitching action. This is display control of the tendency display unit 60.

[0194] In this case, the calculation unit 21 reads out Fig.10 The action tendency information that matches the conditions of the pitcher, viewpoint and stage among the action tendency information stored in the storage medium in step S106 is then displayed based on the read action tendency information, prediction accuracy 61, time series change graph 62, message 63, important action model 64, feature quantity graph 66 and accuracy 67 for each condition.

[0195] Subsequently, in step S206 , the calculation unit 21 performs display control of the pitching motion display unit 40 .

[0196] The calculation unit 21 extracts samples that meet the conditions from the combined data of the EPTS data and the score data SD, and generates bone models 51 and 52 or CG models 53 and 54 through the function of the drawing processing unit 21a. For example, the combined data of the pitcher corresponding to the conditions is extracted, and the combined data is classified according to whether the pitching result is a straight ball or a breaking ball. Then, the bone model 51 or the CG model 53 is generated as the average form of the skeleton capture data BD in the combined data used as a sample of a straight ball. In addition, the bone model 52 or the CG model 54 is generated as the average form of the skeleton capture data BD in the combined data used as a sample of a breaking ball.

[0197] At this time, the calculation unit 21 sets the bone models 51 and 52 or the CG models 53 and 54 to be generated as images seen from the viewpoint specified by the condition selection.

[0198] Then, the calculation unit 21 displays the generated bone models 51 and 52 or the CG models 53 and 54 on the motion comparison display unit 50 of the tendency presentation screen 30 .

[0199] Furthermore, the calculation unit 21 controls to display a pitcher's name 41 , an image selection unit 42 , a highlight selection unit 43 , a play button 44 , a stop button 45 , a drag bar 46 , and an action stage 49 .

[0200] Note that although Fig.11 Although not shown, during the display of the pitching action display unit 40, the computing unit 21 controls the start or stop of the dynamic image playback for the bone models 51 and 52 or the CG models 53 and 54 according to the user's playback operation (operation of the playback button 44) or pause operation (operation of the stop button 45).

[0201] The calculation unit 21 also performs control of the position of the slider 47 in the drag bar 46 according to the progress of the playback, control of changing the playback position according to the operation of the slider 47, and the like.

[0202] In addition, the calculation unit 21 also performs display control of the highlight 55 corresponding to the operation of the highlight selection unit 43, and switching control of the bone models 51 and 52 and the CG models 53 and 54 corresponding to the operation of the image selection unit 42.

[0203] However, all or part of these controls may be performed as controls by an application program on the terminal device 5 side.

[0204] In step S207, the calculation unit 21 confirms whether there is data to be reflected in the display in addition to the data searched in step S202. Then, in the case where there is other corresponding data, the processing of steps S205 and S206 is performed. Therefore, the other corresponding data is also reflected in the display content.

[0205] In step S208, the computing unit 21 determines the end of the display, for example, the end of the application program by the user's operation, and returns to step S201 if the display has not ended. In response to the end determination, the computing unit 21 ends the display. Fig.11 processing.

[0206] Through the above Fig.10 and Fig.11 The processing is performed on the terminal device 5. Figures 4 to 9 The display of the tendency presentation screen 30 is described. Therefore, the user can obtain information useful for determining the pitcher's habits.

[0207] <3. Second Embodiment: Processing of Training Screen>

[0208] As a second embodiment, a description will be given of Fig.12 The service for providing the training screen 100 is also performed by the server device 2.

[0209] The training screen 100 provided by the server device 2 enables confirmation of pitching movements and pitching trajectories from any viewpoint of a batsman, umpire, etc., and enables perception training for predicting the type of ball, etc. from an image (eg, a CG dynamic image).

[0210] Will describe Fig.12 100 training screens.

[0211] On the training screen 100 , a load button 101 , an item selection unit 102 , a pitcher selection unit 103 , a catcher selection unit 104 , a batter selection unit 105 , and a time period selection unit 106 are displayed.

[0212] The item selection unit 102 is a field for the user to select a training item. For example, items such as ball type determination, strike / ball determination, and pitch / pin-down determination are prepared and can be selected by the user.

[0213] Pitcher selection unit 103 is a field for the user to select a target pitcher. For example, the user can select a team name or a pitcher name.

[0214] The catcher selection unit 104 is a field for the user to select a catcher. For example, the user can select the team name or the catcher name. All catchers can also be selected.

[0215] The batter selection unit 105 is a field for the user to select a batter. For example, the user can select the batter's name. It is also possible to select all batters, all right-handed batters, all left-handed batters, etc. The target pitcher selected by the pitcher selection unit 103 can select all batters, etc. by limiting the opponent team that the pitcher is playing against.

[0216] The time period selection unit 106 is a field for selecting a time period for collecting pitching samples to be trained. The time period is set by specifying a date. For example, the user displays a calendar screen by clicking a field of the time period selection unit 106, and can specify the first day and the last day of the time period.

[0217] When the user inputs or selects the item selection unit 102, the pitcher selection unit 103, the catcher selection unit 104, the batter selection unit 105 and the time period selection unit 106, and operates the load button 101 as a click operation of the mouse cursor MK as shown in the diagram, the server device 2 searches for the corresponding pitching data and displays the corresponding pitching data on the pitching table 150.

[0218] In the pitching table 150, pitches corresponding to the conditions are listed, that is, pitches corresponding to the conditions selected by the user among the pitches for which data was acquired in the past as EPTS data and score data SD. In the pitching table 150, one row corresponds to one pitch. Then, for one pitch, information obtained based on the score data SD, etc. is displayed, such as the display center frame, answer result, pitcher name, batter name, game date, inning, and catcher name.

[0219] The pitches comprising pitch table 150 are samples of pitches used for training.

[0220] A pitching video 110 is displayed on the training screen 100. As the pitching video 110, a CG dynamic image of a pitch on the pitching table 150 is displayed. That is, the pitching video is a CG dynamic image that is generated based on the skeleton capture data BD about a certain pitch on the pitching table 150 and represents the pitching form of the pitcher. In addition, the trajectory of the ball can also be displayed based on the EPTS data.

[0221] Furthermore, the training screen 100 is provided with an image condition selection unit 140. The image condition selection unit 140 allows the user to select whether to display the viewpoint position and the trajectory.

[0222] As the viewpoint position, a right-handed batter, a left-handed batter, a catcher, first base, second base, third base, and an umpire may be selected. The computing unit 21 generates a CG corresponding to the selected viewpoint position to generate the pitching video 110. For example, Figure 12 to Figure 15 The pitching video 110 in FIG. 1 is an example of an image of the viewpoint position of a right-handed batter, and Fig.16 The pitching video 110 in FIG. 1 is an example of an image at the viewpoint position of the referee.

[0223] In addition, the trajectory is the trajectory of the ball, and the user can choose whether to display the trajectory. For example, Fig.14 , Fig.15 and Fig.16 The illustration illustrates an example in which, in addition to the ball 116, the trajectory 115 is also displayed.

[0224] As operating elements related to pitching video 110, forward buttons 111 and 112, zoom button 113, play button 114, drag bar 120, playback speed setting unit 130, play button 131, stop button 132, fast rewind button 133, fast forward button 134, repeat button 135, etc. are displayed.

[0225] Forward buttons 111 and 112 are operating members for switching the displayed pitch video 110 to a previous or subsequent pitch on the pitch table 150 .

[0226] Zoom button 113 is an operation element for zooming in and out of pitching video 110 .

[0227] Play buttons 114 and 131 are operating elements for starting playback of pitching video 110 .

[0228] Stop button 132 is an operation element for stopping the playback of pitching video 110 .

[0229] The fast-rewind button 133 and the fast-forward button 134 are operating elements for rewinding or fast-forwarding the playback of the pitching video 110 .

[0230] The repeat button 135 is an operation element for executing repeated playback of the pitching video 110 .

[0231] The playback speed setting unit 130 is a field in which the user sets the playback speed of the pitching video 110 .

[0232] The pitching video 110 is a dynamic image of a time period from the start time point to the end time point of the pitching action, and the end time point in this case is the time point when the ball passes the home plate and is caught by the catcher. Note that among the past pitching samples listed in the pitching table 150, there are samples in which the ball was actually hit but did not pass the home plate, and samples in which the catcher failed to catch the ball, but it is preferable that the pitching video 110 is generated as a CG dynamic image in a state in which the batter does not swing the bat, and the pitching end timing is set to the catch timing regardless of whether the catcher actually catches the ball.

[0233] In the drag bar 120 , the slider 121 indicates the current playback position of the pitching video 110 .

[0234] With the drag bar 120 , pointers 122 , 123 , 124 , 125 , and 126 are shown within a period from a start time point to an end time point of a pitching action.

[0235] Pointer 122 indicates the timing of the start of pitching. Pointer 123 indicates the timing of the ball being released. Pointer 124 indicates the timing of the pitching tunnel. Pointer 125 indicates the timing of passing the home plate. Pointer 126 indicates the timing of the end of pitching.

[0236] The user can recognize each timing in the pitching video 110 through the pointers 122 , 123 , 124 , 125 , and 126 .

[0237] Note that the pitching tunnel generally means a point where it is difficult for a batter to distinguish between types of balls, and it is recommended that the pitcher change the ball after passing through the pitching tunnel. The pitching tunnel is considered to be located about 7m to 10m from the home plate. Although the pitching tunnel is not a quantitatively determined value, in the training screen 100, as a timing for training to distinguish between types of balls, the pointer 124 is displayed at a timing roughly corresponding to the pitching tunnel.

[0238] After the playback starts, the pitching video 110 is automatically paused at the timing of the pointer 124. This timing is the user's answer timing. Note that the pitching video 110 can be reproduced by automatically reducing the playback speed at the timing of the pointer 124. By reducing the playback speed at the timing of the pointer 124, the difficulty of training to distinguish between ball types can be reduced.

[0239] The position of the pointer 124 can be moved back and forth by the user's dragging operation, etc. Therefore, the user can arbitrarily adjust the answer timing. As the position of the pointer 124 moves to the pitch start side, the user is required to determine the ball type at an early stage of the pitching video 110.

[0240] An answer button 160 is provided on the training screen 100 so that the user can input an answer. Fig.12 In the example, an interface is used to allow one to select and answer one of "straight ball" and "changing ball" for training in ball type judgment.

[0241] In addition, detailed information about current pitching video 110 is displayed by pitching information display unit 165. Specifically, the actual pitching date and time, opposing team, stadium, inning, number of strikes / balls, number of outs, baserunner status, etc., which are the basis for displayed pitching video 110, are displayed.

[0242] Furthermore, the result display unit 166 displays a result indicating whether the user's answer is a correct answer or an incorrect answer.

[0243] In addition, the correct answer rate 167 is also shown.

[0244] An example of the progression of the training screen 100 will be described.

[0245] When the user sets each selection unit such as the item selection unit 102, and Fig.12 When the load button 101 is operated as shown in the diagram, a pitching table 150 corresponding to the pitch is displayed, such as Fig.12 For pitches randomly selected from the pitch table 150, the pitching videos 110 are displayed in sequence.

[0246] like Fig.13 As shown in the diagram, when the user operates, for example, play button 114, playback of pitching video 110 starts.

[0247] like Fig.14 As shown in the diagram, the replay of the pitching dynamic image is paused at the timing of the pointer 124, and the user's answer is waited. The user enters the answer by clicking the answer button 160 with the mouse cursor MK or the like.

[0248] Since the user determines the type of ball based on the paused image, it is preferable to display not only the ball 116 but also the trajectory 115 .

[0249] However, for advanced training, it is preferable to turn off the display of the trajectory 115 and determine the ball type in the dynamic image until immediately before the pause.

[0250] In response to the input of the answer, the playback of the pitching video 110 is resumed, and Fig.15 As shown in the diagram in , the pitching is finished. In addition, it is displayed on the result display unit 166 whether the answer is a correct answer or an incorrect answer.

[0251] For example, in the case of a correct answer, "○" or "correct answer" is displayed, and in the case of an incorrect answer, "×" or "incorrect answer" is displayed. In addition, the ball path taken on a scale obtained by dividing the strike zone into 9 is also displayed. Fig.15 An example in which “○” is displayed on the result display unit 166 is illustrated.

[0252] As described above, the user can judge and answer the pitch type from the viewpoint of a right-handed batter or a left-handed batter, for example, and confirm the result. Thus, it is effective as training for the batter to judge the pitch type of the opponent's pitcher.

[0253] Fig.17 and Fig.18 An example of processing by the calculation unit 21 of the server device 2 for providing such a training screen 100 is illustrated.

[0254] Fig.17 The diagram illustrates the process of accumulating data of pitching movements, for example, accumulating data of pitching movements corresponding to a certain game. This process can be performed in real time during the game, or can be performed by aggregating input data after the game, etc.

[0255] exist Fig.17 In the processing of Fig.10 Similar processing is indicated by the same step number. Fig.10 The difference is that the processing of steps S105 and S106 is not performed.

[0256] Since the other steps S101, S102, S103, S104 and S107 are Fig.10 The corresponding steps in are similar, so only these steps are shown in the figure to avoid redundant description of the details of the processing.

[0257] exist Fig.17 In the processing, the calculation unit 21 only needs to store the combined data obtained by combining the score data SD and the EPTS data in step S104 in the storage medium, and no analysis is required for prediction of the ball type.

[0258] If the server device 2 performs both the service for providing the tendency presentation screen 30 and the service for providing the training screen 100, and performs Fig.10 There is no need to perform separate Fig.17 processing.

[0259] Fig.18 This diagram illustrates an example of processing of the calculation unit 21 when the user requests display of the training screen 100. This processing is performed when the user activates an application for a training service provided by the server device 2 on the terminal device 5, for example.

[0260] In step S251, the calculation unit 21 monitors the condition selection made by the user. For example, when the application is activated in the terminal device 5, the server device 2 first monitors the operation of the load button 101, the item selection unit 102, the pitcher selection unit 103, the catcher selection unit 104, the batter selection unit 105, and the time period selection unit 106 on the training screen 100.

[0261] During a period in which the load button 101 is not operated, the calculation unit 21 proceeds to step S264, and returns to step S251 if the application is not ended.

[0262] After receiving the conditions selected from the terminal device 5 according to the user's operation (i.e., the setting of the item selection unit 102, the pitcher selection unit 103, the catcher selection unit 104, the batter selection unit 105, and the time period selection unit 106 and the operation of the load button 101), the calculation unit 21 proceeds from step S251 to step S252 and searches for data corresponding to the conditions. The data to be searched here is Fig.17 (or Fig.10 ) is the combined data of the EPTS data and the score data SD stored in the storage medium in step S104, and is the combined data about the pitching corresponding to the condition.

[0263] In the case where there is no combined data corresponding to the condition, the calculation unit 21 proceeds from step S253 to step S254, instructs the terminal device 5 to display on the training screen 100 that there is no corresponding data, and then proceeds to step S264.

[0264] On the other hand, in the case where there is combined data corresponding to the condition, the calculation unit 21 proceeds from step S253 to step S255 and performs display control of the pitch table 150 .

[0265] That is, a list of information on corresponding pitches extracted by the search is displayed.

[0266] Furthermore, in step S256 , calculation unit 21 randomly selects a pitch to be presented first to the user from among the pitches listed in pitch table 150 .

[0267] For the selected pitch, Fig.12 As with the pitches in the first row of the pitch table 150, the "Showing" field is checked.

[0268] Note that here, calculation unit 21 randomly selects a pitch to be displayed first from pitch table 150, but the pitches may be specified by the user, or may be selected in order of the earliest or latest pitch date and time.

[0269] When a pitch to be displayed is selected, the computing unit 21 generates a pitching video 110 of the pitch in step S257, and displays the pitching video while waiting for playback to start, such as Fig.12 As shown in the diagram in FIG. For example, a still image and a play button 114 are displayed when a pitch is started.

[0270] At this time, calculation unit 21 sets generated pitching video 110 as an image based on the viewpoint position selected by image condition selection unit 140 .

[0271] Then, as the processing of step S257, the calculation unit 21 generates a CG dynamic image of the pitching action seen from the selected viewpoint position based on the skeleton capture data BD of the currently selected pitch, and displays the first frame of the CG dynamic image.

[0272] Note that, in the case where the user changes the viewpoint position by image condition selection unit 140 in the state before playback, calculation unit 21 generates a CG dynamic image of the pitching action seen from the new viewpoint position based on the skeleton capture data BD of the selected pitch, and displays the first frame. Thus, the playback standby state of pitching video 110 whose viewpoint position has been switched is set.

[0273] In response to the user operating play button 114 or play button 131 , calculation unit 21 proceeds to step S258 and starts playing pitching video 110 .

[0274] Note that, for pitching video 110 , calculation unit 21 also controls the presence or absence of display of trajectory 115 according to the setting of the trajectory in image condition selection unit 140 .

[0275] The calculation unit 21 automatically stops the playback of the pitching video 110 at the timing of the pointer 124. In the state where the playback is stopped, the calculation unit 21 waits for the user's answer input in step S259.

[0276] Before detecting the input of an answer, at step S260 , the calculation unit 21 controls so that an alarm display is performed, thereby prompting the user to input an answer.

[0277] When the answer input is detected, the calculation unit 21 proceeds from step S259 to step S261, controls to determine whether the answer result is correct or wrong, and displays the result. Fig.16 As shown in the diagram, a result display unit 166 and a correct answer rate 167 are displayed.

[0278] Furthermore, the calculation unit 21 resumes the playback of the pitching video 110 in step S262, and performs the playback until the pitching action ends.

[0279] Note that the updating of the display of the result display unit 166 and the correct answer rate 167 may be performed together with the restart of the playback of the pitching video 110, or may be performed when the playback ends the pitching (the pointer 126).

[0280] Furthermore, the order of the processes in steps S261 and S262 may be reversed.

[0281] The correct / incorrect determination performed by the calculation unit 21 in step S261 includes the following two types of processing [Case 1] and [Case 2], and may be performed in either of the processing, or the user may select either of the processing.

[0282] [Example 1]

[0283] The remaining actually observed ball types in the score data SD are compared with the user's answer as the correct answer.

[0284] [Example 2]

[0285] The ball type is determined by referring to the information on the trajectory of the ball based on the EPTS data, and the determination result is compared with the user's answer as a correct answer.

[0286] In step S263, the calculation unit 21 determines whether the playback of all pitches on the current pitch table 150 has ended.

[0287] Note that even if playback of all pitches of current pitch table 150 has not yet been completed, calculation unit 21 may determine that playback processing related to current pitch table 150 is ended, for example, if condition settings are changed and load button 101 is operated.

[0288] If playback of current pitch table 150 has not ended, calculation unit 21 proceeds from step S263 to step S265 and randomly selects a pitch to be presented as next pitch video 110 from pitch table 150. For example, the remaining pitches that have not yet been selected are randomly selected.

[0289] Note that the user may be allowed to select the pitch to be displayed next.

[0290] Then, the calculation unit 21 proceeds to step S257, generates a CG image of the pitch selected in step S265, and displays the first frame of the pitch video 110. The same applies hereinafter.

[0291] By repeating steps S257 to S263 , the user can perform perception training on the pitches listed on the pitch table 150 .

[0292] When it is determined in step S263 that the replay of all pitches has ended, the computing unit 21 determines in step S264 that the display of the training screen 100 has ended, for example, that the application has ended by the user's operation, and returns to step S251 if the replay has not ended. In response to the end determination in step S264, the computing unit 21 ends the training screen 100. Fig.18 processing.

[0293] Through the above Fig.17 and Fig.18 The processing is performed in the terminal device 5 with reference to Figure 12 to Figure 15 The display of the training screen 100 is described. Therefore, the user can perform training for determining the habits of the pitcher.

[0294] Note that in the playback control of the pitching video 110 in step S258, step S262, etc., as processing by the application on the computing unit 21 or terminal device 5 side, controls such as stop, playback, fast rewind, fast forward, repeated playback, and change of the playback position of the drag bar 120 through the slider 121 are also performed in accordance with the user's operation.

[0295] In addition, as pitching video 110, images of pitches randomly selected in the order of pitching date and time or by user operation are sequentially displayed among actual pitches that meet the conditions listed in pitching table 150. In this case, the data of pitches in pitching table 150 may be filtered, and a pitch may be selected among the extracted pitches randomly, in the order of pitching date and time, or by user selection.

[0296] at the same time, Fig.16 The example of pitching video 110 from the referee's point of view is illustrated, and the setting of the event selection unit 102 is strike / ball determination.

[0297] In this case, the answer button 160 is a button for selecting "strike" or "ball".

[0298] Through such training screen 100, referees or trainee referees can practice making decisions.

[0299] Note that in the case of umpire training, the pointer 124 for pausing the pitching video 110 is set at a position such as behind the home plate. Alternatively, the user may be asked to select "strike" or "ball" after the pitching is completed without pausing.

[0300] That is, it is preferable that the presence or absence of stopping of pitching video 110 , the timing of stopping, and the timing of requesting an answer input differ according to the training items.

[0301] Furthermore, when the training item is strike / ball judgment, not only umpire training but also batsman training can be performed.

[0302] For example, the viewpoint is set to a right-handed batter or a left-handed batter, and the pitching video 110 from the batter's viewpoint is displayed. The pointer 124 is, for example, the timing of the pitching tunnel.

[0303] Then, the pitching video 110 is reproduced and stopped at the timing of the pointer 124 to ask the user to select "strike" or "ball". From the batter's point of view, the user selects "strike" or "ball" at the timing of deciding whether to swing the bat. Thus, the batter can be trained to improve his ability to select balls.

[0304] In the training of answering these "good balls" and "balls", the correct / incorrect judgment performed by the calculation unit 21 in step S261 includes the following two types of processing [Example 3] and [Example 4], and can be performed in either case, or the user can select either one.

[0305] [Example 3]

[0306] The information of the ball's trajectory based on the EPTS data is referred to, the strike zone is set based on the batter's skeleton and the home plate position, and a strike or a ball is determined based on whether the ball's trajectory passes through the strike zone. The determination result is compared with the user's answer as a correct answer.

[0307] [Example 4]

[0308] The remaining decisions in the actual game in the score data SD are compared with the user's answer as the correct answer. However, for a pitch for which no strike / ball decision was made due to a hit, the correct answer is determined in the above-mentioned [Example 3]. Alternatively, such a pitch may not be used as a sample (not listed on the pitch table 150).

[0309] Furthermore, in consideration of such training of strike / ball judgment, the name of the umpire may be set as a setting item, and pitches in the case of a specific umpire may be listed in the pitch table 150 .

[0310] In this case, the actual verdict is considered the correct response, as in [Example 4].

[0311] In this way, training can be implemented in which the habits of a particular referee's decisions can be perceived. For example, it is an exercise for each individual referee to judge the tendency to expand the strike zone on the outside corners, the tendency to shrink the strike zone overall, etc.

[0312] In addition, as a training item, an item of judging a pitching / pinning ball for practicing stealing a base can also be envisioned.

[0313] The viewpoint of pitching video 110 is set to first base, second base, etc. The samples listed in pitching table 150 include not only pitches to the batter but also pinballs.

[0314] The answer button 160 is a button for selecting "pitch" or "pin down the ball".

[0315] Through such training screen 100 , training for judging the pitcher's habit of pinning the ball from the baserunner's point of view can be performed.

[0316] <4. Summary and Modifications>

[0317] According to the above-described embodiment, the following effects can be obtained.

[0318] The information processing device 70 serving as the server device 2 according to the embodiment includes the calculation unit 21 which performs processing of generating action tendency information indicating an action tendency regarding a specific action (eg, pitching action) as a human action.

[0319] Thus, for example, information indicating a pitcher's pitching habits (e.g., the difference in motion when throwing a straight ball and when throwing a changeup) can be obtained, and this information can be used for research on the pitcher's pitches, practice of pitching by a batter, and the like.

[0320] In the embodiment, an example is described in which the calculation unit 21 generates action tendency information based on captured data such as skeleton shot data obtained from a captured image of a specific action.

[0321] For example, the calculation unit 21 generates an AI model to which a large amount of EPTS data of a pitching action of a certain pitcher is input and to which a motion tendency has been learned. The skeleton capture data makes the movement of the body during the pitching action clear and is suitable for learning the motion tendency.

[0322] Note that the captured data includes the expression captured data and the item captured data as described above, and the characteristics of the process of the pitching action are made clear by generating the action tendency information based on the expression captured data and the item captured data, which is suitable for learning the action tendency.

[0323] In the embodiment, an example is described in which the calculation unit 21 performs a process of generating display information based on the action tendency information.

[0324] For example, the computing unit 21 generates display information based on information obtained from an AI model that has learned the action tendency of a certain pitcher, and displays Figure 4 The tendency presentation screen 30 in the above-mentioned manner can provide the user with appropriate information for determining the habit of pitching, etc. through this display.

[0325] In the embodiment, an example is described in which the calculation unit 21 generates the display information indicating the importance of the feature amount used for action result prediction of a specific action as the display information based on the action tendency information.

[0326] For example, an important action model 64 indicating a focus point 65 that is determined from the AI ​​model of the pitcher and is likely to be a habit of the action, a feature quantity graph 66, etc. are displayed. Thus, the body part action that is highly likely to be determined as a habit of the pitching action is indicated, and information that helps to identify the pitcher's habit can be presented to the user.

[0327] In the embodiment, an example is described in which the calculation unit 21 generates the display information indicating the prediction accuracy of the action result prediction of the specific action as the display information based on the action tendency information.

[0328] For example, the prediction accuracy 61 of the pitch type by the AI ​​model of a certain pitcher, the time series change graph 62 indicating the change of the prediction accuracy, the accuracy 67 of each condition, etc. are displayed. The level of prediction accuracy from the pitching form is an indicator of whether the habit can be found from the pitching form. For example, in the case where the pitcher has no form habit for each pitch type, the value indicated by the prediction accuracy 61 is a low value, and in the case where the pitcher has a form habit for each pitch type, the value indicated by the prediction accuracy 61 is a high value. Therefore, the information on the prediction accuracy is also an indicator of whether the habit can be found through the important action model 64 and the feature quantity graph 66.

[0329] In the embodiment, an example is described in which the calculation unit 21 generates an action comparison image indicating each action result of a specific action as display information based on the action tendency information.

[0330] For example, the action comparison display unit 50 displays an action comparison image (bone models 51 and 52 or CG models 53 and 54) indicating the action in the respective cases of a straight ball and a breaking ball as a result of the action. Therefore, the user can intuitively understand the difference in pitching form depending on the ball type.

[0331] In the embodiment, an example is described in which the action comparison image is set as an image in which the difference point of the action generation of each action result is highlighted.

[0332] For example, in the action comparison display unit 50, a specific portion is clearly indicated as a highlight display 55. Therefore, the user can easily find the difference in pitching forms depending on the types of balls.

[0333] In the embodiment, an example is described in which the calculation unit 21 performs processing to generate display information based on the action tendency information having learned the feature amount of the action observable in the state of viewing from a specific viewpoint position.

[0334] For example, as the action of a certain pitcher, the calculation unit 21 sets the viewpoint position of a right-handed batter, for example, and generates display information of the tendency presentation screen 30 by using the AI ​​model that has learned the feature amount of the action observable from the viewpoint of the right-handed batter. Therefore, in consideration of the difference in viewpoints of a right-handed batter, a left-handed batter, a base runner, an umpire, etc., a tendency presentation screen 30 suitable for each viewpoint can be presented.

[0335] In the embodiment, an example is described in which the calculation unit 21 performs processing for generating display information based on the action tendency information in which the feature amount of an action observable in a part of the action time period of a specific action has been learned.

[0336] For example, as a pitching action of a certain pitcher, the calculation unit 21 divides the time period from the start to immediately after the ball is released into stages 0 to 3, and generates display information of the tendency presentation screen 30 by using, for example, an AI model that has learned the feature amount of the action that can be observed in a specified time period until stage 1. Therefore, the tendency presentation screen 30 focusing only on the action observed in a part of the time period of a specific action can be presented. For example, in the case of pitching, the display can be performed in consideration of the time period that the batter can actually observe.

[0337] In the embodiment, an example is described in which the calculation unit 21 performs a process of generating a dynamic image including a specific action and display information of an operation element by which an action result of the specific action instructed in the dynamic image is answered.

[0338] For example, the calculation unit 21 performs processing to display the training screen 100 including the pitching video 110 and the answer button 160. By displaying a dynamic image of the pitching action on such a training screen 100 and allowing the player to answer the pitch type, it is possible to perform perception training for pitch prediction. In particular, it is very useful as a countermeasure against the opposing pitcher of the opposing team to grasp the presence or absence of the pitcher's habits and important movements on the tendency presentation screen 30 and then actually perform perception training on the training screen 100.

[0339] In particular, after determining the pitcher's habits on the tendency presentation screen 30 , the user can perform training for predicting the type of pitch using the pitcher's CG image on the training screen 100 , thereby providing the user with an effective preparation environment for the game.

[0340] In the embodiment, the example has been described in which the pitching video 110 is generated based on the captured data obtained from the captured image of the actual specific action in the past.

[0341] For example, the computing unit 21 stores the skeleton capture data of each ball of a certain pitcher's past pitches, and generates a pitching video 110 as a CG image based on the skeleton capture data. Therefore, a pitching video 110 corresponding to an actual pitch of a specific pitcher can be displayed, which is suitable for training.

[0342] Furthermore, pitching video 100 may be generated not only based on skeleton capture data but also based on expression capture data, object capture data, etc. This is effective for displaying pitching video 110 corresponding to an actual pitch.

[0343] In the embodiment, an example is described in which control is performed to pause pitching video 110 at the determination timing after the start of playback and to resume playback in accordance with the answer of the action result.

[0344] That is, pitching video 110 is paused at the determination timing indicated by pointer 124 (e.g., the timing of the pitching tunnel), and the user can answer using answer button 160. Then, after the answer, the playback is resumed. This is suitable for training in which the ball type is determined based on the pitching form before the determination timing.

[0345] Furthermore, by changing the timing of pausing playback according to the purpose of training, an appropriate response timing can be set.

[0346] In the embodiment, an example is described in which the calculation unit 21 sets the determination timing based on the user input.

[0347] For example, the calculation unit 21 changes the pause timing by the user arbitrarily moving the pointer 124. This allows the user to specify the decision timing. For example, the difficulty level of the training can be adjusted by changing the decision timing to an earlier stage.

[0348] In the embodiment, an example is described in which calculation unit 21 generates pitching video 110 as a dynamic image of a pitching action observed from a designated viewpoint position.

[0349] The viewpoint position can be specified as a right-handed batter, a left-handed batter, a first base runner, an umpire, etc., and the pitching video 110 displayed can be made suitable for training purposes by generating a pitching video 110 with the specified viewpoint position. That is, the user can receive training at the viewpoint position in the user's actual game. For example, dynamic images suitable for prediction training of the type of ball from the viewpoint of a right-handed batter, prediction training of whether it is a pitch or a pinball from the viewpoint of a base runner, and judgment training of whether it is a strike or a ball from the viewpoint of an umpire can be displayed.

[0350] In the embodiment, an example is described in which calculation unit 21 performs a process of generating and displaying pitching video 110 by using data of a specific action randomly selected among data of a plurality of specific actions in the past extracted under a specified condition.

[0351] For example, a search is performed using a pitcher's name, time period / date and time, etc. as conditions, a pitching table 150 of the search results is obtained, and a pitch is randomly selected from the pitching table to generate a pitching video 110. Therefore, a pitching video 110 as a training problem can be displayed in a state where the user cannot predict actual pitches in the past.

[0352] In the embodiment, an example is described in which the calculation unit 21 performs a process of displaying the correct or incorrect result of an answer of an action result of a specific action.

[0353] By displaying the correct or incorrect results of the answers, the user can know how good the user's own judgment ability is.

[0354] Note that the determination of a correct or incorrect answer may be based on the type of ball actually thrown in the past pitching or the actual umpire's determination of a strike or a ball, or may be determined based on data. For example, the type of ball thrown or strike / ball may be determined based on the trajectory of the ball. These may be used differently depending on the purpose of the training.

[0355] In an embodiment, the specific motion is a pitching motion of a baseball or softball.

[0356] Generate action tendency information about the pitching action and display the tendency presentation screen 30 based on the action tendency information, or generate a CG dynamic image of the pitching action and use it as a pitching video 110 of the training screen 100, which is suitable for assisting or training the prediction of the pitching action of the batsman, base runner and umpire.

[0357] Note that operations other than the pitching motion of a baseball or softball may be set as specific operations of the present technology.

[0358] For example, the present invention can be applied to baseball or softball swings. For example, by analyzing a batter's swing and presenting characteristic quantities, important movements, etc., the analysis can be used to find the differences between good swings, bad swings, etc.

[0359] In addition, by setting the actions of other games and penalty kicks in soccer as specific actions, it is effective to provide information such as the habits of the kicker for the goalkeeper to predict the direction of the shot, and to perform prediction training for the goalkeeper. In addition, for example, operations such as serving in tennis, volleyball or table tennis can be used as specific operations.

[0360] Furthermore, the present invention is not limited to sports, but can also be applied to prediction of results and training of prediction of specific actions such as game actions and daily actions.

[0361] The processing related to the tendency presentation screen 30 and the processing related to the training screen 100 described in the embodiment are mainly performed by the control of the server device 2, but can be executed by the CPU 71 on the terminal device 5 side by defining an application program for these processing.

[0362] The program according to the embodiment is a program for causing a CPU, a digital signal processor (DSP), an AI processor, etc., or an information processing device 70 including a CPU, a DSP, or an AI processor to perform Fig.10 , Fig.11 , Fig.17 and Fig.18 The processing procedure is shown in the diagram.

[0363] That is, the program according to the embodiment is a program that causes the information processing apparatus 70 to execute a process of generating action tendency information indicating an action tendency regarding a specific action as a human action.

[0364] By such a program, the information processing device 70 constituting the information analysis system 1 according to the embodiment can be realized in, for example, a computer device, a mobile terminal device, or other devices capable of performing information processing.

[0365] Such a program may be recorded in advance in a HDD as a recording medium built in a device such as a computer device, a ROM in a microcomputer having a CPU, or the like.

[0366] Alternatively, the program may be temporarily or permanently stored (recorded) in a removable recording medium such as a floppy disk, a compact disk read only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such a removable recording medium may be provided as so-called packaged software.

[0367] Furthermore, such a program may be installed from a removable recording medium into a personal computer or the like, or may be downloaded from a download site via a network such as a local area network (LAN) or the Internet.

[0368] Furthermore, such a program is suitable for providing a wide range of information processing devices 70 included in the information analysis system 1 according to the embodiment. For example, the program is downloaded to mobile terminal devices such as smartphones, tablet computers, imaging devices, mobile phones, personal computers, game devices, video devices, personal digital assistants (PDAs), etc., so that smartphones, etc. can be used as the information processing device 70 constituting the information analysis system 1 of the present disclosure.

[0369] Note that the effects described in this specification are merely examples and are not limiting, and other effects may be provided.

[0370] Note that the present technology may also have the following configurations.

[0371] (1) An information processing device includes a calculation unit that performs processing to generate action tendency information indicating an action tendency regarding a specific action as a human action.

[0372] (2) The information processing device according to (1) above, wherein the calculation unit generates the action tendency information based on captured data obtained from a captured image of a specific action.

[0373] (3) The information processing device according to (1) or (2) above, wherein the calculation unit performs processing of generating display information based on the action tendency information.

[0374] (4) The information processing device according to (3) above, wherein the calculation unit generates, as the display information based on the action tendency information, display information indicating the importance of the feature amount used for action result prediction of the specific action.

[0375] (5) The information processing device according to (3) or (4) above, wherein the calculation unit generates display information indicating a prediction accuracy of an action result prediction of a specific action as the display information based on the action tendency information.

[0376] (6) The information processing device according to any one of (3) to (5) above, wherein the calculation unit generates an action comparison image indicating an action result of each specific action as display information based on the action tendency information.

[0377] (7) The information processing device according to (6) above, wherein the calculation unit sets the action comparison image as an image in which a difference point of action generation of each action result is highlighted.

[0378] (8) An information processing device according to any one of the above (1) to (7), wherein the computing unit performs processing to generate display information based on action tendency information, in which characteristic quantities of actions that can be observed when viewed from a specific viewpoint position have been learned.

[0379] (9) An information processing device according to any one of the above (1) to (8), wherein the computing unit performs processing to generate display information based on action tendency information, in which characteristic quantities of actions that can be observed in a part of an action time period of a specific action have been learned.

[0380] (10) An information processing device according to any one of the above (1) to (9), wherein the computing unit performs processing to generate a dynamic image including a specific action and display information of an operating element, and through the operating element, an action result of the specific action indicated in the dynamic image is answered.

[0381] (11) The information processing device according to (10) above, wherein the calculation unit generates a dynamic image based on captured data obtained from a captured image of an actual specific action in the past.

[0382] (12) The information processing device according to (10) or (11) above, wherein the calculation unit performs control to pause the dynamic image at the determination timing after the start of playback, and resume playback according to the answer of the action result.

[0383] (13) The information processing apparatus according to (12) above, wherein the calculation unit sets the determination timing based on a user input.

[0384] (14) The information processing device according to any one of (10) to (13) above, wherein the calculation unit generates the dynamic image as a dynamic image obtained by observing a specific action from a specified viewpoint position.

[0385] (15) An information processing device according to any one of (10) to (14) above, wherein the calculation unit performs processing for generating and displaying a dynamic image by using data randomly selected from data of a plurality of specific actions in the past extracted under specified conditions.

[0386] (16) The information processing device according to any one of (10) to (15) above, wherein the calculation unit performs processing to display a correct or incorrect result of an answer to an action result of a specific action.

[0387] (17) The information processing device according to any one of (1) to (16) above, wherein the calculation unit generates the action tendency information based on the skeleton capture data obtained from the capture image of the specific action.

[0388] (18) The information processing device according to any one of (1) to (17) above, wherein the specific motion is a pitching motion of a baseball or a softball.

[0389] (19) An information processing method includes executing, by an information processing device, a process of generating action tendency information indicating an action tendency regarding a specific action as a human action.

[0390] (20) An information analysis system comprising:

[0391] Imaging equipment;

[0392] a data generating unit that generates captured data based on an image captured by the imaging device regarding a specific action as a human action; and

[0393] A calculation unit that generates action tendency information indicating an action tendency regarding a specific action based on the captured data generated by the data generation unit.

[0394] Reference numerals list

[0395] 1Information Analysis System

[0396] 2. Server Equipment

[0397] 3Score Data Server

[0398] 4 Sensors

[0399] 5. Terminal Equipment

[0400] 10 Imaging equipment

[0401] 12EPTS data generation unit

[0402] 21 Computational Units

[0403] 21a Graphics Processing Unit

[0404] 21b Capture Data Processing Unit

[0405] 21c Score Data Processing Unit

[0406] 21d Analysis and Processing Unit

[0407] 22 Presentation Control Unit

[0408] 23 Storage control unit

[0409] 30 Tendency to present the picture

[0410] 70 Information processing equipment

[0411] 71CPU

[0412] 100 training screens

Claims

1. An information processing device comprising a calculation unit that performs processing to generate action tendency information indicating an action tendency regarding a specific action as a human action. 2 . The information processing device according to claim 1 , wherein the calculation unit generates the action tendency information based on captured data obtained from a captured image of a specific action. 3 . The information processing apparatus according to claim 1 , wherein the calculation unit performs a process of generating display information based on the action tendency information. 4 . The information processing apparatus according to claim 3 , wherein the calculation unit generates, as the display information based on the action tendency information, display information indicating the importance of the feature amount used for action result prediction of the specific action. 5 . The information processing device according to claim 3 , wherein the calculation unit generates display information indicating a prediction accuracy of an action result prediction of a specific action as the display information based on the action tendency information. 6 . The information processing device according to claim 3 , wherein the calculation unit generates an action comparison image indicating an action of each action result of a specific action as the display information based on the action tendency information. 7 . The information processing apparatus according to claim 6 , wherein the calculation unit sets the action comparison image as an image in which a difference point of action generation of each action result is highlighted.

8. The information processing device according to claim 1, wherein the calculation unit performs processing to generate display information based on action tendency information in which feature quantities of actions that can be observed in a state of viewing from a specific viewpoint position have been learned.

9. The information processing device according to claim 1, wherein the calculation unit performs processing to generate display information based on action tendency information in which a feature amount of an action that can be observed in a part of an action time period of a specific action has been learned.

10. The information processing device according to claim 1, wherein the calculation unit performs processing to generate a dynamic image including a specific action and display information of an operation element, and an action result of the specific action instructed in the dynamic image is answered through the operation element. 11 . The information processing apparatus according to claim 10 , wherein the calculation unit generates a dynamic image based on captured data obtained from a captured image of an actual specific action in the past.

12. The information processing device according to claim 10, wherein the calculation unit performs control to pause the dynamic image at a determination timing after the playback starts, and resume the playback according to an answer of the action result. 13 . The information processing apparatus according to claim 12 , wherein the calculation unit sets the determination timing based on a user input. 14 . The information processing apparatus according to claim 10 , wherein the calculation unit generates the dynamic image as a dynamic image obtained by observing a specific action from a designated viewpoint position. 15 . The information processing apparatus according to claim 10 , wherein the calculation unit performs a process of generating and displaying a dynamic image by using data randomly selected from data of a plurality of specific actions in the past extracted under a specified condition. 16 . The information processing apparatus according to claim 10 , wherein the calculation unit performs a process of displaying a correct or incorrect result of an answer of an action result of a specific action. 17 . The information processing apparatus according to claim 1 , wherein the calculation unit generates the motion tendency information based on skeleton capture data obtained from a capture image of a specific motion.

18. The information processing device according to claim 1, wherein the specific motion is a pitching motion of a baseball or a softball.

19. An information processing method, comprising executing, by an information processing device, a process of generating action tendency information indicating an action tendency regarding a specific action as a human action.

20. An information analysis system comprising: Imaging equipment; a data generating unit that generates captured data based on an image captured by the imaging device regarding a specific action as a human action; and A calculation unit that generates action tendency information indicating an action tendency regarding a specific action based on the captured data generated by the data generation unit.

Images