Information processing system, information processing device, terminal device, and program

Abstract

To indicate information on swing of a player in an easily understandable manner.SOLUTION: An information processing system processes information of a competition in which a player swings a tool to hit a moving object. The information processing system includes: acquisition means that acquires a moving image showing the player swinging the tool; and display means that displays information on an angle of the tool with respect to a track of the object specified using the moving image, together with a swing image showing the player swinging in the moving image.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, a terminal device, and a program. [Background technology]

[0002] Patent Document 1 discloses a baseball training system for baseball training, characterized by including multiple cameras that continuously record pitches thrown by a pitcher from the batter's viewpoint, a recording device that records the images captured by the cameras as multiple video files for each pitch, a display device that synchronously displays the multiple images recorded on the recording device on the same screen, and a controller that controls these. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-40071 Summary of the Invention [Problem to be solved by the invention]

[0004] In sports where players swing equipment to hit targets such as balls, instruction to improve a player's swing has largely depended on the instructor's ability, such as their experience and knowledge. In contrast, if the information necessary to improve a player's swing could be presented to the player in an easy-to-understand manner, it would be possible to improve the player's swing efficiently, regardless of the instructor's ability.

[0005] The present invention aims to present information about a player's swing in an easy-to-understand manner. [Means for solving the problem]

[0006] With this objective in mind, the technology disclosed in this specification is an information processing system that processes information about a sport in which an athlete swings an implement to hit a moving object, and is equipped with an acquisition means that acquires a video of the athlete swinging the implement, and a display means that displays information about the angle of the implement relative to the trajectory of the object, identified using the video, together with an image of the athlete swinging the implement in the video.

[0007] Here, the display means may display adaptation information relating to an adapted trajectory, which is the trajectory of the implement included within a range determined according to the trajectory of the object. The display means may also display a trajectory image showing the trajectory of the swing of the implement on the swing image, and may display the trajectory corresponding to the adapted trajectory in the trajectory image so as to be identifiable. The display means may also display, on the swing image, a contact position image indicating a contact position when the implement comes into contact with the target object. The display means may also display evaluation information for evaluating the positional relationship between the contact position and the adapted trajectory. The display means may also display, on the swing image, an object trajectory image showing the trajectory of the object moving toward the player and the trajectory of the object hit by the implement. Also, a guide image that guides at least the equipment and the athlete to the positions to which they should move may be displayed on the shooting screen of a shooting terminal that shoots the video of the athlete swinging the equipment. The video system may also include a detection means for detecting the player, the equipment, and the object from the video. The detection means may also detect the player, the target object, and the equipment using information about objects that are not moving in the video. The detection means may set a detection range for detecting the object in the video based on a swing trajectory of the implement. Furthermore, it is preferable to set a time period for analyzing the movements of the athlete and the equipment in the video based on information about the time when the equipment is in a specific state.

[0008] Furthermore, with this objective in mind, the technology disclosed in this specification is an information processing system that processes information about a sport in which an athlete swings equipment to hit a moving object, and is equipped with a guide display means that displays a guide image that guides at least the position in which the equipment and the athlete appear on the shooting screen of a shooting terminal that shoots a video of the athlete swinging the equipment, an acquisition means that acquires a video of the athlete swinging the equipment that is shot by the single shooting terminal, a display means that displays angle information regarding the angle of the equipment relative to the trajectory of the object, identified using the video, together with an image of the athlete swinging the equipment in the video, and an output means that outputs evaluation information about the athlete's swing of the equipment based on the angle information.

[0009] Furthermore, with this objective in mind, the technology disclosed in this specification is an information processing device that processes information about a sport in which an athlete swings an implement to hit a moving object, and that includes an acquisition means for acquiring a video of the athlete swinging the implement, and a display means for displaying information about the angle of the implement relative to the trajectory of the object, identified using the video, together with an image of the athlete swinging the implement in the video.

[0010] Furthermore, with this objective in mind, the technology disclosed in this specification is an information processing device that processes information about a sport in which an athlete swings an implement to hit a moving object, and is a terminal device that includes a filming means that shoots a video of the athlete swinging the implement, and a display means that displays information about the angle of the implement relative to the trajectory of the object, identified using the video, along with an image of the athlete swinging the implement in the video.

[0011] Furthermore, with this objective in mind, the technology disclosed in this specification is a program for enabling a computer that functions as an information processing device that processes information about a sport in which an athlete swings equipment to hit a moving object to acquire a video of the athlete swinging the equipment, and to display information about the angle of the equipment relative to the trajectory of the object, identified using the video, along with an image of the athlete swinging the equipment in the video.

[0012] Furthermore, with this objective in mind, the technology disclosed in this specification is a program for enabling a computer that functions as an information processing device that processes information about a sport in which an athlete swings an implement to hit a moving object to capture a video of the athlete swinging the implement, and to display information about the angle of the implement relative to the trajectory of the object, identified using the video, along with an image of the athlete swinging the implement in the video. [Effects of the Invention]

[0013] According to the present invention, information about a player's swing can be displayed in an easy-to-understand manner. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram of a practice support system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal device. [Figure 3] FIG. 2 illustrates an example of a hardware configuration of a server device. [Figure 4] FIG. 2 is a functional block diagram of a user terminal device according to the present embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a shooting guide displayed on a screen of a user terminal device. [Figure 6] FIG. 2 is a functional block diagram of a server device according to the present embodiment. [Figure 7] FIG. 10 is a schematic diagram of a batting motion analysis performed by the video analysis unit of the present embodiment. [Figure 8] 10A and 10B are explanatory diagrams of object detection performed by a video analysis unit of the present embodiment. [Figure 9] FIG. 10 is an explanatory diagram of an impact analysis performed by a video analysis unit of the present embodiment. [Figure 10] 10A to 10C are explanatory diagrams illustrating a rotational motion analysis performed by a video analysis unit of the present embodiment. [Figure 11] 10A and 10B are explanatory diagrams of a compatibility analysis performed by a video analysis unit of the present embodiment. [Figure 12] 10 is an explanatory diagram of a swing analysis video output by the analysis result output unit of the present embodiment. FIG. [Figure 13] 10A and 10B are explanatory diagrams of swing trajectory information output by the analysis result output unit of the present embodiment. [Figure 14] 10 is an explanatory diagram of rotation information output by an analysis result output unit of the present embodiment. FIG. [Figure 15] FIG. 10 is an explanatory diagram of comprehensive evaluation information output by the analysis result output unit of the present embodiment. [Figure 16] 10 is an explanatory diagram of reference information output by a reference information presenting unit of the present embodiment. FIG. [Figure 17] 10 is an explanatory diagram of personal ability evaluation information output by the analysis result output unit of the present embodiment. FIG. [Figure 18-1] 10 is an explanatory diagram of personal ability evaluation information output by the analysis result output unit of the present embodiment. FIG. [Figure 18-2] 10 is an explanatory diagram of personal ability evaluation information output by the analysis result output unit of the present embodiment. FIG. [Figure 19] 10 is an explanatory diagram of practice information output by a reference information presenting unit of the present embodiment. FIG. [Figure 20] 10 is an explanatory diagram of practice information output by a reference information presenting unit of the present embodiment. FIG. [Figure 21] 10 is a flowchart of the operation of the server device of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. <Practice Support System 1> FIG. 1 is a schematic diagram of a practice support system 1 according to this embodiment.

[0016] As shown in FIG. 1, the practice assistance system 1 of this embodiment includes a user terminal device 10 used by a user of the system, and a server device 30 that processes information related to sports competitions. The user terminal device 10 and the server device 30 are connected to each other via a network. The network is not particularly limited as long as it is a communication network used for data communication between the devices.

[0017] In the practice support system 1 of this embodiment, there is no particular limit to the number of user terminal devices 10. The user terminal devices 10 are used by the athletes competing in the competition or the instructors who train the athletes. For example, when multiple athletes on a team use this system, the practice support system 1 of this embodiment is connected with user terminal devices 10 corresponding to the number of athletes and instructors who make up the team. In this embodiment, when no particular distinction is made between athletes (batters) and coaches, users of the system will be referred to as "users" in the following description.

[0018] The practice support system 1 of this embodiment can be applied to sports in which the athlete hits a game ball by swinging a game tool, such as baseball, softball, cricket, golf, tennis, badminton, and table tennis. In this case, the game tool used by the athlete may be a bat, a club, a racket, or the like. The object to be hit with the game tool may be a ball, a shuttlecock, or the like.

[0019] In the description of the practice support system 1 of this embodiment, batting in baseball is used as an example. Specifically, the practice support system 1 captures a video of the batter swinging a bat to hit back a ball thrown at him. The practice support system 1 then provides the batter with information useful to him as a player and information related to batting.

[0020] However, if the coaching environment for a sports competition is not adequate, there is a possibility that the coach may force the athletes to practice or teach inefficiently.

[0021] Therefore, in the practice support system 1, for example, a video of a batter as an athlete swinging a batting ball is captured by the user terminal device 10, and the captured video is uploaded to the server device 30. The batter's swing can be captured by a single user terminal device 10 (a single camera 108). The server device 30 then analyzes the batter's batting and returns the analysis results to the user terminal device 10. Users, including the batter, can improve their swing motion, such as their batting form, by referring to the analysis results on the user terminal device 10.

[0022] The practice support system 1 of this embodiment acquires a video of a batter swinging a bat, and displays information about the angle of the bat relative to the trajectory of the ball, determined using the video, along with an image of the athlete's swing in the video. In this way, the practice support system 1 of this embodiment provides a practice support service to users who are batters or coaches. In other words, the practice support system 1 of this embodiment provides the athlete with objective information about their swing. The practice support system 1 of this embodiment will be described in detail below.

[0023] First, the hardware configuration of the user terminal device 10 and the server device 30 will be described.

[0024] (User terminal device 10) FIG. 2 is a diagram showing an example of the hardware configuration of the user terminal device 10. As shown in FIG.

[0025] As shown in FIG. 2, the user terminal device 10 includes a CPU 101, a RAM 102, a ROM 103, a storage unit 104, a display 105, a speaker 106, a microphone 107, a camera 108, and a network I / F 109.

[0026] The CPU 101 loads various programs stored in the ROM 103 and the like into the RAM 102 and executes them to realize various functions of the terminal device. The RAM 102 is a memory used as a working memory for the CPU 101. The ROM 103 is a memory that stores various programs executed by the CPU 101. The storage unit 104 is, for example, a flash memory that stores various application programs and various data used by the programs.

[0027] The display 105 displays a screen that conveys various information to the user, etc. The display 105 may also be, for example, a touch panel that accepts operation input from the user. The speaker 106 outputs various sounds including the received voice from the other party. The microphone 107 acquires various sounds including the user's speech. The camera 108 includes a lens and an imaging element (image sensor) such as a CMOS (Complementary Metal-Oxide Semiconductor) and captures an image of a subject. The network I / F 109 transmits and receives data to and from other communication devices, and communicates with the server device 30 and the like via the network.

[0028] The user terminal device 10 configured as described above allows the user to make phone calls and use Internet communication services. The user terminal device 10 can also take images (moving images, still images) using the camera 108. Furthermore, the user terminal device 10 can upload the taken images to, for example, the server device 30 using a communication service. On the other hand, the user terminal device 10 can obtain (download or stream) images from the server device 30 and display them on the display 105. Note that a moving image is made up of a plurality of still images (frame images). Therefore, in this embodiment, a moving image can be considered as an image. That is, in this embodiment, an image will be described as a concept that includes both a moving image and a still image.

[0029] (Server device 30) FIG. 3 is a diagram showing an example of the hardware configuration of the server device 30. As shown in FIG.

[0030] 3, the server device 30 includes a CPU (Central Processing Unit) 301, which is a computing means, a RAM (Random Access Memory) 302, which is a program execution area, and a ROM (Read Only Memory) 303, which is a storage area for storing programs such as a BIOS (Basic Input Output System). The server device also includes an HDD (Hard Disk Drive) 304, which is a storage area for storing various programs such as an OS (Operating System) and applications, input data for the various programs, output data from the various programs, etc. The server device also includes a communication interface (communication I / F) 305 for communicating with the outside.

[0031] Next, the functional configuration of the user terminal device 10 will be described. FIG. 4 is a functional block diagram of the user terminal device 10 of this embodiment.

[0032] As shown in FIG. 4, the user terminal device 10 includes a video shooting unit 11 that performs processing related to video shooting, and an image display unit 13 that displays analysis results, videos, and the like, which will be described later, on the screen.

[0033] The video shooting unit 11 performs various processes when shooting the batter for analysis. The video shooting unit 11 associates the video shot by the user terminal device 10 with the user's account information (described later) and transmits the video to the server device 30. Furthermore, video shooting unit 11 assists the user in shooting the batter's swing motion. Specifically, video shooting unit 11 displays shooting guide 200 (see FIG. 5 described later) on the screen. By displaying shooting guide 200, video shooting unit 11 provides guidance for shooting video so that video that can be analyzed by server device 30 or video that will result in higher analysis accuracy is shot.

[0034] The image display unit 13 displays on the screen the analysis results of the swing motion performed by the server device 30, a reference video containing information useful for practice, etc. The image display unit 13 displays on the screen according to the logged-in account.

[0035] Next, the shooting guide 200 displayed on the screen by the video shooting unit 11 will be described. FIG. 5 is a diagram showing an example of a shooting guide displayed on the screen of the user terminal device 10. As shown in FIG.

[0036] 5, shooting guide 200 (an example of a guide image) has a base area guide 210, a batter area guide 220, a bat area guide 230, and a ball area guide 240. When shooting batter P batting, shooting guide 200 indicates the position of each element to be shot within the video screen.

[0037] The base area guide 210 uses a frame image to indicate the area in which home base H should be captured. By specifying that home base H should be captured, the base area guide 210 ensures that the video captured includes the feet of the batter P, who move up and down in response to his batting movements. The batter area guide 220 uses a frame image to indicate the area in which the batter P should be photographed. The batter area guide 220 specifies that the batter P should be photographed, so that a video that includes the entire body of the batter P, who is the subject of analysis, is captured. The bat area guide 230 uses a frame image to indicate the area where the bat T should be photographed. The bat area guide 230 specifies that the bat T should be photographed, thereby allowing a video that includes the entire bat T, which is the subject of analysis, to be shot.

[0038] The ball region guide 240 uses a frame image to indicate the region in which the pitched ball thrown toward the batter P and the batted ball hit by the batter P should be photographed. In this embodiment, the ball region guide 240 is set to an area extending from home base H to a predetermined distance (for example, one meter) in the direction of the batted ball. This allows a video to be captured that includes the trajectory of the thrown ball B before and after the impact when the ball B is hit with the bat T.

[0039] In the shooting guide 200 of this embodiment, the elements of the object to be shot (for example, "batter," "bat," "home base," "pitching zone," etc.) are displayed in text for each area guide. This makes the user aware that when shooting a video of a batting game, they need to shoot at least the elements displayed in text.

[0040] 5 shows an example of a photography guide 200 for a right-handed batter who is positioned to the right of home base H as seen from the pitcher's side. Depending on the selection of a setting button image (not shown), photography guide 200 displays an image for a left-handed batter, which is the opposite of a right-handed batter. In addition, the shooting guide 200 can automatically switch between a guide for right-handed batters and a guide for left-handed batters without accepting a user selection, depending on image analysis of the orientation of home base H included in the captured image.

[0041] Furthermore, the photography guide 200 may detect elements included in the photographed image and determine whether or not the image contains elements corresponding to each of the area guides of the photography guide 200. If the photography guide 200 does not contain an element corresponding to an area guide, or if the element is not included in its entirety but only in part, the photography guide 200 may output a notification (error) to the user.

[0042] In this way, the user uses the user terminal device 10 to film a batting scene in which batter P is batting. In this embodiment, the filming is performed using only a single user terminal device 10. When filming is performed using the user terminal device 10, the user terminal device 10 is fixed in position and not moved.

[0043] Next, the functional configuration of the server device 30 will be described. FIG. 6 is a functional block diagram of the server device 30 of this embodiment.

[0044] The server device 30 of this embodiment includes an account management unit 31 that manages the accounts of users who use this service, a video analysis unit 33 that analyzes the batter's athletic movements from video, an analysis result output unit 35 that outputs the analysis results, and a reference information presentation unit 37 that presents reference information to the user.

[0045] The account management unit 31 manages the accounts of users who use this service. An account can be created, for example, for each user who uses this service or for each team consisting of multiple users. In this embodiment, various information such as captured videos and analysis results is managed in association with the account.

[0046] The video analysis unit 33 analyzes the video obtained by filming the batter P swinging the bat. The video analysis unit 33 of this embodiment performs (1) batting motion analysis, (2) impact analysis, (3) landing analysis, (4) line of sight analysis, (5) rotational motion analysis, and (6) compatibility analysis. Then, the video analysis unit 33 sends the analysis results based on each viewpoint to the analysis result output unit 35.

[0047] (1) Batting motion analysis FIG. 7 is a schematic diagram of the batting motion analysis performed by the video analysis unit of this embodiment.

[0048] The video analysis unit 33 analyzes the movements of the batter P from the video captured by the user terminal device 10. As shown in FIG. 7, the video analysis unit 33 performs object detection in the filmed video. Specifically, the video analysis unit 33 uses object detection AI (artificial intelligence) to detect the batter P, bat T, and ball B in the video. In this embodiment, the video analysis unit 33 can use machine learning such as a support vector machine or an algorithm that uses deep learning such as YOLO as the detection algorithm for the batter P, bat T, and ball B. For example, YOLO detects the batter P, bat T, and ball B from the filmed video using a trained model obtained by deep learning using training data consisting of a large number of images to which correct labels (teaching information) of feature objects such as the batter and the bat are attached.

[0049] The video analysis unit 33 estimates the two-dimensional postures of the batter P and the bat T from the object-detected image information of the batter P and the bat T, and extracts two-dimensional skeletal data of the batter P and the bat T. In this embodiment, the video analysis unit 33 can use a two-dimensional posture estimation AI using a posture estimation model such as HRNet (high-resolution net) to identify the two-dimensional postures of the batter P and the bat T.

[0050] The video analysis unit 33 then estimates a three-dimensional posture from the two-dimensional skeletal data and extracts three-dimensional skeletal data of the batter P and the bat T. In this embodiment, the video analysis unit 33 can use a three-dimensional posture estimation AI such as GAST (Graph Attention Spatio-temporal Convolutional Network) to identify the three-dimensional postures of the batter P and the bat T. For example, for a video of a batting game, a trained model is created using a large amount of data for machine learning that associates the two-dimensional skeletal data of the batter P with three-dimensional motion information obtained by attaching a pointing device to the batter P. The video analysis unit 33 can use this trained model to obtain three-dimensional skeletal data from the two-dimensional skeletal data.

[0051] Next, object detection from a competition video executed by the video analysis unit 33 of this embodiment will be described. FIG. 8 is an explanatory diagram of object detection performed by the video analysis unit 33 of this embodiment.

[0052] As shown in Fig. 8, a video of a batting game contains various elements, including a batter P. In this embodiment, the video analysis unit 33 applies a filter that selects elements contained in the image in order to accurately detect the batter P, bat T, and ball B from the video. Then, the video analysis unit 33 detects the batter P, bat T, and ball B from the video.

[0053] The video analysis unit 33 performs a process of excluding objects that do not move in the video from the object detection targets. That is, the video analysis unit 33 compares the positions of the detection target objects included in each of the frame images that make up the video among multiple frame images. The video analysis unit 33 then excludes objects whose positions do not change among multiple frame images from the analysis targets. The video analysis unit 33 of this embodiment uses information about objects that do not move in the captured video to detect the moving batter P, bat T, and ball B.

[0054] In the image shown in Figure 8, for example, the net 41 and posts 43 of a ball-stopping netting device that protects people in the vicinity from batted balls do not move in position across multiple frame images. Therefore, the video analysis unit 33 excludes the net 41 and posts 43 from the analysis target. The net 41 and posts 43 are similar in shape to the bat T, which is a detection target that is actively desired to detect in this embodiment. The video analysis unit 33 of this embodiment improves the detection accuracy of the bat T by excluding images of linear objects such as the net 41 and posts 43 that have a similar shape to the bat T.

[0055] Furthermore, the video analysis unit 33 adds a condition to the detection of the bat T that the bat T must be a rod-shaped object closest to the wrist of the batter P. Objects similar to the bat T, such as a support pole 43 and a net 41, appear in the captured video. In order to accurately detect the bat T from such a captured video, the video analysis unit 33 also uses the detected image of the batter P to detect the bat T.

[0056] 8, the position of the ball 45 rolling on the ground does not change in the multiple frame images. Therefore, the video analysis unit 33 excludes these balls 45 from the analysis target. The ball 45 lying on the ground has the same shape as the moving ball B (pitched or batted), which is the object to be actively detected in this embodiment. The video analysis unit 33 of this embodiment improves the detection accuracy of the moving ball B by excluding the object image of the ball 45 on the ground.

[0057] Then, the video analysis unit 33 detects the ball B (pitched or batted) moving toward the batter P in the video. As shown in FIG. 8, the video analysis unit 33 of this embodiment sets a detection target area FW (an example of a detection range) in the video (image) to be analyzed in order to identify the moving ball B. The detection target area FW can be set as an area extending from the lowest point in the trajectory of the bat T identified from the image to a predetermined height (for example, 0.5 meters vertically upward from the lowest point). In this way, in order to detect the moving ball B, the video analysis unit 33 sets the detection target area FW in the captured video based on the swing trajectory ST of the bat T (see FIG. 11 described below). Then, the video analysis unit 33 spatially limits the target of analysis processing for the captured video. This system improves the detection accuracy of the ball B while reducing the load on the system related to the analysis (the amount of data of the image to be processed).

[0058] Furthermore, the video analysis unit 33 sets a period (an example of a time period) to be analyzed in the video (image) in order to identify the batter P, the bat T, and the ball B in the video. In this embodiment, the time period to be analyzed in the video is a few seconds before and after the lowest point timing T0, when the trajectory of the bat T is at its lowest (lowest point). The video analysis unit 33 sets an analysis start time T1 and an analysis end time T2 (see FIG. 10, described below). By setting a time period in this way to identify the batter P, the bat T, and the ball B in the video, the image to be processed is temporally limited. Furthermore, this system reduces the load on the system related to analysis (the amount of data of the images to be processed).

[0059] Note that the reference time point used to set the time period for video analysis may be the impact timing IT in addition to the lowest point timing T0. In this way, the video analysis unit 33 can set the time period for analyzing the movements of the batter P and the bat T in the filmed video based on information about the time when the bat T reaches a specific state.

[0060] As described above, the video analysis unit 33 of this embodiment detects the batter P, bat T, and ball B as objects contained in the video from two-dimensional video data of a batting scene captured by a single user terminal device 10. The video analysis unit 33 then identifies the two-dimensional movements of the detected objects. Furthermore, the video analysis unit 33 can also identify the three-dimensional movements of the detected objects. In this way, the practice support system 1 of this embodiment is capable of identifying the two-dimensional and three-dimensional movements of the batter P, bat T, and ball B from a video of a batting scene captured by a single user terminal device 10, such as a smartphone.

[0061] (2) Impact analysis FIG. 9 is an explanatory diagram of the impact analysis executed by the video analysis unit 33 of this embodiment.

[0062] The video analysis unit 33 performs an analysis of the impact, which is an event that occurs when the ball B is hit (collides) with the bat T, from the two-dimensional motion information obtained by the above-described batting motion analysis. The video analysis unit 33 of this embodiment identifies the impact timing IT (see FIG. 10, which will be described later) and the impact point IP in the image (see FIG. 11, which will be described later).

[0063] The video analysis unit 33 performs object detection of ball B in multiple frame images, which are images taken at different points in time. Then, for each of the multiple frame images, the unit identifies the ball position, which is the position of ball B in the frame image. Then, as shown in FIG. 9, the multiple ball positions in the multiple frame images are displayed in a superimposed manner. Here, for example, the ball position as a pitched ball is displayed on the image as pitched ball position b1, pitched ball position b2, pitched ball position b3, ..., pitched ball position b7. Furthermore, the ball position as a batted ball is displayed as batted ball position r1, batted ball position r2, batted ball position r3. Note that a pitched ball and a batted ball can be distinguished by the direction of movement of ball B, which is identified by comparing the multiple frame images.

[0064] Then, a pitch line BL is identified that passes through multiple pitching positions (centers) indicating ball B, which is a pitched ball. The pitch line BL corresponds to the trajectory of the pitched ball. On the other hand, a batted ball line RL is identified that passes through multiple batted ball positions (centers) of ball B, which is a batted ball. The batted ball line RL corresponds to the trajectory of the batted ball.

[0065] Furthermore, the intersection XP where the pitch line BL and the batted ball line RL intersect is identified. The intersection XP can be obtained as specific coordinates corresponding to the point where the intersection XP is located after setting coordinates on the image shown in Figure 9. The position of the specific coordinates identified as this intersection XP becomes the impact point IP. The time when ball B is located at the intersection XP is identified. The speed of ball B can be identified from multiple frame images. The time when ball B is located at the intersection XP is the impact timing IT.

[0066] As described above, the video analysis unit 33 of this embodiment can identify the impact point IP and the impact timing IT in the image from the video.

[0067] It is conceivable that the impact point IP and impact timing IT could be directly determined from an image by detecting the ball B in a video. However, the moment of impact is when the ball B collides with the bat T, and the ball B is significantly deformed and comes into close contact with the bat T. This makes it extremely difficult to detect the ball B from an image. Therefore, in the practice support system 1 of this embodiment, an image of the ball B moving without contacting the bat T is used to indirectly identify the impact point IP and the impact timing IT.

[0068] (3) Landing analysis The video analysis unit 33 identifies the timing of the batter's landing from the two-dimensional motion information obtained by the above-mentioned batting motion analysis. There is a batting form in which a batter steps with his or her foot on the side of the ball raised, then places his or her foot on the ground again before rotating his or her body and the bat. When a batter uses this type of batting form, the video analysis unit 33 of this embodiment identifies the timing from when the batter's foot is raised to when the batter lands from the image. The video analysis unit 33 sends information about the timing of when the batter raised his or her foot and when the batter landed to the analysis result output unit 35.

[0069] (4) Eye movement analysis The video analysis unit 33 identifies the width and angle of the eye movement from when the batter lifts his foot until he lands, based on the two-dimensional movement information from the above-mentioned batting movement analysis. The video analysis unit 33 identifies the batter's eye position based on features that are relatively easy to detect in an image, such as the outline of the face or the brim of the helmet. It then calculates the eye positions from when the batter lifts his or her foot until he or she lands. The video analysis unit 33 then sends information about the identified width and angle of eye movement to the analysis result output unit 35.

[0070] (5) Rotational motion analysis FIG. 10 is an explanatory diagram of the rotational motion analysis performed by the video analysis unit 33 of this embodiment.

[0071] The video analysis unit 33 determines the rotational motion analysis of the batter and the bat from the three-dimensional motion information obtained by the above-mentioned batting motion analysis. As shown in Figure 10, the graph visualizes the rotational speed of the batter and the bat. The vertical axis of the graph represents the rotational speed (degrees / second) according to the rotation direction, and the horizontal axis represents an arbitrary elapsed time. Then, from the three-dimensional motion information of the batter and the bat, the video analysis unit 33 determines the rotational speed of the batter's hips, shoulders, arms, and bat, respectively, in time series.

[0072] 10 shows the results of rotational motion analysis including a time period other than the moment of batting. Of these, the time period that is actually the subject of rotational motion analysis is the time period when the bat T is being swung to hit the ball B. Therefore, the video analysis unit 33 of this embodiment targets the rotational motion analysis from the analysis start time T1 to the analysis end time T2, which is set based on the lowest point timing T0.

[0073] As described above, the video analysis unit 33 can identify the rotational speed of the batter (hips, shoulders, arms) and the bat. The video analysis unit 33 can then compare the rotational power and timing of the peak rotational speed of each element of the batter's body. In particular, the coordination achieved by the order of rotation of the body elements is considered to be important in batting. The video analysis unit 33 of this embodiment can also evaluate the coordination of body rotation during batting.

[0074] (6) Conformance analysis FIG. 11 is an explanatory diagram of the compatibility analysis performed by the video analysis unit 33 of this embodiment.

[0075] The video analysis unit 33 identifies the relationship of the swing trajectory ST of the bat T with respect to the trajectory of the pitched ball from the two-dimensional and three-dimensional motion information obtained by the above-mentioned batting motion analysis. Here, in batting, it is preferable to align the swing trajectory ST of the bat T with the pitching trajectory (line) of the ball in order to more accurately hit the ball B with the bat T or to more effectively transmit power from the bat T to the ball B. Therefore, the video analysis unit 33 of this embodiment identifies the angle of the bat T with respect to the pitching trajectory based on the above-mentioned batting motion analysis, allowing the user to understand the swing of the bat T.

[0076] The video analysis unit 33 of this embodiment identifies the relationship of conformity between the swing trajectory ST and the pitching trajectory. 11, the video analysis unit 33 can determine the pitch line BL from the image, similar to the impact analysis described above. The video analysis unit 33 can also identify the swing trajectory ST of the bat T. Furthermore, the video analysis unit 33 can also identify the impact point IP.

[0077] In this embodiment, a first reference line SL1 that forms an angle of +5 degrees with respect to the pitching line BL and a second reference line SL2 that forms an angle of -5 degrees with respect to the pitching line BL are set as standards for conformance of the swing trajectory ST of the bat T with respect to the pitching line BL. FIG. 11 shows, as an example, the first reference line SL1 and the second reference line SL2 that pass through the impact point IP. Note that, in this embodiment, the comparison of the angle between the pitching trajectory (pitching line BL) and the swing trajectory ST is determined using the vertical angle component.

[0078] The video analysis unit 33 then analyzes compatibility information related to the compatibility of the swing trajectory ST of the bat T with the pitching trajectory (pitching line BL). In this embodiment, the video analysis unit 33 identifies the angle of the bat T included in the range from the first reference line SL1 to the second reference line SL2, with the pitching line BL in between, as the compatible trajectory CT. In other words, the video analysis unit 33 determines that the trajectory of the swing trajectory ST that is within a range of +5 degrees and -5 degrees of the pitching line BL, which corresponds to the pitching trajectory, is the compatible trajectory CT. In this way, the compatibility information is the trajectory of the bat T included in a range determined according to the trajectory of the ball B.

[0079] Then, the video analysis unit 33 can set coordinates on the image and obtain the location corresponding to the adaptive trajectory CT as specific coordinates. The video analysis unit 33 can also identify the positional relationship between the adaptive trajectory CT and the impact point IP. Furthermore, the video analysis unit 33 identifies an adaptive ratio, which is the ratio of the adaptive trajectory CT to the length of the bat swing trajectory ST. Then, the video analysis unit 33 can identify the adaptive time corresponding to the adaptive trajectory CT within the bat swing time.

[0080] Next, the analysis result output unit 35 of this embodiment will be described in detail. The analysis result output unit 35 outputs the analysis result created based on the analysis of the moving image by the moving image analysis unit 33 to the user.

[0081] FIG. 12 is an explanatory diagram of a swing analysis video 500 output by the analysis result output unit 35 of this embodiment.

[0082] The analysis result output unit 35 outputs a swing analysis video 500 in which information obtained by analyzing the video, such as the swing trajectory ST and the trajectory of the ball B, is superimposed on the video of the batter P.

[0083] 12, swing analysis video 500 is displayed on the screen of user terminal device 10. In swing analysis video 500, a captured video 510, a trajectory image 520, a ball trajectory image 530, an impact position image 540, and a detailed fitting information display 550 are displayed.

[0084] The filmed video 510 (an example of a swing image) is a video taken when filming for analyzing the swing of the bat T, etc. The practice support system 1 of this embodiment shows the user the analysis results of the swing trajectory ST and the trajectory of the ball B. The swing analysis video 500 does not simply show the analysis results of the swing, but rather displays the analysis results obtained by analyzing the filmed video 510 superimposed on the filmed video 510 of the batting used for the analysis.

[0085] Trajectory image 520 (an example of a trajectory image) includes a swing trajectory image 521 showing the trajectory of the swing of bat T obtained by analyzing photographed video 510. Swing trajectory image 521 is displayed superimposed on photographed video 510 as a line (curve) corresponding to swing trajectory ST.

[0086] Furthermore, the trajectory image 520 includes an adapted trajectory image 523 that indicates an adapted trajectory CT, which is information related to the angle of the bat T obtained by analyzing the captured video 510. The adapted trajectory CT is part of the swing trajectory ST. However, the adapted trajectory CT is the swing trajectory of the bat T, which is particularly important in batting. Therefore, in this embodiment, the adapted trajectory CT that constitutes the swing trajectory ST is displayed as the adapted trajectory image 523 in a different color and line thickness from the other trajectories, making it identifiable. Furthermore, the swing trajectory image 250 includes a text image of "adaptive trajectory" that explains what the adapted trajectory image 523 indicates.

[0087] Ball trajectory image 530 (an example of an object trajectory image) shows the trajectory of ball B obtained by analyzing captured video 510. Ball B exists as a pitched ball before being hit with bat T, and a batted ball after being hit with bat T. Ball trajectory image 530 includes pitched ball trajectory image 531 showing the trajectory of ball B as a pitched ball before being hit with bat T, and batted ball trajectory image 532 showing the trajectory of ball B as a batted ball after being hit with bat T. Pitched ball trajectory image 531 and batted ball trajectory image 532 are displayed superimposed on captured video 510.

[0088] The impact position image 540 (an example of a contact position image) is an image showing the position of the impact point IP, which is the position where the bat T collides with the ball B. The impact position image 540 includes a text image of "impact point" that explains what the impact position image 540 shows.

[0089] The compatibility information detail display 550 displays details of compatibility information regarding the compatibility of the swing trajectory ST with the pitching trajectory obtained by analyzing the captured video 510. The adaptation information detailed display 550 of this embodiment displays adaptation time information 551 and adaptation rate information 552 together with the captured video 510. The adaptation time information 551 indicates the swing time during which the bat T is in line with the adaptation trajectory CT. The adaptation rate information 552 indicates the proportion of the swing trajectory ST of the bat T that is in line with the adaptation trajectory CT.

[0090] Furthermore, the conformance information detail display 550 (an example of evaluation information) displays evaluations regarding the conformance trajectory CT. Specifically, the conformance information detail display 550 displays trajectory evaluation information 553 and timing evaluation information 554. The trajectory evaluation information 553 is information that evaluates whether the suitable trajectory CT, which is the angle of approach of the bat T relative to the pitched ball trajectory, is appropriate. Furthermore, the longer the suitable trajectory CT, the better. Therefore, the trajectory evaluation information 553 also includes evaluation information regarding the length of the suitable trajectory CT. Specifically, the trajectory evaluation information 553 displays a text image saying, "The bat is long on the line, Good!"

[0091] The timing evaluation information 554 is information that evaluates the positional relationship between the impact point IP and the suitable trajectory CT. Specifically, when the suitable trajectory CT is located in front of the impact point IP, a text image stating "suitable trajectory is before impact, Good!" is displayed as the timing evaluation information 554. On the other hand, when the suitable trajectory CT is located behind the impact point IP (toward the pitching ball), a text image stating "suitable trajectory is after impact, NG" is displayed as the timing evaluation information 554.

[0092] The user can check the impact position image 540 and the adapted trajectory image 523 on the screen. Furthermore, in this embodiment, the adapted information detail display 550 also indicates to the user in text whether or not the adapted trajectory CT has been created with the swing of the batter P.

[0093] Then, in swing analysis video 500 configured as described above, filmed video 510, trajectory image 520, ball trajectory image 530, and impact position image 540 can be displayed as a moving image to the user. That is, when played back as a moving image, batter P, bat T, and ball B in filmed video 510 move over time. Furthermore, swing trajectory image 521 and fitted trajectory image 523 of trajectory image 520 are sequentially displayed to follow the movement of bat T in filmed video 510. Similarly, ball trajectory image 530 is also sequentially displayed to follow the movement of ball B in filmed video 510.

[0094] FIG. 13 is an explanatory diagram of swing trajectory information 600 output by the analysis result output unit 35 of this embodiment.

[0095] The analysis result output unit 35 outputs swing trajectory information 600 including information relating to the swing trajectory of the bat.

[0096] 13, swing trajectory information 600 is displayed on the screen of user terminal device 10. Swing trajectory information 600 displays captured video 610, a three-dimensional representation 620 of batter P and bat T, swing specification information 630, and check items 640 related to the swing.

[0097] The captured video 610 is a captured image of the batter P used to analyze the swing, etc. The user can play back the captured video 610 repeatedly or from any position.

[0098] The three-dimensional display 620 is a display of the movements of the batter P and bat T based on three-dimensional skeletal information obtained by analyzing the filmed video 610. In the three-dimensional display 620, the bat T and the batter P are expressed as so-called rod-like displays. The three-dimensional display 620 also displays the trajectory of the approximate tip of the bat T. By viewing the swing movement based on the simplified three-dimensional skeletal information display, the user can easily understand the batting form of the batter P.

[0099] The specification information 630 is the result of the movement of the batter P obtained by analyzing a three-dimensional image of the batter P. The specification information 630 of this embodiment discloses information on "swing time," "level swing time," "upper angle," and "pull angle." Specifically, the swing time is the time from the time the bat lands to the impact timing IT. The level swing time is the time during which the bat T is swung parallel to the ground. The upper angle is the angle of the bat T in the direction of the ball's launch (upward relative to the ground) at the time of impact. The pull angle is the angle of the bat T in the left-right direction of the hit ball relative to the pitching direction at the time of impact. The "level swing time," "upper angle," and "pull angle" are each displayed as numerical values.

[0100] Check item 640 indicates batter P's evaluation of his swing. In this embodiment, check item 640 evaluates batter P using a four-point scale. The evaluation items in check item 640 are "swing time" and "swing trajectory." Swing time indicates the time the bat T is swung. A shorter swing time can be considered to indicate a higher swing speed. Swing trajectory is an example of an item used to evaluate whether the swing form makes it easier for batter P to hit ball B than, for example, a level swing, i.e., whether it increases the batting average.

[0101] FIG. 14 is an explanatory diagram of rotation information 700 output by the analysis result output unit 35 of this embodiment.

[0102] The analysis result output unit 35 outputs rotation information 700 including information relating to the swing trajectory of the bat.

[0103] 14, rotation information 700 is displayed on the screen of user terminal device 10. Rotation information 700 displays a captured image 710, swing specification information 730, and check items 740 related to the swing.

[0104] The captured image 710 is a captured image of the batter P used to analyze the swing, etc. In the example shown in Fig. 14, a still image at a specific timing from the captured video is displayed. The captured image 710 is displayed, for example, at the time of maximum hip rotation, maximum shoulder rotation, and maximum arm rotation.

[0105] The specification information 730 displays the speed at which the hips, shoulders, and arms rotate at their maximum. The user can check the specific maximum rotation speed of the hips, shoulders, and arms in numerical form in order to ultimately improve the rotation speed of the bat swung by the arms.

[0106] Check item 740 indicates an evaluation of batter P regarding his swing. In this embodiment, check item 740 evaluates batter P using a four-point scale. The evaluation items in check item 740 are "kinetic chain," "rotational power," and "posture at landing." "Kinetic chain" is an item that evaluates whether the maximum rotational speed is transferred in order from the waist, to the shoulders, and to the arms. Rotational power is an item that evaluates the magnitude of the rotational force, represented by the rotational speed. "Posture at landing" is an item that evaluates batter P's posture at landing based on the position of his line of sight.

[0107] FIG. 15 is an explanatory diagram of the comprehensive evaluation information 800 output by the analysis result output unit 35 of this embodiment.

[0108] The analysis result output unit 35 outputs comprehensive evaluation information 800 including information relating to the bat swing trajectory.

[0109] 15, overall evaluation information 800 is displayed on the screen of user terminal device 10. The overall evaluation information 800 displays a captured video 810, a rotational power evaluation 820, a linkage evaluation 830, a swing time evaluation 840, an eye line evaluation 850, and ranking information 860.

[0110] The filmed video 810 is a filmed image of the batter P used to analyze the swing, etc. The filmed video 810 can be played back repeatedly by the user, or can be played back from any position.

[0111] The rotational power evaluation 820 is an item that objectively evaluates the level of batter P in terms of rotational power, such as rotational speed. In this embodiment, three indicators of rotational power, for example, "independent," "professional level," and "major level," are shown using a graph. In this example, the levels of other players, such as independent league players, professional baseball league players, and major league players, are used as a guide.

[0112] The linkage evaluation 830 is an item for objectively evaluating whether the rotation of the hips, shoulders, and arms is smoothly linked. As with the rotational power evaluation, the level of batter P is shown relative to the levels of other players.

[0113] The swing time evaluation 840 is an item for objectively evaluating the swing time related to the swing speed. As with the rotational power evaluation, the level of the batter P is shown in relation to the levels of other players.

[0114] The gaze evaluation 850 is an item that objectively evaluates the vertical movement of the gaze of batter P. As with the rotational power evaluation, the level of batter P is shown relative to the levels of other players.

[0115] The ranking information 860 shows the results of a comparison with other batters who use the service in terms of the rotational power, motility, swing time, and eye line evaluation described above. The ranking information 860 can use rankings based on the data of all batters who use the service.

[0116] The reference information presenting unit 37 presents reference information such as helpful videos and advice to the batter P based on the analysis of the batting of the batter P by the video analyzing unit 33.

[0117] FIG. 16 is an explanatory diagram of reference information 900 output by the reference information presenting unit 37 of this embodiment.

[0118] 16, reference information 900 is displayed on the screen of the user terminal device 10. In the reference information 900, a reference video 910 and advice information 920 are displayed.

[0119] The reference information presenting unit 37 of this embodiment can evaluate the batting of the batter P as described above by analyzing the video of the batting of the batter P. Then, for example, for an item with a low evaluation, the reference information presenting unit 37 presents on the screen reference information for improving that item.

[0120] The reference video 910 displays a video that serves as a reference for, for example, eye direction, posture upon landing, and coordination. These videos may be stored in the database of the practice support system 1 of this embodiment or may be displayed as videos that are publicly available on a website or the like. Furthermore, videos of other batters previously filmed by this system may be used. In this case, the videos of other batters that have been highly evaluated through video analysis may be used.

[0121] The advice information 920 specifically displays in a text message to batter P issues that should be prioritized for future practice, such as the need to improve rotation speed. The advice information 920 can adopt information that serves as a reference for items that are poorly evaluated as a result of an analysis of batter P's batting form.

[0122] 17, 18-1 and 18-2 are explanatory diagrams of the individual ability evaluation information 940 output by the analysis result output unit 35 of this embodiment.

[0123] The analysis result output unit 35 outputs individual ability evaluation information 940 including information related to the batter P's swing.

[0124] 17, the individual ability evaluation information 940 is displayed on the screen of the user terminal device 10. The individual ability evaluation information 940 displays a radar chart 941 and detailed analysis information 960.

[0125] For example, the radar chart 941 uses a radar chart graph to grade the evaluation of five items for each batter P: 1. pelvic rotation, 2. thoracic rotation, 3. whole-body coordination, 4. bat trajectory, and 5. ball vision. Specifically, the radar chart 941 displays the ability scores of the batter P with respect to the above five elements. The practice support system 1 of this embodiment is capable of visualizing and displaying multiple evaluation results of the swing of each batter P, which are obtained based on video footage of the batter P.

[0126] "Pelvic rotation" shows the evaluation results for the batter P's pelvic rotation speed and pelvic sway (axis movement) based on predetermined standard values. "Thoracic rotation" shows the evaluation results for the difference in torsion at landing and the upward angle of the ribs based on predetermined standard values. "Whole body coordination" shows the evaluation results for the order of rotation between the pelvis and thoracic spine and the order of rotation between the thoracic spine and arms based on predetermined standard values. "Bat trajectory" shows the evaluation results for the upper swing angle and pull angle based on predetermined standard values. And "ball vision" shows the evaluation results for the range of eye movement and changes in neck angle based on predetermined standard values.

[0127] The reference values ​​for each item can be standard values ​​obtained by analyzing, for example, video of batter P as a user of this system or video of a batter obtained from an external source.

[0128] As shown in Figures 18-1 and 18-2, detailed analysis information 960 displays analysis results of batter P as a user along with videos and specific numerical values ​​for detailed information on pelvic rotation 961, detailed information on thoracic rotation 962, detailed information on whole body coordination 963, detailed information on bat trajectory 964, and detailed information on how to see the ball 965. As shown in Figures 18-1 and 18-2, each piece of information is displayed individually in card format, and can be displayed individually on the screen by, for example, horizontal scrolling, which is an operation of moving a finger horizontally across the screen (display).

[0129] As shown in FIG. 18-1(A), the detailed information 961 of pelvic rotation indicates a model image 961M (video), a rotation speed evaluation 9611, a sway distance evaluation 9612, and advice information 9613. Model image 961M shows a captured image of batter P who has been highly evaluated for rotation speed and sway distance. Rotation speed evaluation 9611 displays the rotation speed (rotation angles / second) of batter P's pelvis using specific numerical values ​​and a graded evaluation. Sway distance evaluation 9612 displays the sway distance (m) of batter P's pelvis using specific numerical values ​​and a graded evaluation. Advice information 9613 displays advice information for batter P regarding pelvic rotation in the form of text.

[0130] As shown in FIG. 18-1(B), the detailed information 962 on thoracic rotation indicates a model image 962M (moving image), a torsion difference evaluation 9621, a rib angle evaluation 9622, and advice information 9623. Model image 962M shows a photographed image and 3D posture video of batter P who has received high evaluations for twisting difference and rib angle. Torsion difference evaluation 9621 displays the twisting difference (rotation angle) of batter P at the time of landing using specific numerical values ​​and a graded evaluation. Rib angle evaluation 9622 displays the upward angle (angle) of batter P's ribs using specific numerical values ​​and a graded evaluation. Advice information 9623 displays advice information for batter P regarding thoracic vertebrae rotation in the form of text.

[0131] As shown in FIG. 18-1(C), the detailed information 963 of the whole body coordination indicates a model image 963M (moving image), a first rotation order evaluation 9631, a second rotation order evaluation 9632, and advice information 9633, respectively. Model image 963M shows a captured image of batter P who has received a high evaluation for whole-body rotation. First rotation order evaluation 9631 displays the rotation order (time difference) of the pelvis and thoracic vertebrae using specific numerical values ​​and a graded evaluation. Second rotation order evaluation 9632 displays the rotation order (time difference) of the thoracic vertebrae and arms using specific numerical values ​​and a graded evaluation. Advice information 9633 displays advice information for batter P regarding whole-body coordination in the form of text.

[0132] As shown in FIG. 18-2(D), the detailed bat trajectory information 964 indicates a model image 964M (video), an upper swing evaluation 9641, a pull angle evaluation 9642, and advice information (not shown). Model image 964M shows a captured image of batter P who received a high evaluation for bat trajectory. Upper swing evaluation 9641 displays the upper swing angle (angle) as a specific numerical value along with an image that visualizes the three-dimensional posture (bat) of batter P. Pull angle evaluation 9642 displays the pull angle (angle) of batter P as a specific numerical value. Advice information displays advice information for batter P regarding bat trajectory in the form of text.

[0133] As shown in FIG. 18-2(E), the detailed information 965 on how to look at the ball indicates a model image 965M (video), an eye line evaluation 9651, a neck angle evaluation 9652, and advice information 9653. Model image 965M shows a captured image of batter P who has received a high evaluation for how to see the ball. Eye line evaluation 9651 displays the range (distance) of eye line movement using specific numerical values ​​and a graded evaluation. Neck angle evaluation 9652 displays the change in neck angle (angle) using specific numerical values ​​and a graded evaluation. Advice information 9653 displays advice information for batter P regarding how to see the ball in the form of text.

[0134] As described above, the practice support system 1 of this embodiment is configured to specifically present important elements obtained from the analysis results of the batter P's swing and the like using video and numerical values.

[0135] 19 and 20 are explanatory diagrams of the practice information 970 output by the reference information presenting unit 37 of this embodiment.

[0136] As shown in FIG. 19, the reference information presenting unit 37 outputs practice information 970, which is information relating to individual practice for the batter P. 19, practice information 970 is displayed on the screen of the user terminal device 10. The practice information 970 includes goal plan information 971 and practice suggestion information 972.

[0137] The goal plan information 971 proposes a practice goal and plan necessary for the batter P based on the evaluation results obtained by analyzing the batter P's swing, etc. The goal plan information 971 specifically displays the content that the batter P should improve using text information. The goal plan information 971 also proposes a practice plan and information on the amount of practice to the batter P using a plan graph 971G.

[0138] 20, the practice suggestion information 972 displays practice videos that can be used as reference for the batter P regarding the practice content proposed to the batter P, which is displayed as the goal plan information 971. In the practice suggestion information 972 of this embodiment, for one practice goal, multiple (e.g., three) practice videos are displayed in card format, and each practice video can be viewed by the user by scrolling horizontally.

[0139] Specifically, the practice suggestion information 972 includes goal information 9721 indicating a practice goal for the user in the form of text information, a reference video 9722, objective information 9723, and point information 9724. Goal information 9721 specifically displays the content of the practice goal proposed to the user in text. Reference video 9722 displays a video that serves as a reference for the practice goal. Objective information 9723 displays the purpose of the practice goal in text. Point information 9724 displays important points during practice in text.

[0140] The practice support system 1 of this embodiment can present, for example, multiple pieces of practice suggestion information 972 for one task of the batter P. This allows the batter P as a user to view all of the multiple pieces of practice suggestion information 972 or selectively view the practice suggestion information 972 that he or she prefers. As described above, the practice support system 1 of this embodiment can specifically present practice plans and goals for the batter P based on the evaluation of the analysis results of the batter P's swing and the like.

[0141] Next, a series of operations relating to practice support by the server device 30 of this embodiment configured as described above will be described. FIG. 21 is a flowchart showing the operation of the server device 30 of this embodiment.

[0142] 21, the server device 30 acquires a video of a batter P swinging a bat T, which has been captured by a single user terminal device 10 (S2101). Next, the server device 30 performs object detection in the acquired video (S2102). Specifically, the server device 30 detects the batter P, bat T, and ball B contained in the frame images as objects, targeting a plurality of frame images constituting the captured video.

[0143] Next, the server device 30 determines whether video analysis is possible (S2103). Specifically, video analysis is possible when the batter P, bat T, and ball B can be detected in the video. On the other hand, video analysis is not possible when the batter P, bat T, and ball B cannot be detected in the video, or when only a portion of each element is captured.

[0144] If it is determined that video analysis is possible (YES in S2103), the server device 30 identifies the lowest point timing T0, which is the time when the bat T reaches its lowest position (S2104). The server device 30 of this embodiment identifies the time when the bat T detected from the frame images constituting the captured image reaches its lowest position.

[0145] Next, the server device 30 performs two-dimensional video analysis to identify the two-dimensional posture (position) of each element using the partial images of the batter P, the bat T, and the ball B obtained by performing object detection from the video (S2105). At this time, the server device 30 performs analysis on the video for time periods of one second before and after the above-mentioned lowest point timing T0, which is used as the base point.

[0146] Next, the server device 30 performs three-dimensional video analysis of the batter P and the bat T using the two-dimensional posture information obtained by the two-dimensional video analysis (S2106). Furthermore, the server device 30 performs motion analysis of the batter P and the bat T, such as the trajectory of the ball B, the swing trajectory of the bat T, the rotation and eye movement of the batter P, based on object detection and two-dimensional and three-dimensional video analysis (S2107).

[0147] Then, the server device 30 generates evaluation information for evaluating the movement of the batter P based on reference information, such as information on an ideal swing prepared in advance, regarding the movement analysis of the bat T and the batter P (S2108). Furthermore, the server device 30 generates advice information for pointing out areas for improvement to the user based on the evaluation results (S2109).

[0148] If it is determined in step 2103 that video analysis is not possible (NO in S2103), the server device 30 instructs the user terminal device 10 to re-film the swing of batter P and transmit the filmed video again (S2110).

[0149] As described above, the practice support system 1 of this embodiment is able to display information about the batter P's swing in an easy-to-understand manner.

[0150] In the practice support system 1 of this embodiment, the server device 30 performs analysis of the swing using the captured video, but the present invention is not limited to this. For example, the user terminal device 10 may analyze the swing of the captured video captured using the camera 108 and display the analysis results superimposed on the captured video, thereby providing the user with information about the swing.

[0151] Here, the practice support system 1 is an example of an information processing system. The server device 30 is an example of an information processing device. The user terminal device 10 is an example of a terminal device and a photographing terminal. The video analysis unit 33 is an example of an acquisition means and a detection means. The analysis result output unit 35 is an example of a display means and an output means. The image display unit 13 is an example of a guidance display means. The camera 108 is an example of a photographing means.

[0152] Furthermore, although various embodiments and modifications have been described above, these embodiments and modifications may be combined together to form a configuration. Furthermore, the present disclosure is not limited to the above-described embodiments, and can be implemented in various forms without departing from the gist of the present disclosure. [Explanation of symbols]

[0153] 1... Practice support system, 10... User terminal device, 11... Video shooting unit, 13... Image display unit, 30... Server device, 31... Account management unit, 33... Video analysis unit, 33... Motion analysis unit, 35... Analysis result output unit, 37... Reference information presentation unit

Claims

1. An information processing system for processing information on a sport in which a player swings a tool to hit a moving object, an acquisition means for acquiring a video of the athlete swinging the implement; a display means for displaying information about the angle of the implement relative to the trajectory of the object, determined using the video, together with an image of the player's swing in the video; An information processing system comprising:

2. The information processing system according to claim 1 , wherein the display means displays adaptation information relating to an adapted trajectory, which is a trajectory of the implement included within a range determined according to the trajectory of the object.

3. the display means displays a trajectory image showing a trajectory of a swing of the tool on the swing image; The information processing system according to claim 2 , wherein the trajectory corresponding to the adapted trajectory is displayed in the trajectory image in a distinguishable manner.

4. The information processing system according to claim 3 , wherein the display means displays a contact position image, which indicates a contact position when the implement comes into contact with the object, on the swing image.

5. The information processing system according to claim 4 , wherein the display means displays evaluation information for evaluating the positional relationship between the contact position and the adapted trajectory.

6. 2. The information processing system according to claim 1, wherein the display means displays, on the swing image, an object trajectory image showing the trajectory of the object moving toward the player and the trajectory of the object hit by the implement.

7. 2. The information processing system according to claim 1, wherein a guide image that guides at least the equipment and the athlete to a position to which the equipment will move is displayed on a shooting screen of a shooting terminal that shoots a video of the athlete swinging the equipment.

8. 2. The information processing system according to claim 1, further comprising a detection means for detecting the player, the equipment, and the object from the video.

9. 9. The information processing system according to claim 8, wherein the detection means detects the player, the target object, and the equipment using information about objects that are not moving in the video.

10. The information processing system according to claim 8 , wherein the detection means sets a detection range for detecting the object in the video based on a swing trajectory of the implement.

11. The information processing system according to claim 1 , wherein a time period for analyzing the movements of the athlete and the equipment in the video is set based on information about the time when the equipment is in a specific state.

12. An information processing system for processing information on a sport in which a player swings a tool to hit a moving object, a guidance display means for displaying a guidance image indicating at least the position where the equipment and the athlete are to appear on a shooting screen of a shooting terminal that shoots a video of the athlete swinging the equipment; an acquisition means for acquiring a video of the athlete swinging the equipment, the video being captured by the single capturing device; a display means for displaying angle information relating to the angle of the implement relative to the trajectory of the object, which information is determined using the video, together with an image of the player's swing in the video; an output means for outputting evaluation information regarding the player's swing of the equipment based on the angle information; An information processing system comprising:

13. An information processing device that processes information on a sport in which a player swings a tool to hit a moving object, an acquisition means for acquiring a video of the athlete swinging the implement; a display means for displaying information relating to the angle of the implement relative to the trajectory of the object, determined using the video, together with an image of the player's swing in the video; An information processing device comprising:

14. An information processing device that processes information on a sport in which a player swings a tool to hit a moving object, a video capture means for capturing a video of the athlete swinging the implement; a display means for displaying information about the angle of the implement relative to the trajectory of the object, determined using the video, together with an image of the player's swing in the video; A terminal device comprising:

15. A computer that functions as an information processing device that processes information on a sport in which a player swings a tool to hit a moving object, capturing video of the athlete swinging the equipment; a function of displaying information about the angle of the implement relative to the trajectory of the object identified using the video together with an image of the player's swing in the video; A program to achieve this.

16. A computer that functions as an information processing device that processes information on a sport in which a player swings a tool to hit a moving object, a function of capturing video of the athlete swinging the equipment; a function of displaying information about the angle of the implement relative to the trajectory of the object identified using the video together with an image of the player's swing in the video; A program to achieve this.

Images