Information output system and information output method

Abstract

We provide technology that allows users to understand how well they are utilizing their own abilities. [Solution] The information output system disclosed herein comprises: a competition video acquisition unit 121 that acquires video footage of a specific competition action involving movement of the subject's position in a sport being played; a recognition unit 130 that recognizes a standardized reference object in the sport from the acquired video; a real feature acquisition unit 141 that measures the position of characteristic points of the subject's body in a specific competition action over time from the acquired video and acquires actual movement features, which are movement features in a specific competition action, based on the measurement results and the recognition results of the reference object; a reference feature acquisition unit 142 that acquires reference movement features, which are movement features in a reference action performed by the subject; and an output unit 160 that outputs the comparison result between the actual movement features and the reference movement features.

Description

Technical Field

[0001] The present disclosure relates to an information output system and an information output method.

Background Art

[0002] Based on the comparison result between the motion data generated from a video representing the motion state of a person receiving guidance and the standard motion data generated by collecting data of a plurality of top athletes and ordinary people, a motion education system that displays improvement points for the motion state of the person receiving guidance on a terminal is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Regarding the actions performed by athletes in sports, there are cases where athletes cannot fully demonstrate their abilities, while there are also cases where athletes can demonstrate their abilities better than usual. Therefore, grasping how much an athlete can demonstrate their abilities is extremely important for improving the athlete's competitive ability.

[0005] In view of the above, an object of the present disclosure is to provide a technology that enables grasping how much one's own ability can be demonstrated.

Means for Solving the Problems

[0006] To solve the above problems, an information output system in one embodiment of the present disclosure includes: a competition video acquisition unit that acquires video footage of a specific competition action involving movement of a subject's position in a sport in which a match is being played; a recognition unit that recognizes a standardized reference object in the sport from the acquired video; a real feature acquisition unit that measures the position of characteristic points of the subject's body in the specific competition action over time from the acquired video and acquires actual movement features, which are movement features in the specific competition action, based on the measurement results and the recognition results of the reference object; a reference feature acquisition unit that acquires reference movement features, which are movement features in a reference action performed by the subject; and an output unit that outputs the result of comparing the actual movement features and the reference movement features.

[0007] Other aspects of the information output method of this disclosure include: acquiring video footage of a specific athletic movement involving movement of a subject's position in a sport in which a match is being played; recognizing a standardized reference object in the sport from the acquired video footage; measuring the positions of characteristic points of the subject's body in the specific athletic movement over time from the acquired video footage, and acquiring actual movement features, which are movement features in the specific athletic movement, based on the measurement results and the recognition results of the reference object; acquiring reference movement features, which are movement features in a reference movement performed by the subject; and outputting the results of a comparison between the actual movement features and the reference movement features.

[0008] Furthermore, any combination of the above components, as well as conversions of the expression of the present invention between methods, apparatus, systems, computer programs, data structures, recording media, etc., are also valid embodiments of the present invention. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a technology that makes it possible to understand how well one's own abilities are being utilized. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing the general configuration of the performance evaluation system. [Figure 2] This is a functional block diagram showing the various components of the operation evaluation system according to the first embodiment. [Figure 3] This figure illustrates the standard operation in the first embodiment. [Figure 4] This is a diagram illustrating a standard object in tennis. [Figure 5] This diagram illustrates the process of estimating the location of feature points from competition footage. [Figure 6] This figure shows an example screen displaying the comparison results of the actual operating features and the reference operating features of the first embodiment. [Figure 7] This is a functional block diagram showing the configurations of the operation evaluation system of the second embodiment. [Figure 8] This figure shows an example screen displaying the comparison results of the actual operating features and the reference operating features of the second embodiment. [Figure 9] This is a diagram illustrating a reference object in basketball. [Figure 10] This figure illustrates the standard operation in the third embodiment. [Figure 11] This is a functional block diagram showing the various components of the operation evaluation system according to the third embodiment. [Modes for carrying out the invention]

[0011] The present disclosure will be described below with reference to the drawings, based on preferred embodiments. The same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant descriptions will be omitted as appropriate. Furthermore, the embodiments are illustrative and not limiting to the invention, and not all features or combinations thereof described in the embodiments are necessarily essential to the invention.

[0012] First Embodiment Figure 1 shows a schematic configuration of a motion evaluation system 1 for evaluating the movements of a subject. The motion evaluation system 1 in Figure 1 comprises an information terminal 50 and a server 100. Figure 1 shows an example in which the information terminal 50 uses its camera function to film a tennis player 10 performing a tennis return motion in response to a serve from an opponent player 20 during a tennis match. Tennis is an example of a sport in which a match is played facing an opponent. The tennis return motion is an example of a specific competitive movement that involves movement of the subject 10's position. Here, "sports in which a match is played facing an opponent" includes sports in which players compete for points against an opponent, such as tennis, badminton, volleyball, basketball, handball, and futsal, as well as sports in which players compete side-by-side with an opponent, such as swimming and track and field events. Furthermore, "specific athletic actions involving movement of the subject 10's position" include, for example, receiving actions in tennis, badminton, and volleyball; dribbling actions in basketball, handball, and futsal; crawl stroke in swimming; and sprinting actions in track and field events. "Specific athletic actions involving movement of the subject 10's position" are not limited to a single action, but may be a combination of multiple actions, or may be, for example, the actions of the subject 10 throughout the entire match. The action evaluation system 1 of this embodiment is an example of an information output system.

[0013] The information terminal 50 is a portable device, such as a smartphone or tablet, capable of capturing moving images, displaying images, recording audio, inputting and outputting audio, and sending and receiving information. The information terminal 50 and the server 100 are connected via a predetermined communication network, such as the Internet. The information terminal 50 acquires information about the subject's actions and transmits it to the server 100.

[0014] Server 100 acquires video footage of a specific athletic movement performed by subject 10 from information terminal 50, calculates the actual movement features of that specific athletic movement from the video footage, and outputs the comparison result with the reference movement features of the reference movement described later. This allows subject 10 to understand how well they are performing their abilities during the competition.

[0015] FIG. 2 is a functional block diagram showing each component of the operation evaluation system 1 of the first embodiment. In this figure, a block diagram focusing on functions is drawn, and these functional blocks can be realized in various forms by hardware, software, or a combination thereof. The information terminal 50 and the server 100 may mainly consist of a mobile terminal or a computer including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device, a display device, a communication device, a camera module, etc., and programs stored in the mobile terminal or the computer.

[0016] The information terminal 50 includes an operation information generation unit 51, an output unit 52, and a communication unit 53.

[0017] The operation information generation unit 51 generates information regarding the operation of the subject 10. The operation information generation unit 51 of the present embodiment acquires a video regarding the operation of the subject 10. The operation information generation unit 51 includes a competition video generation unit 54 and a reference video generation unit 55. The operation information generation unit 51 includes, for example, a camera module as hardware.

[0018] The competition video generation unit 54 generates a competition video by photographing a specific competition operation performed by the subject 10 during a sports competition. The competition video generation unit 54 generates a competition video by photographing an operation involving the movement of the position of the subject 10 as a specific competition operation, for example, the reception operation of the subject 10 as shown in FIG. 1. The reception operation here is, for example, the operation from when the opponent player 20 serves until the subject 10 returns the served ball.

[0019] The reference video generation unit 55 generates a reference video by filming a reference movement performed by the subject 10. The reference movement is the same as or similar to a competitive movement. By performing the reference movement, it becomes possible to measure the subject 10's basic physical fitness, which forms the basis of the subject 10's achievable abilities.

[0020] Figure 3 illustrates the reference action in the first embodiment. The reference action in this embodiment is, for example, the action of the subject 10 dashing from the starting line SL to cone C, which is 3 meters away, and touching the cone with a racket, as shown in Figure 3. The reference video generation unit 55 generates a reference video by, for example, filming the action of the subject 10 from starting from the starting line SL, dashing to cone C, which is 3 meters away, and touching the cone with a racket. The reference video may be filmed before or after the generation of the competition video.

[0021] The output unit 52 transmits the competition video and reference video to the server 100 via the communication unit 53, associating them, for example, with the identification information of the subject 10. The output unit 52 also acquires the information transmitted from the server 100 via the communication unit 53 and displays it on the display of the information terminal 50. The display of the information terminal 50 in this embodiment is an example of an output device.

[0022] The server 100 comprises a communication unit 110, an operation information acquisition unit 120, a recognition unit 130, a feature quantity determination unit 140, a storage unit 150, and an output unit 160.

[0023] The motion information acquisition unit 120 acquires information about the subject's movements from the information terminal 50. The motion information acquisition unit 120 includes a competition video acquisition unit 121 and a reference video acquisition unit 122. The competition video acquisition unit 121 and the reference video acquisition unit 122 acquire competition videos and reference videos, respectively, associated with the subject's identification information, from the information terminal 50 via the communication unit 110 as information about the subject's movements.

[0024] The recognition unit 130 recognizes standardized reference objects in sports from the acquired competition video. Figure 4 illustrates standard objects in tennis. In the example in Figure 4, the reference objects are the baseline S1, sideline S2, service line S3, service center line S4, and post S5. For example, the length L1 of the baseline S1 is standardized to 8.23m, the length L2 of the sideline S2 to 23.77m, the length L3 of the service line S3 to 8.23m, the length L4 of the service center line S4 to 6.4m, and the height L5 of the post S5 to 1.07m. Also, since the diameter of a tennis ball is standardized to 6.54 to 6.86 cm, a tennis ball can also be a reference object. The recognition unit 130 recognizes these reference objects from the competition video using known image recognition technology and supplies the recognition results to the real feature acquisition unit 141.

[0025] The feature determination unit 140 includes an actual feature acquisition unit 141 and a reference feature acquisition unit 142. The actual feature acquisition unit 141 measures the positions of feature points of the subject 10's body during specific athletic movements from the acquired athletic video over time, and acquires actual motion features based on the measurement results and the recognition results of the reference object. Actual motion features are motion features in specific athletic movements. The actual feature acquisition unit 141 estimates the three-dimensional coordinates of feature points from the images of the subject 10 included in the athletic video. The feature points here include not only body parts such as the joints of the subject 10, but also feature points of the shoes worn by the subject 10 and feature points of equipment such as rackets held by the subject 10. The actual feature acquisition unit 141 measures at least one of the motion features, such as the position, trajectory, and movement speed of the feature points, based on the time changes of the feature points estimated in the athletic video.

[0026] Furthermore, the actual feature acquisition unit 141 measures the distance traveled by each feature point in the game video from the start to the end of the action, and measures the actual action features based on the measurement results and the recognition results of the reference object. For example, if the subject 10 moves from one sideline S2 to the other sideline S2 of the tennis court during a receiving action in the game video, the actual distance traveled by each feature point of the subject 10 is estimated to be the length L1 (=8.23m) of the baseline S1. Alternatively, for example, the actual feature acquisition unit 141 may estimate the distance traveled by each feature point by determining the scale of the game video using the dimensions of the reference object. By measuring the actual action features based on the distance traveled by the feature points estimated from the dimensions of the reference object in this way, it becomes possible to measure action features more accurately from the game video. The actual feature acquisition unit 141 stores the measurement results of the actual action features in the storage unit 150, linked to the identification information of the subject 10.

[0027] The actual motion features measured by the actual feature acquisition unit 141 include not only features as time history characteristics that can change over the entire time course of the motion, but also features as instantaneous characteristics that occur at a specific moment in the motion. Features as time history characteristics include, for example, the position and trajectory of feature points, movement speed history, joint angles, joint angular velocity, joint angular acceleration, upper limb movement speed, upper limb movement trajectory, time history of trunk posture, stride length, and face orientation. Features as instantaneous characteristics that occur at a specific moment in the motion include, for example, the speed and stride length of the first step of the motion.

[0028] Figure 5 illustrates the process of estimating the location of feature points from a video of a sporting event. The video of the sporting event 5 in Figure 5 shows a subject 10 performing a tennis receiving motion. By performing image processing on the video of the sporting event 5 using known motion analysis techniques, the locations of major joints and other physical feature points, which are the physical feature points of subject 10, are estimated as three-dimensional coordinates from the image portion of subject 10's body. In Figure 5, the coordinates of the estimated feature points are shown by multiple circles 32. By connecting the multiple circles 32 representing the feature points with thick lines 34, the skeleton of subject 10 is shown in the form of a so-called stick picture. The positions of the circles 32 and thick lines 34 are shown to follow the movement of the feature points according to the movement of subject 10 in the video of the sporting event 5. The position and movement of these feature points are estimated by the actual feature acquisition unit 141, and motion features such as the position, trajectory, and movement speed of the feature points are extracted based on the time changes of the estimated feature points.

[0029] The reference feature acquisition unit 142 acquires reference motion features. Similar to the actual feature acquisition unit 141, the reference feature acquisition unit 142 in this embodiment acquires reference motion features by measuring motion features from the reference motion of the subject 10 in the reference video using the physical feature points of the subject 10. The reference feature acquisition unit 142 stores the measurement results of the acquired reference motion features in the storage unit 150, linked to the identification information of the subject 10.

[0030] The memory unit 150 stores the measurement results of the actual motion features and the reference motion features for each subject 10, based on the subject's identification information linked to the measurement results of the actual motion features and the reference motion features. The memory unit 150 also stores in advance the reference objects for each sport and the dimensions defined for those reference objects.

[0031] The output unit 160 outputs the comparison result between the actual operating features and the reference operating features to the information terminal 50. For example, the output unit 160 outputs the ratio of the actual operating features to the reference operating features as the comparison result. Alternatively, for example, the output unit 160 may output a graph showing the time-course characteristics of the reference operating features and the actual operating features side by side as the comparison result. The output unit 160 may output the comparison result for each type of operating feature, or it may output the comparison result using an overall operating feature calculated by summing each operating feature, for example, by multiplying all operating features by weight coefficients according to their respective importance.

[0032] Figure 6 shows an example screen displaying the comparison results of the actual operation features and the reference operation features of the first embodiment. In the example in Figure 6, the comparison results generated by extracting the movement speed of feature points from the start to the end of the operation as operation features are shown. The first column 41 shows, for example, a graph of the time history characteristics of the actual operation features and the reference operation features. Line 42 represents the actual operation features, and line 43 represents the reference operation features. The second column 44 displays the ratio of the actual operation features to the reference operation features, such as "ability performance," as an indicator of how well the ability was demonstrated. In the example in Figure 6, the ratio V / Vref = 0.69 of the average value of the actual operation features (i.e., average movement speed in the competitive operation) to the average value Vref of the reference operation features (i.e., average movement speed in the reference operation) from the start to the end of the operation is shown as the ability performance. In the graph in column 41, the actual performance features are consistently smaller than the baseline performance features, and the "Performance Level" in column 44 indicates that the subject was only able to perform about 70% of the performance level they demonstrated in the baseline performance during the competition. Therefore, subject 10 can understand that they were not fully utilizing their abilities in a particular competition. In this way, by comparing actual performance features with baseline performance features, it becomes possible to understand how well one is utilizing their abilities in a specific competition during the competition.

[0033] Second Embodiment The second embodiment will now be described. In the drawings and description of the second embodiment, components and members that are the same as or equivalent to those in the first embodiment will be denoted by the same reference numerals. Descriptions that overlap with those of the first embodiment will be omitted as appropriate, and the description will focus on the configurations that differ from those of the first embodiment.

[0034] Figure 7 is a functional block diagram showing the configurations of the operation evaluation system 1 of the second embodiment. As shown in Figure 7, the operation evaluation system 1 of the second embodiment further includes a factor acquisition unit 170 and an additional information provision unit 180. The operation information generation unit 51 of the second embodiment further includes a voice information generation unit 56. The operation information acquisition unit 120 of the second embodiment further includes a voice information acquisition unit 123.

[0035] The voice information generation unit 56 inputs ambient sounds from the information terminal 50 and generates voice information representing those sounds. The voice information generation unit 56 includes, for example, a microphone as hardware. For example, the voice information generation unit 56 acquires the voices of athletes, the cheers of spectators, and sounds emitted by sports equipment (such as buzzer sounds) when the subject 10 is filmed performing sports. The output unit 52 outputs the voice information to the server 100 via the communication unit 53, associating it with the identification information of the subject 10. The voice information acquisition unit 123 acquires the voice information associated with the identification information of the subject 10 from the information terminal 50 via the communication unit 110.

[0036] The factor acquisition unit 170 acquires factor information from the competition video, including factors that may affect the quality of specific competition movements. The factor acquisition unit 170 includes an internal factor information acquisition unit 171 and an external factor information acquisition unit 172. The factor information in this embodiment includes internal factor information acquired by the internal factor information acquisition unit 171 and external factor information acquired by the external factor information acquisition unit 172. The factor acquisition unit 170 acquires information related to movements from the movement information acquisition unit 120 and the recognition results of the reference object by the recognition unit 130. The factor acquisition unit 170 stores the factor information in the storage unit 150.

[0037] The internal factor information acquisition unit 171 acquires internal factor information indicating the psychological state of the subject 10 as factors from the competition video. "The psychological state of the subject 10" includes, for example, the subject 10's level of tension, concentration, and confidence. For example, the internal factor information acquisition unit 171 generates internal factor information including an evaluation value of the psychological state by extracting elements related to the tendency of the psychological state, including at least one of the subject 10's facial expression, posture, movement speed, trajectory, and position, from the competition video and performing an analysis to quantify the psychological state. For example, the internal factor information acquisition unit 171 reads a prediction model for the subject 10 that has been generated in advance by machine learning using the competition video and the correct data of the evaluation value of the subject 10's psychological state (for example, a numerical value judged and input by the subject 10) as training data from the storage unit 150, and obtains an evaluation value of the psychological state from the prediction model by inputting the competition video into the prediction model.

[0038] The internal factor information acquisition unit 171 may further perform an analysis to quantify the psychological state by extracting at least one of the pitch, speed, volume, and intonation of the voice spoken by the subject 10 from the audio information as an element relating to the tendency of the psychological state. Furthermore, if the subject 10 is wearing a wearable measurement device such as smart glasses or a smartwatch, the internal factor information acquisition unit 171 may perform an analysis to quantify the psychological state by extracting biometric information, including at least one of the subject 10's heart rate, blood oxygen saturation, and gaze measured by the wearable measurement device, as an element relating to the tendency of the psychological state. By analyzing the psychological state using audio information and biometric information, objective internal factor information can be obtained from the subject 10's unconscious actions and biometric indicators, enabling an objective factor analysis of the extent to which abilities were demonstrated. Additionally, the internal factor information acquisition unit 171 may administer a questionnaire to the subject 10 to assess their psychological state and extract the subject 10's responses to the questionnaire as an element relating to the tendency of the psychological state. By analyzing psychological states using questionnaires, subjective internal factor information of the 10 subjects can be obtained, enabling subjective factor analysis of the extent to which abilities were demonstrated. Furthermore, comparing subjective internal factor information obtained from questionnaires with objective internal factor information is also effective. In these cases, a predictive model can be machine-learned using voice information, biometric information, and the 10 subjects' responses to the questionnaire as training data.

[0039] The external factor information acquisition unit 172 acquires external factor information as factors from the competition video regarding the surrounding conditions of the subject 10. The external factor information acquisition unit 172 includes an opponent feature acquisition unit 173, a ball feature acquisition unit 174, an equipment state acquisition unit 175, an audio feature acquisition unit 176, and a location state acquisition unit 177.

[0040] The opponent feature acquisition unit 173 acquires motion features and physical features of the opponent player 20 by measuring the positions of characteristic points on the opponent player 20's body over time during the period in which a specific athletic action is being performed, based on the acquired athletic video footage. The method for measuring motion features in the opponent feature acquisition unit 173 is the same as the method for measuring motion features in the actual feature acquisition unit 141. In addition, the opponent feature acquisition unit 173 measures the physical features of the opponent player 20 based on, for example, the distance between predetermined characteristic points on the opponent player 20's body. Examples of the opponent player's physical features include height, arm length, and body width. The opponent player 20 is an example of a moving object other than the subject 10.

[0041] The ball feature acquisition unit 174 acquires motion features of the ball by measuring over time the positions of feature points of the ball used by the subject 10 during a specific athletic action from the acquired athletic video. For example, the motion features of the ball include the position of the ball and the speed of the ball's movement. The ball is an example of a moving object other than the subject 10.

[0042] The equipment status acquisition unit 175 acquires equipment status information indicating the state of equipment by detecting equipment used to indicate the situation of a match in a sport from the acquired competition video. "The state of equipment used to indicate the situation of a match" refers to, for example, the score on the scoreboard or the remaining time on the timer. "The situation of the match" refers to, for example, the score, the remaining time on the match, the number of sets in the match, etc. The equipment status acquisition unit 175 acquires equipment status information by detecting the state of equipment used to indicate the situation of a match by recognizing the score on the scoreboard, the remaining time on the timer, etc. from the competition video using known image recognition technology.

[0043] The audio feature acquisition unit 176 acquires audio features related to at least one of the following from audio information, including audio corresponding to the competition video: spectator vocalizations and sounds emitted by sports equipment. "Spectator vocalizations" include, for example, cheers, boos, shouts, and jeers. "Sounds emitted by sports equipment" include, for example, the sound of a referee's whistle, a buzzer, and the sound of a racket hitting the ball. "Audio features" include volume, pitch, and rhythm. The audio feature acquisition unit 176 measures audio features by recognizing spectator vocalizations and sounds emitted by sports equipment in the audio information using known speech recognition technology.

[0044] The location state acquisition unit 177 acquires location state information indicating the state of a location by detecting the state of the location where a specific athletic action was performed from the acquired athletic video footage. Location state refers to things like the condition of the court (tennis court surface, wetness, slipperiness, etc.), wind speed, weather, season, temperature, etc. The location state acquisition unit 177 acquires location state information by recognizing the condition of the court, etc., from the athletic video footage using known image recognition technology and detecting the state of the location where a specific athletic action was performed.

[0045] The external factor information in the second embodiment includes motion characteristics and physical characteristics of the opposing player 20, motion characteristics of the ball, equipment status information, audio characteristics, and location status information.

[0046] The additional information provision unit 180 provides additional information regarding the subject's athletic performance based on the subject's actual performance characteristics, reference performance characteristics, and factor information stored in the memory unit 150. For example, the additional information provision unit 180 provides feedback information as additional information that indicates factors that allowed the subject to perform well or poorly in athletic performance. For example, if the actual performance characteristics when the tennis court surface is a hard court are significantly inferior to the actual performance characteristics for other surfaces stored in the memory unit 150, the additional information provision unit 180 generates feedback information indicating that the subject is not good at hard courts as a factor in their inability to perform well. The additional information provision unit 180 supplies the additional information to the output unit 160.

[0047] The output unit 160 of this embodiment outputs factor information and additional information in relation to the comparison results.

[0048] Figure 8 shows an example screen displaying the comparison results of actual operation features and reference operation features in the second embodiment. For example, the third column 45 displays a table showing the score and number of sets on the scoreboard as tool state information 45A, and the type of surface and weather as location state information 45B. For example, the fourth column 46 displays comments illustrating feature quantities that show characteristic evaluation values ​​in the internal factor information of the subject 10. For example, the fifth column 47 displays comments illustrating feedback information.

[0049] According to the second embodiment, factor information including factors that may influence the quality of specific athletic movements is output in association with the comparison results between actual movement features and reference movement features, making it possible to understand the factors that influenced the quality of athletic movements. Furthermore, by including internal factor information in the factor information, it becomes possible to understand how the psychological state of the subject 10 influenced the quality of athletic movements. Moreover, by including external factor information in the factor information, it becomes possible to understand how the surrounding conditions of the subject 10 influenced the quality of athletic movements.

[0050] By including the feature quantities of moving objects other than subject 10 in the external factor information, it becomes possible to understand how moving objects other than subject 10 influenced the quality of the athletic movements. By including the motion and physical feature quantities of opponent player 20 in the external factor information, it becomes possible to understand how opponent player 20 influenced the quality of the athletic movements. By including the motion feature quantities of the ball used by subject 10 in the external factor information, it becomes possible to understand how the ball's motion influenced the quality of the athletic movements.

[0051] By including audio features related to at least one of the following in the audio information corresponding to the competition video: spectator vocalizations and sounds emitted by sports equipment, it becomes possible to understand how spectator vocalizations and sounds emitted by sports equipment influenced the quality of the athletic movements. By including equipment state information indicating the state of equipment used to represent the situation of a match in a sport, it becomes possible to understand how the situation of the match influenced the quality of the athletic movements. By including location state information indicating the state of the place where a particular athletic movement took place, it becomes possible to understand how the state of the location influenced the quality of the athletic movements.

[0052] Third Embodiment The third embodiment will now be described. In the drawings and description of the third embodiment, components and members that are the same as or equivalent to those in the first embodiment will be denoted by the same reference numerals. Descriptions that overlap with those of the first embodiment will be omitted as appropriate, and the description will focus on the configurations that differ from those of the first embodiment.

[0053] In the above embodiment, tennis was used as an example sport, but in the third embodiment, basketball will be used as an example. Figure 9 illustrates reference objects in basketball. In addition to the reference objects, Figure 9 also shows a subject 10, an opposing player 20, and a teammate 25. In Figure 9, for simplification, the subject 10 and teammate 25 are shown with circles, and the opposing player 20 is shown with an "x". In the example in Figure 9, the reference objects are the end line S6, the side line S7, the free throw line S8, the ring 27, etc. For example, the length L6 of the end line S6 is standardized to 15m, the length L7 of the side line S7 is standardized to 28m, the distance L8 from the end line S6 to the free throw line S8 is standardized to 5.8m, and the height from the floor to the top of the ring 27 is standardized to 3.05m. The recognition unit 130 recognizes these reference objects from the game video using known image recognition technology and supplies the recognition results of the reference objects to the actual feature acquisition unit 141 and the factor acquisition unit 170.

[0054] Furthermore, Figure 9 shows an example in which subject 10 is performing a dribbling motion towards the ring 27 from a position outside the three-point line 28 near the front of the ring 27. This dribbling motion is a specific game action in this embodiment.

[0055] Figure 10 illustrates the reference motion in the third embodiment. As shown in Figure 10, the reference motion in this embodiment is the action of the subject 10 dribbling towards the ring 27 from a position outside the three-point line 28 near the front of the ring 27, assuming that cone C is an opposing player. The reference video generation unit 55 generates a reference video by filming the action of the subject 10 dribbling towards the ring 27 from a position outside the three-point line 28 near the front of the ring 27.

[0056] Figure 11 is a functional block diagram showing the configurations of the operation evaluation system 1 of the third embodiment. As shown in Figure 11, the factor acquisition unit 170 of the third embodiment further includes a friendly feature acquisition unit 178.

[0057] The Ally Feature Acquisition Unit 178 acquires motion features and physical features of the Ally 25 by measuring the positions of characteristic points of the Ally 25's body over time during a predetermined athletic action from the acquired athletic video. The method for measuring motion features and physical features in the Ally Feature Acquisition Unit 178 is the same as the method for measuring motion features and physical features in the Opponent Feature Acquisition Unit 173. The Ally 25 is an example of a moving object other than the subject 10.

[0058] The external factor information of the third embodiment includes motion and physical characteristics of the opposing player 20, motion characteristics of the ball, equipment status information, voice characteristics, location status information, and motion and physical characteristics of the teammate 25.

[0059] According to the third embodiment, it becomes possible to understand how the teammate player 25 influenced the quality of the subject's athletic performance.

[0060] The present invention is not limited to the embodiments described above, and each configuration can be modified as appropriate without departing from the spirit of the invention. Modifications are described below.

[0061] In this embodiment, sports in which players face each other, such as tennis, were given as examples, but the system is not limited to these, and the motion evaluation system 1 may also be applied to sports in which players do not face each other, such as jumping events like the long jump or throwing events like the javelin throw in track and field.

[0062] In the embodiment, the factor information includes, but is not limited to, internal factor information and external factor information, and may include at least one of these. In the embodiment, the external factor information includes, but is not limited to, the motion and physical characteristics of the opposing player 20, the motion characteristics of the ball, the equipment state information, the voice characteristics, the location state information, and the motion and physical characteristics of the teammate player 25.

[0063] For example, the additional information providing unit 180 may provide type information indicating the type of subject 10 as additional information by diagnosing the type of subject 10 based on the actual operation characteristics, reference operation characteristics, and factor information of the subject 10 stored in the memory unit. For example, if the distance traveled is less than a predetermined distance traveled reference value and the maximum speed is greater than a predetermined maximum speed reference value, the additional information providing unit 180 may diagnose the type of subject 10 as a "speed type". For example, if the distance traveled is greater than a predetermined distance traveled reference value and the rate of decrease between the maximum speed at the end of the game and the maximum speed at the beginning of the game is less than a predetermined rate of decrease reference value, the additional information providing unit 180 may diagnose the type of subject 10 as a "type that performs well under pressure". For example, if the additional information provision unit 180 determines that the difference between the actual operation feature and the reference operation feature is greater than a predetermined difference threshold, it may diagnose the subject 10 as a "type that performs poorly under pressure." This allows the subject 10 to understand their own type and improve their performance by developing their strengths or overcoming their weaknesses.

[0064] Furthermore, the additional information provision unit 180 may provide additional information indicating countermeasures that show how to take measures to effectively enable the subject 10 to demonstrate their abilities according to the diagnosed type. For example, if the subject 10 is diagnosed as a "type that performs poorly under pressure," the additional information provision unit 180 may generate countermeasures such as "increase the number of matches you play to get used to playing matches." This allows the subject 10 to appropriately improve their performance according to their type.

[0065] Furthermore, the additional information provision unit 180 may provide motion characteristics of athletic movements performed by other athletes of the same type as the subject 10 as additional information. This makes it possible to compare the actual motion characteristics of the subject 10 with those of other athletes of the same type.

[0066] For example, the additional information provision unit 180 may generate growth potential indication information, which shows the movement features among the subject's 10 actual movement features that have room for improvement greater than a predetermined standard, based on the subject's 10 actual movement features, reference movement features, and factor information stored in the memory unit 150, and provide it as additional information. For example, if the movement speed as a movement feature is smaller than a predetermined standard when the tennis court surface is a hard court, the additional information provision unit 180 may provide growth potential indication information as additional information, which shows that there is room for improvement in the movement speed when the tennis court surface is a hard court. As a result, the subject 10 can grasp the movement features that have room for improvement, and thus efficiently work to improve their movements.

[0067] Furthermore, for example, the additional information providing unit 180 may provide improvement suggestion information as additional information, which offers suggestions to encourage improvement regarding the motion features indicated by the growth potential information. For example, if the growth potential information indicates that there is room for improvement in movement speed when the tennis court surface is a hard court, the additional information providing unit 180 may provide improvement suggestion information that includes a comment encouraging practice of footwork on hard courts. This allows the subject 10 to effectively improve the performance characteristics that have room for improvement. Additionally, for example, the additional information provision unit 180 may present the target values ​​for the performance characteristics indicated in the performance characteristics information within the performance characteristics information.

[0068] The additional information provision unit 180 may provide, as additional information, an image video that simulates what would happen if the subject 10's movements improved through training based on the growth potential information. This makes it easier for the subject 10 to visualize their improved self, thereby improving their motivation.

[0069] For example, the additional information provision unit 180 may generate a highlight video of the subject 10's athletic performance based on the subject 10's actual performance characteristics, reference performance characteristics, and factor information stored in the memory unit 150, and provide it as additional information. For example, the additional information provision unit 180 may generate a highlight video of the athletic performance based on athletic video and audio information from a predetermined period including when the volume of the spectators' cheers is at its maximum or when the subject 10's movement speed is at its maximum, and provide it as additional information.

[0070] For example, if the additional information providing unit 180 determines that a subject 10's athletic performance is a performance they excel at, based on the subject 10's actual performance characteristics, reference performance characteristics, and factor information stored in the memory unit 150, it may provide a video of the performance they excel at, including multiple videos of the performance they excel at, as additional information. For example, if one or more of the subject 10's athletic performance characteristics exceed a predetermined reference value, the additional information providing unit 180 may determine that an athletic performance is a performance they excel at, and generate a video of the performance they excel at, including multiple videos of the performance they excel at, and provide it as additional information.

[0071] For example, the additional information provision unit 180 may provide tactical information as additional information that suggests recommended tactics based on the actual movement characteristics, reference movement characteristics, and factor information of the subject 10 stored in the memory unit 150. For example, the additional information provision unit 180 may generate and provide as additional information tactical information that suggests actively using the competitive movements that have been judged as being the subject's strong points, as described above. Also, for example, if the actual movement characteristics obtained when facing a specific opponent are greater than or equal to a predetermined threshold value compared to the actual movement characteristics obtained when facing other opponents, the additional information provision unit 180 may generate and provide as additional information tactical information that suggests actively playing against that specific opponent. Also, for example, if the actual movement characteristics obtained when playing a tennis match on a specific surface are greater than or equal to a predetermined threshold value compared to the actual movement characteristics obtained when playing a tennis match on other surfaces, the additional information provision unit 180 may generate and provide as additional information tactical information that suggests actively playing on that specific surface.

[0072] For example, the additional information provision unit 180 may provide the actual movement characteristics of past athletic movements performed by the subject 10, which are stored in the memory unit 150, as additional information.

[0073] In this embodiment, the recognition result of a reference object was used to measure actual operating features, but this is not limited to this. For example, the recognition unit 130 may recognize a reference object in a reference video, and the reference feature acquisition unit 142 may measure reference operating features based on the recognition result of the reference object in the reference image.

[0074] In this embodiment, reference motion features were acquired based on a reference video, but the system is not limited to this. For example, the subject 10 may input the measurement results as reference motion features via the information terminal 50 by performing various measurements (e.g., the time taken for a 3m dash) on the reference motion of the subject 10. In this case, the storage unit 150 stores the measurement results of the reference motion features received via the communication unit 110, and the reference feature acquisition unit 142 acquires the measurement results of the reference motion features from the storage unit 150. In this case, the reference video generation unit 55 and the reference video acquisition unit 122 do not necessarily have to be used.

[0075] Opponent player information may include the level of competition of opponent player 20. For example, if opponent player 20 or the organization to which opponent player 20 belongs can be identified from the uniform worn by opponent player 20 in the competition video, the opponent feature acquisition unit 173 may acquire the level of competition, such as professional level or high school level, based on that identification result. Similarly, teammate player information may also include the level of competition of teammate player 25.

[0076] In this embodiment, the operation evaluation system 1 is composed of an information terminal 50 and a server 100, but it is not limited to this. For example, the operation evaluation system 1 may be configured using the information terminal 50 by assigning the functions of the server 100 to the information terminal 50. Alternatively, the operation evaluation system 1 may be configured using the server 100 by assigning the functions of the information terminal 50 to the server 100.

[0077] The embodiments and modifications described above can be generalized to obtain the following embodiments.

[0078] [Aspect 1] A sports video acquisition unit that acquires video footage of specific sports movements involving the movement of the subject's position during a match, A recognition unit that recognizes a standardized object in the sport from the acquired video, A real feature acquisition unit that measures the position of characteristic points of the subject's body during a specific athletic movement from the acquired video over time, and acquires actual movement feature quantities, which are movement feature quantities in the specific athletic movement, based on the measurement results and the recognition results of the reference object. A reference feature acquisition unit acquires reference motion feature quantities, which are motion feature quantities in the reference motion performed by the subject, An output unit that outputs the comparison result between the actual operating feature quantity and the reference operating feature quantity, An information output system equipped with the following features.

[0079] [Aspect 2] The system includes a factor acquisition unit that acquires factor information from the aforementioned video, which includes factors that may affect the quality of the specific athletic movements. The information output system according to claim 1, wherein the output unit outputs the factor information in relation to the comparison result.

[0080] [Aspect 3] The factor acquisition unit includes an internal factor information acquisition unit that acquires internal factor information indicating the psychological state of the subject as the factor. The information output system described in Embodiment 2.

[0081] [Aspect 4] The factor acquisition unit includes an external factor information acquisition unit that acquires external factor information relating to the subject's surrounding conditions as the factor. The information output system according to embodiment 2 or 3.

[0082] [Aspect 5] The aforementioned external factor information includes feature quantities obtained by measuring over time the positions of feature points of moving objects other than the subject during the period in which the specific athletic action is being performed, from the video. The information output system described in Embodiment 4.

[0083] [Aspect 6] The aforementioned external factor information includes motion characteristics and physical characteristics obtained by measuring the positions of the characteristic points of the opposing player's body in the sport over time, as the positions of the characteristic points of the moving object. The information output system described in Embodiment 5.

[0084] [Aspect 7] The aforementioned external factor information includes motion characteristics and physical characteristics obtained by measuring over time the positions of the physical characteristics of the subject's teammate in the sport, as the positions of the characteristic points of the moving object. The information output system described in embodiment 5 or 6.

[0085] [Aspect 8] The aforementioned external factor information includes motion features obtained by measuring over time the position of the feature points of a ball used by the subject as the position of the feature points of the moving object. An information output system as described in any of embodiments 5 to 7.

[0086] [Aspect 9] The system further includes an audio information acquisition unit that acquires audio information indicating the audio corresponding to the aforementioned video, The aforementioned external factor information includes an audio feature quantity relating to at least one of the spectator's vocalizations included in the audio information and the sounds emitted by the equipment used in the sport. An information output system as described in any of embodiments 4 to 8.

[0087] [Aspect 10] The aforementioned external factor information includes equipment condition information indicating the condition of the equipment used to show the situation of the match in the sport, An information output system according to any one of embodiments 4 to 9.

[0088] [Aspect 11] The aforementioned external factor information includes location status information indicating the state of the place where the specific athletic action was performed. An information output system according to any one of embodiments 4 to 10.

[0089] [Aspect 12] This involves acquiring video footage of specific athletic actions in which the subject's position changes during a sporting event, and From the acquired video footage, recognize a standard object whose dimensions are standardized in the sport, From the acquired video, the positions of characteristic points of the subject's body during the specific athletic movement are measured over time, and based on the measurement results and the recognition results of the reference object, actual movement characteristics, which are the movement characteristics of the specific athletic movement, are obtained. To obtain the reference motion feature quantity, which is the motion feature quantity in the reference motion performed by the subject, Output the comparison result between the actual operating features and the reference operating features, An information output method comprising the following features.

[0090] [Aspect 13] A sports video acquisition unit that acquires video footage of specific sports movements involving the movement of the subject's position during a match, A recognition unit that recognizes a standardized object in the sport from the acquired video, A real feature acquisition unit that measures the position of characteristic points of the subject's body during a specific athletic movement from the acquired video over time, and acquires actual movement feature quantities, which are movement feature quantities in the specific athletic movement, based on the measurement results and the recognition results of the reference object. A reference feature acquisition unit acquires reference motion feature quantities, which are motion feature quantities in the reference motion performed by the subject, An output unit that outputs the comparison result between the actual operating feature quantity and the reference operating feature quantity, An information output device equipped with the following features.

[0091] [Aspect 14] On the computer, This involves acquiring video footage of specific athletic actions in which the subject's position changes during a sporting event, and From the acquired video footage, recognize a standard object whose dimensions are standardized in the sport, From the acquired video, the positions of characteristic points of the subject's body during the specific athletic movement are measured over time, and based on the measurement results and the recognition results of the reference object, actual movement characteristics, which are the movement characteristics of the specific athletic movement, are obtained. To obtain the reference motion feature quantity, which is the motion feature quantity in the reference motion performed by the subject, Output the comparison result between the actual operating features and the reference operating features, A program that executes the command. [Explanation of Symbols]

[0092] 1 Motion evaluation system, 10 Subject, 20 Opponent player, 25 Teammate player, 50 Information terminal, 51 Motion information generation unit, 52, 160 Output unit, 53, 110 Communication unit, 100 Server, 120 Motion information acquisition unit, 130 Recognition unit, 140 Feature determination unit, 150 Storage unit, 170 Factor acquisition unit, 171 Internal factor information acquisition unit, 172 External factor information acquisition unit, 180 Additional information provision unit.

Claims

1. A sports video acquisition unit that acquires video footage of specific sports movements involving the movement of the subject's position during a match, A recognition unit that recognizes a standardized object in the sport from the acquired video, A real feature acquisition unit that measures the position of characteristic points of the subject's body during a specific athletic movement from the acquired video over time, and acquires actual movement feature quantities, which are movement feature quantities in the specific athletic movement, based on the measurement results and the recognition results of the reference object. A reference feature acquisition unit acquires reference motion feature quantities, which are motion feature quantities in the reference motion performed by the subject, An output unit that outputs the comparison result between the actual operating feature quantity and the reference operating feature quantity, An information output system equipped with the following features.

2. The system includes a factor acquisition unit that acquires factor information from the aforementioned video, which includes factors that may affect the quality of the specific athletic movements. The information output system according to claim 1, wherein the output unit outputs the factor information in relation to the comparison result.

3. The factor acquisition unit includes an internal factor information acquisition unit that acquires internal factor information indicating the psychological state of the subject as the factor. The information output system according to claim 2.

4. The factor acquisition unit includes an external factor information acquisition unit that acquires external factor information relating to the subject's surrounding conditions as the factor. The information output system according to claim 2.

5. The aforementioned external factor information includes feature quantities obtained by measuring over time the positions of feature points of moving objects other than the subject during the period in which the specific athletic action is being performed, from the video. The information output system according to claim 4.

6. The aforementioned external factor information includes motion characteristics and physical characteristics obtained by measuring the positions of the characteristic points of the opposing player's body in the sport over time, as the positions of the characteristic points of the moving object. The information output system according to claim 5.

7. The aforementioned external factor information includes motion characteristics and physical characteristics obtained by measuring over time the positions of the physical characteristics of the subject's teammate in the sport, as the positions of the characteristic points of the moving object. The information output system according to claim 5.

8. The aforementioned external factor information includes the measurement results of motion features obtained by measuring over time the position of the feature points of a ball used by the subject as the position of the feature points of the moving object. The information output system according to claim 5.

9. The system further includes an audio information acquisition unit that acquires audio information indicating the audio corresponding to the aforementioned video, The aforementioned external factor information includes an audio feature quantity relating to at least one of the spectator's vocalizations included in the audio information and the sounds emitted by the equipment used in the sport. The information output system according to claim 4.

10. The aforementioned external factor information includes equipment condition information indicating the condition of the equipment used to show the situation of the match in the sport, The information output system according to claim 4.

11. The aforementioned external factor information includes location status information indicating the state of the place where the specific athletic action was performed. The information output system according to claim 4.

12. This involves acquiring video footage of specific athletic actions in which the subject's position changes during a sporting event, and From the acquired video footage, recognize a standard object whose dimensions are standardized in the sport, From the acquired video, the positions of characteristic points of the subject's body during the specific athletic movement are measured over time, and based on the measurement results and the recognition results of the reference object, actual movement characteristics, which are the movement characteristics of the specific athletic movement, are obtained. To obtain the reference motion feature quantity, which is the motion feature quantity in the reference motion performed by the subject, The comparison result between the actual operating feature quantity and the reference operating feature quantity is output to the output device, An information output method comprising the following features.

Images