Football activity classification

Abstract

A method of determining events participated by an athlete includes receiving, at a computing device, a plurality of motion determinations generated by a monitor from motion data captured from a motion of the athlete during a monitoring window. Athletic determination includes actions performed by the athlete, performance metrics of the athlete, or both. The method further includes classifying which of the plurality of predetermined events the athlete participates during the monitoring window, by using a machine learning model stored on the computing device, based at least in part on the plurality of motion determinations.

Description

Technical Field

[0001] The described embodiments generally relate to the application of machine learning models to determine which of a plurality of predefined possible events an athlete will participate in, based on determinations about the athlete's activities derived from motion data captured from the athlete's motion. Background Art

[0002] In the field of professional sports, it has become common practice to record the activities of athletes during games and in preparation for games for the purpose of analyzing athlete performance and optimizing training. Information recorded for this purpose can include video recordings of the areas where the athletes are active as well as measurements of specific quantitative factors for each athlete. For example, an athlete can wear a motion sensor or a biometric sensor while participating in a game or during training to facilitate the analysis of the athlete's individual performance.

[0003] A large amount of data related to an athlete's team can be difficult to manage. Records are difficult to identify as having been obtained during a game, training, or other event unless manually classified. Data from different sources (such as video recordings and motion sensor data) are not synchronized when created and thus cannot be considered together unless collated by a human analyst. Therefore, attempting to gain more insightful information from the type and quantity of data currently available about athlete performance can be a time-consuming task. Summary of the Invention

[0004] The systems and methods herein apply machine learning techniques to identify, when generating a record of an athlete's performance, the type of activity in which the athlete is engaged. According to some embodiments, a system can include a wearable monitor configured to measure the motion of an athlete, thereby capturing motion data from the athlete's motion and generating a motion determination based on the motion data. In some embodiments, the wearable monitor is configured with a machine learning model for generating a motion determination based on the motion data. The motion classification can include individual actions performed by the athlete, performance metrics of the athlete, or both. According to some additional embodiments, the machine learning model can analyze a set of motion classifications and classify the motion data, the motion determination, or both, based on which of a plurality of predefined events the athlete was participating in when the motion data was captured. According to some additional embodiments, the machine learning model can run on a computing device separate from the wearable monitor.

[0005] In some embodiments, using a machine learning model to classify which event or events an athlete participated in and to determine when the athlete's participation in a single event began and ended based on information reported by a wearable monitor can accelerate the analysis of the athlete's performance, including providing improved activity recognition and classification. For example, after the machine learning model separates the data into warm-up and game data, the system can provide analysis tools to facilitate the analysis of warm-up data and game data over multiple days to evaluate which warm-up types lead to better game performance. The use of the machine learning model can also speed up the synchronization of wearable monitor information with video footage because the machine learning model can identify the start or end times of specific events in the information and the footage without manual supervision.

[0006] Aspects of the present disclosure relate to a method for determining an event in which an athlete participated. The method may include: receiving, at a computing device, a plurality of motion determinations generated based on motion data of an athlete captured during a monitoring window. The motion determination may include at least one of an action performed by the athlete and a performance metric of the athlete. The method may include: classifying the event at least in part based on the plurality of motion determinations by using a machine learning model stored on the computing device and based on the motion determinations. The event may represent a classification of the plurality of motion determinations. The method may include generating a graphical user interface that visualizes the event with respect to time-related parameters.

[0007] In some embodiments according to the above, the method may include: generating, by the computing device, a timeline of an event in which the athlete participated at least in part based on a set of motion determinations including the plurality of motion determinations and by applying the machine learning model to the set of motion determinations.

[0008] In some embodiments according to any of the above, generating the timeline of the event may include: classifying each of the plurality of motion determinations as being generated from motion data captured during each of the events in the timeline of the event.

[0009] In some embodiments according to any of the above, the timeline of the event in which the athlete participated may be an output timeline. The method may include: training the machine learning model by submitting a training timeline to the machine learning model prior to the receiving step to identify the event in which the athlete participated. Each training timeline may include a plurality of sample motion determinations and an indication of when the sample event occurred.

[0010] In some embodiments according to any of the above, the indication of when the sample event occurred may include an event type tag associated with the sample motion determination among the plurality of sample motion determinations.

[0011] In some embodiments according to any of the above, the timeline may be a filtered timeline. Generating the timeline may include: generating an unfiltered timeline of events in which the athlete participates by applying a machine learning model to a set of motion determinations and based in part on the set of motion determinations. Generating the timeline may further include: filtering the unfiltered timeline of events by changing the start times of individual events within the timeline of events to conform to filtering rules.

[0012] In some embodiments according to any of the above, the filtering rules may include the possible durations of events among a plurality of predetermined events.

[0013] In some embodiments according to any of the above, the plurality of predetermined events may include exercise, sports training, and sports competitions.

[0014] In some embodiments according to any of the above, the method may include: training a machine learning model to classify events in which an athlete participates based on motion determinations. The training may include: creating a plurality of test motion determinations based on the motion of a test athlete during a test window. The training may further include: using the machine learning model and based on the plurality of test motion determinations, outputting a test event classification of which event among a plurality of predetermined events the test athlete participated in during the test window. The training may further include: correcting the test event classification based on a record of the events in which the test athlete participated during the test window.

[0015] In some embodiments according to any of the above, the method may include: determining the role of an athlete in a team sport by applying a machine learning model to a plurality of motion determinations, at least in part based on the plurality of motion determinations, by a computing device.

[0016] In some embodiments according to any of the above, the motion determination may include an action performed by the athlete. The action performed by the athlete may include any one or any combination of kicking a ball, stepping, dribbling, and running.

[0017] In some embodiments according to any of the above, the motion determination may include performance metrics. The performance metrics may include any one or any combination of distance traveled, speed of travel, and kicking force.

[0018] In some embodiments according to any of the above, classifying which event among a plurality of predetermined events the athlete participated in during a monitoring window by using a machine learning model may further be based on a video recording of the athlete during the monitoring window.

[0019] In some embodiments according to any of the above, the method may include: training a machine learning model to classify an event in which an athlete participates based on video footage. The training may include: creating a labeled video by labeling a training video footage of an athlete participating in an event among a plurality of predetermined events with the start time of the event among the plurality of predetermined events. The training may further include: training a machine learning model based on the labeled video to identify participation in an event among the plurality of predetermined events.

[0020] Some aspects of the present disclosure relate to a system. The system may include a wearable sensor configured to measure the movement of a wearer of the wearable sensor. The system may further include a controller configured to generate a motion determination from motion data captured by the wearable sensor. The motion determination may include any one or both of an action performed by the wearer and a performance metric of the wearer. The system may include a computing device including a processor and a non-transitory computer-readable medium, wherein the non-transitory computer-readable medium bears instructions that, when read by the processor, cause the processor to classify which event among a plurality of predetermined events the wearer participated in when the wearable sensor captured motion data, at least in part based on a plurality of motion determinations.

[0021] In some embodiments according to any of the above, the wearable sensor may be configured to be integrated into a wearable item.

[0022] In some embodiments according to any of the above, the wearable item may include a shoe.

[0023] In some embodiments according to any of the above, the system may include a wearable monitor including the wearable sensor and the controller.

[0024] In some embodiments according to any of the above, the computing device may be remote from the wearable sensor.

[0025] In some embodiments according to any of the above, the computing device may include any one or any combination of a smart device, a laptop computer, a desktop computer, or a cloud computing system.

[0026] In some embodiments according to any of the above, the instructions may be part of a machine learning model.

[0027] In some embodiments according to any of the above, the system may include an equipment sensor configured to be integrated into a sports equipment and measure the movement of the sports equipment, wherein when the instructions are read by the processor, the instructions cause the processor to classify which event among a plurality of predetermined events the wearer participated in when the wearable sensor captured motion data, at least in part based on the measurements made by the equipment sensor.

[0028] In some embodiments according to any of the above, the events among the plurality of predetermined events may include any one or any combination of warm-up, training, and competition.

[0029] In some embodiments according to any of the above, when the instruction is read by the processor, the instruction may cause the processor to classify which event among the plurality of predetermined events the wearer has participated in, based in part on the rules of the sport.

[0030] In some embodiments according to any of the above, the rules of the sport may include the duration of the competition.

[0031] In some embodiments according to any of the above, when the instruction is read by the processor, it may cause the processor to classify which event among the plurality of predetermined events the wearer has participated in, based in part on the video recording of the wearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1A A system according to some aspects of the present disclosure is shown.

[0033] Figure 1B The data flow within the system according to some aspects of the present disclosure is shown.

[0034] Figure 2A It is a diagram of action classification according to some aspects of the present disclosure.

[0035] Figure 2B It is a diagram of performance indicators according to some aspects of the present disclosure.

[0036] Figure 2C It is a diagram of equipment indicators according to some aspects of the present disclosure.

[0037] Figure 3A It is a schematic diagram of the input and output of a machine learning model according to some aspects of the present disclosure.

[0038] Figure 3B An alternative output of the machine learning model according to some aspects of the present disclosure is shown.

[0039] Figure 4 It is a flowchart of a training method for a machine learning model according to some aspects of the present disclosure.

[0040] Figure 5 It is a flowchart of another training method for a machine learning model according to some aspects of the present disclosure. DETAILED DESCRIPTION

[0041] The concepts of the present disclosure will be described in detail with reference to its embodiments as illustrated in the accompanying drawings. References to "some embodiments", "an embodiment", "embodiments", "exemplary embodiments", etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but each embodiment does not necessarily include that particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is considered within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0042] Figure 1A System 100 is shown including athlete monitor 110 and computing device 120. Athlete monitor 110 can be a wearable monitor. Athlete monitor 110 can be configured to generate a motion determination of the wearer of athlete monitor 110 (e.g., athlete 118) based on the wearer's motion. FIG. 1 shows one athlete 118 and athlete monitor 110 for simplicity, but system 100 can include any number of athlete monitors 110 worn by different athletes 118. For example, each member of athlete 118's team can wear a corresponding athlete monitor 110, and all of the athlete monitors 110 can be part of system 100.

[0043] Computing device 120 can receive and store motion data from any number of athlete monitors 110 and provide the received motion data to a trained machine learning model 160 to classify one or more activities or events in the motion data. For example, machine learning model 160 can be trained to identify different events (e.g., games, training, or workouts) in the motion data and to identify different athletes in the motion data. Machine learning model 160 can then provide these classifications to a graphical user interface and / or analysis tool for further processing. As a non-limiting example, the motion data can include video data (e.g., video of a game, training, and / or workout session), and machine learning model 160 can automatically add visual annotations to the video data to indicate the identified classifications. In this particular context, a workout can refer to an activity whose primary purpose is to improve the physical health aspects of athlete 118. In contrast, in this particular comparison, training can refer to an activity whose primary purpose is to develop the skills of athlete 118. As another example, machine learning model 160 can be configured to generate a visual report including the classifications, accompanied by timestamped images or data indicating where the classifications occur within the video data.

[0044] Machine learning model 160 according to various embodiments can be any type of machine learning model 160. Thus, in some embodiments, machine learning model 160 can be, for example, a neural network.

[0045] The machine learning model 160 classifies motion determinations based on motion data provided by the monitor 110. These classifications can include one or more events from a plurality of predetermined types of events. Depending on the desired output, the machine learning model 160 can be trained to identify different types of events based on the target activity, such as soccer, rugby, tennis, etc. For example, for soccer, different types of events can include: activities not involving soccer (e.g., jogging, sprinting), training using a soccer ball (e.g., passing, penalty kicks, free kicks), and activities simulating a soccer game. For the present disclosure, the terms "event" and "activity" can be used interchangeably to refer to classifications applicable to motion data.

[0046] In some embodiments, the machine learning model 160 can be trained using annotated video data. In some embodiments, the video data can be annotated with information such as which portions of the video depict a particular activity in a set of predetermined activities or predetermined events (e.g., training, game, or workout). In further embodiments, the video data can be further annotated with subtypes of any predetermined activity or predetermined event, e.g., the type of workout being performed.

[0047] In some embodiments, the machine learning model 160 can be further trained based on the information stream of the monitor 110 (e.g., one or more sets of motion determinations generated by the monitor 110). By learning from the annotated video data, in combination with the motion determinations generated by the monitor 110 - the motion data captured during the events described in the annotated video - the machine learning model 160 can learn which patterns of motion determinations tend to occur in the motion data captured during each type of event from a plurality of predetermined events. For example, according to some embodiments, the machine learning model 160 can learn that an extended sequence of jogging or sprinting tends to occur during a workout, while an extended repetition of actions such as passing, penalty kicks, free kicks, etc. tends to occur during training. For example, according to some embodiments, the machine learning model 160 can learn that an uneven mixture of various actions (e.g., running, kicking, and passing) tends to occur during a game.

[0048] In some embodiments, the machine learning model 160 is trained to output the event in which the athlete 118 is participating at the time the motion data submitted to the machine learning model 160 is captured. For example, the machine learning model 160 according to some embodiments is trained to receive a plurality of motion determinations and classify the event or activity in which the athlete 118 is participating while wearing the monitor 110 that generates the motion determinations.

[0049] The athletic data is captured by an athlete monitor 110, which may be a wearable athlete monitor. Athlete monitor 110 may include sensor 111 and controller 115. Because athlete monitor 110 may be a wearable athlete monitor, sensor 111 may be a wearable sensor. Similarly, controller 115 may be a wearable controller.

[0050] Sensors 111 may include motion sensors configured to capture motion data associated with the motion of athlete 118 over a configurable time period. The motion sensors may include, for example, accelerometers, magnetometers, gyroscopes, global positioning systems (“GPS”), pedometers, posture sensors, impact sensors, pressure sensors, and any number of the above sensors, any one or any combination thereof. In further embodiments, sensors 111 may include a heart rate monitor, a temperature sensor, a respiration sensor, a posture sensor, a lactate sensor, and any number of the above sensors, any one or any combination thereof.

[0051] Controller 115 may be configured to generate motion determinations from motion data captured by sensor 111. In the present disclosure, the motion determinations generated by athlete monitor 110 are distinct from the event classifications generated by machine learning model 160. The motion determinations are generated based on the motion data provided by sensor 111, while the event classifications are generated based on the motion determinations provided by athlete monitor 110.

[0052] Since the athlete monitor 110 includes a controller 115, the athlete monitor 110 according to some embodiments may be configured to generate a motion determination from the motion data captured by the sensor 111. The motion determination may include any one or both of the actions performed and the performance index. In some embodiments, the athlete monitor 110 may be configured with a machine learning model to process the motion data captured by the sensor 111, and generate a motion determination. Similar to the machine learning model 160 of the computing device, the machine learning model of the athlete monitor 110 may be any one of various types of machine learning models. In some embodiments, the machine learning model of the athlete monitor 110 is a neural network. In some embodiments, the motion data captured by the athlete monitor 110 may also include a timestamp, and the timestamp identifies the time when the captured motion is recorded. Therefore, for example, in some embodiments where the athlete monitor 110 generates a motion determination that the athlete 118 kicks a ball, the athlete monitor 110 may be further configured to determine when the athlete 118 kicks a ball based on the corresponding timestamp. In another example, in certain embodiments where the athlete monitor 110 generates a motion determination that the athlete 118 travels, the athlete monitor 110 may be further configured to determine the distance that the athlete 118 travels within a specific time interval.

[0053] In some embodiments, the athlete monitor 110 may be configured to be integrated into a wearable item. The wearable item may include, for example, a shoe 116, a wristband, a glove, a shirt, a headband, a hat, or any other wearable item. The athlete monitor 110 of the illustrated embodiment may be integrated into the shoe 116 by inserting the athlete monitor 110 into the insole 112 and then disposing the insole 112 within the shoe 116. Thus, the athlete 118 may wear the athlete monitor 110 of the illustrated embodiment by wearing the shoe 116, while the athlete monitor 110 remains inserted within the insole 112 and the insole 112 remains disposed within the shoe 116. The athlete monitor 110 according to other embodiments may be integrated into the shoe 116 or other wearable item—by any process and tool suitable for the type of item and type of activity that the athlete monitor 110 is intended for.

[0054] The computing device 120 may include a processor and a memory. The memory may store instructions that, when read by the processor, cause the processor to determine which of a plurality of predetermined events the athlete 118 (when performing the movements on which the plurality of movement determinations are based) participated in, at least in part based on the plurality of movement determinations. The instructions stored on the medium may include a portion of the machine learning model 160.

[0055] In some embodiments, the computing device 120 may be remote from the athlete monitor 110. That is, the computing device 120 may be a separate physical device from the athlete monitor 110. Thus, in some embodiments, the computing device 120 may include, for example, any one or any combination of a smart device, a laptop computer, a desktop computer, and a cloud computing system. In such an embodiment, the athlete monitor 110 is configured to transmit, as an input, the output (i.e., the movement determination) from its machine learning model to the machine learning model 160 in the computing device 120. Thus, within the system 100 according to some embodiments, the athlete monitor 110 may host a first machine learning model, while the computing device 120 hosts the second machine learning model 160. In such an embodiment, the first machine learning model and the second machine learning model 160 play different roles in different steps in the process—from capturing the movement data of the athlete 118 to ultimately classifying the activity or event that the athlete participated in while wearing the athlete monitor 110.

[0056] In some embodiments, the athlete monitor 110 may generate a movement determination and report it to the computing device 120. In such an embodiment, the machine learning model 160 stored on the computing device 120 may run on hardware separate from the athlete monitor 110 to determine what event the athlete 118—when performing the movements that generated the movement determination—participated in, at least in part based on the movement determination.

[0057] An athlete monitor 110 according to various embodiments may be configured to communicate with a computing device 120 to report motion determinations via any electronic communication process, hardware, and protocol. In some embodiments, the athlete monitor 110 may be configured to wirelessly report motion determinations to the computing device 120. In further embodiments, the athlete monitor 110 may be configured to report motion determinations to the computing device 120 via a physical electronic connector. In further embodiments, the athlete monitor 110 may be configured to: report motion determination results to the computing device 120 by electronically connecting the athlete monitor 110 to one or more wires of the device 120. In some embodiments, the athlete monitor 110 may be configured to: report motion determinations to the computing device 120 when motion determinations are made in real time, such as when the athlete monitor 110 is worn by an active athlete 118. In further embodiments, the athlete monitor 110 may be configured to batch report motion determinations to the computing device 120, such as after an athlete has completed participation in one or more events while wearing the athlete monitor 110. In further embodiments, the athlete monitor 110 may be configured to transmit information to an intermediary, and the intermediary may be configured to transmit the information to the computing device 120. The intermediary may be, for example, a smart phone, a cloud computing system, or any other computing device. In some embodiments, the athlete monitor 110 may transmit motion determinations to the intermediary, and then the intermediary may transmit the motion determinations to the computing device. In further embodiments, the athlete monitor 110 may be configured to transmit motion data to an intermediary, the intermediary may store a machine learning model configured to derive motion determinations from the motion data, and the intermediary may be configured to transmit the motion determinations derived by the machine learning model to the computing device 120. Thus, in other embodiments, all processes described herein as being performed by the processor 115 may alternatively be performed on the intermediary device.

[0058] In some embodiments, the system 100 may optionally further include an equipment monitor 124. The equipment monitor 124 may be configured to be integrated into a sports equipment 128. The sports equipment 128 in the illustrated embodiment is a ball 128. According to various further embodiments, the ball 128 may be a ball suitable for any type of sport, including, for example, a soccer ball. According to further embodiments, the sports equipment 128 may include, for example, any type of sports ball, any type of sports "bat" (e.g., a baseball bat, a hockey stick, a golf club, a table tennis racket, or a tennis racket), a sports glove (e.g., a boxing glove), a bicycle, a paddle, a ski, a skateboard, or a surfboard for personal use (e.g., by an athlete 118) during a sports activity.

[0059] Similar to athlete monitor 110, equipment monitor 124 can include a controller and sensors. The sensors can include motion sensors, such as the motion sensors described above with respect to sensor 111 of athlete monitor 110. The controller of the equipment monitor can be similarly configured to generate an equipment determination based on motion data captured by the sensors of equipment monitor 124. The equipment determination can include, for example, the motion of equipment 128, the position of equipment 128, impacts on equipment 128, and actions performed on equipment 128. Equipment monitor 124 can also be configured to report the equipment determination to computing device 120 in any way that athlete monitor 110 can communicate with a computing device. However, in some embodiments, system 100 can lack equipment monitor 124.

[0060] Figure 1B An exemplary flow 190 of data within system 100 is shown, the components of which can be implemented as a motion determination machine learning model 192, an event classification machine learning model 194, a filtering component 196, and a visualizer 198. In some embodiments, the motion determination machine learning model 192 can be implemented in athlete monitor 110, and the event classification machine learning model 194 can be implemented in computing device 120 (e.g., machine learning model 160). The visualizer 198 can be configured to generate a visualization from the output of the event classification machine learning model 194. Examples of visualizations include: arranging the event classifications provided by the motion classification machine learning model in a time-based manner, such as in a timeline (e.g., showing a sequence of event classifications over a period of time, see Figure 3A ) or as a table (e.g., showing the relationships between event classifications, with respect to how much time the classifications occur in the source data, see Figure 3B ).

[0061] In some embodiments, the motion determination machine learning model 192 is configured to receive motion data (such as received from sensor 111) and generate a motion determination based on the motion data. In some embodiments, there can be separate motion determination machine learning models for different target activities, such as for different sports. That is, the motion determination machine learning model 192 can be trained to generate motion determinations for a specific sport (e.g., soccer), while there are other motion determination machine learning models for other sports.

[0062] In some embodiments, the motion determination machine learning model 192 can be implemented in the athlete monitor 110 based on user selection. For example, a user can upload the motion determination machine learning model 192 to the athlete monitor 110 during a soccer training session on one day, and upload a different motion determination machine learning model during a rugby training session on another day. Thus, the type of motion determination generated by the athlete monitor 110 can be customized based on the type of installed motion determination machine learning model.

[0063] In some embodiments, the output of the event classification machine learning model 194 is provided to the filtering component 196 to perform additional processing on the motion classification based on filtering rules for further organizing the motion classification.

[0064] In some embodiments, the visualizer 198 is implemented in a computing device separate from the computing device 120. For example, the visualizer 198 can be implemented in a user device such as a smart phone, laptop computer, tablet computer, or personal computer, and can be configured to generate and display the requested visualization of the event classification.

[0065] As Figure 2AAs shown, in some embodiments, motion determination includes an action 130 performed by athlete 118. The action 130 performed by athlete 118 and recognizable by controller 115 as a motion determination may include instances of actions among multiple predetermined action types. Thus, as shown in the example, controller 115 may be configured to determine when athlete 118 performs an action of a first predetermined action type 131, a second predetermined action type 132, or a third predetermined action type 133. In other embodiments, the number of predetermined action types may vary based on the type of activity and the annotated video data (on which machine learning model 160 is trained). For example, in some embodiments, athlete monitor 110 may be configured to determine only one predetermined action type. In a further example, athlete monitor 110 may be configured to determine two predetermined action types, or any other number of predetermined action types. According to some embodiments, the predetermined action types may include, for example, kicking, stepping, dribbling, running, jogging, and walking. As will be discussed below, the machine learning model on athlete monitor 110 may be trained to detect any number of predetermined action types based on the target activity (e.g., soccer, football, baseball). The action types may be predetermined based on the target activity on which the machine learning model is trained. In some embodiments, motion determination of the action type may be further based on an additional processing layer using information in addition to that obtained by sensor 111. For example, the travel speed or distance indicating walking, jogging, or running may vary according to the age, gender, or size of athlete 118. Thus, in some embodiments, motion determination of the action type may be performed based on the information obtained by sensor 111 in combination with the demographic information of the wearer of monitor 110. As a non-limiting example, a travel speed of 10 km / h may be classified as a running action or a jogging action according to the age of athlete 118. There may be configurable thresholds associated with the additional processing layer, such as a threshold age for motion determination of certain action types. If athlete 118 is 10 years old, 10 km / h may be more defined as running, while if athlete 118 is 18 years old, 10 km / h may be defined as jogging. The additional processing layer may include one or more rules and thresholds for defining motion of the action type, the motion of the action type including walking, jogging, running, high-speed running, sprinting. The additional processing layer may also be configured to perform motion determination of the action type specific to a sports activity. For example, for soccer, the processing layer may determine the kicking type according to the speed of the ball. In addition to age and sports activity-specific activities, other parameters may also affect the additional processing layer, including gender-specific actions.

[0066] As a non-limiting example of determining the type of action, when athlete 118 kicks a ball while wearing monitor 110, sensor 111 can capture motion data from the motion of athlete 118 and report the motion data to controller 115. Controller 115 can process the motion data, determine that the measurement result is consistent with kicking a ball, and thus generate a motion determination that athlete 118 performed a kick. Athlete monitor 110 can also confirm the occurrence time of each action 130, which is determined by monitor 110 from the motion data captured by sensor 111.

[0067] As Figure 2B shown, in some embodiments, the motion determination includes performance metric 140 of athlete 118. In further embodiments, the motion determination includes both action 130 performed by athlete 118 and performance metric 140 of athlete 118. In some embodiments, performance metric 140 includes a quantitative assessment of the action performed by athlete 118. Thus, monitor 110 can determine the magnitude of performance metric 140. According to some embodiments, performance metric 140 can include any one or any combination of distance traveled, travel speed, and kicking force. Thus, for example, when athlete 118 runs while wearing monitor 110, sensor 111 can capture motion data from the motion of athlete 118 and report the motion data to controller 115. Controller 115 can process the motion data, determine that the motion data is consistent with athlete 118 traveling at a specific speed, and thus generate a motion determination that athlete 118 is traveling at a specific speed. In some embodiments, controller 115 can also generate a motion determination of the distance traveled by athlete 118 within a certain time period. Athlete monitor 110 can also assign a time to each performance metric 140, indicating when performance metric 140 accurately reflects the performance of athlete 118. In some embodiments, the performance metric can be segmented into predetermined time intervals such that each motion determination related to certain performance metrics can include the value of the performance metric within a discrete time interval. For example, the travel speed performance metric can be segmented into predetermined time intervals such that each motion determination related to travel speed includes the average travel speed within one of the predetermined intervals. In a further example, the distance traveled performance metric can be segmented into predetermined time intervals such that each motion determination related to the distance traveled includes the total distance traveled within one of the predetermined time intervals. The time interval can be, for example, 0.5 seconds, 1.0 seconds, 2.0 seconds, 3.0 seconds, 4.0 seconds, 5.0 seconds, or any other time length. In some examples, the performance metric can be used to make a motion determination related to the type of action. For example, in some embodiments, controller 115 can make a motion determination of the type of action - whether athlete 118 is standing, walking, jogging, or running - based on the performance metric of travel speed or distance traveled within a time interval.

[0068] In embodiments where system 100 includes equipment monitor 124, equipment monitor 124 may generate equipment determination 150. As described above, equipment determination 150 may include, for example, the movement of equipment 128, the position of equipment 128, an impact on equipment 128, and an action performed on equipment 128. Thus, equipment monitor 124 may assign a value to each equipment determination 150. The nature of the value may depend on the type of equipment determination 150. For example, in some embodiments, the value may be the type of event that occurs to sports equipment 128 among a plurality of predetermined events. In further embodiments, the value may be the magnitude of a measured quantity associated with sports equipment 128, such as the magnitude of the force applied to sports equipment 128, the travel speed of sports equipment 128, or the acceleration of sports equipment 128. Equipment monitor 124 may also assign a time to each equipment determination 150, indicating the time at which the equipment determination 150 is accurate with respect to the state of sports equipment 128.

[0069] As Figure 3A shown, movement determinations (such as action 130, performance metric 140, or both action 130 and performance metric 140) may be submitted to machine learning model 160. Machine learning model 160 may then classify which event among a plurality of predetermined events 171 the athlete 118 participated in, while sensor 111 captures movement data from which movement determinations (i.e., either or both of action 130 and performance metric 140) are generated. The plurality of predetermined events 171 of the illustrated embodiment include workouts and competitions, such as sports competitions or games. The plurality of predetermined events 171 may also include an unknown category that machine learning model 160 may use to label movement determinations that machine learning model 160 cannot assign to another event type. In further embodiments, the plurality of predetermined events 171 may also include warm-ups for another type of event 171. In further embodiments, the plurality of predetermined events may also include training, such as soccer training or another specific type of sports training. In further embodiments, machine learning model 160 may be trained to label movement determinations as being generated by an activity that occurs in any non-zero number of predetermined events 171. In further embodiments, the plurality of predetermined events 171 may include any group of the example predetermined events 171 provided herein. In further embodiments, machine learning model 160 may be further trained to determine the role of athlete 118 in a team sport based on a plurality of movement determinations. For example, in some embodiments, machine learning model 160 may be trained to determine whether athlete 118 is participating in a soccer game as a goalkeeper or a field player while wearing monitor 110.

[0070] The machine learning model 160 can be trained to classify the events participated in by the athlete 118, at least in part, based on multiple consecutive motion determinations. Thus, the machine learning model 160 can be trained to classify the events participated in by the athlete 118, at least in part, based on the sequence of events represented by the multiple consecutive motion determinations. The times at which the monitor 110 captures motion data from the athlete 118 can be considered a series of monitoring windows, such as monitoring windows 172, 174, and 176, where the corresponding sequences of events that occur are interpreted by the machine learning model 160 as representing the athlete 118's participation in a particular type of predefined event 171. For illustrative purposes, Figure 3A monitoring windows 172, 174, 176 are shown, but the time period during which the monitor 110 is used to capture motion data from the motion of the athlete 118 can be divided into any number of monitoring windows with any ratio. Thus, in some embodiments, the method implemented by the system 100 can include: receiving, at the computing device 120, multiple motion determinations that are generated from motion data captured by the monitor 110 from the motion of the athlete 118 during a monitoring window (e.g., any one of monitoring windows 172, 174, 176). The method can also include: determining, by using the machine learning model 160 stored on the computing device 120, which of the multiple predefined events 171 the athlete 118 participated in during the monitoring window in which the motion data was captured.

[0071] Monitor 110 can obtain sufficient motion data during each of monitoring windows 172, 174, 176 for multiple motion determinations. Thus, a set of motion determinations presented to machine learning model 160 can include multiple motion determinations, and each of the multiple motion determinations can be generated by athlete monitor 110 based on motion data captured during different monitoring windows. Accordingly, in some embodiments, the method implemented by system 100 can further include: generating, by a computing device and at least partially based on a set of motion determinations, a timeline 170 of multiple events 171 in which athlete 118 participates by applying machine learning model 160 to the set of motion determinations. In this method, the set of motion determinations can include one or more pluralities of motion determinations, and each of the one or more pluralities of motion determinations can be generated from motion data captured during different individual monitoring windows. The generated timeline 170 of events 171 can thus span multiple monitoring windows 172, 174, 176, where machine learning model 160 can infer that events occurring in some monitoring windows are different from events occurring in other monitoring windows. In some embodiments, generating timeline 170 of events 171 includes classifying each of the multiple motion determinations as being generated from motion data captured during respective ones of events 171 in timeline 170 of events 171.

[0072] In embodiments where the system includes equipment monitor 124, equipment determination 150 can also be submitted to machine learning model 160. Accordingly, in some embodiments, machine learning model 160 can be trained to further classify events 171 in which athlete 118 participates, at least partially based on equipment determination 150. However, in some embodiments, machine learning model 160 can be trained to classify events 171 in which athlete 118 participates without equipment determination 150.

[0073] In some embodiments, video recordings 180 of athlete 118 during one or more monitoring windows 172, 174, 176 can also be submitted to machine learning model 160. Accordingly, in some embodiments, machine learning model 160 can be trained to further classify events 171 in which the athlete participates during one or more monitoring windows 172, 174, 176, at least partially based on the video recordings of athlete 118 during the one or more monitoring windows 172, 174, 176.

[0074] In some embodiments, the timeline 170 can be synchronized with the video recording 180 such that each motion determination in the timeline 170 corresponds to a portion of the video recording 180. Thus, the timeline 170 according to some embodiments can be used within a user interface, such as a digital graphical user interface. In some such interfaces, the user can navigate to a point within the video recording 180 by selecting a point along the timeline 170 or by selecting a motion determination within the timeline 170. That is, a graphical user interface according to some embodiments can display multiple motion determinations within the timeline and be configured such that selecting one of the motion determinations causes the device displaying the graphical user interface to further display a portion of the video recording 180 that is temporally corresponding to the motion data upon which the selected motion determination is based. In a further embodiment, the graphical user interface can indicate the start and end of the monitoring windows 172, 174, 176 along the timeline. In some such embodiments, the graphical user interface can be configured such that selecting one of the monitoring windows 172, 174, 176 causes the device displaying the graphical user interface to further display a portion of the video recording 180 that is temporally corresponding to the selected one of the monitoring windows 172, 174, 176. The timeline 170 can be synchronized with the video recording 180 even if the machine learning model 160 did not generate the timeline 170 based on any portion of the video recording 180. In some embodiments, the machine learning model 160 can synchronize the timeline 170 with the video recording 180.

[0075] In some embodiments, the output from the machine learning model 160 can be refined based on external considerations. For example, in some embodiments, the output from the machine learning model 160 can be refined based on classification rules, such as rules related to a particular activity or sport, such as the duration of a regular game of a sport, a training schedule for a sport, a game plan for a sport, or any combination of the above. In some embodiments, the output of the machine learning model 160 can be considered unfiltered and then passed through a filter according to rules derived from real-world considerations to obtain a filtered result. For example, in some embodiments, the timeline 170 can be a filtered timeline, and the process of generating the timeline can include: first generating an unfiltered timeline by using the machine learning model 160 as described above. Then, the computing device 120 can filter the unfiltered timeline by changing the start times of the individual events 171 within the unfiltered timeline to conform to the filtering rules. Classification (which can also be considered a filtering rule) can include the real-world considerations described above, such as the possible duration of an event 171. For example, when the unfiltered timeline includes a "game" that lasts longer than the duration of a game according to the rules of the sport, the computing device 120 can create the filtered timeline 170 by delaying the start time of the game within the timeline or advancing the start time of the event 171 immediately following the game. In further embodiments, the machine learning model 160 can be trained according to any of the above real-world considerations, classification rules, or rules related to a particular sport, and can weigh these considerations when determining groupings of sports activities to assign to the individual events 171.

[0076] Figure 3B Alternative embodiments are provided that visualize event classification as different groups. The visualization of event classification can be based on time-related parameters, such as the duration of an event relative to other classifications. Figure 3B Depicted are "training", "practice", and "game" as exemplary event classifications, and duration as a time-related parameter. For example, the machine learning model 160 determines event classifications as a 10-minute training, a 15-minute practice, and a 20-minute game.

[0077] In some embodiments, a second - order analysis can be applied between an initial event classification or activity classification of the machine - learning model 160 and an output final event classification, for example, by sending the final event classification to the visualizer 198. This second - order analysis can be within the filtering component 196. According to various embodiments, the filtering component 196 can be an aspect of the machine - learning model 160 or a separate process or algorithm of the machine - learning model 160. This second - order analysis can include: applying classification rules (such as the classification rules discussed above) to the initial event classification created by the machine - learning model 160 to refine the initial event classification based on other initial event classifications created from the same batch of motion determinations. Thus, the second - order analysis can include: changing the initial event classification in the multiple event classifications to a final event classification by changing the initial event classification based on other initial event classifications in the same plurality of event classifications.

[0078] In some examples, the second - order analysis can include: re - classifying a motion determination or time portion initially classified as "unknown" to match adjacent other event classifications. In some such examples, individual motion determinations, group motion determinations, or small time portions initially classified as "unknown" and occurring within a larger window classified as a particular event (i.e., a game, a workout, a warm - up, etc.) can be re - classified as the same event as the larger window. Thus, for example, a portion of an "unknown" classification that appears in the middle of a "game" classification can be re - classified as "game". An "unknown" initial event classification that appears between two different types of other classifications can be re - classified according to which classification rule appears to be the more likely correct event classification. For example, when the classification rule includes that a certain sport has a 90 - minute game and a warm - up phase beforehand, the second - order analysis according to some embodiments can convert an initial event classification that includes 25 minutes of warm - up, 10 minutes of unknown activity after the warm - up, and 85 minutes of game after the unknown activity into a final output event classification that includes 30 minutes of warm - up and then 90 minutes of game.

[0079] In a further example, the second - order analysis according to some embodiments can also convert non - "unknown" initial classifications into different final classifications according to the classification rules. In a further example, where the classification rule includes that a particular sport has a 90 - minute game and a training session that typically lasts more than an hour, the second - order analysis can include converting an initial event classification that includes 15 minutes of workout, 15 minutes of training after the workout, and 90 minutes of game after the training into a final output event classification that includes 30 minutes of warm - up and then 90 minutes of game. In an even further example, within the same set of classification rules, the second - order analysis can include: converting an initial event classification that includes 10 minutes of workout, 50 minutes of training after the workout, and 30 minutes of game into a final output event classification of a single 90 - minute training session.

[0080] Figure 4 FIG. 2 shows a training method 200 according to some embodiments for iteratively training a machine learning model 160 by identifying errors in the output from the machine learning model 160. The monitoring step 204 includes monitoring an athlete 118 with a monitor 110 while the athlete 118 participates in one or more predetermined events 171. The athlete 118 for the training method 200 can be regarded as a test athlete 118. The time when the monitoring step 204 is performed can be regarded as a test window. A motion classification generated from the motion data captured during the monitoring step 204 can be regarded as a test motion determination.

[0081] The submission step 208 includes submitting the test motion determination to the machine learning model 160, where the test motion determination is generated by the monitor 110 from the motion data captured during the monitoring step 204. The output step 212 includes the machine learning model 160 generating an output, such as one or more event classifications, that relate to which event or events 171 among the plurality of predetermined events 171 the test athlete 118 participated in during the monitoring step 204. The output generated in the output step 204 can be regarded as a test event classification. In some embodiments, the output can include, for example, an output timeline 170 of the events 171 that the athlete 118 participated in during the monitoring step 204. The output timeline 170 can be regarded as a test timeline 170.

[0082] The correction step 216 includes creating training data by correcting the test event classification obtained in the output step 212. In some embodiments, the test event classification can be corrected based on one or more records of which event or events 171 the test athlete 118 participated in during the test window of the monitoring step 204 and when the athlete 118 started and ended participation in a particular event 171. In some embodiments, the one or more records can include a video recording of the athlete 118 during the monitoring step 204. In some embodiments, the training data can include a corrected timeline of the events 171 that the test athlete 118 participated in during the monitoring step 204. In a further embodiment, the training data includes an identification of the correction made to the test event classification obtained during the output step 212. The training step 220 includes training the machine learning model 160 based on the training data created during the correction step 216.

[0083] Figure 5Illustrated is a training method 300 according to some embodiments for training a machine learning model 160 to identify an event 171 in which an athlete 118 participates among a plurality of predetermined events 171. In the training method 300, a recording step 304 includes: recording the activities of the athlete 118 when the athlete 118 participates in an event 171 among the plurality of predetermined events 171. The event 171 that occurs during the recording step 304 can be regarded as a sample event. In some embodiments, the recording step 304 may include: each participating athlete 118 uses a monitor 110 to capture motion data from each participating action of the athlete 118. The motion classification performed by the monitor 110 based on the motion data captured during monitoring of the athlete 118's participation in the sample event can be regarded as a sample motion determination. In some embodiments, the recording step 304 may further include: obtaining a video recording of the athlete 118 participating in the sample event.

[0084] A labeling step 308 includes: labeling the records generated in the recording step 304 with an indication of when the sample event occurs. According to some embodiments, the labeling step 308 may include: labeling the video recording with the start time of the sample event. According to some embodiments, the labeling step 308 may include: labeling the video recording with the end time of the sample event. The labeling may correspond to each of the plurality of predetermined events 171 described above. Thus, event type labels related to the predetermined events can be applied, such as sports training, ball training, competitions, warm-ups, and workouts. According to some embodiments, the labeling step 308 may include: using the sample motion determination to label a portion of the video recording, where the sample motion determination is generated from the motion data obtained when the event described in the portion of the video recording occurs. According to some embodiments, the labeling step 308 may include: using activity labels to label a portion of the video recording that corresponds to the activities of the athlete 118 visible in the portion of the video recording.

[0085] According to some embodiments, the tagging step 308 may include: generating a training timeline from a sample motion determination generated from the motion data captured during the recording step 304. The tagging step 308 may further include: tagging the sample motion determination within the training timeline using event type tags that indicate the type of event 171 in which the associated athlete 118 participated during the recording step 304. Thus, using tagged records that indicate the time of occurrence of sample events may include: using event type tags to tag the sample motion determination within the training timeline. The applied tags may further include sub-event tags that indicate shorter times of occurrence within an event. For example, sports training may include sub-events such as jogging and sprinting. In a further example, ball training may include sub-events such as a particular type of kick. Thus, in some embodiments, sports training event tags may also include an indication of sub-events, such as a 25-meter jog, a 50-meter jog, a 15-meter sprint, a 25-meter sprint, or a 50-meter sprint. In some embodiments, ball training event tags may also include an indication of sub-events, such as a corner kick, a free kick, a hard kick, a penalty kick, and a pass. In some embodiments, including sub-event tags in the training data of the machine learning model 160 may help the machine learning model 160 identify sub-events and understand which sequences of sub-events tend to occur in each of the multiple predefined events 171.

[0086] The training step 312 includes: training the machine learning model 160 on the tagged records created in the tagging step 308 to identify the participation of the athlete 118 in the events 171 among the multiple predefined events 171. The training step 312 may thus include: training the machine learning model 160 on the tagged records created in the tagging step 308 to identify the participation of the athlete 118 in the events 171 among the multiple predefined events 171.

[0087] The training method 300 can be used to train an untrained machine learning model 160. That is, the training method 300 does not require the machine learning model 160 to be trained to be ready to generate an output, such as in the output step 212 of the training method 200. However, when used on a machine learning model 160 that is ready to generate an output, the training method 300 can be used in combination with the training method 200 - by implementing the output step 212, the calibration step 216, and the training step 220 after the training step 312.

[0088] The embodiments described herein can also be directed to a computer program product that includes software stored on any computer-usable medium. When executed in one or more data processing devices, such software causes the data processing devices to operate as described herein. The embodiments described herein can employ any computer-usable or readable medium. Examples of computer-usable media include, but are not limited to, primary storage devices (e.g., any type of random access memory), secondary storage devices (e.g., hard disk drives, floppy disks, CD ROMs, ZIP disks, magnetic tapes, magnetic storage devices, and optical storage devices, MEMS, nanotechnology storage devices, etc.).

[0089] It should be understood that the detailed description section, rather than the summary and abstract sections, is intended to be used to interpret the claims. The summary and abstract sections may set forth one or more, but not all, of the exemplary embodiments contemplated by the inventors, and thus are not intended to limit the invention and the appended claims in any way.

[0090] The present invention has been described above in terms of functional building blocks that illustrate the implementation of specified functions and their relationships. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries can be defined as long as the specified functions and their relationships are appropriately performed.

[0091] The description of the above specific embodiments will so fully reveal the general nature of the invention that others can, by applying the knowledge of those skilled in the art, readily modify and / or adapt these specific embodiments for various applications without undue experimentation and without departing from the general concept of the invention. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments based on the teachings and guidance presented herein. It should be understood that the language or terminology herein is for the purpose of description and not of limitation, such that the terminology or language of this specification is to be interpreted by those skilled in the art in light of the teachings and guidance.

[0092] The breadth and scope of the concepts of the present disclosure should not be limited by any of the above exemplary embodiments, but should be defined only in accordance with the appended claims and their equivalents.

Claims

1. A method for determining an event in which an athlete is involved, the method comprising: receiving, at a computing device, a plurality of motion determinations generated based on motion data captured from motion data of the athlete during a monitoring window, wherein the motion determinations include at least one of an action performed by the athlete and a performance metric of the athlete; classifying an event based at least in part on the plurality of motion determinations by using a machine learning model stored on a computing device and based on the motion determinations, wherein the event represents a classification of the plurality of motion determinations; A graphical user interface is generated that visualizes events with respect to time-dependent parameters.

2. The method according to claim 1, comprising: A timeline of events in which the athlete participated is generated by the computing device based at least in part on a set of motion determinations including the plurality of motion determinations and by applying the machine learning model to the set of motion determinations.

3. The method of claim 2, wherein generating a timeline of events comprises: Respective motion determinations of the plurality of motion determinations are categorized as being generated from motion data captured during respective events in the timeline of events.

4. The method according to claim 2, wherein: The timeline of events in which the athlete participates is the output timeline, The method includes, prior to the receiving step, training the machine learning model to identify events in which athletes participate by submitting a training timeline to the machine learning model, and Each training timeline includes a plurality of sample motion determinations and indications of when the sample events occurred.

5. The method of claim 4, wherein the indication of when the sample event occurred comprises: An event type tag associated with a sample motion determination of the plurality of sample motion determinations.

6. The method of claim 2, wherein the timeline is a filtered timeline, and generating the timeline comprises: generating an unfiltered timeline of events in which the athlete participated by applying the machine learning model to the set of athletic determinations and based in part on the set of athletic determinations; as well as The unfiltered timeline is filtered by changing the start time of each event in the timeline of events to comply with the filtering rules. The method of claim 6 , wherein the filtering criteria include a possible duration of an event from among a plurality of predetermined events.

8. The method of claim 1, wherein the plurality of predetermined events include exercise, sports training, and sports games.

9. The method of claim 1, comprising training the machine learning model to classify events of engagement based on motion determination, wherein the training comprises: creating a plurality of test movement determinations based on the test athlete's movement during the testing window; determining, based on the plurality of test sports, using the machine learning model to output a test event classification for which of the plurality of predetermined events the test athlete participated in during the test window; as well as The test event classification is corrected based on a record of events in which the test athlete participated during the test window.

10. The method according to claim 1, comprising: The computing device determines a role for the player in a team sport based at least in part on the multiple motion determinations by applying the machine learning model to the multiple motion determinations.

11. The method of claim 1, wherein the motion determination comprises an action performed by the athlete, and the action performed by the athlete comprises any one or any combination of kicking, stepping, dribbling, and running.

12. The method of claim 1, wherein the motion determination includes a performance metric, the performance metric including any one or any combination of distance traveled, speed traveled, and kicking force.

13. The method of claim 1, wherein classifying which of a plurality of predetermined events the athlete participated in during the monitoring window by using the machine learning model is also based on video recordings of the athlete during the monitoring window.

14. The method of claim 13, further comprising training the machine learning model to classify events in which athletes are involved based on video recordings, wherein: The training includes: creating a tagged recording by tagging a training video recording of an athlete participating in an event in the plurality of predetermined events with the start time of the event in the plurality of predetermined events; and The machine learning model is trained based on the labeled video recordings to identify attendance at an event in the plurality of predetermined events.

15. A system comprising: a wearable sensor configured to measure motion of a wearer of the wearable sensor; a controller configured to generate a motion determination from the motion data captured by the wearable sensor, wherein the motion determination includes any or both of an action performed by the wearer and a performance indicator of the wearer; and A computing device comprising a processor and a non-transitory computer-readable medium, wherein the non-transitory computer-readable medium carries instructions that, when read by the processor, cause the processor to classify which of a plurality of predetermined events the wearer was engaged in when the motion data was captured by the wearable sensor based at least in part on a plurality of motion determinations.

16. The system of claim 15, wherein the wearable sensor is configured to be integrated into a wearable article.

17. The system of claim 15, wherein the article of wear comprises a shoe.

18. The system of claim 15, comprising a wearable monitor comprising the wearable sensor and the controller.

19. The system of claim 15, wherein the computing device is remote from the wearable sensor.

20. The system of claim 19, wherein the computing device comprises any one or any combination of a smart device, a laptop computer, a desktop computer, or a cloud computing system.

Images