Automated video tracking of athletes using inertial sensors

Abstract

A method of tracking video objects within a video recording includes receiving, at a computing device, video recording and event data from a space of a plurality of monitors. Each monitor includes a motion sensor configured to measure motion of a respective monitored video object within a space. The event data includes motion events transmitted by the monitor. The method further includes identifying and tracking monitored video objects in the video recording using a machine learning model and identifying video events performed by the monitored video objects from the video recording. The method further includes assigning, using the computing device, a respective permanent identifier to each monitored video object within the video recording based at least in part on a commonality between a video event performed by the monitored video object and a motion event transmitted by one of the monitors.

Description

Technical Field

[0001] The described embodiments generally relate to using motion data obtained from monitors on moving objects and video recordings of the moving objects to improve object tracking and motion analysis, particularly in situations where multiple overlapping objects are being tracked (e.g., monitoring a team of athletes). Background Art

[0002] In the field of professional sports, it has become common practice to record the activities of athletes during games and in preparation for games for the purpose of analyzing their performance and optimizing training. Information recorded for this purpose can include video recordings of the areas where the athletes are active and measurements of specific quantitative factors for each athlete. For example, athletes can wear motion sensors or biometric sensors when participating in games or during training to facilitate the analysis of their individual performance.

[0003] The effectiveness of video recordings can be enhanced by using image analysis tools to find athletes in the video recordings and applying trackers (commonly known as "tracklets") to these athletes. Existing image analysis tools may lose track of athletes when the images of the athletes are blurred, such as when an athlete leaves the frame, collides with another athlete, there are multiple moving and crossing objects within the same frame, or when an athlete moves behind another object within the field of view. In addition, existing image analysis tools are generally not sufficient to reliably identify athletes by their prominent features (such as jersey numbers or facial appearance), especially in situations where multiple athletes are being tracked and monitored. Therefore, it is difficult for existing tools to continuously track athletes throughout a sports event. Summary of the Invention

[0004] The systems and methods herein apply machine learning techniques to identify video events in video recordings. The systems and methods herein also coordinate the identified video events with motion data obtained from wearable monitors worn by athletes within the video recordings for analysis purposes. For example, some of the processes herein involve evaluating the similarity between video events identified by a machine learning model and data transmitted by the wearable monitors. In some embodiments, the results of such an evaluation can be used to determine, at one or more points during the duration of the video recording, which athlete wore which monitor 110. In some embodiments, this determination can be used to correct the time information in the data transmitted by the wearable monitors, thereby overcoming potential inaccuracies in the monitor timing. In some embodiments, this determination can also be used to facilitate the continuous tracking of athletes throughout the video recording, such as by confirming the identity of the athletes before and after an event where an athlete is occluded in the video recording.

[0005] Some aspects of the present disclosure relate to a method for tracking an object within a video recording. The method may include: at a computing device, receiving video recording and event data from a monitored space. The monitor may include a motion sensor configured to measure the motion of a monitored video object within the space. The event data may include motion events transmitted by the monitor. The method may further include: using a machine learning model on the computing device to identify video events performed by the monitored video object from the video recording. The method may further include: assigning a corresponding permanent identifier to the monitored video object within the video recording, at least in part based on a commonality between the video events performed by the monitored video object and the motion events transmitted by the monitor.

[0006] In some embodiments according to the above, the monitored video object may be one of a plurality of video objects. The monitor may be one of a plurality of monitors, each monitor including a motion sensor configured to measure the motion of a corresponding monitored video object. The method may include: assigning a corresponding permanent identifier to each of the plurality of monitored video objects. Assigning the corresponding permanent identifier may include: at multiple different times within the video recording, assigning a corresponding preliminary identifier to the monitored video object within the video recording. Assigning the corresponding permanent identifier may further include: finding, at least in part based on a commonality between the video events performed by the same monitored video object and the motion events transmitted by one of the monitors, the preliminary identifiers assigned to the same monitored video object within different portions of the video recording. Assigning the corresponding permanent identifier may further include: merging the found preliminary identifiers assigned to the same monitored video object into a single preliminary identifier. Assigning the corresponding permanent identifier may further include: after the merging step, converting the preliminary identifier into a permanent identifier.

[0007] In some embodiments according to any of the above, assigning the corresponding preliminary identifier may include: at a discontinuity point within the video recording, assigning a new preliminary identifier to a particular one of the monitored video objects, where the computing device is unable to determine an identity between the particular one of the monitored video objects and any of the monitored video objects that were assigned preliminary identifiers within an earlier portion of the video recording based on the video recording.

[0008] In some embodiments according to any of the above, allocating a corresponding preliminary identifier may include: identifying critical moments within the video recording, where each monitored video object is individually identifiable from the video recording at each critical moment by a machine learning model. Allocating a corresponding preliminary identifier may also include: at at least one critical moment, calculating a similarity between a motion event in the event data transmitted by each wearable monitor and a video event performed by each monitored video object to which a preliminary identifier has been allocated.

[0009] In some embodiments according to any of the above, the method may include: prior to transmitting the event data to a computing device, processing motion data captured by a motion sensor of a wearable monitor to identify a motion event.

[0010] In some embodiments according to any of the above, the machine learning model may be a first machine learning model. Processing of the motion data may include: applying a second machine learning model stored by the wearable monitor to the motion data.

[0011] In some embodiments according to any of the above, the method may include: for each permanent identifier, identifying a corresponding one of the monitors including the motion sensor, the motion sensor configured to measure the motion of the monitored video object to which the permanent identifier has been allocated.

[0012] In some embodiments according to any of the above, the method may include: applying a timestamp to one of the motion events transmitted by one of the corresponding monitors, the timestamp matching the time at which the monitored video object to which the permanent identifier has been allocated performs a corresponding video event.

[0013] In some embodiments according to any of the above, the method may include applying a timestamp to at least one of the motion events to match the time at which one of the video events occurs.

[0014] In some embodiments according to any of the above, the event data may include: a preliminary timestamp applied by the monitor to the motion event, and applying the timestamp to at least one of the motion events includes replacing one of the preliminary timestamps with a final timestamp.

[0015] In some embodiments according to any of the above, the video object may be an athlete participating in a sports event.

[0016] In some embodiments according to any of the above, each athlete may wear one of the monitors.

[0017] In some embodiments according to any of the above, the motion event may include any one or any combination of kicking a ball, running, walking, and standing.

[0018] In some embodiments according to any of the above, the motion event may include amplitude information in any form of any one or any combination of kicking force, travel distance, and speed.

[0019] In some embodiments according to any of the above, the monitor may include a controller configured to identify a motion event from a motion sensor of the monitor.

[0020] Some aspects of the present disclosure relate to a system. The system may include a wearable monitor that includes a motion sensor. The system may further include a computing device configured to receive event data transmitted by the wearable monitor. The event data may include a motion event. The computing device may include a non-transitory computer-readable medium storing a machine learning model. The machine learning model may be configured to identify a video event from a video recording. The computing device may be configured to associate the video event with the motion event.

[0021] In some embodiments according to any of the above, the wearable monitor may be configured to be integrated into a wearable item.

[0022] In some embodiments according to any of the above, the wearable item may be an insole.

[0023] In some embodiments according to any of the above, the wearable monitor may include a motion sensor and a controller configured to identify a motion event from measurements obtained from the motion sensor.

[0024] In some embodiments according to any of the above, the machine learning model may be configured to track a video object within a video recording based on the event data.

[0025] In some embodiments according to any of the above, the machine learning model may be a first machine learning model, and each of the plurality of wearable monitors stores a second machine learning model configured to identify a motion event from motion data obtained from a motion sensor of the wearable monitor.

[0026] In some embodiments according to any of the above, the wearable monitor may be a first wearable monitor, the system includes a plurality of wearable monitors including the first wearable monitor, each of the plurality of wearable monitors includes a motion sensor, and the computing device is configured to receive event data transmitted by the wearable monitor. Description of the Drawings

[0027] Figure 1AShows a system according to some aspects of the present disclosure.

[0028] Figure 1B Shows a portion of a video recording processed by the Figure 1A system.

[0029] Figure 1C Is a flowchart of a process performed by a wearable monitor according to some aspects of the present disclosure.

[0030] Figure 1D Is a flowchart of a process performed by a computing device according to some aspects of the present disclosure.

[0031] Figure 2A Is a graph of video analysis data according to some aspects of the present disclosure.

[0032] Figure 2B Is a graph of event data according to some aspects of the present disclosure.

[0033] Figure 2C Is a flowchart of a method for calibration time data according to some aspects of the present disclosure.

[0034] Figure 3A Is a flowchart of a process for analyzing video analysis data and event data according to some aspects of the present disclosure.

[0035] Figure 3B Is an illustration of a hypothesis tree according to some aspects of the present disclosure.

[0036] Figure 4 Is a flowchart of a matching analysis according to some aspects of the present disclosure.

[0037] Figure 5 Is a flowchart of a process for analyzing video analysis data according to some aspects of the present disclosure. DETAILED DESCRIPTION

[0038] The concepts of the present disclosure will be described in detail with reference to its embodiments as shown in the accompanying drawings. References to "some embodiments", "an embodiment", "embodiments", "exemplary embodiments", etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but each embodiment does not necessarily include that particular feature, structure, or characteristic. Further, such phrases do not necessarily refer to the same embodiment. Additionally, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is considered within the knowledge of those skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0039] Figure 1ASystem 100 is shown including one or more monitors 110 and a computing device 120. Each monitor 110 can be a wearable monitor. The monitor 110 can be configured to generate an activity determination of the wearer in the monitor 110 based on the movement of the wearer (e.g., athlete 118). The computing device 120 stores a machine learning model 160 trained to parse video footage.

[0040] The monitor 110 can be a wearable monitor. The monitor 110 can include a sensor 111 and a controller 115. Since the monitor 110 can be a wearable monitor, the sensor 111 can be a wearable sensor. Similarly, the controller 115 can be a wearable controller.

[0041] The sensor 111 can include a motion sensor configured to measure the movement of the athlete 118. The motion sensor can include, for example, an accelerometer, a magnetometer, a gyroscope, a global positioning system (“GPS”), a pedometer, an attitude sensor, an impact sensor, a pressure sensor, and any number of the above sensors, any one or any combination of the above.

[0042] The controller 115 can be configured to identify a motion event based on the motion data captured by the sensor 111. Since the monitor 110 includes the controller 115, the monitor 110 according to some embodiments can be configured to identify a motion event based on the measurements made by the sensor 111, where the motion event includes any one or both of the actions performed by the athlete 118 in a plurality of predetermined motion types and the performance metrics in a plurality of predetermined metrics. In some embodiments, the monitor 110 can also be configured to: determine or at least approximately determine the time at which the identified motion event occurs. Thus, for example, in some embodiments where the monitor 110 identifies a motion event including the athlete 118 kicking a ball, the monitor 110 can also be configured to determine or at least approximately determine the time at which the athlete 118 kicks the ball. In some embodiments, the motion types can include, for example, any one or any combination of kicking a ball, running, walking, and standing. In some embodiments, the metrics can include amplitude information. In some embodiments, the amplitude information can include, for example, any one or any combination of kick force, travel distance, and speed.

[0043] In some embodiments, the monitor 110 may host a machine learning model 117 to process motion data captured by the sensor 111 and generate a motion determination. Similar to the machine learning model 160 of the computing device, the machine learning model 117 of the monitor 110 can be any one of various types of machine learning models 117. In some embodiments, the machine learning model 117 of the monitor 110 is a neural network. In some embodiments, the motion data captured by the monitor 110 may further include a timestamp that identifies the time at which the captured motion was recorded. Thus, for example, in some embodiments in which the monitor 110 generates a motion determination that an athlete 118 kicks a ball, the monitor 110 may further be configured to determine when the athlete 118 kicks the ball based on the corresponding timestamp.

[0044] In some embodiments, the monitor 110 may be configured to be integrated into a wearable item. The wearable item may include, for example, a shoe 116, a wristband, gloves, a shirt, a headband, a hat, or any other wearable item. By inserting the monitor 110 into the insole 112 and then setting the insole 112 within the shoe 116, the monitor 110 of the illustrated embodiment can be integrated into the shoe 116. Thus, the athlete 118 may wear the monitor 110 of the illustrated embodiment by wearing the shoe 116, while the monitor 110 remains inserted within the insole 112 and the insole 112 remains set within the shoe 116. An athlete monitor 110 according to other embodiments may be integrated into the shoe 116 or other wearable item—by any process and tool suitable for the type of item and type of activity that the athlete monitor 110 is intended for.

[0045] The computing device 120 may include a processor and a non-transitory computer-readable medium, where the non-transitory computer-readable medium carries instructions that, when read by the processor, cause the processor to parse video footage. The instructions stored on the medium may include a portion of the machine learning model 160. The machine learning model 160 according to various embodiments can be any type of machine learning model 160. Thus, in some embodiments, the machine learning model 160 can be, for example, a neural network.

[0046] In some embodiments, the computing device 120 may be remote from the monitor 110. That is, the computing device 120 can be a separate physical device from the monitor 110. Accordingly, in some embodiments, the computing device 120 may include, for example, any one or any combination of a smart device, a laptop computer, a desktop computer, and a cloud computing system. In some embodiments, the computing device 120 can be a single device. In other embodiments, the computing device 120 can be a distributed group of collaborative devices.

[0047] In some embodiments, when implemented using controller 115, monitor 110 may be further configured to identify motion events based on motion data captured over a predetermined period of time. Non-limiting examples of motion data include, but are not limited to, speed, angular velocity, acceleration, and position. Non-limiting examples of motion events include, but are not limited to, kicking a ball, traveling at a certain rate, and being in a certain position. Controller 115 may be configured to convert the motion data into event data based on the type of object being monitored and how the monitor is applied to the object being monitored. For example, in the case where monitor 110 is located within shoe 116 and is configured to monitor athlete 118, controller 115 may be configured such that the speed in the motion data does not exactly correspond to the magnitude of the rate of the traveling motion event, since the foot of athlete 118 may move at a different speed than the overall athlete 118. Controller 115 may collect the identified motion event 155, along with the specific time of motion event 155, into the event data. Thus, the event data may include a plurality of motion events identified from the motion data, as well as time information of the motion events. As further explained below, according to some embodiments, controller 115 creates the time information based on a local clock (e.g., the clock included in monitor 110 or the clock of a mobile device with which monitor 110 communicates). In some cases, the local clock used for this purpose may not be entirely accurate. Thus, according to some embodiments, the time information in the event data may include the true time at which the motion event occurred, and according to further embodiments, the event data may include an approximation of the time at which the motion event occurred. According to further embodiments, the time information in the event data may include the specific times of the motion events relative to each other, without reference to clock time. Monitor 110 may be further configured to transmit the event data, which includes the motion events, to computing device 120. In such an embodiment, machine learning model 160 stored on computing device 120 may run on hardware separate from monitor 110.

[0048] Monitor 110 according to various embodiments may be configured to communicate with computing device 120 to transmit event data through any electronic communication process, hardware, and protocol. In a further embodiment, monitor 110 may be configured to transmit event data to computing device 120 through a physical electronic connector. In a further embodiment, monitor 110 may be configured to transmit event data to computing device 120 through one or more wires that electronically connect monitor 110 to device 120. In some embodiments, monitor 110 may be configured to transmit event data to computing device 120 when event data is identified in real time, such as when monitor 110 is worn by an active athlete 118. In a further embodiment, monitor 110 may be configured to transmit event data to computing device 120 in batches, such as after an athlete has completed participation in one or more activities while wearing monitor 110. In a further embodiment, athlete monitor 110 may be configured to transmit information to an intermediary, and the intermediary may be configured to transmit the information to computing device 120. The intermediary may be, for example, a smart phone, a cloud computing system, or any other computing device. In some embodiments, athlete monitor 110 may transmit a motion determination to the intermediary, and then the intermediary may transmit the motion determination to computing device 120. In a further embodiment, athlete monitor 110 may be configured to transmit motion data to an intermediary, the intermediary may store a machine learning model configured to derive a motion determination from the motion data, and the intermediary may be configured to transmit the motion determination derived by the machine learning model to computing device 120. Thus, in other embodiments, all processes described herein as being performed by processor 115 may alternatively be performed on the intermediary device.

[0049] According to some embodiments, system 100 may further include one or more cameras 121. The cameras 121 may be configured to capture video data of a space over a predetermined time period. The video data may include video recordings, such as video recording 122 discussed further below. In some embodiments, the video data may further include time information, such as the time at which the video recording was captured. In some embodiments, the predetermined time period of the captured video recording corresponds in time to the predetermined time period of the captured motion data from monitor 110. In this way, video events may be associated with motion events to improve the accuracy of video object detection within the video data. Video events include recognized actions within the video data and are associated with video objects within the video data. Examples of video events include, but are not limited to, action events (e.g., kicking a ball, kicking force, passing a ball), travel distance (e.g., over a predetermined time period), travel path (e.g., over a predetermined time period), and travel speed (e.g., over a predetermined time period within the video data). Since cameras 121 may capture video data of the space monitored by monitor 110 for a video object (e.g., athlete 118) during the same predetermined time that monitor 110 is used to monitor the video object, events performed by or occurring on the video object may be detected in the video data captured by cameras 121 and the motion data captured by sensors 111 of monitor 110. Thus, by processing the motion data to identify motion events (as described above with respect to monitor 110) and by processing the video data to identify video events, the motion data may be synchronized with the video data and it may be determined which video object monitor 110 applies to that is detectable in the video data. According to some embodiments, cameras 121 may be further configured to transmit the video recording to computing device 120.

[0050] According to some embodiments, system 100 may be configured to track video objects within the video recording. In some embodiments described herein, the video object may be athlete 118. Thus, any process described herein with respect to athlete 118 is an example of a concept that may be applied to generally tracking video objects in a video. Additionally, the processes described herein with respect to tracking video objects in a video may be used to track athletes in a video.

[0051] According to some embodiments, computing device 120 may be further configured to identify video events performed by video objects within the video recording. Within system 100, each monitor 110 may be configured to track motion data and measure the motion of its corresponding video object based on the tracked motion data, while cameras 121 capture video recordings of the video objects. In other words, system 100 may include synchronized motion detection and video capture capabilities, and system 100 may be configured to correlate the motion data from the motion detection with the video data provided from the video capture capabilities to improve the accuracy of object detection within the video data.

[0052] Figure 1B Shows an example frame of video recording 122 with identifier 124. Video recording 122 is included in the video data captured by camera 121. Multiple video objects 130 (athlete 118 in the illustrated example) can be detected in video recording 122. According to some embodiments, machine learning model 160 can assign an identifier 124 to each detectable video object 130 and annotate video recording 122 using the obtained identifier 124. The identifier 124 can highlight the video objects 130 and related motion data (e.g., actions, movement speed, movement path) in video recording 122, making video recording 122 easier to parse. The identifier 124 can also facilitate the analysis of individual video objects 130, for example, by enabling computer analysis to be associated with each identifier 124. The identifiers 124 can each have a unique label, such as a serial number, to distinguish them from each other. The identifier 124 can also be given different visual features to further assist in easily viewing video recording 122. Thus, in some embodiments, the identifiers 124 can each be given a different color, which allows different video objects 130 to be easily distinguished by the human eye. In a further embodiment, the identifier 124 can be given different colors according to the detectable features of the video object 130. For example, in some embodiments, the video object 130 is an athlete 118 in a team sport, and the computing device 120 can detect the jersey color of the video object 130 and assign different colors to the identifier 124 according to the jersey color.

[0053] Go to Figure 1C and Figure 1D , monitor 110 and computing device 120 can perform different analysis processes 170, 180 to associate the motion events that can be identified from the motion data with the video events that can be identified from video recording 122. Specifically referring to Figure 1C , monitor 110 can be configured to execute analysis process 170. Analysis process 170 can include an acquisition step 172. The acquisition step 172 can include: obtaining, in acquisition step 172, the motion data of video object 130 (e.g., athlete 118) wearing monitor 110 using sensor 111.

[0054] In processing step 174, the controller 115 of monitor 110 can process the motion data captured by sensor 111 of the same monitor 110 to create event data. In some embodiments, the event data created in processing step 174 can be as described below with respect to Figure 2BThe described event data 150. Accordingly, processing the motion data in processing step 174 may include identifying motion events from the motion data, such as the motion event 155 described below. As described above, in some embodiments, each controller 115 may store a machine learning model 117 different from the machine learning model 160 of the computing device 120. The machine learning model 117 stored by the controller 115 can be trained to identify motion events 155 from the motion data. Accordingly, in some embodiments, processing the motion data within processing step 174 may include: applying the machine learning model 117 stored by the controller 115 to the motion data to identify motion events 155 from the motion data.

[0055] In reporting step 176, the monitor 110 may transmit the event data 150 created in processing step 174 to the computing device 120.

[0056] Go to Figure 1D , the computing device 120 may be configured to execute process 180. Process 180 may include an assignment step 182. The assignment step 182 may include using the computing device 120 to assign an identifier 124 to a video object 130 that can be detected in the video recording 122, as described above. In some embodiments, the computing device 120 may use the machine learning model 160 to assign the identifier 124. Thus, according to some embodiments, the machine learning model 160 may be trained to identify video objects 130 within the video recording 122. In some embodiments, the identifier 124 applied in the assignment step 182 may be a preliminary identifier, which will be converted into a permanent identifier in the process described below.

[0057] The association step 184 may include: using the computing device 120 to associate video events with the identifier 124. As described above, the machine learning model 160 can be trained to identify video events from the video recording 122, such as the video event 145 described below with respect to Figure 2A Accordingly, according to some embodiments, the association step 184 may include: further using the machine learning model 160 to associate the identified video event 145 with the identifier 124, where the identifier is assigned to the same video object 130 that performs the video event 145. The video event 145 may be identified before or during the association step 184. In some embodiments, the video data 140 described further below may include the video event 145 and the association between the video event 145 and the identifier 124.

[0058] Figure 2A And 2B are exemplary events that can be compared to associate the identifier 124 with the monitor 110. Figure 2AAn example of video analysis data 140 is shown. The video analysis data 140 includes information created by analyzing video data captured by the camera 121. Thus, while the video data captured by the camera 121 may include the video recording 122 itself, the video analysis data 140 generated by the computing device 120 may include an identifier 124, the location of the identifier 124, a video event 145, and an association between the video event 145 and the identifier 124. In some embodiments, the video analysis data 140 may also include one or more portions of the video data itself. The machine learning model 160 may be trained to create the video analysis data 140 based on the video recording 122. The video analysis data 140 may include a video event 145, which is performed by a video object 130 within the video recording 122 and detected by the machine learning model 160. The video event 145 may be an event of the same predefined type as the motion events recognizable by the monitor 110. Thus, the video analysis data 140 created by the machine learning model 160 may include an identification of the same event transmitted by the monitor 110 worn by the video object 130 that appears in the video recording 122.

[0059] In some embodiments, the machine learning model 160 may also be trained to associate the video event 145 with the video object 130 identified as performing the video event 145. In some embodiments, the machine learning model 160 may associate the video event 145 with a particular video object 130 performing the video event 145 by assigning each video event 145 to one of a plurality of identifiers 141, 142, 143 (each identifier being similar to the above-mentioned identifier 124), where the identifier is assigned to the video object 130 performing the video event 145. Thus, as Figure 2A shown, the machine learning model 160 may find that the video object 130 assigned the third identifier 143 performs two video events 145, followed by the video object 130 assigned the second identifier 142 performing two video events 145, and then the video object 130 assigned the first identifier 141 performing three video events 145.

[0060] In some embodiments, the machine learning model 160 can be configured to deactivate an identifier when the video object 130 becomes blurred. In some embodiments, the machine learning model 160 can be configured to deactivate an identifier whenever the machine learning model 160 cannot find the video object 130 assigned the identifier within the video recording 122. Once the machine learning model 160 fails to find the video object 130 assigned the identifier, the identifier is immediately deactivated, which can be regarded as "fast" deactivation. In other embodiments, the machine learning model 160 can be configured to deactivate an identifier only when the video object 130 cannot be located with threshold certainty within a period of time after the video object 130 becomes undetectable within the video recording 122. Deactivating the identifier only when the video object 130 assigned the identifier cannot be located with threshold certainty within a certain period of time after the video object 130 becomes undetectable can be regarded as "slow" deactivation. According to some embodiments, the machine learning model 160 can be configured or instructed to perform fast deactivation or slow deactivation. The machine learning model 160 can also be configured to: create a new identifier whenever the machine learning model 160 finds a video object 130 without an identifier within the video recording 122. Thus, different identifiers 141, 142, 143 can be associated with the same video object 130 at different times within the video recording 122. According to some embodiments, the machine learning model 160 can be trained to perform further analysis to find multiple identifiers assigned to the same video object 130 at different times. Thus, Figure 2A the identifiers 141, 142, 143 in the analysis phase represented in

[0061] Go to Figure 2B , the event data 150 transmitted by the monitor 110 to the computing device 120 can include: a motion event 155, and the identity of the monitor 110 that transmitted each part of the event data 150. In the illustrated example, the event data 150 transmitted by the first monitor 151 in the monitor 110 includes three motion events 155. Also in the illustrated example, the event data 150 transmitted by the second monitor 152 in the monitor 110 includes four motion events 155.

[0062] According to some embodiments, computing device 120 may be configured to find pairs of preliminary identifiers assigned to the same monitored video object 130 within different portions of video recording 122, at least in part based on a commonality between video event 145 performed by the same video object 130 and motion event 155 transmitted by one of monitors 110 (including monitors 151, 152). That is, computing device 120 may be configured to compare video analysis data 140 with event data 150 to find the sequence of video events 145 that is most similar to the sequence of motion events 155 transmitted by a particular monitor 110 (including monitors 151, 152). In some embodiments, evaluating the commonality may include, for example, matrix step 412 of matching analysis 400, as further described below. In further embodiments, evaluating the commonality may include analysis phase 408 of matching analysis 400 as further described below. In further embodiments, evaluating the commonality may include matching analysis 400 as further described below. Thus, in the illustrated example, computing device 120 may conclude from the similarity of the particular times and patterns of video events 145 (assigned to second identifier 142 and third identifier 143) and the particular times and patterns of motion events 155 (transmitted by second monitor 152) that both second identifier 142 and third identifier 143 were assigned to a single video object 130 wearing second monitor 152 at different times. Computing device 120 may also conclude from the similarity of the particular times and patterns of video events 145 (assigned to first identifier 141) and the particular times and patterns of motion events 155 (transmitted by first monitor 151) that first identifier 141 was assigned to a video object 130 wearing first monitor 151. In some embodiments, computing device 120 may be configured to find pairs of preliminary identifiers assigned to the same monitored video object 130 within different portions of video recording 122 by applying machine learning model 160. According to some such embodiments, machine learning model 160 may be trained to evaluate the commonality between video analysis data 140 and event data 150. Thus, in some embodiments, any of the steps described above that are performed by the machine learning model - regarding drawing conclusions based on a comparison of video analysis data 140 and event data 150 - may be implemented by applying machine learning model 160.

[0063] After determining which initial identifier or identifiers 141, 142, 143 are assigned to the same video object 130 throughout the video recording 122, the machine learning model 160 can replace the initial identifiers 141, 142, 143 with permanent identifiers that are assigned one-to-one to the corresponding video objects 130. The machine learning model 160 can assign permanent identifiers to each monitored video object 130 within the video recording 122 based at least in part on commonalities between video events 145 (performed by the monitored video object 130) and motion events 155 (transmitted by one of the monitors 110, 151, 152). Thus, by using motion events 155 to merge multiple initial identifiers into a single permanent identifier, the machine learning model 160 can track video objects 130 within the video recording 122 based on event data 150. Replacing the initial identifiers 141, 142, 143 with permanent identifiers can include merging any two or more initial identifiers 141, 142, 143 that are assigned to the same video object 130 at different times within the video recording 122 into a single permanent identifier. Thus, in the example shown, the machine learning model 160 can replace the first initial identifier 141 with a permanent identifier corresponding to the first monitor 151. Additionally, in the example shown, the machine learning model 160 can merge the second initial identifier 142 and the third initial identifier 143 into a single permanent identifier corresponding to the second monitor 152.

[0064] Using computing device 120 to associate video event 145 with motion event 155 by finding which identifiers (e.g., any of the identifiers 124, 141, 142, 143 above) correspond to which monitors 110, 151, 152 can also help correct or assign time information to event data 150. For example, in some embodiments, monitor 110 may transmit event data 150 with a slightly inaccurate timestamp or no timestamp at all. In some examples, monitors 110 according to some embodiments may have internal clocks that may not be fully synchronized to real time, resulting in slightly inaccurate timestamps. In a further example, monitor 110 may transmit event data that only indicates the interval during which a motion event detected by monitor 110 occurred, but does not indicate a timestamp. Thus, after matching monitors 110, 151, 152 with at least one preliminary identifier 141, 142, 143, computing device 120 may apply a time offset to event data 150 received from that monitor 110. The time offset may be a constant amount of time added to or subtracted from the time within event data 150 to synchronize event data 150 with video analysis data 140. Each monitor 110, 151, 152 may have an independent clock, and thus different magnitudes of time offsets may be assigned. The time offsets can then be used in the matching steps 316, 516 and the matching analysis 400 described below.

[0065] Figure 2C Synchronization method 200 is shown. Data acquisition step 204 includes: acquiring video recording 122 of video object 130 wearing monitors 110, 151, 152. Submission step 208 includes: submitting video recording 122 and event data 150 to machine learning model 160. Association step 212 includes: after submitting event data 150 and video recording 122 to machine learning model 160, using machine learning model 160 to determine which identifiers 141, 142, 143 correspond to which monitors 110, 151, 152, as described above with respect to Figure 2B described. Timestamp step 216 includes: after machine learning model 160 determines which monitor 110, 151, 152 corresponds to which identifier or identifiers, machine learning model 160 applying a timestamp to video event 145 within event data 150 to match the time at which the corresponding video event 145 appears in video recording 122.

[0066] Video recording 122 may be accompanied by real-time information, such that in some embodiments, timestamps applied based on when video events 145 appear in video segment 122 may match the real time at which those events occur. Accordingly, timestamps applied to motion events 155 (transmitted in this manner by a particular one of monitors 110, 151, 152) may match the actual time at which a single video object 130 (to which a permanent identifier corresponding to one of monitors 110, 151, 152 is assigned) performs video event 145. In embodiments where monitors 110, 151, 152 transmit event data 150 with inaccurate timestamps, applying timestamps to event data 150 by machine learning model 160 based on video analysis data 140 may include correcting the inaccurate timestamps. In some embodiments, timestamps within event data 150 transmitted by monitors 110, 151, 152 may be considered preliminary timestamps, and correcting inaccurate timestamps may include replacing a preliminary timestamp associated with a particular motion event 155 with a final timestamp that matches the occurrence time of the corresponding video event 145.

[0067] Accordingly, in the illustrated example, machine learning model 160 may apply a timestamp to a motion event 155 transmitted by first monitor 151 that matches the time at which a video object 130 assigned a first preliminary identifier 141 may be seen to perform a corresponding video event 145 within video recording 122. Further, according to the illustrated example, machine learning model 160 may apply a timestamp to a motion event 155 transmitted by second monitor 152 that matches the time at which a video object 130 assigned second identifier 142 and third identifier 143 may be seen to perform a corresponding video event 145 within video recording 122.

[0068] Figure 3A A process 300 for merging preliminary identifiers into permanent identifiers is shown. In process 300, detection step 304 includes using machine learning model 160 to detect video object 130 within video recording 122. Assignment step 308 includes assigning a preliminary identifier to the detected video object 130. In some embodiments, machine learning model 160 may be further configured to cause a preliminary identifier to follow the video object 130 to which it is assigned.

[0069] When a video object 130 with a previously deactivated identifier becomes detectable within the video recording 122, a discontinuity point may occur. Such a discontinuity situation may be because the machine learning model 160 cannot confirm the identity between the re-detected video object 130 and any video object 130 that was previously assigned a preliminary identifier based solely on the video recording 122. Therefore, it cannot be assumed that the machine learning model 160 can reliably apply the same preliminary identifier before and after these discontinuity points. Thus, in some embodiments, the machine learning model 160 may be configured to deactivate the preliminary identifier at any point within the video recording 122 where the machine learning model 160 cannot locate the video object 130 assigned the preliminary identifier based solely on the video recording 122. The machine learning model 160 according to such embodiments may be further configured to generate a new preliminary identifier and assign the newly generated preliminary identifier to the detected video object 130 that lacks a preliminary identifier at each discontinuity point. Thus, according to some embodiments, when the video object 130 enters and exits a condition that occludes it from the video recording 122, the machine learning model 160 may repeatedly deactivate the preliminary identifier and generate and assign new preliminary identifiers throughout the video recording 122. Thus, a series of different preliminary identifiers may be assigned to a single video object 130 during the course of the video recording 122. Thus, the assignment step 308 may include: at multiple different times within the video recording, assigning corresponding preliminary identifiers to the monitored video objects within the video recording.

[0070] The generation step 312 may include: generating a hypothesis tree for each monitor 110, such as Figure 3B the hypothesis tree 323 shown in. Each branch of the hypothesis tree 323 represents a different preliminary identifier that may be assigned to the video object 130 that is the same as the previous preliminary identifier based on the analysis of the video recording 122 by the machine learning model 160. Thus, the hypothesis tree 323 may include a branch of the first preliminary identifier 324. Each first preliminary identifier 324 may perhaps be the earliest preliminary identifier assigned to the video object 130 wearing the monitor 110 that provides the root of the hypothesis tree 323.

[0071] Each branch in the hypothesis tree ends with the deactivation 325 of the preliminary identifier corresponding to that branch. Thus, the deactivation 325 may correspond to a discontinuity point, meaning that multiple preliminary identifiers may follow after the deactivation 325, and these preliminary identifiers may be assigned to the same video object 130 together with the deactivated preliminary identifier. Thus, as Figure 3BAs shown in the example of , each first preliminary identifier 324 may end at a deactivation 325. Additionally, multiple second preliminary identifiers 326 may follow the deactivation 325 of the first preliminary identifier 324, and these second preliminary identifiers may have been assigned to the same video object 130 as the deactivated first preliminary identifier 324. Each second preliminary identifier 326 may eventually reach its own deactivation 325, after which one or more third preliminary identifiers may follow, and these third preliminary identifiers may have been assigned to the same video object 130 as the deactivated second preliminary identifier 326, and so on until the end of the video recording 122. Any number of deactivations 325 and new preliminary identifiers may be generated based on events within the video recording 122. Generally, any preliminary identifier generated and assigned after an earlier preliminary identifier is deactivated may be assigned to the same video object 130 as the deactivated preliminary identifier.

[0072] Returning to Figure 3A , the matching step 316 may follow the generation step 312. The matching step 316 may include: analyzing the video recording 122 and the event data 150 and the hypothesis tree 323 to determine which one or more preliminary identifiers have been assigned to the same video object 130 as each monitor 110.

[0073] The matching step 316 may include a matching analysis 400 as shown in Figure 4 . The matching analysis 400 includes a critical moment identification step 404. The critical moment identification step 404 may include: identifying critical moments in the video recording 122. In this case, a critical moment is a moment in the video recording 122 when each monitored video object 130 can be detected within a frame by the machine learning model 160. Thus, according to some embodiments, a critical moment may include a portion of the video recording 122 during which the number of video objects 130 detectable within a frame is at least as great as the number of monitors 110 active during the event recorded by the video recording 122. Critical moments are separated by certain portions of the video recording 122 during which not all monitored video objects 130 are so identifiable by the machine learning model 160. At the end of each critical moment, at least one preliminary identifier may be deactivated. Additionally, each critical moment begins with the detection of at least one previously undetected video object 130, which means that a critical moment may coincide with a discontinuity point as described above.

[0074] In the analysis phase 408, each critical moment identified in the critical moment identification step 404 is analyzed separately. The analysis phase 408 for each critical moment includes a matrix step 412. The matrix step 412 may include: calculating a similarity matrix between the video event 145 (associated with each preliminary identifier active at the critical moment) and the motion event 155 (associated with each monitor 110). Thus, for each pair of a monitor 110 and one of the preliminary identifiers (active at the relevant critical moment), matrix elements may exist. The similarity can be calculated in various ways. In some examples, the similarity can be calculated as the mean square error between the motion event 155 of the monitor 110 of the matrix element and the video event 145 of the preliminary identifier of the same matrix element. In some embodiments, a pair of a monitor 110 and one of the preliminary identifiers that does not correspond to any branch of any hypothesis tree 323, or a pair that conflicts with the result of any completed binary analysis 416 in the analysis tree 323, may be omitted as matrix elements.

[0075] The analysis phase 408 may further include a binary analysis step 416 after the matrix step 412. The binary analysis step 416 may include applying a binary matching process to the matrix created in the matrix step 412. The binary analysis can be performed in various ways. In some examples, the binary analysis can be performed according to a linear assignment method. Before performing the binary matching process, the hypothesis tree 323 may be referred to. Thus, for the purpose of binary analysis, any matrix elements corresponding to a pair of a monitor 110 and a preliminary identifier that cannot occur in any hypothesis tree can be ignored. Thus, in the binary matching process, impossible pairs of preliminary identifiers and monitors 110 can be excluded from consideration.

[0076] The analysis phase 408 is performed separately for each critical moment identified in the critical moment identification step 404. Accordingly, for each analysis phase 408 performed after the first analysis phase 408 for a single video recording 122, the previously performed analysis phase 408 can be considered in the binary analysis step 416. For example, each completed analysis phase 408 identifies some branches in some hypothesis trees 323 as true. Thus, for subsequent analysis phases 408, it can be assumed that the previous analysis phase 408 is correct, thereby excluding other branches of the hypothesis tree 323. Thus, the binary analysis step 416 may include excluding any pair of a preliminary identifier and a monitor 110 that is incompatible with all previously completed analysis phases 408 in the hypothesis tree 323.

[0077] After completion of the analysis phase 408, an examination 420 can be performed to determine whether the analysis phase 408 has been completed for each critical moment. If the analysis phase 408 has not been completed for each critical moment, the matching analysis 400 can continue with the analysis phase 408 for the next critical moment. In some embodiments, the analysis phase 408 is performed in chronological order of the critical moments. It is assumed that the hypothesis tree 323 has fewer branches at an earlier critical moment than at a later critical moment. Thus, by performing the analysis phase 408 in chronological order of the critical moments, the conclusions from each analysis phase 408 can be used to eliminate the branches considered in a later analysis phase 408, thereby saving computational resources.

[0078] If it is determined at the examination 420 that the analysis phase 408 has been completed for each critical moment, the matching analysis 400 can proceed to an end step 424. The end step 424 can include merging all the preliminary identifiers found to correspond to the same monitor 110 into one permanent identifier. Thus, performing the analysis phase 408 for each critical moment can find the preliminary identifiers assigned to the same monitored video object 130, and the end step 424 can include converting the found preliminary identifiers assigned to the same monitored video object 130 into a single permanent identifier. Each resulting permanent identifier can be assigned to a single video object 130 in the entire video recording 122. The machine learning model 160 can annotate the video recording 122 with the permanent identifiers so that each permanent identifier follows the video object 130 wearing the corresponding monitor 110 throughout the video recording 122.

[0079] Figure 5 A process 500 according to a further embodiment is shown. The process 500 according to some embodiments can be substantially similar to the above process 300. Thus, the process 500 includes a detection step 504, an assignment step 508, a generation step 512, and a matching step 516, which may be similar to the above detection step 304, assignment step 308, generation step 312, and matching step 516 in some or all respects. Thus, similar to the above matching step 316, the matching step 516 can include a matching analysis 400.

[0080] Process 500 also includes an intermediate processing step 510 between the allocation step 508 and the generation step 512. The intermediate processing step 510 includes: analyzing the video recording 122 using the computing device 120 to combine some of the plurality of preliminary identifiers into a single preliminary identifier before generating the hypothesis tree. Accordingly, the generation step 512 may include: generating the hypothesis tree only from the preliminary identifiers remaining after the intermediate processing step 510. In some embodiments, the intermediate processing step 510 may be performed without using the event data 150. In further embodiments, the intermediate processing step 510 may be performed relying on the video recording 122 and the expected number of monitors 110 as the sole inputs. In some embodiments, the computing device 120 may perform the intermediate processing step 510 by applying the machine learning model 160. In further embodiments, the computing device 120 may perform the intermediate processing step 510 without using the machine learning model 160.

[0081] In process 500, the allocation step 508 may include: deactivating the preliminary identifier at any point where the video object 130 assigned the preliminary identifier becomes undetectable within the video recording 122, and then assigning a new preliminary identifier when any video object 130 is detected that was not detectable in the immediately preceding frame of the video recording 122. Thus, in some embodiments, the allocation step 508 may include: assigning a new preliminary identifier when a previously undetected video object 130 is detected, without attempting to determine whether any previously undetected video object 130 has been detected and assigned a subsequently deactivated preliminary identifier in an earlier portion of the video recording 122. The intermediate processing step 510 may include: analyzing the video recording 122 and the preliminary identifiers after the allocation step 508 to find preliminary identifiers that may have been assigned to the same video object 130 at different times, and combining these preliminary identifiers into a single preliminary identifier for subsequent use in the generation step 512 and the matching step 516.

[0082] According to various embodiments, the intermediate processing step 510 may include any one of a variety of analysis techniques and combinations thereof. The intermediate processing step 510 of the illustrated embodiment may include a filtering step 520 that includes applying a filter to the video recording 122, the video analysis data 140, or both the video recording 122 and the video analysis data 140 to distinguish signals and noise. In some embodiments, the filtering step may be performed by the computing device 120. The filter may be any type of filter capable of reducing noise in the video information, amplifying the signal, or both. In some embodiments, the filter may be a Kalman filter. For example, a Kalman filter may be applied to a group of multiple frames or portions of the video recording 122 to improve the accuracy of the speed determination of the video object 130. The improved accuracy determination and any other information obtained through the filtering step 520 may be used in the overlapping step 524 and the feature step 528 to merge the preliminary identifiers. In some embodiments, the Kalman filter may be used to predict the position of an object by knowing the object's current speed and position. For example, the video recording 122 may include data indicating that the speed of a first object is zero and another object passes the first object (overlaps) at a higher speed (e.g., 12 km / h); then this video data may be used to predict the position where the moving object will continue in the next frame. In this embodiment, the trajectory identifier of the first object may remain unchanged, and the trajectory identifier of the moving object may also remain unchanged.

[0083] The result of applying the filter in the filtering step 520 may be used to extend the preliminary identifiers. After extending the preliminary identifiers based on the result of applying the filter, some preliminary identifiers may overlap by being assigned to the same video object 130 within the same portion of the video recording 122. The overlapping step 524 may include: discovering the overlapping preliminary identifiers and merging them into a single preliminary identifier.

[0084] Feature step 528 can include viewing video recording 122 to identify features exhibited by video object 130 and then combining any number of preliminary identifiers assigned to athletes exhibiting mutually consistent identified features. In some embodiments, the identified features can include, for example, features such as a jersey number and physical features (e.g., facial characteristics). In some such embodiments, even if video recording 122 does not enable computing device 120 to identify the jersey number or consistently identify physical features throughout video recording 122, computing device 120 may be able to identify specific instances where the jersey number or the same physical feature becomes clear enough for the computing device to accurately identify. Accordingly, in some embodiments, computing device 120 can be configured to read the jersey number at any point in video recording 122 where the jersey number becomes clearly detectable, and feature step 528 can include using computing device 120 to combine any number of preliminary identifiers assigned to video objects 130 having the same jersey number found in different portions of video recording 122. Similarly, in some embodiments, computing device 120 can be configured to identify facial characteristics of video object 130 at any point in video recording 122 where the facial characteristics become clearly detectable, and feature step 528 can include using computing device 120 to combine any number of preliminary identifiers assigned to video objects 130 having the same facial characteristics found in different portions of video recording 122.

[0085] In some embodiments, the identifying features available in feature step 528 may further include features that may not be unique to a given video object 130 or consistent throughout the video recording 122, but that can be evaluated in combination with other circumstances to assess the likelihood of assigning two different preliminary identifiers to the same video object 130 in different portions of the video recording 122. Thus, feature step 528 may include calculating a similarity factor for the identifying features and the associated circumstances by applying a predetermined weight to the identifying features and the associated circumstances, and then merging two preliminary identifiers into a single preliminary identifier if the similarity factor exceeds a predetermined threshold. In some embodiments, the identifying features may include jersey color, position, speed, or any combination of the foregoing. In some embodiments, the associated circumstances may include the number of video objects 130 wearing the same jersey color expected to be on the field at the same time, the ratio of the active preliminary identifiers to the expected maximum number of preliminary identifiers in a portion of the video recording 122, and the deactivation time of one or more preliminary identifiers. Thus, for example, within feature step 528, computing device 120 may determine that a later preliminary identifier that appears soon after an earlier preliminary identifier assigned to a video object 130 in a nearby location and wearing the same jersey color has been deactivated is assigned to the same video object 130 as the earlier preliminary identifier—if no other preliminary identifiers assigned to video objects 130 wearing the same jersey color have been deactivated at substantially the same location and time. In a further example, when two preliminary identifiers assigned to two video objects 130 wearing different jersey colors are deactivated near each other and then two new preliminary identifiers are assigned to the same two video objects 130 soon thereafter, computing device 120 may be able to rely on the jersey color to determine which new preliminary identifier is assigned to the same video object 130 as either deactivated preliminary identifier. In a further example, in feature step 528, if a preliminary identifier is deactivated while assigned to a video object 130 traveling at a certain speed and a later preliminary identifier appears near the location where the video object 130 would arrive soon after the deactivation (if the video object 130 continues to travel at that speed for a gap time between the deactivation of the earlier preliminary identifier and the appearance of the later preliminary identifier), then computing device 120 may determine that the earlier preliminary identifier and the later preliminary identifier are assigned to the same video object 130 (if no other preliminary identifiers appear or are deactivated at substantially the same location and time). Similarly, if a preliminary identifier is deactivated while assigned to a stationary or slowly moving video object 130, then computing device 120 may determine that a newly assigned preliminary identifier to the video object 130 soon after the deactivation, but that is relatively far from the last location of the deactivated preliminary identifier, is less likely to be assigned to the same video object 130 as the deactivated preliminary identifier.

[0086] In the illustrated embodiment, the feature step 528 occurs after the filtering step 520 and the overlapping step 524. However, in other embodiments, the feature step 528 may occur before, between, or during the filtering step 520 and the overlapping step 524.

[0087] By applying the above filtering step 520, overlapping step 524, and feature step 528, according to some embodiments, the intermediate processing step 510 may reduce the effective number of preliminary identifiers before the process 500 reaches the generation step 512. Thus, in some embodiments, the intermediate step 510 may make the generation step 512 and the matching step 516 less computationally intensive and more efficient. According to other embodiments, the intermediate processing step 510 may include more or fewer steps similar in purpose to the filtering step 520, overlapping step 524, and feature step 528 of the illustrated embodiment.

[0088] The embodiments described herein may also be directed to a computer program product that includes software stored on any computer-usable medium. When executed in one or more data processing devices, such software causes the data processing devices to operate as described herein. The embodiments described herein may employ any computer-usable or readable medium. Examples of computer-usable media include, but are not limited to, primary storage devices (e.g., any type of random access memory), secondary storage devices (e.g., hard disk drives, floppy disks, CD ROMs, ZIP disks, magnetic tapes, magnetic storage devices, and optical storage devices, MEMS, nanotechnology storage devices, etc.).

[0089] It should be understood that the detailed description, rather than the summary and abstract sections, is intended to be used to interpret the claims. The summary and abstract sections may set forth one or more, but not all, of the exemplary embodiments contemplated by the inventors, and thus are not intended to limit the invention and the appended claims in any way.

[0090] The present invention has been described above by means of functional building blocks that illustrate the implementation of specified functions and their relationships. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined as long as the specified functions and their relationships are appropriately performed.

[0091] The description of the above specific embodiments will so fully disclose the general nature of the present invention that others can, by applying the knowledge of those skilled in the art, easily modify and / or adapt these specific embodiments for various applications without undue experimentation, without departing from the general concept of the present invention. Therefore, based on the teachings and guidance presented herein, such adaptations and modifications are intended to be within the meaning and scope of the equivalents of the disclosed embodiments. It should be understood that the language or terminology herein is for the purpose of description and not of limitation, such that the terminology or language of this specification is to be interpreted by those skilled in the art in light of the teachings and guidance.

[0092] The breadth and scope of the concept of the present disclosure should not be limited by any of the above exemplary embodiments, but should be defined only in accordance with the appended claims and their equivalents.

Claims

1. A method for tracking an object in a video recording, the method comprising: receiving, at a computing device, video footage and event data of a space from a monitor, wherein the monitor includes a motion sensor configured to measure motion of monitored video objects within the space, and the event data includes motion events transmitted by the monitor; identifying, using a machine learning model on a computing device, from the video footage a video event performed by the monitored video subject; as well as A corresponding permanent identifier is assigned to the monitored video object within the video recording based at least in part on commonalities between video events performed by the monitored video object and motion events transmitted by the detector.

2. The method of claim 1 , wherein the monitored video object is one of a plurality of video objects, the monitor is one of a plurality of monitors, each monitor comprising a motion sensor configured to measure motion of the corresponding monitored video object, the method comprising assigning a corresponding permanent identifier to each of the plurality of monitored video objects, and assigning the corresponding permanent identifier comprises: assigning corresponding preliminary identifiers to the monitored video objects within the video recording at a plurality of different times within the video recording; finding a preliminary identifier assigned to the same monitored video subject within a different portion of the video recording based at least in part on commonalities between video events performed by the same monitored video subject and motion events transmitted by one of the monitors; merging the discovered preliminary identifiers assigned to the same monitored video object into a single preliminary identifier; as well as After the merging step, the preliminary identifier is converted into the permanent identifier.

3. The method of claim 2, wherein assigning the corresponding preliminary identifier comprises: A new preliminary identifier is assigned to a particular one of the monitored video objects at a discontinuity point within the video recording, when the computing device is unable to determine, based on the video recording, the identity of the particular one of the monitored video objects from any of the monitored video objects that were assigned the preliminary identifier within an earlier portion of the video recording.

4. The method of claim 3, wherein the assigning of the corresponding preliminary identifier comprises: identifying key moments within the video recording, wherein at each key moment, each monitored video object is individually identifiable from the video recording by the machine learning model; At at least one key moment, similarities are calculated between motion events in the event data transmitted by each wearable monitor and video events performed by each monitored video subject assigned a preliminary identifier.

5. The method according to claim 1, comprising: Prior to transmitting the event data to the computing device, motion data captured by the motion sensor of the wearable monitor is processed to identify the motion event.

6. The method of claim 5, wherein the machine learning model is a first machine learning model, and the processing of the motion data comprises: A second machine learning model stored in the wearable monitor is applied to the motion data.

7. The method according to claim 2, comprising: A corresponding one of the monitors is identified for each permanent identifier that includes the motion sensor configured to measure motion of the monitored video object to which the permanent identifier is assigned.

8. The method according to claim 7, comprising: A timestamp is applied to one of the motion events transmitted by a corresponding one of the monitors, the timestamp matching a time at which the corresponding video event was performed by the monitored video object assigned the permanent identifier.

9. The method according to claim 1, comprising: A timestamp is applied to at least one of the motion events to match a time at which one of the video events occurred.

10. The method of claim 9, wherein the event data comprises: A preliminary timestamp is applied by the monitor to the motion events, and applying the timestamp to at least one of the motion events includes replacing one of the preliminary timestamps with a final timestamp.

11. The method of claim 2, wherein the video subjects are athletes participating in a sporting event.

12. The method of claim 11, wherein each athlete wears one of the monitors.

13. The method according to claim 11, wherein the sports event comprises any one or any combination of kicking a ball, running, walking and standing.

14. The method according to claim 13, wherein the sports event includes amplitude information in any one or any combination of kicking force, travel distance and speed.

15. The method of claim 1, wherein the monitor comprises a controller configured to identify the motion event from the motion sensor of the monitor.

16. A system comprising: a wearable monitor including a motion sensor; as well as A computing device configured to receive event data transmitted by the wearable monitor, the event data comprising motion events, wherein the computing device includes a non-transitory computer-readable medium storing a machine learning model, the machine learning model configured to identify video events from video recordings, and the computing device is configured to associate the video events with the motion events.

17. The system of claim 16, wherein the wearable monitor is configured to be integrated into an article of wear.

18. The system of claim 17, wherein the article of wear is a shoe insole.

19. The system of claim 16, wherein the wearable monitor comprises a motion sensor and a controller configured to identify the motion event from measurements obtained by the motion sensor.

20. The system of claim 16, wherein the machine learning model is configured to track video objects within a video recording based on the event data.

Images