SYSTEM AND METHOD FOR ANALYZING VIDEO IN REAL TIME - Patent application

Abstract

A method and a sports analysis system (SAS) for analyzing a live video broadcast stream (LVBS) of a sporting event are provided. The SAS divides the LVBS into a real-time messaging protocol (RTMP) stream and a hypertext transfer protocol live stream (HLS), and analyzes the RTMP stream using a phase difference between the RTMP stream and the HLS. The SAS uses a first set of cues to detect people present in frames of the RTMP stream and tracks the detected people by analyzing previous frames. The SAS recognizes the tracked people using a second set of cues, assigns individual weights to each of the second set of cues, and compares the assigned weights of each recognized person with prior data of all players to identify the players in the frames.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of Indian Patent Application No. 202041006309, filed on February 14, 2020, the contents of which are incorporated herein by reference in their entirety.

[0002] SYSTEMS AND METHODS FOR ANALYZING VIDEO STREAMS FIELD OF THEINVENTION The present invention relates to systems and methods for analyzing video streams, and more particularly to sports analysis systems and methods for analyzing live video broadcast streams of sporting events. [Background technology]

[0003] (background) Broadcasting sporting events is a very common practice that allows spectators to experience the events they missed in person. It also increases the potential audience reach to a wider group of spectators, thereby providing significant benefits for broadcasters, teams, and sports franchises. Real-time contextual overlays for live broadcast feeds are traditionally done by inserting and presenting contextual information relevant to potential viewers to increase audience reach. However, for sporting events, complex data tracking and analysis are required to present contextual interactive content in real time to actively engage spectators with the live broadcast feed. Thus, there is a need for a system and method for analyzing live video broadcast streams of sporting events to deliver real-time contextual interactive content for spectators.

[0004] Real-time motion capture and object tracking are essential when it comes to analyzing sporting events. A key aspect in analyzing live broadcast feeds of sporting events is the recognition of players in real-time by actively tracing or tracking players over time and gaining meaningful insights before presenting it to spectators. Motion capture methods using optical systems, various sensors to measure the three-dimensional (3D) location of players' organs, and the use of various pattern recognition techniques exist in the prior art. However, they require complex systems with arrays of sensors, as well as dedicated hardware to accurately identify and pinpoint the measurements. Furthermore, this makes it more difficult to perform analysis and deliver contextual interactive content in real-time and with accuracy on the live broadcast feed. Thus, there is a need for a system and method for analyzing live video broadcast streams of sporting events to recognize players with multifaceted techniques in order to deliver contextual interactive content to spectators.

[0005] In sporting events, the scene is mostly dynamic as players manipulate themselves in parallel with other objects based on the nature of the game and the situation. Existing tracking techniques are inclined to provide qualitative data about players and objects, which requires sophisticated techniques and manual efforts to bring contextual interactive content onto the live broadcast feed. However, the accuracy of the data is unreliable and ineffective. Systems using multiple cameras, sensors, and identifiers are used to fetch useful information. However, these techniques require cumbersome solutions to perform analysis and extract contextual interactive content from the obtained data, which further increases the cost and time spent. Optical 3D measurement techniques fail to provide accurate data due to poor viewing angles and distortions as a result of dynamic images. The aforementioned drawbacks lead to aggravation of the complexity in performing analysis and retrieving useful insights and metrics. Hence, there is a need for a system and method for analyzing live video broadcast streams of sporting events to perform accurate sports analysis in order to deliver contextual interactive content with many insights and metrics for spectators.

[0006] Live events usually generate significant revenues for broadcasters and advertisers. When live events such as sports are augmented with contextual interactive content, this will further expand their business and subsequently generate higher revenues. Sports teams conduct a lot of analysis and research on their team's players and games, and use accurate data extraction and advanced analytical modules to formulate strategies and decisions. Similarly, broadcasters pursue such techniques to increase the monetization potential of live broadcast feeds. For sporting events, spectators expect more engagement with the game, and therefore there is a need for systems and methods for analyzing live video broadcast streams of sporting events to deliver more contextual interactive content in real time. Traditional techniques require complex hardware for implementation, and additional cameras are needed to capture data, either on the playing field / court or elsewhere, which leads to expensive system requirements.

[0007] A live video broadcast stream consists of a series of frames. Frames per second (fps) indicates the number of frames present within a one second time frame. The duration of a single frame is referred to as frame lifetime duration or frame duration (1,000 ms / fps). For example, in a 25 fps video, each second consists of 25 frames, and each frame lasts on the screen for a total of 1,000 / 25=40 ms. To provide contextual interactive content to the audience in real time, all processing activities should occur within this time frame to keep up with the live broadcast feed. Otherwise, the time it takes for one frame to be processed will cause subsequent frames to be missed. Thus, there is a long felt but unmet need for a system and method for analyzing a live video broadcast stream within frame lifetime duration or frame duration.

[0008] Thus, there is a long felt but unmet need for a system and method for analyzing live video broadcast streams of sporting events to deliver contextual, interactive content for spectators in real time to increase their audience reach and increase the monetization potential of existing content for broadcasters and advertisers. Summary of the Invention [Means for solving the problem]

[0009] SUMMARY OF THEINVENTION This Summary is provided to introduce a selection of concepts in a simplified form that are further disclosed in the Detailed Description of the Invention. This Summary is not intended to determine the scope of the claimed subject matter.

[0010] The sports analysis system and method disclosed herein address the above-listed needs for analyzing live video broadcast streams of sporting events to deliver contextual interactive content for spectators in real time to improve their audience and increase the monetization potential of existing content for broadcasters and advertisers. Furthermore, the sports analysis system and method disclosed herein utilizes a phase difference between a real-time messaging protocol stream and a hypertext transfer protocol live stream to complete processing activities. The sports analysis system incorporates a computer server to analyze the live video broadcast streams of sporting events. The sports analysis system includes a video demultiplexer, a non-transitory computer-readable storage medium, and at least one processor. The video demultiplexer receives the live video broadcast streams of sporting events from a video capture device and splits the received live broadcast streams into a real-time messaging protocol stream and a hypertext transfer protocol live stream. The non-transitory computer-readable storage medium stores computer program instructions defined by modules of the sports analysis system. The processor is communicatively coupled to the non-transitory computer-readable storage medium, and the processor is configured to execute computer program instructions defined by modules of the sports analysis system. The analysis module analyzes the real-time messaging protocol stream within a predetermined time interval. The predetermined time interval corresponds to a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream. The predetermined time interval is less than a frame duration of the frame. The analysis module includes a detection module, a tracking module, a recognition module, a comparison module, and a transmission module.The detection module detects one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from the first set of cues, comprising face detection, head detection, person detection, and jersey number detection, based on availability of the first set of cues.

[0011] The tracking module tracks the detected person in the frame by analyzing one or more prior frames of the real-time messaging protocol stream. The recognition module recognizes the tracked person in the frame using one or more cues selected from a second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and a commentary of the sporting event based on availability of the second set of cues. The jersey recognition comprises one or more of a jersey number classification and a team recognition. An individual weighting is assigned to each of the facial recognition, person recognition, and jersey recognition when the facial recognition, person recognition, and jersey recognition are performed in the frame. The individual weighting is determined based on clarity of each person recognized in the frame. If any of the facial recognition, person recognition, and jersey recognition is unavailable, the recognition module uses one or more of the live score feed and the commentary for identification of the one or more players in the frame. The comparison module identifies the one or more players in the frame by comparing the assigned weighting for each of the recognized person to the prior data of all players. The transmission module transmits the HyperText Transfer Protocol live stream and the analysis data including the contextual interactive content of the identified player to a client application provided by the sports analysis system on the user device within a predetermined time interval. The live video broadcast stream is converted into an interactive video including the contextual interactive content of the identified player. The client application includes an interaction module to provide user interaction with the interactive video.

[0012] The identified player's contextual interactive content is displayed on the user device without overlapping with existing graphics of the HyperText Transfer Protocol live stream. The identified player's contextual interactive content comprises a dynamic display of one or more of scorecards, player information, and match statistics, and the identified player's contextual interactive content can be customized and shared on social media. The user can interact with one or more of the identified players in the interactive video. The sports analysis system increases the implementation of on-demand enhanced content by recognizing a player of interest and displaying to the user device key metrics of the recognized player, such as the player's biography, season, career totals, social media posts, and related videos. The sports analysis system also provides rich content for users to view head-to-head comparisons of the statistics of the player of interest. The sports analysis system further helps to engage users in live video broadcast streams by embedding the game experience and increasing the monetization potential of existing content for various broadcasters and advertisers. The present invention provides, for example, the following: (Item 1) 1. A sports analysis system incorporating a computer server for analyzing a live video broadcast stream of a sporting event, the sports analysis system comprising: a video demultiplexer for receiving the live video broadcast stream of the sporting event from a video capture device and splitting the received live broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream; a non-transitory computer readable storage medium for storing computer program instructions defined by modules of the sports analysis system; At least one processor communicatively coupled to the non-transitory computer readable storage medium, the at least one processor configured to execute the computer program instructions defined by the modules of the sports analysis system, the modules comprising: an analysis module for analyzing the real-time messaging protocol stream within a predetermined time interval, the predetermined time interval corresponding to a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream, the analysis module comprising: a detection module for detecting one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from a first set of cues based on availability of the first set of cues, the cues comprising face detection, head detection, person detection, and jersey number detection; a tracking module for tracking the detected person within one or more previous frames of the real-time messaging protocol stream by analyzing the frames; a recognition module for recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and a commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; a comparison module for identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; an analysis module comprising: a transmission module for transmitting the HyperText Transfer Protocol live stream and analysis data including the contextual interactive content of the identified player to a client application provided by the sports analysis system on a user device within the predetermined time interval, the live video broadcast stream being converted into an interactive video including the contextual interactive content of the identified player; and At least one processor comprising: A sports analysis system comprising: (Item 2) 2. The sports analysis system of claim 1, wherein if any one of the facial recognition, the person recognition, and the jersey recognition is unavailable, the recognition module uses one or more of the live score feed and the commentary for the identification of the one or more players in the frame. (Item 3) 2. The sports analysis system of claim 1, wherein the predetermined time interval is less than a frame duration of the frame. (Item 4) 2. The sports analysis system of claim 1, wherein the jersey recognition comprises one or more of jersey number classification and team recognition. (Item 5) 2. The sports analysis system of claim 1, wherein the individual weightings are determined based on the clarity of each person recognized in the frame. (Item 6) 2. The sports analysis system of claim 1, wherein the client application comprises an interaction module for providing user interaction with the interactive video. (Item 7) 2. The sports analysis system of claim 1, wherein the contextual interactive content of the identified player is displayed on the user device without overlapping with existing graphics of the HyperText Transfer Protocol live stream. (Item 8) 8. The sports analysis system of claim 7, wherein the contextual interactive content of the identified player comprises a dynamic display of one or more of a scorecard, player information, and match statistics, and the contextual interactive content of the identified player is configured to be customized and shared on social media. (Item 9) 1. A method for analyzing a live video broadcast stream of a sporting event, the method employing a sports analysis system comprising at least one processor configured to execute computer program instructions for implementing the method, the method comprising: receiving, by a video demultiplexer of the sports analysis system, the live video broadcast stream of the sporting event from a video capture device; splitting the live broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream by the video demultiplexer of the sports analysis system; analyzing the real-time messaging protocol stream within a predetermined time interval, the predetermined time interval comprising a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream, the analysis comprising: detecting one or more people present within frames of the real-time messaging protocol stream using one or more cues selected from a first set of cues based on availability of the first set of cues, the cues comprising face detection, head detection, person detection, and jersey number detection; Tracking the detected person within one or more previous frames of the real-time messaging protocol stream by analyzing the frames; Recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and a commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons to prior data of all players; and transmitting, by the sports analysis system, the hypertext transfer protocol live stream and analysis data including contextual interactive content of the identified player to a client application hosted on a user device within the predetermined time interval; Including, Whereby, the live video broadcast stream is converted into an interactive video including the contextual interactivity content of the identified player and transmitted to the user device. (Item 10) 10. The method of claim 9, wherein if any one of the facial recognition, the person recognition, and the jersey recognition is unavailable, one or more of the live score feed and the commentary are used to identify the one or more players in the frame. (Item 11) 10. The method of claim 9, wherein the predetermined time interval is less than a frame duration of the frame. (Item 12) 10. The method of claim 9, wherein the jersey recognition comprises one or more of a jersey number classification and a team recognition. (Item 13) 10. The method of claim 9, wherein the individual weights are determined based on the clarity of each person recognized in the frame. (Item 14) 10. The method of claim 9, wherein a user can interact with one or more of the identified players within the interactive video. (Item 15) 10. The method of claim 9, wherein the contextual interactive content of the identified player is displayed on the user device without overlapping with graphics of the HyperText Transfer Protocol live stream. (Item 16) Item 16. The method of item 15, wherein the contextual interactive content of the identified player comprises a dynamic display of one or more of a scorecard, player information, and match statistics, and the contextual interactive content of the identified player is configured to be customized and shared on social media. (Item 17) 1. A non-transitory computer readable storage medium having embodied thereon computer program code comprising instructions executable by at least one processor for analyzing a live video broadcast stream of a sporting event, the computer program code comprising: first computer program code for receiving the live video broadcast stream of the sporting event from one or more video capture devices; second computer program code for splitting the live broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream; third computer program code for analyzing the real-time messaging protocol stream within a predetermined time interval, the predetermined time interval comprising a phase difference between the real-time messaging protocol stream and the HyperText Transfer Protocol live stream; fourth computer program code for detecting one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of the first set of cues; fifth computer program code for tracking the detected person within one or more previous frames of the real-time messaging protocol stream by analyzing the frames; sixth computer program code for recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; seventh computer program code for identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; an eighth computer program code for transmitting, by the sports analysis system, the HyperText Transfer Protocol live stream and analysis data including contextual interactive content of the identified player to a client application hosted on a user device within the predetermined time interval; 1. A non-transitory computer-readable storage medium comprising: (Item 18) 20. The non-transitory computer-readable storage medium of claim 17, wherein if any one of the facial recognition, the person recognition, and the jersey recognition is unavailable, one or more of the live score feed and the commentary are used to identify the one or more players in the frame. (Item 19) Item 18. The non-transitory computer-readable storage medium of item 17, wherein the computer program code further comprises a ninth computer program code for providing user interaction with the interactive video. (Item 20) Item 18. The non-transitory computer-readable storage medium of item 17, wherein the predetermined time interval is less than a frame duration of the frame. (Item 21) Item 18. The non-transitory computer-readable storage medium of item 17, wherein the jersey recognition comprises one or more of a jersey number classification and a team recognition. (Item 22) Item 18. The non-transitory computer-readable storage medium of item 17, wherein the individual weights are determined based on the clarity of each person recognized in the frame. (Item 23) Item 18. The non-transitory computer-readable storage medium of item 17, wherein the contextual interactive content of the identified player is displayed on the user device without overlapping with graphics of the HyperText Transfer Protocol live stream. (Item 24) 24. The non-transitory computer-readable storage medium of claim 23, wherein the contextual interactive content of the identified player comprises a dynamic display of one or more of a scorecard, player information, and match statistics, and the contextual interactive content of the identified player is configured to be customized and shared on social media. (Item 25) 1. A sports analysis system incorporating a computer server for analyzing pre-recorded video streams of a sporting event, the sports analysis system comprising: a non-transitory computer readable storage medium for storing computer program instructions defined by modules of the sports analysis system; At least one processor communicatively coupled to the non-transitory computer readable storage medium, the at least one processor configured to execute the computer program instructions defined by the modules of the sports analysis system, the modules comprising: an analysis module for analyzing the pre-recorded video stream, the analysis module comprising: a detection module for detecting one or more people present within a frame of the pre-recorded video stream using one or more cues selected from a first set of cues based on availability of the first set of cues, the cues comprising face detection, head detection, person detection, and jersey number detection; a tracking module for tracking the detected person in the frame by analyzing one or more previous frames; a recognition module for recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a score feed of the sporting event, and a commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; a comparison module for identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; an analysis module comprising: a transmission module for transmitting the pre-recorded video stream and analytics data including the contextual interactive content of the identified player to a client application provided on a user device by the sports analysis system, the pre-recorded video stream being converted into an interactive video including the contextual interactive content of the identified player; and At least one processor comprising: A sports analysis system comprising: (Item 26) 1. A method for analyzing a pre-recorded video stream of a sporting event, the method employing a sports analysis system comprising at least one processor configured to execute computer program instructions for implementing the method, the method comprising: analyzing the pre-recorded video stream, the analysis comprising: detecting one or more people present within a frame of the pre-recorded video stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of a first set of cues; tracking the detected person in one or more previous frames by analyzing the frames; Recognizing the tracked person in the frame using one or more cues selected from a second set of cues comprising facial recognition, person recognition, jersey recognition, a score feed of the sporting event, and a commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons to prior data of all players; transmitting, by the sports analysis system, the pre-recorded video stream and analytics data including contextual interactive content for the identified player to a client application hosted on a user device; Including, Whereby, the pre-recorded video broadcast stream is converted into an interactive video containing the contextual interactivity content for the identified player and transmitted to the user device. [Brief description of the drawings]

[0013] The foregoing summary, as well as the following detailed description of the invention, will be better understood when read in conjunction with the accompanying drawings. For the purpose of illustrating the invention, exemplary constructions of the invention are shown in the drawings. The invention is not limited, however, to the specific methods and components disclosed therein.

[0014] [Figure 1] FIG. 1 illustrates an exemplary sports analysis system incorporating a computer server for analyzing a live video broadcast stream of a sporting event.

[0015] [Diagram 2] FIG. 2 illustrates an exemplary computer server architecture employed by a sports analysis system for analyzing live video broadcast streams of sporting events.

[0016] [Diagram 3] FIG. 3 illustrates an exemplary method for analyzing a live video broadcast stream of a sporting event.

[0017] [Figure 4A] FIG. 4A exemplarily illustrates people detection within frames of a HyperText Transfer Protocol live stream.

[0018] [Figure 4B]FIG. 4B exemplarily illustrates head detection within a frame of a HyperText Transfer Protocol live stream.

[0019] [Figure 4C] FIG. 4C exemplarily illustrates face detection within frames of a HyperText Transfer Protocol live stream.

[0020] [Figure 4D] FIG. 4D exemplarily illustrates jersey number detection within a frame of a HyperText Transfer Protocol live stream.

[0021] [Figure 4E] FIG. 4E illustrates an exemplary diagram of person detection, head detection, face detection, jersey number detection, and associations thereof within a frame of a HyperText Transfer Protocol live stream.

[0022] [Diagram 5] FIG. 5 exemplarily illustrates tracking of a detected person within a frame by analyzing one or more previous frames of a real-time messaging protocol stream.

[0023] [Figure 6A] FIG. 6A exemplarily illustrates face recognition using extracted embeddings.

[0024] [Figure 6B] FIG. 6B illustrates an exemplary jersey number classification.

[0025] [Figure 6C] FIG. 6C exemplarily illustrates team recognition by extracting embeddings.

[0026] [Figure 6D] FIG. 6D exemplarily illustrates a live score feed and commentary to identify one or more players in the frame.

[0027] [Figure 7A] 7A-7D exemplarily illustrate screenshots of on-demand enhanced contextual interactivity content overlaid on a HyperText Transfer Protocol live stream of a sporting event. [Figure 7B] 7A-7D exemplarily illustrate screenshots of on-demand enhanced contextual interactivity content overlaid on a HyperText Transfer Protocol live stream of a sporting event. [Figure 7C] 7A-7D exemplarily illustrate screenshots of on-demand enhanced contextual interactivity content overlaid on a HyperText Transfer Protocol live stream of a sporting event. [Figure 7D] 7A-7D exemplarily illustrate screenshots of on-demand enhanced contextual interactivity content overlaid on a HyperText Transfer Protocol live stream of a sporting event.

[0028] [Figure 8] FIG. 8 exemplarily illustrates a screenshot of rich contextual interactive content overlaid on a HyperText Transfer Protocol live stream.

[0029] [Figure 9A] 9A-9B exemplarily illustrate screenshots of contextual interactive content overlaid on a HyperText Transfer Protocol live stream for an immersive and gaming experience for a user. [Figure 9B] 9A-9B exemplarily illustrate screenshots of contextual interactive content overlaid on a HyperText Transfer Protocol live stream for an immersive and gaming experience for a user.

[0030] [Figure 10A]10A-10D exemplarily illustrate screenshots of contextual interactive content overlaid onto a HyperText Transfer Protocol live stream to increase the monetization potential of existing content. [Figure 10B] 10A-10D exemplarily illustrate screenshots of contextual interactive content overlaid onto a HyperText Transfer Protocol live stream to increase the monetization potential of existing content. [Figure 10C] 10A-10D exemplarily illustrate screenshots of contextual interactive content overlaid onto a HyperText Transfer Protocol live stream to increase the monetization potential of existing content. [Figure 10D] 10A-10D exemplarily illustrate screenshots of contextual interactive content overlaid onto a HyperText Transfer Protocol live stream to increase the monetization potential of existing content.

[0031] [Figure 11] FIG. 11 exemplarily illustrates a screenshot of a user engagement to improve and increase the monetization potential of existing content at the consumer end. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Detailed Description of the Invention FIG. 1 exemplarily illustrates a sports analysis system 100 incorporating a computer server for analyzing a live video broadcast stream of a sporting event. As used herein, a "live video broadcast stream" refers to a live feed of a video broadcast of a sporting event. The sports analysis system 100 includes a video demultiplexer 101, a non-transitory computer-readable storage medium 202, and at least one processor 201, as exemplarily illustrated in FIG. 2. The video demultiplexer 101 receives a live video broadcast stream of a sporting event from a video capture device (not shown) and splits the received live broadcast stream into a real-time messaging protocol stream 101b and a hypertext transfer protocol live stream 101a. The real-time messaging protocol stream 101b is sent to an analysis module 102 for analyzing the sporting event. The hypertext transfer protocol live stream 101a is encoded for broadcast to any consumer device platform, such as the web, any iOS device, any Android device, or other integrated platform. Further, offset-based frame synchronization logic is implemented to achieve frame-level synchronization across the split streams. The non-transitory computer-readable storage medium 202 stores computer program instructions defined by the modules of the sports analysis system 100. The processor 201 is communicatively coupled to the non-transitory computer-readable storage medium 202 and configured to execute the computer program instructions defined by the modules of the sports analysis system 100. The analysis module 102 analyzes the real-time messaging protocol stream 101b within a predetermined time interval. The predetermined time interval corresponds to a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream. The predetermined time interval is less than the frame duration of a frame.The analysis module 102 comprises a detection module 103 , a detection / comparison queue 104 , a tracking module 105 , a recognition module 106 , a comparison module 107 , a configuration data module 108 , and a transmission module 109 .

[0033] The detection module 103 detects one or more people present in a frame of the real-time messaging protocol stream 101b using one or more cues selected from the first set of cues, including face detection 103a, head detection 103b, person detection 103b, and jersey number detection 103c, based on the availability of the first set of cues. As used herein, a "jersey" refers to any type of clothing worn by a person in the frame. The sports analysis system 100 can detect people in the frame from various angles and orientations. The detection module 103 uses a first set of pre-trained models 103d to detect various cues in the frame. The face detection 103a includes detecting the face of a person in the frame and creating a bounding box around the detected face such that the face is tightly cropped by the bounding box, as illustrated in FIG. 4C. Person Detection 103b involves detecting human bodies in the frame in various complex situations, such as when a person's body is either partially visible or fully visible, or when a person's body overlaps with another person's body, etc. Person Detection 103b involves creating bounding boxes around all detected human bodies in the frame, as illustrated in FIG.

[0034] Similarly, head detection 103b includes detecting the head of a person in the frame in situations where the head is either partially visible or fully visible, or where the head is in various orientations and poses. For example, the person's head may be seen from behind, from the front, from below or above, or from the left or right. In some cases, the person's head may overlap with the head of another person. The output of head detection 103b includes creating a bounding box around all detected heads in the frame. Each detected head is associated with a body, and a bounding box is created inside the bounding box of the associated person and as close to the head as possible, as illustrated in FIG. 4B. Jersey number detection 103c includes detecting jersey numbers on one or more of the back, front, shoulder, and other locations on the jersey of the person in the frame, as illustrated in FIG. 4D. Additionally, jersey number detection 103c includes detecting each digit on the jersey individually (even if they act as part of a multi-digit number) using a pre-trained model 103d. The hierarchy of the detection sequence is then a face bounding box contained within a head bounding box, which is contained within a person bounding box, as illustrated in Figure 4E. The output of the detection module 103 is fed into a shared detection / comparison queue 104 for further processing.

[0035] The tracking module 105 tracks the detected person in the frame by analyzing one or more previous frames of the real-time messaging protocol stream 101b as illustrated in FIG. 5. The tracking module 105 correlates the detected person across a successive set of frames, essentially capturing the person's movement. Furthermore, the tracking module 105 employs a hybrid tracking algorithm to track the detected person in the frame using a multi-object tracking approach. The detected person is tracked in the frame persistently in real time, and the frames are processed independently to identify multiple objects in a particular frame. The hybrid tracking algorithm implements learning techniques to track the detected person using Kalman filter prediction 105a, feature tracking 105b, and intersection of unions (IOU) 105c, using a second set of pre-trained models 105d.

[0036] The recognition module 106 recognizes the tracked person in the frame using one or more cues selected from the second set of cues comprising face recognition 106a, person recognition 106b, jersey recognition 106c, live score feed of the sporting event 106d, and commentary of the sporting event 106d based on the availability of the second set of cues. Recognition involves processes such as classification, extraction of embeddings, computation of similar features using metrics such as cosine distance, squared Euclidean distance, etc. Each recognition has a confidence value associated with it, and a prediction is considered only if the confidence value of the predicted recognition exceeds a pre-set threshold confidence value. This reduces the chances of false positives and makes the recognition more accurate.

[0037] Facial recognition 106a includes extracting embeddings from detected faces, as illustrated in FIG. 6A, which are compared to another array of embeddings obtained from a pre-collected static set of high-resolution images. Facial recognition 106a is based on similarity between embeddings. Furthermore, parameters such as pose, face size, skin color, and orientation toward the camera do not affect facial recognition 106a. Jersey recognition 106c comprises one or more of jersey number classification and team recognition. Jersey number classification is used to classify each of the detected digits of the jersey into one or more classes, as illustrated in FIG. 6B. Furthermore, multiple digits belonging to the same jersey number are correlated with each other based on either their relative position or font. Jersey number classification maintains a queuing system and minimizes the possibility of edge cases of false positives. Team recognition includes extracting embeddings from person bounding boxes and comparing them to an array of pre-computed embeddings, as illustrated in FIG. 6C. Team recognition uses jersey colors as distinguishing parameters and performs recognition based on similarities between embeddings.

[0038] An individual weighting is assigned to each of the facial recognition 106a, person recognition 106b, and jersey recognition 106c when they are performed in the frame. The individual weighting is determined based on the clarity of each person recognized in the frame. Recognitions predicted by the recognition module 106 have an associated confidence metric based on which a final inference is made. At that point, the recognition is then pushed to the detection / comparison queue 104, and predictions with low confidence metrics are reserved for reflection in future decisions. As used herein, "individual weighting" refers to a score generated for detection of one or more of the second set of cues. For example, the sports analysis system 100 actively recognizes tracked people in frames of a live cricket video and assigns individual weightings to each of those people depending on one or more of the second set of cues determined for the people in the frame.

[0039] If any of the facial recognition 106a, person recognition 106b, and jersey recognition 106c are unavailable, the recognition module 106 uses one or more of the live score feed 106d and commentary 106e to identify one or more players in the frame, as illustrated in FIG. 6D. The recognition module 106 uses a third set of predefined models 106f to perform the recognition. For example, when player A is present in the frame and his jersey number is slightly occluded but his face is clearly visible, the weighting given to "facial recognition" is higher than that given to "jersey number recognition". On the other hand, when player B is present in the frame and his jersey number is clearly visible but his face is slightly occluded, the weighting given to "jersey number recognition" is higher than that given to "facial recognition". Furthermore, the sports analysis system 100 uses player tracking to backfill the detection. For example, if a person's detection cues, such as face detection 103a and jersey number detection 103b, are not visible at a time "t" within a particular trajectory to provide sufficient reliability to uniquely detect a player, but using backfill and tracking are available at some other time "t+n" in the same trajectory, the system can identify the person through trajectory survival, as illustrated in FIG. 5.

[0040] The comparison module 107 identifies one or more players in the frame by comparing the assigned weight for each recognized person with the prior data of all players. The detection, tracking, and recognition are sent to a decision / comparison queue 104 along with various confidence levels extracted from the detection module 103, tracking module 105, and recognition module 106. The sports analysis system 100 uses the outputs extracted from the independent modules to make a final inference, significantly reducing the expected false positives and improving the output of the system. From the perspective of player recognition, each sport has its own characteristics in terms of the markers available for the various modules used to perform the recognition. For example, the face of an American football player is not available as a marker for recognition because the player wears a helmet, while a golfer does not have a number on his playing uniform. Therefore, the system considers the weights 107a of the various available cues, which are essential in the context of sports player recognition. In addition, knowledge of previous inferences 107b is also used to arrive at the final prediction and is processed in the output monitoring unit 107c via a buffer 107d.

[0041] The sports analysis system 100 processes the output from the various modules and generates the contextual interactivity content within a predetermined time interval. The sports analysis system 100 utilizes a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream to deliver the contextual interactivity content. The predetermined time interval is less than the frame duration of the frame. The sports analysis system 100 uses an inherent time lag to enable additional time buffers required by the various modules. Furthermore, each module in the sports analysis system 100 can be configured with configuration data 108 via the network 110. The configuration data 108 can be at least one of machine learning models, game specific data, video / stream specific data, and the like. Furthermore, the configuration data 108 can be updated in real-time during the processing of the real-time messaging protocol stream 101b. In an embodiment, a dashboard is configured to control and monitor the processes performed by the sports analysis system 100.

[0042] The transmission module 109 of the sports analysis system 100 is configured to transmit the hypertext transfer protocol live stream and the analysis data, including the contextual interactive content of the identified player, to the client application 111a provided by the sports analysis system 100 on the user device 111 within a predetermined time interval. In effect, the sports analysis system 100 converts the live video broadcast stream into an interactive video, including the contextual interactive content of the identified player, and makes the interactive video available on the user device 111. The client application 111a includes an interaction module 111b to provide user interaction with the interactive video. As used herein, a "client application" refers to an application installed on a user device that is used for user interaction. Also, as used herein, a "user device" refers to any device that is used for user interaction, including but not limited to mobile communication devices, mobile phones, personal digital assistants (PDAs), digital cameras, notebook computers, tablet computers, and netbooks.

[0043] The identified player's contextual interactive content is displayed on the user device 111 without overlapping with existing graphics of the HyperText Transfer Protocol live stream. The contextual interactive content is converted in real time into the live video broadcast stream as on-demand enhanced content to enhance user interaction and audience growth. The identified player's contextual interactive content comprises a dynamic display of one or more of scorecards, player information, and match statistics, and the identified player's contextual interactive content can be customized and shared on social media. The identified player's contextual interactive content also includes information about the player, player statistics, and player bio. The identified player's contextual interactive content can be customized according to requirements and shared on any social media platform at will. In some embodiments, the identified player's contextual interactive content can be customized to a particular season and the corresponding player's statistics in that season. In another embodiment, one or more social media posts of the player and team are provided as on-demand enhanced content. In another embodiment, customized player match statistics are provided as rich content specifically for the user to enhance their viewing interaction. The interaction module 111b provides rich content for the user in real time, provides a deeper engagement and gaming experience platform for interested users, and increases the monetization potential of existing content for various broadcasters and advertisers. The user can interact with one or more of the identified players in the interactive video. As used herein, "user" refers to any end user of the application, for example, a sports fan watching a game on one of the aforementioned devices.

[0044] In one embodiment, the sports analysis system 100 provides recognition-driven interactivity, allowing viewers to obtain information about identified players. The sports analysis system 100 allows users to interact with identified players on the screen of a user device. For example, once a player is identified, visual cues such as associated icons and / or markers are displayed on the screen, and the user is notified about the recognized player, thereby opening a gateway for interaction. The contextual interactivity content is customized based on detection of the cues to meet the user's requirements. In another embodiment, the sports analysis system 100 allows users to interact with more than one recognized player in the frame by notifying the user with a choice to select a player of interest from the identified players. For example, if multiple players are recognized at any one time, an array of associated icons and / or markers is displayed to allow the user to select any player of interest for interaction.

[0045] The sports analysis system 100 provides user interactivity with and around existing graphic assets in the hypertext transfer protocol live stream 101a. The sports analysis system 100 dynamically overlays contextually interactive content layouts around graphics embedded in the hypertext transfer protocol live stream 101a. As used herein, "graphics" refers to any information displayed with the hypertext transfer protocol live stream 101a, such as scorecards, player statistics, player information, etc. The sports analysis system 100 recognizes the coordinates of graphic assets on the screen that are part of the hypertext transfer protocol live stream 101a, ensuring that additional overlays do not overlap each other and provide a good user experience. Additionally, the sports analysis system 100 provides a wide range of information for the user as on-demand contextually interactive content on top of the overlaid graphics in the hypertext transfer protocol live stream 101a. For example, when a user clicks on a scorecard, an expanded version of the scorecard opens, revealing various information that was not previously available. In addition, users will also be able to interact with and customize the expanded version of the scorecard, and users will have access to richer, on-demand, contextual and interactive content in real time.

[0046] 2 exemplarily illustrates the architecture of a computer server employed by the sports analysis system 100 for analyzing live video broadcast streams of sporting events. The computer system 200 includes, for example, a processor 201, a non-transitory computer-readable storage medium such as a memory unit 202 for storing programs and data, an input / output (I / O) controller 203, a network interface 204, a data bus 205, a display unit 206, an input device 207, a fixed media drive 208, an output device 210, etc. The computer server 200 may optionally include a removable media drive 209 for receiving removable media. The processor 201 reads instructions from the memory unit 202, for example, for executing modules 103, 104, 105, 106, 107, 108, and 109 of the sports analysis system 100.

[0047] 3 illustrates a method for analyzing a live video broadcast stream of a sporting event. The method disclosed herein employs a sports analysis system 100 comprising at least one processor 201 configured to execute computer program instructions for analyzing a live video broadcast stream of a sporting event. The sports analysis system 100 uses a video demultiplexer 101 to receive 301 a live video broadcast stream of a sporting event from a video capture device. The video demultiplexer 101 of the sports analysis system 100 splits 302 the live broadcast stream into a real-time messaging protocol stream 101b and a hypertext transfer protocol live stream 101a. The sports analysis system 100 analyzes 303 the real-time messaging protocol stream 101b within a predetermined time interval. The predetermined time interval comprises a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream. The predetermined time interval is less than a frame duration of a frame. The sports analysis system 100 detects 303a one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from the first set of cues, comprising face detection, head detection, person detection, and jersey number detection, based on availability of the first set of cues.

[0048] The sports analysis system 100 tracks 303b the detected person in the frame by analyzing one or more previous frames of the real-time messaging protocol stream. The sports analysis system 100 recognizes 303c the tracked person in the frame using one or more cues selected from a second set of cues comprising facial recognition 106a, person recognition 106b, jersey recognition 106c, a live score feed of the sporting event 106d, and a commentary of the sporting event 106e based on the availability of the second set of cues. The jersey recognition 106c comprises one or more of a jersey number classification and a team recognition. An individual weighting is assigned to each of the facial recognition 106a, person recognition 106b, and jersey recognition 106c when the facial recognition 106a, person recognition 106b, and jersey recognition 106c are performed in the frame. The individual weighting is determined based on the clarity of each person recognized in the frame. If any of facial recognition 106a, person recognition 106b, and jersey recognition 106c are unavailable, one or more of live score feed 106d and commentary 106e are used to identify one or more players in the frame. The sports analysis system 100 identifies 303d one or more players in the frame by comparing the assigned weighting for each of the recognized people to the prior data of all players.

[0049] The sports analysis system 100 transmits 303e the hypertext transfer protocol live stream 101a and the analysis data, including the contextual interactive content of the identified players, to a client application 111a provided by the sports analysis system 100 on the user device 111 within a predetermined time interval. Thus, the live video broadcast stream is converted into an interactive video, including the contextual interactive content of the identified players. The contextual interactive content of the identified players is displayed on the user device without overlapping the graphics of the live video broadcast stream of the sporting event. The contextual interactive content of the identified players comprises a dynamic display of one or more of a scorecard, player information, match statistics, and the contextual interactive content may be customized for sharing on social media. The user may interact with one or more of the identified players in the interactive video.

[0050] 7A-7D exemplarily illustrate screenshots of on-demand enhanced contextual interactivity content overlaid on a HyperText Transfer Protocol live stream 101a of a sporting event. The contextual interactivity content is displayed without overlapping with existing graphics displayed on the HyperText Transfer Protocol live stream 101a of a sporting event. As used herein, "contextual interactivity content" refers to any on-demand enhanced content streamed in real-time onto a user device 111 to enhance user interaction and audience growth. FIG. 7A illustrates an exemplary screenshot of a contextual overlay rendering that renders contextual interactivity content comprising a dynamic display of one or more of a scorecard, player information, and match statistics. FIG. 7B illustrates an exemplary screenshot of a rendering of a batsman's biography in a cricket game. FIG. 7C illustrates an exemplary screenshot of a rendering of a batsman's player statistics in a cricket game. The sports analysis system 100 renders the recognized player's contextual interactive content to display the player's overall statistics across different formats of the game, such as total matches, number of catches, total runs, strike rate, batting average, etc. The contextual interactive content can be customized according to requirements and shared on any social media platform at will. In some embodiments, a user can access the player's statistics for a particular season. In another embodiment, one or more social media posts of the player and team are provided as on-demand enhanced content, as exemplarily illustrated in FIG. 7D, which includes the player's interactive information, such as photos, videos, etc.

[0051] FIG. 8 exemplarily illustrates screenshots of rich contextual interactive content overlaid on the HyperText Transfer Protocol live stream 101a. Customized player match statistics are provided as rich content specifically for users to enhance their viewing interaction. The sports analysis system 100 shows match comparisons of key players in the match related to the games played on that match day on their statistics. FIG. 9A-9B exemplarily illustrates screenshots of rich contextual interactive content overlaid on the HyperText Transfer Protocol live stream 101a for user engagement and viewing experience. The rich contextual interactive content is customized to improve engagement and provide an enhanced viewing experience for users. For example, as shown in FIG. 9A, for sporting events such as cricket, rich contextual interactive content is delivered on the HyperText Transfer Protocol live stream 101a in the form of a poll to have users select their favorite team. Moreover, this type of rich contextual interactive content significantly improves user engagement to increase the audience base. In some embodiments, this may be customized as a legal gambling game, allowing users to bet on their favorite teams or players for the exchange of any rewards, however, the sports analysis system 100 may also be used for other sporting events to achieve desired results.

[0052] As exemplarily shown in FIG. 9B, in one embodiment, rich contextual interactive content in the form of a fantasy sports game is overlaid to keep the user actively engaged in the HyperText Transfer Protocol live stream 101a.

[0053] 10A-10D exemplarily illustrate screenshots of rich contextual interactive content overlaid on a HyperText Transfer Protocol live stream 101a to increase the monetization potential of existing content. Rich contextual interactive content is generated to increase the monetization potential of existing content for broadcasters and advertisers. The rich contextual interactive content can be at least one of targeted advertising, ticket sales, and branded merchandise. As illustrated in FIG. 10C, in another embodiment, the sports analysis system 100 increases the monetization potential by transmitting rich contextual interactive content regarding branded merchandise associated with the identified players and teams, encouraging users to purchase online directly from their portals. Similarly, one or more targeted advertisements can be presented based on the recognition of a cue, as illustrated in FIG. 10C. For example, when the sports analysis system 100 recognizes and tracks player A from team A for a predetermined time interval, it can push targeted advertisements for any brands associated with player A. In another embodiment, as shown in Fig. 10D, the monetization possibilities are also increased by transmitting a ticket sales platform on the user device 111, allowing the user to reserve tickets for himself / herself for any subsequent games of the season in a sporting event. For example, in a cricket league knockout game, when team A is playing team B, the sports analysis team recognizes the queue, detects the players and / or teams, and transmits it to the ticket sales platform, which gives the user the option to purchase tickets for the next game depending on the outcome of the ongoing game.

[0054] The sports analysis system 100 enhances video streams by providing an interactivity layer for user-engagement. The sports analysis system 100 increases viewer engagement by introducing interactivity, enabling content personalization, and performing multifaceted video and user analytics. The sports analysis system 100 captures user information about the user accessing the content. The user's demographics, favorite brands, social profile, digital spending information, favorite teams and / or players, time spent on the platform to access contextual interactive content, etc. are captured to deliver personalized content delivery, thereby increasing monetization. Similarly, the sports analysis system 100 enables content owners to deliver rich contextual advertising by recognizing who is on screen and identifying what the user is actually clicking on, on contextual interactive content, in a hypertext transfer protocol live stream 101a. As shown in FIG. 11, real-time recognition is a crucial part of this enhancement, as it helps provide context to make the content more relevant. The sports analysis system 100 can be made available on the client's website through a plug-in or integrated with the client's application (Android / iOS) through an SDK. Additionally, the sports analysis system 100 provides real-time frame-level recognition data to a real-time database along with other data components such as configuration data, player / team information, live game statistics, games for fans such as voting or fantasy, or online marketplaces. This data is downloaded periodically in the plug-in / SDK using which interactivity layers are presented to the consumer. With this configuration, the user can access / interact with the aforementioned value-adds. The sports analysis system 100 receives video streams in various formats, such as HyperText Transfer Protocol live streams, which are embedded in the plug-in / SDK.The plugin / SDK also handles synchronization of HyperText Transfer Protocol live streams at the frame level with downloaded recognition data.

[0055] A non-transitory computer readable storage medium is embodied with computer program code comprising instructions executable by at least one processor 201 in a sports analysis system 100 for analyzing a live video broadcast stream of a sporting event. The computer program code includes first computer program code for receiving a live video broadcast stream of the sporting event from one or more video capture devices, second computer program code for splitting the live broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream, third computer program code for analyzing the real-time messaging protocol stream within a predetermined time interval, the predetermined time interval comprising a phase difference between the real-time messaging protocol stream and the hypertext transfer protocol live stream, the predetermined time interval being less than a frame duration of a frame, and analyzing the real-time messaging protocol stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of the first set of cues. a fourth computer program code for detecting one or more persons present within a frame of a real-time messaging protocol stream; a fifth computer program code for tracking the detected persons within the frame by analyzing one or more previous frames of the real-time messaging protocol stream; and a sixth computer program code for recognizing the tracked persons within the frame using one or more cues selected from a second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and a commentary of the sporting event based on availability of a second set of cues, the jersey recognition comprising one or more of a jersey number classification and a team recognition, and when the facial recognition, person recognition, and jersey recognition are performed within a frame, individual weights are assigned to each of the facial recognition, person recognition, and jersey recognition, and any of the facial recognition, person recognition, and jersey recognition is / are:sixth computer program code, where if unavailable, one or more of the live score feed and commentary are used to identify one or more players in the frame, with individual weightings determined based on clarity of each person recognized in the frame; seventh computer program code, for identifying one or more players in the frame by comparing the assigned weightings for each of the recognized people to prior data for all players; and eighth computer program code, for transmitting, within a predetermined time interval, the HyperText Transfer Protocol live stream and the analytical data including the contextual interactive content of the identified players to a client application provided on the user device by the sports analysis system.

[0056] The computer program code further comprises a ninth computer program code for providing user interaction with the interactive video. The identified player's contextual interactive content is displayed on the user device without overlapping existing graphics of the HyperText Transfer Protocol live stream. The identified player's contextual interactive content comprises a dynamic display of one or more of a scorecard, player information, and match statistics, and the identified player's contextual interactive content may be customized and shared on social media. Similarly, the system enables users to share their favorite moments of the game online in real time on one or more social media platforms.

[0057] The computer program code is executed to display the contextual interactive content without overlapping with existing graphics displayed on the live video broadcast stream of the sporting event. The contextual interactive content also includes any information of interest about player statistics and biographies. The contextual interactive content can be customized according to requirements and shared on any social media platform at will. In some embodiments, the contextual interactive content can be customized to a particular season of the game and the corresponding player statistics in that season. In another embodiment, one or more social media posts of the players and teams are provided as on-demand enhanced content on the live video broadcast stream for more insight into the players' and teams' non-sporting activities. In another embodiment, customized player match statistics are provided as rich content specifically for users to enhance their viewing interaction. In another embodiment, the contextual interactive content is customized to improve engagement and provide an enhanced gaming experience for users.

[0058] The computer program code is stored in a computer usable or computer readable memory capable of instructing a computer or other programmable data processing device to implement functionality in a particular manner. The computer program code may also be implemented by an article of manufacture containing instructions for implementing each block of the flowchart or the functions described in the flowchart. The computer program instructions may be loaded onto a computer or other programmable data processing device, and a series of operational steps may be performed on the computer or other programmable data processing device to execute the instructions for the sports analysis system 100.

[0059] The sports analysis system 100 for analyzing live video broadcast streams of sporting events provides significant benefits for end users, content owners, advertisers, broadcasters, sports companies, sports franchises, teams, etc. in a multifaceted manner. Users can simply interact with the contextual content displayed around the players and share their favorite moments of the game online in one or more social media platforms. The platform also allows users to purchase tickets for games of interest and merchandise sold by the associated team or franchise. The sports analysis system 100 allows content owners to capture users' demographics, brands followed by users, social profiles, digital spending information, favorite teams and / or players, time spent on an AI-powered contextual interactive content delivery platform, etc. Furthermore, this information can be used to provide personalized content delivery and profitability. The sports analysis system 100 helps generate incremental revenue through pay-per-fan interaction mode and does not require any additional hardware, such as cameras and sensors, for implementation. Existing hardware is sufficient to implement the system in a cost-effective and hassle-free manner. The sports analysis system 100 allows content owners to choose from a wide range of interactive options and provide them to users, allowing content owners to deliver contextual advertising based on who is on screen and what viewers actually click on. Users get a significant advantage by following their favorite players, their statistics, and their social media updates in one place, getting richer information to enjoy the game, and being more interactive with the live video broadcast stream.

[0060] The foregoing examples are provided merely for illustration and are not to be construed in any way as limiting the method and sports analysis system 100 disclosed herein. Although the method and sports analysis system 100 have been described with reference to various embodiments, it is understood that the words used herein are words of description and illustration, rather than words of limitation. Furthermore, although the method and sports analysis system 100 have been described herein with reference to specific means, materials, and embodiments, the method and sports analysis system 100 are not intended to be limited to the particular items disclosed herein, but rather, the method and sports analysis system 100 extends to all functionally equivalent structures, methods, and uses, such as those within the scope of the appended claims. While the embodiments described herein are directed to analyzing live video broadcast streams of sporting events, the embodiments are also directed to analyzing pre-recorded video streams of sporting events, as recited in the claims. While multiple embodiments are disclosed, it will be understood by those skilled in the art having the benefit of the present teachings herein that the methods and sports analysis systems 100 disclosed herein may be modified, other embodiments may be produced, and changes may be made thereto without departing from the scope and spirit of the methods and sports analysis systems 100 disclosed herein.

Claims

1. 1. A sports analysis system incorporating a computer server for analyzing a live video broadcast stream of a sporting event, the sports analysis system comprising: a video demultiplexer for receiving the live video broadcast stream of the sporting event from a video capture device and splitting the received live video broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream; a non-transitory computer readable storage medium for storing computer program instructions defined by a plurality of modules of the sports analysis system; at least one processor communicatively coupled to the non-transitory computer-readable storage medium; Equipped with the at least one processor is configured to execute the computer program instructions defined by the plurality of modules of the sports analysis system; The plurality of modules include: an analysis module for analyzing the real-time messaging protocol stream within a predetermined time interval, the analysis module comprising: a detection module for detecting one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of a first set of cues; a tracking module for tracking the detected person within one or more previous frames of the real-time messaging protocol stream by analyzing the frames; a recognition module for recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; a comparison module for identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; an analysis module comprising: a transmission module for transmitting the HyperText Transfer Protocol live stream and analysis data including the identified player's interactive content to a client application provided by the sports analysis system on a user device within the predetermined time interval, wherein the live video broadcast stream is converted into an interactive video including the identified player's interactive content, and the identified player's interactive content is displayed on the user device without the identified player's interactive content overlapping with existing graphics of the HyperText Transfer Protocol live stream; A sports analysis system comprising:

2. 2. The sports analysis system of claim 1, wherein if any one of the facial recognition, the person recognition, and the jersey recognition is unavailable, the recognition module uses one or more of the live score feed and the commentary for the identification of the one or more players in the frame.

3. The sports analysis system of claim 1 , wherein the predetermined time interval is less than a frame duration of the frame.

4. The sports analysis system of claim 1 , wherein the jersey recognition comprises one or more of jersey number classification and team recognition.

5. The sports analysis system of claim 1 , wherein the individual weights are determined based on the clarity of each person recognized in the frame.

6. The sports analysis system of claim 1 , wherein the client application comprises an interaction module for providing user interaction with the interactive video.

7. 2. The sports analysis system of claim 1, wherein the interactive content of the identified player comprises a dynamic display of one or more of a scorecard and player information and match statistics, and the interactive content of the identified player is configured to be customized and shared on social media.

8. 1. A method for analyzing a live video broadcast stream of a sporting event, the method employing a sports analysis system comprising at least one processor configured to execute computer program instructions for performing the method; The method comprises: a video demultiplexer of the sports analysis system receiving the live video broadcast stream of the sporting event from a video capture device; the video demultiplexer of the sports analysis system splitting the live video broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream; analyzing the real-time messaging protocol stream within a predetermined time interval, the analyzing comprising: detecting one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of a first set of cues; Tracking the detected person within one or more previous frames of the real-time messaging protocol stream by analyzing the frames; recognizing the tracked person in the frame using one or more cues selected from a second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; identifying one or more players within the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; and transmitting the HyperText Transfer Protocol live stream and analysis data including the identified player's interactive content to a client application provided on a user device by the sports analysis system within the predetermined time interval; Including, the live video broadcast stream is converted into an interactive video including the interactive content of the identified player and transmitted to the user device; The interactive content of the identified player is displayed on the user device without the interactive content of the identified player overlapping with graphics of the HyperText Transfer Protocol live stream.

9. The method of claim 8, wherein if any one of the facial recognition and the person recognition and the jersey recognition is unavailable, one or more of the live score feed and the commentary are used to identify the one or more players in the frame.

10. The method of claim 8 , wherein the predetermined time interval is less than a frame duration of the frame.

11. The method of claim 8 , wherein the jersey recognition comprises one or more of a jersey number classification and a team recognition.

12. The method of claim 8 , wherein the individual weights are determined based on the clarity of each person recognized in the frame.

13. The method of claim 8 , wherein a user can interact with one or more of the identified players within the interactive video.

14. 9. The method of claim 8, wherein the interactive content for the identified player comprises a dynamic display of one or more of a scorecard, player information, and match statistics, and the interactive content for the identified player is configured to be customized and shared on social media.

15. A non-transitory computer-readable storage medium having a plurality of computer program code embodied thereon, the computer program code comprising instructions executable by at least one processor in a sports analysis system for analyzing a live video broadcast stream of a sporting event; The plurality of computer program codes include: first computer program code for receiving the live video broadcast stream of the sporting event from one or more video capture devices; second computer program code for splitting the live video broadcast stream into a real-time messaging protocol stream and a hypertext transfer protocol live stream; third computer program code for analyzing the real-time messaging protocol stream within a predetermined time interval; fourth computer program code for detecting one or more people present within a frame of the real-time messaging protocol stream using one or more cues selected from the first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of a first set of cues; fifth computer program code for tracking the detected person within one or more previous frames of the real-time messaging protocol stream by analyzing the frames; sixth computer program code for recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a live score feed of the sporting event, and commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; seventh computer program code for identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; an eighth computer program code for transmitting, within the predetermined time interval, the HyperText Transfer Protocol live stream and analysis data including interactive content of the identified player to a client application provided by the sports analysis system on a user device, wherein the interactive content of the identified player is displayed on the user device without the interactive content of the identified player overlapping existing graphics of the HyperText Transfer Protocol live stream; 1. A non-transitory computer-readable storage medium comprising:

16. 16. The non-transitory computer-readable storage medium of claim 15, wherein if any one of the facial recognition and the person recognition and the jersey recognition is unavailable, one or more of the live score feed and the commentary are used to identify the one or more players in the frame.

17. The non-transitory computer readable storage medium of claim 15, wherein the eighth computer program code further causes the live video broadcast stream to be converted into an interactive video including the interactive content for the identified player, and the plurality of computer program codes further comprises a ninth computer program code for providing user interaction with the interactive video.

18. 16. The non-transitory computer-readable storage medium of claim 15, wherein the predetermined time interval is less than a frame duration of the frame.

19. 16. The non-transitory computer readable storage medium of claim 15, wherein the jersey recognition comprises one or more of a jersey number classification and a team recognition.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the individual weights are determined based on clarity of each person recognized in the frame.

21. 16. The non-transitory computer-readable storage medium of claim 15, wherein the interactive content for the identified player comprises a dynamic display of one or more of a scorecard and player information and match statistics, and the interactive content for the identified player is configured to be customized and shared on social media.

22. 1. A sports analysis system incorporating a computer server for analyzing pre-recorded video streams of a sporting event, the sports analysis system comprising: a non-transitory computer readable storage medium for storing computer program instructions defined by a plurality of modules of the sports analysis system; at least one processor communicatively coupled to the non-transitory computer-readable storage medium; Equipped with the at least one processor is configured to execute the computer program instructions defined by the plurality of modules of the sports analysis system; The plurality of modules include: an analysis module for analyzing the pre-recorded video stream, the analysis module comprising: a detection module for detecting one or more people present within a frame of the pre-recorded video stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of a first set of cues; a tracking module for tracking the detected person in the frame by analyzing one or more previous frames; a recognition module for recognizing the tracked person in the frame using one or more cues selected from the second set of cues comprising facial recognition, person recognition, jersey recognition, a score feed of the sporting event, and commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; a comparison module for identifying one or more players in the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; an analysis module comprising: a transmission module for transmitting the pre-recorded video stream and analytics data including the identified player's interactive content to a client application provided on a user device by the sports analysis system, the pre-recorded video stream being converted into an interactive video including the identified player's interactive content, and the identified player's interactive content being displayed on the user device without the identified player's interactive content overlapping existing graphics of the pre-recorded video stream; A sports analysis system comprising:

23. 1. A method for analyzing a pre-recorded video stream of a sporting event, the method employing a sports analysis system comprising at least one processor configured to execute computer program instructions for performing the method; The method comprises: analyzing the pre-recorded video stream, the analyzing comprising: detecting one or more people present within a frame of the pre-recorded video stream using one or more cues selected from a first set of cues comprising face detection, head detection, person detection, and jersey number detection based on availability of a first set of cues; tracking the detected person in one or more previous frames by analyzing the frames; recognizing the tracked person in the frame using one or more cues selected from a second set of cues comprising facial recognition, person recognition, jersey recognition, a score feed of the sporting event, and commentary of the sporting event based on availability of a second set of cues, wherein when the facial recognition, the person recognition, and the jersey recognition are performed in the frame, individual weights are assigned to each of the facial recognition, the person recognition, and the jersey recognition; identifying one or more players within the frame by comparing the assigned weights for each of the recognized persons with prior data of all players; and transmitting the pre-recorded video stream and analysis data including the interactive content of the identified player to a client application provided on a user device by the sports analysis system; Including, the pre-recorded video stream is converted into an interactive video containing the interactive content for the identified player and transmitted to the user device; The method, wherein the interactive content of the identified player is displayed on the user device without the interactive content of the identified player overlapping with existing graphics of the pre-recorded video stream.

Images