Accuracy detection in competitive performances

Abstract

Disclosed are various approaches for accuracy detection in competitive performances using artificial intelligence (AI) technology. A system can include a plurality of cameras in a plurality of different positions surrounding a competitive event including a plurality of individuals performing a plurality of actions. A computing device can determine a skill level associated with the competitive event, select a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level, obtain a plurality of images from the plurality of cameras, and generate a plurality of inputs based at least in part on the plurality of images. The computing device can apply the plurality of inputs to a skill accuracy model, and determine an accuracy associated with at least one action of the plurality of actions based at least in part on an output of the skill accuracy model.

Description

TITLE: ACCURACY DETECTION IN COMPETITIVE PERFORMANCESInventors: Foy N. Chalk and Alison DitkoCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to, and the benefit of, U.S. Provisional Patent Application 63 / 530,790, entitled “ACCURACY DETECTION IN COMPETITVE PERFORMANCES,” which was filed on August 4, 2023, and is incorporated by reference as if set forth herein in its entirety.BACKGROUND

[0002] The sport of competitive performance, whether cheer, dance, etc., is the competition between teams of performance athletes who are evaluated and scored against each other. A panel of judges scores the teams based on difficulty, execution, technique, etc. The judges must have a keen eye to watch every individual on the mat to evaluate at least every motion, jump, and tumbling sequence. Without a one-to-one match of judges to performers, it is impossible for the judges to assess every performer uniformly. Teams are aware of the judge’s incapability of seeing everything and often exploit this weakness by, for example, hiding their lowest jumpers or non-tumblers in the back of the group. This weakness and its exploitation along with bias in human judgement can lead to inconsistent and, thus, ineffective scoring.SUMMARY

[0003] Aspects of the present disclosure are related to accuracy detection in competitive performances using artificial intelligence (Al) technology. In one aspect, a system is provided, where the system includes a plurality of cameras positioned in a plurality of different positions surrounding a competitive event. The competitive event can include a plurality of individuals performing a plurality of actions during a duration of the competitive event. The system can further include a computing device comprising a processor and a memory, and machine-readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least determine a skill level associated with the competitive event, select a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level, obtain a plurality of images from the plurality of cameras (where the plurality of images capture the plurality of individuals performing at least one action of the plurality of actions), and generate a plurality of inputs based at least in part on the plurality of images. The machine-readable instructions can further cause the computing device to at least apply the plurality of inputs to a skill accuracy model, and determine an accuracy associated with at least one action of the plurality of actions based at least in part on an output of the skill accuracy model.

[0004] In some examples, the skill level is one of a plurality of skill levels, where each skill level requires a performance of a different set of actions during the competitive event. In some examples, individual skill accuracy models of the plurality of skill accuracy models correspond to a respective skill level of the plurality of skill levels. According to at least one example, the competitive event comprises a competitive cheerleading competition performance.

[0005] The individuals can perform the plurality of actions on a stage, where a first camera is positioned about a front position relative to the stage, a second camera is positioned about a first back position relative to the stage, and a third camera is positioned about a second back position relative to the stage. According to various aspects, the skill accuracy model is trained to analyze the plurality of inputs associated with the plurality of images to determine whether a threshold number of individuals of the plurality of individuals performed the action accurately.

[0006] The machine-readable instructions can further cause the computing device to at least generate a report detailing a respective accuracy summary for each action performed of the plurality of actions, and transmit the report to a client device. Additionally, the machine- readable instructions can further cause the computing device to at least obtain a plurality of training images associated with a plurality of other individuals performing the plurality of actions associated with the skill level, and train the skill accuracy model associated with the skill level based at least in part on the plurality of training images. In some examples, the plurality of images corresponds to video content. In some examples, a first subset of images from the plurality of images are obtained from a first camera of the plurality of cameras and a second subset of images are obtained from a second camera of the plurality of cameras.

[0007] In another aspect, a method is provided, the method including the steps of: determining, by at least one computing device, a skill level associated with a competitive event comprising a plurality of individuals performing a plurality of actions during a duration of the competitive event; selecting, by the at least one computing device, a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level; obtaining, by the at least one computing device, a plurality of images from a plurality of cameras positionedin a plurality of different positions surrounding the competitive event, where the plurality of images capture the plurality of individuals performing at least one action of the plurality of actions; generating, by the at least one computing device, a plurality of inputs based at least in part on the plurality of images; applying, by the at least one computing device, the plurality of inputs to a skill accuracy model; and determining, by the at least one computing device, an accuracy associated with the at least one action of the plurality of actions based at least in part on an output of the skill accuracy model. In some examples, the skill level is one of a plurality of skill levels, each skill level requiring a performance of a different set of actions during the competitive event. Individual skill accuracy models of the plurality of skill accuracy models can correspond to a respective skill level of the plurality of skill levels. The competitive event can comprise a competitive cheerleading competition performance. In some examples, the individuals perform the plurality of actions on a stage, where a first camera is positioned about a front position relative to the stage, a second camera is positioned about a first back position relative to the stage, and a third camera is positioned about a second back position relative to the stage.

[0008] The skill accuracy model can be trained to analyze the plurality of inputs associated with the plurality of images to determine whether a threshold number of individuals of the plurality of individuals performed the action accurately. According to various aspects, the method can further include the steps of: generating a report detailing a respective accuracy summary for each action performed of the plurality of actions; and transmitting the report to a client device. Similarly, the method can further include the steps of obtaining a plurality of training images associated with a plurality of other individuals performing the plurality of actions associated with the skill level; and training the skill accuracy model associated withthe skill level based at least in part on the plurality of training images. The plurality of images can correspond to video content. A first subset of images from the plurality of images can be obtained from a first camera of the plurality of cameras and a second subset of images can be obtained from a second camera of the plurality of cameras.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, with emphasis instead being placed upon clearly illustrating the principles of the disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

[0010] FIG. 1 is a drawing depicting an example scenario of an accuracy detection system in use during a competitive performance according to one of several embodiments of the present disclosure.

[0011] FIG. 2 is a drawing of a network environment according to various embodiments of the present disclosure.

[0012] FIGS. 3A-3F include tables defining various skills / actions required in a performance of a given skill level for a cheerleading competition according to various embodiments of the present disclosure.

[0013] FIG. 4 is a flowchart illustrating one example of functionality implemented as portions of an application executed in a computing environment in the network environment of FIG. 2 according to various embodiments of the present disclosure.

[0014] FIG. 5 is a flowchart illustrating one example of functionality implemented as portions of an application executed in a computing environment in the network environment of FIG. 2 according to various embodiments of the present disclosure.DETAILED DESCRIPTION

[0015] Disclosed are various approaches for accuracy detection in competitive performances using artificial intelligence (Al) technology. In particular, the present disclosure relates to developing and training a series of custom computer vision models that will analyze competitive performances (e.g., cheer routines) in real time. According to various embodiments, the models of present disclosure are trained to identify anomalies and classify activities or motions e.g., stunts, jumps, movements, etc.) performed by one or more individuals in a competitive performance, thereby providing a more objective and unbiased assessment of each routine. In various examples, the developed models are deployed to a local edge server at the competition sites for real-time inferencing.

[0016] Competitive performances e.g., cheer competitions, dance competitions, etc.) currently experience a lack of accuracy in judging the competitions, which is largely due to the bias and errors inherent in human judgement. Therefore, it would be beneficial to provide an accuracy detection system that can analyze competitive performances in real-time and remove subjectivity and bias from the scoring process, in order to improve the credibility and professionalism of the sport. Improvements resulting from the accuracy detection system of the present disclosure can include, for example: (1) a reduction in the number of human judges needed per event, leading to labor cost savings; (2) reduced subjectivity, thus legitimizing the sport and enabling its growth; and (3) providing teams with the validation that their fundsspent on each competition will result in a fair and accurate outcome. According to various examples, the accuracy detection system of the present disclosure focuses on identifying outliers and detecting anomalies with the intent of eliminating bias from judging and to help confirm existing judges' scores. In addition, the accuracy detection system is configured to classify skills and provide the ability to judge the quality of skill execution relative to an existing quality baseline. Furthermore, the accuracy detection system of the present disclosure can showcase how it can leverage computer vision technology to further drive innovation and competitive differentiation as it scales its business.

[0017] Turning now to FIG. 1, shown is an example scenario of a competitive performance and the use of the accuracy detection system 100 to accurately detect and identify outliers and anomalies with the intent of eliminating bias from judging and to help confirm existing judges' scores. For example, FIG. 1 illustrates a group of individuals e.g., cheer squad, dance team, etc.) that are on a performance stage and participating in a competitive event performance. In various examples, the competitive performance can comprise the group of individuals performing one of more actions (e.g., stunts, movements, jumps, etc.) during a predefined time for the purpose of being evaluated and scored based at least in part on a variety of factors including, difficulty, execution, technique, etc., of the performed actions.

[0018] In accordance with various embodiments, FIG. 1 illustrates a plurality of sensors 103 (e.g., 103a, 103b, 103c, etc.) that are distributed about the stage and in view of the performance athletes / indivi duals. In various examples, the sensors 103 (e.g., cameras) can be positioned at different angles about the performance stage to capture the competitive performances. In various examples, the sensors 103 are positioned about the stage and in view of the individuals in order to capture content (e.g., images, video, etc.) associated with theperformance of a given routine or competitive event. Although FIG. 1 illustrates three cameras where a first camera 103a is positioned in front of the stage, a second camera 103b is positioned in a first back position relative to the stage and a third camera 103c is positioned in a second back position relative to the stage, it should be appreciated that the accuracy detection system 100 can comprise any number of sensors 103 positioned at any position (e.g., front, back, left side, right side, diagonal, etc.) relative to the stage.

[0019] In various examples, the accuracy detection system 100 of the present disclosure can comprise a plurality of sensors 103 in data communication with a computing environment (FIG. 2) configured to analyze image or video content obtained from the sensors 103 using one or more trained skill accuracy models 106 to detect inaccuracies and / or identify anomalies associated with the performance of the group of individuals. In various examples, the content obtained from the sensors 103 can be used to generate input features 109 (e.g., 109a, 109b, 109c, etc.) that can be used as inputs to a trained skill accuracy model 106. The skill accuracy model 106 can be trained to detect individual performers. In some examples, the skill accuracy model 106 can be trained to isolate individual performers and generate a video for each individual performer. The skill accuracy model 106 can be trained to detect anomalies among the performing individuals, annotate and classify activities / actions, and judge the quality of skill execution associated with a skill level of a given competitive performance.

[0020] In various examples, the output 112 of the skill accuracy model 106 can include one or more predictions associated with the detection and accuracy of one or more actions performed by the one or more individuals. For example, if a given type of performance requires all individuals to perform a forward roll during the routine at the same time, the skill accuracy model 106 can be trained to detect the occurrence of a forward roll during the routine(e.g, classification of forward roll) and further identify whether all individuals performed the forward roll accurately (e.g., detect anomalies with the given action). The output 112 can indicate whether the action (e.g., forward roll) was performed correctly by the correct number of individuals in the performance. In some examples, the output 112 can include annotations for each video of an individual performer. The annotations can be representative of one or more scores for skill, quality, or other evaluative marker of performance. The annotations can be used to score an individual’s performance as well as can be used for future training sets.

[0021] In various examples, the output 112 can be used to generate a report that includes analytics derived from the output 112 of the skill accuracy model 106. In various examples, the report can comprise a user interface that is rendered on a display of a client device in the form of a dashboard. The dashboard / user interface can be designed to display analytics derived from predictions and further provide real-time data insights on the routines being performed, including anomalies detected and the classification of activities.

[0022] According to various examples, different skill accuracy models 106 can be trained to analyze competitive performances for different skill levels. For example, in competitive cheerleading, there may be multiple competitions of teams based on skill levels where different skill levels require different actions (e.g., stunts, jumps, movements, etc.) where performers in the higher skill level competitions are required to perform actions that may be considered more complex or difficult than the actions required in a lower skill level. According to various examples, the skill accuracy model 106 can be trained using annotated single-person videos to recognize specific skills and their associated quality levels. In some examples, a SlowFast Networks for Video Recognition can be used as a classification model for incorporation into the skill accuracy model 106. For example, a “SlowFast Network forVideo Recognition” can comprise a network having a first “slow” pathway for processing semantics of an object in a video and a second “fast” pathway for processing motion of the object in the video. The slow pathway comprises a convolutional model which models a video clip as a spatiotemporal volume. The fast pathway comprises a convolutional model analogous to the slow pathway, except that the fast pathway has a high frame rate, high temporal resolution features, and low channel capacity. The network is created by fusing the slow pathway and the fast pathway with lateral connections. A SlowFast Network can be trained using the annotated single-person videos and incorporated into the skill accuracy model 106.

[0023] In the following discussion, a general description of the system and its components is provided, followed by a discussion of the operation of the same. Although the following discussion provides illustrative examples of the operation of various components of the present disclosure, the use of the following illustrative examples does not exclude other implementations that are consistent with the principals disclosed by the following illustrative examples.

[0024] With reference to FIG. 2, shown is a network environment 200 according to various embodiments. The network environment 200 can include a computing environment 203, a client device 206, and one or more sensors 103, which can be in data communication with each other via a network 209.

[0025] The network 209 can include wide area networks (WANs), local area networks(LANs), personal area networks (PANs), or a combination thereof. These networks can include wired or wireless components or a combination thereof. Wired networks can include Ethernet networks, cable networks, fiber optic networks, and telephone networks such as dial-up, digital subscriber line (DSL), and integrated services digital network (ISDN) networks.Wireless networks can include cellular networks, satellite networks, Institute of Electrical andElectronic Engineers (IEEE) 802.11 wireless networks (z.e., WI-FI®), BLUETOOTH® networks, microwave transmission networks, as well as other networks relying on radio broadcasts. The network 209 can also include a combination of two or more networks 209. Examples of networks 209 can include the Internet, intranets, extranets, virtual private networks (VPNs), and similar networks.

[0026] The computing environment 203 can include one or more computing devices that include a processor, a memory, and / or a network interface. For example, the computing devices can be configured to perform computations on behalf of other computing devices or applications. As another example, such computing devices can host and / or provide content to other computing devices in response to requests for content.

[0027] Moreover, the computing environment 203 can employ a plurality of computing devices that can be arranged in one or more server banks or computer banks or other arrangements. Such computing devices can be located in a single installation or can be distributed among many different geographical locations. For example, the computing environment 203 can include a plurality of computing devices that together can include a hosted computing resource, a grid computing resource, an edge computing resource, or any other distributed computing arrangement. In some cases, the computing environment 203 can correspond to an elastic computing resource where the allotted capacity of processing, network, storage, or other computing-related resources can vary over time.

[0028] Various applications or other functionality can be executed in the computing environment 203. The components executed on the computing environment 203 include acompetitive event service 212, and other applications, services, processes, systems, engines, or functionality not discussed in detail herein.

[0029] The competitive event service 212 can be executed to train one or more skill accuracy models 106 using images and / or video content e.g., event content data 215) generated fortraining data, obtained from past competitive events associated with a given skill level, and / or obtained from another type of resource. For example, a skill accuracy model 106 for a given skill level can be trained by the competitive event service 212 to (1) classify activities (e.g., movements, stunts, jumps, etc.) expected during a competitive performance for the skill level and (2) detect anomalies in the performance of the actions / activities, as well as perform various other actions.

[0030] The competitive event service 212 can further be executed to facilitate the use and execution of a skill accuracy model 106 to classify actions being performed and detect anomalies in the actions being performed in real time. For example, the competitive event service 212 can be executed to determine a skill level associated with a given performance and select a trained skill accuracy model 106 corresponding to the determined skill level. In some examples, the skill level is provided by a user interacting with a user interface 218 associated with the competitive event service 212.

[0031] During the duration of the performance, the competitive event service 212 can obtain event content data 215 from the one or more sensors 103 capturing content associated with the performing individuals. The competitive event service 212 can detect individual performers and track each performer separately for the duration of the performance. Similarly, in some examples, the competitive event service 212 can convert the event content data 215 from the one or more sensors 103 into individual videos for each tracked performer. Then,the competitive event service 212 can analyze the movements of each individual based at least in part on their respective video. The competitive event service 212 can generate input features from an analysis of the event content data 215 obtained from the one or more sensors 103 and apply the input features as inputs to the selected skill accuracy model 106. The output 112 (FIG. 1) of the skill accuracy model 106 can include a classification of an action being performed during the performance and an identification of any anomalies associated with the action in accordance to the skill level data 221 and corresponding skill rules 224.

[0032] In various examples, the competitive event service 212 can generate a user interface 218 or corresponding user interface data for generating a user interface 218 that includes analytics associated with the output 112 of the skill accuracy model 106. For example, the user interface 218 can include a report that identifies the different skills performed and whether or not the skills were accurately performed by the required number of individuals.

[0033] Also, various data is stored in a data store 227 that is accessible to the computing environment 203. The data store 227 can be representative of a plurality of data stores 227, which can include relational databases or non-relational databases such as object-oriented databases, hierarchical databases, hash tables or similar key -value data stores, as well as other data storage applications or data structures. Moreover, combinations of these databases, data storage applications, and / or data structures may be used together to provide a single, logical, data store. The data stored in the data store 227 is associated with the operation of the various applications or functional entities described below. This data can include sensor data 230, event content data 215, an accuracy report 233, skill level data 221, skill accuracy models 106, network content data 236, and potentially other data.

[0034] The sensor data 230 relates to data associated with a given sensor 103. For example, the sensor data 230 can comprise a sensor brand, a sensor version, a sensor name, one or more application programming interface (API) calls for communicating with the sensor 103, or other types of characteristic data. In addition, the sensor data 230 may comprise sensor location data 239 the given sensor 103 relative to the performance stage.

[0035] The event content data 215 can include image or video content associated with the performance of actions by one or more individuals. In various examples, the event content data 215 can include historical image or video content data associated with prior competitions. The event content data 215 can further include real time image and video content collected by the one or more sensors 103 during a given performance and can be used to generate the input features that are input into a skill accuracy model 106 for detecting anomalies within a performance and the classification of a given action or activity. In various examples, the event content data 215 can be used to train and / or update a skill accuracy model 106 for a given skill level.

[0036] The accuracy report 233 can include analytics derived from the output 112 of the skill accuracy model 106. In various examples, the report 233 can comprise a user interface that is rendered on a display 242 of a client device 206 in the form of a dashboard. The dashboard / user interface 218 can be designed to display the accuracy report 233 derived from predictions of the skill accuracy model 106, and further provide real-time data insights on the routines being performed, including anomalies detected and the classification of activities.

[0037] The skill level data 221 can include skill rules 224 that define the requirements and skills of a given performance for a given skill level. In various examples, a skill accuracy model 106 is trained to classify activities and detect anomalies of activities that are requiredto be performed during a competitive performance associated with a given skill level. The skill rules 224 can include a list of required skills, threshold values associated with a number of individuals required to perform a given skill / action during a routine, rules governing a given competition (e.g., US All Star Federation, etc.), and / or other data. The skill rules 224 can be used in conjunction with the training event content data 215 to train a given sill accuracy model 106. FIGS. 3A-3F include examples of required skills / actions for a given skill level during a given cheerleading competition.

[0038] The skill accuracy model 106 comprises a machine learning model trained to analyze event content data 215 to detect anomalies within a performance and classify actions / activities occurring during the performance for a given skill level. The skill accuracy model 106 can include, for example, a decision tree classifier, a gradient boost classifier, a Gaussian naive Bayes classifier, a reinforcement learning algorithm, a logistic regression classifier, a random forest classifier, a decision tree classifier, a multi-layer perceptron classifier, a recurrent neural network, a neural network, a label-specific attention network, an ensemble model, and / or any other type of trained model as can be appreciated.

[0039] The network content data 236 can include various data employed in generating user interfaces 218 and / or other network pages. The network content data 236 can include hypertext markup language (HTML), extensible markup language (XML), cascading style sheets (CSS), images, text, audio, video, templates, and / or other data.

[0040] The sensor 103 may comprise a camera device that captures a video or an image of one or more objects (e.g., performing individuals) positioned in the viewing range of the sensor 103. In some examples, the sensor 103 can comprise a heat sensor, an infrared sensor,a motion sensor, or other form or combination of sensors which are capable of detecting detailed motions.

[0041] The client device 206 is representative of a plurality of client devices that can be coupled to the network 209. The client device 206 can include a processor-based system such as a computer system. Such a computer system can be embodied in the form of a personal computer (e.g., a desktop computer, a laptop computer, or similar device), a mobile computing device e.g., personal digital assistants, cellular telephones, smartphones, web pads, tablet computer systems, music players, portable game consoles, electronic book readers, and similar devices), media playback devices (e.g., media streaming devices, BluRay® players, digital video disc (DVD) players, set-top boxes, and similar devices), a videogame console, or other devices with like capability. The client device 206 can include one or more displays 242, such as liquid crystal displays (LCDs), gas plasma-based flat panel displays, organic light emitting diode (OLED) displays, electrophoretic ink (“E-ink”) displays, projectors, or other types of display devices. In some instances, the display 242 can be a component of the client device 206 or can be connected to the client device 206 through a wired or wireless connection.

[0042] The client device 206 can be configured to execute various applications such as a client application 245 or other applications. The client application 245 can be executed in a client device 206 to access network content served up by the computing environment 203 or other servers, thereby rendering a user interface 218 on the display 242. To this end, the client application 245 can include a browser, a dedicated application, or other executable, and the user interface 218 can include a network page, an application screen, or other user mechanism for obtaining user input The client device 206 can be configured to execute applicationsbeyond the client application 245 such as email applications, social networking applications, word processors, spreadsheets, or other applications.

[0043] Referring next to FIG. 4, shown is a flowchart that provides one example of the operation of a portion of the competitive event service 212. The flowchart of FIG. 4 provides merely an example of the many different types of functional arrangements that can be employed to implement the operation of the depicted portion of the competitive event service 212. As an alternative, the flowchart of FIG. 4 can be viewed as depicting an example of elements of a method implemented within the network environment 203.

[0044] Beginning with block 403, the competitive event service 212 determines a skill level associated with a performance. For example, a competitive team may be performing a performance under a skill level 1. In various examples, the skill level being performed can be determined based at least in part on user interactions with a user interface 218 associated with the competitive event service 212 and rendered on a client device 206 and / or other display associated with the computing environment 203. At block 406, the competitive event service 212 selects a skill accuracy model 106 associated with the determined skill level.

[0045] At block 409, the competitive event service 212 obtains event content data 215 from the one or more sensors 103 configured to capture images and / or video content of the individuals performing the performance. At block 412, the competitive event service 212 generates input data based at least in part on the event content data 215. For example, the input data can include feature data associated with the event content data 215 obtained from the various sensors 103. The input data can include individual frames and / or frame segments of images and / or videos obtained from the sensors 103. Input data can include individual videos corresponding to each individual performer. At block 415, the competitive eventservice 212 applies the input data to the skill accuracy model 106. At block 418, the competitive event service 212 obtains an output 112 from the skill accuracy model 106.

[0046] At block 421, the competitive event service 212 determines if a classified skill is accurately performed based at least in part on the output 112 of the skill accuracy model 106. If the classified action is accurate, the competitive event service 212 proceeds to block 427. Otherwise, the competitive event service 212, at block 424, identifies the inaccuracies based at least in part on the output 112 of the skill accuracy model 106.

[0047] At block 427, the competitive event service 212 determines if the performance is complete. In some examples, a performance is a predefined period of time (e.g., 2.5 minutes). Therefore, if the predefined period of time has elapsed, the performance can be determined to be complete. If the performance is not complete, the competitive event service 212 returns to block 409. Otherwise, the competitive event service 212 proceeds to block 430.

[0048] At block 430, the competitive event service 212 generates an accuracy report 233 including the analytics associated with the performance and identified from the output 112 of the skill accuracy model. At block 433, the competitive event service 212 can transmit the accuracy report 233 to a client device 206 for rendering on a display 242 and / or render the accuracy report 233 on a display 242 associated with the computing environment 203. Thereafter, this portion of the process proceeds to completion.

[0049] Referring next to FIG. 5, shown is a flowchart that provides one example of the operation of a portion of the competitive event service 212. The flowchart of FIG. 5 provides merely an example of the many different types of functional arrangements that can be employed to implement the operation of the depicted portion of the competitive event service212. As an alternative, the flowchart of FIG. 5 can be viewed as depicting an example of elements of a method implemented within the network environment 203.

[0050] Beginning with block 503, the competitive event service 212 determines a skill level associated with a skill accuracy model 106 to be trained. At block 506, the competitive event service 212 obtains the training data based on the skill level. In various examples, the training data can include historical event content data 215 collected during prior competitive performances. At block 509, the competitive event service 212 generates feature inputs based at least in part on the training data. At block 512, the competitive event service 212 trains the skill accuracy model 205 using the feature inputs. Thereafter, this portion of the process proceeds to completion.

[0051] A number of software components previously discussed are stored in the memory of the respective computing devices and are executable by the processor of the respective computing devices. In this respect, the term "executable" means a program file that is in a form that can ultimately be run by the processor. Examples of executable programs can be a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of the memory and run by the processor, source code that can be expressed in proper format such as object code that is capable of being loaded into a random access portion of the memory and executed by the processor, or source code that can be interpreted by another executable program to generate instructions in a random access portion of the memory to be executed by the processor. An executable program can be stored in any portion or component of the memory, including random access memory (RAM), read-only memory (ROM), hard drive, solid-state drive, Universal Serial Bus (USB) flash drive,memory card, optical disc such as compact disc (CD) or digital versatile disc (DVD), floppy disk, magnetic tape, or other memory components.

[0052] The memory includes both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon a loss of power. Thus, the memory can include random access memory (RAM), read-only memory (ROM), hard disk drives, solid-state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical discs accessed via an optical disc drive, magnetic tapes accessed via an appropriate tape drive, or other memory components, or a combination of any two or more of these memory components. In addition, the RAM can include static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. The ROM can include a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other like memory device.

[0053] Although the applications and systems described herein can be embodied in software or code executed by general purpose hardware as discussed above, as an alternative the same can also be embodied in dedicated hardware or a combination of software / general purpose hardware and dedicated hardware. If embodied in dedicated hardware, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies can include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions upon an application of one or more data signals, application specific integrated circuits (ASICs) having appropriatelogic gates, field-programmable gate arrays (FPGAs), or other components, etc. Such technologies are generally well known by those skilled in the art and, consequently, are not described in detail herein.

[0054] The flowcharts show the functionality and operation of an implementation of portions of the various embodiments of the present disclosure. If embodied in software, each block can represent a module, segment, or portion of code that includes program instructions to implement the specified logical function(s). The program instructions can be embodied in the form of source code that includes human-readable statements written in a programming language or machine code that includes numerical instructions recognizable by a suitable execution system such as a processor in a computer system. The machine code can be converted from the source code through various processes. For example, the machine code can be generated from the source code with a compiler prior to execution of the corresponding application. As another example, the machine code can be generated from the source code concurrently with execution with an interpreter. Other approaches can also be used. If embodied in hardware, each block can represent a circuit or a number of interconnected circuits to implement the specified logical function or functions.

[0055] Although the flowcharts show a specific order of execution, it is understood that the order of execution can differ from that which is depicted. For example, the order of execution of two or more blocks can be scrambled relative to the order shown. Also, two or more blocks shown in succession can be executed concurrently or with partial concurrence. Further, in some embodiments, one or more of the blocks shown in the flowcharts can be skipped or omitted. In addition, any number of counters, state variables, warning semaphores, or messages might be added to the logical flow described herein, for purposes of enhancedutility, accounting, performance measurement, or providing troubleshooting aids, etc. It is understood that all such variations are within the scope of the present disclosure.

[0056] Also, any logic or application described herein that includes software or code can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as a processor in a computer system or other system. In this sense, the logic can include statements including instructions and declarations that can be fetched from the computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a "computer-readable medium" can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system. Moreover, a collection of distributed computer-readable media located across a plurality of computing devices (e.g., storage area networks or distributed or clustered filesystems or databases) may also be collectively considered as a single non-transitory computer-readable medium.

[0057] The computer-readable medium can include any one of many physical media such as magnetic, optical, or semiconductor media. More specific examples of a suitable computer- readable medium would include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, the computer-readable medium can be a random access memory (RAM) including static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other type of memory device.

[0058] Further, any logic or application described herein can be implemented and structured in a variety of ways. For example, one or more applications described can be implemented as modules or components of a single application. Further, one or more applications described herein can be executed in shared or separate computing devices or a combination thereof. For example, a plurality of the applications described herein can execute in the same computing device, or in multiple computing devices in the same computing environment 203.

[0059] Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., can be either X, Y, or Z, or any combination thereof (e.g., X; Y; Z; X or Y; X or Z; Y orZ; X, Y, orZ; etc.). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.

[0060] In addition to the forgoing, the various embodiments of the present disclosure include, but are not limited to, the embodiments set forth in the following clauses.

[0061] Clause 1 - A system, comprising a plurality of cameras positioned in a plurality of different positions surrounding a competitive event comprising a plurality of individuals performing a plurality of actions during a duration of the competitive event; a computing device comprising a processor and a memory; and machine-readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least determine a skill level associated with the competitive event; select a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level; obtain a plurality of images from the plurality of cameras, the plurality of images capturing theplurality of individuals performing at least one action of the plurality of actions; generate a plurality of inputs based at least in part on the plurality of images; apply the plurality of inputs to a skill accuracy model; and determine an accuracy associated with the at least one action of the plurality of actions based at least in part on an output of the skill accuracy model.

[0062] Clause 2 - The system of clause 1, wherein the skill level is one of a plurality of skill levels, each skill level requiring a performance of a different set of actions during the competitive event.

[0063] Clause 3 - The system of clause 2, wherein individual skill accuracy models of the plurality of skill accuracy models correspond to a respective skill level of the plurality of skill levels.

[0064] Clause 4 - The system of any of clauses 1-3, wherein the competitive event comprises a competitive cheerleading competition performance.

[0065] Clause 5 - The system of any of clauses 1-4, wherein the individuals perform the plurality of actions on a stage, a first camera being positioned about a front position relative to the stage, a second camera being positioned about a first back position relative to the stage, and a third camera being positioned about a second back position relative to the stage.

[0066] Clause 6 - The system of any of clauses 1-5, wherein the skill accuracy model is trained to analyze the plurality of inputs associated with the plurality of images to determine whether a threshold number of individuals of the plurality of individuals performed the action accurately.

[0067] Clause 7 - The system of any of clauses 1-6, wherein, when executed, the machine-readable instructions further cause the computing device to at least generate a reportdetailing a respective accuracy summary for each action performed of the plurality of actions; and transmit the report to a client device.

[0068] Clause 8 - The system of any of clauses 1-7, wherein, when executed, the machine-readable instructions further cause the computing device to at least obtain a plurality of training images associated with a plurality of other individuals performing the plurality of actions associated with the skill level; and train the skill accuracy model associated with the skill level based at least in part on the plurality of training images.

[0069] Clause 9 - The system of any of clauses 1-8, wherein the plurality of images corresponds to video content.

[0070] Clause 10 - The system of any of clauses 1-9, wherein a first subset of images from the plurality of images are obtained from a first camera of the plurality of cameras and a second subset of images are obtained from a second camera of the plurality of cameras.

[0071] Clause 11 - A method, comprising determining, by at least one computing device, a skill level associated with a competitive event comprising a plurality of individuals performing a plurality of actions during a duration of the competitive event; selecting, by the at least one computing device, a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level; obtaining, by the at least one computing device, a plurality of images from a plurality of cameras positioned in a plurality of different positions surrounding the competitive event, the plurality of images capturing the plurality of individuals performing at least one action of the plurality of actions; generating, by the at least one computing device, a plurality of inputs based at least in part on the plurality of images; applying, by the at least one computing device, the plurality of inputs to a skill accuracy model; and determining, by the at least one computing device, an accuracy associated withthe at least one action of the plurality of actions based at least in part on an output of the skill accuracy model.

[0072] Clause 12 - The method of clause 11, wherein the skill level is one of a plurality of skill levels, each skill level requiring a performance of a different set of actions during the competitive event.

[0073] Clause 13 - The method of clause 12, wherein individual skill accuracy models of the plurality of skill accuracy models correspond to a respective skill level of the plurality of skill levels.

[0074] Clause 14 - The method of any of clauses 11-13, wherein the competitive event comprises a competitive cheerleading competition performance.

[0075] Clause 15 - The method of any of clauses 11-14, wherein the individuals perform the plurality of actions on a stage, a first camera being positioned about a front position relative to the stage, a second camera being positioned about a first back position relative to the stage, and a third camera being positioned about a second back position relative to the stage.

[0076] Clause 16 - The method of any of clauses 11-15, wherein the skill accuracy model is trained to analyze the plurality of inputs associated with the plurality of images to determine whether a threshold number of individuals of the plurality of individuals performed the action accurately.

[0077] Clause 17 - The method of any of clauses 11-16, further comprising generating a report detailing a respective accuracy summary for each action performed of the plurality of actions; and transmitting the report to a client device.

[0078] Clause 18 - The method of any of clauses 11-17, further comprising obtaining a plurality of training images associated with a plurality of other individuals performing theplurality of actions associated with the skill level; and training the skill accuracy model associated with the skill level based at least in part on the plurality of training images.

[0079] Clause 19 - The method of any of clauses 11-18, wherein the plurality of images corresponds to video content.

[0080] Clause 20 - The method of any of clauses 11-19, wherein a first subset of images from the plurality of images are obtained from a first camera of the plurality of cameras and a second subset of images are obtained from a second camera of the plurality of cameras.

[0081] It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clear understanding of the principles of the disclosure. Many variations and modifications can be made to the above-described embodiments without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims

CLAIMSTherefore, the following is claimed:

1. A system, comprising: a plurality of cameras positioned in a plurality of different positions surrounding a competitive event comprising a plurality of individuals performing a plurality of actions during a duration of the competitive event; a computing device comprising a processor and a memory; and machine-readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least: determine a skill level associated with the competitive event; select a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level; obtain a plurality of images from the plurality of cameras, the plurality of images capturing the plurality of individuals performing at least one action of the plurality of actions; generate a plurality of inputs based at least in part on the plurality of images; apply the plurality of inputs to a skill accuracy model; and determine an accuracy associated with the at least one action of the plurality of actions based at least in part on an output of the skill accuracy model.

2. The system of claim 1, wherein the skill level is one of a plurality of skill levels, each skill level requiring a performance of a different set of actions during the competitive event.

3. The system of claim 2, wherein individual skill accuracy models of the plurality of skill accuracy models correspond to a respective skill level of the plurality of skill levels.

4. The system of any of claims 1-3, wherein the competitive event comprises a competitive cheerleading competition performance.

5. The system of any of claims 1-4, wherein the individuals perform the plurality of actions on a stage, a first camera being positioned about a front position relative to the stage, a second camera being positioned about a first back position relative to the stage, and a third camera being positioned about a second back position relative to the stage.

6. The system of any of claims 1-5, wherein the skill accuracy model is trained to analyze the plurality of inputs associated with the plurality of images to determine whether a threshold number of individuals of the plurality of individuals performed the action accurately.

7. The system of any of claims 1-6, wherein, when executed, the machine-readable instructions further cause the computing device to at least: generate a report detailing a respective accuracy summary for each action performed of the plurality of actions; and transmit the report to a client device.

8. The system of any of claims 1-7, wherein, when executed, the machine-readable instructions further cause the computing device to at least: obtain a plurality of training images associated with a plurality of other individuals performing the plurality of actions associated with the skill level; and train the skill accuracy model associated with the skill level based at least in part on the plurality of training images.

9. The system of any of claims 1-8, wherein the plurality of images corresponds to video content.

10. The system of any of claims 1-9, wherein a first subset of images from the plurality of images are obtained from a first camera of the plurality of cameras and a second subset of images are obtained from a second camera of the plurality of cameras.1 1. A method, comprising: determining, by at least one computing device, a skill level associated with a competitive event comprising a plurality of individuals performing a plurality of actions during a duration of the competitive event; selecting, by the at least one computing device, a skill accuracy model from a plurality of skill accuracy models based at least in part on the skill level; obtaining, by the at least one computing device, a plurality of images from a plurality of cameras positioned in a plurality of different positions surrounding the competitive event, the plurality of images capturing the plurality of individuals performing at least one action of the plurality of actions; generating, by the at least one computing device, a plurality of inputs based at least in part on the plurality of images; applying, by the at least one computing device, the plurality of inputs to a skill accuracy model; and determining, by the at least one computing device, an accuracy associated with the at least one action of the plurality of actions based at least in part on an output of the skill accuracy model.

12. The method of claim 11, wherein the skill level is one of a plurality of skill levels, each skill level requiring a performance of a different set of actions during the competitive event.

13. The method of claim 12, wherein individual skill accuracy models of the plurality of skill accuracy models correspond to a respective skill level of the plurality of skill levels.

14. The method of any of claims 11-13, wherein the competitive event comprises a competitive cheerleading competition performance.

15. The method of any of claims 11-14, wherein the individuals perform the plurality of actions on a stage, a first camera being positioned about a front position relative to the stage, a second camera being positioned about a first back position relative to the stage, and a third camera being positioned about a second back position relative to the stage.

16. The method of any of claims 11-15, wherein the skill accuracy model is trained to analyze the plurality of inputs associated with the plurality of images to determine whether a threshold number of individuals of the plurality of individuals performed the action accurately.

17. The method of any of claims 11-16, further comprising: generating a report detailing a respective accuracy summary for each action performed of the plurality of actions; and transmitting the report to a client device.

18. The method of any of claims 11 -17, further comprising: obtaining a plurality of training images associated with a plurality of other individuals performing the plurality of actions associated with the skill level; and training the skill accuracy model associated with the skill level based at least in part on the plurality of training images.

19. The method of any of claims 11-18, wherein the plurality of images corresponds to video content.

20. The method of any of claims 11-19, wherein a first subset of images from the plurality of images are obtained from a first camera of the plurality of cameras and a second subset of images are obtained from a second camera of the plurality of cameras.

Images