Sports training through artificial intelligence guidance on biomechanics and methods of use

Abstract

Sports Training through Artificial Intelligence (Al) Guidance on Biomechanics system and method is an interactive, individualized Al training loop that assesses and provides guidance on an individual's biomechanics during a sports activity. The Al system and method uses a sport's specific method of scoring, gaining an in-game advantage, or winning as the Al model's training result and returns the critical biomechanics and multi-frame human pose estimation attributes identified in the model as 'guidance' or instructions on optimizing their movement or sequence of movements to achieve the desired sports result.

Description

TITLESPORTS TRAINING THROUGH ARTIFICIAL INTELLIGENCE GUIDANCE ON BIOMECHANICS AND METHODS OF USEBACKGROUND

[0001] The invention generally relates to sports training, and more specifically, using artificial intelligence to provide biomechanical guidance.SUMMARY OF THE INVENTION

[0002] Provided herein are systems and methods for Sports Training through Artificial Intelligence Guidance on Biomechanics.

[0003] The methods, systems, and apparatuses are set forth in part in the description which follows, and in part will be obvious from the description, or can be learned by practice of the methods, apparatuses, and systems. The advantages of the methods, apparatuses, and systems will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the methods, apparatuses, and systems, as claimed.

[0004] Accordingly, it is an object of the invention not to encompass within the invention any previously known product, process of making the product, or method of using the product such that Applicants reserve the right and hereby disclose a disclaimer of any previously known product, process, or method. It is further noted that the invention does not intend to encompass within the scope of the invention any product, process, or making of the product or method of using the product, which does not meet the written description and enablement requirements of the USPTO (35 U.S.C. § 112, first paragraph) or the EPO (Article 83 of the EPC), such that Applicants reserve the right and hereby disclose a disclaimer of any previously described product, process of making the product, or method of using the product. It may be advantageous in the practice of the invention to be in compliance with Art. 53(c) EPC and Rule 28(b) and (c) EPC. All rights to explicitly disclaim any embodiments that are the subject of any granted patent(s) of applicant in the lineage of this application or in any other lineage or in any prior filed application of any third party is explicitly reserved. Nothing herein is to be construed as a promise.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In the accompanying figures, like elements are identified by like reference numerals among the several preferred embodiments of the present invention.

[0006] FIG. 1 is a schematic flow chart of the Sports Training through Artificial Intelligence Guidance on Biomechanics.

[0007] FIG. 2 is a block logical architecture diagram showing an embodiment of an Al system and method in accordance with some of the present techniques.

[0008] FIG. 3 is a schematic of a computing device by which the present techniques may be implemented.DETAILED DESCRIPTION OF THE INVENTION

[0009] The foregoing and other features and advantages of the invention are apparent from the following detailed description of exemplary embodiments, read in conjunction with the accompanying drawings. The detailed description and drawings are merely illustrative of the invention rather than limiting, the scope of the invention being defined by the appended claims and equivalents thereof.

[0010] Embodiments of the invention will now be described with reference to the Figures, wherein like numerals reflect like elements throughout. The terminology used in the description presented herein is not intended to be interpreted in any limited or restrictive way, simply because it is being utilized in conjunction with detailed description of certain specific embodiments of the invention. Furthermore, embodiments of the invention may include several novel features, no single one of which is solely responsible for its desirable attributes or which is essential to practicing the invention described herein.

[0011] The words proximal and distal are applied herein to denote specific ends of components of the instrument described herein. A proximal end refers to the end of an instrument nearer to an operator of the instrument when the instrument is being used. A distal end refers to the end of a component further from the operator and extending towards the surgical area of a patient and / or the implant.

[0012] The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. It will be further understood that the terms “comprises,” “comprising,” “includes,” and / or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do notpreclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0013] Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. The word “about,” when accompanying a numerical value, is to be construed as indicating a deviation of up to and inclusive of 10% from the stated numerical value. The use of any and all examples, or exemplary language (“e.g.” or “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any nonclaimed element as essential to the practice of the invention.

[0014] References to “one embodiment,” “an embodiment,” “example embodiment,” “various embodiments,” etc., may indicate that the embodiment(s) of the invention so described may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Further, repeated use of the phrase “in one embodiment,” or “in an exemplary embodiment,” do not necessarily refer to the same embodiment, although they may.

[0015] As used herein the term “method” refers to manners, means, techniques and procedures for accomplishing a given task including, but not limited to, those manners, means, techniques and procedures either known to, or readily developed from known manners, means, techniques and procedures by practitioners of the mechanical, software, electrical, and electromechanical arts. Unless otherwise expressly stated, it is in no way intended that any method or aspect set forth herein be construed as requiring that its steps be performed in a specific order. Accordingly, where a method claim does not specifically state in the claims or descriptions that the steps are to be limited to a specific order, it is no way intended that an order be inferred, in any respect. This holds for any possible non-express basis for interpretation, including matters of logic with respect to arrangement of steps or operational flow, plain meaning derived from grammatical organization or punctuation, or the number or type of aspects described in the specification.

[0016] Description of Embodiments

[0017] Generally speaking, the Sports Training through Artificial Intelligence Guidance on Biomechanics system and method is an interactive, individualized Al training loop that assessesand provides guidance on an individual’s biomechanics during a sports activity. As shown in FIG. 1, the Al system and method 100 uses a sport’s specific method of scoring 130, gaining an ingame advantage, or winning as the Al model’s training result and returns the critical biomechanics and multi-frame human pose estimation attributes 120 identified in the model as ‘guidance’ or instructions on optimizing their movement or sequence of movements to achieve the desired sports result 140. One example would be a basketball player shooting the ball, where the model's training result would be whether the shot went in or not. Another example would be a quarterback throwing the football, where the training result would be the ball reaching the receiver in a catchable zone relative to their body.

[0018] As shown in FIG. 1, the Al system and method 100 is trained by inputs into the model 110, which would come from the various human pose estimation attributes at various stages of the subject’s movement 120. In one embodiment, the human pose estimation attributes at various stages of the subject’s movement are collected by a non-wearable sensor such as a camera or wearable sensors on the user. The inputs would also be linked to the level of interaction (if any) between the device with the sensor (such as a robotic arm designed to block a basketball shot) and the subject performing the measurable act (such as shooting a basketball at the basketball goal).

[0001] The multi-frame human pose estimation incorporates 1 or more of the following model types of method of scoring movement: Kinematic, Volumetric. In some embodiments, the Al system and method 100 may be used in various sports such as soccer (e.g. goalkeeper), football (blocking kicks or passes), volleyball (e.g. defender on the net), tennis, badminton, etc. for training purposes.

[0019] The individual Al model receiving & assessing the human pose attributes will be a deeplearning model, such as a neural network, that may be aided by insights gained from algorithms and / or basic and advanced statistical methods such as Hidden Markov or regression. This model will also be aided by the collaborative guidance fine-tuning model 150 on the central platform, as shown in FIG. 1.

[0020] The individualized models are aggregated on a central platform and utilized to initially generate and ultimately speed up the time to “first guidance” for newer subjects. The selection of “origin models” to seed a new individual’s model is based on comparing critical biomechanics identified for the given sport activity or initial similarities in biomechanics.[0021 J In one embodiment, the Al model responsible for fine-tuning guidance based on postguidance results across multiple subjects will follow the pattern of reinforcement learning.

[0022] In one embodiment, the mathematical approaches include, but not limited to, Hidden Markov models may be used to fine-tune the guidance as well.

[0023] In one embodiment, the guidance provided by the models and accepted by the subject is tracked for impact in regard to the original biomechanical attributes that are being targeted for adjustment. Over time, this allows for a much higher success rate in initial guidance / recommendations.

[0024] Corrections to the perceived training results can be made by the subject or somebody associated with the subject or provider.

[0025] Defender

[0026] In one embodiment, a robotic defender 10 may include a body 12, a camera / sensor system 14, an external camera / sensor 18, and a controller 16 to implement the Al system and method 100, as shown in Figure 2.

[0027] In some embodiments, the body 12 may be made of various impact-resistant materials, such as polymers (e.g., plastic, rubber, etc.), woods, metals, and a combination thereof. In some embodiments, the shape and dimensions of the body are approximately match those of a basketball player (e.g. an adult basketball player or a high school basketball player).

[0028] In some embodiments, the body 12 may include a first arm and a second arm, symmetrically disposed on opposing sides. The first and the second arms may be elongate rigid or extendable. The arms may have dimensions similar to human arms (e.g., between about 2 and about 4 feet), to simulate the real game conditions, or bigger dimensions to simulate a harder than real game conditions for training purposes. The arms may have the same length, or different length (e.g. one arm fully extended and one arm only partially extended). In some embodiments, the length of each arm is adjustable. The length of each of the arms may vary from twenty to forty inches. The arms’ movements may be in unison or independently, with speed and timing of each arm being adjustable.

[0029] In some embodiments, arms may extend at a pre-determined noncontact angle above the horizontal (e.g. parallel to the floor) to simulate posture of a defender in a real-game condition. The arms angle may be adjusted between about 30 to about 90 degrees from a plane parallel to the floor. In some embodiments, arms may have elbow joints that can provide means of movementssimilar to the movements of a defender forearms in real-game condition. The arms may have shoulder joints that can provide means of movements to raise or lower the arms. In some cases, the arms have three degrees of freedom each (elbow movement and shoulder rotation about two axes) and a corresponding number of actuators.

[0030] In some embodiments, the body 12 may have a vertical movement degree of freedom (e.g., a telescoping core) and corresponding actuator to simulate jumping of a defender in the real-game condition. The body may also have an adjustable height. For example, the height of the robotic defender 10 may be adjusted to match the height of a defender that a player, who is the subject of the training, will be facing in the future. In some embodiments, the height of the robotic defender may be adjusted to simulate basketball defenders of various heights, arm reach, and jumping abilities. The height of the robotic defender may be adjusted by means of frictional adjustment screws. The height of the robotic defender may be adjusted from 5 feet to 8 feet to challenge various sizes of opponents on the court.|0031] In some embodiments, the body 12 may be moveable and adapted to rest on a basketball court floor in a position to withstand contact by a basketball player. The body 12 may have sufficient weight to provide stability to the robotic defender 10 when robotic defender 10 is bumped by a player or hit by a ball during use.

[0032] In some embodiments, the body 12 may be configured to rest at a specific position on basketball floor. In some embodiments, the body 12 may have movement actuators (e.g., wheels or rollers coupled to motors) to simulate the movements of a defender in real-game condition. The body may be configured to move along a line in one degree of freedom or in a plane with two degrees of freedom. Support for movement may also allow the robotic defender to be easily rolled on or off the court or repositioned on the court as desired during a training session.

[0033] In some embodiments, an actuating system may include a structure that links to the body 12 and has an actuator for moving the parts of the body. The actuating system may include electric motors and gearboxes, solenoids, pneumatic actuators, hydraulic actuators or a combination thereof. Structural materials employed may be metal, plastic, carbon fiber or wood, while cosmetic outer coverings can comprise vinyl, rubber or plastic.

[0034] In some embodiments, the controller 16 of the robotic defender 10 may be configured to prioritize and localize Objects of Interest (e.g., a basketball player) in their surroundings and move in response to the location or trajectory of those Objects of Interest, in some cases, withoutrequiring an input from an operator. The controller 16 also assesses and provides guidance on an individual’s biomechanics during a sports activity as input from the sensor suite.

[0035] In some embodiments, the controller 16 is coupled to a sensor suite that includes a camera 14, a depth camera, ultrasonic sensors, infrared transmitters and receiver, contact sensors, audio sensors, wearable sensors 18, and the like. The controller may be configured to calculate the relative distance between Objects of Interest and the robotic defender 10 assesses and provides guidance on an individual’s biomechanics during a sports activity. Based on the determined distance, the controller may choose a defending strategy (e.g. blocking jumping, extending arms, etc.). In some embodiments, the controller may store data from the sensor suite or metrics computed based on such data. In other embodiments, the sensor suite may operate as a computer vision system. The sensor suite may sensor the position and location of a ball and may operate to avoid the ball or defend the ball.

[0036] In some embodiments, the robotic defender may be configured to autonomously position itself relative to an Object of Interest (e.g., a shooter.) In some embodiment, the robotic defender may be set at a fix position on the floor (e.g., a basketball court) and it may move only vertically (e.g., simulating jumping or extending arms) to block the Objects of Interest (e.g., the shooter or the ball.) In some embodiments, the robotic defender may be configured to move on the floor with one degree of freedom (e.g. moving along a line). In some embodiments, the robotic defender may be configured to move on the floor with two degrees of freedom (e.g. moving on an x-y plane parallel to the floor plane). In some embodiments, the robotic defender may be configured to move with 3 degrees of freedom (e.g. moving on the floor and extending vertically to block a shooter or rotating about the vertical axis).

[0037] In some embodiments, the controller may be configured to process data (e.g. data from the sensor suite) and determine the number and the location of Object(s) of Interest relative to the robotic defender. In some embodiments, the controller may process the received data at a rapid enough pace to capture the trajectory of fast-moving Object(s) of Interest. In some embodiments, the controller may perform data association tasks by using machine learning techniques, including techniques to predict object positions and movements (e.g., studying the movement of a basketball player to predict how fast a player can catch a ball and take a shot).

[0038] In some embodiments, machine learning techniques may be used to predict the behavior of a shooter (e.g., timing, jumping height, speed, extension of arms, form, body posture, etc.) toincrease the chance of the robotic defender successfully blocking the shot. Tn some embodiments, a machine learning model may be tuned for simulating the defending (e.g. blocking) skills of a specific player. For example, if a user wants to train specifically for a defender whom the user will be facing in his next match, the user can setup the robotic defender to simulate the movements and blocking postures of that defender.

[0039] In some embodiments, the controller may be configured to rank detected Objects of Interest based on their class (e.g. player with the ball, player closer to the robot, player in better position to take a shot, etc.). In some embodiments, the controller may rank the Objects of Interest based on their distance from the robotic defender. Based on these rankings the controller may identify a singular object of interest to be the highest priority. For example, the player in possession of the ball may be determined as the highest priority object of interest and a player located as the closest player or best position to take a shot may be associated as the second highest ranking Object of Interest.|0040] In some embodiments, the controller may include a plurality of sensors that can monitor the surrounding area (e.g. motion sensors and cameras) and specifically target a user (e.g. basketball player) as the Object of Interest. In some embodiments, the sensors may be configured to determine the moment that the player will shoot. Sensors may be focused on the movements of the player in possession of the ball. For example, the sensors may be tracking the player’s eyes, knees, arms, or other body parts to monitor when the player will shoot.

[0041] The sensors of the robotic defender may monitor the movements of the player to determine when to attempt blocking the player’s shot, in the non-extended format (e.g., simulating a basketball player in the defending position), shown in Figure 2B. Once the robotic defender detects that the shooter is about to take a shot, the robotic defender may transform into the extended mode (e.g. extended the amis or increasing the height) to simulate defending behavior of a basketball player in the defending position,

[0042] Sensor suite

[0043] Various aspects of the described logic may be implemented with a computing device like that shown in Figure 3 and described below.

[0044] Some embodiments may afford the Al system and method 100 with a relatively robust sensor suite, relatively large numbers of degrees of freedom, and machine learning capabilities to track player pose and ball movement and assesses and provides guidance on an individual’sbiomechanics during a sports activity. Some embodiments may include a remote server storing profiles of defensive players (such as profiles with configuration parameters that tune robot behavior to match that of various professional basketball players identified in association with the profiles, such that users can select a given professional basketball player as a defender to be loaded into a model controlling operation of the robotic defender over the Internet from the remote server). In some embodiments, the remote server may further receive, via the Internet, metrics indicating performance of a player practicing with the robotic defender, for example, shot locations, shot percentages, block percentages, and various biomechanical metrics or recordings of the player taking shots.

[0045] In some embodiments, the system may include a sensor suite located local to a facility where the robotic shot blocker is being used, with sensors being located on the robotic shot blocker itself or deployed in the same room and communicating via a local area network connected to an access point configured to communicate with the remote server. Examples of such sensors include stereoscopic optical cameras, for instance located at head height in positions analogous to the eyes of a player on the robotic shot blocker, as well as oriented in the opposite direction to capture video of the backboard of a basket to track whether shots are made. Some embodiments may include arrays of more than two stereoscopic cameras, such as optical video cameras to obtain higher fidelity depth information. Some embodiments may include other cameras, like video cameras positioned off the robot, for instance with a substantial portion, like more than 1 / 4 or more than half of the court including the robotic shot defender and the player in their field-of-view to capture video suitable for characterizing biomechanics of the shots being taken, for instance, from the sides and behind the player. The biomechanics are characterized for the ideal form of the subject’s body for shooting the basketball, which may be analyzed for training purposes. In some embodiments, the video cameras may have a relatively high frame rate to capture relatively fast movements of the player, for instance, greater than about 50 or greater than about 100 frames per second. Some embodiments may further include depth sensing cameras, for example, those configured to transmit structured light patterns onto a scene and capture frames of video depicting position of the structured light to infer depth, lidar sensors, ultrasonic sensors, time-of-flight sensors, and the like. Some embodiments may further include a floormat with a matrix of pressure sensors to capture pressure readings and locations from the player’s feet as they move and go up for a shot as an additional channels of information to characterize biomechanics and detect shots. Othersensors include audio sensors to detect sliding shoes and contact sensors to detect when contact is made with the ball or player. The controller is configured to trigger the actuating system to rotate and extend the first arm and the second position the hand portion in the extended position to automatically block a sports player in response to a sensor input, wherein the sensor input is a motion by the sports player.

[0046] Output from the sensors may be used by a controller (e.g. and obtained via a CAN bus or local area network) described below to engage various actuators of the robotic shot defender. Examples include a lateral movement actuator, such as a side to side or back-to-back actuator, like a set of wheels with a servo or stepper motor drive. Some embodiments may be configured to implement more degrees of freedom in lateral movement of the robot within the frame of reference of a court. For example, some embodiments may implement two or three degrees of freedom of movement of the robotic shot defender with mecanum and wheels or omni wheels controlled by a set of corresponding drives, like pneumatic drives, stepper motors, servomotors, or the like. In some embodiments, the actuator and wheels may be mounted to a chassis adjacent the floor that supports the other components of the robotic defender above the chassis.

[0047] In some embodiments, other components may include a robot core, extending upwards from the chassis, analogous to the core of the human body to which movable arms are mounted and stereoscopic cameras and other sensors are mounted. In some embodiments, the robot core may have one or more degrees of freedom relative to the chassis, for instance one, two, or three of freedom, such as corresponding to leaning forward, leaning sideways, and twisting about the vertical axis, and some embodiments may be further configured to translate vertically up and down relative to the chassis to mimic a player jumping to block a shot.

[0048] In some embodiments, actuators for one or two arms may be mounted to the core, in some cases with the arms having one, two, or three degrees of freedom relative to the core. In some embodiments, the core and arm actuators may be driven by pneumatic motors, stepper motors, servomotors, or the like. In some embodiments, movement may be actuated by a resilient member, like a spring or an elastomeric band that is released after having been placed in a stretched configuration by a slower-acting electrical or pneumatic actuator to obtain relatively fast, powerful movements with motors that are less expensive than those that would support such movements directly. For instance, the resilient members may be stressed over a first duration of time between shots, and then released over a second, much shorter duration of time, to block a shot with a fastmovement.

[0049] In some embodiments, the sensor suite and the actuators may couple to a controller, which in some cases may be entirely onboard the rope robotic defender, may be in the same room as the robotic defender, or may be remote, for instance at a remote server, or in some embodiments, various components may be distributed among these various locations for computing equipment. In some embodiments, relatively latency sensitive computation may be located closer to the robotic defender, for instance, on board the robotic defender for things like shot detection to trigger a movement of the actuators to block a shot, while other types of computation may be more compute intensive and may be distributed remotely, for instance, analysis of various biomechanics over a collection of samples of time-stamp synchronized channels of sensor data corresponding to different shots. In some embodiments, the controller includes a Linux-based or Windows-based computer executing various processes described below to control the actuators responsive to these channels of information from the sensor suite.|0050] In some embodiments, the controller includes a human pose estimator which may ingest various (e.g., all) channels of sensor information including video data and infer a pose of the player practicing shots in real time, for instance with less than hundred milliseconds, less than 50 ms, or less than 5 ms of latency). In some embodiments, the human pose estimator may implement a skeleton-based model, contour-based model, or a volume-based model of the player. In some embodiments, the model may further include a relative position of the player to the robotic shot defender. In some embodiments, inferred pose of the player may be updated relatively frequently, for instance consistence with the latency examples discussed above, and in some cases, updates may be based on a previous pose, e.g., as an input feature to the model. In some embodiments, the human pose estimator may ingest multiple channels of information from the sensor suite in addition to an array of cameras, for example including channels indicative of depth, pressure on a floormat with the pressure sensor matrix, ultrasonic readings indicative of distance, lidar readings, and the like.

[0051] In some embodiments, the human pose estimator may implement a deep convolutional neural network (for instance having 5 or more, 15 or more, or 40 or more layers of perceptrons). In some embodiments, the human pose estimator may implement temporal depth-separable convolutions, for instance, like those implemented in MobileNet version 3 (as described by Howard et al, in “Searching for MobileNetV3,” arXiv: 1905.02244, the contents of which arehereby incorporated by reference), to expedite operations with fewer computing resources used in non-depth separable convolutions, potentially at the expense of a relatively small loss of accuracy. In some embodiments, the human pose estimator may implement the techniques described by Pavillo et al, in “3D human pose estimation in video with temporal convolutions and semisupervised training,” arXiv;1811.11742, the contents of which are hereby incorporated by reference. In some embodiments, the human pose estimator may run on a computer vision hardware accelerator, like an edge tensor processing unit, or the like to obtain more performance.

[0052] Some embodiments may further include a ball tracker model, for instance, implemented with the same class of neural networks, drawing from the same sensor suite, to track position of a ball relative to the robotic defender or in a coordinate system of the court. Embodiments may include a deep convolutional neural network operative to perform object detection and localization of the ball in frames of video from multiple cameras and translate such detections from various cameras at different poses into a position of the ball in the room.|0053] In operation, the controller may detect shots and engage the actuators, for instance, moving up the core and the arms and laterally positioning the robotic defender, to attempt to block the shot. In some embodiments, these movements may be tuned based upon profiles of professional basketball players downloaded from the remote server, for instance, with some professional basketball players having a higher reach but a slower response time or vice versa or some professional black basketball players favoring their left arm with her right arm to block shots. Other examples include distance between the player and the shot blocker robot that may depend upon player profile. In some embodiments, the human pose estimator and ball tracker may feed into a shot classifier, for instance, an ensemble model, that infers whether the player is taking a shot based upon things like current and past estimations in human pose (like gaze orientation, head position, knees position, and movement of the player center of body and arms) and movement of the ball (like being held in both hands momentarily and moving upward well in contact with a human’s hand). Upon detecting a shot, some embodiments may actuate a shot block maneuver. In some embodiments, the professional basketball player profiles may further include tuning parameters based upon how quickly professional basketball players are to detect that a shot is being taken, with some responding more slowly than others. In some embodiments, these profiles may be learned by training a human pose estimator and ball tracker on video of professional basketball games.

[0054] Other operations of the controller may include positioning the robotic shot defender on the court to block a shot. Some embodiments may compute a difference in position and orientation of the robotic shot blocker and the player (in some cases modeling a future position of the player and computing a delta as well to be controlled) and adjust the that difference to position the robotic shot blocker to mimic the behavior of a professional basketball player or otherwise attempt to block the shot. In some embodiments, contact sensors on the robotic shot blocker may detect when a player makes contact and disengage all movements with the lateral movement actuators. Some embodiments may implement a controller, like a proportional integral differential controller, to attempt to maintain an optimal relative distance of the robotic shot blocker in the player to determine how to actuate the lateral movement actuators. In some embodiments, core actuators may also be engaged as part of lateral movements, for instance leaning into a movement in a manner similar to how a human does. For example, some embodiments may model and control the core (which is core in the sense of a human body core, not in the sense of being required in all instances) as an inverted pendulum and position the core to reduce torque as the chassis accelerates.

[0055] Some embodiments may classify various movements of the player. Examples include detecting and classifying believability of head fakes of the player. Some embodiments may score a trailing set of movements of the player based upon likelihood of the player being about to engage in a shot, and some embodiments may detect missed classifications of shots or near miss classifications as head fakes (e.g., upon a shot-classification score exceeding a threshold) and provide a score based upon the believability of the head fakes for the score player to store in their profile and displayed to the player.

[0056] Some embodiments may classify various metrics of biomechanics of shots of the player, like time to get a shot off from the time at which the player shot movement began, elbow position at various stages of shot, head position, balance, jumping height, knee position through various movements of the shot, and correlations therebetween. Some embodiments may compute a deviation from ideal form and score biomechanics based upon an aggregate measure of deviation or detect particular problematic areas. Examples include an aggregate measure of deviation across a plurality of joints, or metrics of max deviation among such a set. Some embodiments may store these results and corresponding video in the player profile, and in some cases, the player may access these profiles, for instance during rafter practice section to see a video overlay of the actual shot, for instance from a camera from the side of a court overlaid with a transparent model showingwhat an optimal biomechanics movement would look like in that shot.

[0057] In some embodiments, the controller may further implement various user interfaces related to gestures. For example, in some embodiments may renew a blocking mode or change profiles responsive to the player slapping a ball to change modes, for instance, from a non-shock blocking mode to a shot blocking mode or the player raising their hand and waving at left to right to indicate that the shot blocker should respond faster be more aggressive or less aggressive.

[0058] In some embodiments, the controller may analyze shots of the player and provide feedback. In some embodiments, the feedback may be provided to the player after each shot to improve the player’s skill at making shots by identifying the strengths and weaknesses of the player, including the elbow position at various stages of shot, head position, balance, jumping height, knee position through various movements of the shot, and correlations therebetween. In some embodiments, the feedback may be provided after a series of shots (e.g. after about 15 minutes of training or after a full training session) to analyze the performance of the player (speed, form, accuracy, etc.) The analysis may include qualitative (e.g. too high or too low) and quantitative (e.g. height of the jump, angle of the elbow, etc.) feedback.

[0059] In some embodiments, the controller may provide some statistics after a training session to the player. Such statistics may include the number of shots, the location of each shot, number of makes, misses, clean shots, close misses, progress during the training session, average jumping height, speed, angle, number of running shots, lay-up shots, jump shots, bank shots, hook shots, set-shots, free throws, etc.

[0060] In some embodiments, the feedback may be comminuted to the player during a training session via various means, including an interface (e.g. a monitor) showing the feedback, an audio system, and data reported to the coach. In some cases, players may pay a subscription to obtain feedback and coaching via a native application on their mobile device or on the defender. Examples of feedback are described herein and include how high the player is jumping, whether they are jumping right or left, forward or backward, whether the player is leaning when shooting and a direction, etc., in some cases shown with a delta relative to an ideal form. Feedback may also include shot positions, where shots were missed and made, total shotes, heatmaps of shot percentages on the court, etc. In some cases, user interfaces may include speakers and video of the shots may be captured with an application on the user’s mobile device by which a subscription is managed. In some cases, the UI includes lasers that indicate where on the floor the user’s feetshould be positioned for a shot. The laser or projectors project a laser line or optical markings on the floor that are used to help align the shooter and for safety. This laser line or optical markings shows the shooter to try not to cross so they do not contact the arm portion, and for computer vision processing. The laser or projector may be mounted in the base or may be on the body portion.

[0061] The present techniques have been described with reference to basketball, but the robotic shot blocker is expected to have application in a variety of other forms of fitness and sports, examples including volleyball, soccer, football, badminton, tennis, pickle ball, and the like.

[0062] In some embodiments, various types of robots may be used to attain relatively fast movement of a member configured to block shots, like a hand-shaped end effector. In some cases, such an end effector may be mounted to five-bar robot or scissor, having two motors controlling two degrees of freedom (e.g., in a plane that bisects a line from the player to the basket). Angular position of the motors may specify a location of the end effector. As described above, a camera may be used to track the ball and position the end effector to block a shot. In some cases, the end effector and arms may be made of a relatively lightweight material, like carbon fiber composites to facilitate fast movement. In some cases, only a right arm or a left arm (having a mechanical movement like that of a human arm) may be mounted to the defender to favor passing on the other side. Or embodiments may have arms on both sides.

[0063] Figure 5 is a diagram that illustrates an exemplary computing system 1000 by which Al models and embodiments of the present technique may be implemented. Various portions of systems and methods described herein, may include or be executed on one or more computer systems similar to computing system 1000. Further, Al models, processes, and modules described herein may be executed by one or more processing systems similar to that of computing system 1000.

[0064] Computing system 1000 may include one or more processors (e.g., processors 1010a- lOlOn) coupled to system memory 1020, an input / output I / O device interface 1030, and a network interface 1040 via an input / output (I / O) interface 1050. A processor may include a single processor or a plurality of processors (e.g., distributed processors). A processor may be any suitable processor capable of executing or otherwise performing instructions. A processor may include a central processing unit (CPU) that carries out program instructions to perform the arithmetical, logical, and input / output operations of computing system 1000. A processor may execute code(e.g., processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof) that creates an execution environment for program instructions. A processor may include a programmable processor. A processor may include general or special purpose microprocessors. A processor may receive instructions and data from a memory (e.g., system memory 1020). Computing system 1000 may be a uni-processor system including one processor (e.g., processor 1010a), or a multi-processor system including any number of suitable processors (e.g., lOlOa-lOlOn). Multiple processors may be employed to provide for parallel or sequential execution of one or more portions of the techniques described herein. Processes, such as logic flows, described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating corresponding output. Processes described herein may be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Computing system 1000 may include a plurality of computing devices (e.g., distributed computer systems) to implement various processing functions.

[0065] I / O device interface 1030 may provide an interface for connection of one or more I / O devices 1060 to computer system 1000. I / O devices may include devices that receive input (e.g., from a user) or output information (e.g., to a user). I / O devices 1060 may include, for example, graphical user interface presented on displays (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor), pointing devices (e.g., a computer mouse or trackball), keyboards, keypads, touchpads, scanning devices, voice recognition devices, gesture recognition devices, printers, audio speakers, microphones, cameras, or the like. I / O devices 1060 may be connected to computer system 1000 through a wired or wireless connection. I / O devices 1060 may be connected to computer system 1000 from a remote location. I / O devices 1060 located on remote computer system, for example, may be connected to computer system 1000 via a network and network interface 1040.

[0066] Network interface 1040 may include a network adapter that provides for connection of computer system 1000 to a network. Network interface may 1040 may facilitate data exchange between computer system 1000 and other devices connected to the network. Network interface 1040 may support wired or wireless communication. The network may include an electronic communication network, such as the Internet, a local area network (LAN), a wide area network(WAN), a cellular communications network, or the like.

[0067] System memory 1020 may be configured to store program instructions 1100 or data 1110. Program instructions 1100 may be executable by a processor (e.g., one or more of processors lOlOa-lOlOn) to implement one or more embodiments of the present techniques. Instructions 1100 may include modules of computer program instructions for implementing one or more techniques described herein with regard to various processing modules. Program instructions may include a computer program (which in certain forms is known as a program, software, software application, script, or code). A computer program may be written in a programming language, including compiled or interpreted languages, or declarative or procedural languages. A computer program may include a unit suitable for use in a computing environment, including as a stand-alone program, a module, a component, or a subroutine. A computer program may or may not correspond to a fde in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program may be deployed to be executed on one or more computer processors located locally at one site or distributed across multiple remote sites and interconnected by a communication network.

[0068] System memory 1020 may include a tangible program carrier having program instructions stored thereon. A tangible program carrier may include a non-transitory computer readable storage medium. A non-transitory computer readable storage medium may include a machine readable storage device, a machine readable storage substrate, a memory device, or any combination thereof. Non-transitory computer readable storage medium may include non-volatile memory (e.g., flash memory, ROM, PROM, EPROM, EEPROM memory), volatile memory (e.g., random access memory (RAM), static random access memory (SRAM), synchronous dynamic RAM (SDRAM)), bulk storage memory (e.g., CD-ROM and / or DVD-ROM, hard-drives), or the like. System memory 1020 may include a non-transitory computer readable storage medium that may have program instructions stored thereon that are executable by a computer processor (e.g., one or more of processors lOlOa-lOlOn) to cause the subject matter and the functional operations described herein. A memory (e.g., system memory 1020) may include a single memory device and / or a plurality of memory devices (e.g., distributed memory devices). Instructions or other program code to provide the functionality described herein may be stored on a tangible, non-transitory computer readable media. In some cases, the entire set of instructions may be stored concurrently on the media, or in some cases, different parts of the instructions may be stored on the same media at different times.

[0069] I / O interface 1050 may be configured to coordinate I / O traffic between processors 1010a- lOlOn, system memory 1020, network interface 1040, I / O devices 1060, and / or other peripheral devices. I / O interface 1050 may perform protocol, timing, or other data transformations to convert data signals from one component (e.g., system memory 1020) into a format suitable for use by another component (e.g., processors lOlOa-lOlOn). I / O interface 1050 may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard.

[0070] Embodiments of the techniques described herein may be implemented using a single instance of computer system 1000 or multiple computer systems 1000 configured to host different portions or instances of embodiments. Multiple computer systems 1000 may provide for parallel or sequential processing / execution of one or more portions of the techniques described herein.

[0071] Those skilled in the art will appreciate that computer system 1000 is merely illustrative and is not intended to limit the scope of the techniques described herein. Computer system 1000 may include any combination of devices or software that may perform or otherwise provide for the performance of the techniques described herein. For example, computer system 1000 may include or be a combination of a cloud-computing system, a data center, a server rack, a server, a virtual server, a desktop computer, a laptop computer, a tablet computer, a server device, a client device, a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a vehicle-mounted computer, or a Global Positioning System (GPS), or the like. Computer system 1000 may also be connected to other devices that are not illustrated, or may operate as a stand-alone system. In addition, the functionality provided by the illustrated components may in some embodiments be combined in fewer components or distributed in additional components. Similarly, in some embodiments, the functionality of some of the illustrated components may not be provided or other additional functionality may be available.

[0072] Those skilled in the art will also appreciate that while various items are illustrated as being stored in memory or on storage while being used, these items or portions of them may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments some or all of the software components mayexecute in memory on another device and communicate with the illustrated computer system via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or a portable article to be read by an appropriate drive, various examples of which are described above. In some embodiments, instructions stored on a computer-accessible medium separate from computer system 1000 may be transmitted to computer system 1000 via transmission media or signals such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as a network or a wireless link. Various embodiments may further include receiving, sending, or storing instructions or data implemented in accordance with the foregoing description upon a computer- accessible medium. Accordingly, the present techniques may be practiced with other computer system configurations.[0073 | In block diagrams, illustrated components are depicted as discrete functional blocks, but embodiments are not limited to systems in which the functionality described herein is organized as illustrated. The functionality provided by each of the components may be provided by software or hardware modules that are differently organized than is presently depicted, for example such software or hardware may be intermingled, conjoined, replicated, broken up, distributed (e.g. within a data center or geographically), or otherwise differently organized. The functionality described herein may be provided by one or more processors of one or more computers executing code stored on a tangible, non-transitory, machine readable medium. In some cases, notwithstanding use of the singular term “medium,” the instructions may be distributed on different storage devices associated with different computing devices, for instance, with each computing device having a different subset of the instructions, an implementation consistent with usage of the singular term “medium” herein. In some cases, third party content delivery networks may host some or all of the information conveyed over networks, in which case, to the extent information (e.g., content) is said to be supplied or otherwise provided, the information may provided by sending instructions to retrieve that information from a content delivery network.

[0074] All publications and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication or patent application was specifically and individually indicated to be incorporated by reference.

[0075] While the invention has been described in connection with various embodiments, it will be understood that the invention is capable of further modifications. This application is intended tocover any variations, uses or adaptations of the invention following, in general, the principles of the invention, and including such departures from the present disclosure as, within the known and customary practice within the art to which the invention pertains.

Claims

CLAIMSWhat is claimed is:

1. A Sports Training through Artificial Intelligence Guidance on Biomechanics method, comprising: a. an Al training loop model that assesses and provides guidance on an individual’s biomechanics during a sports activity; b. a sport’s specific method of scoring and training the Al training loop model; c. returning the critical biomechanics and multi-frame human pose estimation attributes identified in the Al training loop model; and d. providing instructions on optimizing their movement or sequence of movements to achieve the desired sports result.

2. The method of Claim 1 , wherein the sports activity is basketball and the Al model's method of scoring and training result would be whether the basketball shot was made or not.

3. The method of Claim 1, wherein the sports activity is football and the Al model’s training result would be the ball reaching the receiver in a catchable zone relative to the receiver’s body.

4. The method of Claim 1, wherein the training of the Al model are inputs from the various human pose estimation attributes at various stages of the subject’s movement, and combining the human pose estimation attributes with data collected via other methods to influence the biomechanics processing and the resulting guidance or tailored feedback.

5. The method of Claim 4, wherein the human pose estimation attributes at various stages of the subject’s movement are collected by a non-wearable sensor or wearable sensors on the user.

6. The method of Claim 5, wherein the inputs are linked to the level of interaction between the non-wearable sensor or wearable sensors and the subject performing the measurable act.

7. The method of Claim 6, wherein the multi-frame human pose estimation incorporates a method of scoring movement selected from Kinematic or Volumetric.

8. The method of Claim 7, wherein the Al model is a deep-learning neural model, including a neural network, that is aided by algorithms or basic and advanced statistical methods.

9. The method of Claim 8, wherein the Al model is aided by the collaborative guidance fine- tuning model on the central platform.

10. The method of Claim 9, further comprising aggregating individualized models a central platform and generating first guidance for a subject; and selecting an origin models to seed a new individual’s model based on initial similarities in biomechanics.

11. The method of Claim 10, further comprising fine-tuning guidance based on post-guidance results across multiple subjects by reinforcement learning.

12. The method of Claim 11, wherein the fine tuning guidance includes Hidden Markov models.

Images