Interactive basketball system

Abstract

An interactive basketball system is provided. Methods, systems, and apparatuses for a basketball backboard including a computer program encoded on a computer storage medium. The basketball backboard includes a display screen, a plurality of sensors configured to generate sensor data regarding a shot attempt by a user, an imaging device configured to generate image data of the shot attempt, a speaker, and a control unit. The control unit can receive (i) the sensor data from the plurality of sensors and (ii) the image data from the imaging device. Based on the received sensor data, the control unit can determine whether the shot attempt was successful. Based on the received image data and whether the shot attempt was successful, the control unit can generate an analysis indicating characteristics of the user and the shot attempt and suggestions for improving the shot attempt for a subsequent shot attempt. The control unit can provide output data representing the analysis.

Description

[0001] This application is a divisional application of the application filed on August 18, 2021, with application number 202180063418.5 and invention title "Interactive Basketball System".

[0002] Cross-references to related applications

[0003] This application claims the benefit of U.S. Provisional Application No. 63 / 067,422, filed August 19, 2020, which is incorporated herein by reference. Technical Field

[0004] This specification generally relates to basketball, and a particular implementation relates to systems and methods for basketball games and various training programs. Background Technology

[0005] Basketball is a sport that involves two teams competing against each other on a court. The goal is for each team to score a basket against the opposing team while preventing the opposing team from scoring against them. Athletes of varying skill levels can play basketball and regularly practice to improve their chances of success against opponents. Furthermore, athletes can hire coaches, trainers, or others to assist in developing their basketball skills over time. Summary of the Invention

[0006] This specification describes a basketball system that includes specific components for monitoring users playing basketball. Specifically, the basketball system may include a backboard, a rim, and poles supporting the backboard. The backboard may comprise multiple layers, each housing different components. These components may be positioned within the backboard in a specific manner and with different layouts to enable them to monitor one or more characteristics of one or more users interacting with the basketball system within a short distance of the system on the court.

[0007] In some implementations, the basketball system may be located at a basketball court. A basketball court may include, for example, driveways, streets, courts within stadiums of professional, college, or younger scale, and various other locations. The court may comprise, for example, two halves, each with a free throw line, three-point line, half-court line, and other features. The basketball system may monitor the characteristics of users playing basketball up to the half-court line or the entire length of the court. In other instances, the basketball system may monitor the characteristics of users playing basketball while playing in their driveways, streets, or other locations.

[0008] The basketball system can monitor the characteristics of one or more users playing basketball. Specifically, a user can interact with the basketball system to request that he / she be tracked by the system. Users can use the basketball system to play basketball, and the system can generate characteristics describing the user's performance. For example, these characteristics may include the number of shots the user attempted, the number of shots the user made, the user's movement, body posture during shooting attempts, characteristics of each shooting attempt, and other characteristics. The basketball system can provide these characteristics to the user's client device for later review, or display them in real-time on the basketball system when the user interacts with it.

[0009] In some implementations, the backboard of a basketball system may include components for tracking one or more users playing basketball using the system and the ability to provide feedback to those users. For example, the backboard may comprise multiple layers, each housing different components. The front layer of the backboard may include a transparent coating to protect the components within the backboard. The second layer of the backboard may include one or more components for monitoring users interacting with the basketball system. In some implementations, the second layer may also include a display for providing feedback to the user playing basketball, along with sensors. In other implementations, the display may be located in a third layer, and the components may be located in the second layer. In some instances, the layers of the backboard may be ordered from front to back as: a first layer, a second layer, and / or a third layer.

[0010] In some implementations, components within the backboard can be configured to monitor a user playing basketball and provide feedback. For example, these components may include a camera sensing system, one or more speakers, one or more microphones, multiple sensors, a control unit, a display screen, and a power unit. The control unit can receive data from each of these components, generate user characteristics, and provide feedback to the user to help improve their basketball skills or performance. In some instances, the control unit can train a machine learning model to track and generate user characteristics while playing basketball. The machine learning model may be, for example, a convolutional neural network (CNN). These components and their functionality will be described in further detail below.

[0011] In some implementations, the basketball system can offer a variety of games for users to participate in. For example, the basketball system allows users to compete against another user or other users located at different geographical locations. In this example, another user can play basketball using their own basketball system, and the two basketball systems can communicate with each other in real time while the two users are playing using their respective basketball systems. For example, the two backboards can display shooting statistics, real-time video feeds of the other user's basketball game, and other information that each user can view while interacting with the basketball system. In another example, the basketball system allows users to play games such as training session mode, local head-to-head matchups, live streaming mode, and global competition mode. These games will be described further below.

[0012] In one general aspect, a basketball backboard includes: a display screen; a plurality of sensors configured to generate sensor data about a user's throwing attempts; one or more imaging devices configured to generate image data of the throwing attempts; a speaker; and a control unit configured to: receive (i) sensor data from one or more of the plurality of sensors and (ii) image data from the one or more imaging devices; determine, based on the received sensor data, whether the throwing attempt was successful; generate, based on the received image data and whether the throwing attempt was successful, an analysis indicating (i) user characteristics, (ii) characteristics of the throwing attempt, and (iii) suggestions for improving the throwing attempt for subsequent throwing attempts, and (iv) game performance; and provide output data representing the analysis to one or more of (i) the speaker, (ii) the display screen, and (iii) the user's client device.

[0013] Other embodiments of these and other aspects of this disclosure include corresponding systems, devices, and computer programs configured to perform actions of methods encoded on a computer storage device. A system of one or more computers may be configured in this way by means of software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the actions during operation. One or more computer programs may be configured in this way by means of instructions having instructions that, when executed by a data processing device, cause the device to perform the actions.

[0014] The above and other embodiments may optionally include one or more of the following features individually or in combination. For example, one embodiment includes all of the following features in combination.

[0015] In some embodiments, the backboard includes, wherein the plurality of sensors include one or more of a LIDAR sensor, a motion sensor, a travel sensor, and an accelerometer, and wherein the LIDAR sensor is configured to detect a user’s throwing attempt and one or both of the angle and height of the basketball from the throwing attempt; the motion sensor is configured to detect one or more users on the court within close range of the backboard; the travel sensor is configured to determine whether the throwing attempt was successful; and the accelerometer is configured to determine an indication of the basketball’s position relative to the backboard based on accelerometer data and vibration patterns.

[0016] In some embodiments, the backboard includes, wherein the one or more imaging devices include one or more depth-sensing cameras or one or more RGB cameras, wherein the one or more depth-sensing cameras are configured to perform one or more of the following: (i) detecting a user on the court, (ii) tracking the movement of the user, (iii) detecting a basketball used by the user for a throwing attempt, (iv) tracking the movement of the basketball, (v) detecting the user's posture, and wherein the one or more RGB cameras are configured to (i) record image data of the field of view of the court and (ii) record image data of the area below the backboard to detect when the user's throwing attempt corresponds to a layup.

[0017] In some implementations, the backboard includes a basket that is attached to the backboard.

[0018] In some implementations, the backboard includes, wherein the plurality of sensors include a travel sensor configured to determine whether the basketball has passed through the rim during a throw attempt.

[0019] In some implementations, the backboard includes a speaker configured to provide an audible output in response to receiving output data representing analysis from the control unit.

[0020] In some implementations, the backboard includes a display screen configured to display one or more of the following: (i) image data from the one or more imaging devices; (ii) a head-up display (HUD) showing throwing attempts and any other data related to the user and / or the game / training course; and (iii) image data from a second control unit connected via a network.

[0021] In some implementations, the backboard further includes a protective layer that connects to the display screen.

[0022] In some implementations, the backboard includes a protective layer comprising tempered glass.

[0023] In some implementations, the backboard includes a basket that is attached to a protective layer.

[0024] In some implementations, the backboard includes a control unit configured to provide received image data to a trained machine learning model to generate (i) user characteristics and (ii) characteristics of a throwing attempt, and (iii) game performance, wherein the user characteristics include user identification and the user's position relative to the backboard, wherein the characteristics of a throwing attempt include the angle of the basketball trajectory and indication of whether the basketball passed the rim, and wherein the game performance includes data associated with the game the user is playing; and the control unit is configured to store the user characteristics and characteristics of the throwing attempt in a user profile on a server outside the backboard.

[0025] In some implementations, the backboard includes a trained machine learning model configured to simultaneously identify and track multiple users on the court, and a control unit configured to: associate each of the multiple users identified by the trained machine learning model with a stored user profile; and update each of the stored user profiles with the characteristics of each user and the characteristics of each user's throwing attempts.

[0026] In some implementations, the backboard includes a server that stores multiple profiles corresponding to different users.

[0027] In some implementations, the backboard includes a control unit configured to: generate suggestions for improving subsequent throwing attempts, said suggestions including one or more of (i) body posture, (ii) arm angle, (iii) the ball's release point, and (iv) the ball's trajectory; display the generated suggestions on a display screen; and provide audible voice output to a speaker to transmit the generated suggestions to a user.

[0028] In some implementations, the backboard includes a control unit configured to provide the generated recommendations to a user's client device via a network.

[0029] In some implementations, the backboard includes a control unit configured to: determine obtained data based on received sensor data, said obtained data including one or more of the following: (i) whether a throwing attempt resulted in the basketball passing through the rim, (ii) whether a throwing attempt resulted in the basketball bouncing off the front side of the backboard without passing through the rim, (iii) whether a throwing attempt resulted in the basketball bouncing off the rim without passing through the rim, (iv) the position of the user's throwing attempt, and (v) the trajectory of the basketball during the user's throwing attempt; combine the obtained data with (i) the user's characteristics and (ii) the characteristics of the throwing attempt output from a trained machine learning model; and store the received sensor data, received image data, the combined obtained data, the user's characteristics, the characteristics of the throwing attempt, and the generated recommendations in a user profile on a server.

[0030] In some embodiments, the backboard includes a control unit configured to: receive an instruction from a user to participate in a competition with a second user; connect via a network to a second control unit associated with a second backboard used by the second user, wherein the second control unit is located at a geographically different location from the control unit; provide received image data to the second control unit via the network; receive second image data from the second control unit via the network; provide the received second image data from the second control unit to a display screen; count the number of throwing attempts by the user based on received sensor data and received image data; receive a second number of throwing attempts by the second user from the second control unit; provide (i) the number of throwing attempts by the user and (ii) the second number of throwing attempts by the second user to the display screen, wherein the display screen overlays the number of throwing attempts and the second number of throwing attempts on top of the received second image data; and provide the number of throwing attempts by the user to the second control unit.

[0031] In some embodiments, the backboard includes a control unit configured to: receive an instruction from a user to participate in a competition with a second user in a local contest; provide received image data to a display screen; count the number of throwing attempts by the user based on received sensor data and received image data; count the number of throwing attempts by the second user based on received sensor data and received image data; and provide (i) the number of throwing attempts by the user and (ii) the number of throwing attempts by the second user to the display screen, wherein the display screen overlays the number of throwing attempts and the second number of throwing attempts on top of the received image data.

[0032] In some implementations, the backboard includes a control unit configured to: determine whether a throwing attempt resulted in a basketball passing through the rim based on: generating an inner cone and an outer cone in received image data, the inner cone including a first cone having a base conforming to the rim and a first height, the outer cone including a second cone having a radius centered at the center of the rim and a second height; determining whether the basketball entered a first portion of the outer cone and exited a second portion of the outer cone; in response to exiting the second portion of the outer cone, determining whether the basketball entered a third portion of the inner cone and exited the base of the inner cone; and in response to determining that the basketball exited the base of the inner cone, determining that the throwing attempt resulted in a successful shot because the basketball passed through the rim.

[0033] In some implementations, the backboard includes a control unit configured to: in response to determining that the basketball has entered the third portion of the inner cone and has not left the base of the inner cone, determine that the result of the throwing attempt was a miss because the basketball did not pass the rim.

[0034] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. Attached Figure Description

[0035] Figure 1A This is a block diagram illustrating an example of a system used to monitor users playing basketball.

[0036] Figure 1B This is a block diagram illustrating an example of a backboard system.

[0037] Figure 1C This is another block diagram illustrating an example of a backboard system.

[0038] Figure 2A This is a block diagram illustrating an example of video analysis used to detect a basketball passing through the hoop.

[0039] Figure 2B This is a block diagram illustrating an example of video analysis used to detect when a basketball misses the hoop.

[0040] Figure 3 This is a block diagram illustrating an example of a system where two users play basketball using a connected basketball system.

[0041] Figure 4 This is a block diagram illustrating an example computational system for the backboard system.

[0042] Figure 5 This is a flowchart illustrating an example of a process used to generate characteristics for users who play basketball.

[0043] The same reference numerals and symbols in each drawing indicate the same components. Detailed Implementation

[0044] Figure 1A This is a block diagram illustrating an example of a system 100 for monitoring one or more users (e.g., one or more users playing basketball) interacting with system 100 through an activity. System 100 includes a basketball system 107, a basketball court 108, one or more users 102, and client devices 104 associated with the one or more users 102. System 100 may also include a network and servers external to basketball system 107. In simple terms, system 100 can monitor the one or more users 102 playing basketball 106 using basketball system 107, generate data describing the characteristics of the one or more users 102 playing basketball 106, and provide the data as feedback to the one or more users 102. System 100 may provide the data as feedback to client device 104 or to a display on basketball system 107. Figure 1AVarious operations in exemplary phases (A) to (G) that can be performed in the indicated sequence or another sequence are shown.

[0045] In some implementations, basketball system 107 may include a backboard 110, a pole 113, and a rim 112. Basketball system 107 may include various components and algorithms that enable tracking and monitoring of one or more users 102 playing a basketball game. Furthermore, the various components of basketball system 107 may generate suggestions to improve the basketball skills of user 102. These suggestions may focus on improving, for example, the trajectory of a user's shot, the user's posture during the shot, the user's dribbling technique, and other basketball skills. As will be further described below, basketball system 107 may track multiple users 102, track the users' basketball characteristics, and store this data in user profiles on a server located outside basketball system 107.

[0046] Furthermore, the basketball system 107 enables the one or more users 102 to play basketball games with each other within the same basketball system and with other users of their respective basketball systems located far from themselves (including other basketball systems located in various places around the world). In some embodiments, user 102 can request to play basketball games or other activities, such as basketball training sessions, by interacting with the basketball system 107. In other embodiments, user 102 can request to play basketball games using the basketball system 107 by interacting with a client device 104 that communicates with the basketball system 107 via a network, such as one or more of Bluetooth, Wi-Fi, the Internet, cloud access, and cellular data networks (e.g., networks with 4G and 5G capabilities). As will be further described below, basketball games or other activities may include, for example, training session modes, head-to-head matchup modes, global competition modes, and live streaming modes, to name just a few.

[0047] In some embodiments, the basketball system 107 may include a rim 112 and a backboard 110, vertically supported above a basketball court 108 or a court surface such as a driveway, street, lawn, or other suitable surface. The basketball system 107 includes a pole 113 or support on which the backboard 110 and rim 112 are supported. In some instances, the pole 113 may be inserted or embedded into the ground to a certain depth to maintain the stability of the basketball system 107. In other instances, the pole 113 may be inserted into a base platform above the ground that maintains the stability of the basketball system 107. In some cases, the basketball system 107 may be, for example, a small basketball system adapted to fit above a door frame in an office or bedroom.

[0048] In some embodiments, the backboard 110 may comprise multiple layers, each housing different components. Specifically, the front layer of the backboard 110 may include a transparent coating for protecting the components within the backboard. For example, the front layer of the backboard 110 may include tempered glass, which (i) protects the components within the backboard 110 and (ii) allows a user to see a display screen behind the front layer. The backboard 110 may include a second layer, which is placed or positioned behind the front layer. The second layer may include one or more components, such as sensors and cameras, for monitoring and generating data (e.g., sensor data and image data) associated with users on the court. In some embodiments, the second layer of the backboard may also include a display for providing feedback to the one or more users playing basketball on the basketball court 108. In this case, the sensors and cameras may be coupled to the display screen. In other embodiments, the backboard may include a third layer positioned behind the second layer. In this embodiment, the third layer may include a display for providing feedback to the one or more users playing basketball on the basketball court 108.

[0049] In some implementations, the basketball system 107 may include a control unit. The control unit may include one or more central processing units (CPUs), one or more graphics processing units (GPUs), and memory components for executing software via the CPU and GPU. In some instances, the control unit may be located behind the backboard 110. In other instances, the control unit may be located within a second or third layer of the backboard 110.

[0050] Typically, the control unit receives sensor and image data from one or more components within the backboard. Based on the received sensor and image data, the control unit generates data regarding the basketball characteristics of one or more users 102 playing basketball on the basketball court 108 using the basketball system 107. The control unit may, for example, identify a user profile associated with user 102 and store the generated user's basketball characteristics in the user's profile. The control unit may store the user profile and associated data within the backboard 110. Alternatively, the control unit may access a server outside the backboard 110 via a network and store the generated basketball characteristics in the identified user profile on the server. The network may include one or more of, for example, Bluetooth, Wi-Fi, the Internet, cloud access, and cellular data networks (e.g., 4G and 5G capabilities).

[0051] In some implementations, the control unit may generate recommendations tailored to a specific user based on data about the user's basketball characteristics. The control unit may display the generated recommendations on a display screen on the backboard. Additionally, the control unit may provide the generated recommendations to the user 102's client device 104. The control unit may also store the generated recommendations along with a generated profile of the identified user (e.g., along with the profile of user 102).

[0052] In some implementations, the basketball system 107 may include a power source for powering the one or more components coupled to the backboard 110. For example, the power source may power the one or more components within a second layer of the backboard 110, a display within the backboard 110, and a control unit. The power source may include, for example, a power plug inserted into a socket, a solar panel coupled to the basketball system 107, or a rechargeable battery pack attached to or coupled to the basketball system 107.

[0053] In some implementations, the backboard 110 may include one or more components for monitoring a user 102 playing basketball 106 on a basketball court 108. Specifically, the one or more components may include multiple sensors and multiple cameras positioned within the backboard 110. The multiple sensors may include, for example, one or more vibration sensors, one or more travel sensors, one or more accelerometers, a light detection and ranging (LIDAR) sensor, one or more motion sensors, and one or more pressure sensors. The multiple cameras may include depth-sensing cameras (e.g., real-time depth cameras) and red-green-blue (RGB) cameras. Control data may be received from each of the multiple sensors and from each of the multiple cameras to generate characteristics of the user 102 playing basketball.

[0054] Each of the sensors contained within the backboard 110 can be configured for a different purpose. For example, a vibration sensor can be configured to detect vibrations in the basketball system 107 based on the basketball 106 (i) bouncing off the rim 112, (ii) bouncing off the backboard 110, (iii) passing through the rim 112, and (iv) the user 102 dribbling on the basketball court 108. A travel sensor can be configured to determine whether the user 102's throwing attempt was successful. A successful throwing attempt indicates that the basketball 106 passed through the rim 112. An unsuccessful throwing attempt indicates that the basketball 106 did not pass through the rim 112. This will be further explained and described below.

[0055] An accelerometer can be configured to determine the position of the basketball relative to the backboard based on accelerometer data and vibration patterns. For example, when user 102 throws basketball 106 towards the basketball system in an unsuccessful throw, basketball 106 may bounce off the rim 112 at a specific location. This location could be, for example, the front, rear, or side of the rim 112. Furthermore, this location could be, for example, a specific position on the front side of the backboard. Based on where basketball 106 bounces off the backboard 110, the basketball system 107 can exhibit a specific vibration pattern. This specific vibration pattern may correspond to the speed or frequency of vibration of components of the basketball system 107 (e.g., backboard 110, rim 112, pole 113, or a combination thereof). The accelerometer can be configured to measure the speed or frequency of the vibration pattern and provide the detected vibration pattern to the control unit.

[0056] In some instances, the accelerometer can determine the location where the basketball 106 bounces off the basketball system 107 based on a determined vibration pattern. The accelerometer can compare the determined vibration pattern with one or more stored vibration patterns. Each stored vibration pattern indicates the location where the basketball 106 impacts the basketball system 107. In this case, the accelerometer can provide the control unit with the vibration pattern and location of the basketball 106 bouncing off the basketball system 107 during successful or unsuccessful throwing attempts. In other instances, the control unit can use accelerometer data combined with signal processing algorithms to determine specific vibration patterns inherent in different impact locations of the rim. For example, the control unit can sample the accelerometer data, run the sampled data through one or more matched filters to attempt to identify vibration patterns, and identify the vibration pattern that best matches the matched filter. In other instances, the control unit can apply other signal processing algorithms, such as low-pass filters, high-pass filters, sound modeling, waveform matching, fast Fourier transform, acceleration signal matching, and matching between signals based on statistical properties.

[0057] In some implementations, the LIDAR sensor can be configured to detect the characteristics of a user 102's throwing attempt and the throwing attempt of the basketball 106 made by the user 102. For example, the LIDAR sensor can indicate the characteristics of the throwing attempt based on the user 102's shooting point of the basketball 106 and the user 102's posture during the throwing attempt. The LIDAR sensor can generate thousands of points per second at millimeter resolution for objects within its range, which may be, for example, 15 meters. The LIDAR sensor can detect when the basketball 106 separates from the user 102 during the throwing attempt. Furthermore, the LIDAR sensor can detect the characteristics of the basketball 106's throwing attempt, which may include the angle and height of the basketball 106 during the throwing attempt. The LIDAR sensor can indicate the angle of the basketball 106 relative to the basket 112 during the time increment of the trajectory of the basketball 106's throwing attempt. Furthermore, the LIDAR sensor can indicate the height of the basketball 106 relative to the basketball court 108 during the time increment of the trajectory of the basketball 106's throwing attempt. For example, a LiDAR sensor can indicate the angle of basketball 106 as 45 degrees at time t0, 30 degrees at time t1, 22.5 degrees at t2, until the basketball reaches the basket 112 at -35 degrees at time tN. The LiDAR sensor can provide this data to the control unit while user 102 is playing basketball. In some instances, the LiDAR sensor can measure and provide the shooting angle by measuring the angle between the basketball court at t0 and the tangent to the initial shooting arc. Typically, the shooting angle with a probability of making the shot is between 35 and 60 degrees, which the LiDAR sensor can measure.

[0058] In some implementations, a motion sensor may be configured to detect one or more users 102 within close range of the backboard 110 on a basketball court 108. For example, the motion sensor may detect movement on the basketball court 108 and movement of the basketball 106. Based on the detection of movement, in some instances, the motion sensor may indicate whether the movement corresponds to movement of user 102 or movement of basketball 106. In other instances, the motion sensor may provide the detected motion data to a control unit. As will be further described below, the control unit may process the motion data, other sensor data, and image data to determine whether the movement corresponds to user 102 or basketball 106.

[0059] In some implementations, a pressure sensor may be configured to detect and calculate the position where the basketball 106 impacts the backboard 110. Based on the position of the basketball 106 impacting the backboard 110, the control unit may calculate the trajectory of the ball from the user 102's throwing attempt. The position of the basketball 106 impacting the backboard 110 may be provided in position coordinates (e.g., Cartesian or polar coordinates) relative to the front of the backboard 110. In other instances, the control unit may determine the position of the basketball 106 impacting the backboard 110 based on image data and sensor data from other sensors without using a pressure sensor. For example, the control unit may predict the impact position of the basketball 106 on the backboard 110 based on the initial conditions of the basketball 106 being thrown by the user 102, known gravitational constants and air resistance, and parabolic mathematical equations. Data from a LIDAR sensor may aid in this position determination.

[0060] As mentioned above, the multiple cameras may include depth-sensing cameras and RGB cameras. Each of the multiple cameras or imaging devices may be configured to perform different functions. For example, the backboard 110 may include one to three depth-sensing cameras. Other examples are also possible. For example, the backboard 110 may include more than three depth-sensing cameras, such as eight or more. The depth-sensing cameras may be configured to (i) detect user 102 on the basketball court 108, (ii) track the movement of user 102, (iii) detect the basketball 106 used by user 102 for a throwing attempt, (iv) track the movement of the basketball 106, and (v) detect the posture of user 102. For example, three depth-sensing cameras may have an overlapping field of view (FOV) to encompass the widest possible view of the basketball court 108.

[0061] For example, a depth-sensing camera located in backboard 110 can be configured to detect user 102 on basketball court 108 based on software that detects and identifies user 102's movement. The depth-sensing camera can detect and identify user 102 without using machine learning models such as convolutional neural networks (CNNs), offering advantages over previous systems. Furthermore, the depth-sensing camera can track user 102's movement as the user moves along basketball court 108. The user may, for example, move behind the free-throw line to attempt a shot, move behind the three-point line to attempt a shot, or move to any other location on basketball court 108.

[0062] A depth-sensing camera can track user 102 by generating the position coordinates of user 102 along the basketball court 108 and providing these coordinates to the control unit within the image data. Similarly, a depth-sensing camera can track one or more basketballs 106 by generating the position coordinates of basketball 106 along the basketball court 108 and providing these coordinates to the control unit within the image data. For example, in each frame of image data recorded by the depth-sensing camera, the camera can draw boxes around the identified user 102 and basketball 106 and attach position coordinates to each box. In this way, the control unit can determine the position of user 102 and basketball 106 as they move from each frame of image data. Furthermore, the depth-sensing camera can track multiple basketballs and multiple users on the basketball court 108. The depth-sensing camera can also track and associate throwing attempts and successful / missed shots for each of the multiple users.

[0063] The depth-sensing camera can also track the body posture of user 102. User 102's body posture can correspond to the user's posture during dribbling, during a throwing attempt, or after a throwing attempt (e.g., referred to as a follow-through, and movement to bounce the basketball 106 back if user 102's throwing attempt is unsuccessful) (to name just a few). The body posture detected by the depth-sensing camera can indicate the body's position in position coordinates. The depth-sensing camera can provide the posture detection to the control unit, where further analysis of the posture detection data can be performed.

[0064] In some implementations, the backboard 110 may utilize a LIDAR sensor instead of a depth-sensing camera. In addition to the functionality described above regarding the LIDAR sensor, the LIDAR sensor may also be configured to perform the functionality described by the depth-sensing camera. This functionality may include detecting the identification and movement of one or more users on the basketball court. Furthermore, the LIDAR sensor may be configured to track the movement of identified users over time. The LIDAR sensor may be configured to identify and monitor user movement without using a machine learning model. Similarly, the LIDAR sensor may identify and track the movement of one or more basketballs on the basketball court 108, similar to how the depth-sensing camera tracks the movement of one or more basketballs.

[0065] In some implementations, the backboard may include one or more RGB cameras configured to perform specific functions. Specifically, these functions may include (i) recording image data of the field of view of the basketball court 108 and (ii) recording image data of the area below the backboard to detect when a user 102's throwing attempt corresponds to a specific type of shot. For example, the backboard 110 may include one or two RGB cameras. The RGB cameras may record in real-time the area of ​​the basketball court 108 within the vicinity of the backboard 110. This area may include, for example, the area extending to the half-court line on the basketball court 108, the entire length of the court in the case of a standard basketball court, the lane area, or some other area. Furthermore, the RGB cameras may record image data of the area below the rim 112, which can be used when the user 102 is making a layup or another type of shot below or near the rim 112. The RGB cameras may provide the recorded image data to a control unit for further analysis and use, as will be described further below.

[0066] See Figure 1A During phase (A), user 102 may request interaction with basketball system 107 by, for example, requesting to play a basketball game. In some embodiments, user 102 may access a basketball application on their client device 104. User 102 may log in to the basketball application using authentication credentials such as a username and password, and may access applications provided by basketball system 107, such as games, training courses, etc. User 102 may choose to interact with basketball system 107 using basketball 106. For example, as shown in system 100, user 102 may select a game to count the number of shots made within a predetermined amount of time.

[0067] In other embodiments, user 102 may communicate with basketball system 107 to request access to a basketball game using basketball system 107. User 102 may provide a verbal command, such as “Hi Huupe,” to basketball system 107 or perform a waving gesture to wake basketball system 107. Components within backboard 110, such as a microphone and / or a depth-sensing camera, may detect user communication with basketball system 107 and perform user detection functions. For example, the microphone and depth-sensing camera may provide verbal commands, such as “Hi Huupe,” and image data detecting user 102 to the control unit, respectively. The control unit may determine (i) user 102’s identification and (ii) an instruction from the audio and image data that the control unit will provide user 102 with access to a list of available games.

[0068] The control unit can provide a list of available games to the display screen of the backboard 110 or to the user's device 104. Alternatively, the control unit can provide the list of available games audibly to the speakers of the backboard 110. The user 102 can select which game to play by speaking to the basketball system 107 or by making a selection via the user's client device 104. For example, the user 102 can instruct "Hey Huupe, play the timer game," and the control unit can recognize the voice command and determine that the user 102 requests the timer game. Alternatively, the user 102 can select which game to play by waving in front of the basketball system 107. The display screen of the backboard 110 can list the available games, and the user 102 can raise their arm to act as a mouse on the display. A depth-sensing camera can recognize that the user 102 has raised their arm via recorded image data and provide the recorded image data to the control unit. The control unit can associate the arm raising with this request from the user 102 to select an available game from the game list. User 102 can move their arm continuously, and the depth-sensing camera, control unit, and display work together to display a mouse that moves in a manner similar to user 102's arm to allow for selection of available games. The display can visually show the mouse's movement above the game list to manipulate the movement as user 102 moves their arm. In another example, each hand position of user 102 can represent an area of ​​the display screen. In this case, if user 102 raises their hand vertically upwards, the depth camera, control unit, and display will associate this movement with a button highlighted at the top center of the display screen. If user 102 raises their right hand at a 45-degree angle (e.g., the upper right of the display screen), the depth camera, control unit, and display will associate this movement with a button highlighted in the upper right corner of the display screen. The process is similar if the user places their right hand at a -45-degree angle (which corresponds to a button highlighted in the lower right of the display screen). If the user raises their left hand at a 45-degree angle, this corresponds to a button highlighted in the upper left of the display screen. If the user places their left hand at a -45-degree angle, this corresponds to a button highlighted in the lower left of the display screen. This process is similar for other hand positions around the display. Furthermore, when a user changes from an open hand to a clenched hand (e.g., from an open palm to a clenched fist), the control unit can recognize that the user wishes to select or enter that selection. The selection can also be performed using a highlighting for a predetermined amount of time (e.g., 3 seconds).

[0069] User 102 can select an available game by performing a selection based on arm movement. The selection may be, for example, pointing, a verbal command, or another gesture instructing the user to select a particular game. In an example of system 100, user 102 may select a timed throwing game, and in response, the control unit may initiate the execution of the timed throwing game. For example, the control unit may provide a throwing timer 116 and a throwing counter 114 in digital format to a display on the backboard. The control unit may instruct user 102 to begin throwing the basketball 106 through the hoop 112 once the throwing timer 116 begins its countdown.

[0070] In some implementations, the control unit can adjust the throw counter 114 when user 102 attempts to throw. For example, user 102 may perform a throw attempt corresponding to throwing a basketball 106 through the rim 112. A depth-sensing camera and sensors may generate image data and sensor data, respectively, and provide the image data and sensor data to the control unit, whereby the control unit can determine (i) whether the user attempted to throw and (ii) whether the throw attempt was successful. The control unit can then update the throw counter 114 in the head-up display (HUD) on the display based on whether the user attempted to throw and whether the throw attempt was successful. For example, if user 102 misses the first five throws, the control unit may display "0 / 5" for the throw counter on the HUD on the backboard.

[0071] Furthermore, the control unit can display images recorded by an RGB camera contained within the backboard. For example, when user 102 throws basketball 106, the RGB camera can record an area of ​​the basketball court 108 and provide the recorded image data to the control unit. The control unit can receive the recorded image data and display it in real time on the backboard 110's display while user 102 is playing. In this way, user 102 can visually see themselves on the backboard 110 while playing a timed game. Additionally, the control unit can digitally overlay a throw timer 116 and a throw counter 114 on the backboard's display over the recorded image data. The recorded image data from the RGB camera can fill the entire display of the backboard 110. For example, the throw timer 116 can be positioned in the upper left corner of the display, and the throw counter can be positioned in the lower right corner. Other positions are also possible.

[0072] During phase (B), the one or more sensors included in backboard 110 acquire sensor data 118 corresponding to user 102 on basketball court 108. As previously mentioned, sensor data 118 may include data from travel sensors, accelerometers, LiDAR sensors, and motion sensors. For a specific throwing attempt demonstrated in system 100, user 102's throwing attempt is successful. Travel sensors may indicate that user 102's throwing attempt was successful because the basketball 106 passes the rim 112. For example, backboard 110 may include an array of two or more laser travel sensors vertically aligned and overlapping with the rim system. If the ball passes through a series of laser travel sensors sequentially from high to low, the control unit may determine that the throwing attempt was successful. Alternatively, if the ball does not pass through the laser travel sensors sequentially from high to low, the control unit may determine that the throwing attempt was unsuccessful.

[0073] The accelerometer can indicate small vibration patterns as the path of the basketball 106 through the rim 112 results in a swish, or a successful throw that does not hit the rim 112 or backboard 110. Alternatively, if the basketball 106 does hit the backboard 110 or rim 112 with a successful throw attempt, the accelerometer can record specific vibration patterns to indicate where the basketball 106 landed. A LIDAR sensor can detect the user 102's throw attempt and its characteristics, such as the angle and height of the basketball 106 during the throw attempt. A motion sensor can detect the user 102 on the basketball court 108. Furthermore, the motion sensor can detect the movement of the user 102 and the movement of the basketball 106.

[0074] In some implementations, the backboard 110 may periodically acquire sensor data 118 from multiple sensors. For example, the sensors may be configured to acquire sensor data 118 every 2 seconds, 5 seconds, or 10 seconds. In other instances, the sensors may be configured to acquire sensor data 118 based on the type of game the user 102 is playing. For example, if the user 102 is playing a timed throwing game, the sensors may be configured to acquire sensor data 118 at more frequent intervals to ensure that all throwing attempts, whether successful or not, are taken into account. Furthermore, if the user 102 notifies the basketball system 107 that the count of throwing attempts is incorrect, as well as the counts of successful and missed shots (and vice versa), the user 102 may adjust the sensitivity or frequency of sensor acquisition via the client device 104 until the count is accurate. In another instance, if the user 102 is playing a game where the user needs to throw from a pre-defined location on the basketball court 108, the sensors may be configured to acquire sensor data 118 less frequently because between each throw, the user 102 must bounce the ball back and move it to a different pre-defined location. This reduces the amount of processing that sensors and control units must perform during games where the user's goal is to throw fewer balls. More precisely, it reduces overall complexity and the amount of processing performed by sensors and control units.

[0075] During phase (C), the one or more cameras included in backboard 110 acquire image data 122 corresponding to user 102 on basketball court 108. As previously mentioned, image data 122 may include image data from depth-sensing cameras and RGB cameras. For example, image data 122 may include images or videos from each of the cameras. For example, as shown in system 100, user 102's throwing attempt is successful because it passes the basket 112. The depth-sensing cameras may generate (i) detection data of user 102 on basketball court 108, (ii) movement data of user 102, (iii) detection data of basketball 106 used by user 102, (iv) tracking movement data of basketball 106, and (v) detection data of user's posture.

[0076] As previously described, the depth-sensing camera can generate and track each of the different detection and movement data of user 102 and basketball 106. The depth-sensing camera continuously records image data 122 of user 102 monitoring the basketball court 108, and generates this detection / movement data for each frame of the recorded image data 122. In some instances, the depth-sensing camera can provide this image data 122 and the detection / movement to the control unit in real time. Additionally, an RGB camera can record user 102 on the basketball court 108 and can provide the recorded image data 122 to the control unit. The control unit can receive sensor data 118 and image data 122 to generate characteristics of the user and the throwing attempt.

[0077] During phase (D), the control unit may receive image data 122 from depth sensing and RGB cameras and provide the image data 122 to a trained machine learning model. The control unit may train the machine learning model to perform a variety of functions. The functions may include (i) classifying or identifying each user on the basketball court 108 as a unique and persistent user, (ii) identifying user throws, (iii) generating characteristics of users on the basketball court 108, and (iv) generating characteristics of user throw attempts.

[0078] For example, a trained machine learning model can correspond to a convolutional neural network (CNN). The control unit can train the machine learning model using different image data of successful and unsuccessful throwing attempts from various positions on different courts. The control unit can also use different image data of throwing attempts by professional basketball players to train the machine learning model to understand ideal throwing attempts, thereby assisting other users interacting with the basketball system 107. The control unit can provide image data of these professional athletes dribbling, shooting from various positions on the court, and moving while dribbling. In another example, RGB and depth sensing cameras can be used to determine the user's posture during a throwing attempt and can train a CNN based on the identified posture, either as the ideal posture during the throwing attempt (e.g., the ideal posture of a professional athlete or other individual) or which posture identification provides the best result.

[0079] In some implementations, the control unit may also train a machine learning model to identify users interacting with the basketball system 107. For example, when a user, such as user 102, attempts to initially use the basketball system 107, an application on client device 104 may request the user to enter their credentials and take a photo or selfie of themselves. This provides the control unit with initial images to train the machine learning model to detect user 102. The control unit may then instruct user 102 to use the basketball system 107 to play the game by making throwing attempts. The control unit may acquire image data 122 of user 102 and use the newly acquired image data 122 to train the machine learning model. The control unit may provide an indication on the display that the basketball system 107 is in a learning mode for learning user 102. Then, at a later point in time, and once the machine learning model is sufficiently trained, the control unit may apply the trained machine learning model to (i) identify user 102 as a unique user, (ii) identify the user's throws, (iii) generate characteristics of the user on the basketball court 108, and (iv) generate characteristics of the user's throwing attempts.

[0080] In some implementations, a trained machine learning model may output labeled data 126. Labeled data 126 may indicate the identification of user 102 and the position of user 102 relative to the backboard 110 on frames of image data 122. For example, labeled data 126 may indicate that user 102 is "Bob," and indicate the (X,Y,Z) coordinates of where user 102 is located on the basketball court 108 relative to the basketball system 107. The labeled data may also indicate the angle of the trajectory of basketball 106 for a specific image data frame, and whether basketball 106 passed the rim 112. For example, the angle of the trajectory of basketball 106 may indicate that the basketball is at 101 degrees relative to the basketball system at a specific time point. Furthermore, labeled data 126 may indicate that basketball 106 did indeed pass the rim 112, e.g., successfully. In some instances, the labeled data 126 may include statistics or percentages indicating the probability that user 102 is “Bob”, the probability that user 102 appears to be located, the probability of the angle of the basketball trajectory, and the probability that the basketball passes through the hoop 112. These probabilities may, for example, be in the range of 0-100% or 0-1.

[0081] If multiple users 102 are interacting with the basketball system 107, the control unit can simultaneously identify and track each of the users 102. For example, the control unit can receive sensor data from each of the sensors within the backboard 110 that monitors the characteristics of each of the different users. The control unit can also receive image data from each of the cameras within the backboard 110 and provide the image data to a trained machine learning model. In response, the machine learning model can identify each of the users on the basketball court 108 and track each of the users on the basketball court 108 and their corresponding movements with and / or without a basketball. The labeled data 126 output by the trained machine learning model can include image data frames with labels for each of the users in the frame and characteristics corresponding to each of the users, as described above. For example, the trained machine learning model can track each of the users interacting with the basketball system 107 based on their jersey, jersey number, specific clothing type, and body characteristics. In this case, the trained machine learning model can also correlate each successful and missed shot from each of the users on the basketball court.

[0082] In some implementations, the trained machine learning model can also be used to generate real-time predictions of a user's throws. For example, the trained machine learning model can generate real-time predictions for each throw as the basketball travels toward the rim 112. For instance, based on previous sensor data, image data, and current input data from sensors and cameras, the basketball system 107 can initially predict the probability of a successful throw when the ball is ten feet from the rim 112, for example, a probability of 70.0%. The trained machine learning model can adjust the probability after the basketball 106 hits the rim 112, for example, a probability of 55.0%. The trained machine learning model can be continuously updated / retrained using predictions and results from throw attempts, utilizing image data and sensor data captured from the basketball system 107.

[0083] During phase (E), the control unit may analyze the received sensor data 118 to determine whether the throwing attempt was successful. For example, the control unit may first determine from the LIDAR sensor whether a throwing attempt was detected. If the control unit determines that a throwing attempt was detected, the control unit may analyze data from a travel sensor located on the backboard 110 behind the rim to determine whether the basketball 106 passed the rim 112. If the throw did not pass the rim 112, the control unit may analyze data provided by the accelerometer to determine the position of the basketball 106 relative to the backboard impact based on the vibration pattern. If no vibration pattern is detected and an unsuccessful throwing attempt is detected, the control unit may determine that the user did not hit the rim 112 and backboard 110 during the unsuccessful throwing attempt, for example, referred to as an "airball". In another example, if the control unit determines that the throw did not pass the rim and a vibration pattern is detected, the control unit may determine whether the basketball 106 bounced off the front side of the backboard 110, the bottom side of the backboard 110, the top portion of the backboard, the side portion of the backboard, or some other location on the rim 112 (or a combination of each). The control unit can analyze sensor data provided by motion sensors and LIDAR sensors to determine the location and trajectory or arc of the throwing attempt. If the control unit determines that the throw did indeed pass the rim, it can determine the vibration pattern to indicate how the basketball passed the rim 112, such as leaving the backboard 110, leaving the rim, or a swish.

[0084] During phase (F), the control unit may perform analysis on labeled data 126 provided by the trained machine learning model 124 and data generated from the analysis of sensor data 118 during phase (E). Based on the labeled data 126 and the analysis performed on the sensor data 118, the control unit may generate output data 130. Output data 130 may include the user's position 132, user identification 134, ball trajectory angle 136, throw attempt success / miss 138, and improvement suggestions 140. For example, the control unit may generate the output data and store the data in a digital data type such as structure, class, or other. For example, as shown in system 100, the user's position 132 may indicate the X, Y, Z coordinates relative to the basketball system 107 and the basketball court 108 – “12.00, 1.01, 0.00”, measured in feet. The user identification 134 may correspond to the name or other identifier of the identified user 102, such as “Bob”. The ball trajectory angle 136 can correspond to an angle on the flight time trajectory, such as 5 degrees at t9, where time can be measured in seconds or milliseconds, for example. For example, the throw attempt to hit / miss 138 can correspond to "hit". In addition, the control unit can generate a suggestion 140 based on the performed analysis.

[0085] For example, the control unit may generate suggestions to be provided to user 102 for improving subsequent throwing attempts based on user 102's current and previous throwing attempts. These suggestions may correspond to improvements for: (i) the user's posture, (ii) the user's arm angle, (iii) the ball's release point during the throwing attempt, and (iv) the ball's trajectory during the throwing attempt. For example, the control unit may compare user 102's posture during a throwing attempt with stored image data of a professional athlete's posture during a throwing attempt. User 102 may indicate via an application on client device 104 that they wish to throw like a specific professional athlete. The control unit may display image data of user 102's shot and image data of a professional athlete's shot side-by-side on the display of client device 104 and / or backboard 110. In this way, the user can practice their shooting form to match the shooting form of a professional athlete displayed on the display. The control unit can determine the degree of similarity in shooting form by comparison and, for example, provide the user with a percentage to indicate how close their posture is to that of a professional athlete during the throwing attempt. In this way, user 102 is able to improve their posture over time during throwing attempts.

[0086] The control unit can also analyze the user 102's arm angle or limbs during the throwing attempt. For example, the arm angle could be an outward angle, an inward angle, or an angle in between during the throwing attempt. The control unit can instruct the user 102 to adjust their arm angle for subsequent throws to better align with the arm angle of a professional. Alternatively, the control unit can instruct the user 102 to adjust their arm angle to improve the probability of success in subsequent throwing attempts. The instructions can be displayed as image data (e.g., video) or in another form on the display of the backboard 110 or on the client device 104. Similarly, the control unit can analyze the shooting point of the basketball 106 and its trajectory during the user's throwing attempt, and provide suggestions to improve these based on another player's shooting point and trajectory, thereby improving the probability of successful throws.

[0087] In some implementations, the control unit may generate a profile of user 102. This profile may include user 102's identification (e.g., username Bob), user 102's credentials, one or more client devices associated with user 102 (e.g., client device 104), and user 102's characteristics during a throwing attempt. For example, user 102's characteristics during a throwing attempt may include the user's classification and the user's position on the basketball court 108 relative to the backboard 110 during the throwing attempt. Furthermore, user 102's characteristics may include the user's wingspan, height, hand size, and speed. These characteristics may also include user 102's previous throwing attempts, such as the characteristics of the basketball 106's trajectory during those attempts, including height, angle, trajectory points, user's vertical jump, and shooting point at different times. The control unit can also store in the profile whether the previous throwing attempt was successful or unsuccessful, and if successful, what type of throw it was, such as a shot that bounced off the backboard 110, a shot that bounced off the rim 112, or a shot that went through the net of the basketball system 107. The control unit can also store in the profile sensor data 118, image data 122, tagged data 126, and output data 130.

[0088] In some implementations, the control unit may also store game performance data, or game data associated with the game played by user 102, in a profile corresponding to user 102. The type of game played may include, for example, training course mode, local head-to-head matchup, live stream mode, and global competition mode. For example, game performance may include what game was played, the date and time of the game, the number of players in the game, the identification of each player participating in the game, the final score of the game, each player's made and missed shots during the game, the position of each player on the basketball court for each made and missed shot, and the time when each player's made and missed shots occurred, both in absolute and relative time (relative to the start of the game). The control unit may store game data as tuples, structures, classes, or some other computer format. If multiple users are playing a single game, the control unit may store the game data of each user in the corresponding profile of that single game.

[0089] The control unit can store a profile for user 102 on an external server. When the profile is updated based on newly received sensor and image data, the control unit can access and retrieve user 102's profile. For example, the control unit can identify the user profile based on facial recognition from received image data, user 102 entering a username and password, iris recognition from received image data, fingerprint matching, or some other suitable authentication or identification method. Once the control unit identifies the user profile corresponding to user 102, it can access the corresponding profile from the external server and update its contents. Once the update of user 102's profile is complete, the control unit can update the external server with the revised profile.

[0090] During phase (G), the control unit can provide output data 130 for review by user 102. The control unit can provide output data 130 for review by user 102 in several ways. One way is that the control unit can provide output data 130 to client device 104 for review by user 102. In this way, user 102 can view output data 130 to analyze recent shooting attempts, such as ball trajectory angle 136, suggestions 140, and user position 132. In some instances, the control unit can provide output data 130 to a display on backboard 110, and the user can interact with backboard 110 via verbal or gestural commands to view output data 130. In this way, user 102 can attempt to improve their basketball skills for subsequent shooting attempts based on the output data 130 characterized by the basketball system 107 for the most recent shooting attempt.

[0091] In some implementations, the control unit may also provide the generated user profile to client device 104 for review by user 102. User 102 can access an application on their client device 104 to view the profile generated for them by the control unit. Specifically, user 102 can view data from previous throwing attempts to analyze how the user's throwing attempts progressed in the profile. For example, user 102 can view sensor data 118, image data 122, tagged data 126, and output data 130 for each previous throwing attempt on their client device 104. In some instances, the control unit may display the profile on a display on backboard 110, and the user may interact with the profile via verbal or gesturing commands.

[0092] In some implementations, the control unit may provide encouraging suggestions to user 102 while user 102 is playing the game. For example, if the control unit informs user 102 of a low shot-to-attempt ratio, the control unit may provide an audible message 146 via a speaker on backboard 110. The audible message 146 can be heard by user 102, for example, saying, "6 minutes left! Don't stop shooting." In another instance, if the control unit informs user 102 of a successful shot attempt, the control unit may provide another audible message 144 via a speaker on backboard 110, saying, "Great shot!" The control unit may also provide other audible messages to the speaker on backboard 110 or to client device 104. Each of these encouraging messages may also be stored in user 102's profile.

[0093] User 102 can continue playing the throwing timer game until the time displayed on the throwing timer 116 has elapsed. Afterward, user 102 can choose another game available using the basketball system 107, review the analysis generated by the basketball system 107 on the backboard 110 display or client device 104, or shut down the basketball system 107. In some instances, the user can turn the basketball system 107 on and off via an application on the client device 104.

[0094] In some implementations, a miniature backboard may be incorporated into system 100. The miniature backboard may include components and functionality similar to those described for backboard 110 relative to system 100. Instead of including pole 113, the miniature backboard may be placed in various locations within a residence, office building, or other area. For example, a user may be able to mount the miniature backboard to a wall in a residence or hang it above a specific side of a door or cabinet. The miniature backboard may include multiple anchor points on the rear side of the backboard for attachment to various attachment points. These anchor points may be attached by, for example, ropes, hooks, screws, fasteners, and other attachments. The miniature backboard may include a basket frame with a net attached to its front side. In other instances, a user may be able to mount the miniature backboard on a surface in a garage, basement, or other area within a residence or company property. In other instances, the miniature backboard may include a small pole with a base for support that can be placed in any location, and a user may be able to throw a smaller basketball at its basket. These locations may include, for example, bedrooms, basements, kitchens, office spaces, living rooms, and other places.

[0095] In some implementations, the miniature backboard may contain fewer components than backboard 110 while maintaining similar functionality. The number of sensors within the miniature backboard can be reduced because its size can be significantly smaller than backboard 110. For example, the miniature backboard may contain two depth-sensing cameras instead of three to eight, with overlapping fields of view for inspecting the area where the user is shooting. The rim of the miniature backboard is also significantly smaller than that of backboard 110. Due to the smaller rim size of the miniature backboard, the basketball used for it is also significantly smaller.

[0096] The miniature backboard may contain a similar number of playable games as backboard 110 and include the ability to connect via a network to other miniature backboards located in other geographic areas. In some cases, the miniature backboard may include the ability to play basketball with users at other standard-sized backboards (e.g., backboard 110) in different geographic areas via the network. Users can interact with the miniature backboard by communicating with it verbally, through hand gestures, or by interacting with a smart application on its client device that communicates with the miniature backboard, similar to how users would interact with backboard 110.

[0097] Figure 1B This is a block diagram illustrating an example of a backboard system 101. System 101 shows the structure of a backboard 110 in detail. The backboard 110 shown in system 101 is similar to the backboard 110 shown in system 100.

[0098] System 101 includes a backboard 110, a network 150, a client device 104, and an external server 170. The backboard 110 includes a front layer 164, a second layer 166, and a display 168. The front layer 164, second layer 166, and display 168 are positioned longitudinally, in a layered, or stacked manner, but can be arranged in any suitable configuration. System 101 includes fasteners 158A and 158B, which secure the various layers of the backboard 110 together in place. In some cases, the backboard 110 may have fewer than three layers, as will be further explained and described below. The backboard 110 is IP67 waterproof and can, for example, weigh nearly 300 pounds.

[0099] Fasteners 158A and 158B may be shock-absorbing fasteners connecting different layers of the backboard 110. Fasteners 158A and 158B may be any suitable device for absorbing and / or minimizing the transmission of impact forces and / or vibrations from the rim 112 and / or the front layer 164 to the display 168. In one example, fasteners 158A and 158B may include one or more brackets and springs, screws, rivets, bolts, or other suitable mechanisms. In another example, fasteners 158A and 158B may include one or more adhesives, sealants, or other suitable mechanisms.

[0100] In some implementations, the front layer 164 of the backboard 110 protects components within the backboard 110. For example, the front layer 164 may be tempered glass covered with a protective coating having a translucent or transparent material (e.g., an anti-reflective coating) or both. The front layer 164 may also be configured to allow a user interacting with the backboard 110 (e.g., user 102) to view the display screen 168 within a third layer of the backboard 110. The front layer 164 may also be connected to the basket 112 at its bottom portion. A second layer 166 of the backboard 110 may contain a section between the front layer 164 and the display screen 168 that includes one or more components of the backboard 110.

[0101] For example, the second layer 166 may include a camera sensing system 156, a streaming camera 157, a sensor 154, and speakers 160A and 160B. The camera sensing system 156 may include one or more depth-sensing cameras that examine the basketball court 108 and are positioned at the top portion and center of the second layer 166. The streaming camera 157 may include one or more RGB cameras that examine the basketball court 108 and are positioned below the camera sensing system 156. In some instances, the second layer 166 may be a compressible material such as rubber or foam filler.

[0102] Sensor 154 can be located in various positions within the second layer. For example, a LiDAR sensor can be located within camera sensing system 156. A motion sensor can be located within camera sensing system 156. A travel sensor 162 can be a laser travel sensor, vertically positioned at the bottom of backboard 110 on the front layer 164 behind the rim 112. Speakers 160A and 160B can be located within the second layer 166 of backboard 110. Speaker 160A is capable of playing music and / or providing audible feedback to user 102. An accelerometer can be located within sensor 154 in the second layer 166. A microphone can be located within sensor 154 in the second layer 166.

[0103] As previously mentioned, sensor 154, including stroke sensor 162, can sense the player, the ball, and / or the forces applied to basketball system 107. Sensor 154 generates data that is processed and analyzed by control unit 148. Each of the sensors 154 can be configured in a specific manner to appropriately detect, acquire, and generate sensor data for the user playing basketball. Sensor 154 can be coupled to front layer 164, display 168, pole 113, or second layer 166. In some instances, sensor 154 may also be located remotely to basketball system 107 or disconnected from basketball system 107. For example, sensor 154 may be coupled to existing court lighting system and / or auxiliary support structure along the side or end of basketball court 108.

[0104] The control unit 148 may be positioned behind the display 168 of the backboard 110. Each of the sensors, cameras, speakers, and microphones may be connected to the control unit 148 in a bidirectional manner. The control unit 148 may communicate with an external server 170 and client device 104 via a network 150. The network 150 may be, for example, the Internet, Wi-Fi, Bluetooth, Ethernet, or some other form of wireless or wired connection.

[0105] Display screen 168 can display the throw timer 116 and the throw counter 114. User 102 can see the throw timer 116 and the throw counter 114 through the front layer 164 and the second layer 166. User 102 can also see image data displayed on display screen 168 by the control unit, such as a live video of user 102 playing basketball, or a video of another user playing basketball from another connected control unit. The control unit can digitally overlay the throw timer 116 and the throw counter 114 on top of the video image provided to display screen 168. In addition, a square of backboard 110 can be generated and displayed on display screen 168.

[0106] Display screen 168 can visually display information, signs, videos, and / or images. Display screen 168 may have a brightness of 1000 nits or more. Display screen 168 can be any suitable display panel, such as an LED or LCD display. For example, display screen 168 can be a smart TV. Display screen 168 can also be a screen for projecting information, signs, videos, and / or images. For example, a projector can project information onto the display screen. In another example, display screen 168 may include a short-throw projector as a display for providing information, signs, videos, and / or images. Client devices can stream any image data to display screen 168 for user viewing.

[0107] System 101 also includes a power source 152. As previously mentioned, power source 152 can power one or more components within the second layer 166 of backboard 110, the display within backboard 110, and the control unit. Power source 152 may include, for example, a power plug inserted into a socket, a solar panel connected to basketball system 107, or a rechargeable battery pack attached to or connected to basketball system 107. Power source 152 may be located on the rear side of display screen 168.

[0108] Figure 1C This is another block diagram illustrating an example of a backboard system 103. System 103 contains components similar to those of the backboard 110 shown in system 101. However, the backboard 110 in system 103 comprises two layers, such as a front layer 164 and a display screen 167. The different components of the backboard 110 can be positioned and configured to operate in three layers and two layers, respectively, as shown and described in systems 101 and 103.

[0109] In system 103, the front layer 164 and the display screen 168 are spaced apart by a predetermined amount to reduce the impact during a user's throwing attempt. For example, the front layer 164 and the display screen 168 may be spaced 0.5-1.0 inches apart, such that when the basketball hits the front layer 164, as the front layer 164 vibrates or flexes, the front layer 164 will not contact the display screen 168.

[0110] In another example, the display screen 168 is tightly mated to the front layer 164. In this example, the front layer 164 is formed of generally rigid Plexis glass or other suitable material, and there is a minimal or no space between the front layer 164 and the display screen 168. Thus, when the basketball contacts the front layer 164, the front layer 164 will not move relative to the display screen 168. Therefore, the display screen 168 will not be damaged when the basketball contacts the front layer 164. In this example, the rod 113 or support member is embedded in the ground, and the connection between the rod 113 and the display screen 168 and / or the front layer 164 is rigid, such that the force exerted on the front layer 164 by the basketball is transmitted to the ground, thereby minimizing vibration and jitter of the display screen 168.

[0111] Figure 2A This is a block diagram illustrating an example of video analytics 200 used to detect a basketball passing through the rim. Specifically, video analytics 200 illustrates the process performed by the control unit when it determines that a throwing attempt was successful (e.g., the basketball passed the rim of the backboard). Video analytics 200 illustrates components similar to those in systems 100, 101, and 103, and such components need not be described again here.

[0112] Video analytics 200 includes a basketball 202 thrown by the user, a backboard 204, and a virtual zone. The virtual zone includes a virtual inner cone 208 and a virtual outer cone 210. The control unit can apply the virtual inner cone 208 and the virtual outer cone 210 to received image data from a depth-sensing camera.

[0113] The virtual inner cone 208 and virtual outer cone 210 may include one or more features positioned above the basket rim. For example, the virtual inner cone 208 may include an inner radius 206 matching the radius of the basket rim. The virtual inner cone 208 includes a base conforming to the basket rim. The virtual inner cone 208 also has a specific height extending from the basket rim. For example, the height of the virtual inner cone 208 may be 1 foot. Similarly, the virtual outer cone 210 may include an outer radius 212, which includes the radius of the basket rim and a distance greater than the radius of the basket rim. For example, the outer radius 212 may correspond to 2.5 feet or another distance. The virtual outer cone 210 may include an extruded profile of the size of the virtual inner cone 208 such that the two virtual cones do not overlap each other. The virtual outer cone 210 also has a specific height extending from the basket rim. For example, the height of the virtual outer cone 210 may be 4 feet or another height. The control unit may utilize the features of the virtual cones displaced above the basket rim to determine whether a throwing attempt results in a successful or unsuccessful throw.

[0114] For example, as shown in video analysis 200, a user performs a throwing attempt by throwing a basketball 202 toward the backboard 204. The control unit may consider the throwing attempt successful if: (i) the basketball 202 enters the virtual outer cone 210, then (ii) the basketball 202 leaves the virtual outer cone 210, then (iii) the basketball 202 enters the virtual inner cone 208, and finally (iv) the basketball 202 leaves the virtual inner cone 208 (e.g.) only via the base of the virtual inner cone 208 and enters the basketball system's hoop. At this point, the control unit may consider the throwing attempt successful because the basketball 202 has passed the basketball system's hoop. The control unit may perform these video analyses during analysis 128 of system 100 to determine whether the throwing attempt was successful.

[0115] Figure 2B This is a block diagram illustrating an example of video analytics 201 used to detect when a basketball fails to pass the rim. Video analytics 201 shows components similar to those in video analytics 200, which will not be described again here. Video analytics 201 may illustrate the process performed by the control unit when it determines that a throwing attempt has failed (e.g., the basketball has failed to pass the rim of the backboard). Furthermore, video analytics 201 shows components similar to those in systems 100, 101, and 102, and it is unnecessary to describe these similar components again.

[0116] As shown in video analysis 201, a user performs a throwing attempt by throwing a basketball 202 toward the backboard 204. The control unit can determine that the throwing attempt is unsuccessful in several ways. In one instance, if the basketball 202 does not enter the virtual outer cone 210 or the virtual inner cone 208, the control unit may consider the throwing attempt unsuccessful. In another instance, if the basketball 202 enters the virtual outer cone 210 and then leaves the virtual outer cone 210 without entering the virtual inner cone 208, the control unit may consider the throwing attempt unsuccessful. In yet another instance, if the basketball 202 enters the virtual outer cone 210, then leaves the virtual outer cone 210, and finally leaves the virtual inner cone 208 without passing the base, the control unit considers the throwing attempt unsuccessful. The control unit may perform these video analyses during analysis 128 of system 100 to determine whether the throwing attempt is unsuccessful.

[0117] Figure 3 This is a block diagram illustrating an example of a system 300 in which two users play basketball using a connected basketball system. As previously mentioned, for example... Figure 1A Basketball systems such as Basketball System 107 enable users to play various basketball games. These games may include, for example, training course modes, head-to-head matchup modes, global competition modes, and live streaming modes, to name just a few. System 300 illustrates an example of a head-to-head matchup mode between two different basketball systems.

[0118] System 300 illustrates basketball systems 302-1 and 302-N engaging in head-to-head competition. Basketball systems 302-1 and 302-N communicate with each other via network 301 through their respective control units. Network 301 may be, for example, the Internet, Wi-Fi, or another form of connection.

[0119] At basketball system 302-1, user 304 can register with basketball system 302-1 using their client device 314 to play a head-to-head game. User 304 can use basketball 312 to play on court 310. Similarly, at basketball system 302-N, another user 306 can register with basketball system 302-N using their client device 316 to play a head-to-head game. User 306 can use basketball 315 to play on court 308.

[0120] In some implementations, user 304 may transmit a request to participate in a head-to-head match to basketball system 302-1. User 304 may provide an instruction or request to participate in a head-to-head match to basketball system 302-1 via an application on client device 314. The instruction or request may also indicate whether user 304 wishes to play against a friend or be randomly matched. If user 304 chooses a friend, basketball system 302-1 may attempt to send the request to a basketball system associated with said friend, such as basketball system 302-N. Alternatively, if user 304 chooses random matching, basketball system 302-1 may retrieve each of the other basketball systems currently online and listed in the queue that are also requesting to participate in a head-to-head match, and may randomly select one of those basketball systems in the queue to play against user 304 from basketball system 302-1. Alternatively, basketball system 302-1 may select the top basketball system in the queue to play against user 304.

[0121] Similarly, user 306 can provide an instruction or request to participate in a head-to-head competition game to basketball system 302-N via an application on client device 316. Basketball system 302-N can determine whether user 306 requests a friend match or a random match. Based on the request, basketball system 302-N can communicate with another basketball system to set up a head-to-head competition. For example, the control unit of basketball system 302-1 can transmit request 318 to the control unit of basketball system 302-N. The control unit of basketball system 302-N can respond to receiving request 318 and set up a head-to-head competition by accepting the response request 318.

[0122] Basketball system 302-1 and basketball system 302-N can be located in different geographical areas. For example, basketball system 302-1 can be located in a stadium in New York City, New York, and basketball system 302-N can be located in a stadium in Geneva, Switzerland. In other instances, basketball system 302-1 can be located at one house, and basketball system 302-N can be located at a neighbor's house. In other instances, basketball system 302-1 and basketball system 302-N can be located at opposite ends of the same basketball court. Basketball systems can connect to each other as long as they can be connected to a network connection (e.g., the Internet). In other embodiments, basketball systems do not require an Internet connection to communicate with each other. In fact, basketball systems can communicate via a cellular connection with specific throughput and constant network connectivity.

[0123] In response to basketball systems 302-1 and 302-N accepting a head-to-head competition, the head-to-head competition module is executed at both basketball systems. For example, the backboard of basketball system 302-1 displays a live video stream recorded by user 306 playing basketball. Similarly, the backboard of basketball system 302-N displays a live video stream recorded by user 304 playing basketball. One or more RGB cameras at basketball system 302-1 record image data of the court 310 where user 304 is playing and provide the recorded image data to the control unit of basketball system 302-1. The control unit of basketball system 302-1 transmits the recorded image data via network 301 to the control unit of basketball system 302-N for display on the display screen of basketball system 302-N. Similarly, one or more RGB cameras at basketball system 302-N record image data of the court 308 where user 306 is playing basketball and provide the recorded image data to the control unit of basketball system 302-N. The control unit of basketball system 302-N transmits the recorded image data to the control unit of basketball system 302-1 via network 301 for display on the screen of basketball system 302-1. This process occurs simultaneously, so user 304 can see user 306 playing the ball on the backboard of basketball system 302-1, and therefore user 306 can see user 304 playing the ball on the backboard of basketball system 302-N.

[0124] Furthermore, the two users can converse with each other during a head-to-head match. For example, user 306 can say, "I'm about to beat you, John!" The microphone of basketball system 302-N can pick up audible messages, provide them to the control unit, and the control unit can transmit the audible messages to the control unit of basketball system 302-1. Here, the control unit of basketball system 302-1 can play the audible messages through the speakers of basketball system 302-1, where user 304 can hear the message, "I'm about to beat you, John!"

[0125] User 304 can respond by uttering the verbal message "It won't be like this soon!" The microphone of basketball system 302-1 picks up the verbal message and transmits it to the control unit of basketball system 302-1. The control unit of basketball system 302-1 can then transmit the verbal message to the control unit of basketball system 302-N, where the control unit of basketball system 302-N receives the verbal message and provides it for display by the speaker of basketball system 302-N, for example, the speaker plays "It won't be like this soon!"

[0126] When a head-to-head competition begins with a "3-point contest," "free throw contest," or "most successful shots within a predetermined time," basketball systems 302-1 and 302-N will begin a visually and / or audibly playing countdown, such as "3, 2, 1, start!" Users 304 and 306 will then begin throwing their respective basketballs 312 and 315 toward their respective basketball systems 302-1 and 302-N. Both basketball systems 302-1 and 302-N can use their respective cameras and sensors to monitor each user's throws and throwing attempts using the process described above. The display screen of basketball system 302-1 can show the ratio of successful shots to throwing attempts for user 304 (e.g., 6 / 20) and the ratio for successful shots to throwing attempts for user 306 (e.g., 5 / 10). Similarly, the display screen of basketball system 302-N can show a similar ratio. Both control units track shot successes and throw attempts, providing this information, along with image data recorded by an RGB camera, to the other control unit. In this way, the control unit of basketball system 302-N can display the ratio of shot successes to throw attempts by user 304 on the display screen of basketball system 302-N, and the control unit of basketball system 302-1 can display the ratio of shot successes to throw attempts by user 306 on the display screen of basketball system 302-1. Any updates regarding the throw attempts and shots of the two users are provided to the two control units via network 301, allowing their displays to be updated separately.

[0127] If this is a timed competition, the displays on both basketball systems, 302-1 and 302-N, can show the timer. The timers on the two displays will be synchronized to ensure that both users have an equal amount of time to compete head-to-head. Once the timer expires, the player with the highest field goal to shot attempt ratio is considered the winner of the competition.

[0128] Similar to stage (G) of system 100, the control unit of basketball system 302-1 can provide the output data of each attempted throw to client device 314 for review by user 304. The control unit of basketball system 302-N can provide the output data of each attempted throw to client device 316 for review by user 306. The control unit of basketball system 302-N can also provide output data to the control unit of basketball system 302-1 and / or client device 314 for user 304 to review the results of their opponent. The control unit of basketball system 302-1 can also provide output data to the control unit of basketball system 302-N and / or client device 316 for user 306 to review the results of their opponent. Similarly, the two control units can associate the image data, sensor data, and output data of each throw attempt with the corresponding user's profile. For example, the control unit of basketball system 302-1 can store the image data, sensor data, and output data of each throw attempt by user 304 in its profile. The control unit of the basketball system 302-N can execute the same stored program for user 306 and associated profile.

[0129] If users 306 and 304 are playing basketball on the same basketball system (e.g., basketball system 302-1), a process similar to that described above with respect to system 300 can be performed. In this case, components of basketball system 302-1 can monitor both user 304 and the corresponding basketball 312, and user 306 and the corresponding basketball 315. The control unit of basketball system 302-1 can display the shot-to-attack ratio of the two users as they perform shooting attempts on the display screen of basketball system 302-1. In this example, the display screen of basketball system 302-1 can display video recorded from the RGB cameras of basketball system 302-1 for the two users playing on basketball court 310. The shot-to-attack ratio of the two users can be overlaid on the display screen above the video recorded from the RGB cameras. In this case, the two users can see their respective scores (e.g., their respective ratios) and the recorded video of the two users playing the game.

[0130] In some implementations, these games can rate users interacting with the basketball system globally. For example, a head-to-head match could result in user 304 winning and user 306 losing. In the list of other users, user 304's rating would move up, while user 306's rating would move down. Users can then attempt to compete against other users using this player rating through their respective client devices or the basketball system.

[0131] In another game mode, live video highlights or real-time basketball can be streamed to the basketball system. For example, when user 304 makes various shooting attempts with basketball 312, basketball system 302-1 can display highlights from college or professional basketball games. These games can include NBA games, college games, and high school games.

[0132] In another game mode, the basketball system can enable a global competition mode. In this mode, two basketball systems connect and compete similarly to a head-to-head matchup. Furthermore, the global competition mode allows two users to invest stakes in their head-to-head matches. These stakes are escrowed to the winner's account and can be used to purchase additional paid features of the system. These paid features may include premium real-time-remote one-on-one training, special guest group coaching events, and other exclusive activities.

[0133] In another game mode, the basketball system allows users to play basketball in a training mode. When a user attempts to improve his or her basketball skills, the user (e.g., user 304) can select a training mode. For example, when training mode begins, the user can select a series of inputs from the user's client device or the backboard of the basketball system. This series of inputs may include, for example, "focus on shooting," "ball control," "foot speed," "catch and shoot," and other practice modes. The user can select which input or multiple inputs they wish to practice. The user is then instructed to begin practicing based on instructions provided by the training module. The control unit of the basketball system 302-1 can sense throwing parameters based on data provided by sensors and cameras during the training module, such as throwing attempts, successful shots, missed shots, dribbling, posture, ball movement, and body movement, to name a few. In response, the control unit can generate output data (e.g., output data 130) for each attempted action, such as throwing, dribbling, ball movement, and user movement, and can store the output data for each attempted action in association with the user's profile.

[0134] The training module can terminate when a specific event occurs (e.g., the user makes 50 free throws, 50 three-point shots, or learns how to dribble with their legs). In response to the termination of the training module, the control unit can provide output data to the basketball system's display and / or to the user's client device. The output data may include a training plan and other suggestions to help the user improve their basketball skills. If the user later returns to perform a similar training module, the control unit can determine whether the user's ability has improved or declined compared to one or more previous training modules. The control unit can then indicate this information to the user via the client device or display, for example, displaying "Your shooting percentage has improved by 20% in the 5 minutes since the last training session."

[0135] Figure 4 This is a block diagram illustrating an example computing system 400 for a backboard. The computing system 400 may include a backboard 110, a backboard 204, a basketball system 302-1, and a basketball system 302-N. As previously mentioned, the basketball system includes a control unit 402 that receives data from sensors 420 and cameras 418, and the control unit 402 can process and / or analyze the data from both sensors 420 and cameras 418. The control unit 402 includes a memory 404, a processor 406, and a network interface card (NIC) 408, and is connected to other components of the computing system 400 via wired or wireless communication links.

[0136] Control unit 402 may receive data from other input devices, such as user input device 422. In one example, user input device 422 is a user's personal smartphone or personal computer. User input device 422 may be connected to control unit 402 via a wired connection through NIC 408, for example, user input device 422 may be connected to the basketball system via a USB connector, HDMI connector, or wireless connection. In one example, the wireless connection may be a Bluetooth connection between user input device 422 and a transceiver and control unit connected to the basketball system. In another example, the wireless connection may be a Wi-Fi network or cellular data network connected to the Internet or cloud 424. In this example, the Internet 424 may provide wireless access / connection between control unit 402 and user input device 422 (e.g., personal smartphone). In some examples, software stored in memory analyzes input data and generates output data, which is transmitted to the player via user input device 422 and / or display panel 412. Control unit 402 may also transmit data to and / or receive data from software modules or mobile applications of user input device 422. For example, the mobile application of user input device 422 may display data from control unit 402 and / or provide the user with input fields for input data sent to control unit 402 via wired or wireless connection.

[0137] Power supply 410 can power components within the basketball system, as previously described. Control unit 402 can provide audible messages and music to speaker 414. Control unit 402 can receive audible messages and sound from microphone 416.

[0138] As described above, control unit 402 analyzes input data to generate output data. Processor 406 processes the input data and uses software programs or modules stored on memory 404. Different modules can use the output data, allowing users to use the basketball system in different ways. For example, users can stream personalized training sessions or engage in head-to-head basketball games with other players in remote locations (such as other basketball courts or a local basketball court). Output data can also be stored on memory 404, user input device 422, and / or cloud storage system 424, allowing performance and other metrics to be tracked over time as the user interacts with the basketball system. Accordingly, users can access the output dataset to understand how their basketball skills are developing and performance trends (e.g., the number of missed shots), analyze or plot the shot trajectory relative to the basket 112, and other characteristics. Control unit 402 can also display the user's performance data on display panel 412.

[0139] Figure 5 This is a flowchart illustrating an example of a process 500 for generating characteristics of a user playing basketball. Process 500 can be executed by the backboard 110 of basketball system 107 and the backboards 204 of systems 200 and 201.

[0140] The backboard can receive sensor data from multiple sensors and image data from one or more imaging devices coupled to the backboard (502) regarding a user's throwing attempts. For example, a user can request to play a basketball game using the backboard. The user can authenticate with their client device and / or the backboard using an authentication component and can select the basketball game to play. For example, the basketball game can include a training session mode, a local head-to-head matchup, a live broadcast mode, and a global competition mode. Once the user has selected the basketball game to play via their client device or the backboard, the backboard's control unit can instruct the user to begin shooting towards the basket. This can occur when a shooting timer starts counting down or when some other indication of the selected basketball game begins.

[0141] When a user engages in a game (e.g., moves with the basketball or attempts to throw), cameras and sensors can generate image data and sensor data, respectively, and provide these data to a control unit, which can determine (i) whether the user attempted a throw and (ii) whether the throw attempt was successful. Each of the sensors and cameras can be configured to perform different and / or similar functions. For example, a LiDAR sensor can be configured to detect a user's throw attempt, and one or both of the angle and height of the basketball from said throw attempt. A motion sensor can be configured to detect one or more users on the court within close range of the backboard. A travel sensor can be configured to determine if a throw attempt was successful. An accelerometer can be configured to determine an indication of the basketball's position relative to the backboard based on accelerometer data and vibration patterns during the throw attempt.

[0142] In addition, the backboard may contain one or more cameras. The cameras may include one or more depth-sensing cameras and / or one or more RGB cameras. Each of the cameras may be configured to perform different and / or similar functions. For example, the depth-sensing camera may be configured to perform one or more of the following: (i) detecting a user on the court, (ii) tracking the user's movement, (iii) detecting the basketball used by the user for a throwing attempt, (iv) tracking the movement of the basketball, and (v) detecting the user's posture. The one or more RGB cameras may be configured to perform one or more of the following: (i) recording image data of the field of view on the court, and (ii) recording image data of the area below the backboard to detect when a user's throwing attempt corresponds to a layup.

[0143] The backboard may comprise multiple layers, each housing different components. The front layer of the backboard may include a transparent coating to protect the components within the backboard. For example, the front layer may include tempered glass, which (i) protects the components within the backboard 110 and (ii) allows a user to see a display screen behind the front layer. The backboard may also include a second layer located behind the front layer. The second layer may include one or more components, such as sensors and cameras, for monitoring and generating data (e.g., sensor and image data) associated with users on the court. The backboard may also include a third layer housing the display screen. In some embodiments, the second layer of the backboard may include a display screen for providing feedback to the one or more users playing basketball. The rim may be attached to a protective layer of the backboard or the front layer of the backboard.

[0144] The backboard may also include a control unit housing a CPU and GPU for processing sensor and image data and providing output data to a display screen, one or more speakers, and / or a client device of the one or more users. The one or more speakers may provide audible output corresponding to the output data to the user.

[0145] The backboard can determine whether a throwing attempt was successful based on received sensor data (504). In some implementations, a user's throwing attempt with a basketball may result in an unsuccessful attempt. In an unsuccessful attempt, the basketball may bounce off the rim, backboard, or fail to hit the backboard or components at all. The backboard may further include one or more travel sensors configured to determine whether the basketball passed the rim during the throwing attempt.

[0146] In other implementations, a user's throwing attempt with a basketball may result in a successful attempt. In a successful attempt, the basketball may pass through the rim via the throwing attempt. The basketball may pass through the rim by bouncing off the backboard, without bouncing off the backboard or rim, or by bouncing off the rim first and then passing through the rim.

[0147] The backboard control unit can instruct the sensors to acquire sensor data periodically. In other instances, the backboard control unit can instruct the sensors to acquire sensor data based on the type of basketball game in progress. The control unit can also adjust the sensor sensitivity to improve the detection of throwing attempts, successful throwing attempts, and unsuccessful throwing attempts.

[0148] The backboard can generate analysis indicating (i) user characteristics, (ii) characteristics of the throwing attempt, (iii) suggestions for improving subsequent throwing attempts based on received image data and the success of the throwing attempt, and (iv) game performance (506). Cameras within the backboard can acquire image data from depth-sensing cameras and RGB cameras. For example, the image data may include images or videos of users playing basketball on the court. For example, the depth-sensing camera can generate (i) detection data of users on the basketball court, (ii) user movement data, (iii) detection data of the basketball used by the user, (iv) tracking movement data of the basketball, and (v) detection data of the user's posture. The depth-sensing camera continuously generates and tracks each of the different detection and movement data of the user and the basketball within a time period.

[0149] The control unit receives image data from depth sensing and RGB cameras and feeds the image data to a trained machine learning model. The machine learning model generates data that (i) classifies or identifies each user on the court as a unique and persistent user, (ii) identifies user throws, (iii) generates characteristics of users on the basketball court, and (iv) generates characteristics of user throw attempts. The trained machine learning model can simultaneously identify and track each user on the court. User characteristics may include user identification and the user's position on the court relative to the backboard. Throw attempt characteristics may include the angle of the basketball trajectory during the throw attempt and an indication of whether the basketball passed the rim. Furthermore, the control unit can associate users identified by the trained machine learning model with stored user profiles.

[0150] Once identified, the control unit can update the user's stored profile with the newly generated characteristics of the user (including characteristics describing the user's throwing attempt). More specifically, the control unit can determine the obtained data based on received sensor data, which includes one or more of the following: (i) whether the throwing attempt resulted in the basketball passing through the rim, (ii) whether the throwing attempt resulted in the basketball bouncing off the front of the backboard without passing through the rim, (iii) whether the throwing attempt resulted in the basketball bouncing off the rim without passing through the rim, (iv) the position of the user's throwing attempt, and (v) the trajectory of the basketball during the user's throwing attempt. The control unit can store the obtained data along with the user's characteristics, the characteristics of the throwing attempt output from a trained machine learning model, the received sensor data, and the received image data in a corresponding profile. The control unit can then provide the updated profile to an external server that stores multiple profiles, each corresponding to a different user.

[0151] In some implementations, the control unit can use image data to determine whether a throwing attempt was successful. For example, the control unit can apply virtual inner and outer cones to received image data from a depth-sensing camera. The virtual inner cone may include an inner radius matching the radius of the basketball hoop, a base conforming to the hoop, and a height extending from the hoop. The virtual outer cone may include an outer radius extending a certain distance from the hoop, an extruded profile the size of the virtual inner cone such that the two cones do not overlap, and a height extending from the hoop that is greater than the height of the virtual inner cone. Based on these cones, the control unit can determine whether the throwing attempt was successful or unsuccessful. For example, if the basketball passes through each cone and leaves the base of the virtual inner cone, the control unit can determine that the throwing attempt was successful. Alternatively, if the basketball does not pass through either cone or does not leave the base of the virtual inner cone, the control unit can determine that the throwing attempt was unsuccessful.

[0152] In some implementations, the control unit may use data generated from a trained machine learning model and received data to generate suggestions for the user. For example, the suggestions may include improvements for the user's subsequent throwing attempts. The suggestions may focus on one or more of the following during the user's subsequent throwing attempts: (i) body posture, (ii) arm angle, (iii) the point of contact with the ball, and (iv) the trajectory of the ball. The suggestions may also be stored with a profile of the specific identified user.

[0153] In some implementations, the control unit may store game data or performance associated with a game played by a user in the user's profile. The type of game played may include, for example, training course mode, local head-to-head matchup, live stream mode, and global competition mode. For example, game performance may include what game was played, the date and time the game was played, the number of players in the game, the identification of each player involved, the final score of the game, each player's made and missed shots during the game, the position of each player on the basketball court for each made and missed shot, and the time when each player's made and missed shots occurred, both in absolute and relative time (relative to the start of the game). The control unit may store game data as tuples, structures, classes, or some other computer format. If multiple users are playing a single game, the control unit may store the game data of each user in the corresponding profile of that single game.

[0154] The backboard can provide output data representing the analysis to one or more of the following: (i) a speaker, (ii) a display screen, and (iii) a user's client device (508). For example, the backboard's control unit can provide output data containing generated suggestions to the backboard's display screen. The output data can also correspond to audible voice output, which can be provided to the backboard's speaker to transmit the generated suggestions to the user. In another example, the control unit can provide the generated suggestions to the user's client device via a network. The control unit can also provide media content from an RGB camera to the backboard's display screen. In other examples, the control unit can, for example, receive media content from another control unit associated with another backboard during a specific game type, and display the received media content from said other control unit on the backboard.

[0155] In some instances, users can review their profiles, which include generated suggestions and determined analyses. Users can review their respective profiles on their client devices and / or on the backboard display. The control unit can receive instructions from the user to access the corresponding profile, and the control unit can identify the corresponding profile of the user. The control unit can determine which user profile to access based on the user's authentication and identification. The control unit can then provide the corresponding profile and its contents to the user's client device and / or to the backboard display. Users can review analyses of previous shooting attempts, such as recorded footage, suggestions, comparisons between their shooting attempts and those of professional athletes, data associated with the basketball game they played (e.g., the type of game played), shooting attempts, successful / missed shots, and opponent shooting attempts, successful / missed shots, timestamp information, and other basketball information associated with users interacting with the basketball system.

[0156] Generally, the terms apparatus, system, computing entity, entity, and / or similar terms used interchangeably herein can refer to, for example, one or more computers, computing entities, desktops, mobile phones, tablets, phablets, laptops, distributed systems, game consoles (e.g., Xbox, PlayStation, Wii), watches, glasses, keychains, radio frequency identification (RFID) tags, headphones, scanners, cameras, wristbands, kiosks, input terminals, servers or server networks, USB flash drives, gateways, switches, processing devices, processing entities, set-top boxes, repeaters, routers, network access points, base stations, etc., and / or any combination of apparatus or entities suitable for performing the functions, operations, and / or processes described herein. These functions, operations, and / or processes may include, for example, transmitting, receiving, retrieving, operating, processing, displaying, storing, determining, creating, generating, monitoring, evaluating, comparing, and / or similar terms used interchangeably herein. In various embodiments, these functions, operations, and / or processes may be performed on data, content, information, and / or similar terms used interchangeably herein. Furthermore, in embodiments of the present invention, the client device 104 may be a mobile device and may be operated by a user participating in an interactive physical combat game.

[0157] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include embodiments in one or more computer programs that can be executed and / or interpreted on a programmable system comprising at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0158] These computer programs (also referred to as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be implemented in high-level programming and / or object-oriented programming languages, and / or in assembly / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term machine-readable signal refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0159] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) (through which the user can provide input to the computer). Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including sound, speech, or tactile input.

[0160] The systems and techniques described herein can be implemented in computing systems that include back-end components (e.g., as data servers), middleware components (e.g., application servers), front-end components (e.g., client computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0161] A computing system can contain clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other.

[0162] Although several embodiments have been described in detail above, other modifications are possible. For example, while the client application is described as an access delegate, in other embodiments, the delegate may be adopted by other applications implemented by one or more processors (e.g., applications running on one or more servers). Furthermore, the logical flow depicted in the figures does not require a specific order or sequence to achieve the desired result. Additionally, other actions may be provided, or actions may be removed from the described flow, and other components may be added to or removed from the described system. Accordingly, other embodiments are within the scope of the appended claims.

[0163] While this specification contains numerous details of specific embodiments, these details should not be construed as limiting the scope of any invention or potentially claimed matter, but rather as descriptions of features specific to particular embodiments of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations or even initially claimed, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may be for sub-combinations or variations thereof.

[0164] Similarly, although operations are depicted in a specific order in the diagrams, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential manner, or requiring all shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring this separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products.

[0165] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions described in the claims can be performed in different orders and still achieve the desired result. As an example, the processes depicted in the drawings do not necessarily require a specific order or sequential sequence to achieve the desired result. In some embodiments, multitasking and parallel processing may be advantageous.

Claims

1. A basketball throwing attempt analysis system, comprising: The housing includes: Fasteners configured to attach the housing to a basketball backboard; Multiple sensors are configured to generate sensor data about the user's basketball throwing attempts; One or more imaging devices are configured to generate image data of the basketball throwing attempt; Control unit, wherein the control unit is configured to: Receive sensor data from one or more of the plurality of sensors and image data from the one or more imaging devices; Based on the received sensor data, determine whether the basketball throwing attempt was successful; Analysis is generated based on the generated image data and whether the basketball throwing attempt was successful; and Generate output data representing the analysis; A display screen, wherein the control unit is configured to provide real-time or recorded video data to the display screen; and A network interface, which is connected to the control unit, is configured to connect the system to one or more additional systems via a network.

2. The system according to claim 1, wherein, The display screen is configured to display user image data for one or more users.

3. The system according to claim 2, wherein, The network interface is configured to provide the control unit with user image data received from the one or more additional systems for display on the display screen.

4. The system according to claim 1, wherein, The control unit is configured to provide output data representing the analysis to one or more of the display screen and the user's client device.

5. The system according to claim 4, wherein, The analysis indicates (i) the characteristics of the user, (ii) the characteristics of the basketball throwing attempt, (iii) suggestions for improving the basketball throwing attempt for subsequent basketball throwing attempts, and (iv) game performance.

6. The system according to claim 1, wherein, The sensors include one or more of a LiDAR sensor, a motion sensor, a travel sensor, and an accelerometer.

7. The system according to claim 6, wherein, The LIDAR sensor is configured to generate sensor data indicating one or both of the user's basketball throwing attempt and the angle and height of the basketball from the basketball throwing attempt. The motion sensor is configured to generate sensor data that indicates one or more users or one or more basketballs on a basketball court. The stroke sensor is configured to generate sensor data that indicates whether the basketball throwing attempt was successful. The accelerometer is configured to generate sensor data that indicates the position of the basketball relative to the basketball backboard based on accelerometer data and vibration patterns. as well as The control unit is configured to: (i) use sensor data from the LIDAR sensor to detect one or both of the user's basketball throwing attempt and the angle and height of the basketball from the basketball throwing attempt; (ii) use sensor data from the motion sensor to detect one or more users on the basketball court; (iii) use sensor data from the travel sensor to determine whether the basketball throwing attempt was successful; and (iv) use sensor data from the accelerometer, based on the accelerometer data and the vibration pattern, to determine the position of the basketball relative to the basketball backboard.

8. The system according to claim 7, wherein, The one or more imaging devices include one or more depth sensing cameras or one or more RGB cameras, wherein the one or more depth sensing cameras are configured to perform one or more of the following: (i) detect the user on the basketball court, (ii) track the movement of the user, (iii) detect the basketball used by the user for the basketball throwing attempt, (iv) track the movement of the basketball, (v) detect the user's posture, and wherein the one or more RGB cameras are configured to record image data of the field of view of the basketball court.

9. The system of claim 1, further comprising a speaker, wherein the speaker is configured to provide an audible output representing the analysis in response to receiving output data representing the analysis from the control unit.

10. The system according to claim 2, wherein, The display screen is configured to display one or more of the following: (i) image data from the one or more imaging devices, (ii) a head-up display (HUD) showing the user's basketball throwing attempts and successful shots, and (iii) image data received from a second control unit connected via a network.

11. The system according to claim 5, wherein, The control unit is configured to provide image data received from the one or more imaging devices to a trained machine learning model to generate (i) the characteristics of the user, (ii) the characteristics of the basketball throwing attempt, and (iii) the game performance. The user's characteristics include the user's identification and the user's location. The characteristics of the basketball throwing attempt include the angle of the basketball's trajectory. The game performance includes data associated with the game played by the user; and The control unit is configured to store the user's characteristics, the characteristics of the basketball throwing attempt, and the game performance in the user's profile on an external server.

12. The system according to claim 11, wherein, The trained machine learning model is configured to simultaneously identify and track multiple users on the court, and the control unit is configured to: Associating each of the multiple users identified by the trained machine learning model with a stored user profile; as well as Update each user profile in the stored profile by utilizing the characteristics of each user and the characteristics of each user's basketball throwing attempts.

13. The system according to claim 11, wherein, The external server stores multiple profiles corresponding to different users.

14. The system according to claim 11, wherein, The control unit is configured to: Generate suggestions for improving the basketball throwing attempt for subsequent throwing attempts, wherein the suggestions include one or more of the following: (i) body posture, (ii) arm angle, (iii) the throwing point of the basketball, and (iv) the trajectory of the basketball; Provide the generated suggestions for display on the display screen; as well as Provide an output signal for transmitting the generated recommendations to the user.

15. The system according to claim 14, wherein, The control unit is configured to provide the generated suggestions and the output signals to the user's client device.

16. The system according to claim 1, wherein, The control unit is configured to: Receive instructions from the user to participate in a competition against a second user; The second control unit is connected via the network to a second system used by the second user, wherein the second control unit is located at a geographically different location from the control unit. The received image data is provided to the second control unit via the network; Receive second image data from the second control unit via the network; Provide second image data received from the second control unit for display on the display screen; The number of throwing attempts made by the user is counted based on the received sensor data and received image data; Receive the second number of throwing attempts made by the second user from the second control unit; Provide (i) the number of throwing attempts made by the user and (ii) a second number of throwing attempts made by the second user for display on the display screen; as well as The number of throwing attempts made by the user is provided to the second control unit.

17. The system according to claim 1, wherein, The control unit is configured to: The user receives instructions to participate in a local competition against a second user. The received image data is provided to the display screen; The number of basketball throwing attempts made by the user is counted based on the received sensor data and received image data; The number of basketball throwing attempts made by the second user is counted based on the received sensor data and received image data; as well as The display screen provides (i) the number of basketball throwing attempts made by the user and (ii) the number of basketball throwing attempts made by the second user, wherein the display screen overlays the number of basketball throwing attempts made by the user and the number of basketball throwing attempts made by the second user on top of the received image data.

18. The system according to claim 17, wherein, Instructions from the user may include audible voice commands, input from the user's client device, or visual commands.

19. The system according to claim 1, wherein, The control unit is configured to: In the generated image data, generate two or more virtual geometric regions near the basketball hoop; Determine whether the basketball has entered the virtual geometric area; as well as In response to determining that the basketball has entered the virtual geometry, it is determined whether the basketball throwing attempt was successful or unsuccessful.

20. The system according to claim 1, wherein, The control unit is configured to: Receive commands from the user; as well as In response to receiving a command from a user, one or more of the generated image data, the sensor data, and the output data representing the analysis are provided to a client device via a network.

21. The system according to claim 1, further comprising: A cloud storage system configured to store user metrics that can be tracked over time and to provide the user metrics to one or more of a client device, the control unit, and the network.

22. The system according to claim 1, wherein, The control unit is configured to provide one or more users' recorded images to at least one of the display screen, client device, and cloud storage system.

23. The system according to claim 1, wherein, The control unit is configured to measure the user's speed and store the user's speed in a corresponding user profile over time.

24. The system according to claim 1, wherein, The control unit is configured to provide image data received from the one or more imaging devices to a trained machine learning model to identify users attempting to throw a basketball.

25. The system according to claim 1, wherein, The control unit is configured to receive image data from the user's client device for display on the display screen.

26. The system according to claim 1, wherein, The control unit is configured to receive game costs from one or more users and provide the game costs via the network.

27. The system according to claim 1, wherein, The system is configured to rate users of the system and share those ratings with other systems via a network.

28. The system according to claim 9, wherein, The control unit is configured to connect to one or more user input devices via Bluetooth, and to receive audio signals from the one or more user input devices and provide the audio signals to the speaker.

29. The system according to claim 1, wherein, The control unit is configured to provide recorded footage from one or more users via a network.

Images